U jg]+ @s dZddlmZddlmZddlZddlZddlmZddl m Z m Z m Z dd l mZmZmZmZeZeZejjejjd d d d dddgddddddgdddZejjddZejjejjdddgddgdejjd d!d"gfd#d$d%gfd$d&gfd$d'gfd$d%d'gfgd(d)d*d+d,d-gdejjd.e d/fe d0fgd1d2gdd3d4Zejjejd.e d/fe d0fgd5d6Zejjejd.e d/fe d0fgd7d8Zd9d:Zd;d<Zejjejd.e d/fe d0fgd=d>Zejjejd.e d/fe d0fgd?d@ZdS)Az Test the processor hooks )Path)TemporaryDirectoryN)Pooch)UnzipUntar Decompress)pooch_test_urlpooch_test_registrycheck_tiny_data capture_logzmethod,ext,name)autoxzN)lzmarN)rrN)bzip2bz2N)gzipgzN)rrzdifferent-name.txtrrrrrname)idsc CsFt||d}t(}t|}|dkr@t|dd|dg}n t||}t|ttd}tr}|j d||d} | } | } t | d kst | d d d kst | d d st || d kst W5QRX| |kst t| t*}|j d||d} | dkst W5QRX| |ks0t t| W5QRXdS)z=Check that decompression after download works for all formats)methodrN. tiny-data.txtdecomppathbase_urlregistryztiny-data.txt. processorrr DownloadingZ Decompressing)rrrstrjoinrBASEURLREGISTRYr fetchgetvalue splitlineslenAssertionErrorsplit startswithr ) rextrr local_storer true_pathpuplog_filefnameZlogslinesr6?/tmp/pip-unpacked-wheel-vga3p65l/pooch/tests/test_processors.pytest_decompresss,    r8c Cst}t|}t|ttd}tt.}t |j dt dddW5QRXW5QRX|j j ddsttd|j j dksttt.}t |j dt d ddW5QRXW5QRX|j j dd std|j j dksttt.}t |j dt d ddW5QRXW5QRX|j j dd sFtd|j j dks\ttt.}t |j d t dddW5QRXW5QRX|j j ddstd|j j dkstW5QRXdS)zGShould fail if method='auto' and no extension is given in the file namerrr)rrrz"Unrecognized file extension '.txt'zpooch.Unzip/UntarZblaz Invalid compression method 'bla'zipz Invalid compression method 'zip'z store.zipz"Unrecognized file extension '.zip'N)rrrr&r'pytestZraises ValueErrorwarningscatch_warningsr(rvalueargsr.r,)r0rr2 exceptionr6r6r7test_decompress_failsEs.   (  (  (  (rA target_pathZsome_custom_pathZ default_pathZ custom_pathzarchive,members tiny-datar)storeNrDstore/tiny-data.txtstore/subdir/tiny-data.txtz store/subdirZ single_fileZ archive_allZ archive_fileZarchive_subdir_fileZarchive_subdirZarchive_multiplezprocessor_class,extensionz.zipz.tar.gzrrc Cs|||d}|dkr"|||j}t}t|}t|||||j\}}t|ttd} t2} | j |||d} t | |kst t | |W5QRX| D] } t | qt2} | j |||d} t | |kst t | gW5QRX| D] } t | qW5QRXdS)zGTests the behaviour of processors for unpacking archives (Untar, Unzip))members extract_dirNrr)suffixrr"_unpacking_expected_paths_and_logs__name__rr&r'r r(setr, _check_logsr ) processor_class extensionrBarchiverGr r true_pathsZ expected_logr2r3fnamesr4r6r6r7test_unpackingds0  rSc Cst}tt|ttd}|dgd}|jd||d}t|dksHtt|d|ddgd}|jd||d}t|d kstt|dt|d|jd||d}t|dkstt|dW5QRXd S) zHTest that multiple subsequent calls to a processor yield correct resultsrrErGrDrr rrFrN rrrr&r'r(r+r,r ) rNrOr0r2 processor1 filenames1 processor2 filenames2Z filenames3r6r6r7test_multiple_unpackings     rZc CsbtR}tt|ttd}|dgd}|jd||d}t|dksHtt|dW5QRXdS) zDTest that unpack members can also be specifed both with a leading ./rz./store/tiny-data.txtrTrDrr rNrU)rNrOr0r2rVrWr6r6r7$test_unpack_members_with_leading_dots  r[cCsF|}t|t|ks tt||D]\}}||s*tq*dS)zC Assert that the lines in the log match the expected ones. N)r)r*r+r,r9r.)r3Zexpected_linesr5lineZ expected_liner6r6r7rMs rMcCsdg}|dkr(t|dh}|dn|dkrv|dkrvt|ddt|dddh}|||dd nn|dkr|dk rg}|D]L}|t|d }t|ds|d}|t||d |d qt|}||fS) zo Generate the appropriate expected paths and log message depending on the parameters for the test. r!rCrzExtracting 'tiny-data.txt'rDNsubdirr"z ing contents/z Extracting '')r$appendrr-endswithrL)rPrGrrZ log_linesrQmemberr1r6r6r7rJs& rJc Cs~tn}tt|ttd}|dgd}|jd||d}t|dksHt|}|jd||d}t|dksptW5QRXdS)zx Test that calling with valid members then without them works. https://github.com/fatiando/pooch/issues/364 rrErTrDrr Nrrrr&r'r(r+r,rNrOr0r2rVrWrXrYr6r6r7&test_unpacking_members_then_no_memberss  rec Cs~tn}tt|ttd}|dgd}|jd||d}t|dksHt|}|jd||d}t|dksptW5QRXdS)zz Test that calling with invalid members then without them works. https://github.com/fatiando/pooch/issues/364 rznot-a-valid-file.csvrTrDrrNrcrdr6r6r7,test_unpacking_wrong_members_then_no_members s  rf) __doc__pathlibrtempfilerr<r:r#rZ processorsrrrutilsr r r r r'r&marknetworkZ parametrizer8rArSrZr[rMrJrerfr6r6r6r7s