[ENH] Allow tck.gz files to be used as input

[chiuhoward]

Enhancement for #14 to accommodate .tck.gz files by first unzipping them.

This is important because qsirecon pipeline produces tck.gz outputs.

Before this change, these files would not be recognized, leading to a crash later on.

[arokem]

Looks good to me! Does it work for your use-case?

[chiuhoward]

I haven’t actually tested oops, let me test this in a conda environment and get back!

[chiuhoward]

@arokem sorry I tested this last night - it still writes a .tck file to the qsirecon folder and doesn’t delete it after pyAFQ completes...

Do I need something like a os.remove(temp_tck) line on line 79 after is_trx = false?

[arokem]

That's weird. Can you print the value of temp_tck? I would expect this to be a file-system location entirely different from your qsirecon folder, so it's odd that anything ends up there.

[chiuhoward]

I also tried this but the .tck file remains in the qsirecon folder even after the job completes

    elif streamlines.endswith(".tck.gz"):
        # uncompress tck.gz to a temporary tck:
        temp_dir = mkdtemp()
        temp_tck = op.join(temp_dir, streamlines.replace(".gz", ""))
        logger.info(f"Temporary tck file created at: {temp_tck}")
        try:
            with gzip.open(streamlines, 'rb') as f_in:
                with open(temp_tck, 'wb') as f_out:
                    shutil.copyfileobj(f_in, f_out)
            # initialize stateful tractogram from tck file:
            tg = load_tractogram(
                temp_tck, data_imap["dwi"], Space.VOX, 
                bbox_valid_check=False)
            is_trx = False
        finally:
            shutil.rmtree(temp_dir)

The logger line prints:

2024-11-12 19:41:44,218 [    INFO] Saving /scratch/groups/jyeatman/howard/DBPFullwT1derivOct24-pyAFQ/sub-16004/sub-16004_acq-b700_dir-PA_space-subject_desc-template_dwi.nii.gz
2024-11-12 19:41:44,232 [    INFO] Temporary tck file created at: /scratch/groups/jyeatman/howard/DBPFullwT1derivOct24-recon/derivatives/qsirecon-MRtrix3_fork-SS3T_act-None/sub-16004/dwi/sub-16004_acq-b700_dir-PA_space-T1w_model-ifod2_streamlines.tck
2024-11-12 19:42:25,474 [    INFO] Preprocessing Streamlines

[arokem]

That's very odd. I'd expect mkdtemp to give you a completely different file-system location:

>>> from tempfile import mkdtemp
>>> mkdtemp()
'/var/folders/px/t5ym05pn5k16nz97mzcpqqkr0000gn/T/tmp8p0d9rda'

Is streamlines a full path at that point or a relative path?

[chiuhoward]

streamlines is an absolute path here, was this because I passed in import_tract as an absolute path?

2024-11-13 10:58:39,780 [    INFO] Streamlines path: /scratch/groups/jyeatman/howard/DBPFullwT1derivOct24-recon/derivatives/qsirecon-MRtrix3_fork-SS3T_act-None/sub-16004/dwi/sub-16004_acq-b700_dir-PA_space-T1w_model-ifod2_streamlines.tck.gz
2024-11-13 10:58:39,780 [    INFO] Temporary tck file created at: /scratch/groups/jyeatman/howard/DBPFullwT1derivOct24-recon/derivatives/qsirecon-MRtrix3_fork-SS3T_act-None/sub-16004/dwi/sub-16004_acq-b700_dir-PA_space-T1w_model-ifod2_streamlines.tck

[chiuhoward]

@arokem that worked! the temporary folder was used and the qsirecon no longer has a .tck file after pyAFQ completes!

2024-11-13 13:53:56,163 [    INFO] Temporary tck file created at: /tmp/tmpj0pshudg/sub-16004_acq-b700_dir-PA_space-T1w_model-ifod2_streamlines.tck

[arokem]

Great! Sorry that it took so many iterations to get to a working solution...

@36000 : WDYT? Should we leave that line of logging in or is that just noise?

[36000]

I think its fine! Should we merge this?