KeyError: "Metadata term 'RepetitionTime' unavailable for file

[m-petersen]

What version of fMRIPrep are you using?

21.0.0rc0

What kind of installation are you using? Containers (Singularity, Docker), or "bare-metal"?

Singularity container

What is the exact command-line you used?

   singularity run --cleanenv --userns -B $PROJ_DIR -B /work/fatx405/tmp/:/tmp \
   $ENV_DIR/fmriprep-21.0.0rc0 \
   data/raw_bids data participant \
   -w /tmp \
   --participant-label $1 \
   --output-spaces $OUTPUT_SPACES \
   --nthreads $SLURM_CPUS_PER_TASK \
   --omp-nthreads $OMP_NTHREADS \
   --mem-mb $MEM_MB \
   --bids-database-dir $CODE_DIR/bids_database \
   --stop-on-first-crash \
   --ignore t2w \
   --fs-subjects-dir data/freesurfer \
   --use-aroma \
   --random-seed 12345 \
   --use-syn-sdc \
   --cifti-output 91k \
   --notrack \
   --skip_bids_validation \
   --fs-license-file envs/freesurfer_license.txt"

Have you checked that your inputs are BIDS valid?

Yes, they are BIDS valid

Are you reusing previously computed results (e.g., FreeSurfer, Anatomical derivatives, work directory of previous run)?

No

fMRIPrep log

If you have access to the output logged by fMRIPrep, please make sure to attach it as a text file to this issue.
#################################################################

Because it's rather short I'll put it in the post directly

         Building fMRIPrep's workflow:
           * BIDS dataset path: /work/fatx405/projects/CSI_HCHS/data/raw_bids.
           * Participant list: ['000480a3'].
           * Run identifier: 20210915-094801_82bbc2c0-a350-4c53-a171-5e869b40972c.
           * Output spaces: fsnative fsaverage:den-164k MNI152NLin6Asym:res-native.
           * Pre-run FreeSurfer's SUBJECTS_DIR: /work/fatx405/projects/CSI_HCHS/data/freesurfer.
Matplotlib is building the font cache; this may take a moment.
210915-09:53:39,389 nipype.workflow CRITICAL:
         SyN processing is not yet implemented.
210915-09:55:00,804 nipype.workflow INFO:
         No single-band-reference found for sub-000480a3_ses-1_task-rest_bold.nii.gz.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/bids/layout/utils.py", line 18, in __getitem__
    return super().__getitem__(key)
KeyError: 'RepetitionTime'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/fmriprep/cli/workflow.py", line 118, in build_workflow
    retval["workflow"] = init_fmriprep_wf()
  File "/usr/local/miniconda/lib/python3.8/site-packages/fmriprep/workflows/base.py", line 85, in init_fmriprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "/usr/local/miniconda/lib/python3.8/site-packages/fmriprep/workflows/base.py", line 349, in init_single_subject_wf
    func_preproc_wf = init_func_preproc_wf(bold_file, has_fieldmap=has_fieldmap)
  File "/usr/local/miniconda/lib/python3.8/site-packages/fmriprep/workflows/bold/base.py", line 332, in init_func_preproc_wf
    tr=metadata["RepetitionTime"],
  File "/usr/local/miniconda/lib/python3.8/site-packages/bids/layout/utils.py", line 20, in __getitem__
    raise KeyError(
KeyError: "Metadata term 'RepetitionTime' unavailable for file /work/fatx405/projects/CSI_HCHS/data/raw_bids/sub-000480a3/ses-1/func/sub-000480a3_ses-1_task-rest_bold.nii.gz."

The issue is the corresponding json sidecar actually contains the repetition time.

$cat ses-1/func/sub-000480a3_ses-1_task-rest_bold.json

____
"ReconMatrixPE": 94,
  "RefLinesPE": 46,
  "RepetitionTime": 2.5,
  "SAR": 0.140098,
  "ScanOptions": "FS",
____

[smeisler]

I also get this error, particularly when working with large pybids database layouts (so large that BIDS validator has issues).

[effigies]

I guess this would be a PyBIDS bug. Did previous versions of fMRIPrep work on these data?

[smeisler]

I received similar errors when using 20.2.3 and 21.0.0rc0. I have not tried any older version.

[m-petersen]

If I remember correctly 20.2.1 worked on these data. Should have mentioned that.

[effigies]

It would be good if you could retry with those versions so we could try to narrow down the cause.

fMRIPrep version	PyBIDS version
20.2.1	0.12.3
20.2.3	0.13.1
21.0.0rc0	0.13.2

I notice in the original report:

   --bids-database-dir $CODE_DIR/bids_database \

If you're using pre-computed PyBIDS database files, you may want to delete and re-compute them. I believe there was a change to the database format, and it could have subtle effects rather than throwing an intelligible error.

[smeisler]

I tried reverting to a 0.12.3 PyBIDS version layout on the unstable 21.0.0 release, same error. Have not tried going back to 20.2.1 yet. When having fMRIPrep recompute the dataset I appear to get farther, reaching the numpy import error referenced before (#2532), but obviously for large datasets that's not ideal.

[effigies]

I tried reverting to a 0.12.3 PyBIDS version layout on the unstable 21.0.0 release, same error.

We definitely don't want to go back to an older version of PyBIDS to create the layout. Apologies if that was unclear. You should use the same version of PyBIDS as fMRIPrep to create the layout. My concern was that you might be reusing a layout created by an older version.

[smeisler]

When I use a premade pybids layout v0.13.1 on 20.2.3 or 0.13.2 on 21.0.0rc0 I get the error in this issue. However, when I have fMRIPrep make the layouts I do not.

[effigies]

Can you share your command for pre-computing the layout?

[smeisler]

pybids layout $bids $bids/code/pybids_db

[effigies]

Okay, I reproduced this issue, and the fix is that you need to add --index-metadata to the pybids layout command:

pybids layout --index-metadata --reset-db $bids $bids/code/pybids_db

It is unclear why that's disabled by default. We should add it to our docs.

[effigies]

This seems to be resolved.