Skip to content

Commit

Permalink
Improve documentation for running QSIPrep with TemplateFlow on non-in…
Browse files Browse the repository at this point in the history
…ternet nodes (#907)

* Add note for nodes without internet access for templateflow

* Clean update to docs regarding non-internet node usage for rst

* Update docs/installation.rst

---------

Co-authored-by: Matt Cieslak <mattcieslak@gmail.com>
  • Loading branch information
pierre-nedelec and mattcieslak authored Feb 3, 2025
1 parent ee9aa2e commit 46579b3
Showing 1 changed file with 33 additions and 0 deletions.
33 changes: 33 additions & 0 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,39 @@ As with Docker, you will need to bind the Freesurfer license.txt when running Ap
$HOME/fullds005 $HOME/dockerout participant \
--fs-license-file /opt/freesurfer/license.txt

.. note::
**Running QSIPrep with Apptainer on Non-Internet Nodes**

QSIPrep relies on TemplateFlow to provide standard anatomical templates. By default, it downloads
necessary files into the ``$TEMPLATEFLOW_HOME`` directory (default: ``$HOME/.cache/templateflow``).
However, when running with ``--containall``, the default location may not be accessible, and any
missing templates will be tentatively downloaded into the temporary QSIPrep working directory instead.
To avoid this, you must always set and bind ``TEMPLATEFLOW_HOME`` explicitly.

Steps to ensure successful execution:

1. On an **internet-enabled node**, set and bind the ``TEMPLATEFLOW_HOME`` variable to a persistent
directory before running QSIPrep. This will ensure all necessary templates are downloaded into the
specified location:

.. code-block:: sh
export TEMPLATEFLOW_HOME=/path/to/persistent/templateflow
apptainer run --cleanenv --containall \
-B ${TEMPLATEFLOW_HOME}:${TEMPLATEFLOW_HOME} \
--env "TEMPLATEFLOW_HOME=$TEMPLATEFLOW_HOME" \
/path/to/qsiprep_<VERSION>.sif <your commands>
2. If the ``TEMPLATEFLOW_HOME`` directory is not accessible on your target HPC node, manually copy it
to the target system after the first run.

3. On nodes without internet access, bind the copied ``TEMPLATEFLOW_HOME`` directory and set the
environment variable as described above before running QSIPrep.
4. It may help to run a single subject or session on its own before running many jobs that access the templates. The single run will download the necessary templates and prevent multiple jobs from attempting to download the templates simultaneously.


For additional troubleshooting, see `fmriprep docs <https://fmriprep.org/en/stable/faq.html#how-do-you-use-templateflow-in-the-absence-of-access-to-the-internet>`_
or `this thread on Neurostars <https://neurostars.org/t/issue-with-qsiprep-templateflow-on-hpc-host-with-no-internet-access/31259/10?u=pierre-nedelec>`_.

*********************
External Dependencies
Expand Down

0 comments on commit 46579b3

Please sign in to comment.