-
Notifications
You must be signed in to change notification settings - Fork 298
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
singularity and template flow #2480
Comments
Have you tried running |
Hi, sorry for the delay. When I run:
that directory has all of the templates in it. TEMPLATEFLOW_HOME in the singularity shell points to /home/fmriprep/.cache/templateflow/. If I bind directories as in the second command above, I get the same thing. Also, I tried:
and all the templates are in /templateflow, and TEMPLATEFLOW_HOME points to /templateflow. That command to run singularity still ends up with it trying to download templates. |
Can you try running https://github.com/nipreps/fmriprep/blob/master/scripts/fetch_templates.py inside your container and reporting the output? |
It doesn't work, because I don't have internet access on my compute nodes. Per the fMRIprep documentation I did run on our login node:
and placed the output in |
@ejcorn do you know what file(s) it is downloading? |
Yes, sorry should have included that! It's
|
Okay, If you want to verify, try the following minimal example in a shell - lines starting with # are comments and lines starting with $ are commands
|
Tried the above and everything checks out. The Previously I was pre-downloading the templateflow files to a folder called Now I'm having a problem getting the freesurfer license to be recognized. I'm not sure if this is upstream or downstream of the templateflow issue. With this command:
In singularity shell, everything seems to available. I also tried binding to /opt/freesurfer/license.txt, and tried pointing it directly to the license files. I've tried multiple license files. |
Please post the error traceback. It sounds like something is not right in your command - what I found helps is making the in bash, it'd look something like: cmd="singularity run..."
echo $cmd
$cmd |
Sorry about that -- new to this github help framework. Thank you for being so responsive.
I've now tried several variations of this that produce this error, where I've used different license files in different locations, all of which singularity should be able to see based on what paths I'm binding. |
Something is wrong with your license, either it is not being found or is not formatted properly. Singularity shell is very useful when debugging, you can check the file exists inside the container and contains the expected text ( |
Okay, so now I'm running this command, using a different license that I've used to successfully start running recon-all through fmriprep on a different machine:
and getting the same error. I did singularity shell and cat /license.txt gives the expected text in the license (don't want to post the license here) |
When you use include variables (like $BASE) that are not defined in the snippet, I can't really help debug. Please look at my suggestion in #2480 (comment) to convert your singularity run command into a string (expanding variables along the way) and share that |
Sorry about that! here you go:
produces
|
Hm, nothing jumps out. This might be a long shot, but theres a readonly binding within a binding, which might introduce some complexity. Try removing this binding |
same error unfortunately. I know you can only do so much without sitting at my machine, but would appreciate any other suggestions you have! it's odd because singularity shell can find the license, and I've tried it with many different licenses. |
Alternatively, you can use environment variables: https://www.nipreps.org/apps/singularity/#handling-environment-variables
Overall, a deep read of the singularity documentation there (https://www.nipreps.org/apps/singularity/) may be of particular help in this case. |
That command works and shows that
I can see the license file (this is a new license I just downloaded from freesurfer's website). When I run this:
In the log file, |
I wonder if you are getting an unrelated FreeSurfer error that is being picked up by our test function: Can you try running the following after defining
and paste the output you get? |
|
Aha, looks like we found the problem - the issue is actually not related to the license file. Taken from this post in the FreeSurfer mailing list:
I looks like the best way forward is to either:
|
Looks like we can check |
Ah, thank you very much!! Yes, when I run |
Hi, I'm having exactly the same issue of binding templateflow. Everything worked correctly based on this test. I found that templateflow can be accessed without problem if with python even inside singularity shell, but will be problematic with command line only (either singularity shell / singularity run / singularity exec). I'm not that lucky as @ejcorn, changing directory name of templateflow did not help me. Just wonder if there's anything else I can do to debug this? |
Hello,
I'm having issues similar to #1778 and those listed in #1801. However, I have tried multiple solutions and still, when I run fmriprep v20.2.3 on my HPC (no internet access) with singularity, it keeps trying to download template flow even though the templateflow folder with all templates exists in multiple locations. As best I can tell, I've bound the necessary directories. Here are a couple examples of what I've tried:
export SINGULARITYENV_https_proxy=http://micc:8899
singularity run --cleanenv --home /home/fmriprep/ -B /project:/project \
-B $HOME:/home/fmriprep/ \
$fmriprep \
$SUBJDIR \
${OUTDIR}derivatives/ \
participant \
--skip_bids_validation \
--notrack \
--participant-label $SUBJ \
--output-spaces MNI152NLin2009cAsym:res-2 T1w \
--fs-license-file $FSLIC
export SINGULARITYENV_TEMPLATEFLOW_HOME=/home/fmriprep/.cache/templateflow/
singularity run --cleanenv -B /project:/project \
-B ~/.cache/templateflow:/home/fmriprep/.cache/templateflow \
-B ~/.cache/fmriprep:/home/fmriprep/.cache/fmriprep \
$fmriprep \
$SUBJDIR \
${OUTDIR}derivatives/ \
participant \
--skip_bids_validation \
--notrack \
--participant-label $SUBJ \
--output-spaces MNI152NLin2009cAsym:res-2 T1w \
--fs-license-file $FSLIC
Do you have any suggestions? Thanks in advance for your time!
The text was updated successfully, but these errors were encountered: