Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pipeline is working with SRA run IDs, but failing with corresponding Biosample IDs #129

Closed
amizeranschi opened this issue Nov 17, 2022 · 10 comments · May be fixed by #269
Closed

Pipeline is working with SRA run IDs, but failing with corresponding Biosample IDs #129

amizeranschi opened this issue Nov 17, 2022 · 10 comments · May be fixed by #269
Assignees
@drpatelh
Labels
bug Something isn't working
Milestone
1.10.0

Comments

@amizeranschi
Copy link

Description of the bug

The pipeline works when supplying SRA run IDs (SRR***), but fails with corresponding Biosample IDs (SAMN***).

For instance, the following works:

echo 'SRR9041539
SRR9041540
SRR9041541
SRR9041542
SRR9041543
SRR9041544' > yeast-test-SRArun-ids.csv

nextflow run nf-core/fetchngs -r 1.8 --input yeast-test-SRArun-ids.csv --outdir yeast-test-SRArun-ids -profile docker

and the following fails:

echo 'SAMN11619543
SAMN11619542
SAMN11619539
SAMN11619538
SAMN11619541
SAMN11619540' > yeast-test-sample-ids.csv

nextflow run nf-core/fetchngs -r 1.8 --input yeast-test-sample-ids.csv --outdir yeast-test-sample-ids -profile docker

Command used and terminal output

$ nextflow run nf-core/fetchngs -r 1.8 --input yeast-test-sample-ids.csv --outdir yeast-test-sample-ids -profile docker 
N E X T F L O W  ~  version 22.10.2
Launching `https://github.com/nf-core/fetchngs` [friendly_brahmagupta] DSL2 - revision: 249210f185 [1.8]


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/fetchngs v1.8
------------------------------------------------------
Core Nextflow options
  revision       : 1.8
  runName        : friendly_brahmagupta
  containerEngine: docker
  launchDir      : /data/share/nf-core
  workDir        : /data/share/nf-core/work
  projectDir     : /home/amizeranschi/.nextflow/assets/nf-core/fetchngs
  userName       : amizeranschi
  profile        : docker
  configFiles    : /home/amizeranschi/.nextflow/assets/nf-core/fetchngs/nextflow.config

Input/output options
  input          : yeast-test-sample-ids.csv
  outdir         : yeast-test-sample-ids

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/fetchngs for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.5070524

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/fetchngs/blob/master/CITATIONS.md
------------------------------------------------------
executor >  local (8)
executor >  local (8)
[a4/4cb5a2] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)                       [ 25%] 2 of 8, failed: 2...
executor >  local (9)
[e3/4cc40b] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)                       [ 25%] 2 of 8, failed: 2...
executor >  local (10)
[c7/9e9094] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)                       [ 33%] 3 of 9, failed: 3...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
executor >  local (10)
[c7/9e9094] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)                       [ 33%] 3 of 9, failed: 3...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
executor >  local (11)
[35/d8ab51] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)                       [ 40%] 4 of 10, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
executor >  local (12)
[f9/7d2f51] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)                       [ 45%] 5 of 11, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
executor >  local (13)
[60/d956f8] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)                       [ 50%] 6 of 12, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
executor >  local (14)
[57/8d60bd] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)                       [ 53%] 7 of 13, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
executor >  local (15)
[0e/099e8d] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)                       [ 57%] 8 of 14, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
executor >  local (15)
[0e/099e8d] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)                       [ 57%] 8 of 14, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
executor >  local (16)
[43/f354a6] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)                       [ 60%] 9 of 15, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET                                   -
executor >  local (16)
[43/f354a6] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)                       [ 60%] 9 of 15, failed: ...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET                                   -
executor >  local (17)
[61/a0069e] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)                       [ 62%] 10 of 16, failed:...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET                                   -
[-        ] process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG                                 -
[-        ] process > NFCORE_FETCHNGS:SRA:CUSTOM_DUMPSOFTWAREVERSIONS                             -
executor >  local (18)
[ae/7f1586] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)                       [ 66%] 12 of 18, failed:...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET                                   -
[-        ] process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG                                 -
[-        ] process > NFCORE_FETCHNGS:SRA:CUSTOM_DUMPSOFTWAREVERSIONS                             -
[b2/8f976e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) --executor >  local (19)
[5a/305aef] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)                       [ 66%] 12 of 18, failed:...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET                                   -
[-        ] process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG                                 -
[-        ] process > NFCORE_FETCHNGS:SRA:CUSTOM_DUMPSOFTWAREVERSIONS                             -
[b2/8f976e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) -- Execution is retried (1)
[a4/4cb5a2] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)` terminated with an error exit status (1) -- Execution is retried (1)
[f6/368e1e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)` terminated with an error exit status (1) -- Execution is retried (1)
[b3/b9bb77] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)` terminated with an error exit status (1) -- Execution is retried (1)
[54/a149f0] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)` terminated with an error exit status (1) -- Execution is retried (1)
[f9/63adbc] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)` terminated with an error exit status (1) -- Execution is retried (1)
[f6/68d470] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) --executor >  local (19)
[5a/305aef] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)                       [100%] 13 of 13, failed:...
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP                                           -
[ba/8919d2] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_... [100%] 1 of 1 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOL... -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET                                      -
[-        ] process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET                                   -
[-        ] process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG                                 -
[-        ] process > NFCORE_FETCHNGS:SRA:CUSTOM_DUMPSOFTWAREVERSIONS                             -
[b2/8f976e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) -- Execution is retried (1)
[a4/4cb5a2] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)` terminated with an error exit status (1) -- Execution is retried (1)
[f6/368e1e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)` terminated with an error exit status (1) -- Execution is retried (1)
[b3/b9bb77] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)` terminated with an error exit status (1) -- Execution is retried (1)
[54/a149f0] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)` terminated with an error exit status (1) -- Execution is retried (1)
[f9/63adbc] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)` terminated with an error exit status (1) -- Execution is retried (1)
[f6/68d470] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) -- Execution is retried (2)
[e3/4cc40b] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)` terminated with an error exit status (1) -- Execution is retried (2)
[c7/9e9094] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)` terminated with an error exit status (1) -- Execution is retried (2)
[35/d8ab51] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)` terminated with an error exit status (1) -- Execution is retried (2)
[60/d956f8] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)` terminated with an error exit status (1) -- Execution is retried (2)
[f9/7d2f51] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)` terminated with an error exit status (1) -- Execution is retried (2)
Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)'

Caused by:
  Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1)

Command executed:

  echo SAMN11619542 > id.txt
  sra_ids_to_runinfo.py \
      id.txt \
      SAMN11619542.runinfo.tsv \
  
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO":
      python: $(python --version | sed 's/Python //g')
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  [ERROR] The server couldn't fulfill the request.
  [ERROR] Status: 400 Bad Request

Work dir:
  /data/share/nf-core/work/57/8d60bd8d90ce9f15f3a574a7568f7c

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

Relevant files

nextflow.log

System information

N E X T F L O W ~ version 22.10.2

Local; Docker; Ubuntu 20.04

nf-core/fetchngs v1.8
@amizeranschi amizeranschi added the bug Something isn't working label Nov 17, 2022
@amizeranschi amizeranschi changed the title Pipeline failing with SRA run IDs, but failing with corresponding Biosample IDs Pipeline is working with SRA run IDs, but failing with corresponding Biosample IDs Nov 22, 2022
@Midnighter
Copy link
Contributor

Hi @amizeranschi,

Thank you for the report. I looked into this and it comes down to the following request to NCBI failing:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?id=SAMN11619542&db=sra&rettype=runinfo&retmode=text

@drpatelh the corresponding request to ENA succeeds

https://www.ebi.ac.uk/ena/portal/api/filereport?accession=SAMN11619542&fields=run_accession%2Cexperiment_accession&format=tsv&result=read_run

Should we just always resolve SRA identifiers using the ENA API? Or was there a reason not to?

@Midnighter Midnighter mentioned this issue Nov 24, 2022
Closed
@Midnighter
Copy link
Contributor

Did a bit more testing. I do get a hit using the NCBI esearch:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=sra&term=SAMN11619542

One could parse that output for the identifier and then use that in efetch:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=sra&id=7819691&rettype=runinfo&retmode=text

which is successful. However, that means extra requests while ENA provides direct access to this information.

@amizeranschi
Copy link
Author

Hi @Midnighter and @drpatelh

Has anyone had a chance to look into this issue?

@Midnighter
Copy link
Contributor

Well, I've been working on a tool that can resolve these correctly. There's no clear roadmap for when that tool will end up integrated into the pipeline, though.

My best suggestion at the moment is: Use the ENA Portal yourself to transform the BioSamples into run identifiers. Then use the run identifiers as input to fetchngs.

@drpatelh drpatelh added this to the 1.10 milestone Apr 25, 2023
@drpatelh drpatelh self-assigned this May 5, 2023
drpatelh added a commit to drpatelh/nf-core-fetchngs that referenced this issue May 6, 2023
@drpatelh
Fix nf-core#129
2aca881
@drpatelh drpatelh mentioned this issue May 6, 2023
Merged
@drpatelh
Copy link
Member

drpatelh commented May 6, 2023

This has been fixed in #149

Should we just always resolve SRA identifiers using the ENA API? Or was there a reason not to?

We could do but the problem has always been getting the identifier resolution to work as expected when you have multiple run ids associated with any given "meta" id.

In this case, I found an id SAMN13021459 that has multiple SRR* ids and the fix works as expected in terms of the resolution.

Will close for now but if you observe the same behaviour for other ids then please feel free to re-open or create a new issue.

@drpatelh drpatelh closed this as completed May 6, 2023
@amizeranschi
Copy link
Author

Hi @drpatelh

Thanks for working on this issue. I have tested the current dev code, using the command from the original post above, but it's still not working for me. Here is the output:

[9f/657233] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)` terminated with an error exit status (1) -- Execution is retried (1)
[91/0213a0] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)` terminated with an error exit status (1) -- Execution is retried (1)
[98/00918c] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)` terminated with an error exit status (1) -- Execution is retried (1)
[ca/a4ae4b] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)` terminated with an error exit status (1) -- Execution is retried (1)
[cd/785fd6] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)` terminated with an error exit status (1) -- Execution is retried (1)
[d5/10ebd2] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) -- Execution is retried (1)
[5c/992ad5] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)` terminated with an error exit status (1) -- Execution is retried (2)
[01/3af175] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619543)` terminated with an error exit status (1) -- Execution is retried (2)
[a1/334a96] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619540)` terminated with an error exit status (1) -- Execution is retried (2)
[e1/793c8a] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619539)` terminated with an error exit status (1) -- Execution is retried (2)
[f5/0aab56] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619541)` terminated with an error exit status (1) -- Execution is retried (2)
[74/ac6e4e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619542)` terminated with an error exit status (1) -- Execution is retried (2)
ERROR ~ Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)'

Caused by:
  Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (SAMN11619538)` terminated with an error exit status (1)

Command executed:

  echo SAMN11619538 > id.txt
  sra_ids_to_runinfo.py \
      id.txt \
      SAMN11619538.runinfo.tsv \
  
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO":
      python: $(python --version | sed 's/Python //g')
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  [ERROR] The server couldn't fulfill the request.
  [ERROR] Status: 400 Bad Request

Work dir:
  /data/share/reproducible-sarek-error/work/6e/d43223f432e2317b51b1844eab32f4

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

@amizeranschi
Copy link
Author

Wait, nevermind. I see now that the fix hasn't been merged into the dev branch yet.

@drpatelh
Copy link
Member

drpatelh commented May 6, 2023

Merged into dev now @amizeranschi !

You will have to pull the latest code before re-running

nextflow pull nf-core/fetchngs -r dev

@amizeranschi
Copy link
Author

I tested the fix now and it works as expected. Thanks again for addressing this issue!

@drpatelh
Copy link
Member

drpatelh commented May 6, 2023

Awesome! Thanks for testing so quickly 🚀

@edmundmiller edmundmiller mentioned this issue Feb 6, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@drpatelh drpatelh

Labels
bug Something isn't working
Projects
None yet
Milestone
1.10.0
Development

Successfully merging a pull request may close this issue.

optimized resources nf-core/fetchngs
3 participants
@Midnighter @amizeranschi @drpatelh