NfcoreSchema.validateParameters JSONObject error

[Leon-Bichmann]

Description of the bug

Updating the pipelines MHCquant and DIAproteomics to the template 1.13.1 resulted in an error calling the function:

NfcoreSchema.validateParameters(params, json_schema, log)

Unknown method invocation `containsKey` on JSONObject type

Steps to reproduce

The error occured both here:
nf-core/diaproteomics#103

and here:
nf-core/mhcquant#160

[KevinMenden]

@Leon-Bichmann to solve this issue, you have to change line 222 in NfcoreSchema.groovy
to this:

if(rawSchema.keySet().contains('properties') && rawSchema.get('properties').keySet().contains(ignore_param)) {

Basically replacing containsKey() with keySet().contains()

[ewels]

@Leon-Bichmann to solve this issue, you have to change line 222 in NfcoreSchema.groovy

This fix makes the groovy code handle the problem, but it would be nice to also know why it's a problem in the first place. With this GitHub Actions test I thought I tracked it down to the fact that the schema has params.name in it whilst the config doesn't.

I was fairly satisfied that this was the root cause and that the mismatch should have been caught by the nf-core lint command had it not failed for other reasons.

However, in these PRs above the nf-core lint command is not picking up any mismatch and the schema doesn't contain params.name. @Leon-Bichmann - was there any other parameters that you were stripping out of the config at the time? If you delete nextflow_schema.json and generate a new one with nf-core schema build . --no-prompts and run the pipeline again does the error go away?

[KevinMenden]

Yes I couldn't find any name param here so that's why I figured it must be something else.

The problem happens in the code when checking for rawSchema for properties and then for ignore params. In the pipeline template, there is no properties, but it is here: https://github.com/Leon-Bichmann/diaproteomics/blob/merging-template-updates/nextflow_schema.json

So I think the code couldn't handle this before - might not be a failure on the pipeline side. Maybe I'll try with a fresh template and adding to the schema to confirm this

[KevinMenden]

Okay confirmed, this is the part that breaks the code:

    "properties": {
        "max_multiqc_email_size": {
            "type": "string",
            "default": "25 MB"
        }
    }

Adding this to a fresh template breaks it (at the end of the nextflow_schema.json)

[KevinMenden]

The difference might just be that this internally is a JSONObject which doesn't have the containsKey(), function (only works for a Map). But we can use .keySet().contains() instead.

[ewels]

JSONObject again?!

[ewels]

ok so then the easiest quick fix @Leon-Bichmann is to run nf-core schema build . on these pipelines and make sure that all parameters are in groups.

But yeah, we should fix this 👍🏻 Are you on a PR @KevinMenden?

[KevinMenden]

#939

😄

[Leon-Bichmann]

Ok awesome thanks for checking and fixing this !

[chasemc]

Similar error occurs if "fasta" param is deleted from nextflow.config

[chasemc]

Error occurs because nextflow_schema.json goes from:

 "reference_genome_options": {
            "title": "Reference genome options",
            "type": "object",
            "fa_icon": "fas fa-dna",
            "description": "Reference genome related files and options required for the workflow.",
            "properties": {
                "fasta": {
                    "type": "string",
                    "format": "file-path",
                    "mimetype": "text/plain",
                    "pattern": "^\\S+\\.fn?a(sta)?(\\.gz)?$",
                    "description": "Path to FASTA genome file.",
                    "help_text": "This parameter is *mandatory* if `--genome` is not specified. If you don't have a BWA index available this will be generated for you automatically. Combine with `--save_reference` to save BWA index for future runs.",
                    "fa_icon": "far fa-file-code"
                }
            }
        },

to:

        "reference_genome_options": {
            "title": "Reference genome options",
            "type": "object",
            "fa_icon": "fas fa-dna",
            "description": "Reference genome related files and options required for the workflow."
        },