Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create update section #5687

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open
81 changes: 55 additions & 26 deletions docs/dsl1.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,28 @@
(dsl1-page)=

# Migrating from DSL 1
# Migrate from DSL1

In Nextflow version `22.03.0-edge`, DSL2 became the default DSL version. In version `22.12.0-edge`, DSL1 support was removed, and the Nextflow documentation was updated to use DSL2 by default. Users who are still using DSL1 should migrate their pipelines to DSL2 in order to use the latest versions of Nextflow. This page describes the differences between DSL1 and DSL2, and how to migrate to DSL2.
In Nextflow version `22.03.0-edge`, DSL2 became the default DSL version. In version `22.12.0-edge`, DSL1 support was removed and the Nextflow documentation was updated to use DSL2 by default. Users who are still using DSL1 should migrate their pipelines to DSL2 to use the latest versions of Nextflow. This page describes the differences between DSL1 and DSL2 and how to migrate to DSL2.

In Nextflow versions prior to `22.03.0-edge`, you must enable DSL2 explicitly in order to use it. You can either set the feature flag in your pipeline script:
## Enable DSL2

```nextflow
nextflow.enable.dsl=2
```
To use DSL2 prior to Nextflow version `22.03.0-edge`, you must explicitly enable it. You can enable the feature flag in one of two ways:

Or set the environment variable where you launch Nextflow:
- Set the feature flag in your pipeline script:

```bash
export NXF_DEFAULT_DSL=2
```
```nextflow
nextflow.enable.dsl=2
```

- Set the environment variable where you launch Nextflow:

```bash
export NXF_DEFAULT_DSL=2
```

## Processes and workflows

In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels using `from` and `into`. Here is the {ref}`your-first-script` example written in DSL1:
In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels using `from` and `into`. You can see a basic Nextflow script written in DSL1 here:

```nextflow
nextflow.enable.dsl=1
Expand Down Expand Up @@ -53,19 +57,44 @@ result.view { it.trim() }

To migrate this code to DSL2, you need to move all of your channel logic throughout the script into a `workflow` definition. Additionally, you must call each process explicitly, passing any input channels as arguments (instead of `from ...`) and receiving any output channels as return values (instead of `into ...`).

Refer to the {ref}`workflow-page` page to learn how to define a workflow. The DSL2 version of the above script is duplicated here for your convenience:
See {ref}`workflow-page` page to learn how to define a workflow.

```{literalinclude} snippets/your-first-script.nf
:language: nextflow
```
You can see the DSL1 Nextflow script from above written in DSL2 here:

## Channel forking
```nextflow
params.str = 'Hello world!'

process splitLetters {
output:
path 'chunk_*'

script:
"""
printf '${params.str}' | split -b 6 - chunk_
"""
}

In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must be forked using the `into` operator.
process convertToUpper {
input:
path x

In DSL2, channels are automatically forked when connecting two or more consumers.
output:
stdout

For example, this would not work in DSL1 but is not a problem in DSL2:
script:
"""
cat $x | tr '[a-z]' '[A-Z]'
"""
}

workflow {
splitLetters | flatten | convertToUpper | view { v -> v.trim() }
}
```

## Channel forking

In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must be forked using the `into` operator. In DSL2, channels are automatically forked when connecting two or more consumers. For example:

```nextflow
Channel
Expand All @@ -81,16 +110,16 @@ cheers
.view()
```

Similarly, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.
Similarly, in DSL2, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.

## Modules

In DSL1, the entire Nextflow pipeline must be defined in a single file (e.g. `main.nf`). This restriction becomes quite cumbersome as a pipeline becomes larger, and it hinders the sharing and reuse of pipeline components.
In DSL1, the entire Nextflow pipeline must be defined in a single file. For example, `main.nf`. This restriction becomes cumbersome as a pipeline grows and hinders the sharing and reuse of pipeline components.

DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2 by any means, modules can help you organize a large pipeline into multiple smaller files, and take advantage of modules created by others. Check out the {ref}`module-page` to get started.
DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2, modules can help you organize a large pipeline into multiple smaller files and take advantage of modules created by others. See {ref}`module-page` to learn more about modules.

:::{note}
DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be split into modules to avoid this limit.
DSL2 scripts cannot exceed 64 KB in size. Split large DSL1 scripts into modules to avoid this limit.
:::

## Deprecations
Expand All @@ -103,14 +132,13 @@ DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be spli

- The `mode flatten` option for process outputs is no longer available. Use the {ref}`operator-flatten` operator on the corresponding output channel instead.

- Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit `val` or `path` qualifier.

For example:
- Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit `val` or `path` qualifier. For example:

```nextflow
process foo {
input:
tuple X, 'some-file.sam'

output:
tuple X, 'some-file.bam'

Expand All @@ -127,6 +155,7 @@ DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be spli
process foo {
input:
tuple val(X), path('some-file.sam')

output:
tuple val(X), path('some-file.bam')

Expand Down
11 changes: 10 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@ notifications
secrets
sharing
vscode
dsl1
```

```{toctree}
Expand Down Expand Up @@ -123,6 +122,16 @@ reference/channel
reference/operator
```

```{toctree}
:hidden:
:caption: Updates
:maxdepth: 1

updates
dsl1
```


```{toctree}
:hidden:
:caption: Contributing
Expand Down
54 changes: 10 additions & 44 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

# Installation

Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). This page describes how to install Nextflow.

(install-requirements)=

## Requirements

Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). It requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. You can see which version you have using the following command:
Nextflow requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. To see which version of Java you have, run the following command:

```{code-block} bash
:class: copyable
Expand All @@ -17,7 +19,7 @@ java -version
Support for Java versions prior to 17 was dropped.
:::

If you don't have a compatible version of Java installed in your computer, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest LTS version of Temurin. See [this website](https://whichjdk.com/) for more information.
If you don't have a compatible version of Java installed, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest Long-Term-Support (LTS) version of Temurin. See [Which version of JDK should I use?](https://whichjdk.com/) for more information about different versions of Java.

To install Java with SDKMAN:

Expand Down Expand Up @@ -50,15 +52,15 @@ To install Java with SDKMAN:

Nextflow is distributed as a self-installing package, in order to make the installation process as simple as possible:

1. Install Nextflow:
To install Nextflow:

1. Download Nextflow:

```{code-block} bash
:class: copyable
curl -s https://get.nextflow.io | bash
```

This will create the `nextflow` executable in the current directory.

:::{tip}
Set `export CAPSULE_LOG=none` to make the installation logs less verbose.
:::
Expand Down Expand Up @@ -93,27 +95,12 @@ Nextflow is distributed as a self-installing package, in order to make the insta
nextflow info
```

## Updates

With Nextflow installed in your environment, you can update to the latest version using the following command:

```{code-block} bash
:class: copyable
nextflow self-update
```

You can also temporarily switch to a specific version of Nextflow with the `NXF_VER` environment variable. For example:

```{code-block} bash
:class: copyable
NXF_VER=23.10.0 nextflow info
```

## Seqera Platform

You can launch workflows directly from [Seqera Platform](https://seqera.io/platform/) without installing Nextflow locally.

Launching from Seqera Platform provides you with:

- User-friendly launch interfaces.
- Automated cloud infrastructure creation.
- Organizational user management.
Expand All @@ -122,34 +109,13 @@ Launching from Seqera Platform provides you with:
Seqera Cloud Basic is free for small teams. Researchers at qualifying academic institutions can apply for free access to Seqera Cloud Pro.
See the [Seqera Platform documentation](https://docs.seqera.io/platform) for set-up information and tutorials to get started.

## Stable and edge releases

A *stable* version of Nextflow is released every six months, in the 4th and 10th month of each year.

Additionally, an *edge* version is released on a monthly basis. The edge releases can be used to access the latest updates and experimental features.

To use the latest edge release, set `NXF_EDGE=1` when updating:

```{code-block} bash
:class: copyable
NXF_EDGE=1 nextflow self-update
```

You can also use `NXF_VER` to temporarily switch to any edge release. For example:

```{code-block} bash
:class: copyable
NXF_VER=24.06.0-edge nextflow info
```

## Standalone distribution

The Nextflow standalone distribution (i.e. the `dist` distribution) consists of self-contained `nextflow` executable file
that includes all the application dependencies for core functionalities, and it can run without downloading third parties
that includes all the application dependencies for core functionalities and can run without downloading third parties
libraries. This distribution is mainly useful for offline environments.

Note however the support for cloud services e.g. AWS, Seqera Platform, Wave, etc. still require the download
of the corresponding Nextflow plugins.
Support for cloud services, such as AWS, Seqera Platform, and Wave, still require downloading the corresponding Nextflow plugins.

To use the standalone distribution:

Expand Down
56 changes: 56 additions & 0 deletions docs/updates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Update Nextflow

This page describes Nextflow release cadence, how to self-update Nextflow, and how select your version of Nextflow.

## Releases

A stable version of Nextflow is released in the 4th and 10th month of each year. A edge version of Nextflow is released on a monthly basis. The edge version can be used to access the latest updates and experimental features.

You can find an exhaustive list of releases and updates in the [Nextflow changelog](https://github.com/nextflow-io/nextflow/blob/master/changelog.txt).

:::{warning}
Nextflow will update its executable during the self-update process. The update can fail if the Nextflow executable is in a directory with restricted permissions.
:::

## Self-update

To update to the latest stable release of Nextflow, run the `self-update` command:

```{code-block} bash
:class: copyable
nextflow self-update
```

To use the latest edge release, set `NXF_EDGE=1` when you self-update Nextflow:

```{code-block} bash
:class: copyable
NXF_EDGE=1 nextflow self-update
```

## Version selection

The `NXF_VER` environment variable can be used to define which version of Nextflow to use. To switch to a specific version of Nextflow for a single run, set the `NXF_VER` environment variable in your execution command. For example:

```{code-block} bash
:class: copyable
NXF_VER=23.10.0 nextflow info
```

To set a specific version of Nextflow for a terminal session, export the `NXF_VER` environment variable. For example:

```{code-block} bash
:class: copyable
export NXF_VER=23.10.0
```

To set a specific version of Nextflow for your user profile, add the above `NXF_VER` export command to your shell configuration file, such as `~/.bashrc` or `~/.zshrc`, and restart your session.

:::{tip}
You can use `NXF_VER` to switch to an edge release. For example:

```{code-block} bash
:class: copyable
NXF_VER=24.06.0-edge nextflow info
```
:::
Loading