Tweak reflow and running sections.

src/glossary.rst

@@ -43,27 +43,36 @@ Glossary
 
 
    retry
-      Tasks configured to retry on failure will return to the ``waiting`` state
-      with a :term:`clock trigger` to delay the next try.
+   task retry
+   try number
+      Tasks can be configured to retry automatically on failure, one or more
+      times. They return to the ``waiting`` state with a :term:`clock trigger`
+      to delay the retry, and only go to the ``failed`` :term:`state <task
+      state>` once the final try fails.
 
-      Any number of retries, with configurable delays between them, are possible.
-      Task jobs can get their own try number from ``$CYLC_TASK_TRY_NUMBER``.
-
-      If the final try fails, the task goes to the ``failed`` :term:`state
-      <task state>`.
+      The task try number increments with every automatic retry, and is
+      passed to the job environment as ``$CYLC_TASK_TRY_NUMBER``.
 
       .. seealso::
 
          * :ref:`Cylc User Guide <TaskRetries>`
 
 
+   submit number
+   task submit number
+      Every time a task re-runs, whether by automatic :term:`retry` or manual
+      triggering, its submit number increments. Submit number appears in the
+      job log path so that log files don't get overwritten.
+
+
    window
    n-window
    active window
    workflow window
+   active task pool
       This is a :term:`graph`-based window or view of the workflow at runtime,
-      including tasks out to ``n`` graph edges from current active tasks.
-      The *active window* is ``n=0``.
+      including tasks out to ``n`` graph edges from current :term:`active
+      tasks<active task>`. The *active window* is ``n=0``.
 
       .. seealso::
 
@@ -143,14 +152,17 @@ Glossary
          the same as :term:`workflow id`.
 
 
-   active waiting task
-      An active waiting task is a task in the :term:`scheduler's <scheduler>`
-      active window that is "actively waiting" on (i.e. periodically checking)
-      an :term:`external trigger` or :term:`clock trigger`.
+   active
+   active task
+      An active task is a task in the submitted or running state.
+
 
-      These are the only waiting tasks that matter to the :term:`scheduler`.
-      Waiting tasks ahead of the active window in Cylc 8 are entirely
-      abstract.
+   active-waiting
+   active-waiting task
+      An active-waiting task is a task in the :term:`scheduler's <scheduler>`
+      ``n=0`` :term:`active window` that is ready to run according to its task
+      prerequisites, but is still waiting on a limiting mechanism such as a
+      :term:`clock trigger`, task :term:`hold`, or :term:`internal queue`.
 
 
    external trigger
@@ -172,6 +184,7 @@ Glossary
          * :ref:`Cylc User Guide <Built-in Workflow State Triggers>`
 
 
+   queue
    internal queue
       Internal queues (so called to distinguish them from external batch
       queueing systems) allow you to limit how many :term:`tasks <task>` can be
@@ -1424,14 +1437,15 @@ Glossary
    expected output
       Task outputs that are not marked as :term:`optional <optional output>`
       in the :term:`graph` are expected to be completed at runtime. If not, the
-      :term:`scheduler` retains the task as :term:`incomplete <incomplete
-      task>` pending user intervention.
+      :term:`scheduler` retains the task as :term:`incomplete` pending user
+      intervention.
 
       .. seealso::
 
          * :ref:`Cylc User Guide <expected outputs>`
 
 
+   incomplete
    incomplete task
       Incomplete tasks are :term:`tasks <task>` that finish (succeed or fail)
       without completing all :term:`expected outputs <expected output>`. They

src/user-guide/running-workflows/dynamic-behaviour.rst

@@ -3,19 +3,18 @@
 Cylc Broadcast
 --------------
 
-The ``cylc broadcast`` command overrides :cylc:conf:`[runtime]`
-settings in a running workflow.
-
-Broadcast can be used to communicate information to downstream tasks by
-setting environment variables. Broadcasts can target all tasks, groups of
-tasks, or specific tasks, at all cycle points or at specific cycle points.
-See broadcast command help for details.
-
-Broadcast settings targeted at a specific task ID or cycle point expire and
-are forgotten as the workflow moves on. Un-targeted variables and those
-targeted at a task name persist throughout the workflow run, even across
-restarts, unless manually cleared using the broadcast command - and so
-should be used sparingly.
+The ``cylc broadcast`` command overrides task :cylc:conf:`[runtime]`
+settings in a running scheduler. You can think of it as communicating
+changed settings (including information via environment variables) to selected
+upcoming tasks via the scheduler.
+
+Broadcasts can target all tasks, groups of tasks, or specific tasks, at all
+cycle points or at specific cycle points. See ``cylc broadcast --help`` for
+detailed information.
+
+Broadcast settings targeting a specific task ID or cycle point expire as the
+workflow moves on. Non-specific broadcasts persist throughout the run and
+across restarts unless manually cleared using the broadcast command.
 
 
 .. _Sub-Workflows:

src/user-guide/running-workflows/external-command-execution.rst

@@ -1,19 +1,19 @@
 .. _Managing External Command Execution:
 
-External Command Execution
---------------------------
+Scheduler Subprocesses
+----------------------
 
 Job submission commands, event handlers, and job poll and kill commands, are
-executed by the :term:`scheduler` in a "pool" of asynchronous
-subprocesses, in order to avoid blocking the workflow process. The process pool
-is actively managed to limit it to a configurable size, using
+executed by the :term:`scheduler` in a pool of asynchronous subprocesses.
+
+The process pool is size is configurable via
 :cylc:conf:`global.cylc[scheduler]process pool size`.
-Custom event handlers should be lightweight and quick-running because they
-will tie up a process pool member until they complete, and the workflow will
-appear to stall if the pool is saturated with long-running processes.
-However, to guard against rogue commands that hang indefinitely, processes
-are killed after a configurable timeout
-(:cylc:conf:`global.cylc[scheduler]process pool timeout`).
-All process kills are
-logged by the :term:`scheduler`. For killed job submissions, the associated
-tasks also go to the *submit-failed* state.
+
+Event handlers should be lightweight and quick-running because they tie up
+a process pool member until they complete, and the workflow will appear to
+stall if the pool is saturated with long-running processes.
+
+To protect the scheduler, processes are killed after a configurable timeout
+(:cylc:conf:`global.cylc[scheduler]process pool timeout`). This will be
+logged by the :term:`scheduler`. For killed job submissions, associated tasks
+go to the *submit-failed* state.

src/user-guide/running-workflows/index.rst

@@ -9,15 +9,15 @@ Running Workflows
    scheduler-start-up
    task-job-states
    tasks-jobs-ui
-   reflow
    retrying-tasks
-   workflow-completion
    tracking-task-state
-   authentication-files
+   workflow-completion
+   dynamic-behaviour.rst
+   reflow
    simulation-modes
    scheduler-log-files
-   external-command-execution
-   handling-job-preemption
-   dynamic-behaviour.rst
+   authentication-files
    workflow-databases.rst
    workflow-migration.rst
+   handling-job-preemption
+   external-command-execution

src/user-guide/running-workflows/reflow.rst

@@ -1,25 +1,28 @@
 .. _user-guide-reflow:
 
-Multiple Flows
-==============
+Concurrent Flows
+================
 
 .. versionadded:: 8.0.0
 
 In Cylc, a *flow* is a self-propagating run through the workflow :term:`graph`
-from some initial task or tasks.
+from some initial task(s).
 
-Often there is only one flow: the original one started automatically at the
-beginning of the graph. But Cylc can manage multiple flows at once.
+At start-up the :term:`scheduler` automatically launches a flow from the start
+of the graph. But you can use ``cylc trigger --flow=new ID`` to start additional 
+flows anywhere in the graph, while the original (and any other) flow still
+runs. New flows continue on from triggered tasks as dictated by the graph.
 
 When a flow advances to a new task in the :term:`graph`, the task will only run
 if it did not already run in the same flow.
 
-See below for suggested :ref:`use cases<flow-trigger-use-cases>` and an
-:ref:`example<new-flow-example>`.
+See below for suggested :ref:`use cases<flow-trigger-use-cases>` of this
+multi-flow capability, and a concrete :ref:`example<new-flow-example>`.
 
 .. note::
-   Flows are not entirely independent: they :ref:`merge<flow-merging>`
-   where (and if) tasks collide in the ``n=0`` :term:`active window`.
+   Flows :ref:`merge<flow-merging>` where (and if) tasks collide in the ``n=0``
+   :term:`active window`. Downstream of a merge, tasks are considered to belong
+   to all constituent flows.
 
 
 Flow Numbers
@@ -77,8 +80,8 @@ Triggering in Specific Flows
    This triggers the task with flow numbers ``1`` and ``2``.
 
    The result is like the default above, except that tasks in the new front
-   belong only to the specified flow(s), regardless of which flows are active
-   at triggering time.
+   belong only to the specified flow(s), regardless of which flows are
+   :term:`active` at triggering time.
 
 Triggering a New Flow
    ``cylc trigger --flow=new ID``
@@ -100,41 +103,42 @@ Triggering a Flow-Independent Single Task
    .. image:: ../../img/no-flow-n.png
 
 Special Case: Triggering ``n=0`` Tasks
-   Tasks in the ``n=0`` window are active, active-waiting, or incomplete. Their
-   flow membership is already determined - that of the parents that spawned them.
+   Tasks in the ``n=0`` window are :term:`active`, :term:`active-waiting`, or
+   :term:`incomplete`. Their flow membership is already determined - that of
+   the parent tasks that spawned them.
 
-   - Triggering an active task has no effect (it is already triggered).
-   - Triggering an active-waiting task runs it immediately in the same flow.
-   - Triggering an incomplete task re-runs it immediately in the same flow.
+   - Triggering an :term:`active task` has no effect (it is already triggered).
+   - Triggering an :term:`active-waiting task` runs it immediately in the same flow.
+   - Triggering an :term:`incomplete task` re-runs it immediately in the same flow.
 
 
 .. _flow-merging:
 
-Flow Merging In ``n=0``
+Flow Merging in ``n=0``
 -----------------------
 
 If a task spawning into the ``n=0`` :term:`window` finds another instance
-of itself (same name and cycle point, different flow) already there, a single
-instance of it will carry both (sets of) flow numbers forward from that point.
-Downstream tasks belong to both flows.
+of itself already there (i.e., same name and cycle point, different flow
+number) a single instance will carry both (sets of) flow numbers forward from
+that point. Downstream tasks belong to both flows.
 
-Flow merging in ``n=0`` means flows are not entirely independent. One flow
-might not be able to overtake another because one or more of its tasks might
-merge in ``n=0``. Merging is necessary while task IDs - and associated log
-directory paths etc. - do not incorporate flow numbers, because task IDs must
-be unique in the active task pool.
+Flow merging in ``n=0`` means flows are not completely independent. One flow
+might not be able to entirely overtake another because one or more of its tasks
+might merge in ``n=0``. Merging is necessary while task IDs - and associated
+log directory paths etc. - do not incorporate flow numbers, because task IDs
+must be unique in the :term:`active task pool`.
 
-Incomplete tasks
-^^^^^^^^^^^^^^^^
+Merging with Incomplete tasks
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Incomplete tasks are retained in the active window in expectation of
-retriggering once fixed, to complete expected outputs and continue the flow.
+:term:`Incomplete<incomplete>` tasks are retained in the active window in
+expectation of retriggering to complete :term:`expected outputs<expected
+output>` and continue their flow.
 
-If another flow encounters an incomplete task, one task will carry both flow
-numbers forward on successfully completing its expected outputs.
-
-.. TODO whether or not it automatically reruns in the later flow is still an
-   open question: https://github.com/cylc/cylc-flow/pull/4737
+If another flow encounters an incomplete task (i.e. if another instance of the
+same task collides with it in the ``n=0`` :term:`active window`) the task will
+run again and carry both flow numbers forward if it successfully completes its
+expected outputs.
 
 
 Stopping Flows
@@ -143,22 +147,23 @@ Stopping Flows
 By default, ``cylc stop`` halts the workflow and shuts the scheduler down.
 
 It can also stop specific flows: ``cylc stop --flow=N`` removes the flow number
-``N`` from tasks in the active pool. Tasks that have no flow numbers left as a
-result do not spawn children at all. If there are no active flows left, the
-scheduler shuts down.
+``N`` from tasks in the :term:`active task pool`. Tasks that have no flow
+numbers left as a result do not spawn children at all. If there are no active
+flows left, the scheduler shuts down.
 
 .. TODO update this section post https://github.com/cylc/cylc-flow/issues/4741
 
 
 .. _flow-trigger-use-cases:
 
-Use Cases
----------
+Some Use Cases
+--------------
 
 Running Tasks Ahead of Time
-   To run a task even though its prerequisites are not satisfied, just trigger
-   it. Use ``--wait`` if you don't want the new front to continue immediately.
-   Triggered task(s) will not re-run when the main flow front catches up.
+   To run a task within the existing flow(s) even though its prerequisites are
+   not satisfied yet, just trigger it. Use ``--wait`` if you don't want the new
+   flow front to continue immediately. Triggered task(s) will not re-run when
+   the main front catches up.
 
 Regenerating Products Behind a Flow
    To re-run a sub-graph (e.g. because the original run was affected by a
@@ -178,20 +183,21 @@ Rewinding a Workflow
 
 Test-running Tasks in a Live Workflow
    You can trigger individual tasks as many times as you like with
-   ``--flow=none``, without affecting the workflow.
+   ``--flow=none``, without affecting the workflow. The task :term:`submit
+   number` will increment each time.
 
-Processing Flow-specific Data
-   Flow numbers are passed to task environments, so it is possible to have
+Processing Flow-Specific Data
+   Flow numbers are passed to job environments, so it is possible to have
    different flows process different datasets through the same graph. However
-   we do not recommend doing this. Generally, that's what cycling is for; and
-   besides, each task would have to be capable of processing multiple datasets
-   at once in case of :ref:`flow-merging`.
+   **we do not recommend this**. That's what cycling is for; and besides, every
+   task would have to be capable of processing multiple datasets at once in
+   case of :ref:`flow-merging`.
 
 
 .. _new-flow-example:
 
-Example: Rerun a Past Sub-graph
--------------------------------
+Example: Rerun a Sub-Graph
+---------------------------
 
 The following :term:`cycling workflow` runs a :term:`task` called ``model`` in
 every cycle, followed by a postprocessing task, two product-generating tasks,

src/user-guide/running-workflows/scheduler-log-files.rst

@@ -1,7 +1,7 @@
-.. _Workflow Server Logs:
+.. _Scheduler Logs:
 
-Workflow Server Logs
---------------------
+Scheduler Logs
+--------------
 
 Each workflow maintains its own log of time-stamped events in the
 :term:`workflow log directory` (``$HOME/cylc-run/<workflow-id>/log/workflow/``).