Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve cycling docs #454

Merged
merged 3 commits into from
Apr 13, 2022
Merged

Improve cycling docs #454

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
704447c
Update cycling docs
MetRonnie Apr 5, 2022
c013847
Improve installation docs slightly
MetRonnie Apr 7, 2022
9dd5fc6
Improve cycling docs (address code review)
MetRonnie Apr 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 53 additions & 12 deletions src/glossary.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,8 +338,8 @@ Glossary
The unique label given to tasks that belong to a particular :term:`cycle`.
For :term:`integer cycling` these will be integers, e.g. ``1``, ``2``,
``3``, etc.
For :term:`datetime cycling` they will be :term:`ISO8601` datetimes, e.g.
``2000-01-01T00:00Z``.
For :term:`datetime cycling` they will be :term:`ISO 8601` datetimes,
e.g. ``2000-01-01T00:00Z``.

.. seealso::

Expand Down Expand Up @@ -415,9 +415,9 @@ Glossary

datetime cycling
A datetime :term:`cycling` workflow uses
:term:`ISO8601 datetime <ISO8601 datetime>` :term:`cycle points <cycle
point>` (e.g. ``2000-01-01T00:00Z``) and :term:`recurrences
<recurrence>` (e.g. ``P3D`` means every third day).
:term:`ISO 8601 datetime` :term:`cycle points <cycle point>`
(e.g. ``2000-01-01T00:00Z``) and :term:`recurrences <recurrence>`
(e.g. ``P3D`` means every third day).

.. seealso::

Expand All @@ -435,23 +435,25 @@ Glossary
* :term:`clock trigger`


ISO 8601
ISO8601
ISO8601 is an international standard for writing datetimes, durations,
ISO 8601 is an international standard for writing datetimes, durations,
and :term:`recurrences <recurrence>` (sequences of datetimes). Cylc uses
ISO8601 for :term:`datetime cycling`.
ISO 8601 for :term:`datetime cycling`.

.. seealso::

* :term:`ISO8601 datetime`
* `Wikipedia (ISO8601) <https://en.wikipedia.org/wiki/ISO_8601>`_
* :term:`ISO 8601 datetime`
* `Wikipedia (ISO 8601) <https://en.wikipedia.org/wiki/ISO_8601>`_
* `International Organisation For Standardisation
<https://www.iso.org/iso-8601-date-and-time-format.html>`_
* `a summary of the international standard date and time notation
<http://www.cl.cam.ac.uk/%7Emgk25/iso-time.html>`_


ISO 8601 datetime
ISO8601 datetime
A datetime written in the :term:`ISO8601` format, e.g:
A datetime written in the :term:`ISO 8601` format, e.g:

* ``2000-01-01T00:00Z``: midnight on the 1st of January 2000, UTC.

Expand All @@ -460,14 +462,15 @@ Glossary
* :ref:`Cylc tutorial <tutorial-iso8601-datetimes>`


ISO 8601 duration
ISO8601 duration
A duration written in the ISO8601 format e.g:
A duration written in the ISO 8601 format e.g:

* ``PT1H30M``: one hour and thirty minutes.

.. seealso::

* :term:`ISO8601`
* :term:`ISO 8601`
* :ref:`Cylc tutorial <tutorial-iso8601-durations>`


Expand All @@ -481,6 +484,44 @@ Glossary
syntax, but are much simpler.


exact datetime unit
An exact datetime unit is any unit of a datetime that has a fixed
duration, which does not depend on its position in the calendar.
In Cylc, the following are exact units:

* second: SI base unit
* minute: 60 seconds
* hour: 60 minutes
* day: 24 hours
* week: 7 days

.. note::
Although the :term:`ISO 8601` standard specifies that weeks and days
are :term:`inexact <inexact datetime unit>` due to the possibility of
daylight saving time, leap seconds etc., they are always exact in
Cylc because workflows always maintain the same time zone, and leap
seconds are not supported.

.. seealso::

* :term:`inexact datetime unit`


inexact datetime unit
nominal duration
An inexact datetime unit is any unit of a datetime that does not have
a fixed duration; it instead depends on its position in the calendar.
In Cylc, the following are inexact units (when using the Gregorian
calendar):

* year: either 365 or 366 days depending on whether it is a leap year
* month: between 28 - 31 days depending on the specific month and year

.. seealso::

* :term:`exact datetime unit`


clock trigger
Clock triggers connect cycle points to the :term:`wallclock time`, in
:term:`datetime cycling` workflows. Tasks that depend on a clock trigger
Expand Down
12 changes: 10 additions & 2 deletions src/installation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ Via Pip (+npm)
Non-Python Requirements
-----------------------

.. _Graphviz: https://graphviz.org/download/
.. _configurable-http-proxy: https://anaconda.org/conda-forge/configurable-http-proxy

These dependencies are not installed by Conda or pip:
Expand All @@ -65,9 +66,16 @@ These dependencies are not installed by Conda or pip:
* GNU `coreutils`_
* ``mail`` (optional: for automated email functionality)

These dependencies are installed by Conda but not by pip (you can use npm):
These dependencies are installed by Conda but not by pip:

* `configurable-http-proxy`_ (optional: for multi-user setups)
* `Graphviz`_ (optional - used by ``cylc graph`` for displaying workflow
graphs)
* `configurable-http-proxy`_ (optional - for multi-user setups; can also be
installed using npm)

.. seealso::

:ref:`SyntaxHighlighting`


Installing On Mac OS
Expand Down
121 changes: 90 additions & 31 deletions src/user-guide/writing-workflows/scheduling.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,12 +289,15 @@ number of repetitions.

.. _writing_flows.scheduling.syntax_rules:

Syntax Rules
^^^^^^^^^^^^
Cycling Syntax Rules
--------------------

:term:`Datetime cycling <datetime cycling>` information is made up of a
starting :term:`datetime <ISO8601 datetime>`, an interval, and an optional
limit.
:term:`Datetime cycling <datetime cycling>` information is made up of:

* a :term:`datetime <ISO 8601 datetime>` that typically specifies the start
point of the sequence
* an interval between points in the sequence
* and an optional limit on the number of points in the sequence

The time is assumed to be in UTC unless you set
:cylc:conf:`[scheduler]cycle point time zone`.
Expand All @@ -312,7 +315,7 @@ The time is assumed to be in UTC unless you set
The calendar is assumed to be the proleptic Gregorian calendar unless
you set :cylc:conf:`[scheduling]cycling mode`.

The syntax is based on the :term:`ISO8601` datetime standard, which includes
The syntax is based on the :term:`ISO 8601` datetime standard, which includes
the representation of datetimes and intervals. Cylc (optionally) allows these
representations to be heavily condensed by omitting information that can be
inferred from context (rules below).
Expand All @@ -325,8 +328,27 @@ inferred from context (rules below).
except where specific tasks, if any, depend on :term:`clock triggers <clock
trigger>`.

There are three ISO 8601 recurrence formats supported by Cylc, detailed below
in order from most commonly used to least commonly used.


Format 3: ``R[limit?]/[datetime]/[interval]``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Comment on lines +335 to +336
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels a bit awkward talking about format 3, then 4, then 1. But that's probably the most sensible order

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a quick note just underneath the title giving a reason? Especially as the first sentence is that reason anyway...


The most common full form for recurrences is
``R[limit?]/[datetime]/[interval]``. In Cylc this can be condensed to:
``R[limit?]/[datetime]/[interval]``. This is format number 3 in the ISO 8601
standard. The datetime specifies the start of the cycling sequence.
For example, ``R3/2000-01-01T00Z/P2D`` means "run 3 times, every 2 days,
starting at 2000-01-01T00Z (midnight, Jan 1st 2000)"; the list of points
on this sequence is:

.. code-block:: none

2000-01-01T00Z
2000-01-03T00Z
2000-01-05T00Z

In Cylc, this form can be condensed to:

.. code-block:: sub

Expand All @@ -341,12 +363,12 @@ Here are some examples for each form:

.. code-block:: sub

R5/T00 # Run 5 times at 00:00 every day
R//PT1H # Run every hour (Note the R// is redundant)
20000101T00Z/P1D # Run every day starting at 00:00 1st Jan 2000
R1 # Run once at the initial cycle point
R1/20000101T00Z # Run once at 00:00 1st Jan 2000
P1Y # Run every year
R5/T00 # Run 5 times at 00:00 every day
R//PT1H # Run every hour (Note the R// is optional)
20000101T06Z/P1D # Run every day starting at 06:00 1st Jan 2000
R1 # Run once at the initial cycle point
R1/20000101T00Z # Run once at 00:00 1st Jan 2000
P1Y # Run every year

.. note::

Expand All @@ -368,19 +390,27 @@ hours, so the inferred interval is 1 day (daily), ``P1D``.
If the limit is omitted, unlimited cycling is assumed. This will be
bounded by the workflow's final cycle point if given.

Another supported form of ISO8601 :term:`recurrence` is:
``R[limit?]/[interval]/[datetime]``. This form uses the
datetime as the end of the cycling sequence rather than the start.
For example, ``R3/P5D/20140430T06`` means:

Format 4: ``R[limit?]/[interval]/[datetime]``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Another supported recurrence form is:
``R[limit?]/[interval]/[datetime]`` (format number 4 in the ISO 8601 standard).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A second reason why the section order and the format number don't match to add to the note at the top of the Format3 section?

This uses the datetime as the end of the cycling sequence rather than the start.
For example, ``R3/P5D/2014-04-30T06`` means "run 3 times, every 5 days, ending
at 2014-04-30T06 (06:00, April 30th 2014)"; the list of points on this
sequence is:

.. code-block:: none

20140420T06
20140425T06
20140430T06
2014-04-20T06
2014-04-25T06
2014-04-30T06

This form can be used to get special behaviour relative to
the final cycle point.

This form can be used to get special behaviour near the end of the workflow, at
the final cycle point. We can also represent this in Cylc with a collapsed form:
We can also represent this in Cylc with a collapsed form:

.. code-block:: none

Expand All @@ -393,12 +423,41 @@ So, for example, you can write:
.. code-block:: sub

R1//+P0D # Run once at the final cycle point
R5/P1D # Run 5 times, every 1 day, ending at the final
# cycle point
P2W/T00 # Run every 2 weeks ending at 00:00 following
# the final cycle point
R//T00 # Run every 1 day ending at 00:00 following the
# final cycle point
R5/P2D # Run 5 times, every 2 days, ending at the final cycle point
P2W/T00 # Run every 2 weeks ending at 00:00 before/at the final cycle point
R//T00 # Run every day ending at 00:00 before/at the final cycle point


Format 1: ``R[limit?]/[datetime]/[datetime]``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A less common recurrence form is ``R[limit?]/[datetime]/[datetime]``
(format number 1 in the ISO 8601 standard). This uses the difference between
the first datetime and the second datetime to set the recurrence interval.
The first datetime is the start point. For example,
``R3/2020-07-10/2020-07-15`` means "run 3 times, every 5 days, starting at
2020-07-10 (midnight, July 10th 2020)"; the list of points on this sequence is:

.. code-block:: none

2020-07-10
2020-07-15
2020-07-20

.. caution::

Cylc will always calculate the interval in
:term:`exact datetime units <exact datetime unit>`. So for the example
of ``R/2004/2005``, the interval will be ``P366D`` (2004 is a leap year)
rather then ``P1Y``, because year is an
:term:`inexact unit <inexact datetime unit>`.

.. note::

In versions of Cylc prior to 8.0.0, this syntax was undocumented and
behaved differently, in a way which was not in accordance with the
:term:`ISO 8601` standard.


.. _referencing-the-initial-and-final-cycle-points:

Expand Down Expand Up @@ -464,7 +523,7 @@ Setting The Initial Cycle Point Relative To The Current Time
It does not work for alternative calendars like the 360, 365 or 366 day
calendars, or integer cycling.

The ``next`` and ``previous`` syntax can be used with truncated ISO8601
The ``next`` and ``previous`` syntax can be used with truncated ISO 8601
representations, to set the initial cycle point:
``next(Thh:mmZ)``, ``previous(T-mm)``; e.g.

Expand Down Expand Up @@ -510,7 +569,7 @@ Offsets used without ``next`` or ``previous`` are interpreted as offsets from "n
==================================== ==================

Relative initial cycle points also work with truncated dates, including
weeks and ordinal date, using ISO8601 truncated date representations.
weeks and ordinal date, using ISO 8601 truncated date representations.
Note that day-of-week should always be specified when using weeks. If a time
is not included, the calculation of the next or previous corresponding
point will be done from midnight of the current day.
Expand Down Expand Up @@ -580,7 +639,7 @@ This graph can be written more concisely, with the same result, like this:
Excluding Dates
^^^^^^^^^^^^^^^

:term:`datetimes <ISO8601 datetime>` can be excluded from a :term:`recurrence`
:term:`datetimes <ISO 8601 datetime>` can be excluded from a :term:`recurrence`
by an exclamation mark for example ``PT1D!20000101`` means run daily except on
the first of January 2000.

Expand Down