Skip to content

Commit

Permalink
gh-37206: Various edits of developer guide
Browse files Browse the repository at this point in the history 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes #1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Mostly small edits and

- updated sections on testing facilities
- added how to go back exited container.
- moved the section "Testing GitHub Actions locally" to an appropriate
place.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes #12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- #12345: short description why this is a dependency
- #34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: #37206
Reported by: Kwankyu Lee
Reviewer(s): Kwankyu Lee, Matthias Köppe
  • Loading branch information
Release Manager committed Feb 21, 2024
2 parents 726052b + 311911e commit abc7099
Show file tree
Hide file tree
Showing 3 changed files with 103 additions and 82 deletions.
118 changes: 40 additions & 78 deletions src/doc/en/developer/portability_testing.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,40 +10,44 @@ Testing on Multiple Platforms

Sage is intended to build and run on a variety of platforms,
including all major Linux distributions, as well as macOS, and
Windows (with WSL).
Windows with WSL (Windows Subsystem for Linux).

There is considerable variation among these platforms.
To ensure that Sage continues to build correctly on users'
machines, it is crucial to test changes to Sage, in particular
when external packages are added or upgraded, on a wide
spectrum of platforms.

GitHub actions
==============

The `GitHub Actions
<https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions>`_
will automatically test your GitHub PR by attempting an incremental build of
Sage and running doctests.
Testing PRs with GitHub Actions
===============================

`GitHub Actions <https://github.com/sagemath/sage/actions>`_ are automatically
and constantly testing GitHub PRs to identify errors early and ensure code
quality. In particular, Build & Test workflows perform an incremental build of
Sage and run doctests on a selection of major platforms including Ubuntu,
macOS, and Conda.


Sage buildbots
==============

The `Sage Release buildbot <https://wiki.sagemath.org/buildbot>`_
builds entire tarballs (e.g., all the development releases) on a
variety of machines.
Before a new release, the release manager runs a fleet of `buildbots
<http://build.sagemath.org>`_ to make it sure that Sage builds correctly on all
of our supported platforms.

Developers' and users' tests on sage-release
============================================

Sage developers and users are encouraged to contribute to testing
releases that are announced on `Sage Release
<https://groups.google.com/forum/#!forum/sage-release>`_ on their
machines and to report test results (success and failures) by
responding to the announcements.
Test reports on sage-release
============================

Testing Sage on a different platform using Docker
=================================================
Sage developers and users are encouraged to test releases that are announced on
`Sage Release <https://groups.google.com/forum/#!forum/sage-release>`_ on their
machines and to report the results (successes and failures) by responding to the
announcements.


Testing on multiple platforms using Docker
==========================================

`Docker <https://www.docker.com>`_ is a popular virtualization
software, running Linux operating system images ("Docker images") in
Expand Down Expand Up @@ -282,22 +286,32 @@ Let us install a subset of these packages::
Committing a container to disk
------------------------------

After terminating the container, we can create a new image
corresponding to its current state::
After terminating the container, the following command shows the status
of the container you just exited::

root@39d693b2a75d:/sage# ^D
[mkoeppe@sage worktree-ubuntu-latest]$ docker ps -a | head -n3
CONTAINER ID IMAGE COMMAND CREATED STATUS
39d693b2a75d ubuntu:latest "/bin/bash" 8 minutes ago Exited (0) 6 seconds ago
9f3398da43c2 ubuntu:latest "/bin/bash" 8 minutes ago Exited (0) 8 minutes ago

We can go back to the container with the command::

[mkoeppe@sage worktree-ubuntu-latest]$ docker start -a -i 39d693b2a75d
root@9f3398da43c2:/#

Here, ``39d693b2a75d`` is the container id, which appeared in the
shell prompts and in the output of ``docker ps``.

We can create a new image corresponding to its current state::

root@39d693b2a75d:/# ^D
[mkoeppe@sage worktree-ubuntu-latest]$ docker commit 39d693b2a75d ubuntu-latest-minimal-17
sha256:4151c5ca4476660f6181cdb13923da8fe44082222b984c377fb4fd6cc05415c1

Here, ``39d693b2a75d`` was the container id (which appeared in the
shell prompts and in the output of ``docker ps``), and
``ubuntu-latest-minimal-17`` is an arbitrary symbolic name for the new
image. The output of the command is the id of the new image. We can
use either the symbolic name or the id to refer to the new image.
where ``ubuntu-latest-minimal-17`` is an arbitrary symbolic name for the new
image. The output of the command is the id of the new image. We can use either
the symbolic name or the id to refer to the new image.

We can run the image and get a new container with the same state as
the one that we terminated. Again we want to mount our worktree into
Expand Down Expand Up @@ -1180,58 +1194,6 @@ are available:
.. include:: portability_platform_table.rst


Testing GitHub Actions locally
==============================

`act <https://github.com/nektos/act>`_ is a tool, written in Go, and using Docker,
to run GitHub Actions locally; in particular, it speeds up developing Actions.
We recommend using ``gh extension`` facility to install ``act``. ::

[alice@localhost sage]$ gh extension install https://github.com/nektos/gh-act

Extra steps needed for configuration of Docker to run Actions locally can be found on
`act's GitHub <https://github.com/nektos/act>`_

Here we give a very short sampling of ``act``'s capabilities. If you installed standalone
``act``, it should be invoked as ``act``, not as ``gh act``.
After the set up, one can e.g. list all the available linting actions::

[alice@localhost sage]$ gh act -l | grep lint
0 lint-pycodestyle Code style check with pycodestyle Lint lint.yml push,pull_request
0 lint-relint Code style check with relint Lint lint.yml push,pull_request
0 lint-rst Validate docstring markup as RST Lint lint.yml push,pull_request
[alice@localhost sage]$

run a particular action ``lint-rst`` ::

[alice@localhost sage]$ gh act -j lint-rst
...

and so on.

By default, ``act`` pulls all the data needed from the next, but it can also cache it,
speeding up repeated runs quite a lot. The following repeats running of ``lint-rst`` using cached data::

[alice@localhost sage]$ gh act -p false -r -j lint-rst
[Lint/Validate docstring markup as RST] Start image=catthehacker/ubuntu:act-latest
...
| rst: commands[0] /home/alice/work/software/sage/src> flake8 --select=RST
| rst: OK (472.60=setup[0.09]+cmd[472.51] seconds)
| congratulations :) (474.10 seconds)
...
[Lint/Validate docstring markup as RST] Success - Main Lint using tox -e rst
[Lint/Validate docstring markup as RST] Run Post Set up Python
[Lint/Validate docstring markup as RST] docker exec cmd=[node /var/run/act/actions/actions-setup-python@v4/dist/cache-save/index.js] user= workdir=
[Lint/Validate docstring markup as RST] Success - Post Set up Python
[Lint/Validate docstring markup as RST] Job succeeded

Here ``-p false`` means using already pulled Docker images, and ``-r`` means do not remove Docker images
after a successful run which used them. This, and many more details, can be found by running ``gh act -h``, as well
as reading ``act``'s documentation.

.. This sectuion is a stub.
More Sage-specfic details for using ``act`` should be added. PRs welcome!
Using our pre-built Docker images for development in VS Code
============================================================

Expand Down
65 changes: 62 additions & 3 deletions src/doc/en/developer/tools.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

.. _chapter-tools:

========================================
Additional Development and Testing Tools
========================================
=============================
Development and Testing Tools
=============================

.. _section-tools-tox:

Expand Down Expand Up @@ -108,6 +108,7 @@ detailed HTML report is generated.
**coverage** package is installed into the normal Sage environment, and
Sage is invoked from there.


.. _section-tools-coverage:

Coverage
Expand Down Expand Up @@ -277,6 +278,7 @@ Cython-lint
`Cython-lint <https://pypi.org/project/cython-lint/>`_ checks Cython source files
for coding style.


.. _section-tools-ruff:

Ruff
Expand All @@ -288,6 +290,7 @@ for Python code, written in Rust.
It comes with a large choice of possible checks, and has the capacity
to fix some of the warnings it emits.


.. _section-tools-relint:

Relint
Expand Down Expand Up @@ -402,3 +405,59 @@ Pyright
Pyflakes
========
`Pyflakes <https://github.com/PyCQA/pyflakes>`_ checks for common coding errors.


.. _section-act:

Act
===

`act <https://github.com/nektos/act>`_ is a tool, written in Go, and using Docker,
to run GitHub Actions locally; in particular, it speeds up developing Actions.
We recommend using ``gh extension`` facility to install ``act``. ::

[alice@localhost sage]$ gh extension install https://github.com/nektos/gh-act

Extra steps needed for configuration of Docker to run Actions locally can be found on
`act's GitHub <https://github.com/nektos/act>`_

Here we give a very short sampling of ``act``'s capabilities. If you installed standalone
``act``, it should be invoked as ``act``, not as ``gh act``.
After the set up, one can e.g. list all the available linting actions::

[alice@localhost sage]$ gh act -l | grep lint
0 lint-pycodestyle Code style check with pycodestyle Lint lint.yml push,pull_request
0 lint-relint Code style check with relint Lint lint.yml push,pull_request
0 lint-rst Validate docstring markup as RST Lint lint.yml push,pull_request
[alice@localhost sage]$

run a particular action ``lint-rst`` ::

[alice@localhost sage]$ gh act -j lint-rst
...

and so on.

By default, ``act`` pulls all the data needed from the next, but it can also cache it,
speeding up repeated runs quite a lot. The following repeats running of ``lint-rst`` using cached data::

[alice@localhost sage]$ gh act -p false -r -j lint-rst
[Lint/Validate docstring markup as RST] Start image=catthehacker/ubuntu:act-latest
...
| rst: commands[0] /home/alice/work/software/sage/src> flake8 --select=RST
| rst: OK (472.60=setup[0.09]+cmd[472.51] seconds)
| congratulations :) (474.10 seconds)
...
[Lint/Validate docstring markup as RST] Success - Main Lint using tox -e rst
[Lint/Validate docstring markup as RST] Run Post Set up Python
[Lint/Validate docstring markup as RST] docker exec cmd=[node /var/run/act/actions/actions-setup-python@v4/dist/cache-save/index.js] user= workdir=
[Lint/Validate docstring markup as RST] Success - Post Set up Python
[Lint/Validate docstring markup as RST] Job succeeded

Here ``-p false`` means using already pulled Docker images, and ``-r`` means do not remove Docker images
after a successful run which used them. This, and many more details, can be found by running ``gh act -h``, as well
as reading ``act``'s documentation.

.. This section is a stub.
More Sage-specfic details for using ``act`` should be added. PRs welcome!
2 changes: 1 addition & 1 deletion src/doc/en/developer/workspace.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
.. _chapter-workspace-setup:

=========================
Setting up your workspace
Setting up Your Workspace
=========================

.. _section-ide:
Expand Down

0 comments on commit abc7099

Please sign in to comment.