.github/workflows/: Test all optional packages, add workflow for testing all experimental packages

[mkoeppe]

Follow up from #29341.

Depends on #29174
Depends on #29823
Depends on #20104

CC: @jhpalmieri @novoselt @kliem @orlitzky

Component: porting

Author: Matthias Koeppe

Branch/Commit: 6afe959

Reviewer: Jonathan Kliem

Issue created by migration from https://trac.sagemath.org/ticket/29901

[mkoeppe]

Dependencies: #29174

[mkoeppe]

Changed dependencies from #29174 to #29174, #29823

[mkoeppe]

Branch: u/mkoeppe/_github_workflows___test_all_optional_packages__add_workflow_for_testing_all_experimental_packages

[mkoeppe]

comment:5

The present version has an explicit list of the packages to the workflow scripts. This could be made less maintenance-intensive by #29902

---

New commits:

c182f48	trac 29754: remove packages backporting Python 3 components to Python 2.
05a6937	Merge commit 'c559f26caf56378c8c4a5652f577986b4f5d431c' of git://trac.sagemath.org/sage into t/29754/no-py2-backports
332778d	.github/workflows/ci-cygwin*: Remove build of removed package singledispatch
b389d77	Merge branch 't/29745/cleanup-spkg-rst-files' into t/29174/remove_autotools_spkg
963737e	build/pkgs/autotools: Remove
91b5ae8	Merge branches 't/29823/_github_workflows_ci_cygwin___remove_build_of_removed_packages' and 't/29174/remove_autotools_spkg' into t/29901/_github_workflows___test_all_optional_packages__add_workflow_for_testing_all_experimental_packages
f582222	.github/workflows/: Test all optional packages, add workflow for testing all experimental packages

[mkoeppe]

Commit: f582222

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f87c885

.github/workflows/: Test all optional packages, add workflow for testing all experimental packages

[sagetrac-git]

Changed commit from f582222 to f87c885

[mkoeppe]

comment:7

.github/workflows/: Test all optional packages, add workflow for testing all experimental packages · mkoeppe/sage@f87c885
https://github.com/mkoeppe/sage/actions/runs/141135018

.github/workflows/: Test all optional packages, add workflow for testing all experimental packages · mkoeppe/sage@f87c885
https://github.com/mkoeppe/sage/actions/runs/141135010

[mkoeppe]

Changed dependencies from #29174, #29823 to #29174, #29823, #20104

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

94ec3c9	Add sage --package update-latest command
cb7a291	Fixup rebase
1d36a2e	build/bin/sage-system-python: Try python first
a91ecab	Merge branch 't/29890/build_bin_sage_system_python__try__python__first' into t/20104/automatic_updates_of_pip_type_packages
ba6fbb5	Merge branch 't/20104/automatic_updates_of_pip_type_packages' into t/29901/_github_workflows___test_all_optional_packages__add_workflow_for_testing_all_experimental_packages
4dc7367	.github/workflows: Replace explicit optional/experimental lists by using sage-package
efb14c3	Suppress linebreaks

[sagetrac-git]

Changed commit from f87c885 to efb14c3

[mkoeppe]

comment:10

https://github.com/mkoeppe/sage/runs/789828733

Error processing tar file(exit status 1): write /sage/local/share/pari/elldata/ell366: no space left on device
Error response from daemon: Error processing tar file(exit status 1): write /sage/local/share/stein_watkins/a.010.bz2: no space left on device

[jhpalmieri]

comment:11

Maybe we don't need to test database_stein_watkins much: if database_stein_watkins_mini works, then since they use the same spkg-install.in, the only issue, I think, is whether the big tarball is corrupt.

[mkoeppe]

comment:12

Right. There was apparently a point to having that old category of "huge" packages. Wondering how that got lost. --> #29919 (Restore "huge" package type)

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2a38f51

.github/workflows/tox-optional.yml: Do not test huge packages

[sagetrac-git]

Changed commit from efb14c3 to 2a38f51

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

1fcafe8

.github/workflows/tox-optional.yml: Do not test huge packages

[sagetrac-git]

Changed commit from 2a38f51 to 1fcafe8

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

bd703e4

.github/workflows/tox-optional.yml: Do not test huge packages

[sagetrac-git]

Changed commit from 1fcafe8 to bd703e4

[mkoeppe]

comment:16

New test at https://github.com/mkoeppe/sage/actions/runs/141954265

[mkoeppe]

comment:17

Filtering out the known huge packages works now.

But there are many timeouts. Will need to split "optional" into 2 parts.

[sagetrac-git]

Changed commit from bd703e4 to fc59886

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

fc59886

Merge tag '9.2.beta5' into t/29901/_github_workflows___test_all_optional_packages__add_workflow_for_testing_all_experimental_packages

[sagetrac-git]

Changed commit from fc59886 to 1a408a8

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

1a408a8

.github/workflows/tox-optional.yml: Split jobs

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

50556a7

Fixup syntax

[sagetrac-git]

Changed commit from 1a408a8 to 50556a7

[mkoeppe]

comment:22

Tests for experimental packages at https://github.com/mkoeppe/sage/actions/runs/170318716

Tests for optional packages at https://github.com/mkoeppe/sage/actions/runs/170362341

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

comment:24

Still exceeds 2 x 6h on ubuntu-xenial, need to split it up more

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

6e3051d

Split up into more jobs

[sagetrac-git]

Changed commit from 50556a7 to 6e3051d

[mkoeppe]

comment:26

Tests at https://github.com/mkoeppe/sage/actions/runs/170932419 and https://github.com/mkoeppe/sage/actions/runs/170932412

[mkoeppe]

comment:27

Should remove the testing of sage_numerical_backends_cplex, sage_numerical_backends_gurobi, which require the commercial packages to be installed first

[sagetrac-git]

Changed commit from 6e3051d to 6afe959

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

6c82896	Even more splitting
6afe959	Do not try to test packages that need cplex, gurobi

[kliem]

comment:29

Why are there green checkmarks, if things haven't worked out?

https://github.com/mkoeppe/sage/runs/876312084?check_suite_focus=true

[mkoeppe]

comment:30

Targets listed in TARGETS_OPTIONAL ignore errors.

[kliem]

comment:31

Why is that? There is probably a good reason, but I'm curious. How do I efficiently use it then.

[mkoeppe]

comment:32

Unfortunately a test run can only get a red or a green status on GH Actions.
TARGETS_OPTIONAL is designed for additional tests of things that are likely to fail - such as ptest in tox.yml, which always has 1 or 2 failures that we don't want to be signaled by a red status.

There's only a rather shallow reason why currently use TARGETS_OPTIONAL for testing the optional and experimental packages. At the moment the majority of the optional and experimental packages fail, so all of these test runs will fail. Thus there is no information either way. But the green checkmarks look more friendly.

We can change this when we have made some progress in #29900 (Meta-ticket: Fix optional and experimental packages for Sage 9.2).

[mkoeppe]

comment:33

Replying to @kliem:

How do I efficiently use it then.

Look at the "This run" tab of the whole workflow and search for package names that you're interested in.

[kliem]

comment:34

Thanks. Yes, sounds reasonable.

From my point of view, optional should mean almost as safe as sage itself and experimental is for all stuff that might or might not work (I guess there are runtime failures as well that make stuff experimental). So adding a doctesting framework for this seems great.

[mkoeppe]

comment:35

Replying to @kliem:

From my point of view, optional should mean almost as safe as sage itself and experimental is for all stuff that might or might not work (I guess there are runtime failures as well that make stuff experimental).

Yes, I agree with this interpretation.

[kliem]

Reviewer: Jonathan Kliem

[kliem]

comment:36

I think this is a good improvement as it is.

[mkoeppe]

comment:37

Thanks!

[vbraun]

Changed branch from u/mkoeppe/_github_workflows___test_all_optional_packages__add_workflow_for_testing_all_experimental_packages to 6afe959