Stuck on a mysterious error during CI?

Sometimes, tests pass locally but not during CI in GitHub Actions. Those errors can be hard to track down because they are linked to building the documentation or checks of example codes.

Here's a short list of such known errors that can happen, to avoid being stuck on those again once we've lost track months (years?) later:

1. The typical: Package versions tend to change during CI's setup, updating to newest versions. Quite often those contain new warning about future deprecations or syntax changes. Those can be impossible to correct in our own package if they depend on another package (e.g. numpy warning in a scikit package), and can result in failure during tests. Solution: list those in the exclusion list of specific tests or, in case of the documentation test, in a dedicated list found in tests/test_docs.py:

ignored_warnings = ["Starting a Matplotlib GUI outside of the main thread",
                        ".*fetching the attribute.*Polygon.*",]

2. The sneaky: Defining a function with def in an example code (in /docs/source/code, which is used to check that the code snapshots shown in our documentation do work or to create documentation-specific figures) will result in a failure of the CI in GitHub Actions. This failure happens for the exec() in the function test_example_code in tests/test_docs.py. This issue is related to defining global variables through exec(), details here. Solution: don't define any function within those scripts or, if it is unavoidable, define them directly in xdem!

exec(infile.read().replace("plt.show()", "plt.close()"))

3. The mysterious: Building sphinx during CI in GitHub Actions with several cores might result in failure. The cause is as-yet unknown (tried reverting to older sphinx versions without any success). Solution: use a single core.

sphinx.cmd.build.main(["-j", str(N_cores), os.path.join(self.docs_dir, "source/"), os.path.join(self.docs_dir, "build/html")])

4. The unexpected: Initialization of np.ndarray full of NaNs could historically be done in many ways: multiplying or summing a np.zeros, np.ones or np.empty with a np.nan. However, recently, RuntimeWarning have started to pop up a bit randomly with these methods, resulting in test failres. We could not track down the exact version change that triggered this (might be unexpected behaviour, so not in the logs), but it does seems to depend on the version of numpy. Solution: use np.full(shape, np.nan, dtype) to initialize those arrays!

values = np.empty((n_bins, unique_indices.shape[0]), dtype=float) * np.nan
E       RuntimeWarning: invalid value encountered in multiply

5. The baffling: For some reason, numba sometimes does not recognized expressions such as sum (e.g., https://github.com/GlacioHack/xdem/runs/7742646090?check_suite_focus=true) while clearly supported (https://numba.readthedocs.io/en/stable/reference/pysupported.html?highlight=sum). Probably because of a slightly different environment setup by mamba, but still... The one who solves that one will get a cake.

6. The futuristic: Typing annotations are cool, but sometimes they don't exist yet. If you want to use tuple[] or list[] to directly annotate your function input and output, make sure that you first do from __future__ import annotations, otherwise you'll get a not-so-nice:

TypeError: 'type' object is not subscriptable

7. The parallelized: Getting errors in CI while reading/saving a vector file? a raster? or maybe even more sneaky: a bit randomly, a different nodata is associated with the raster data? Check out if pytest-xdist is used, i.e. if an argument such as -n=auto is passed to pytest. Errors can be mysterious as they can simply lead to a Raster full of nodatas:

Warning: All input values are the same.

8. The timely: Getting error about Permission access for writing temporary files on Windows? The tempfile module is a bit more tricky on Windows, for example for NamedTemporaryFiles:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later). To be able to write/read several times, we need to ensure the mode="w" and delete=False options are selected (and for some files with the right suffix as well):

    outfile = tempfile.NamedTemporaryFile(suffix=".tif", mode="w", delete=False)