Closed
Description
Is your feature request related to a problem? Please describe.
There are a number of things with our CI/CD setup that could be optimised or fixed.
- The
environment-dev.yml
file isn't well tested for various environments. For example, on Ubuntu 20.04 trying to create a new environment withconda env create -f environment-dev.yml
requires hours to resolve the environment configuration, and then fails when attempting to buildtokenizers
which is used bytransformers
. This is caused by there being no pre-build wheels for Python 3.11 which is what the environment file will select by default. Our CI/CD system doesn't seem to catch this kind of issue. - Currently the full tests with every dependency installed only use Python 3.8, this version will be dropped soon but all supported versions should be checked with these tests weekly to catch the above issue.
- The versions of libraries mentioned in requirements/environments, such as Pytorch or Numpy, files should be upgraded to more recent ones on a set schedule.
- What tests to run can be set in the workflows so that tests can be skipped when files which aren't source or unit tests are changed, eg. when changing docs. This can be done with Actions syntax.