Skip to content

CI/CD Improvements #7250

Closed
Closed
@ericspod

Description

@ericspod

Is your feature request related to a problem? Please describe.
There are a number of things with our CI/CD setup that could be optimised or fixed.

  • The environment-dev.yml file isn't well tested for various environments. For example, on Ubuntu 20.04 trying to create a new environment with conda env create -f environment-dev.yml requires hours to resolve the environment configuration, and then fails when attempting to build tokenizers which is used by transformers. This is caused by there being no pre-build wheels for Python 3.11 which is what the environment file will select by default. Our CI/CD system doesn't seem to catch this kind of issue.
  • Currently the full tests with every dependency installed only use Python 3.8, this version will be dropped soon but all supported versions should be checked with these tests weekly to catch the above issue.
  • The versions of libraries mentioned in requirements/environments, such as Pytorch or Numpy, files should be upgraded to more recent ones on a set schedule.
  • What tests to run can be set in the workflows so that tests can be skipped when files which aren't source or unit tests are changed, eg. when changing docs. This can be done with Actions syntax.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions