ci: faster ci

[bolinocroustibat]

• Parallelise the execution of the tests in python job using 2 CircleCI executors, similar to what have been done in ci: parellelize tests in CI datagouv/hydra#238, except than here, the tests that are executed in parallel must be manually specified in the CI file, since some tests can't be parallelised. Therefore, the CI code is more complicated.
  Building CI workflow is now around 1m40s faster on average compared to before parallelisation (which is ~30% faster , thanks to ~45% faster specifically on the python-tests step)

Before:

After:

• Modify inv tests task so we can pass specific tests, to be used by parallelization
• Add fallback cache in CI

FYI, this other strategies to improve CI speed have also been tried and abandoned:

• Parallelise the execution of the 3 tasks in assets job using 3 CircleCI executors. -> While this was working well, this didn't result in any time gain as it was already probably executed in parallel
• Cache npm dependencies more aggressively -> didn't result in any significant improvements
• Use npm ci instead of npm install. Advantages:
  • Faster installation (can be 2-3x faster) because it skips certain user-oriented features
  • Deterministic installations - strictly uses package-lock.json
  • Removes node_modules before installation to ensure clean slate
  • Won't write to package.json or package-lock.json
    Disadvantage: many issues related to the command not being recognized in later steps

[maudetes]

I'm unclear about the way it works, but don't we now have 1,4K tests instead of 2K?