Save site-packages as cache in CircleCI job

[ydshieh]

What does this PR do?

Currently, we save ~/.cache/pip as cache. Take check_repository_consistency job as example:

• it installs [all, quality]
• loading cache takes ~ 45 seconds
• pip install takes ~ 3-4 minutes

If we save .pyenv/versions as cache too:

• loading this new extra cache takes ~ 2 min
• pip install takes 20 ~ 30 seconds

We gain 30 ~ 90 seconds (depending on CircleCI's states). Not a big absolute improvement. But for this job which total runtime is ~ 5m30s, we can say > 20% reduction. As check_repository_consistency and check_code_quality will be run for each PR's each push, probably it's nice to have such reduction.

WDYT?

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

Do we still get updates if there is a release of one of the libraries?

[amyeroberts]

If we save .pyenv/versions as cache too:

loading this new extra cache takes ~ 2 min
pip install takes 20 ~ 30 seconds

@ydshieh Wait - are the timings of these in the right order?

[ydshieh]

Do we still get updates if there is a release of one of the libraries?

@sgugger Yes if we use -U flag. I can do that if you are ok

[ydshieh]

If we save .pyenv/versions as cache too:

loading this new extra cache takes ~ 2 min
pip install takes 20 ~ 30 seconds

@ydshieh Wait - are the timings of these in the right order?

Hmm, yes. But could you explain why you have doubts so I can reply in more details?

[sgugger]

@sgugger Yes if we use -U flag. I can do that if you are ok
YEs, please!

[amyeroberts]

Hmm, yes. But could you explain why you have doubts so I can reply in more details?

@ydshieh I just realised my mistake 🙃 I thought it was saying that it takes 2 mins to load with the cache and 30-40s to install by pip. Whereas it's (45 secs + 3-4 mins ) -> (2 mins + 20-30s). My bad!

[ydshieh]

Hmm, yes. But could you explain why you have doubts so I can reply in more details?

@ydshieh I just realised my mistake 🙃 I thought it was saying that it takes 2 mins to load with the cache and 30-40s to install by pip. Whereas it's (45 secs + 3-4 mins ) -> (2 mins + 20-30s). My bad!

The new one should be (45 secs + 2 mins + 20-30s): The first part of cache (in .cache/pip) is not changed.
But we still have a little gain overall.

[ydshieh]

Although already been approved - FYI: I just added -U everywhere