From 02e2c0d0edf422583a522553dfdbd4a47741caa1 Mon Sep 17 00:00:00 2001 From: William Chargin Date: Thu, 23 Jan 2020 10:30:16 -0800 Subject: [PATCH 1/2] =?UTF-8?q?ci:=20don=E2=80=99t=20store=20Pip=20HTTP=20?= =?UTF-8?q?cache?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Summary: The Pip HTTP cache stores all downloaded wheels and is never evicted. With new `tf-nightly` wheels every day, this adds up quickly. We last cleared our Travis caches about a month ago, and they’re up to 14.3 GB. Investigation shows that the Pip HTTP cache accounts for the majority of the cache (about 70% after about a month of cache accrual), and also that jobs with larger caches have significantly longer startup times, with delta on the order of 8 minutes (again, after about a month). Also, uploading large caches at the end of a job can take minutes, and Travis doesn’t report success until this finishes. Fetching `tf-nightly` should be comparatively cheap. Caches may need to be cleared for this to take effect. We’ll find out. Test Plan: See what Travis thinks. wchargin-branch: ci-drop-pip-http-cache --- .travis.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 4fe52c9b4c..6e4b0d26a3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -23,8 +23,6 @@ env: - TF_VERSION_ID= # Do not install TensorFlow in this case cache: - # Reuse the pip cache directory across build machines. - pip: true # Cache directories for Bazel. See ci/bazelrc for details. directories: - $HOME/.cache/tb-bazel-repo From 5af3b8249e678e40e80217baacf13504d3584cc5 Mon Sep 17 00:00:00 2001 From: William Chargin Date: Thu, 23 Jan 2020 11:29:45 -0800 Subject: [PATCH 2/2] [update patch] wchargin-branch: ci-drop-pip-http-cache wchargin-source: 87d25ebd4ffd6db7bff21fa755b897892df1488d --- .travis.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.travis.yml b/.travis.yml index 6e4b0d26a3..3bdd2ea0a8 100644 --- a/.travis.yml +++ b/.travis.yml @@ -23,6 +23,10 @@ env: - TF_VERSION_ID= # Do not install TensorFlow in this case cache: + # Don't cache the Pip directory. We pull in a new `tf-nightly` wheel + # every day, and Pip caches are never evicted, so this quickly bloats + # to many gigabytes and adds minutes to the CI time. + pip: false # Cache directories for Bazel. See ci/bazelrc for details. directories: - $HOME/.cache/tb-bazel-repo