Skip to content

Conversation

@wchargin
Copy link
Contributor

Summary:
Our Travis caches are steadily growing. I suspect that the Pip cache of
TensorFlow nightlies plays a role here, as there are new packages every
day and the old ones are never evicted. Let’s find out.

This is a test PR just to trigger CI. (It would be possible to actually
merge some logging like this, but we’d want to make sure that it doesn’t
slow down the actual success reporting. Therefore:
DO NOT SUBMIT

wchargin-branch: ci-inspect-cache

Summary:
Our Travis caches are steadily growing. I suspect that the Pip cache of
TensorFlow nightlies plays a role here, as there are new packages every
day and the old ones are never evicted. Let’s find out.

This is a test PR just to trigger CI. (It would be possible to actually
merge some logging like this, but we’d want to make sure that it doesn’t
slow down the actual success reporting. Therefore:
DO NOT SUBMIT

wchargin-branch: ci-inspect-cache
@wchargin
Copy link
Contributor Author

Okay, so for the job running on tf-nightly we have (du -b output):

7316426842	/home/travis/.cache
5176123975	/home/travis/.cache/pip
5174556037	/home/travis/.cache/pip/http
1442037696	/home/travis/.cache/tb-bazel-disk
697183623	/home/travis/.cache/tb-bazel-repo

…whereas for the notf job we have:

2292477836	/home/travis/.cache
124713379	/home/travis/.cache/pip
122817873	/home/travis/.cache/pip/http
1469499286	/home/travis/.cache/tb-bazel-disk
697183623	/home/travis/.cache/tb-bazel-repo

This generally makes sense: the Bazel cache sizes are the same, but the
Pip cache size is far larger on the tf-nightly since it’s downloading
a new 450 MB wheel every day and never evicting old ones.

(There’s about a megabyte unaccounted for here, which seems negligible
and which I will therefore neglect.)

Also, just getting to the point where our Travis script is actually
executing takes 9m40s on tf-nightly and 1m31s on notf, so it seems
like there’s a bunch to gain here.

I will disable Pip caching and see what happens.

@wchargin
Copy link
Contributor Author

This PR has served its purpose, and may now rest.

(See #3167.)

@wchargin wchargin closed this Jan 23, 2020
@wchargin wchargin deleted the wchargin-ci-inspect-cache branch January 23, 2020 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants