CI: reduce docker image sizes

[klensy]

Reduces docker image sizes by using simple tips like: cleaning packet managers cache, squashing sequential installation steps into one.

For some images this gives ~40mb for apt-based images (not so much), but ~200mb(!) for centos one.

[rustbot]

r? @jyn514

(rustbot has picked a reviewer for you, use r? to override)

[klensy]

This change forwards --no-install-recommends for this packages too.

[klensy]

This change forwards --no-install-recommends for this packages too.

[Mark-Simulacrum]

Can you say more about what motivates the reduction in size here? It seems clear we can do better, but what use case is this impacting?

In general the complexity of the Dockerfiles matters much more to me than saving a little bit of time on image downloads and such.

[klensy]

Motivation is reducing size/traffic (slightly faster download, lower required disk space), given that added complexity of changes is trivial(cleaning apt/yum cache is non intrusive change used by many docker images).

[jyn514]

r? @Mark-Simulacrum

[Mark-Simulacrum]

I disagree that this is trivial. When we update to a new version of apt/yum, this directory path can change, and the costs of thinking about this if we do it is still constantly there. If it was a global setting somewhere ("give me container optimized defaults"), then this would be relatively speaking more reasonable, but if we have to do it across each of our builders that adds a good bit of overhead to maintaining them.

The savings of a few hundred megabytes at most are sort of nice, but they're also quite small - I don't think they justify having to think about this on every PR. Doing it as a one off isn't worth it either I think.

[klensy]

For apt: location /var/lib/apt/lists/ taken from man https://linux.die.net/man/8/apt-get, and looking at random google threads, it was here at least 10+ years, so it quite stable. Plus, this tip can be found at https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

For yum: it's exactly option yum clean all

For pip: it's option too, no manual rm -rf: --no-cache-dir

[Mark-Simulacrum]

OK. Well, I'm not convinced this is a good idea, but we can let it hit CI and see if anything breaks as a result of trying to be cautious here. It'll bounce at least a few times while we rebuild caches in any case, I suspect.

@bors r+ rollup=never

[bors]

📌 Commit eadf69a has been approved by Mark-Simulacrum

It is now in the queue for this repository.

[bors]

⌛ Testing commit eadf69a with merge e1d8195...

[bors]

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing e1d8195 to master...

[rust-timer]

Finished benchmarking commit (e1d8195): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.0%	[3.0%, 3.0%]	1
Regressions ❌ (secondary)	2.5%	[2.5%, 2.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	3.0%	[3.0%, 3.0%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

[ComputerDruid]

I noticed this seems to be adding a warning:

[WARNING]: Empty continuation line found in:
    RUN apt-get update && apt-get install -y --no-install-recommends   g++   gcc-multilib   make   ninja-build   file   curl   ca-certificates   python2.7   python3.9   git   cmake   sudo   gdb   llvm-13-tools   llvm-13-dev   libedit-dev   libssl-dev   pkg-config   zlib1g-dev   xz-utils   nodejs     apt-transport-https software-properties-common &&     curl -s "https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb" > packages-microsoft-prod.deb &&     dpkg -i packages-microsoft-prod.deb &&     apt-get update &&     apt-get install -y powershell     && rm -rf /var/lib/apt/lists/*
Warning: : Empty continuation lines will become errors in a future release.

For example in https://github.com/rust-lang/rust/actions/runs/3625431239/jobs/6113472422

[klensy]

@ComputerDruid it's about inline comments like here

rust/src/ci/docker/host-x86_64/x86_64-gnu-llvm-13/Dockerfile

Lines 27 to 30 in c5351ad

	nodejs \
	# Install powershell so we can test x.ps1 on Linux
	apt-transport-https software-properties-common && \

Should be fixed long ago moby/moby#35004, but idk why it still here.

[ComputerDruid]

I think it's actually complaining about the empty line above the comment, not the comment itself