Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/cmd/relui: reduce Go download sizes #27151

Closed
bradfitz opened this issue Aug 22, 2018 · 27 comments
Closed

x/build/cmd/relui: reduce Go download sizes #27151

bradfitz opened this issue Aug 22, 2018 · 27 comments
Labels
binary-size Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@bradfitz
Copy link
Contributor

bradfitz commented Aug 22, 2018

Go1's source download was 11 MB.
Go 1.11rc1's source download is 20MB.
Go 1.4.3's Linux download was 58 MB.
Go 1.11rc1's Linux download was 169 MB.

Clearly 169 MB is pretty ridiculous.

https://go-review.googlesource.com/c/build/+/129955 removed some useless *.a files to save 20 MB.

https://go-review.googlesource.com/c/build/+/130575 removed the blog to save 34MB.

We're now down to 116 MB for what will be the Go 1.11rc2 Linux download. Better, but not great.

This is a tracking bug for further improvements.

/cc @ianlancetaylor @andybons @dmitshur

@bradfitz bradfitz added this to the Go1.12 milestone Aug 22, 2018
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Aug 22, 2018
@bradfitz bradfitz added NeedsFix The path to resolution is known, but the work has not been done. and removed Builders x/build issues (builders, bots, dashboards) labels Aug 22, 2018
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Aug 22, 2018
@bradfitz
Copy link
Contributor Author

bradfitz commented Aug 22, 2018

Untarring the release and gzip-compressing each file before doing du stats gives a reasonable way to measure the cost of removing bits:

$ find . -type f -not -name '*.gz' -exec gzip {} \;

Here are the top directories:

127731439       .
127727343       ./go
83486614        ./go/pkg
54847292        ./go/pkg/tool
54843196        ./go/pkg/tool/linux_amd64
21066018        ./go/src
16095151        ./go/bin
15354919        ./go/pkg/linux_amd64_race
13273773        ./go/pkg/linux_amd64
8094311 ./go/src/cmd
3127907 ./go/src/cmd/vendor
2732975 ./go/doc
2138255 ./go/src/runtime
2102037 ./go/doc/gopher
2051880 ./go/pkg/linux_amd64_race/net
1933218 ./go/pkg/linux_amd64_race/go
1773043 ./go/misc
1752589 ./go/src/cmd/vendor/golang.org
1750311 ./go/pkg/linux_amd64/net
1748493 ./go/src/cmd/vendor/golang.org/x
  • It turns out that go/pkg/linux_amd64_race directory is not required. Go 1.11 will rebuild that stuff as needed. I verified with a Dockerfile (https://play.golang.org/p/O-IW_vVnol_g). That'll save us 15 MB.

  • Do we really need 2MB of Gopher images? We need some for the local godoc perhaps.

Biggest binaries:

$ du -b -a | sort -n -r | grep gz$ | head -20
8881040 ./go/pkg/tool/linux_amd64/compile.gz
7957541 ./go/bin/godoc.gz
6467215 ./go/pkg/tool/linux_amd64/pprof.gz
6435168 ./go/bin/go.gz
5961018 ./go/pkg/tool/linux_amd64/tour.gz
5183556 ./go/pkg/tool/linux_amd64/trace.gz
3756217 ./go/pkg/tool/linux_amd64/vet.gz
2739853 ./go/pkg/tool/linux_amd64/link.gz
2451059 ./go/pkg/tool/linux_amd64/cover.gz
2257606 ./go/pkg/tool/linux_amd64/cgo.gz
2187914 ./go/pkg/tool/linux_amd64/doc.gz
2145237 ./go/pkg/tool/linux_amd64/asm.gz
2060780 ./go/pkg/tool/linux_amd64/objdump.gz
1863347 ./go/pkg/tool/linux_amd64/addr2line.gz
1842210 ./go/pkg/tool/linux_amd64/nm.gz
1739594 ./go/pkg/tool/linux_amd64/dist.gz
1698346 ./go/bin/gofmt.gz
1608565 ./go/pkg/tool/linux_amd64/fix.gz
1324476 ./go/pkg/tool/linux_amd64/test2json.gz
1295801 ./go/pkg/tool/linux_amd64/buildid.gz
  • Do we really need to include the tour?

  • Can any binaries be combined into one binary and have different behavior based on invocation name, busybox style?

We have 6.6 MB of testdata:

$ du -b -c $(find -name testdata) | tail -1
6664279 total
  • Do we need tests to pass for people who download binary distributions?

  • Perhaps we should have a full download and a minimal download? (Minimal could lack godoc, gopher images, tests, test data, prebuilt *.a files, etc)

@bradfitz
Copy link
Contributor Author

bradfitz commented Aug 22, 2018

We ship three copies of Mark Twain's Tom Sawyer?

$ ls -l $(find . | grep -i twain)
-rw-r--r-- 1 bradfitz bradfitz 118509 Nov 22  2017 ./compress/bzip2/testdata/Mark.Twain-Tom.Sawyer.txt.bz2
-rw-r--r-- 1 bradfitz bradfitz 387851 Nov 22  2017 ./compress/testdata/Mark.Twain-Tom.Sawyer.txt
-rw-r--r-- 1 bradfitz bradfitz 387851 Nov 22  2017 ./net/testdata/Mark.Twain-Tom.Sawyer.txt

@FiloSottile
Copy link
Contributor

About the pkg files that get rebuilt, a common issue in the past has been users not having write permissions on the system GOROOT, hence breaking or rebuilding every time. Will that be a problem?

@bradfitz
Copy link
Contributor Author

@FiloSottile, that hasn't been a problem since at least Go 1.10. It writes to the build cache instead. You can experiment with the Dockerfile linked above.

@ALTree
Copy link
Member

ALTree commented Aug 22, 2018

Do we really need to include the tour?

Currently we ship the tour so one can do go tool tour to run it locally, but alas that doesn't work if you built from source or you installed Go using the OS package manager, so we also have to explain in the welcome that if go tool tour doesn't work you have to go get it from the x repo.

This is all quite convoluted and I've seen reports of people confused by this. If we remove the tour from the binary releases we will also be able to simplify our instructions for running it locally: it's always "go get it, and then run the gotour command".

@bradfitz
Copy link
Contributor Author

bradfitz commented Aug 22, 2018

In that case I would like to remove the tour, maybe even for Go 1.11.

@andybons
Copy link
Member

/cc @katiehockman

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/131156 mentions this issue: cmd/release: remove the tour from the releases

gopherbot pushed a commit to golang/build that referenced this issue Aug 24, 2018
This removes the tour from the release binary for go1.12.x and later
releases, lowering the size of each release. It will now redirect to
tour.golang.org, unless the user runs "go get -u golang.org/x/tour" to
cache it locally, in which case the local copy will be preferred.

The release size was 170.0 MiB for go1.11rc1, this change reduces the
size by 6.3 MiB, a 3.7% reduction. After this change, and others which
reduced the release size, the release candidate is now 115.2 MiB.

Updates golang/go#27151

Change-Id: I6687c34f3d4ae161c5e6df1f7af8cf3adc016fc4
Reviewed-on: https://go-review.googlesource.com/131156
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@golang golang deleted a comment from networkimprov Aug 26, 2018
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/135495 mentions this issue: compress: reduce the testing copies of the decompressed book from two

gopherbot pushed a commit that referenced this issue Sep 24, 2018
The previous book was 387 KiB decompressed and 119 KiB compressed, the
new book is 567 KiB decompressed and 132 KiB compressed. Overall, this
change will reduce the release binary size by 196 KiB. The new book will
allow for slightly more extensive compression testing with a larger
text.

Command to run the benchmark tests used with benchstat:
`../bin/go test -run='^$' -count=4 -bench=. compress/bzip2 compress/flate`

When running the benchmarks locally, changed "Newton" to "Twain" and
filtered the tests with the -bench flag to include only those which were
relevant to these changes.

benchstat results below:

name                            old time/op    new time/op     delta
DecodeTwain-8                     19.6ms ± 2%     24.1ms ± 1%  +23.04%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e4-8         140µs ± 3%      139µs ± 5%     ~     (p=0.886 n=4+4)
Decode/Twain/Huffman/1e5-8        1.27ms ± 3%     1.26ms ± 1%     ~     (p=1.000 n=4+4)
Decode/Twain/Huffman/1e6-8        12.4ms ± 0%     13.2ms ± 1%   +6.42%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e4-8           133µs ± 1%      123µs ± 1%   -7.35%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e5-8          1.20ms ± 0%     1.02ms ± 3%  -15.32%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e6-8          12.0ms ± 2%     10.1ms ± 3%  -15.89%  (p=0.029 n=4+4)
Decode/Twain/Default/1e4-8         131µs ± 6%      108µs ± 5%  -17.84%  (p=0.029 n=4+4)
Decode/Twain/Default/1e5-8        1.06ms ± 2%     0.80ms ± 1%  -24.97%  (p=0.029 n=4+4)
Decode/Twain/Default/1e6-8        10.0ms ± 3%      8.0ms ± 3%  -20.06%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e4-8     128µs ± 4%      115µs ± 4%   -9.70%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e5-8    1.04ms ± 2%     0.83ms ± 4%  -20.37%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e6-8    10.4ms ± 4%      8.1ms ± 5%  -22.25%  (p=0.029 n=4+4)
Encode/Twain/Huffman/1e4-8        55.7µs ± 2%     55.6µs ± 1%     ~     (p=1.000 n=4+4)
Encode/Twain/Huffman/1e5-8         441µs ± 0%      435µs ± 2%     ~     (p=0.343 n=4+4)
Encode/Twain/Huffman/1e6-8        4.31ms ± 4%     4.30ms ± 4%     ~     (p=0.886 n=4+4)
Encode/Twain/Speed/1e4-8           193µs ± 1%      166µs ± 2%  -14.09%  (p=0.029 n=4+4)
Encode/Twain/Speed/1e5-8          1.54ms ± 1%     1.22ms ± 1%  -20.53%  (p=0.029 n=4+4)
Encode/Twain/Speed/1e6-8          15.3ms ± 1%     12.2ms ± 3%  -20.62%  (p=0.029 n=4+4)
Encode/Twain/Default/1e4-8         393µs ± 1%      390µs ± 1%     ~     (p=0.114 n=4+4)
Encode/Twain/Default/1e5-8        6.12ms ± 4%     6.02ms ± 5%     ~     (p=0.486 n=4+4)
Encode/Twain/Default/1e6-8        69.4ms ± 5%     59.0ms ± 4%  -15.07%  (p=0.029 n=4+4)
Encode/Twain/Compression/1e4-8     423µs ± 2%      379µs ± 2%  -10.34%  (p=0.029 n=4+4)
Encode/Twain/Compression/1e5-8    7.00ms ± 1%     7.88ms ± 3%  +12.49%  (p=0.029 n=4+4)
Encode/Twain/Compression/1e6-8    76.6ms ± 5%     80.9ms ± 3%     ~     (p=0.114 n=4+4)

name                            old speed      new speed       delta
DecodeTwain-8                   19.8MB/s ± 2%   23.6MB/s ± 1%  +18.84%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e4-8      71.7MB/s ± 3%   72.1MB/s ± 6%     ~     (p=0.943 n=4+4)
Decode/Twain/Huffman/1e5-8      78.8MB/s ± 3%   79.5MB/s ± 1%     ~     (p=1.000 n=4+4)
Decode/Twain/Huffman/1e6-8      80.5MB/s ± 0%   75.6MB/s ± 1%   -6.03%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e4-8        75.2MB/s ± 1%   81.2MB/s ± 1%   +7.93%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e5-8        83.4MB/s ± 0%   98.6MB/s ± 3%  +18.16%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e6-8        83.6MB/s ± 2%   99.5MB/s ± 3%  +18.91%  (p=0.029 n=4+4)
Decode/Twain/Default/1e4-8      76.3MB/s ± 6%   92.8MB/s ± 4%  +21.62%  (p=0.029 n=4+4)
Decode/Twain/Default/1e5-8      94.4MB/s ± 3%  125.7MB/s ± 1%  +33.24%  (p=0.029 n=4+4)
Decode/Twain/Default/1e6-8       100MB/s ± 3%    125MB/s ± 3%  +25.12%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e4-8  78.4MB/s ± 4%   86.8MB/s ± 4%  +10.73%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e5-8  95.7MB/s ± 2%  120.3MB/s ± 4%  +25.65%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e6-8  96.4MB/s ± 4%  124.0MB/s ± 5%  +28.64%  (p=0.029 n=4+4)
Encode/Twain/Huffman/1e4-8       179MB/s ± 2%    180MB/s ± 1%     ~     (p=1.000 n=4+4)
Encode/Twain/Huffman/1e5-8       227MB/s ± 0%    230MB/s ± 2%     ~     (p=0.343 n=4+4)
Encode/Twain/Huffman/1e6-8       232MB/s ± 4%    233MB/s ± 4%     ~     (p=0.886 n=4+4)
Encode/Twain/Speed/1e4-8        51.8MB/s ± 1%   60.4MB/s ± 2%  +16.43%  (p=0.029 n=4+4)
Encode/Twain/Speed/1e5-8        65.1MB/s ± 1%   81.9MB/s ± 1%  +25.83%  (p=0.029 n=4+4)
Encode/Twain/Speed/1e6-8        65.2MB/s ± 1%   82.2MB/s ± 3%  +26.00%  (p=0.029 n=4+4)
Encode/Twain/Default/1e4-8      25.4MB/s ± 1%   25.6MB/s ± 1%     ~     (p=0.114 n=4+4)
Encode/Twain/Default/1e5-8      16.4MB/s ± 4%   16.6MB/s ± 5%     ~     (p=0.486 n=4+4)
Encode/Twain/Default/1e6-8      14.4MB/s ± 6%   17.0MB/s ± 4%  +17.67%  (p=0.029 n=4+4)
Encode/Twain/Compression/1e4-8  23.6MB/s ± 2%   26.4MB/s ± 2%  +11.54%  (p=0.029 n=4+4)
Encode/Twain/Compression/1e5-8  14.3MB/s ± 1%   12.7MB/s ± 3%  -11.08%  (p=0.029 n=4+4)
Encode/Twain/Compression/1e6-8  13.1MB/s ± 4%   12.4MB/s ± 3%     ~     (p=0.114 n=4+4)

name                            old alloc/op   new alloc/op    delta
DecodeTwain-8                     3.63MB ± 0%     3.63MB ± 0%   +0.15%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e4-8        42.0kB ± 0%     41.3kB ± 0%   -1.62%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e5-8        43.5kB ± 0%     45.1kB ± 0%   +3.74%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e6-8        71.7kB ± 0%     80.0kB ± 0%  +11.55%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e4-8          41.2kB ± 0%     41.3kB ± 0%     ~     (p=0.286 n=4+4)
Decode/Twain/Speed/1e5-8          45.1kB ± 0%     43.9kB ± 0%   -2.80%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e6-8          72.8kB ± 0%     81.3kB ± 0%  +11.72%  (p=0.029 n=4+4)
Decode/Twain/Default/1e4-8        41.2kB ± 0%     41.2kB ± 0%   -0.22%  (p=0.029 n=4+4)
Decode/Twain/Default/1e5-8        44.4kB ± 0%     43.0kB ± 0%   -3.02%  (p=0.029 n=4+4)
Decode/Twain/Default/1e6-8        71.0kB ± 0%     61.8kB ± 0%  -13.00%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e4-8    41.3kB ± 0%     41.2kB ± 0%   -0.29%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e5-8    43.3kB ± 0%     43.0kB ± 0%   -0.72%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e6-8    69.1kB ± 0%     63.7kB ± 0%   -7.90%  (p=0.029 n=4+4)

name                            old allocs/op  new allocs/op   delta
DecodeTwain-8                       51.0 ± 0%       51.2 ± 1%     ~     (p=1.000 n=4+4)
Decode/Twain/Huffman/1e4-8          15.0 ± 0%       14.0 ± 0%   -6.67%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e5-8          20.0 ± 0%       23.0 ± 0%  +15.00%  (p=0.029 n=4+4)
Decode/Twain/Huffman/1e6-8           134 ± 0%        161 ± 0%  +20.15%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e4-8            17.0 ± 0%       18.0 ± 0%   +5.88%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e5-8            30.0 ± 0%       31.0 ± 0%   +3.33%  (p=0.029 n=4+4)
Decode/Twain/Speed/1e6-8             193 ± 0%        228 ± 0%  +18.13%  (p=0.029 n=4+4)
Decode/Twain/Default/1e4-8          17.0 ± 0%       15.0 ± 0%  -11.76%  (p=0.029 n=4+4)
Decode/Twain/Default/1e5-8          28.0 ± 0%       32.0 ± 0%  +14.29%  (p=0.029 n=4+4)
Decode/Twain/Default/1e6-8           199 ± 0%        158 ± 0%  -20.60%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e4-8      17.0 ± 0%       15.0 ± 0%  -11.76%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e5-8      28.0 ± 0%       32.0 ± 0%  +14.29%  (p=0.029 n=4+4)
Decode/Twain/Compression/1e6-8       196 ± 0%        150 ± 0%  -23.47%  (p=0.029 n=4+4)

Updates #27151

Change-Id: I6c439694ed16a33bb4c63fbfb8570c7de46b4f2d
Reviewed-on: https://go-review.googlesource.com/135495
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/138495 mentions this issue: compress: move benchmark text from src/ to src/compress/testdata

gopherbot pushed a commit that referenced this issue Sep 28, 2018
This text is used mainly for benchmark compression testing, and in one
net test. The text was prevoiusly in a src/testdata directory, but since
that directory would only include one file, the text is moved to the
existing src/compression/testdata directory.

This does not cause any change to the benchmark results.

Updates #27151

Change-Id: I38ab5089dfe744189a970947d15be50ef1d48517
Reviewed-on: https://go-review.googlesource.com/138495
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/138737 mentions this issue: Revert "compress: move benchmark text from src/testdata to src/compress/testdata"

gopherbot pushed a commit that referenced this issue Oct 1, 2018
…ss/testdata"

This reverts commit 067bb44.

Reason for revert:
Failing Darwin-arm builds because that testing environment does not access testdata
from sibling directories. A future change will likely be made to move this testdata
out of src/testdata to create a solution that doesn't require the single-file directory.

Updates #27151

Change-Id: I8dbf5dd9512c94a605ee749ff4655cb00b0de686
Reviewed-on: https://go-review.googlesource.com/138737
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/143537 mentions this issue: cmd/release: fix tour after rename, do less tour work for Go 1.12+ releases

@bradfitz
Copy link
Contributor Author

bradfitz commented Oct 24, 2018

Now that the tour is removed from cmd/release (for Go 1.12), and CLI support is removed from godoc so it's only a webserver, is there any value in shipping the godoc webserver with releases?

I think we should remove it and trim the size of releases further. Anybody wants to read the docs offline can go get golang.org/x/tools/cmd/godoc first.

/cc @andybons @katiehockman @ianlancetaylor @FiloSottile @bcmills

@bradfitz
Copy link
Contributor Author

(Braindump about whether we make "lite" releases.... I'm leaning towards no)

We currently ship 15 downloads with releases at https://golang.org/dl/ ....

  • 1 source tarball
  • 3 installers (Mac amd64, Windows 32-bit, Windows 64-bit)
  • 11 binary archives (zip or tar.gz)

The binary archives take about 112MB compressed, 329MB extracted (linux-amd64 sizes). People sometimes complain that these are too large, and they've been growing over time. (See meta bug #27151)

One thing we could do is ship "lite" versions without:

  • any test data (saves 31MB extracted)
  • any pre-built *.a files (saves 89MB extracted)
  • docs & gopher images (4.5MB)
  • godoc (16MB)

Maybe also:

  • the trace viewer (~15 MB)
  • pprof (~13 MB)
  • objdump (~ 4.4 MB)
  • the "fix" tool (3.2MB)
  • api/* version history (6.5M)
  • zoneinfo
  • etc

It looks like we could halve the sizes of the downloads. (compressed & on-disk)

The question is whether we want to.

If we had "lite" versions, who is their intended audience?

If it's CI users doing builds in loops rather than humans, then one might counter argue that they'd benefit from pre-built *.a files (for faster builds) and the bandwidth isn't a concern for a cloud-connected CI system.

If it's for people building Docker images, one could counter argue that multi-stage Dockerfiles solve the problem: you should build your application in some heavy stage, then copy your final binary to a small image.

@andybons
Copy link
Member

andybons commented Oct 24, 2018

Go for it. I have no issue as long as there's a path to having offline docs (which one can get through go doc).

gopherbot pushed a commit to golang/build that referenced this issue Oct 24, 2018
…leases

The tour binary was renamed from gotour to tour in CL 141857 as part
of golang/go#28163, but cmd/release wasn't updated to know about the new
name. This fixes that.

Additionally, don't push tour source to the buildlet if the releaselet
isn't going to include it anyway.

Tested that running it with a Go 1.11 release includes the tour:

   $ release -rev release-branch.go1.11 -tools release-branch.go1.11 \
             -target=linux-amd64 -skip_tests -version=go1.11.99 -watch

And with Go 1.12 does not:

   $ release -rev master -tools master \
             -target=linux-amd64 -skip_tests -version=go1.12.99 -watch

Updates golang/go#28163
Updates golang/go#27151

Change-Id: I246fc274c9dea81a24d27322bcbbe5a59e682a5e
Reviewed-on: https://go-review.googlesource.com/c/143537
Reviewed-by: Katie Hockman <katie@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/144281 mentions this issue: cmd/release: don't ship race detector *.syso for other platforms

gopherbot pushed a commit to golang/build that referenced this issue Oct 24, 2018
Saves about 800k compressed, 2.2MB on disk.

Updates golang/go#27151

Change-Id: Ie78e02c3956b81c18643c80620f97612d55fdd23
Reviewed-on: https://go-review.googlesource.com/c/144281
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@darkfeline
Copy link
Contributor

Per #29713, does it make sense to omit the test/ subdir for Go distributions? As far as I can tell, it's not needed for, e.g,, a Linux distribution packaging Go.

@bradfitz
Copy link
Contributor Author

@darkfeline, if there are some users who don't want to be able to run tests, yes. See discussion above about "lite" builds and who their audience might be.

@Foxboron

This comment has been minimized.

@bradfitz

This comment has been minimized.

@dmitshur
Copy link
Contributor

dmitshur commented Jan 31, 2019

Now that the tour is removed from cmd/release (for Go 1.12), and CLI support is removed from godoc so it's only a webserver, is there any value in shipping the godoc webserver with releases?

I think we should remove it and trim the size of releases further. Anybody wants to read the docs offline can go get golang.org/x/tools/cmd/godoc first.

This sounds reasonable to me. There's a minor anticipated benefit of doing this. I expect that godoc will gain module support before the next major release, and so this would mean when people do go get -u golang.org/x/tools/cmd/godoc to install it, there's less of a chance of the older godoc in GOROOT/bin interfering.

@bradfitz This hasn't happened by now. Is it too late for 1.12 now?

@bradfitz
Copy link
Contributor Author

@dmitshur, yeah, too late. It would require doc/go1.12.html changes, etc. But I filed #30029 for that for Go 1.13.

@Foxboron

This comment has been minimized.

@andybons andybons modified the milestones: Go1.12, Go1.13 Feb 12, 2019
@bradfitz
Copy link
Contributor Author

Update to first comment:

Go 1.12.4 linux-amd64: 122 MB
Go 1.12.4 source: 21 MB

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/174322 mentions this issue: cmd/release: don't include godoc in releases in Go 1.13+

gopherbot pushed a commit to golang/build that referenced this issue Apr 29, 2019
Tested that it's still present in Go 1.12:

$ release -version=go1.12.999 -target=linux-amd64 -watch -skip_tests -rev=release-branch.go1.12 -tools=release-branch.go1.12 -net=release-branch.go1.12
...
$ ls -lh go1.12.999.linux-amd64.tar.gz
-rw-r--r-- 1 bradfitz bradfitz 122M Apr 29 19:28 go1.12.999.linux-amd64.tar.gz
$ tar -ztf go1.12.999.linux-amd64.tar.gz | grep godoc
go/bin/godoc
go/src/cmd/vendor/github.com/google/pprof/profile/testdata/go.godoc.thread
go/src/cmd/vendor/github.com/google/pprof/profile/testdata/go.godoc.thread.string

But not with Go 1.13:

$ release -version=go1.13beta0 -target=linux-amd64 -watch -skip_tests -rev=8c1f78524e421ac01e35e8805dd7a45bf98c2a79
...
$ ls -lh go1.13beta0.linux-amd64.tar.gz
-rw-r--r-- 1 bradfitz bradfitz 115M Apr 29 19:00 go1.13beta0.linux-amd64.tar.gz
$ tar -ztf go1.13beta0.linux-amd64.tar.gz | grep godoc
$

Fixes golang/go#30029
Updates golang/go#27151

Change-Id: I9ac5c1b2bc76f184bf05fdcac86bb2e37b57c77c
Reviewed-on: https://go-review.googlesource.com/c/build/+/174322
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@chaosmatrix
Copy link

chaosmatrix commented Sep 16, 2019

Why not support more format downloads, for example xz.
It has small size, but it would take more time to compress.

tar.bz created by linux command tar -jcf
tar.xz created by linux command tar -Jcf
tar.gz created by linux command tar -zcf

-rw-rw-r--  1 root root 107M Sep 16 21:26 go1.13.linux-amd64.tar.bz
-rw-rw-r--  1 root root 115M Sep 16 21:13 go1.13.linux-amd64.tar.gz
-rw-rw-r--  1 root root  84M Sep 16 21:18 go1.13.linux-amd64.tar.xz

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@dmitshur dmitshur changed the title x/build/cmd/release: reduce Go download sizes x/build/cmd/relui: reduce Go download sizes May 15, 2023
@heschi
Copy link
Contributor

heschi commented Aug 8, 2023

As of 1.21, releases are much smaller, in the 60-70MiB range. I think that's good enough to close this.

@heschi heschi closed this as completed Aug 8, 2023
@dmitshur dmitshur modified the milestones: Backlog, Go1.21 Aug 8, 2023
@golang golang locked and limited conversation to collaborators Aug 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
binary-size Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
Archived in project
Development

No branches or pull requests