From b60f6e0323c7d7b9cf6780206336dc8011a83d5b Mon Sep 17 00:00:00 2001 From: Qi Luo Date: Mon, 6 Jun 2016 14:10:32 -0700 Subject: [PATCH 1/2] Config apt inside docker images to save disk space: auto clean, gz, no trans --- ThirdPartyLicenses.txt | 194 +++++++++++++++++++++++ build_debian.sh | 2 +- files/apt/apt.conf.d/docker-clean | 19 +++ files/apt/apt.conf.d/docker-gzip-indexes | 11 ++ files/apt/apt.conf.d/docker-no-languages | 5 + 5 files changed, 230 insertions(+), 1 deletion(-) create mode 100644 files/apt/apt.conf.d/docker-clean create mode 100644 files/apt/apt.conf.d/docker-gzip-indexes create mode 100644 files/apt/apt.conf.d/docker-no-languages diff --git a/ThirdPartyLicenses.txt b/ThirdPartyLicenses.txt index 99bc267e11f9..09d2097b48ee 100644 --- a/ThirdPartyLicenses.txt +++ b/ThirdPartyLicenses.txt @@ -976,3 +976,197 @@ Microsoft is offering you a license to use the following components, to the exte * Public License instead of this License. But first, please read * . */ + +4. docker-clean, docker-gzip-indexes, docker-no-languages imported from docker v1.11.1 +/* + * Apache License + * Version 2.0, January 2004 + * https://www.apache.org/licenses/ + * + * TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + * + * 1. Definitions. + * + * "License" shall mean the terms and conditions for use, reproduction, + * and distribution as defined by Sections 1 through 9 of this document. + * + * "Licensor" shall mean the copyright owner or entity authorized by + * the copyright owner that is granting the License. + * + * "Legal Entity" shall mean the union of the acting entity and all + * other entities that control, are controlled by, or are under common + * control with that entity. For the purposes of this definition, + * "control" means (i) the power, direct or indirect, to cause the + * direction or management of such entity, whether by contract or + * otherwise, or (ii) ownership of fifty percent (50%) or more of the + * outstanding shares, or (iii) beneficial ownership of such entity. + * + * "You" (or "Your") shall mean an individual or Legal Entity + * exercising permissions granted by this License. + * + * "Source" form shall mean the preferred form for making modifications, + * including but not limited to software source code, documentation + * source, and configuration files. + * + * "Object" form shall mean any form resulting from mechanical + * transformation or translation of a Source form, including but + * not limited to compiled object code, generated documentation, + * and conversions to other media types. + * + * "Work" shall mean the work of authorship, whether in Source or + * Object form, made available under the License, as indicated by a + * copyright notice that is included in or attached to the work + * (an example is provided in the Appendix below). + * + * "Derivative Works" shall mean any work, whether in Source or Object + * form, that is based on (or derived from) the Work and for which the + * editorial revisions, annotations, elaborations, or other modifications + * represent, as a whole, an original work of authorship. For the purposes + * of this License, Derivative Works shall not include works that remain + * separable from, or merely link (or bind by name) to the interfaces of, + * the Work and Derivative Works thereof. + * + * "Contribution" shall mean any work of authorship, including + * the original version of the Work and any modifications or additions + * to that Work or Derivative Works thereof, that is intentionally + * submitted to Licensor for inclusion in the Work by the copyright owner + * or by an individual or Legal Entity authorized to submit on behalf of + * the copyright owner. For the purposes of this definition, "submitted" + * means any form of electronic, verbal, or written communication sent + * to the Licensor or its representatives, including but not limited to + * communication on electronic mailing lists, source code control systems, + * and issue tracking systems that are managed by, or on behalf of, the + * Licensor for the purpose of discussing and improving the Work, but + * excluding communication that is conspicuously marked or otherwise + * designated in writing by the copyright owner as "Not a Contribution." + * + * "Contributor" shall mean Licensor and any individual or Legal Entity + * on behalf of whom a Contribution has been received by Licensor and + * subsequently incorporated within the Work. + * + * 2. Grant of Copyright License. Subject to the terms and conditions of + * this License, each Contributor hereby grants to You a perpetual, + * worldwide, non-exclusive, no-charge, royalty-free, irrevocable + * copyright license to reproduce, prepare Derivative Works of, + * publicly display, publicly perform, sublicense, and distribute the + * Work and such Derivative Works in Source or Object form. + * + * 3. Grant of Patent License. Subject to the terms and conditions of + * this License, each Contributor hereby grants to You a perpetual, + * worldwide, non-exclusive, no-charge, royalty-free, irrevocable + * (except as stated in this section) patent license to make, have made, + * use, offer to sell, sell, import, and otherwise transfer the Work, + * where such license applies only to those patent claims licensable + * by such Contributor that are necessarily infringed by their + * Contribution(s) alone or by combination of their Contribution(s) + * with the Work to which such Contribution(s) was submitted. If You + * institute patent litigation against any entity (including a + * cross-claim or counterclaim in a lawsuit) alleging that the Work + * or a Contribution incorporated within the Work constitutes direct + * or contributory patent infringement, then any patent licenses + * granted to You under this License for that Work shall terminate + * as of the date such litigation is filed. + * + * 4. Redistribution. You may reproduce and distribute copies of the + * Work or Derivative Works thereof in any medium, with or without + * modifications, and in Source or Object form, provided that You + * meet the following conditions: + * + * (a) You must give any other recipients of the Work or + * Derivative Works a copy of this License; and + * + * (b) You must cause any modified files to carry prominent notices + * stating that You changed the files; and + * + * (c) You must retain, in the Source form of any Derivative Works + * that You distribute, all copyright, patent, trademark, and + * attribution notices from the Source form of the Work, + * excluding those notices that do not pertain to any part of + * the Derivative Works; and + * + * (d) If the Work includes a "NOTICE" text file as part of its + * distribution, then any Derivative Works that You distribute must + * include a readable copy of the attribution notices contained + * within such NOTICE file, excluding those notices that do not + * pertain to any part of the Derivative Works, in at least one + * of the following places: within a NOTICE text file distributed + * as part of the Derivative Works; within the Source form or + * documentation, if provided along with the Derivative Works; or, + * within a display generated by the Derivative Works, if and + * wherever such third-party notices normally appear. The contents + * of the NOTICE file are for informational purposes only and + * do not modify the License. You may add Your own attribution + * notices within Derivative Works that You distribute, alongside + * or as an addendum to the NOTICE text from the Work, provided + * that such additional attribution notices cannot be construed + * as modifying the License. + * + * You may add Your own copyright statement to Your modifications and + * may provide additional or different license terms and conditions + * for use, reproduction, or distribution of Your modifications, or + * for any such Derivative Works as a whole, provided Your use, + * reproduction, and distribution of the Work otherwise complies with + * the conditions stated in this License. + * + * 5. Submission of Contributions. Unless You explicitly state otherwise, + * any Contribution intentionally submitted for inclusion in the Work + * by You to the Licensor shall be under the terms and conditions of + * this License, without any additional terms or conditions. + * Notwithstanding the above, nothing herein shall supersede or modify + * the terms of any separate license agreement you may have executed + * with Licensor regarding such Contributions. + * + * 6. Trademarks. This License does not grant permission to use the trade + * names, trademarks, service marks, or product names of the Licensor, + * except as required for reasonable and customary use in describing the + * origin of the Work and reproducing the content of the NOTICE file. + * + * 7. Disclaimer of Warranty. Unless required by applicable law or + * agreed to in writing, Licensor provides the Work (and each + * Contributor provides its Contributions) on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied, including, without limitation, any warranties or conditions + * of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + * PARTICULAR PURPOSE. You are solely responsible for determining the + * appropriateness of using or redistributing the Work and assume any + * risks associated with Your exercise of permissions under this License. + * + * 8. Limitation of Liability. In no event and under no legal theory, + * whether in tort (including negligence), contract, or otherwise, + * unless required by applicable law (such as deliberate and grossly + * negligent acts) or agreed to in writing, shall any Contributor be + * liable to You for damages, including any direct, indirect, special, + * incidental, or consequential damages of any character arising as a + * result of this License or out of the use or inability to use the + * Work (including but not limited to damages for loss of goodwill, + * work stoppage, computer failure or malfunction, or any and all + * other commercial damages or losses), even if such Contributor + * has been advised of the possibility of such damages. + * + * 9. Accepting Warranty or Additional Liability. While redistributing + * the Work or Derivative Works thereof, You may choose to offer, + * and charge a fee for, acceptance of support, warranty, indemnity, + * or other liability obligations and/or rights consistent with this + * License. However, in accepting such obligations, You may act only + * on Your own behalf and on Your sole responsibility, not on behalf + * of any other Contributor, and only if You agree to indemnify, + * defend, and hold each Contributor harmless for any liability + * incurred by, or claims asserted against, such Contributor by reason + * of your accepting any such warranty or additional liability. + * + * END OF TERMS AND CONDITIONS + * + * Copyright 2013-2016 Docker, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ diff --git a/build_debian.sh b/build_debian.sh index 367c5814fefb..dead26654e27 100755 --- a/build_debian.sh +++ b/build_debian.sh @@ -91,7 +91,7 @@ sudo LANG=C chroot $FILESYSTEM_ROOT mount sysfs /sys -t sysfs ## Pointing apt to public apt mirrors and getting latest packages, needed for latest security updates sudo cp files/apt/sources.list $FILESYSTEM_ROOT/etc/apt/ -sudo cp files/apt/apt.conf.d/81norecommends $FILESYSTEM_ROOT/etc/apt/apt.conf.d/ +sudo cp files/apt/apt.conf.d/{81norecommends,docker-{clean,gzip-indexes,no-languages}} $FILESYSTEM_ROOT/etc/apt/apt.conf.d/ sudo LANG=C chroot $FILESYSTEM_ROOT bash -c 'apt-mark auto `apt-mark showmanual`' ## Note: set lang to prevent locale warnings in your chroot diff --git a/files/apt/apt.conf.d/docker-clean b/files/apt/apt.conf.d/docker-clean new file mode 100644 index 000000000000..886c6d829a3b --- /dev/null +++ b/files/apt/apt.conf.d/docker-clean @@ -0,0 +1,19 @@ +# Since for most Docker users, package installs happen in "docker build" steps, +# they essentially become individual layers due to the way Docker handles +# layering, especially using CoW filesystems. What this means for us is that +# the caches that APT keeps end up just wasting space in those layers, making +# our layers unnecessarily large (especially since we'll normally never use +# these caches again and will instead just "docker build" again and make a brand +# new image). + +# Ideally, these would just be invoking "apt-get clean", but in our testing, +# that ended up being cyclic and we got stuck on APT's lock, so we get this fun +# creation that's essentially just "apt-get clean". +DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; }; +APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; }; + +Dir::Cache::pkgcache ""; +Dir::Cache::srcpkgcache ""; + +# Note that we do realize this isn't the ideal way to do this, and are always +# open to better suggestions (https://github.com/docker/docker/issues). diff --git a/files/apt/apt.conf.d/docker-gzip-indexes b/files/apt/apt.conf.d/docker-gzip-indexes new file mode 100644 index 000000000000..6d29ac4474de --- /dev/null +++ b/files/apt/apt.conf.d/docker-gzip-indexes @@ -0,0 +1,11 @@ +# Since Docker users using "RUN apt-get update && apt-get install -y ..." in +# their Dockerfiles don't go delete the lists files afterwards, we want them to +# be as small as possible on-disk, so we explicitly request "gz" versions and +# tell Apt to keep them gzipped on-disk. + +# For comparison, an "apt-get update" layer without this on a pristine +# "debian:wheezy" base image was "29.88 MB", where with this it was only +# "8.273 MB". + +Acquire::GzipIndexes "true"; +Acquire::CompressionTypes::Order:: "gz"; diff --git a/files/apt/apt.conf.d/docker-no-languages b/files/apt/apt.conf.d/docker-no-languages new file mode 100644 index 000000000000..964d83d6c5b9 --- /dev/null +++ b/files/apt/apt.conf.d/docker-no-languages @@ -0,0 +1,5 @@ +# In Docker, we don't often need the "Translations" files, so we're just wasting +# time and space by downloading them, and this inhibits that. For users that do +# need them, it's a simple matter to delete this file and "apt-get update". :) + +Acquire::Languages "none"; From a3734479860e26767602a69f8fba548e7a1c38f6 Mon Sep 17 00:00:00 2001 From: Qi Luo Date: Mon, 6 Jun 2016 16:01:04 -0700 Subject: [PATCH 2/2] Rename to apt --- ThirdPartyLicenses.txt | 2 +- build_debian.sh | 2 +- files/apt/apt.conf.d/{docker-clean => apt-clean} | 0 files/apt/apt.conf.d/{docker-gzip-indexes => apt-gzip-indexes} | 0 files/apt/apt.conf.d/{docker-no-languages => apt-no-languages} | 0 5 files changed, 2 insertions(+), 2 deletions(-) rename files/apt/apt.conf.d/{docker-clean => apt-clean} (100%) rename files/apt/apt.conf.d/{docker-gzip-indexes => apt-gzip-indexes} (100%) rename files/apt/apt.conf.d/{docker-no-languages => apt-no-languages} (100%) diff --git a/ThirdPartyLicenses.txt b/ThirdPartyLicenses.txt index 09d2097b48ee..376506337e79 100644 --- a/ThirdPartyLicenses.txt +++ b/ThirdPartyLicenses.txt @@ -977,7 +977,7 @@ Microsoft is offering you a license to use the following components, to the exte * . */ -4. docker-clean, docker-gzip-indexes, docker-no-languages imported from docker v1.11.1 +4. apt-clean, apt-gzip-indexes, apt-no-languages imported from docker v1.11.1 /* * Apache License * Version 2.0, January 2004 diff --git a/build_debian.sh b/build_debian.sh index dead26654e27..9252646dff4b 100755 --- a/build_debian.sh +++ b/build_debian.sh @@ -91,7 +91,7 @@ sudo LANG=C chroot $FILESYSTEM_ROOT mount sysfs /sys -t sysfs ## Pointing apt to public apt mirrors and getting latest packages, needed for latest security updates sudo cp files/apt/sources.list $FILESYSTEM_ROOT/etc/apt/ -sudo cp files/apt/apt.conf.d/{81norecommends,docker-{clean,gzip-indexes,no-languages}} $FILESYSTEM_ROOT/etc/apt/apt.conf.d/ +sudo cp files/apt/apt.conf.d/{81norecommends,apt-{clean,gzip-indexes,no-languages}} $FILESYSTEM_ROOT/etc/apt/apt.conf.d/ sudo LANG=C chroot $FILESYSTEM_ROOT bash -c 'apt-mark auto `apt-mark showmanual`' ## Note: set lang to prevent locale warnings in your chroot diff --git a/files/apt/apt.conf.d/docker-clean b/files/apt/apt.conf.d/apt-clean similarity index 100% rename from files/apt/apt.conf.d/docker-clean rename to files/apt/apt.conf.d/apt-clean diff --git a/files/apt/apt.conf.d/docker-gzip-indexes b/files/apt/apt.conf.d/apt-gzip-indexes similarity index 100% rename from files/apt/apt.conf.d/docker-gzip-indexes rename to files/apt/apt.conf.d/apt-gzip-indexes diff --git a/files/apt/apt.conf.d/docker-no-languages b/files/apt/apt.conf.d/apt-no-languages similarity index 100% rename from files/apt/apt.conf.d/docker-no-languages rename to files/apt/apt.conf.d/apt-no-languages