Update ELK Images #16520

elasticmachine · 2024-04-02T06:58:15Z

Update ELK Images

tianon · 2024-04-02T17:10:34Z

The "diff" CI job is failing because we don't support variadic FROM or COPY -- in this case, I'd actually suggest building the binary itself as part of the Dockerfile instead of committing it to Git.

github-actions · 2024-04-04T08:21:33Z

Diff for c4b698c:

diff --git a/_bashbrew-cat b/_bashbrew-cat
index e40534f..5e89fa8 100644
--- a/_bashbrew-cat
+++ b/_bashbrew-cat
@@ -9,10 +9,10 @@ Architectures: amd64, arm64v8
 GitFetch: refs/heads/7.17
 GitCommit: e6eabbb45f3729cefacf138e418aaa12a541741b
 
-Tags: 8.13.0
+Tags: 8.13.1
 Architectures: amd64, arm64v8
 GitFetch: refs/heads/8.13
-GitCommit: 4269a02cf8a56844db30e5e74b23087840ee6fcb
+GitCommit: 4fdbd17dca428a5c64601f380656c124b116fe6f
 
 
 # kibana
@@ -26,10 +26,10 @@ Architectures: amd64, arm64v8
 GitFetch: refs/heads/7.17
 GitCommit: e6eabbb45f3729cefacf138e418aaa12a541741b
 
-Tags: 8.13.0
+Tags: 8.13.1
 Architectures: amd64, arm64v8
 GitFetch: refs/heads/8.13
-GitCommit: 4269a02cf8a56844db30e5e74b23087840ee6fcb
+GitCommit: 4fdbd17dca428a5c64601f380656c124b116fe6f
 
 
 # logstash
@@ -43,7 +43,7 @@ Architectures: amd64, arm64v8
 GitFetch: refs/heads/7.17
 GitCommit: e6eabbb45f3729cefacf138e418aaa12a541741b
 
-Tags: 8.13.0
+Tags: 8.13.1
 Architectures: amd64, arm64v8
 GitFetch: refs/heads/8.13
-GitCommit: 4269a02cf8a56844db30e5e74b23087840ee6fcb
+GitCommit: 4fdbd17dca428a5c64601f380656c124b116fe6f
diff --git a/_bashbrew-list b/_bashbrew-list
index ea60da5..62afc39 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -1,6 +1,6 @@
 elasticsearch:7.17.19
-elasticsearch:8.13.0
+elasticsearch:8.13.1
 kibana:7.17.19
-kibana:8.13.0
+kibana:8.13.1
 logstash:7.17.19
-logstash:8.13.0
+logstash:8.13.1
diff --git a/elasticsearch_8.13.0/Dockerfile b/elasticsearch_8.13.1/Dockerfile
similarity index 94%
rename from elasticsearch_8.13.0/Dockerfile
rename to elasticsearch_8.13.1/Dockerfile
index 236a595..099da21 100644
--- a/elasticsearch_8.13.0/Dockerfile
+++ b/elasticsearch_8.13.1/Dockerfile
@@ -43,7 +43,7 @@ RUN set -eux ; \
 RUN mkdir /usr/share/elasticsearch
 WORKDIR /usr/share/elasticsearch
 
-RUN curl --retry 10 -S -L --output /tmp/elasticsearch.tar.gz https://artifacts-no-kpi.elastic.co/downloads/elasticsearch/elasticsearch-8.13.0-linux-$(arch).tar.gz
+RUN curl --retry 10 -S -L --output /tmp/elasticsearch.tar.gz https://artifacts-no-kpi.elastic.co/downloads/elasticsearch/elasticsearch-8.13.1-linux-$(arch).tar.gz
 
 RUN tar -zxf /tmp/elasticsearch.tar.gz --strip-components=1
 
@@ -135,25 +135,25 @@ RUN /etc/ca-certificates/update.d/docker-openjdk
 
 EXPOSE 9200 9300
 
-LABEL org.label-schema.build-date="2024-03-22T03:35:46.757803203Z" \
+LABEL org.label-schema.build-date="2024-03-29T10:05:29.787251984Z" \
   org.label-schema.license="Elastic-License-2.0" \
   org.label-schema.name="Elasticsearch" \
   org.label-schema.schema-version="1.0" \
   org.label-schema.url="https://www.elastic.co/products/elasticsearch" \
   org.label-schema.usage="https://www.elastic.co/guide/en/elasticsearch/reference/index.html" \
-  org.label-schema.vcs-ref="09df99393193b2c53d92899662a8b8b3c55b45cd" \
+  org.label-schema.vcs-ref="9287f29bba5e270bd51d557b8daccb7d118ba247" \
   org.label-schema.vcs-url="https://github.com/elastic/elasticsearch" \
   org.label-schema.vendor="Elastic" \
-  org.label-schema.version="8.13.0" \
-  org.opencontainers.image.created="2024-03-22T03:35:46.757803203Z" \
+  org.label-schema.version="8.13.1" \
+  org.opencontainers.image.created="2024-03-29T10:05:29.787251984Z" \
   org.opencontainers.image.documentation="https://www.elastic.co/guide/en/elasticsearch/reference/index.html" \
   org.opencontainers.image.licenses="Elastic-License-2.0" \
-  org.opencontainers.image.revision="09df99393193b2c53d92899662a8b8b3c55b45cd" \
+  org.opencontainers.image.revision="9287f29bba5e270bd51d557b8daccb7d118ba247" \
   org.opencontainers.image.source="https://github.com/elastic/elasticsearch" \
   org.opencontainers.image.title="Elasticsearch" \
   org.opencontainers.image.url="https://www.elastic.co/products/elasticsearch" \
   org.opencontainers.image.vendor="Elastic" \
-  org.opencontainers.image.version="8.13.0"
+  org.opencontainers.image.version="8.13.1"
 
 # Our actual entrypoint is `tini`, a minimal but functional init program. It
 # calls the entrypoint we provide, while correctly forwarding signals.
diff --git a/elasticsearch_8.13.0/bin/docker-entrypoint.sh b/elasticsearch_8.13.1/bin/docker-entrypoint.sh
similarity index 100%
rename from elasticsearch_8.13.0/bin/docker-entrypoint.sh
rename to elasticsearch_8.13.1/bin/docker-entrypoint.sh
diff --git a/elasticsearch_8.13.0/bin/docker-openjdk b/elasticsearch_8.13.1/bin/docker-openjdk
similarity index 100%
rename from elasticsearch_8.13.0/bin/docker-openjdk
rename to elasticsearch_8.13.1/bin/docker-openjdk
diff --git a/elasticsearch_8.13.0/config/elasticsearch.yml b/elasticsearch_8.13.1/config/elasticsearch.yml
similarity index 100%
rename from elasticsearch_8.13.0/config/elasticsearch.yml
rename to elasticsearch_8.13.1/config/elasticsearch.yml
diff --git a/elasticsearch_8.13.0/config/log4j2.properties b/elasticsearch_8.13.1/config/log4j2.properties
similarity index 100%
rename from elasticsearch_8.13.0/config/log4j2.properties
rename to elasticsearch_8.13.1/config/log4j2.properties
diff --git a/kibana_8.13.0/Dockerfile b/kibana_8.13.1/Dockerfile
similarity index 91%
rename from kibana_8.13.0/Dockerfile
rename to kibana_8.13.1/Dockerfile
index 46da7c6..b1772d0 100644
--- a/kibana_8.13.0/Dockerfile
+++ b/kibana_8.13.1/Dockerfile
@@ -16,7 +16,7 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y curl
 RUN cd /tmp && \
   curl --retry 8 -s -L \
     --output kibana.tar.gz \
-     https://artifacts.elastic.co/downloads/kibana/kibana-8.13.0-linux-$(arch).tar.gz && \
+     https://artifacts.elastic.co/downloads/kibana/kibana-8.13.1-linux-$(arch).tar.gz && \
   cd -
 
 
@@ -104,25 +104,25 @@ RUN groupadd --gid 1000 kibana && \
       --home-dir /usr/share/kibana --no-create-home \
       kibana
 
-LABEL org.label-schema.build-date="2024-03-22T04:07:01.866Z" \
+LABEL org.label-schema.build-date="2024-03-29T12:06:31.613Z" \
   org.label-schema.license="Elastic License" \
   org.label-schema.name="Kibana" \
   org.label-schema.schema-version="1.0" \
   org.label-schema.url="https://www.elastic.co/products/kibana" \
   org.label-schema.usage="https://www.elastic.co/guide/en/kibana/reference/index.html" \
-  org.label-schema.vcs-ref="2e3a5cd43e835baa1d596b1aa54735992259ecb9" \
+  org.label-schema.vcs-ref="091f486ab05863258cf2f3fa18ea0c59097dee80" \
   org.label-schema.vcs-url="https://github.com/elastic/kibana" \
   org.label-schema.vendor="Elastic" \
-  org.label-schema.version="8.13.0" \
-  org.opencontainers.image.created="2024-03-22T04:07:01.866Z" \
+  org.label-schema.version="8.13.1" \
+  org.opencontainers.image.created="2024-03-29T12:06:31.613Z" \
   org.opencontainers.image.documentation="https://www.elastic.co/guide/en/kibana/reference/index.html" \
   org.opencontainers.image.licenses="Elastic License" \
-  org.opencontainers.image.revision="2e3a5cd43e835baa1d596b1aa54735992259ecb9" \
+  org.opencontainers.image.revision="091f486ab05863258cf2f3fa18ea0c59097dee80" \
   org.opencontainers.image.source="https://github.com/elastic/kibana" \
   org.opencontainers.image.title="Kibana" \
   org.opencontainers.image.url="https://www.elastic.co/products/kibana" \
   org.opencontainers.image.vendor="Elastic" \
-  org.opencontainers.image.version="8.13.0"
+  org.opencontainers.image.version="8.13.1"
 
 
 ENTRYPOINT ["/bin/tini", "--"]
diff --git a/kibana_8.13.0/bin/kibana-docker b/kibana_8.13.1/bin/kibana-docker
similarity index 100%
rename from kibana_8.13.0/bin/kibana-docker
rename to kibana_8.13.1/bin/kibana-docker
diff --git a/kibana_8.13.0/config/kibana.yml b/kibana_8.13.1/config/kibana.yml
similarity index 100%
rename from kibana_8.13.0/config/kibana.yml
rename to kibana_8.13.1/config/kibana.yml
diff --git a/logstash_8.13.0/Dockerfile b/logstash_8.13.1/Dockerfile
similarity index 51%
rename from logstash_8.13.0/Dockerfile
rename to logstash_8.13.1/Dockerfile
index e04f34c..e2f7e62 100644
--- a/logstash_8.13.0/Dockerfile
+++ b/logstash_8.13.1/Dockerfile
@@ -2,33 +4,30 @@
 FROM ubuntu:20.04
 
 RUN for iter in {1..10}; do \
-export DEBIAN_FRONTEND=noninteractive && \
+  export DEBIAN_FRONTEND=noninteractive && \
 apt-get update -y && \
 apt-get upgrade -y && \
-apt-get install -y procps findutils tar gzip curl && \
-apt-get install -y locales && \
+apt-get install -y procps findutils tar gzip && \
+  apt-get install -y locales && \
+  apt-get install -y curl && \
 apt-get clean all && \
-locale-gen 'en_US.UTF-8' && \
-apt-get clean metadata && \
+  locale-gen 'en_US.UTF-8' && \
+  apt-get clean metadata && \
 exit_code=0 && break || exit_code=$? && \
 echo "packaging error: retry $iter in 10s" && \
 apt-get clean all && \
-apt-get clean metadata && \
+  apt-get clean metadata && \
 sleep 10; done; \
 (exit $exit_code)
 
 # Provide a non-root user to run the process.
 RUN groupadd --gid 1000 logstash && \
-    adduser --uid 1000 --gid 1000 \
-    --home /usr/share/logstash --no-create-home \
-    logstash
-
+  adduser --uid 1000 --gid 1000 --home /usr/share/logstash --no-create-home logstash
 
 # Add Logstash itself.
-RUN \
- curl -Lo - https://artifacts.elastic.co/downloads/logstash/logstash-8.13.0-linux-$(arch).tar.gz | \
+RUN curl -Lo - https://artifacts.elastic.co/downloads/logstash/logstash-8.13.1-linux-$(arch).tar.gz | \
   tar zxf - -C /usr/share && \
-    mv /usr/share/logstash-8.13.0 /usr/share/logstash && \
+  mv /usr/share/logstash-8.13.1 /usr/share/logstash && \
   chown --recursive logstash:logstash /usr/share/logstash/ && \
   chown -R logstash:root /usr/share/logstash && \
   chmod -R g=u /usr/share/logstash && \
@@ -46,14 +44,28 @@ ENV PATH=/usr/share/logstash/bin:$PATH
 # Provide a minimal configuration, so that simple invocations will provide
 # a good experience.
 COPY config/pipelines.yml config/pipelines.yml
-COPY config/logstash-full.yml config/logstash.yml
+  COPY config/logstash-full.yml config/logstash.yml
 COPY config/log4j2.properties config/
 COPY config/log4j2.file.properties config/
 COPY pipeline/default.conf pipeline/logstash.conf
+COPY env2yaml/env2yaml-amd64 env2yaml/
+COPY env2yaml/env2yaml-arm64 env2yaml/
 RUN chown --recursive logstash:root config/ pipeline/
 # Ensure Logstash gets the correct locale by default.
 ENV LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
-COPY env2yaml/env2yaml /usr/local/bin/
+# Copy over the appropriate env2yaml artifact
+RUN env2yamlarch="$(arch)"; \
+  case "${env2yamlarch}" in \
+    'x86_64') \
+      env2yamlarch=amd64; \
+      ;; \
+    'aarch64') \
+      env2yamlarch=arm64; \
+      ;; \
+    *) echo >&2 "error: unsupported architecture '${env2yamlarch}'"; exit 1 ;; \
+  esac; \
+  cp env2yaml/env2yaml-${env2yamlarch} /usr/local/bin/env2yaml; \
+  rm -rf env2yaml
 # Place the startup wrapper script.
 COPY bin/docker-entrypoint /usr/local/bin/
 
@@ -68,15 +81,14 @@ LABEL  org.label-schema.schema-version="1.0" \
   org.opencontainers.image.vendor="Elastic" \
   org.label-schema.name="logstash" \
   org.opencontainers.image.title="logstash" \
-  org.label-schema.version="8.13.0" \
-  org.opencontainers.image.version="8.13.0" \
+  org.label-schema.version="8.13.1" \
+  org.opencontainers.image.version="8.13.1" \
   org.label-schema.url="https://www.elastic.co/products/logstash" \
   org.label-schema.vcs-url="https://github.com/elastic/logstash" \
   org.label-schema.license="Elastic License" \
   org.opencontainers.image.licenses="Elastic License" \
   org.opencontainers.image.description="Logstash is a free and open server-side data processing pipeline that ingests data from a multitude of sources, transforms it, and then sends it to your favorite 'stash.'" \
-  org.label-schema.build-date=2024-03-21T09:15:34+00:00 \
-  org.opencontainers.image.created=2024-03-21T09:15:34+00:00
-
+  org.label-schema.build-date=2024-04-03T10:39:38-04:00 \
+  org.opencontainers.image.created=2024-04-03T10:39:38-04:00
 
 ENTRYPOINT ["/usr/local/bin/docker-entrypoint"]
diff --git a/logstash_8.13.0/bin/docker-entrypoint b/logstash_8.13.1/bin/docker-entrypoint
similarity index 100%
rename from logstash_8.13.0/bin/docker-entrypoint
rename to logstash_8.13.1/bin/docker-entrypoint
diff --git a/logstash_8.13.0/config/log4j2.file.properties b/logstash_8.13.1/config/log4j2.file.properties
similarity index 100%
rename from logstash_8.13.0/config/log4j2.file.properties
rename to logstash_8.13.1/config/log4j2.file.properties
diff --git a/logstash_8.13.0/config/log4j2.properties b/logstash_8.13.1/config/log4j2.properties
similarity index 100%
rename from logstash_8.13.0/config/log4j2.properties
rename to logstash_8.13.1/config/log4j2.properties
diff --git a/logstash_8.13.0/config/logstash-full.yml b/logstash_8.13.1/config/logstash-full.yml
similarity index 100%
rename from logstash_8.13.0/config/logstash-full.yml
rename to logstash_8.13.1/config/logstash-full.yml
diff --git a/logstash_8.13.0/config/pipelines.yml b/logstash_8.13.1/config/pipelines.yml
similarity index 100%
rename from logstash_8.13.0/config/pipelines.yml
rename to logstash_8.13.1/config/pipelines.yml
diff --git a/logstash_8.13.0/env2yaml/env2yaml b/logstash_8.13.1/env2yaml/env2yaml-amd64
similarity index 57%
copy from logstash_8.13.0/env2yaml/env2yaml
copy to logstash_8.13.1/env2yaml/env2yaml-amd64
index fc22245..2c4a86d 100755
Binary files a/logstash_8.13.0/env2yaml/env2yaml and b/logstash_8.13.1/env2yaml/env2yaml-amd64 differ
diff --git a/logstash_8.13.0/env2yaml/env2yaml b/logstash_8.13.1/env2yaml/env2yaml-arm64
similarity index 32%
rename from logstash_8.13.0/env2yaml/env2yaml
rename to logstash_8.13.1/env2yaml/env2yaml-arm64
index fc22245..f3b8306 100755
Binary files a/logstash_8.13.0/env2yaml/env2yaml and b/logstash_8.13.1/env2yaml/env2yaml-arm64 differ
diff --git a/logstash_8.13.0/pipeline/default.conf b/logstash_8.13.1/pipeline/default.conf
similarity index 100%
rename from logstash_8.13.0/pipeline/default.conf
rename to logstash_8.13.1/pipeline/default.conf

Relevant Maintainers:

elasticsearch: @mark-vieira
kibana: @watson
logstash: @jsvd

tianon · 2024-04-08T20:04:45Z

+# Copy over the appropriate env2yaml artifact
+RUN env2yamlarch="$(arch)"; \
+  case "${env2yamlarch}" in \
+    'x86_64') \
+      env2yamlarch=amd64; \
+      ;; \
+    'aarch64') \
+      env2yamlarch=arm64; \
+      ;; \
+    *) echo >&2 "error: unsupported architecture '${env2yamlarch}'"; exit 1 ;; \
+  esac; \
+  cp env2yaml/env2yaml-${env2yamlarch} /usr/local/bin/env2yaml; \
+  rm -rf env2yaml

😬 this is definitely not ideal (it means everyone gets both copies of the binary regardless since they exist in the layer history)

I would reiterate the suggestion to build this in the Dockerfile instead (checking a binary into Git seems unnecessary for this use case, but perhaps I'm missing something).

jsvd · 2024-04-08T22:16:59Z

@tianon absolutely agreed that this is not ideal. Logstash/Kibana/ES images are built using the tarball artifacts for the release. For Logstash we need to add the extra binary to the image, that is not used in the tarball.

We have a plan to either move the env2yaml binary to its own repository with its own lifecycle and releases (which can then easily be fetch during image creation), or to do away with the binary altogether. But these will take more time, hence this fix for now.

tianon · 2024-04-08T22:22:11Z

Ok, I can live with that, thanks 😄

Can we at least swap from arch (which queries the kernel architecture) to something like dpkg --print-architecture, apk --print-arch, etc that actually query the userspace architecture instead?

navyau09 · 2024-04-30T18:14:59Z

This PR is created as a part of our automated process and looks like it is not created properly
Please close this and consider the manual PR - #16697

Thanks

Update ELK images for 8.13.1

39f53f3

elasticmachine requested a review from a team as a code owner April 2, 2024 06:58

github-actions bot added library/elasticsearch library/kibana library/logstash labels Apr 2, 2024

Empty-Commit

39aa382

Fix

c4b698c

tianon mentioned this pull request Apr 8, 2024

Update ELK Images #16554

Closed

jsvd mentioned this pull request Apr 9, 2024

Backport of env2yaml arm64 Docker build context fixes elastic/logstash#16063

Open

5 tasks

LaurentGoderre closed this Apr 30, 2024

LaurentGoderre mentioned this pull request Apr 30, 2024

[8.13.2] Update ELK images #16698

Closed

jmlrt deleted the update-official-images-8.13.1 branch December 30, 2024 09:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update ELK Images #16520

Update ELK Images #16520

elasticmachine commented Apr 2, 2024

tianon commented Apr 2, 2024

github-actions bot commented Apr 4, 2024

tianon commented Apr 8, 2024

jsvd commented Apr 8, 2024

tianon commented Apr 8, 2024

navyau09 commented Apr 30, 2024

Update ELK Images #16520

Update ELK Images #16520

Conversation

elasticmachine commented Apr 2, 2024

tianon commented Apr 2, 2024

github-actions bot commented Apr 4, 2024

tianon commented Apr 8, 2024

jsvd commented Apr 8, 2024

tianon commented Apr 8, 2024

navyau09 commented Apr 30, 2024