Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Enable wolfi Elastic Agent images in tests #2067

Merged
merged 22 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
8e9b15f
[CI] Enable by default wolfi images in tests
mrodm Sep 2, 2024
7c19075
Fix comment typo
mrodm Sep 2, 2024
1820056
Test with 8.16.0 SNAPSHOT tests
mrodm Sep 2, 2024
ba7e1f5
Update test policies
mrodm Sep 2, 2024
38a4880
Update oracle test package (parallel)
mrodm Sep 2, 2024
3292bc4
[test oracle] Exist provisioning script in case of failures
mrodm Sep 2, 2024
83a8623
[test apache] Fix policy test in data stream error
mrodm Sep 2, 2024
1912f7a
[test oracle custom agent] Fix Dockerfile for wolfi imges
mrodm Sep 2, 2024
56ddb0b
Remove exit commands from Dockerfile [oracle custom agent]
mrodm Sep 3, 2024
758dd17
Revert "[test apache] Fix policy test in data stream error"
mrodm Sep 3, 2024
76e292f
Revert "Update test policies"
mrodm Sep 3, 2024
4af6371
Revert "Test with 8.16.0 SNAPSHOT tests"
mrodm Sep 3, 2024
9fa5e7f
Reapply "Test with 8.16.0 SNAPSHOT tests"
mrodm Sep 3, 2024
3d4c892
Reapply "Update test policies"
mrodm Sep 3, 2024
725f572
Reapply "[test apache] Fix policy test in data stream error"
mrodm Sep 3, 2024
bd3eb61
Merge remote-tracking branch 'upstream/main' into enable-wolfi-images…
mrodm Sep 3, 2024
8b74298
Fix tests if wolfi is enabled
mrodm Sep 3, 2024
dae2f8f
Revert "Fix tests if wolfi is enabled"
mrodm Sep 3, 2024
3d65e8e
Revert "Reapply "[test apache] Fix policy test in data stream error""
mrodm Sep 3, 2024
e37b053
Revert "Reapply "Update test policies""
mrodm Sep 3, 2024
ebf4f95
Revert "Reapply "Test with 8.16.0 SNAPSHOT tests""
mrodm Sep 3, 2024
acc048b
Update tests to be independent from env vars defined in CI
mrodm Sep 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .buildkite/pipeline.serverless.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ env:
ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT: "true"
# Disable comparison of results in pipeline tests to avoid errors related to GeoIP fields
ELASTIC_PACKAGE_SERVERLESS_PIPELINE_TEST_DISABLE_COMPARE_RESULTS: "true"
# Enable/Disable the usage of wolfi images for Elastic Agent
ELASTIC_PACKAGE_DISABLE_ELASTIC_AGENT_WOLFI: "${ELASTIC_PACKAGE_DISABLE_ELASTIC_AGENT_WOLFI:-false}"


steps:
- input: "Input values for the variables"
Expand Down
3 changes: 3 additions & 0 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ env:
ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT: "true"
# Set maximum number of parallel tests to run if package allows it
ELASTIC_PACKAGE_MAXIMUM_NUMBER_PARALLEL_TESTS: 3
# Enable/Disable the usage of wolfi images for Elastic Agent
ELASTIC_PACKAGE_DISABLE_ELASTIC_AGENT_WOLFI: "${ELASTIC_PACKAGE_DISABLE_ELASTIC_AGENT_WOLFI:-false}"


steps:
- label: ":go: Run check-static"
Expand Down
2 changes: 1 addition & 1 deletion internal/install/stack_version.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ package install

const (
// DefaultStackVersion is the default version of the stack
DefaultStackVersion = "8.15.0"
DefaultStackVersion = "8.16.0-SNAPSHOT"
mrodm marked this conversation as resolved.
Show resolved Hide resolved
)
2 changes: 1 addition & 1 deletion scripts/test-check-packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ cleanup() {
kind delete cluster || true
fi

# In case it is tested with Elatic serverless, there should be just one Elastic stack
# In case it is tested with Elastic serverless, there should be just one Elastic stack
# started to test all packages. In our CI, this Elastic serverless stack is started
# at the beginning of the pipeline and must be running for all packages without stopping it between
# packages.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ inputs:
- apache-access
type: logfile
use_output: default
namespaces: []
Copy link
Contributor Author

@mrodm mrodm Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like that for policy tests it should also be needed to be able to mark some fields optional @jsoriano

This namespaces field appeared when testing with 8.16.0-SNAPSHOT. But they are not present in older stack versions (e.g. 8.15.0).

Changes required present in these two commits:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverting these two commits in order to be able to merge this PR.

output_permissions:
default:
_elastic_agent_checks:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ inputs:
- apache-error
type: logfile
use_output: default
namespaces: []
output_permissions:
default:
_elastic_agent_checks:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ inputs:
server_status_path: /server-status
type: apache/metrics
use_output: default
namespaces: []
output_permissions:
default:
_elastic_agent_checks:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,20 @@ vars:
agent:
runtime: docker
provisioning_script:
language: "bash"
language: "sh"
contents: |
apt-get update && apt-get -y install libaio1 wget unzip
set -eu
if grep wolfi /etc/os-release > /dev/null ; then
apk update && apk add libaio wget unzip
else
apt-get update && apt-get -y install libaio1 wget unzip
fi
mkdir -p /opt/oracle
cd /opt/oracle
wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-basic-linux.x64-21.4.0.0.0dbru.zip && unzip -o instantclient-basic-linux.x64-21.4.0.0.0dbru.zip
wget https://download.oracle.com/otn_software/linux/instantclient/217000/instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip && unzip -o instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip
echo /opt/oracle/instantclient_21_4 > /etc/ld.so.conf.d/oracle-instantclient.conf && ldconfig
wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-basic-linux.x64-21.4.0.0.0dbru.zip && unzip -o instantclient-basic-linux.x64-21.4.0.0.0dbru.zip || exit 1
wget https://download.oracle.com/otn_software/linux/instantclient/217000/instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip && unzip -o instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip || exit 1
mkdir -p /etc/ld.so.conf.d/
echo /opt/oracle/instantclient_21_4 > /etc/ld.so.conf.d/oracle-instantclient.conf && ldconfig || exit 1
cp /opt/oracle/instantclient_21_7/glogin.sql /opt/oracle/instantclient_21_7/libsqlplus.so /opt/oracle/instantclient_21_7/libsqlplusic.so /opt/oracle/instantclient_21_7/sqlplus /opt/oracle/instantclient_21_4/
pre_start_script:
language: "sh"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ inputs:
sql_response_format: variables
type: sql/metrics
use_output: default
namespaces: []
output_permissions:
default:
_elastic_agent_checks:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ inputs:
sql_response_format: variables
type: sql/metrics
use_output: default
namespaces: []
output_permissions:
default:
_elastic_agent_checks:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ inputs:
- preserve_original_event
type: httpjson
use_output: default
namespaces: []
output_permissions:
default:
_elastic_agent_checks:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
ARG ES_AGENT_IMAGE=docker.elastic.co/elastic-agent/elastic-agent-complete:latest
FROM $ES_AGENT_IMAGE
USER root
RUN apt-get update && apt-get -y install \
libaio1 \
wget \
unzip
RUN if grep wolfi /etc/os-release > /dev/null ; then \
apk update && apk add libaio wget unzip; \
else \
apt-get update && apt-get -y install libaio1 wget unzip; \
fi
WORKDIR /opt/oracle
RUN wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-basic-linux.x64-21.4.0.0.0dbru.zip && unzip -o instantclient-basic-linux.x64-21.4.0.0.0dbru.zip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/217000/instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip && unzip -o instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip
RUN echo /opt/oracle/instantclient_21_4 > /etc/ld.so.conf.d/oracle-instantclient.conf && ldconfig
RUN wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-basic-linux.x64-21.4.0.0.0dbru.zip && unzip -o instantclient-basic-linux.x64-21.4.0.0.0dbru.zip || exit 1
RUN wget https://download.oracle.com/otn_software/linux/instantclient/217000/instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip && unzip -o instantclient-sqlplus-linux.x64-21.7.0.0.0dbru.zip || exit 1
RUN mkdir -p /etc/ld.so.conf.d ; echo /opt/oracle/instantclient_21_4 > /etc/ld.so.conf.d/oracle-instantclient.conf && ldconfig || exit 1
RUN cp /opt/oracle/instantclient_21_7/glogin.sql /opt/oracle/instantclient_21_7/libsqlplus.so /opt/oracle/instantclient_21_7/libsqlplusic.so /opt/oracle/instantclient_21_7/sqlplus /opt/oracle/instantclient_21_4/
RUN export LD_LIBRARY_PATH=/opt/oracle/instantclient_21_4:$LD_LIBRARY_PATH && export PATH=/opt/oracle/instantclient_21_7:$PATH
ENV LD_LIBRARY_PATH "${LD_LIBRARY_PATH}:/opt/oracle/instantclient_21_4"
ENV PATH "${PATH}:/opt/oracle/instantclient_21_4"
ENV LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/opt/oracle/instantclient_21_4"
ENV PATH="${PATH}:/opt/oracle/instantclient_21_4"
WORKDIR /opt/oracle/instantclient_21_4