Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add config to enable apm-server #1593

Merged
merged 4 commits into from
Dec 13, 2023
Merged

add config to enable apm-server #1593

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8df2f0d
add config to enable apm-server
graphaelli Dec 12, 2023
1180fdd
add integration test
graphaelli Dec 12, 2023
bc284df
trigger integration test from CI
graphaelli Dec 12, 2023
47d9a8c
accommodate old curl
graphaelli Dec 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .buildkite/pipeline.trigger.integration.tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ STACK_COMMAND_TESTS=(
test-stack-command-7x
test-stack-command-86
test-stack-command-8x
test-stack-command-with-apm-server
)

for test in ${STACK_COMMAND_TESTS[@]}; do
Expand Down
5 changes: 4 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,10 @@ test-stack-command-86:
test-stack-command-8x:
./scripts/test-stack-command.sh 8.11.0-SNAPSHOT

test-stack-command: test-stack-command-default test-stack-command-7x test-stack-command-800 test-stack-command-8x
test-stack-command-with-apm-server:
APM_SERVER_ENABLED=true ./scripts/test-stack-command.sh

test-stack-command: test-stack-command-default test-stack-command-7x test-stack-command-800 test-stack-command-8x test-stack-command-with-apm-server

test-check-packages: test-check-packages-with-kind test-check-packages-other test-check-packages-parallel test-check-packages-with-custom-agent test-check-packages-benchmarks test-check-packages-false-positives test-check-packages-with-logstash

Expand Down
4 changes: 4 additions & 0 deletions internal/profile/_static/config.yml.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@
# Region where the Serverless project is going to be created
# stack.serverless.region: aws-us-east-1

## Enable apm-server
# Flag to enable apm-server in elastic-package stack profile config
# stack.apm_server_enabled: true

## Enable logstash for testing
# Flag to enable logstash in elastic-package stack profile config
# stack.logstash_enabled: true
Expand Down
1 change: 1 addition & 0 deletions internal/profile/_testdata/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# An expected setting.
stack.geoip_dir: "/home/foo/Documents/ingest-geoip"
stack.apm_server_enabled: true
stack.logstash_enabled: true

# An empty string, should exist, but return empty.
Expand Down
5 changes: 5 additions & 0 deletions internal/profile/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,11 @@ func TestLoadProfileConfig(t *testing.T) {
expected: "false",
found: true,
},
{
name: "stack.apm_server_enabled",
expected: "true",
found: true,
},
{
name: "stack.logstash_enabled",
expected: "true",
Expand Down
4 changes: 4 additions & 0 deletions internal/stack/_static/docker-compose-stack.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,10 @@ services:
- "../certs/fleet-server:/etc/ssl/elastic-agent"
ports:
- "127.0.0.1:8220:8220"
{{ $apm_server_enabled := fact "apm_server_enabled" }}
{{ if eq $apm_server_enabled "true" }}
- "127.0.0.1:8200:8200"
{{ end }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea to add this to the Fleet Server agent 👍 maybe at some point we can keep it always enabled and/or reuse it for actual performance monitoring of the testing framework.

Another option would be to create another agent, but this will probably bring unnecessary complexity.


fleet-server_is_ready:
image: tianon/true
Expand Down
18 changes: 18 additions & 0 deletions internal/stack/_static/kibana.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ xpack.fleet.packages:
version: latest
- name: fleet_server
version: latest
{{ $apm_server_enabled := fact "apm_server_enabled" }}
{{ if eq $apm_server_enabled "true" }}
- name: apm
version: latest
{{ end }}
xpack.fleet.agentPolicies:
- name: Elastic-Agent (elastic-package)
id: elastic-agent-managed-ep
Expand All @@ -72,6 +77,19 @@ xpack.fleet.agentPolicies:
id: default-fleet-server
package:
name: fleet_server
{{ $apm_server_enabled := fact "apm_server_enabled" }}
{{ if eq $apm_server_enabled "true" }}
- name: apm-1
package:
name: apm
inputs:
- type: apm
vars:
- name: host
value: "0.0.0.0:8200"
- name: secret_token
value: ""
{{ end }}
xpack.fleet.outputs:
- id: fleet-default-output
name: default
Expand Down
5 changes: 3 additions & 2 deletions internal/stack/resources.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,9 @@ func applyResources(profile *profile.Profile, stackVersion string) error {
"username": elasticsearchUsername,
"password": elasticsearchPassword,

"geoip_dir": profile.Config("stack.geoip_dir", "./ingest-geoip"),
"logstash_enabled": profile.Config("stack.logstash_enabled", "false"),
"geoip_dir": profile.Config("stack.geoip_dir", "./ingest-geoip"),
"apm_server_enabled": profile.Config("stack.apm_server_enabled", "false"),
"logstash_enabled": profile.Config("stack.logstash_enabled", "false"),
})

os.MkdirAll(stackDir, 0755)
Expand Down
25 changes: 25 additions & 0 deletions scripts/test-stack-command.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
set -euxo pipefail

VERSION=${1:-default}
APM_SERVER_ENABLED=${APM_SERVER_ENABLED:-false}

cleanup() {
r=$?
Expand All @@ -13,6 +14,11 @@ cleanup() {
# Take down the stack
elastic-package stack down -v

if [ "${APM_SERVER_ENABLED}" = true ]; then
# Create an apm-server profile and use it
elastic-package profiles delete with-apm-server
fi

exit $r
}

Expand All @@ -34,7 +40,22 @@ if [ "${VERSION}" != "default" ]; then
EXPECTED_VERSION=${VERSION}
fi

if [ "${APM_SERVER_ENABLED}" = true ]; then
# Create an apm-server profile and use it
profile=with-apm-server
elastic-package profiles create -v ${profile}
elastic-package profiles use ${profile}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea to use profiles for this 👍


# Create the config and enable apm-server
cat ~/.elastic-package/profiles/${profile}/config.yml.example - <<EOF > ~/.elastic-package/profiles/${profile}/config.yml
stack.apm_server_enabled: true
EOF
fi

OUTPUT_PATH_STATUS="build/elastic-stack-status/${VERSION}"
if [ "${APM_SERVER_ENABLED}" = true ]; then
OUTPUT_PATH_STATUS="build/elastic-stack-status/${VERSION}_with_apm_server"
fi
mkdir -p ${OUTPUT_PATH_STATUS}

# Initial status empty
Expand Down Expand Up @@ -71,4 +92,8 @@ elastic-package stack status -v 2> ${OUTPUT_PATH_STATUS}/running.txt
clean_status_output "${OUTPUT_PATH_STATUS}/expected_running.txt" > ${OUTPUT_PATH_STATUS}/expected_no_spaces.txt
clean_status_output "${OUTPUT_PATH_STATUS}/running.txt" > ${OUTPUT_PATH_STATUS}/running_no_spaces.txt

if [ "${APM_SERVER_ENABLED}" = true ]; then
curl http://localhost:8200/
fi

diff -q ${OUTPUT_PATH_STATUS}/running_no_spaces.txt ${OUTPUT_PATH_STATUS}/expected_no_spaces.txt