Skip to content

Commit 40b7e88

Browse files
Kaan Yaltimergify[bot]
authored andcommitted
Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection (#6907)
* Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated preinstall script, updated service to use uninstall token * enhancmenet(6394): updated the preinstall script * enchancement(6394): started adding integraiton tests * enhancement(6394): updated fixture install, updated endpoint security tests * enhancement(6394): cleaned up fixture_install, added function that exposes fixture's uninstall tokens, updated tests * enhancement(6394): refactored test code so that I can use it with rpm * enhancement(6394): added tests to assert that tamper protection works * enhancement(6394): updated the endpoint testing tools, fixture install functions and the deb rpm upgrade tests * enhancement(6394): added test logs, updated rpm installation to set agent socket path * enhancement(6394): remove commented code * enhancement(6394): remove print statements * enhancement(6394): remove unnecessary comments, refactor unused function * enhancement(6394): revert var name change * enhancement(6394): added changelog * enchancement(6394): update test logs, add non integrative config to deb installation * enhancement(6394): updated the endpoint version comparison and assertion * enhancement(6394): added log in tests * enhancement(6394): resorted to using previous major instead of minor in upgrade test * enhancement(6394): updated endpoint version function in the tests, updated function name in testing tools * enhancement(6394): use previous minor, fix log * enhancement(6394): added comment explaining motive behind simple install functions * enhancement(6394): updated return in tools * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> * enhancement(6394): fixed function call in tests * enhancement(6394): added systemctl start in postinstall, refactored preinstall and added condition to make same version installations work * enhancement(6394): updated the preinstall and postinstall scripts to troubleshoot * enhancement(6394): updated preinstall and postinstall script templates - Updated preinstall to stop endpoint if it is an available service regardless of the version of endpoint that's install - Updated postintall to start endpoint if the old endpoint version and the new version match. * enhancement(6394): removed error exit from postinstall * enhancement(6394): updated postinstall and preinstall templates - Preinstall now does not use a state file. Recovery from failure start ElasticEndpoint if it is not running - Preinstall does not stop endpoint if tamper protection is not enabled - Postinstall does not print an error if service is still running * enhancement(6394): removed debug logs * enhancement(6394): removed unnecessary comment * enhancement(6394): store uninstall token as local var, uninstall through the agent * enhancement(6394): added setclient function * enhancement(6394): added getInstallCommand and replaced SimpleInstall * enhancement(6394): added test case for error recovery. removed unused fixture functions * enhancement(6394): refactored tests, consolidated test scenarios into one function * enhancement(6394): remove unnecessary test functions * enhancement(6394): remove unused fixture function * enhancement(6394): revert unwanted installDeb changes * enhancement(6394): remove unwanted changes in testing tools * enhancement(6394): remove unused function call * enhancement(6394): replacing systemctl instead of adding new one to path * enhancement(6394): update real systemctl path in mock systemctl script * enhancement(6394): fix linting errors * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated print statement * enhancement(6394): remove unnecessary command * enhancement(6394): use addressFromPath and SetClient * enhancement(6394): using service name, fixed indentation * test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging * Revert "test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging" This reverts commit 390c561. * enhancement(6394): renamed SetClient to SetDebRpmClient. Using hardcoded working dir as fixture working dir does not work for determining socket path * enhancement(6394): consolidated same version upgrade and regular upgrdade test functions * enhancement(6394): simplify preinstall script and enhance upgrade tests for tamper protection - Removed unnecessary endpoint handling logic from preinstall script. - Improved checks for service installation and status before upgrade. - Updated upgrade test functions to handle stopping the endpoint service before upgrades. * enhancement(6394): remove mock systemctl script for tamper protection tests * enhancement(6394): remove unused import * enhancement(6394): fixed order of execution in preinstall * enhancement(6394): added tests to make sure deb/rpm upgrades work when endpoint is not tamper protected --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> (cherry picked from commit 8a6531f) # Conflicts: # dev-tools/packaging/templates/linux/preinstall.sh.tmpl
1 parent 404e19f commit 40b7e88

File tree

7 files changed

+539
-41
lines changed

7 files changed

+539
-41
lines changed
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Kind can be one of:
2+
# - breaking-change: a change to previously-documented behavior
3+
# - deprecation: functionality that is being removed in a later release
4+
# - bug-fix: fixes a problem in a previous version
5+
# - enhancement: extends functionality but does not break or fix existing behavior
6+
# - feature: new functionality
7+
# - known-issue: problems that we are aware of in a given version
8+
# - security: impacts on the security of a product or a user’s deployment.
9+
# - upgrade: important information for someone upgrading from a prior version
10+
# - other: does not fit into any of the other categories
11+
kind: enhancement
12+
13+
# Change summary; a 80ish characters long description of the change.
14+
summary: Allow upgrading deb or rpm agents when using Elastic Defend with tamper protection.
15+
16+
# Long description; in case the summary is not enough to describe the change
17+
# this field accommodate a description without length limits.
18+
# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
19+
#description:
20+
21+
# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
22+
component: elastic-agent
23+
# PR URL; optional; the PR number that added the changeset.
24+
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
25+
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
26+
# Please provide it if you are adding a fragment for a different PR.
27+
pr: https://github.com/elastic/elastic-agent/pull/6907
28+
# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
29+
# If not present is automatically filled by the tooling with the issue linked to the PR number.
30+
issue: https://github.com/elastic/elastic-agent/issues/6394

dev-tools/packaging/templates/linux/postinstall.sh.tmpl

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,42 @@ commit_hash="{{ commit_short }}"
66
version_dir="{{agent_package_version}}{{snapshot_suffix}}"
77
symlink="/usr/share/elastic-agent/bin/elastic-agent"
88
new_agent_dir="/var/lib/elastic-agent/data/elastic-agent-$version_dir-$commit_hash"
9+
new_endpoint_component_bin="$new_agent_dir/components/endpoint-security"
910

1011
# delete $symlink if exists
1112
if test -L "$symlink"; then
1213
echo "found symlink $symlink, unlink"
1314
unlink "$symlink"
1415
fi
1516

17+
SERVICE_NAME="ElasticEndpoint"
18+
19+
echo "Checking if $SERVICE_NAME is installed"
20+
if systemctl list-unit-files --type=service | grep -q "^${SERVICE_NAME}.service"; then
21+
22+
installed_endpoint_version="$(/opt/Elastic/Endpoint/elastic-endpoint version)"
23+
archive_endpoint_version="$("$new_endpoint_component_bin" version)"
24+
25+
echo "${SERVICE_NAME} is installed at version ${installed_endpoint_version}"
26+
if ! systemctl is-active --quiet "$SERVICE_NAME"; then
27+
echo "$SERVICE_NAME is installed but not running"
28+
if [[ "$installed_endpoint_version" == "$archive_endpoint_version" ]]; then
29+
echo "New endpoint and installed endpoint versions are the same: \"${installed_endpoint_version}\""
30+
echo "Starting $SERVICE_NAME"
31+
sudo systemctl start ${SERVICE_NAME}
32+
else
33+
echo "New endpoint version \"${archive_endpoint_version}\" is different than the one that's already
34+
installed \"${installed_endpoint_version}\""
35+
fi
36+
else
37+
# Endpoint can already be running if tamper protection is not enabled
38+
echo "$SERVICE_NAME is already running"
39+
fi
40+
else
41+
echo "$SERVICE_NAME is not installed"
42+
fi
43+
44+
1645
# create symlink to the new agent
1746
echo "create symlink "$symlink" to "$new_agent_dir/elastic-agent""
1847
ln -s "$new_agent_dir/elastic-agent" "$symlink"

dev-tools/packaging/templates/linux/preinstall.sh.tmpl

Lines changed: 66 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,33 @@
22

33
set -e
44

5+
SERVICE_NAME="ElasticEndpoint"
6+
7+
# Check if the endpoint service is installed
8+
if systemctl list-unit-files --type=service | grep -q "^${SERVICE_NAME}.service"; then
9+
echo "$SERVICE_NAME is installed"
10+
11+
# Remove the vault directory if it exists
12+
if [ -d "/opt/Elastic/Endpoint/state/vault" ]; then
13+
14+
# Check if the endpoint is running
15+
if systemctl --quiet is-active $SERVICE_NAME; then
16+
echo "$SERVICE_NAME is running, stopping it"
17+
systemctl --quiet stop $SERVICE_NAME
18+
else
19+
echo "$SERVICE_NAME is not running"
20+
fi
21+
22+
echo "$SERVICE_NAME is tamper protected"
23+
echo "Removing $SERVICE_NAME vault"
24+
rm -rf "/opt/Elastic/Endpoint/state/vault"
25+
else
26+
echo "$SERVICE_NAME tamper protection is not enabled"
27+
fi
28+
else
29+
echo "$SERVICE_NAME is not installed"
30+
fi
31+
532
commit_hash="{{ commit_short }}"
633
version_dir="{{agent_package_version}}{{snapshot_suffix}}"
734
symlink="/usr/share/elastic-agent/bin/elastic-agent"
@@ -11,27 +38,27 @@ old_agent_dir=""
1138
# upon upgrade we migrate the current symlink to an upgrade symlink as the previous
1239
# installed version will remove the symlink
1340
if test -L "$symlink"; then
14-
resolved_symlink="$(readlink -- "$symlink")"
15-
if ! [ -z "$resolved_symlink" ]; then
16-
old_agent_dir="$( dirname "$resolved_symlink" )"
17-
echo "previous installation directory $old_agent_dir"
18-
else
19-
echo "unable to read existing symlink"
20-
fi
41+
resolved_symlink="$(readlink -- "$symlink")"
42+
if ! [ -z "$resolved_symlink" ]; then
43+
old_agent_dir="$(dirname "$resolved_symlink")"
44+
echo "previous installation directory $old_agent_dir"
45+
else
46+
echo "unable to read existing symlink"
47+
fi
2148

22-
# copy the state files if there was a previous agent install
23-
if ! [ -z "$old_agent_dir" ] && ! [ "$old_agent_dir" -ef "$new_agent_dir" ]; then
24-
yml_path="$old_agent_dir/state.yml"
25-
enc_path="$old_agent_dir/state.enc"
26-
echo "migrate state from $old_agent_dir to $new_agent_dir"
49+
# copy the state files if there was a previous agent install
50+
if ! [ -z "$old_agent_dir" ] && ! [ "$old_agent_dir" -ef "$new_agent_dir" ]; then
51+
yml_path="$old_agent_dir/state.yml"
52+
enc_path="$old_agent_dir/state.enc"
53+
echo "migrate state from $old_agent_dir to $new_agent_dir"
2754

28-
if test -f "$yml_path"; then
29-
echo "found "$yml_path", copy to "$new_agent_dir"."
30-
mkdir -p "$new_agent_dir"
31-
cp "$yml_path" "$new_agent_dir"
32-
else
33-
echo "didn't find $yml_path"
34-
fi
55+
if test -f "$yml_path"; then
56+
echo "found "$yml_path", copy to "$new_agent_dir"."
57+
mkdir -p "$new_agent_dir"
58+
cp "$yml_path" "$new_agent_dir"
59+
else
60+
echo "didn't find $yml_path"
61+
fi
3562

3663
if test -f "$enc_path"; then
3764
echo "found "$enc_path", copy to "$new_agent_dir"."
@@ -52,5 +79,25 @@ if test -L "$symlink"; then
5279
fi
5380
fi
5481
else
82+
<<<<<<< HEAD
5583
echo "no previous installation found"
84+
=======
85+
echo "no previous installation found"
86+
87+
# create dir in case it does not exist
88+
mkdir -p "$new_agent_dir"
89+
90+
# 2 is upgrade for Fedora, do not upgrade file when upgrading and file exists
91+
if [[ "$1" != "2" ]]; then
92+
if [[ -n "${ELASTIC_AGENT_FLAVOR}" ]]; then
93+
# Do not modify the file if it already exists
94+
echo "using \"${ELASTIC_AGENT_FLAVOR}\" flavor from environment"
95+
echo "${ELASTIC_AGENT_FLAVOR}" > "$flavor_file"
96+
else
97+
# Defaults to basic installation
98+
echo "defaulting to basic flavor"
99+
echo "basic" > "$flavor_file"
100+
fi
101+
fi
102+
>>>>>>> 8a6531f43 (Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection (#6907))
56103
fi

pkg/testing/fixture.go

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -346,7 +346,6 @@ func (f *Fixture) RunBeat(ctx context.Context) error {
346346
process.WithContext(ctx),
347347
process.WithArgs(args),
348348
process.WithCmdOptions(attachOutErr(stdOut, stdErr)))
349-
350349
if err != nil {
351350
return fmt.Errorf("failed to spawn %s: %w", f.binaryName, err)
352351
}
@@ -401,7 +400,8 @@ func RunProcess(t *testing.T,
401400
lp Logger,
402401
ctx context.Context, runLength time.Duration,
403402
logOutput, allowErrs bool,
404-
processPath string, args ...string) error {
403+
processPath string, args ...string,
404+
) error {
405405
if _, deadlineSet := ctx.Deadline(); !deadlineSet {
406406
t.Fatal("Context passed to RunProcess() has no deadline set.")
407407
}
@@ -419,7 +419,6 @@ func RunProcess(t *testing.T,
419419
process.WithContext(ctx),
420420
process.WithArgs(args),
421421
process.WithCmdOptions(attachOutErr(stdOut, stdErr)))
422-
423422
if err != nil {
424423
return fmt.Errorf("failed to spawn %q: %w", processPath, err)
425424
}
@@ -549,7 +548,6 @@ func (f *Fixture) executeWithClient(ctx context.Context, command string, disable
549548
process.WithContext(ctx),
550549
process.WithArgs(args),
551550
process.WithCmdOptions(attachOutErr(stdOut, stdErr)))
552-
553551
if err != nil {
554552
return fmt.Errorf("failed to spawn %s: %w", f.binaryName, err)
555553
}

pkg/testing/fixture_install.go

Lines changed: 38 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,13 @@ import (
2626
"github.com/elastic/elastic-agent-libs/mapstr"
2727
agentsystemprocess "github.com/elastic/elastic-agent-system-metrics/metric/system/process"
2828
"github.com/elastic/elastic-agent/internal/pkg/agent/application/paths"
29+
"github.com/elastic/elastic-agent/pkg/control"
2930
"github.com/elastic/elastic-agent/pkg/control/v2/client"
3031
"github.com/elastic/elastic-agent/pkg/core/process"
3132
)
3233

3334
// ErrNotInstalled is returned in cases where Agent isn't installed
34-
var ErrNotInstalled = errors.New("Elastic Agent is not installed") //nolint:stylecheck // Elastic Agent is a proper noun
35+
var ErrNotInstalled = errors.New("Elastic Agent is not installed") //nolint:staticcheck // Elastic Agent is a proper noun
3536

3637
// CmdOpts creates vectors of command arguments for different agent commands
3738
type CmdOpts interface {
@@ -317,12 +318,12 @@ func (f *Fixture) installNoPkgManager(ctx context.Context, installOpts *InstallO
317318

318319
// environment variable AGENT_KEEP_INSTALLED=true will skip the uninstallation
319320
// useful to debug the issue with the Elastic Agent
320-
if f.t.Failed() && keepInstalledFlag() {
321+
if f.t.Failed() && KeepInstalledFlag() {
321322
f.t.Logf("skipping uninstall; test failed and AGENT_KEEP_INSTALLED=true")
322323
return
323324
}
324325

325-
if keepInstalledFlag() {
326+
if KeepInstalledFlag() {
326327
f.t.Logf("ignoring AGENT_KEEP_INSTALLED=true as test succeeded, " +
327328
"keeping the agent installed will jeopardise other tests")
328329
}
@@ -432,6 +433,19 @@ func getProcesses(t *gotesting.T, regex string) []runningProcess {
432433
return processes
433434
}
434435

436+
func (f *Fixture) SetDebRpmClient() error {
437+
workDir := "/var/lib/elastic-agent"
438+
socketPath, err := control.AddressFromPath(f.operatingSystem, workDir)
439+
if err != nil {
440+
return fmt.Errorf("failed to get control protcol address: %w", err)
441+
}
442+
443+
c := client.New(client.WithAddress(socketPath))
444+
f.setClient(c)
445+
446+
return nil
447+
}
448+
435449
// installDeb installs the prepared Elastic Agent binary from the deb
436450
// package and registers a t.Cleanup function to uninstall the agent if
437451
// it hasn't been uninstalled. It also takes care of collecting a
@@ -465,7 +479,7 @@ func (f *Fixture) installDeb(ctx context.Context, installOpts *InstallOpts, shou
465479
f.t.Logf("error systemctl stop elastic-agent: %s, output: %s", err, string(out))
466480
}
467481

468-
if keepInstalledFlag() {
482+
if KeepInstalledFlag() {
469483
f.t.Logf("skipping uninstall; test failed and AGENT_KEEP_INSTALLED=true")
470484
return
471485
}
@@ -485,6 +499,11 @@ func (f *Fixture) installDeb(ctx context.Context, installOpts *InstallOpts, shou
485499
return out, fmt.Errorf("systemctl start elastic-agent failed: %w", err)
486500
}
487501

502+
err = f.SetDebRpmClient()
503+
if err != nil {
504+
return nil, err
505+
}
506+
488507
if !shouldEnroll {
489508
return nil, nil
490509
}
@@ -503,11 +522,11 @@ func (f *Fixture) installDeb(ctx context.Context, installOpts *InstallOpts, shou
503522
if installOpts.DelayEnroll {
504523
enrollArgs = append(enrollArgs, "--delay-enroll")
505524
}
506-
if installOpts.EnrollOpts.URL != "" {
507-
enrollArgs = append(enrollArgs, "--url", installOpts.EnrollOpts.URL)
525+
if installOpts.URL != "" {
526+
enrollArgs = append(enrollArgs, "--url", installOpts.URL)
508527
}
509-
if installOpts.EnrollOpts.EnrollmentToken != "" {
510-
enrollArgs = append(enrollArgs, "--enrollment-token", installOpts.EnrollOpts.EnrollmentToken)
528+
if installOpts.EnrollmentToken != "" {
529+
enrollArgs = append(enrollArgs, "--enrollment-token", installOpts.EnrollmentToken)
511530
}
512531
out, err = exec.CommandContext(ctx, "sudo", enrollArgs...).CombinedOutput()
513532
if err != nil {
@@ -572,6 +591,11 @@ func (f *Fixture) installRpm(ctx context.Context, installOpts *InstallOpts, shou
572591
return out, fmt.Errorf("systemctl start elastic-agent failed: %w", err)
573592
}
574593

594+
err = f.SetDebRpmClient()
595+
if err != nil {
596+
return nil, err
597+
}
598+
575599
if !shouldEnroll {
576600
return nil, nil
577601
}
@@ -590,11 +614,11 @@ func (f *Fixture) installRpm(ctx context.Context, installOpts *InstallOpts, shou
590614
if installOpts.DelayEnroll {
591615
enrollArgs = append(enrollArgs, "--delay-enroll")
592616
}
593-
if installOpts.EnrollOpts.URL != "" {
594-
enrollArgs = append(enrollArgs, "--url", installOpts.EnrollOpts.URL)
617+
if installOpts.URL != "" {
618+
enrollArgs = append(enrollArgs, "--url", installOpts.URL)
595619
}
596-
if installOpts.EnrollOpts.EnrollmentToken != "" {
597-
enrollArgs = append(enrollArgs, "--enrollment-token", installOpts.EnrollOpts.EnrollmentToken)
620+
if installOpts.EnrollmentToken != "" {
621+
enrollArgs = append(enrollArgs, "--enrollment-token", installOpts.EnrollmentToken)
598622
}
599623
// run sudo elastic-agent enroll
600624
out, err = exec.CommandContext(ctx, "sudo", enrollArgs...).CombinedOutput()
@@ -696,7 +720,7 @@ func (f *Fixture) uninstallNoPkgManager(ctx context.Context, uninstallOpts *Unin
696720
}
697721

698722
if err != nil && topPathStats != nil {
699-
return out, fmt.Errorf("Elastic Agent is still installed at [%s]", topPath) //nolint:stylecheck // Elastic Agent is a proper noun
723+
return out, fmt.Errorf("Elastic Agent is still installed at [%s]", topPath) //nolint:staticcheck // Elastic Agent is a proper noun
700724
}
701725

702726
return out, nil
@@ -799,7 +823,7 @@ func collectDiagFlag() bool {
799823
return v
800824
}
801825

802-
func keepInstalledFlag() bool {
826+
func KeepInstalledFlag() bool {
803827
// failure reports false (ignore error)
804828
v, _ := strconv.ParseBool(os.Getenv("AGENT_KEEP_INSTALLED"))
805829
return v

pkg/testing/tools/tools.go

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,22 @@ func IsPolicyRevision(ctx context.Context, t *testing.T, client *kibana.Client,
3636
}
3737
}
3838

39+
func GetUninstallToken(ctx context.Context, kibClient *kibana.Client, policyId string) (string, error) {
40+
resp, err := kibClient.GetPolicyUninstallTokens(ctx, policyId)
41+
if err != nil {
42+
return "", fmt.Errorf("failed to fetch uninstall tokens: %w", err)
43+
}
44+
if len(resp.Items) == 0 {
45+
return "", fmt.Errorf("expected non-zero number of tokens")
46+
}
47+
48+
if len(resp.Items[0].Token) == 0 {
49+
return "", fmt.Errorf("expected non-empty token")
50+
}
51+
52+
return resp.Items[0].Token, nil
53+
}
54+
3955
// InstallAgentWithPolicy creates the given policy, enrolls the given agent
4056
// fixture in Fleet using the default Fleet Server, waits for the agent to be
4157
// online, and returns the created policy.
@@ -137,7 +153,6 @@ func InstallAgentForPolicyWithToken(ctx context.Context, t *testing.T,
137153

138154
installOpts.URL = fleetServerURL
139155
}
140-
141156
output, err := agentFixture.Install(ctx, &installOpts)
142157
if err != nil {
143158
t.Log(string(output))

0 commit comments

Comments
 (0)