Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection #6907

kaanyalti · 2025-02-18T03:37:35Z

Enhancement

What does this PR do?

Updates the preinstall script for deb and rpm to stop the endpoint security service and remove the vault

Why is it important?

Currently when users upgrade their deb or rpm agents using their respective package managers, if there is endpoint running and if endpoint has tamper protection enabled, endpoint will become unhealthy. This PR fixes it by replicating what enpoint-security uninstall --uninstall-token <token> does in the deb/rpm preinstall script.

Checklist

I have read and understood the pull request guidelines of this project.
My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in ./changelog/fragments using the changelog tool
I have added an integration test or an E2E test

Disruptive User Impact

There shouldn't be anything disruptive for the users.

How to test this PR locally

Start endpoint compatible VMs, one for deb one for rpm
Build the agent for these two platforms
Create an ESS deployment
Create a policy, add endpoint integration
Follow the "Add Agent" instructions and install the version that is shown on the instructions.
Validate that both the agent and endpoint are healthy
Upgrade the agent running either sudo apt install <Agent that you built> or sudo dnf install <Agent that you built>
After the upgrade is done, validate that both the agent and the endpoint are healthy
Just to be sure, validate that the new endpoint is still tamper protected
- Try to uninstall agent without a token and see it fail
- Try to uninstall agent with an uninstall token and validate that it is uninstalled
- Beware that removing the agent using the package managers will indeed uninstall the agent but will leave endpoint. Use the elastic-agent command to uninstall.

Repeat the steps above but test same version upgrades.

Install the agent you packaged from the branch.
Enroll into fleet
Install the same agent again with dpkg -i or rpm -Uvh --force
Check the status of the agent and endpoint
Validate that tamper protection is enabled

Note: When testing you may run into the following error

┌─ fleet
│  └─ status: (STARTING)
└─ elastic-agent
   ├─ status: (DEGRADED) 1 or more components/units in a failed state
   └─ endpoint-default
      ├─ status: (FAILED) Elastic Defend cannot run on an emulated architecture
      ├─ endpoint-default
      │  └─ status: (FAILED) Elastic Defend cannot run on an emulated architecture
      └─ endpoint-default-b73c5b0f-3b0a-4877-a531-35fdd9f9204f
         └─ status: (FAILED) Elastic Defend cannot run on an emulated architecture

This is related to the following bug #6866

Related issues

mergify · 2025-02-18T03:38:16Z

This pull request does not have a backport label. Could you fix it @kaanyalti? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
backport-active-all is the label that automatically backports to all active branches.
backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

elasticmachine · 2025-02-23T15:58:50Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

pkg/testing/fixture_install.go

pkg/testing/tools/tools.go

testing/integration/endpoint_security_test.go

pkg/testing/tools/tools.go

pchila

Code looks good, however there are still some failures in CI about TestUpgradeAgentWithTamperProtectedEndpoint_RPM since we try to upgrade from version 9.0.0-SNAPSHOT to 9.0.0-SNAPSHOT and rpm is not happy about it.
This is not a problem when installing using elastic-agent upgrade/install commands but here we are missing something I guess.

@cmacknz could you please chime in here: are we supposed to support rpm upgrades between two builds of 9.0.0-SNAPSHOT ?

    fixture_install.go:460: [test TestUpgradeAgentWithTamperProtectedEndpoint_RPM] Inside fixture simpleInstallRPM function
    endpoint_security_test.go:233:
        	Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-gcp-1740982239855044947/elastic/elastic-agent/testing/integration/endpoint_security_test.go:233
        	            				/opt/buildkite-agent/builds/bk-agent-prod-gcp-1740982239855044947/elastic/elastic-agent/testing/integration/endpoint_security_test.go:92
        	Error:      	Received unexpected error:
        	            	apt install failed: exit status 1 output:Verifying...                          ########################################
        	            	Preparing...                          ########################################
        	            		package elastic-agent-9.0.0-1.x86_64 is already installed
        	            		file /etc/elastic-agent/.elastic-agent.active.commit from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	            		file /lib/systemd/system/elastic-agent.service from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	            		file /usr/share/elastic-agent/.build_hash.txt from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	            		file /usr/share/elastic-agent/NOTICE.txt from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	Test:       	TestUpgradeAgentWithTamperProtectedEndpoint_RPM
    fixture.go:1286: Temporary directory "/tmp/TestUpgradeAgentWithTamperProtectedEndpoint_RPM3576483495" preserved for investigation/debugging

kaanyalti · 2025-03-03T19:11:07Z

Code looks good, however there are still some failures in CI about TestUpgradeAgentWithTamperProtectedEndpoint_RPM since we try to upgrade from version 9.0.0-SNAPSHOT to 9.0.0-SNAPSHOT and rpm is not happy about it. This is not a problem when installing using elastic-agent upgrade/install commands but here we are missing something I guess.

@cmacknz could you please chime in here: are we supposed to support rpm upgrades between two builds of 9.0.0-SNAPSHOT ?

    fixture_install.go:460: [test TestUpgradeAgentWithTamperProtectedEndpoint_RPM] Inside fixture simpleInstallRPM function
    endpoint_security_test.go:233:
        	Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-gcp-1740982239855044947/elastic/elastic-agent/testing/integration/endpoint_security_test.go:233
        	            				/opt/buildkite-agent/builds/bk-agent-prod-gcp-1740982239855044947/elastic/elastic-agent/testing/integration/endpoint_security_test.go:92
        	Error:      	Received unexpected error:
        	            	apt install failed: exit status 1 output:Verifying...                          ########################################
        	            	Preparing...                          ########################################
        	            		package elastic-agent-9.0.0-1.x86_64 is already installed
        	            		file /etc/elastic-agent/.elastic-agent.active.commit from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	            		file /lib/systemd/system/elastic-agent.service from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	            		file /usr/share/elastic-agent/.build_hash.txt from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	            		file /usr/share/elastic-agent/NOTICE.txt from install of elastic-agent-9.0.0-1.x86_64 conflicts with file from package elastic-agent-9.0.0-1.x86_64
        	Test:       	TestUpgradeAgentWithTamperProtectedEndpoint_RPM
    fixture.go:1286: Temporary directory "/tmp/TestUpgradeAgentWithTamperProtectedEndpoint_RPM3576483495" preserved for investigation/debugging

I reverted back to using PreviousMinor, hopefully this time it will work. I triggered deb and rpm tests locally, and they passed.

elastic-sonarqube · 2025-03-03T20:17:05Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml

dev-tools/packaging/templates/linux/preinstall.sh.tmpl

testing/integration/endpoint_security_test.go

pkg/testing/fixture_install.go

pkg/testing/fixture.go

mergify · 2025-03-13T11:05:05Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection upstream/enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection
git merge upstream/main
git push upstream enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection

pchila · 2025-03-26T13:05:00Z

@kaanyalti could you please fix conflicts and reply to @cmacknz comments ?

mergify · 2025-05-06T15:27:57Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection upstream/enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection
git merge upstream/main
git push upstream enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection

…n endpoint is not tamper protected

elastic-sonarqube · 2025-06-19T19:01:39Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

elasticmachine · 2025-06-19T19:48:00Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: a496935

Failed CI Steps

x86_64:sudo: fleet

History

💔 Build #22736 failed 4d865e0
💔 Build #22716 failed f0b57ec
💔 Build #22705 failed e55aa3d
💔 Build #22145 failed 46ff905
💔 Build #22096 failed 46ff905

cc @kaanyalti

…tion (#6907) * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated preinstall script, updated service to use uninstall token * enhancmenet(6394): updated the preinstall script * enchancement(6394): started adding integraiton tests * enhancement(6394): updated fixture install, updated endpoint security tests * enhancement(6394): cleaned up fixture_install, added function that exposes fixture's uninstall tokens, updated tests * enhancement(6394): refactored test code so that I can use it with rpm * enhancement(6394): added tests to assert that tamper protection works * enhancement(6394): updated the endpoint testing tools, fixture install functions and the deb rpm upgrade tests * enhancement(6394): added test logs, updated rpm installation to set agent socket path * enhancement(6394): remove commented code * enhancement(6394): remove print statements * enhancement(6394): remove unnecessary comments, refactor unused function * enhancement(6394): revert var name change * enhancement(6394): added changelog * enchancement(6394): update test logs, add non integrative config to deb installation * enhancement(6394): updated the endpoint version comparison and assertion * enhancement(6394): added log in tests * enhancement(6394): resorted to using previous major instead of minor in upgrade test * enhancement(6394): updated endpoint version function in the tests, updated function name in testing tools * enhancement(6394): use previous minor, fix log * enhancement(6394): added comment explaining motive behind simple install functions * enhancement(6394): updated return in tools * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> * enhancement(6394): fixed function call in tests * enhancement(6394): added systemctl start in postinstall, refactored preinstall and added condition to make same version installations work * enhancement(6394): updated the preinstall and postinstall scripts to troubleshoot * enhancement(6394): updated preinstall and postinstall script templates - Updated preinstall to stop endpoint if it is an available service regardless of the version of endpoint that's install - Updated postintall to start endpoint if the old endpoint version and the new version match. * enhancement(6394): removed error exit from postinstall * enhancement(6394): updated postinstall and preinstall templates - Preinstall now does not use a state file. Recovery from failure start ElasticEndpoint if it is not running - Preinstall does not stop endpoint if tamper protection is not enabled - Postinstall does not print an error if service is still running * enhancement(6394): removed debug logs * enhancement(6394): removed unnecessary comment * enhancement(6394): store uninstall token as local var, uninstall through the agent * enhancement(6394): added setclient function * enhancement(6394): added getInstallCommand and replaced SimpleInstall * enhancement(6394): added test case for error recovery. removed unused fixture functions * enhancement(6394): refactored tests, consolidated test scenarios into one function * enhancement(6394): remove unnecessary test functions * enhancement(6394): remove unused fixture function * enhancement(6394): revert unwanted installDeb changes * enhancement(6394): remove unwanted changes in testing tools * enhancement(6394): remove unused function call * enhancement(6394): replacing systemctl instead of adding new one to path * enhancement(6394): update real systemctl path in mock systemctl script * enhancement(6394): fix linting errors * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated print statement * enhancement(6394): remove unnecessary command * enhancement(6394): use addressFromPath and SetClient * enhancement(6394): using service name, fixed indentation * test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging * Revert "test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging" This reverts commit 390c561. * enhancement(6394): renamed SetClient to SetDebRpmClient. Using hardcoded working dir as fixture working dir does not work for determining socket path * enhancement(6394): consolidated same version upgrade and regular upgrdade test functions * enhancement(6394): simplify preinstall script and enhance upgrade tests for tamper protection - Removed unnecessary endpoint handling logic from preinstall script. - Improved checks for service installation and status before upgrade. - Updated upgrade test functions to handle stopping the endpoint service before upgrades. * enhancement(6394): remove mock systemctl script for tamper protection tests * enhancement(6394): remove unused import * enhancement(6394): fixed order of execution in preinstall * enhancement(6394): added tests to make sure deb/rpm upgrades work when endpoint is not tamper protected --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> (cherry picked from commit 8a6531f) # Conflicts: # dev-tools/packaging/templates/linux/preinstall.sh.tmpl

…tion (#6907) * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated preinstall script, updated service to use uninstall token * enhancmenet(6394): updated the preinstall script * enchancement(6394): started adding integraiton tests * enhancement(6394): updated fixture install, updated endpoint security tests * enhancement(6394): cleaned up fixture_install, added function that exposes fixture's uninstall tokens, updated tests * enhancement(6394): refactored test code so that I can use it with rpm * enhancement(6394): added tests to assert that tamper protection works * enhancement(6394): updated the endpoint testing tools, fixture install functions and the deb rpm upgrade tests * enhancement(6394): added test logs, updated rpm installation to set agent socket path * enhancement(6394): remove commented code * enhancement(6394): remove print statements * enhancement(6394): remove unnecessary comments, refactor unused function * enhancement(6394): revert var name change * enhancement(6394): added changelog * enchancement(6394): update test logs, add non integrative config to deb installation * enhancement(6394): updated the endpoint version comparison and assertion * enhancement(6394): added log in tests * enhancement(6394): resorted to using previous major instead of minor in upgrade test * enhancement(6394): updated endpoint version function in the tests, updated function name in testing tools * enhancement(6394): use previous minor, fix log * enhancement(6394): added comment explaining motive behind simple install functions * enhancement(6394): updated return in tools * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> * enhancement(6394): fixed function call in tests * enhancement(6394): added systemctl start in postinstall, refactored preinstall and added condition to make same version installations work * enhancement(6394): updated the preinstall and postinstall scripts to troubleshoot * enhancement(6394): updated preinstall and postinstall script templates - Updated preinstall to stop endpoint if it is an available service regardless of the version of endpoint that's install - Updated postintall to start endpoint if the old endpoint version and the new version match. * enhancement(6394): removed error exit from postinstall * enhancement(6394): updated postinstall and preinstall templates - Preinstall now does not use a state file. Recovery from failure start ElasticEndpoint if it is not running - Preinstall does not stop endpoint if tamper protection is not enabled - Postinstall does not print an error if service is still running * enhancement(6394): removed debug logs * enhancement(6394): removed unnecessary comment * enhancement(6394): store uninstall token as local var, uninstall through the agent * enhancement(6394): added setclient function * enhancement(6394): added getInstallCommand and replaced SimpleInstall * enhancement(6394): added test case for error recovery. removed unused fixture functions * enhancement(6394): refactored tests, consolidated test scenarios into one function * enhancement(6394): remove unnecessary test functions * enhancement(6394): remove unused fixture function * enhancement(6394): revert unwanted installDeb changes * enhancement(6394): remove unwanted changes in testing tools * enhancement(6394): remove unused function call * enhancement(6394): replacing systemctl instead of adding new one to path * enhancement(6394): update real systemctl path in mock systemctl script * enhancement(6394): fix linting errors * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated print statement * enhancement(6394): remove unnecessary command * enhancement(6394): use addressFromPath and SetClient * enhancement(6394): using service name, fixed indentation * test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging * Revert "test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging" This reverts commit 390c561. * enhancement(6394): renamed SetClient to SetDebRpmClient. Using hardcoded working dir as fixture working dir does not work for determining socket path * enhancement(6394): consolidated same version upgrade and regular upgrdade test functions * enhancement(6394): simplify preinstall script and enhance upgrade tests for tamper protection - Removed unnecessary endpoint handling logic from preinstall script. - Improved checks for service installation and status before upgrade. - Updated upgrade test functions to handle stopping the endpoint service before upgrades. * enhancement(6394): remove mock systemctl script for tamper protection tests * enhancement(6394): remove unused import * enhancement(6394): fixed order of execution in preinstall * enhancement(6394): added tests to make sure deb/rpm upgrades work when endpoint is not tamper protected --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> (cherry picked from commit 8a6531f)

…tion (#6907) (#8609) * Update pkg/testing/tools/tools.go * enhancement(6394): updated preinstall script, updated service to use uninstall token * enhancmenet(6394): updated the preinstall script * enchancement(6394): started adding integraiton tests * enhancement(6394): updated fixture install, updated endpoint security tests * enhancement(6394): cleaned up fixture_install, added function that exposes fixture's uninstall tokens, updated tests * enhancement(6394): refactored test code so that I can use it with rpm * enhancement(6394): added tests to assert that tamper protection works * enhancement(6394): updated the endpoint testing tools, fixture install functions and the deb rpm upgrade tests * enhancement(6394): added test logs, updated rpm installation to set agent socket path * enhancement(6394): remove commented code * enhancement(6394): remove print statements * enhancement(6394): remove unnecessary comments, refactor unused function * enhancement(6394): revert var name change * enhancement(6394): added changelog * enchancement(6394): update test logs, add non integrative config to deb installation * enhancement(6394): updated the endpoint version comparison and assertion * enhancement(6394): added log in tests * enhancement(6394): resorted to using previous major instead of minor in upgrade test * enhancement(6394): updated endpoint version function in the tests, updated function name in testing tools * enhancement(6394): use previous minor, fix log * enhancement(6394): added comment explaining motive behind simple install functions * enhancement(6394): updated return in tools * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml * enhancement(6394): fixed function call in tests * enhancement(6394): added systemctl start in postinstall, refactored preinstall and added condition to make same version installations work * enhancement(6394): updated the preinstall and postinstall scripts to troubleshoot * enhancement(6394): updated preinstall and postinstall script templates - Updated preinstall to stop endpoint if it is an available service regardless of the version of endpoint that's install - Updated postintall to start endpoint if the old endpoint version and the new version match. * enhancement(6394): removed error exit from postinstall * enhancement(6394): updated postinstall and preinstall templates - Preinstall now does not use a state file. Recovery from failure start ElasticEndpoint if it is not running - Preinstall does not stop endpoint if tamper protection is not enabled - Postinstall does not print an error if service is still running * enhancement(6394): removed debug logs * enhancement(6394): removed unnecessary comment * enhancement(6394): store uninstall token as local var, uninstall through the agent * enhancement(6394): added setclient function * enhancement(6394): added getInstallCommand and replaced SimpleInstall * enhancement(6394): added test case for error recovery. removed unused fixture functions * enhancement(6394): refactored tests, consolidated test scenarios into one function * enhancement(6394): remove unnecessary test functions * enhancement(6394): remove unused fixture function * enhancement(6394): revert unwanted installDeb changes * enhancement(6394): remove unwanted changes in testing tools * enhancement(6394): remove unused function call * enhancement(6394): replacing systemctl instead of adding new one to path * enhancement(6394): update real systemctl path in mock systemctl script * enhancement(6394): fix linting errors * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl * Update pkg/testing/tools/tools.go * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl * Update pkg/testing/tools/tools.go * enhancement(6394): updated print statement * enhancement(6394): remove unnecessary command * enhancement(6394): use addressFromPath and SetClient * enhancement(6394): using service name, fixed indentation * test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging * Revert "test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging" This reverts commit 390c561. * enhancement(6394): renamed SetClient to SetDebRpmClient. Using hardcoded working dir as fixture working dir does not work for determining socket path * enhancement(6394): consolidated same version upgrade and regular upgrdade test functions * enhancement(6394): simplify preinstall script and enhance upgrade tests for tamper protection - Removed unnecessary endpoint handling logic from preinstall script. - Improved checks for service installation and status before upgrade. - Updated upgrade test functions to handle stopping the endpoint service before upgrades. * enhancement(6394): remove mock systemctl script for tamper protection tests * enhancement(6394): remove unused import * enhancement(6394): fixed order of execution in preinstall * enhancement(6394): added tests to make sure deb/rpm upgrades work when endpoint is not tamper protected --------- (cherry picked from commit 8a6531f) Co-authored-by: Kaan Yalti <kaan.yalti@elastic.co> Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co>

cmacknz · 2025-06-23T17:56:51Z

Let's only put this in 9.1 and 8.19 since it is more like a new feature. We can add it to the maintenance branches once we confirm it has no introduced any problems.

I see the 9.0 backport has already merged, we should revert that. This is on the line of being an enhancement vs fixing something that should have worked already, and we lost the bet last time on that one so let's not risk anything again :)

…r protection (elastic#6907) (elastic#8609)" This reverts commit fc01978.

cmacknz · 2025-06-23T21:21:18Z

Revert for 9.0:

[9.0] Revert "Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection" #8638

…r protection (#6907) (#8609)" (#8638) This reverts commit fc01978.

…tion (#6907) * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated preinstall script, updated service to use uninstall token * enhancmenet(6394): updated the preinstall script * enchancement(6394): started adding integraiton tests * enhancement(6394): updated fixture install, updated endpoint security tests * enhancement(6394): cleaned up fixture_install, added function that exposes fixture's uninstall tokens, updated tests * enhancement(6394): refactored test code so that I can use it with rpm * enhancement(6394): added tests to assert that tamper protection works * enhancement(6394): updated the endpoint testing tools, fixture install functions and the deb rpm upgrade tests * enhancement(6394): added test logs, updated rpm installation to set agent socket path * enhancement(6394): remove commented code * enhancement(6394): remove print statements * enhancement(6394): remove unnecessary comments, refactor unused function * enhancement(6394): revert var name change * enhancement(6394): added changelog * enchancement(6394): update test logs, add non integrative config to deb installation * enhancement(6394): updated the endpoint version comparison and assertion * enhancement(6394): added log in tests * enhancement(6394): resorted to using previous major instead of minor in upgrade test * enhancement(6394): updated endpoint version function in the tests, updated function name in testing tools * enhancement(6394): use previous minor, fix log * enhancement(6394): added comment explaining motive behind simple install functions * enhancement(6394): updated return in tools * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> * enhancement(6394): fixed function call in tests * enhancement(6394): added systemctl start in postinstall, refactored preinstall and added condition to make same version installations work * enhancement(6394): updated the preinstall and postinstall scripts to troubleshoot * enhancement(6394): updated preinstall and postinstall script templates - Updated preinstall to stop endpoint if it is an available service regardless of the version of endpoint that's install - Updated postintall to start endpoint if the old endpoint version and the new version match. * enhancement(6394): removed error exit from postinstall * enhancement(6394): updated postinstall and preinstall templates - Preinstall now does not use a state file. Recovery from failure start ElasticEndpoint if it is not running - Preinstall does not stop endpoint if tamper protection is not enabled - Postinstall does not print an error if service is still running * enhancement(6394): removed debug logs * enhancement(6394): removed unnecessary comment * enhancement(6394): store uninstall token as local var, uninstall through the agent * enhancement(6394): added setclient function * enhancement(6394): added getInstallCommand and replaced SimpleInstall * enhancement(6394): added test case for error recovery. removed unused fixture functions * enhancement(6394): refactored tests, consolidated test scenarios into one function * enhancement(6394): remove unnecessary test functions * enhancement(6394): remove unused fixture function * enhancement(6394): revert unwanted installDeb changes * enhancement(6394): remove unwanted changes in testing tools * enhancement(6394): remove unused function call * enhancement(6394): replacing systemctl instead of adding new one to path * enhancement(6394): update real systemctl path in mock systemctl script * enhancement(6394): fix linting errors * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated print statement * enhancement(6394): remove unnecessary command * enhancement(6394): use addressFromPath and SetClient * enhancement(6394): using service name, fixed indentation * test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging * Revert "test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging" This reverts commit 390c561. * enhancement(6394): renamed SetClient to SetDebRpmClient. Using hardcoded working dir as fixture working dir does not work for determining socket path * enhancement(6394): consolidated same version upgrade and regular upgrdade test functions * enhancement(6394): simplify preinstall script and enhance upgrade tests for tamper protection - Removed unnecessary endpoint handling logic from preinstall script. - Improved checks for service installation and status before upgrade. - Updated upgrade test functions to handle stopping the endpoint service before upgrades. * enhancement(6394): remove mock systemctl script for tamper protection tests * enhancement(6394): remove unused import * enhancement(6394): fixed order of execution in preinstall * enhancement(6394): added tests to make sure deb/rpm upgrades work when endpoint is not tamper protected --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> (cherry picked from commit 8a6531f) # Conflicts: # dev-tools/packaging/templates/linux/preinstall.sh.tmpl # Conflicts: # dev-tools/packaging/templates/linux/postinstall.sh.tmpl # testing/integration/endpoint_security_test.go

…tion after elastic-agent (#8646) * [deb/rpm] restart endpoint with tamper protection after elastic-agent (#8637) * fix: use rpm from local build (cherry picked from commit 249885f) # Conflicts: # dev-tools/packaging/templates/linux/postinstall.sh.tmpl # testing/integration/endpoint_security_test.go * Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection (#6907) * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated preinstall script, updated service to use uninstall token * enhancmenet(6394): updated the preinstall script * enchancement(6394): started adding integraiton tests * enhancement(6394): updated fixture install, updated endpoint security tests * enhancement(6394): cleaned up fixture_install, added function that exposes fixture's uninstall tokens, updated tests * enhancement(6394): refactored test code so that I can use it with rpm * enhancement(6394): added tests to assert that tamper protection works * enhancement(6394): updated the endpoint testing tools, fixture install functions and the deb rpm upgrade tests * enhancement(6394): added test logs, updated rpm installation to set agent socket path * enhancement(6394): remove commented code * enhancement(6394): remove print statements * enhancement(6394): remove unnecessary comments, refactor unused function * enhancement(6394): revert var name change * enhancement(6394): added changelog * enchancement(6394): update test logs, add non integrative config to deb installation * enhancement(6394): updated the endpoint version comparison and assertion * enhancement(6394): added log in tests * enhancement(6394): resorted to using previous major instead of minor in upgrade test * enhancement(6394): updated endpoint version function in the tests, updated function name in testing tools * enhancement(6394): use previous minor, fix log * enhancement(6394): added comment explaining motive behind simple install functions * enhancement(6394): updated return in tools * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> * enhancement(6394): fixed function call in tests * enhancement(6394): added systemctl start in postinstall, refactored preinstall and added condition to make same version installations work * enhancement(6394): updated the preinstall and postinstall scripts to troubleshoot * enhancement(6394): updated preinstall and postinstall script templates - Updated preinstall to stop endpoint if it is an available service regardless of the version of endpoint that's install - Updated postintall to start endpoint if the old endpoint version and the new version match. * enhancement(6394): removed error exit from postinstall * enhancement(6394): updated postinstall and preinstall templates - Preinstall now does not use a state file. Recovery from failure start ElasticEndpoint if it is not running - Preinstall does not stop endpoint if tamper protection is not enabled - Postinstall does not print an error if service is still running * enhancement(6394): removed debug logs * enhancement(6394): removed unnecessary comment * enhancement(6394): store uninstall token as local var, uninstall through the agent * enhancement(6394): added setclient function * enhancement(6394): added getInstallCommand and replaced SimpleInstall * enhancement(6394): added test case for error recovery. removed unused fixture functions * enhancement(6394): refactored tests, consolidated test scenarios into one function * enhancement(6394): remove unnecessary test functions * enhancement(6394): remove unused fixture function * enhancement(6394): revert unwanted installDeb changes * enhancement(6394): remove unwanted changes in testing tools * enhancement(6394): remove unused function call * enhancement(6394): replacing systemctl instead of adding new one to path * enhancement(6394): update real systemctl path in mock systemctl script * enhancement(6394): fix linting errors * Update changelog/fragments/1740166208-allow-deb-rpm-upgrade-with-tamper-protected-endpoint.yaml Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update dev-tools/packaging/templates/linux/postinstall.sh.tmpl Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/tools.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * enhancement(6394): updated print statement * enhancement(6394): remove unnecessary command * enhancement(6394): use addressFromPath and SetClient * enhancement(6394): using service name, fixed indentation * test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging * Revert "test(debug): add detailed logging to Fixture.SetClient and installDeb for agent client setup debugging" This reverts commit 390c561. * enhancement(6394): renamed SetClient to SetDebRpmClient. Using hardcoded working dir as fixture working dir does not work for determining socket path * enhancement(6394): consolidated same version upgrade and regular upgrdade test functions * enhancement(6394): simplify preinstall script and enhance upgrade tests for tamper protection - Removed unnecessary endpoint handling logic from preinstall script. - Improved checks for service installation and status before upgrade. - Updated upgrade test functions to handle stopping the endpoint service before upgrades. * enhancement(6394): remove mock systemctl script for tamper protection tests * enhancement(6394): remove unused import * enhancement(6394): fixed order of execution in preinstall * enhancement(6394): added tests to make sure deb/rpm upgrades work when endpoint is not tamper protected --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> (cherry picked from commit 8a6531f) # Conflicts: # dev-tools/packaging/templates/linux/preinstall.sh.tmpl # Conflicts: # dev-tools/packaging/templates/linux/postinstall.sh.tmpl # testing/integration/endpoint_security_test.go * fix: resolve conflicts * fix: use --force-confold for deb tests in TestUpgradeAgentWithTamperProtectedEndpoint_DEB --------- Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co> Co-authored-by: Kaan Yalti <kaan.yalti@elastic.co>

…-hosted * feature/hosted-stack-using-oblt-cli: (26 commits) Use the current official docker image for oblt-cli Mark the elasticinframetrics processor as deprecated and schedule for removal (#8659) [main][Automation] Update versions (#8668) chore: Update create_deployment_csp_configuration.yaml (#8669) Attempt to make test more reliable by querying ES directly (#8422) [test] split up ess and beats serverless integration tests (#8551) Remove resource/k8s processor and use k8sattributes processor for service attributes (#8599) fix: use --force-confold for deb tests in TestUpgradeAgentWithTamperProtectedEndpoint_DEB (#8649) [main][Automation] Bump stack images versions to 9.1.0-ea0b7542 (#8612) chore: Update to elastic/beats@f6594fb72670 (#8640) [deb/rpm] restart endpoint with tamper protection after elastic-agent (#8637) ci: don't preinstall fleet packages on retried CI steps (#8636) chore: Update to elastic/beats@6b6941eed496 (#8619) [main][Automation] Bump VM Image version to 1750467641 (#8617) flaky: skip TestUpgradeAgentWithTamperProtectedEndpoint_RPM (#8626) Add skip-changelog PR label for bump VM PRs (#8627) build(deps): bump github.com/elastic/go-seccomp-bpf from 1.5.0 to 1.6.0 (#8611) [ci] fix k8s integration tests flakiness (#8575) bump apmconfig Otel extension to v0.3.0 (#8600) Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection (#6907) ...

mergify bot assigned kaanyalti Feb 18, 2025

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from e77e9d2 to 441a1a4 Compare February 21, 2025 19:13

kaanyalti marked this pull request as ready for review February 21, 2025 19:15

kaanyalti requested a review from a team as a code owner February 21, 2025 19:15

kaanyalti requested review from michalpristas and pchila February 21, 2025 19:15

pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Feb 23, 2025

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from 84cbc5e to a0f32ea Compare February 24, 2025 22:27

pchila reviewed Feb 26, 2025

View reviewed changes

pkg/testing/fixture_install.go Outdated Show resolved Hide resolved

pkg/testing/fixture_install.go Outdated Show resolved Hide resolved

pkg/testing/tools/tools.go Outdated Show resolved Hide resolved

testing/integration/endpoint_security_test.go Outdated Show resolved Hide resolved

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from c9d1864 to b0930f4 Compare February 27, 2025 00:37

kaanyalti requested a review from pchila February 27, 2025 00:38

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from 090d610 to b47d390 Compare February 28, 2025 15:17

pchila reviewed Feb 28, 2025

View reviewed changes

pkg/testing/tools/tools.go Outdated Show resolved Hide resolved

pkg/testing/tools/tools.go Outdated Show resolved Hide resolved

kaanyalti requested a review from pchila February 28, 2025 18:24

pchila reviewed Mar 3, 2025

View reviewed changes

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from 1c68177 to 2e0d09f Compare March 3, 2025 18:59

kaanyalti requested a review from a team as a code owner March 3, 2025 18:59

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from 2e0d09f to 33fe38d Compare March 3, 2025 19:04

kaanyalti requested a review from pchila March 3, 2025 20:34

cmacknz reviewed Mar 3, 2025

View reviewed changes

kaanyalti marked this pull request as draft April 3, 2025 15:14

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from 297f833 to ba0a3fd Compare April 28, 2025 19:31

kaanyalti force-pushed the enhancement/6394_allow_deb_rpm_to_upgrade_with_endpoint_tamper_protection branch from 5729e4e to 8f33880 Compare May 6, 2025 15:42

enhancement(6394): fixed order of execution in preinstall

5fca587

kaanyalti dismissed pchila’s stale review via 5fca587 June 19, 2025 16:24

kaanyalti added backport-9.0 Automated backport to the 9.0 branch backport-8.18 Automated backport to the 8.18 branch labels Jun 19, 2025

enhancement(6394): added tests to make sure deb/rpm upgrades work whe…

a496935

…n endpoint is not tamper protected

kaanyalti requested a review from pchila June 19, 2025 17:41

cmacknz approved these changes Jun 19, 2025

View reviewed changes

pchila approved these changes Jun 20, 2025

View reviewed changes

kaanyalti merged commit 8a6531f into elastic:main Jun 20, 2025
19 checks passed

mergify bot mentioned this pull request Jun 20, 2025

[8.18](backport #6907) Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection #8607

Closed

6 tasks

This was referenced Jun 20, 2025

[8.19] (backport #6907) Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection #8608

Closed

[9.0](backport #6907) Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection #8609

Merged

cmacknz removed the backport-8.18 Automated backport to the 8.18 branch label Jun 23, 2025

cmacknz mentioned this pull request Jun 23, 2025

[Flaky Test]: TestUpgradeAgentWithTamperProtectedEndpoint_RPM – Condition never satisfied #8613

Closed

cmacknz added a commit to cmacknz/elastic-agent that referenced this pull request Jun 23, 2025

Revert "Enhancement/6394 allow deb rpm to upgrade with endpoint tampe…

28c76c1

…r protection (elastic#6907) (elastic#8609)" This reverts commit fc01978.

cmacknz mentioned this pull request Jun 23, 2025

[9.0] Revert "Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection" #8638

Merged

cmacknz added a commit that referenced this pull request Jun 23, 2025

Revert "Enhancement/6394 allow deb rpm to upgrade with endpoint tampe…

357d4a8

…r protection (#6907) (#8609)" (#8638) This reverts commit fc01978.

pkoutsovasilis mentioned this pull request Jun 24, 2025

[deb/rpm] restart endpoint with tamper protection after elastic-agent #8637

Merged

8 tasks

mergify bot mentioned this pull request Jun 24, 2025

[8.19] (backport #8637) [deb/rpm] restart endpoint with tamper protection after elastic-agent #8646

Merged

8 tasks

Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection #6907

Enhancement/6394 allow deb rpm to upgrade with endpoint tamper protection #6907

Uh oh!

Conversation

kaanyalti commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Disruptive User Impact

How to test this PR locally

Related issues

Uh oh!

mergify bot commented Feb 18, 2025

Uh oh!

elasticmachine commented Feb 23, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pchila left a comment

Choose a reason for hiding this comment

Uh oh!

kaanyalti commented Mar 3, 2025

Uh oh!

elastic-sonarqube bot commented Mar 3, 2025

Quality Gate passed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Mar 13, 2025

Uh oh!

pchila commented Mar 26, 2025

Uh oh!

mergify bot commented May 6, 2025

Uh oh!

elastic-sonarqube bot commented Jun 19, 2025

Quality Gate passed

Uh oh!

elasticmachine commented Jun 19, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

History

Uh oh!

Uh oh!

cmacknz commented Jun 23, 2025

Uh oh!

cmacknz commented Jun 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kaanyalti commented Feb 18, 2025 •

edited

Loading