[Fleet] Fix latest behavior with bundled packages

[joshdover]

Kibana version: 8.1.0+

Steps to reproduce:

1. EPM has 8.0.0
2. Kibana has 8.1.0 bundled
3. Kibana installs 8.0.0, not 8.1.0
4. Testers can't test 8.1.0 installs before it's released have to upgrade manually, breaks UIs and tests

Good news, tests and testers caught something!

@joshdover: This is breaking behavior for 8.1.0 for testers and Cloud staging deployments using this latest feature in their config which is in some examples and documented, and we should fix it the right way if the effort is reasonable.

Once 8.1.0 package is released it will go away, so it may not affect production since we expect this package to be available in the registry by then. So this only ends up affecting tests right now before the release.

Related Issues:

• [Fleet]: Unable to toggle any switched in Elastic APM integration inside Elastic Cloud agent policy #126323

Possible solutions:

• Install from bundled if the bundled version >= latest on the registry
• [Change Proposal] Remove release tag from packages package-spec#225
  • Rough estimate?
• [APM] Broad version support for custom package policy editors in APM Fleet integration #123477

Questions:
For any solution:

• Is this doing this going to change the packages installed for a number of packages that would affect testing integrations for 8.1.0?

Workaround:
From #126487 (comment)
Workaround to force install a given version via the API, so long as a registry instance is accessible:

curl http://<kibana>/api/fleet/epm/packages/apm-8.1.0 \
  -u elastic:<password> \
  -XPOST \
  -H 'kbn-xsrf: true' \
  -H 'Content-Type: application/json' \
  -d'{"force": true}'

This API can be used to force install apm-8.1.0, and the Kibana UI can then be used to upgrade any related policies for testing.

The UI could also be used to install the latest version of APM from the integrations app, but the API approach is likely quicker to unblock, and could be automated.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[kpollich]

Workaround to force install a given version via the API, so long as a registry instance is accessible:

POST http://<kibana>/api/fleet/epm/packages/<name>-<version>
Authorization: Basic <username>:<password>
Content-Type: application/json

{
  "force": true
}

This API can be used to force install apm-8.1.0, and the Kibana UI can then be used to upgrade any related policies for testing.

The UI could also be used to install the latest version of APM from the integrations app, but the API approach is likely quicker to unblock, and could be automated.

[ghost]

Hi @joshdover,
We have revalidated this issue on the released 8.1.0 build and observed that the issue is fixed. Please find below the testing details:

• Elastic APM integration version is updated to v8.1.0

Screen-Recording:

Home.-.Elastic.-.Google.Chrome.2022-03-17.14-45-42.mp4

Build Details:
VERSION: 8.1.0
COMMIT: 4aaeda2
BUILD: 50485

Thanks!