[APM] Migrate apm alerts tests to deployment agnostic #199097

crespocarlos · 2024-11-06T09:52:29Z

Summary

Closes #198959
Part of #193245

This PR contains the changes to migrate alerts test folder to Deployment-agnostic testing strategy.
It also fixes a bug when filtering alerts by kibana.alert.rule.producer. On serverless ,the producer is observability, not apm

How to test

Serverless

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"

It's recommended to be run against MKI - This PR won't pass on MKI while the instance is not created using the kibana.alert.rule.producer fix from this PR

Stateful

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"

Migrate agent_explorer test Update latest_agent_versions test

…fix'

crespocarlos · 2024-11-07T16:31:05Z

/ci

x-pack/test/apm_api_integration/tests/alerts/anomaly_alert.spec.ts

kibanamachine · 2024-11-07T17:00:52Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7348

[✅] x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts: 25/25 tests passed.

see run history

crespocarlos · 2024-11-07T17:14:46Z

/ci

crespocarlos · 2024-11-08T09:17:02Z

/ci

crespocarlos · 2024-11-08T10:48:29Z

/ci

elasticmachine · 2024-11-08T11:31:03Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

wayneseymour · 2024-11-08T11:58:45Z

👀

crespocarlos · 2024-11-08T13:03:48Z

@elasticmachine merge upstream

wayneseymour

Also, I would love to humbly request any codeowner updates for files added in this PR.
There is an on-going effort to have zero test files without an owner, in order to soon have much more granular reporting for teams.

..._integration/deployment_agnostic/apis/observability/apm/alerts/error_count_threshold.spec.ts

x-pack/test/api_integration/deployment_agnostic/services/alerting_api.ts

kpatticha

LGTM

wayneseymour

Looking better Carlos.
Just a few more mentions of supertest.
Again, in order to better protect our releases and for other teams to assist in test failure debugging, the ability to quickly distinguish which privileges are in play is crucial.

.github/CODEOWNERS

..._integration/deployment_agnostic/apis/observability/apm/alerts/error_count_threshold.spec.ts

...i_integration/deployment_agnostic/apis/observability/apm/alerts/transaction_duration.spec.ts

...integration/deployment_agnostic/apis/observability/apm/alerts/transaction_error_rate.spec.ts

…apm-alerts-tests

MiriamAparicio

LGTM

wayneseymour

We're almost there.

..._integration/deployment_agnostic/apis/observability/apm/alerts/error_count_threshold.spec.ts

wayneseymour · 2024-11-11T16:58:16Z

...i_integration/deployment_agnostic/apis/observability/apm/alerts/transaction_duration.spec.ts

-    before(() => {
+  describe('transaction duration alert', () => {
+    let apmSynthtraceEsClient: ApmSynthtraceEsClient;
+    let supertestViewerWithCookieCredentials: SupertestWithRoleScopeType;


elasticmachine · 2024-11-11T17:29:31Z

💚 Build Succeeded

Buildkite Build
Commit: 3a56a72
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-199097-3a56a72fc4d7

Metrics [docs]

✅ unchanged

History

💔 Build #249982 failed 751ec82
💚 Build #249955 succeeded 5d475f1
💚 Build #249926 succeeded 18e8def
💚 Build #249844 succeeded 0cf7785
💚 Build #249716 succeeded 14dc86c

wayneseymour

CR only, LGTM

kibanamachine · 2024-11-12T11:01:42Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11795906207

kibanamachine · 2024-11-12T11:06:16Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - [APM] Migrate `/data_view` to deployment agnostic test (#199296) - [APM] Migrate `/mobile` API tests to deployment agnostic folder (#199021)

Manual backport

To create the backport manually run:

node scripts/backport --pr 199097

Questions ?

Please refer to the Backport tool documentation

## Summary Closes [elastic#198959](elastic#198959) Part of elastic#193245 This PR contains the changes to migrate `alerts` test folder to Deployment-agnostic testing strategy. It also fixes a bug when filtering alerts by `kibana.alert.rule.producer`. On serverless ,the producer is `observability`, not `apm` ### How to test - Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM" ``` It's recommended to be run against [MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki) - This PR won't pass on MKI while the instance is not created using the `kibana.alert.rule.producer` fix from this PR - Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM" ``` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit fb666aa) # Conflicts: # .github/CODEOWNERS # x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts

## Summary Closes [elastic#198959](elastic#198959) Part of elastic#193245 This PR contains the changes to migrate `alerts` test folder to Deployment-agnostic testing strategy. It also fixes a bug when filtering alerts by `kibana.alert.rule.producer`. On serverless ,the producer is `observability`, not `apm` ### How to test - Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM" ``` It's recommended to be run against [MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki) - This PR won't pass on MKI while the instance is not created using the `kibana.alert.rule.producer` fix from this PR - Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM" ``` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

crespocarlos · 2024-11-12T12:32:51Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…#199781) # Backport This will backport the following commits from `main` to `8.x`: - [[APM] Migrate apm alerts tests to deployment agnostic (#199097)](#199097)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

## Summary Closes [elastic#198959](elastic#198959) Part of elastic#193245 This PR contains the changes to migrate `alerts` test folder to Deployment-agnostic testing strategy. It also fixes a bug when filtering alerts by `kibana.alert.rule.producer`. On serverless ,the producer is `observability`, not `apm` ### How to test - Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM" ``` It's recommended to be run against [MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki) - This PR won't pass on MKI while the instance is not created using the `kibana.alert.rule.producer` fix from this PR - Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM" ``` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

crespocarlos force-pushed the 198959-migrate-apm-alerts-tests branch from cb201c6 to f75d220 Compare November 6, 2024 09:55

crespocarlos and others added 3 commits November 7, 2024 10:57

Add foundation to run apm tests

9ad201e

Migrate agent_explorer test Update latest_agent_versions test

Force Git to recognize latest_agent_versions.spec.ts as moved

78c8243

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

2f348eb

…fix'

rmyz mentioned this pull request Nov 7, 2024

[APM] Migrate /anomalies to deployment agnostic tests #199269

Closed

Remove registry service

e1bb126

crespocarlos force-pushed the 198959-migrate-apm-alerts-tests branch 3 times, most recently from e6fb4a4 to 53a9373 Compare November 7, 2024 16:19

crespocarlos changed the title ~~198959 migrate apm alerts tests~~ [APM Migrate apm alerts tests to deployment agnostic Nov 7, 2024

crespocarlos added release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Nov 7, 2024

crespocarlos commented Nov 7, 2024

View reviewed changes

x-pack/test/apm_api_integration/tests/alerts/anomaly_alert.spec.ts Show resolved Hide resolved

crespocarlos force-pushed the 198959-migrate-apm-alerts-tests branch from 53a9373 to 1ef84ef Compare November 7, 2024 17:14

crespocarlos changed the title ~~[APM Migrate apm alerts tests to deployment agnostic~~ [APM] Migrate apm alerts tests to deployment agnostic Nov 7, 2024

crespocarlos force-pushed the 198959-migrate-apm-alerts-tests branch from 1ef84ef to 1f8c541 Compare November 8, 2024 09:13

Migrate APM alerts test

670ee82

crespocarlos force-pushed the 198959-migrate-apm-alerts-tests branch from 1f8c541 to 670ee82 Compare November 8, 2024 10:48

crespocarlos marked this pull request as ready for review November 8, 2024 11:31

crespocarlos requested review from a team as code owners November 8, 2024 11:31

wayneseymour mentioned this pull request Nov 11, 2024

[FTR][Tech Debt] Too many alerting api helpers, can they be reduced? #199619

Open

wayneseymour requested changes Nov 11, 2024

View reviewed changes

..._integration/deployment_agnostic/apis/observability/apm/alerts/error_count_threshold.spec.ts Outdated Show resolved Hide resolved

wayneseymour reviewed Nov 11, 2024

View reviewed changes

x-pack/test/api_integration/deployment_agnostic/services/alerting_api.ts Show resolved Hide resolved

kpatticha approved these changes Nov 11, 2024

View reviewed changes

CODEOWNERs update

18e8def

crespocarlos requested a review from wayneseymour November 11, 2024 12:28

wayneseymour requested changes Nov 11, 2024

View reviewed changes

Rename supertest

5d475f1

crespocarlos requested a review from wayneseymour November 11, 2024 13:42

crespocarlos added 2 commits November 11, 2024 15:47

Merge branch 'main' of github.com:elastic/kibana into 198959-migrate-…

751ec82

…apm-alerts-tests

revert role change

3a56a72

MiriamAparicio approved these changes Nov 11, 2024

View reviewed changes

wayneseymour requested changes Nov 11, 2024

View reviewed changes

crespocarlos requested a review from wayneseymour November 11, 2024 18:50

crespocarlos enabled auto-merge (squash) November 11, 2024 18:52

wayneseymour approved these changes Nov 12, 2024

View reviewed changes

crespocarlos merged commit fb666aa into elastic:main Nov 12, 2024
24 checks passed

kibanamachine added the v9.0.0 label Nov 12, 2024

crespocarlos mentioned this pull request Nov 12, 2024

[8.x] [APM] Migrate apm alerts tests to deployment agnostic (#199097) #199781

Merged

kibanamachine mentioned this pull request Nov 12, 2024

[APM] Migrate /environment to deployment agnostic test #199582

Merged

4 tasks

kibanamachine added the v8.17.0 label Nov 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APM] Migrate apm alerts tests to deployment agnostic #199097

[APM] Migrate apm alerts tests to deployment agnostic #199097

crespocarlos commented Nov 6, 2024 •

edited by kibanamachine

Loading

crespocarlos commented Nov 7, 2024

kibanamachine commented Nov 7, 2024

crespocarlos commented Nov 7, 2024

crespocarlos commented Nov 8, 2024

crespocarlos commented Nov 8, 2024

elasticmachine commented Nov 8, 2024

wayneseymour commented Nov 8, 2024

crespocarlos commented Nov 8, 2024

wayneseymour left a comment

kpatticha left a comment

wayneseymour left a comment

MiriamAparicio left a comment

wayneseymour left a comment

wayneseymour Nov 11, 2024

elasticmachine commented Nov 11, 2024 •

edited

Loading

wayneseymour left a comment

kibanamachine commented Nov 12, 2024

kibanamachine commented Nov 12, 2024

crespocarlos commented Nov 12, 2024

[APM] Migrate apm alerts tests to deployment agnostic #199097

[APM] Migrate apm alerts tests to deployment agnostic #199097

Conversation

crespocarlos commented Nov 6, 2024 • edited by kibanamachine Loading

Summary

How to test

crespocarlos commented Nov 7, 2024

kibanamachine commented Nov 7, 2024

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7348

crespocarlos commented Nov 7, 2024

crespocarlos commented Nov 8, 2024

crespocarlos commented Nov 8, 2024

elasticmachine commented Nov 8, 2024

wayneseymour commented Nov 8, 2024

crespocarlos commented Nov 8, 2024

wayneseymour left a comment

Choose a reason for hiding this comment

kpatticha left a comment

Choose a reason for hiding this comment

wayneseymour left a comment

Choose a reason for hiding this comment

MiriamAparicio left a comment

Choose a reason for hiding this comment

wayneseymour left a comment

Choose a reason for hiding this comment

wayneseymour Nov 11, 2024

Choose a reason for hiding this comment

elasticmachine commented Nov 11, 2024 • edited Loading

💚 Build Succeeded

Metrics [docs]

History

wayneseymour left a comment

Choose a reason for hiding this comment

kibanamachine commented Nov 12, 2024

kibanamachine commented Nov 12, 2024

💔 All backports failed

Manual backport

Questions ?

crespocarlos commented Nov 12, 2024

💚 All backports created successfully

Questions ?

crespocarlos commented Nov 6, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Nov 11, 2024 •

edited

Loading