Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/group2/tests/telemetry/index·ts - alerting api integration security and spaces enabled - Group 2 Alerting and Actions Telemetry "after all" hook in "Alerting and Actions Telemetry" #136153

Closed
kibanamachine opened this issue Jul 11, 2022 · 8 comments · Fixed by #159818 or #161096
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Jul 11, 2022

A test failed on a tracked branch

ResponseError: {"took":4,"timed_out":false,"total":2,"deleted":0,"batches":1,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[{"index":".kibana_task_manager_1","id":"task:Alerting-alerting_telemetry","cause":{"type":"cluster_block_exception","reason":"index [.kibana_task_manager_1] blocked by: [FORBIDDEN/8/index write (api)];"},"status":403},{"index":".kibana_task_manager_1","id":"task:Actions-actions_telemetry","cause":{"type":"cluster_block_exception","reason":"index [.kibana_task_manager_1] blocked by: [FORBIDDEN/8/index write (api)];"},"status":403}]}
    at SniffingTransport.request (node_modules/@elastic/transport/src/Transport.ts:532:17)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Client.DeleteByQueryApi [as deleteByQuery] (node_modules/@elastic/elasticsearch/src/api/api/delete_by_query.ts:71:10)
    at cleanKibanaIndices (node_modules/@kbn/es-archiver/target_node/lib/indices/kibana_index.js:104:18)
    at Transform.transform [as _transform] (node_modules/@kbn/es-archiver/target_node/lib/indices/delete_index_stream.js:41:13) {
  meta: {
    body: {
      took: 4,
      timed_out: false,
      total: 2,
      deleted: 0,
      batches: 1,
      version_conflicts: 0,
      noops: 0,
      retries: [Object],
      throttled_millis: 0,
      requests_per_second: -1,
      throttled_until_millis: 0,
      failures: [Array]
    },
    statusCode: 403,
    headers: {
      'x-elastic-product': 'Elasticsearch',
      'content-type': 'application/vnd.elasticsearch+json;compatible-with=8',
      'content-length': '648'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    },
    warnings: [Getter]
  }
}

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Jul 11, 2022
@botelastic botelastic bot added the needs-team Issues missing a team label label Jul 11, 2022
@kibanamachine kibanamachine added the Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) label Jul 11, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@mikecote
Copy link
Contributor

Closing as we believe it was a single flaky run.

Repository owner moved this from Awaiting Triage to Done in AppEx: ResponseOps - Execution & Connectors Jul 14, 2022
@kibanamachine kibanamachine reopened this Mar 31, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.8

@dmlemeshko
Copy link
Member

The last 2 failures have the same error:

Error: Expected status code of 200, received 500 Internal Server Error: {
statusCode: 500,
error: 'Internal Server Error',
message: '[Saved object index alias [.kibana_8.8.0] not found: index_not_found_exception\n' +
'\tRoot causes:\n' +
'\t\tindex_not_found_exception: no such index [.kibana_8.8.0] and [require_alias] request flag is [true] and [.kibana_8.8.0] is not an alias]: no such index [.kibana_8.8.0] and [require_alias] request flag is [true] and [.kibana_8.8.0] is not an alias'
}

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic
Copy link
Member

mistic commented May 8, 2023

Skipped.

main: f2b4de6
8.8: 17b34f6

@pmuellr
Copy link
Member

pmuellr commented Jun 26, 2023

All the failures on this, this year, have been 500 errors. Which seem more like transient environmental issues in CI, than with the code.

So, I'm going to approve the PR we have open to skip this.

Suggestion if we get more 500's later: move the ./telemetry test so it's not run first, but last, in the file that orders the tests. Perhaps there's something happening so close to startup, and running the other tests first would work around it.

export default function alertingApiIntegrationTests({ loadTestFile }: FtrProviderContext) {
describe('alerting api integration security and spaces enabled - Group 2', function () {
loadTestFile(require.resolve('./telemetry'));
loadTestFile(require.resolve('./actions'));
loadTestFile(require.resolve('./alerting'));
});

doakalexi added a commit that referenced this issue Jun 26, 2023
…on/security_and_spaces/group2/tests/telemetry/index·ts (#159818)

Resolves #136153

## Summary

Removes skip on the `Group 2 Alerting and Actions Telemetry "after all"
hook in "Alerting and Actions Telemetry"` test.

1. Flaky test run 250x -
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2386
2. Flaky test run 250x -
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2418
( a test failed but it's from `Group 2 Alerts alerts
getGlobalExecutionKpi should return KPI from all spaces in the
namespaces param` )
3. Flaky test run 250x -
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2424
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jun 26, 2023
…on/security_and_spaces/group2/tests/telemetry/index·ts (elastic#159818)

Resolves elastic#136153

## Summary

Removes skip on the `Group 2 Alerting and Actions Telemetry "after all"
hook in "Alerting and Actions Telemetry"` test.

1. Flaky test run 250x -
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2386
2. Flaky test run 250x -
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2418
( a test failed but it's from `Group 2 Alerts alerts
getGlobalExecutionKpi should return KPI from all spaces in the
namespaces param` )
3. Flaky test run 250x -
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2424

(cherry picked from commit e2a6a33)
@mikecote mikecote reopened this Jul 10, 2023
doakalexi added a commit that referenced this issue Jul 31, 2023
…on/security_and_spaces/group2/tests/telemetry/index·ts (#161096)

Resolves #140973
Resolves #136153

## Summary

Fixes a telemetry bug and a flaky telemetry test

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2717
x 250

https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2718
x 250
ThomThomson pushed a commit to ThomThomson/kibana that referenced this issue Aug 1, 2023
…on/security_and_spaces/group2/tests/telemetry/index·ts (elastic#161096)

Resolves elastic#140973
Resolves elastic#136153

## Summary

Fixes a telemetry bug and a flaky telemetry test

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2717
x 250

https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2718
x 250
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
7 participants