Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data Discovery] Resolve flaky Serverless tests #166037

Closed
33 tasks done
davismcphee opened this issue Sep 8, 2023 · 2 comments
Closed
33 tasks done

[Data Discovery] Resolve flaky Serverless tests #166037

davismcphee opened this issue Sep 8, 2023 · 2 comments
Assignees
@davismcphee
Labels
impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Project:Serverless Work as part of the Serverless project for its initial release Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@davismcphee
Copy link
Contributor

davismcphee commented Sep 8, 2023 •
edited
Loading

This issue is to track the currently skipped Data Discovery Serverless tests and the progress toward unskipping them. We plan to work through each failure type and open PRs to fix and unskip them with minimum 100x flaky test runs to confirm they're stable.

Failure types

Unique
Beta Give feedback
  1. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/discover_customization_examples/customizations·ts - serverless examples UI Discover customization examples Customizations "before all" hook for "Top nav" #165396
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  2. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/existing_fields·ts - serverless examples UI Unified Field List Examples Fields existence info "after all" hook in "Fields existence info" #165938
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  3. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/field_stats·ts - serverless examples UI Unified Field List Examples Field stats "before all" hook in "Field stats" #165882
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  4. Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/search_oss/search·ts - serverless common API search search post should return 400 for a painless error #165879
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  5. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/field_stats·ts - serverless examples UI Unified Field List Examples Field stats histogram should return an auto histogram for precalculated histograms #165865
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  6. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/field_stats·ts - serverless examples UI Unified Field List Examples Field stats field distribution "before all" hook for "should return an auto histogram for numbers and top values" #165797
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  7. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search_examples/search_example·ts - serverless examples UI search examples Search example with bfetch should have an other bucket #165735
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  8. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch "before each" hook for "shows shard failure warning notifications by default" #165660
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  9. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch shows shard failure warning notifications by default #165635
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  10. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch shows shard failure warning notifications by default #165379
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee

[data-test-subj="kibanaChrome"]
Beta Give feedback
  1. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/discover_customization_examples/customizations·ts - serverless examples UI Discover customization examples Customizations "before all" hook for "Top nav" #165438
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  2. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/field_formats/index·ts - serverless examples UI Field formats example "before all" hook for "renders field formats example 1" #165537
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  3. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/field_formats/index·ts - serverless examples UI Field formats example "before all" hook for "renders field formats example 1" #165545
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  4. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/partial_results/index·ts - serverless examples UI Partial Results Example "before all" hook for "should trace mouse events" #165563
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  5. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/partial_results/index·ts - serverless examples UI Partial Results Example "before all" hook for "should trace mouse events" #165612
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  6. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search_examples/search_example·ts - serverless examples UI search examples Search example with bfetch "before all" hook for "should have an other bucket" #165730
    Team:DataDiscovery failed-test impact:high loe:small skipped-test
    davismcphee
  7. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search_examples/partial_results_example·ts - serverless examples UI search examples Partial results example "before all" hook for "should update a progress bar" #165763
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee
  8. Failing test: Serverless Observability Functional Tests.x-pack/test_serverless/functional/test_suites/common/data_view_mgmt·ts - serverless common UI Data View Management Scripted fields tab is missing #165796
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  9. Failing test: Serverless Security Functional Tests.x-pack/test_serverless/functional/test_suites/common/data_view_mgmt·ts - serverless common UI Data View Management Scripted fields tab is missing #165804
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  10. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/existing_fields·ts - serverless examples UI Unified Field List Examples Fields existence info "before all" hook in "Fields existence info" #165927
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  11. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/existing_fields·ts - serverless examples UI Unified Field List Examples Fields existence info "before all" hook in "Fields existence info" #165931
    Team:DataDiscovery failed-test impact:medium loe:small
    davismcphee

ResponseError: resource_already_exists_exception
Beta Give feedback
  1. Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/rollups·ts - serverless common API rollup data views - fields for wildcard "before all" hook for "returns 200 and best effort response despite lack of rollup support" #165476
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  2. Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/scripted_fields·ts - serverless common API scripted fields disabled "before all" hook for "scripted fields are ignored when disabled" #165511
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  3. Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_view_field_editor/field_preview·ts - serverless common API index pattern field editor Field preview "before all" hook in "Field preview" #165868
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  4. Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_views/fields_for_wildcard_route/params·ts - serverless common API index_patterns index_patterns/_fields_for_wildcard route params "before all" hook for "requires a pattern query param" #165942
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  5. Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_views/es_errors/errors·js - serverless common API index_patterns index_patterns/service/lib index_patterns/* error handler "before all" hook in "index_patterns/* error handler" #165944
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  6. Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_views/fields_for_wildcard_route/conflicts·ts - serverless common API index_patterns index_patterns/_fields_for_wildcard route conflicts "before all" hook for "flags fields with mismatched types as conflicting" #165972
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  7. Failing test: Serverless Observability Functional Tests.x-pack/test_serverless/functional/test_suites/common/data_view_mgmt·ts - serverless common UI Data View Management "before all" hook for "Scripted fields tab is missing" #165425
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee

TimeoutError: Request timed out
Beta Give feedback
  1. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/data_view_field_editor_example/index·ts - serverless examples UI data view field editor example "before all" hook in "data view field editor example" #165384
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  2. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch "before all" hook for "shows shard failure warning notifications by default" #165502
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  3. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch "before all" hook for "shows shard failure warning notifications by default" #165623
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee

ResponseError: index_not_found_exception
Beta Give feedback
  1. Failing test: Serverless Security Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch "after all" hook for "able to handle shard failure warnings and prevent default notifications" #165503
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
  2. Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search/warnings·ts - serverless examples UI Search examples handling warnings with search source fetch "after all" hook for "able to handle shard failure warnings and prevent default notifications" #165624
    Team:DataDiscovery failed-test impact:high loe:small
    davismcphee
@davismcphee davismcphee added loe:medium Medium Level of Effort impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Serverless labels Sep 8, 2023
@davismcphee davismcphee self-assigned this Sep 8, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

This was referenced Oct 4, 2023
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
@watson watson added Project:Serverless Work as part of the Serverless project for its initial release and removed Serverless labels Oct 4, 2023
@davismcphee davismcphee mentioned this issue Oct 4, 2023
Closed
@davismcphee
Copy link
Contributor Author

Closing this since all flaky tests in the issue have now been resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davismcphee davismcphee

Labels
impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Project:Serverless Work as part of the Serverless project for its initial release Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@watson @elasticmachine @davismcphee