Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.12] Discover: Add handling for source column (#91815), enable fields API by default, Fix context view for date_nanos format with custom timestamps (#54089) #91981

Merged
merged 7 commits into from
Feb 19, 2021

Conversation

flash1293
Copy link
Contributor

@flash1293 flash1293 commented Feb 19, 2021

Backports the following commits to 7.12:

@flash1293 flash1293 enabled auto-merge (squash) February 19, 2021 08:38
@flash1293
Copy link
Contributor Author

This PR failed because fields API isn't default yet on 7.12 - I'm going to change that on a separate PR, then update this one

@flash1293 flash1293 merged commit 0b37d8c into elastic:7.12 Feb 19, 2021
@flash1293 flash1293 changed the title [7.12] Discover: Add handling for source column (#91815) [7.12] Discover: Add handling for source column (#91815), enable fields API by default, Fix context view for date_nanos format with custom timestamps (#54089) Feb 22, 2021
@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / APM API Integration tests (trial).x-pack/test/apm_api_integration/tests/correlations/slow_transactions·ts.APM API tests trial apm_8.0.0 with data and default args returns significant terms

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 4 times on tracked branches: https://github.com/elastic/kibana/issues/91673

[00:00:00]       │
[00:00:00]         │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] moving index [.ds-ilm-history-5-2021.02.22-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [ilm-history-ilm-policy]
[00:00:00]         └-: APM API tests
[00:00:00]           └-> "before all" hook in "APM API tests"
[00:00:00]           └-: trial
[00:00:00]             └-> "before all" hook in "trial"
[00:00:30]             └-: apm_8.0.0
[00:00:30]               └-> "before all" hook in "apm_8.0.0"
[00:00:30]               └-> "before all" hook: runBefore in "apm_8.0.0"
[00:00:30]                 │ info (0ms) Loading apm_8.0.0
[00:00:30]                 │ info [apm_8.0.0] Loading "mappings.json"
[00:00:30]                 │ info [apm_8.0.0] Loading "data.json.gz"
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-production-4da8-high_mean_transaction_duration] running delete by query on [.ml-anomalies-shared]
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-staging-56d9-high_mean_transaction_duration] running delete by query on [.ml-anomalies-shared]
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Job [apm-staging-56d9-high_mean_transaction_duration] deleted
[00:00:30]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001/80NIJcOZQUWlLvCWZq2_tQ] deleting index
[00:00:30]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_task_manager_7.12.0_001/FVIESV0VROq_0r-4z_rbvw] deleting index
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Job [apm-production-4da8-high_mean_transaction_duration] deleted
[00:00:30]                 │ info [apm_8.0.0] Deleted existing index ".kibana_task_manager_7.12.0_001"
[00:00:30]                 │ info [apm_8.0.0] Deleted existing index ".kibana_7.12.0_001"
[00:00:30]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.tasks] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]
[00:00:30]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.tasks]
[00:00:30]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:00:30]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_1]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_1][0]]]).
[00:00:31]                 │ info [apm_8.0.0] Created index ".kibana_1"
[00:00:31]                 │ debg [apm_8.0.0] ".kibana_1" settings {"index":{"auto_expand_replicas":"0-1","number_of_replicas":"1","number_of_shards":"1"}}
[00:00:31]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-anomalies-shared/Omv3VZ36RdW2LfsoPUs8kQ] deleting index
[00:00:31]                 │ info [apm_8.0.0] Deleted existing index ".ml-anomalies-shared"
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-anomalies-shared] creating index, cause [api], templates [.ml-anomalies-], shards [1]/[1]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.ml-anomalies-shared]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ml-anomalies-shared][0]]]).
[00:00:31]                 │ info [apm_8.0.0] Created index ".ml-anomalies-shared"
[00:00:31]                 │ debg [apm_8.0.0] ".ml-anomalies-shared" settings {"index":{"auto_expand_replicas":"0-1","hidden":"true","number_of_replicas":"1","number_of_shards":"1","translog":{"durability":"async"}}}
[00:00:31]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-config/Vl6dXoiuTOea87urN2K_og] deleting index
[00:00:31]                 │ info [apm_8.0.0] Deleted existing index ".ml-config"
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-config] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.ml-config]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ml-config][0]]]).
[00:00:31]                 │ info [apm_8.0.0] Created index ".ml-config"
[00:00:31]                 │ debg [apm_8.0.0] ".ml-config" settings {"index":{"auto_expand_replicas":"0-1","max_result_window":"10000","number_of_replicas":"1","number_of_shards":"1"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-error-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-error-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-error-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-error"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-metric-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-metric-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-metric-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-metric"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-span-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-span-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-span-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-span"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-transaction-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-transaction-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-transaction-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-transaction"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:40]                 │ info [apm_8.0.0] Indexed 20 docs into ".kibana_1"
[00:00:40]                 │ info [apm_8.0.0] Indexed 114 docs into ".ml-anomalies-shared"
[00:00:40]                 │ info [apm_8.0.0] Indexed 4 docs into ".ml-config"
[00:00:40]                 │ info [apm_8.0.0] Indexed 207 docs into "apm-8.0.0-error-2020.12.03-000001"
[00:00:40]                 │ info [apm_8.0.0] Indexed 14970 docs into "apm-8.0.0-metric-2020.12.03-000001"
[00:00:40]                 │ info [apm_8.0.0] Indexed 12781 docs into "apm-8.0.0-span-2020.12.03-000001"
[00:00:40]                 │ info [apm_8.0.0] Indexed 4786 docs into "apm-8.0.0-transaction-2020.12.03-000001"
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_1/m8sE-aPNSoiwVrQpc2hIfA] update_mapping [_doc]
[00:00:40]                 │ debg Migrating saved objects
[00:00:40]                 │ proc [kibana]   log   [10:11:55.829] [info][savedobjects-service] [.kibana_task_manager] INIT -> CREATE_NEW_TARGET
[00:00:40]                 │ proc [kibana]   log   [10:11:55.837] [info][savedobjects-service] [.kibana] INIT -> SET_SOURCE_WRITE_BLOCK
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_task_manager_7.12.0_001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:40]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_task_manager_7.12.0_001]
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] adding block write to indices [[.kibana_1/m8sE-aPNSoiwVrQpc2hIfA]]
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] completed adding block write to indices [.kibana_1]
[00:00:40]                 │ proc [kibana]   log   [10:11:55.905] [info][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> MARK_VERSION_INDEX_READY
[00:00:40]                 │ proc [kibana]   log   [10:11:55.922] [info][savedobjects-service] [.kibana] SET_SOURCE_WRITE_BLOCK -> CREATE_REINDEX_TEMP
[00:00:40]                 │ proc [kibana]   log   [10:11:55.939] [info][savedobjects-service] [.kibana_task_manager] MARK_VERSION_INDEX_READY -> DONE
[00:00:40]                 │ proc [kibana]   log   [10:11:55.940] [info][savedobjects-service] [.kibana_task_manager] Migration completed after 113ms
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_reindex_temp] creating index, cause [api], templates [], shards [1]/[1]
[00:00:40]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_7.12.0_reindex_temp]
[00:00:40]                 │ proc [kibana]   log   [10:11:55.990] [info][savedobjects-service] [.kibana] CREATE_REINDEX_TEMP -> REINDEX_SOURCE_TO_TEMP
[00:00:40]                 │ proc [kibana]   log   [10:11:56.004] [info][savedobjects-service] [.kibana] REINDEX_SOURCE_TO_TEMP -> REINDEX_SOURCE_TO_TEMP_WAIT_FOR_TASK
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_reindex_temp/B2CwJYQpQ7WjMNBmMSFm-g] update_mapping [_doc]
[00:00:40]                 │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] 5084 finished with response BulkByScrollResponse[took=39.6ms,timed_out=false,sliceId=null,updated=0,created=20,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.119] [info][savedobjects-service] [.kibana] REINDEX_SOURCE_TO_TEMP_WAIT_FOR_TASK -> SET_TEMP_WRITE_BLOCK
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] adding block write to indices [[.kibana_7.12.0_reindex_temp/B2CwJYQpQ7WjMNBmMSFm-g]]
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] completed adding block write to indices [.kibana_7.12.0_reindex_temp]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.155] [info][savedobjects-service] [.kibana] SET_TEMP_WRITE_BLOCK -> CLONE_TEMP_TO_TARGET
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] applying create index request using existing index [.kibana_7.12.0_reindex_temp] metadata
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001] creating index, cause [clone_index], templates [], shards [1]/[1]
[00:00:40]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_7.12.0_001]
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001/tCfL7v7VQb2iDmJJEUQV1g] create_mapping [_doc]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.268] [info][savedobjects-service] [.kibana] CLONE_TEMP_TO_TARGET -> OUTDATED_DOCUMENTS_SEARCH
[00:00:40]                 │ proc [kibana]   log   [10:11:56.278] [info][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH -> UPDATE_TARGET_MAPPINGS
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001/tCfL7v7VQb2iDmJJEUQV1g] update_mapping [_doc]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.335] [info][savedobjects-service] [.kibana] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK
[00:00:40]                 │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] 5133 finished with response BulkByScrollResponse[took=19.6ms,timed_out=false,sliceId=null,updated=20,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.442] [info][savedobjects-service] [.kibana] UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK -> MARK_VERSION_INDEX_READY
[00:00:40]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_reindex_temp/B2CwJYQpQ7WjMNBmMSFm-g] deleting index
[00:00:40]                 │ proc [kibana]   log   [10:11:56.484] [info][savedobjects-service] [.kibana] MARK_VERSION_INDEX_READY -> DONE
[00:00:40]                 │ proc [kibana]   log   [10:11:56.484] [info][savedobjects-service] [.kibana] Migration completed after 659ms
[00:00:40]                 │ debg [apm_8.0.0] Migrated Kibana index after loading Kibana data
[00:00:41]                 │ debg [apm_8.0.0] Ensured that default space exists in .kibana
[00:00:41]                 │ info (10217ms) Loaded all archives
[00:00:41]               └-: with data and default args
[00:00:41]                 └-> "before all" hook for "returns successfully"
[00:00:41]                 └-> "before all" hook for "returns successfully"
[00:00:41]                 └-> returns successfully
[00:00:41]                   └-> "before each" hook: global before each for "returns successfully"
[00:00:41]                   └- ✓ pass  (1ms) "APM API tests trial apm_8.0.0 with data and default args returns successfully"
[00:00:41]                 └-> returns significant terms
[00:00:41]                   └-> "before each" hook: global before each for "returns significant terms"
[00:00:41]                   └- ✖ fail: APM API tests trial apm_8.0.0 with data and default args returns significant terms
[00:00:41]                   │      Error: expect(received).toMatchInlineSnapshot(snapshot)
[00:00:41]                   │ 
[00:00:41]                   │ Snapshot name: `APM API tests trial apm_8.0.0 with data and default args returns significant terms 1`
[00:00:41]                   │ 
[00:00:41]                   │ - Snapshot  - 1
[00:00:41]                   │ + Received  + 1
[00:00:41]                   │ 
[00:00:41]                   │ @@ -1,11 +1,11 @@
[00:00:41]                   │   Array [
[00:00:41]                   │     "user_agent.name",
[00:00:41]                   │ -   "url.domain",
[00:00:41]                   │     "host.ip",
[00:00:41]                   │     "service.node.name",
[00:00:41]                   │     "container.id",
[00:00:41]                   │ +   "url.domain",
[00:00:41]                   │     "url.domain",
[00:00:41]                   │     "host.ip",
[00:00:41]                   │     "service.node.name",
[00:00:41]                   │     "container.id",
[00:00:41]                   │     "user_agent.name",
[00:00:41]                   │       at toMatchInline (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/snapshots/decorate_snapshot_ui.ts:226:11)
[00:00:41]                   │       at Context.apply (test/apm_api_integration/tests/correlations/slow_transactions.ts:57:61)
[00:00:41]                   │       at Object.apply (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:30)
[00:00:41]                   │ 
[00:00:41]                   │ 

Stack Trace

Error: expect(received).toMatchInlineSnapshot(snapshot)

Snapshot name: `APM API tests trial apm_8.0.0 with data and default args returns significant terms 1`

- Snapshot  - 1
+ Received  + 1

@@ -1,11 +1,11 @@
  Array [
    "user_agent.name",
-   "url.domain",
    "host.ip",
    "service.node.name",
    "container.id",
+   "url.domain",
    "url.domain",
    "host.ip",
    "service.node.name",
    "container.id",
    "user_agent.name",
    at toMatchInline (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/snapshots/decorate_snapshot_ui.ts:226:11)
    at Context.apply (test/apm_api_integration/tests/correlations/slow_transactions.ts:57:61)
    at Object.apply (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:30)

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
discover 348 349 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
discover 408.7KB 409.5KB +745.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants