Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[7.x] [Endpoint][EPM] Retrieve Index Pattern from Ingest Manager (#63016) #63767

Merged
merged 2 commits into from
Apr 17, 2020
Merged

[7.x] [Endpoint][EPM] Retrieve Index Pattern from Ingest Manager (#63016) #63767

merged 2 commits into from
Apr 17, 2020

Conversation

jonathan-buttner
Copy link
Contributor

Backports the following commits to 7.x:

@jonathan-buttner
[Endpoint][EPM] Retrieve Index Pattern from Ingest Manager (elastic#6…
e7f8482 
…3016)

* Endpoint successfully depending on ingest manager to initialize

* Moving the endpoint functional tests to their own directory to avoid enabling ingest in the base tests

* Removing page objects and other endpoint fields from base functional

* Updating code owners with new functional location

* Adding index pattern functionality

* Missed a file

* Pointing resolver tests at endpoint functional tests

* Pointing space tests at the endpoint functional directory

* Adding ingest service to do setup and tests for 500s

* Correcting services path

* Adding jest test names

* Updating es archives with the correct mapping and index names

* Fixing import error

* Adding resolver tests to code owners

* enabling epm flag for functional tests

* adding correct tag to test

* Removing the version information and unneeded xsrf

* Addressing endpoint index pattern feedback

* Removing unused import

* Renaming index pattern to es index pattern

* Fixing missed index pattern calls

* Removing unused import

* Fixing type error

* Moving es_index_pattern outside of installed and fixing function name

* Keeping the event index the same for now

* Wrapping index pattern await in try catch

* Address PR feedback, adding comments
@jonathan-buttner
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Test Failures

Kibana Pipeline / kibana-xpack-agent / X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting Dashboard "before all" hook: initialize tests in "Dashboard"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: reporting app
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: Reporting
[00:00:00]             └-> "before all" hook
[00:00:00]             └-: Dashboard
[00:00:00]               └-> "before all" hook
[00:00:00]               └-> "before all" hook: initialize tests
[00:00:00]                 │ debg ReportingPage:initTests
[00:00:00]                 │ info [reporting/ecommerce] Loading "mappings.json"
[00:00:00]                 │ info [reporting/ecommerce] Loading "data.json.gz"
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] applying create index request using v1 templates []
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [ecommerce] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:00:00]                 │ info [reporting/ecommerce] Created index "ecommerce"
[00:00:00]                 │ debg [reporting/ecommerce] "ecommerce" settings {"index":{"number_of_replicas":"0","number_of_shards":"1"}}
[00:00:00]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_1/Uhz_NW1iRbezthNqGk0vvA] deleting index
[00:00:00]                 │ info [reporting/ecommerce] Deleted existing index [".kibana_1"]
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] applying create index request using v1 templates []
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_1] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:00:00]                 │ info [reporting/ecommerce] Created index ".kibana_1"
[00:00:00]                 │ debg [reporting/ecommerce] ".kibana_1" settings {"index":{"auto_expand_replicas":"0-1","number_of_replicas":"0","number_of_shards":"1"}}
[00:00:02]                 │ proc [kibana]   log   [23:44:59.810] [warning][alerting][plugins][telemetry] Error executing alerting telemetry task: TypeError: Cannot read property 'value' of undefined
[00:00:03]                 │ proc [kibana]   log   [23:44:59.871] [error][plugins][taskManager][taskManager] Task apm-telemetry-task "apm-telemetry-task" failed: [strict_dynamic_mapping_exception] mapping set to strict, dynamic introduction of [counts] within [apm-telemetry] is not allowed :: {"path":"/.kibana/_doc/apm-telemetry%3Aapm-telemetry","query":{"refresh":"wait_for"},"body":"{\"apm-telemetry\":{\"counts\":{\"error\":{\"1d\":0,\"all\":0},\"metric\":{\"1d\":0,\"all\":0},\"span\":{\"1d\":0,\"all\":0},\"transaction\":{\"1d\":0,\"all\":0},\"onboarding\":{\"1d\":0,\"all\":0},\"sourcemap\":{\"1d\":0,\"all\":0},\"agent_configuration\":{\"all\":0},\"max_error_groups_per_service\":{\"1d\":0},\"max_transaction_groups_per_service\":{\"1d\":0},\"traces\":{\"1d\":0},\"services\":{\"1d\":0}},\"tasks\":{\"processor_events\":{\"took\":{\"ms\":121}},\"agent_configuration\":{\"took\":{\"ms\":18}},\"services\":{\"took\":{\"ms\":54}},\"versions\":{\"took\":{\"ms\":24}},\"groupings\":{\"took\":{\"ms\":33}},\"integrations\":{\"took\":{\"ms\":18}},\"agents\":{\"took\":{\"ms\":59}},\"indices_stats\":{\"took\":{\"ms\":16}},\"cardinality\":{\"took\":{\"ms\":42}}},\"has_any_services\":false,\"services_per_agent\":{\"java\":0,\"js-base\":0,\"rum-js\":0,\"dotnet\":0,\"go\":0,\"nodejs\":0,\"python\":0,\"ruby\":0},\"integrations\":{\"ml\":{\"all_jobs_count\":0}},\"agents\":{},\"indices\":{\"shards\":{\"total\":1},\"all\":{\"total\":{\"docs\":{\"count\":0},\"store\":{\"size_in_bytes\":208}}}},\"cardinality\":{\"transaction\":{\"name\":{\"all_agents\":{\"1d\":0},\"rum\":{\"1d\":0}}},\"user_agent\":{\"original\":{\"all_agents\":{\"1d\":0},\"rum\":{\"1d\":0}}}}},\"type\":\"apm-telemetry\",\"references\":[],\"updated_at\":\"2020-04-16T23:44:59.858Z\"}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"strict_dynamic_mapping_exception\",\"reason\":\"mapping set to strict, dynamic introduction of [counts] within [apm-telemetry] is not allowed\"}],\"type\":\"strict_dynamic_mapping_exception\",\"reason\":\"mapping set to strict, dynamic introduction of [counts] within [apm-telemetry] is not allowed\"},\"status\":400}"}
[00:00:04]                 │ info [reporting/ecommerce] Indexed 4675 docs into "ecommerce"
[00:00:04]                 │ info [reporting/ecommerce] Indexed 3 docs into ".kibana_1"
[00:00:04]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_1/yJJ_oNmUSi-RtNO4UrZpug] update_mapping [_doc]
[00:00:04]                 │ debg Migrating saved objects
[00:00:05]                 │ proc [kibana]   log   [23:45:01.984] [info][savedobjects-service] Creating index .kibana_2.
[00:00:05]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] applying create index request using v1 templates []
[00:00:05]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:00:05]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] updating number_of_replicas to [0] for indices [.kibana_2]
[00:00:05]                 │ proc [kibana]   log   [23:45:02.138] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:00:05]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_2/b34_RBmPTAWttKUy1z-adg] update_mapping [_doc]
[00:00:05]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_2/b34_RBmPTAWttKUy1z-adg] update_mapping [_doc]
[00:00:05]                 │ proc [kibana]   log   [23:45:02.386] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:00:05]                 │ proc [kibana]   log   [23:45:02.489] [info][savedobjects-service] Finished in 510ms.
[00:00:05]                 │ info [reporting/ecommerce_kibana] Loading "mappings.json"
[00:00:05]                 │ info [reporting/ecommerce_kibana] Loading "data.json.gz"
[00:00:05]                 │ proc [kibana]   log   [23:45:02.526] [warning][collector-set][plugins][usageCollection] { Error: Saved object [config/7.8.0-SNAPSHOT] not found
[00:00:05]                 │ proc [kibana]     at Function.createGenericNotFoundError (/dev/shm/workspace/install/kibana-6/src/core/server/saved_objects/service/lib/errors.js:136:37)
[00:00:05]                 │ proc [kibana]     at SavedObjectsRepository.get (/dev/shm/workspace/install/kibana-6/src/core/server/saved_objects/service/lib/repository.js:790:46)
[00:00:05]                 │ proc [kibana]     at process._tickCallback (internal/process/next_tick.js:68:7)
[00:00:05]                 │ proc [kibana]   data: null,
[00:00:05]                 │ proc [kibana]   isBoom: true,
[00:00:05]                 │ proc [kibana]   isServer: false,
[00:00:05]                 │ proc [kibana]   output:
[00:00:05]                 │ proc [kibana]    { statusCode: 404,
[00:00:05]                 │ proc [kibana]      payload:
[00:00:05]                 │ proc [kibana]       { statusCode: 404,
[00:00:05]                 │ proc [kibana]         error: 'Not Found',
[00:00:05]                 │ proc [kibana]         message: 'Saved object [config/7.8.0-SNAPSHOT] not found' },
[00:00:05]                 │ proc [kibana]      headers: {} },
[00:00:05]                 │ proc [kibana]   reformat: [Function],
[00:00:05]                 │ proc [kibana]   typeof: [Function: notFound],
[00:00:05]                 │ proc [kibana]   [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/notFound' }
[00:00:05]                 │ proc [kibana]   log   [23:45:02.531] [warning][collector-set][plugins][usageCollection] Unable to fetch data from stack_management collector
[00:00:05]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_2/b34_RBmPTAWttKUy1z-adg] deleting index
[00:00:05]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana_1/yJJ_oNmUSi-RtNO4UrZpug] deleting index
[00:00:05]                 │ info [reporting/ecommerce_kibana] Deleted existing index [".kibana_2",".kibana_1"]
[00:00:05]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] applying create index request using v1 templates []
[00:00:05]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana] creating index, cause [auto(bulk api)], templates [], shards [1]/[1], mappings []
[00:00:05]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] applying create index request using v1 templates []
[00:00:05]                 │ info Taking screenshot "/dev/shm/workspace/kibana/x-pack/test/functional/screenshots/failure/reporting app Reporting Dashboard _before all_ hook_ initialize tests.png"
[00:00:06]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1587078329952608962] [.kibana/0-wPD2eiQ2e8twqsNb0k1w] create_mapping [_doc]
[00:00:06]                 │ info Current URL is: data:/,
[00:00:06]                 │ info Saving page source to: /dev/shm/workspace/kibana/x-pack/test/reporting/configs/failure_debug/html/reporting app Reporting Dashboard _before all_ hook_ initialize tests.html
[00:00:06]                 └- ✖ fail: "reporting app Reporting Dashboard "before all" hook: initialize tests in "Dashboard""
[00:00:06]                 │

Stack Trace

{ Error: [invalid_alias_name_exception] Invalid alias name [.kibana], an index exists with the same name as the alias, with { index_uuid="0-wPD2eiQ2e8twqsNb0k1w" & index=".kibana" }
    at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 400,
  displayName: 'BadRequest',
  message:
   '[invalid_alias_name_exception] Invalid alias name [.kibana], an index exists with the same name as the alias, with { index_uuid="0-wPD2eiQ2e8twqsNb0k1w" & index=".kibana" }',
  path: '/.kibana_1',
  query: { include_type_name: false },
  body:
   { error:
      { root_cause: [Array],
        type: 'invalid_alias_name_exception',
        reason:
         'Invalid alias name [.kibana], an index exists with the same name as the alias',
        index_uuid: '0-wPD2eiQ2e8twqsNb0k1w',
        index: '.kibana' },
     status: 400 },
  statusCode: 400,
  response:
   '{"error":{"root_cause":[{"type":"invalid_alias_name_exception","reason":"Invalid alias name [.kibana], an index exists with the same name as the alias","index_uuid":"0-wPD2eiQ2e8twqsNb0k1w","index":".kibana"}],"type":"invalid_alias_name_exception","reason":"Invalid alias name [.kibana], an index exists with the same name as the alias","index_uuid":"0-wPD2eiQ2e8twqsNb0k1w","index":".kibana"},"status":400}',
  toString: [Function],
  toJSON: [Function] }

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jonathan-buttner jonathan-buttner merged commit e390a7d into elastic:7.x Apr 17, 2020
@jonathan-buttner jonathan-buttner deleted the backport/7.x/pr-63016 branch April 17, 2020 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backport
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jonathan-buttner @kibanamachine @elasticmachine