Stack Management elasticsearch-js client migration

[cjcenizal]

Summary

Per #83910, the new elasticsearch-js client is available in Kibana and we need to migrate our apps over to it prior to 7.14.

Learn more about the new elasticsearch-js client at https://github.com/elastic/elasticsearch-js.

See the ES JS client API reference.

Punch list

Apps

Alison

• Search Profiler ([Search Profiler] Migrate server to new es-js client #88725)
• Grok Debugger ([Grokdebugger] Fix simulate error handling #83036)
• Painless Lab ([Painless Lab] Update server to use new es-js client #88704)
• Upgrade Assistant ([Upgrade Assistant] Migrate server to new es-js client #89207)

Yulia

• Index Lifecycle Policies ([ILM] Replace legacy ES client with the new client #78416, [ILM] Add esErrorHandler for the new es js client #80302)

JL

• Snapshot and Restore (and functional tests) ([Snapshot Restore] Migrate to new ES client #95499)
• Rollup Jobs (and functional tests) ([Rollup] Migrate to new ES client #95926)
• Ingest Node Pipelines (and functional tests) ([Ingest Pipelines] Migrate to new ES client #96406)
• Watcher ([Watcher] Migrate to new ES client #97260)
• License Management ([License Management] Migrate to new ES client #97795)
• Remote Clusters ([Remote clusters] Migrate to new ES client #98747)

CJ

• Console (no migration necessary, as all ES requests are sent manually, not using the client)
• es_ui_shared, I don't think this needs migration other than the deleting the legacy isEsError function once all of our plugins have been migrated and we no longer use it
• Cross-Cluster Replication (Migrate CCR to new ES JS client. #100131)
• Index Management (Migrate Index Management to the new ES JS client #105863)
  • ILM leftovers (https://github.com/elastic/kibana/pull/78416/files#diff-619a4a46f7698e000cf708eb398fe95ca6d9e031e9a9fabd54182ed7f2f57ce3R28)
  • CCR leftovers (https://github.com/elastic/kibana/pull/100131/files#diff-3a466f5999d6c68de1423317258d9cf6c0f2f518d764d9c84084141c626715e3R26)

How to migrate

📚 Be prepared! Review the migration guide.

Basic steps

• Replace instances of import { LegacyAPICaller } from 'src/core/server' with import { ElasticsearchClient } from 'kibana/server'.
• Replace instances of LegacyAPICaller (commonly named callAsCurrentUser) with ElasticsearchClient (commonly named client).
• Replace instances of transport.request with direct calls to the APIs defined on the new client (see API docs).
• Similarly, replace consumption of legacy client extensions (e.g. https://github.com/elastic/kibana/blob/master/x-pack/plugins/cross_cluster_replication/server/client/elasticsearch_ccr.ts) with direct calls to the APIs defined on the new client.
• Replace references to context.core.elasticsearch.legacy.client.callAsCurrentUser with references to context.core.elasticsearch.client.asCurrentUser.

Handling errors

When handling errors, replaces usage of lib.isEsError(e) with a comparison of e.name === 'ResponseError' and replace usage of:

return response.customError({
  statusCode: e.statusCode,
  body: e,
});

with:

// See https://github.com/elastic/kibana/pull/80302 for an example of how to provide handleEsError
return handleEsError({ error, response });

Per #84801, test the app and verify that full ES errors are surfaced in the UI via SectionError, if applicable.

API integration tests

• Migrate references of getService('legacyEs') to getService('es').
• The es service now returns a promise that resolves to an object containing the body. So any helper functions that consume this API like const getIndex = (index) => es.indices.get({ index }); will need to be migrated to handle the resolved promise and pull out the body: ``const getIndex = (index) => es.indices.get({ index }).then(({ body }) => body);`.
• Re-run tests (steps below) to verify the client hasn't changed the way errors are reported.

# tab 1
cd x-pack && node scripts/functional_tests_server.js

# tab 2
node scripts/functional_test_runner.js --config x-pack/test/api_integration/config.ts --grep "Some string from the test case name" 

Functional tests

We also need to migrate functional tests over from using the legacyEs service to the es service. This means replacing instances of getService('legacyEs') with getService('es'), and ensuring that the required APIs are available.

Once you're done

• Update the punch list at the top of this issue by checking off completed items and linking to the relevant PRs
• Update the migration tracker meta issue and check off completed items

[elasticmachine]

Pinging @elastic/es-ui (Team:Elasticsearch UI)