[App services] Permission errors logged when opening Kibana for the first time

[wylieconlon]

This is happening on the main branch as of today.

Steps to reproduce:

1. Open Kibana in an incognito window
2. See the following errors in the console:

server    log   [14:51:20.085] [error][data][data][indexPatterns][plugins] ResponseError: security_exception
    at onBody (kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:333:23)
    at IncomingMessage.onEnd (kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:260:11)
    at IncomingMessage.emit (events.js:327:22)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  meta: {
    body: { error: [Object], status: 401 },
    statusCode: 401,
    headers: {
      'x-opaque-id': '89f382ce-6825-453a-8bb5-1664d97b630e',
      'www-authenticate': 'Basic realm="security" charset="UTF-8"',
      'content-type': 'application/json;charset=utf-8',
      'content-length': '441'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  },
  isBoom: true,
  isServer: true,
  data: null,
  output: {
    statusCode: 500,
    payload: {
      statusCode: 500,
      error: 'Internal Server Error',
      message: 'An internal server error occurred'
    },
    headers: {}
  },
  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/generalError'
}

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[legrego]

Discussed offline, and this appears to be a faulty Route Handler Context:

kibana/src/plugins/data/server/index_patterns/index_patterns_service.ts

Lines 55 to 69 in 2ef7f3b

	core.http.registerRouteHandlerContext<DataRequestHandlerContext, 'indexPatterns'>(
	'indexPatterns',
	async (context, request) => {
	const [coreStart, , dataStart] = await core.getStartServices();
	try {
	return await dataStart.indexPatterns.indexPatternsServiceFactory(
	coreStart.savedObjects.getScopedClient(request),
	coreStart.elasticsearch.client.asScoped(request).asCurrentUser
	);
	} catch (e) {
	logger.error(e);
	return undefined;
	}
	}
	);

These context providers execute on every inbound http request to the Kibana server, so they should be kept as performant as possible.

This particular provider is querying the UI Settings Service to retrieve personalized settings required for index patterns. There are a couple of problems with this approach:

1. This runs on every request, including those on unauthenticated pages such as the login screen. It's not possible to use the UI Settings service without an authenticated user (assuming security is enabled).
2. I expect this context provider is only used by a handful of routes. That means we are doing a lot of work that will be thrown away on every other http request that Kibana has to handle.

I think this context handler should be revisited so that it does not attempt to communicate with ES at all unless explicitly requested by a consumer.

[Dosant]

No longer happening because the pr that introduced this was reverted #95335 (cc @lizozom fyi about the route context: #95094 (comment))