[Index Patterns] Improve handling of the field_caps API request

[kertal]

Soon an index pattern no longer caches its fields in the saved object, so they will be requested by field_caps API more often (#82223). There is a related problem we've encountered that caused some functional tests to be flaky.

Currently an index pattern is checking, if refreshing the fields cache is necessary. Most of our test fixtures contain legacy field data, so it has to be refreshed. Usually this works, but there were several test failures related to cases when the retrieval of the fieldsAPI didn't work. Here are some details about the last known failure:

https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/971/testReport/junit/Chrome%20UI%20Functional%20Tests/test_functional_apps_discover__filter_editor·js/discover_app_discover_filter_editor_filter_editor_should_add_a_phrases_filter/

The test case failing is using the discover fixture.

kibana/test/functional/apps/discover/_filter_editor.js

Line 36 in bf04235

await esArchiver.loadIfNeeded('discover');

Its saved object contained legacy fields data (was updated in the meantime):

This is triggering a request to the field_caps API, here's some logging

http://localhost:61101/37924/bundles/plugin/data/data.plugin.js 0:772727 "API client url" "/api/index_patterns/_fields_for_wildcard"
[00:09:10]               │ debg browser[INFO] http://localhost:61101/37924/bundles/plugin/data/data.plugin.js 0:772761 "API client query" "{\"pattern\":\"logstash-*\",\"meta_fields\":[\"_source\",\"_id\",\"_type\",\"_index\",\"_score\"]}"
[00:09:10]               │ERROR browser[SEVERE] http://localhost:61101/37924/bundles/plugin/data/data.plugin.js 0:772950 "API client Error Message" "Cannot read property 'statusCode' of undefined"
[00:09:10]               │ERROR browser[SEVERE] http://localhost:61101/37924/bundles/plugin/data/data.plugin.js 0:773002 "API client Error" "{}"
[00:09:10]               │ debg browser[INFO] http://localhost:61101/37924/bundles/plugin/data/data.plugin.js 0:594544 "refreshFieldSpecMap ERROR" "{}"

This request failed, there was no valid body returned ("Cannot read property 'statusCode' of undefined). As a result the fields are not updated, and when requesting the data for the Discover histogram @timestamp is missing the aggregatable flag, so it's considered as invalid:

Invalid:

Valid:

To omit this cases the fieldsAPI retrieval should be retried in case of a failure. If it keeps on failing, and there's no valid field cache, it would be good practice to prevent the further usage of this index pattern, until valid fields can be retrieve, to omit confusion by follow up error messages, leading in the wrong direction. Also more logging should be added in case of errors, to provide more information about failing requests.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[mattkime]

I'm going to wait before acting upon this. It seems unlikely to me that this is an index pattern particular problem, rather its either a) something that affects all requests or b) something particular to the field caps api. Also, I'd like to make sure that I'm not solving for a problem that only occurs in CI.

Will keep an eye out for this.

[kertal]

Well, at least it shouldn't be possible to use index patterns with invalid fields, since the follow up error message are confusing. I've seen the misleading error message @ discuss.elastic.co, happened because of an index pattern field update by an external client. This won't be possible in the future, right? But anyway it has to be checked that whatever error happens when fetching fields, the index pattern can't be used (until the data is refetched )

[mattkime]

But anyway it has to be checked that whatever error happens when fetching fields, the index pattern can't be used (until the data is refetched )

Yes, this will be resolved. A field_caps request failure will result in a thrown error.

[ppisljar]

Thank you for contributing to this issue, however, we are closing this issue due to inactivity as part of a backlog grooming effort. If you believe this feature/bug should still be considered, please reopen with a comment.