[Discover] Only request fields that are shown to user when using new fields API

[wylieconlon]

This is a bug in Discover on master

Steps to reproduce:

1. In advanced settings, enable discover:searchFieldsFromSource
2. Go to Discover and click Inspect. You will see the request is fields: [{ field: '*' }, ...date fields], which is normal without excluded fields
3. Edit your index pattern to add an excluded field
4. Go to Discover and click Inspect. You will see the request is fields: [{ field: 'field1' }, { field: 'field2' }]
5. Click a single column, such as bytes, to add to the table
6. Click Inspect. The request still lists every field, even ones that are not bytes.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[majagrubic]

There is no additional request (shouldn't be?) between steps 5 and 6, so I am confused what the issue here is

[majagrubic]

Everything described here is the expected behavior. The inspector shows the last executed request, which was the one requesting all the fields explicitly since you have a source filter configured.

[wylieconlon]

@majagrubic I see, I assumed we were making a new request when adding columns- this doesn't seem like a bug in the Discover app. Do you think it would be a bug in the saved search embeddable, which is making a larger-than-needed request?

[majagrubic]

Search embeddable will always request '*':

kibana/src/plugins/discover/public/application/embeddable/search_embeddable.ts

Line 311 in c37b0e1

const fields: Record<string, string> = { field: '*' };

Or did you have something else in mind?

[wylieconlon]

@majagrubic Right, but maybe we could optimize the request to not need as much data. Are the edge cases where that wouldn't work?

[majagrubic]

I think fields '*' is a pretty safe bet. We can definitely look into requesting just the fields that are already configured in the search embeddable to optimize this further, but I definitely don't think it's a high priority.

[timroes]

We're needing to request all fields, since we're using the same result to show when the document is expanded (i.e. the full document). There is no additional request happening for expanded documents (for now by choice because of the performance impact), i.e. if we would only request columns configured in the table and a user would then watch the whole document it would only contain those columns and not the full document.

[majagrubic]

Can we close this issue as it's not really an issue?

[timroes]

Update: I am not sure if I understood Wylie initially correctly when I read that issue.

I think there is an issue here, we're currently in the saves search embeddable ignoring the field filters completely. Though I am not sure if that is exactly what Wylie meant.

UPDATE2: I actually was looking at some wrong place with that comment above, so ignore it.