Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch default logs template to search all fields by default #102456

Merged
merged 12 commits into from
Nov 29, 2023

Conversation

eyalkoren
Copy link
Contributor

Closes #99872

Added a test that would fail prior to this change.
This adds the risk of hitting the maxClauseLimit error, so a fix for #102378 should follow up

@eyalkoren eyalkoren self-assigned this Nov 22, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Nov 22, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine
Copy link
Collaborator

Hi @eyalkoren, I've created a changelog YAML for you.

@elasticsearchmachine elasticsearchmachine added the external-contributor Pull request authored by a developer outside the Elasticsearch team label Nov 22, 2023
@dakrone dakrone changed the title Switch default logs template to use Switch default logs template to search all fields by default Nov 22, 2023
@eyalkoren
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/docs

Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code change LGTM.

I suggest we also have some docs ready around this that

  1. describes the behaviour when searching on logs-- form 8.x on
  2. describes how the behaviours could be changed by to message only by using logs@custom or similar

@ruflin
Copy link
Member

ruflin commented Nov 29, 2023

@mdbirnstiehl Can you help us on where these docs would fit in?

EDIT BY @eyalkoren: not sure if we need to doc that the default_field had changed, because it wasn't documented before.
However, what we do want to document for sure is that there is a way to set the default_field to explicit field/s through the logs@custom component template, as demonstrated in the added test.
Maybe it can go into the Logs troubleshooting page if the maxClauseLimit occurs.
I think we generally miss documentation about the @custom components.

@eyalkoren eyalkoren added Team:obs-knowledge Meta label for Observability Knowledge team and removed Team:obs-knowledge Meta label for Observability Knowledge team labels Nov 29, 2023
@eyalkoren eyalkoren merged commit 038a852 into elastic:main Nov 29, 2023
14 checks passed
@mdbirnstiehl
Copy link

mdbirnstiehl commented Nov 30, 2023

@ruflin @eyalkoren We don't really document anything about specific component templates or default index templates aside from the fact that they exist (aside from some small blurbs in the "Parse logs" doc. Adding something about maxClauseLimit and changing the default_field to the troubleshooting could be a short term solution, but it seems like long-term there should be a reference section in the logs docs that explains the templates available to our users and how they can use them. Maybe starting with logs@custom?

timgrein pushed a commit to timgrein/elasticsearch that referenced this pull request Nov 30, 2023
@ruflin
Copy link
Member

ruflin commented Nov 30, 2023

long-term there should be a reference section in the logs docs that explains the templates available to our users and how they can use them. Maybe starting with logs@custom?

Agree. Can you follow up with an issue. +1 on starting with logs@custom. Please also have a chat with the docs members from the ingest team as they might already have bits on this or you can collaborate.

@felixbarny
Copy link
Member

Here are some places in our docs that mention *@custom component templates and pipelines: https://github.com/search?q=org%3Aelastic+%40custom+language%3AAsciiDoc&type=code&l=AsciiDoc

@mdbirnstiehl
Copy link

Sounds good, I'll create an issue, check out the current @custom mentions, and reach out to the ingest team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Data streams Data streams and their lifecycles >enhancement external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Data Management Meta label for data/management team v8.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Logs] Switch default logs templates to use default_field[*]
5 participants