Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore timestamp in redis, haproxy and system filebeat module #9855

Merged
merged 3 commits into from
Jan 2, 2019
Merged

Ignore timestamp in redis, haproxy and system filebeat module #9855

merged 3 commits into from
Jan 2, 2019

Conversation

kaiyan-sheng
Copy link
Contributor

Another way to solve this issue #9849 is to ignore timestamp in test_modules.py for redis, haproxy and system filebeat module.

@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner January 2, 2019 21:51
@kaiyan-sheng kaiyan-sheng added the Team:Integrations Label for the Integrations team label Jan 2, 2019
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a better solution so if this passes on CI let's go with it.

@kaiyan-sheng kaiyan-sheng merged commit b9a20f7 into elastic:master Jan 2, 2019
@kaiyan-sheng kaiyan-sheng deleted the ignore_timestamp branch January 2, 2019 23:56
@kaiyan-sheng kaiyan-sheng added Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. labels Jan 3, 2019
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this!

@@ -178,6 +178,7 @@ def _test_expected_events(self, test_file, objects):

for ev in expected:
found = False
clean_keys(ev)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be needed after #9506 if expected json is regenerated, I have opened #9862 to regenerate the expected json files and remove this line.

kaiyan-sheng added a commit that referenced this pull request Jan 3, 2019
…ebeat module (#9855) (#9858)

* Ignore timestamp in redis, haproxy and system filebeat module (#9855)

* Ignore timestamp in redis, haproxy and system filebeat module

* Fixing formatting

(cherry picked from commit b9a20f7)

* Use event.dataset in if statement

* Update golden files for icinga/startup test to exclude timestamp (#9506)

The icing/startup log file does not contain a timestamp. Because of this the timestamp from filebeat is taken. During the generation of golden files still a timestamp was added but every skipped on comparison. Instead now the timestamp is not added to the generated file anymore to now show a diff each time GENERATE is run.

* Regenerate expectations for logs without timestamps (#9862)

* Rerun GENERATE=1 to update expected json files
kaiyan-sheng added a commit that referenced this pull request Jan 3, 2019
…ebeat module (#9855) (#9857)

* Ignore timestamp in redis, haproxy and system filebeat module (#9855)

* Ignore timestamp in redis, haproxy and system filebeat module

* Fixing formatting

(cherry picked from commit b9a20f7)

* Change to use event.dataset instead

* Update golden files for icinga/startup test to exclude timestamp (#9506)

The icing/startup log file does not contain a timestamp. Because of this the timestamp from filebeat is taken. During the generation of golden files still a timestamp was added but every skipped on comparison. Instead now the timestamp is not added to the generated file anymore to now show a diff each time GENERATE is run.

* Regenerate expectations for logs without timestamps (#9862)

* Rerun icinga redis haproxy and system modules with GENERATE=1
@ruflin
Copy link
Member

ruflin commented Jan 7, 2019

Thanks for the quick fix on this.

There is a difference from incinga to redis / haproxy in that incinga does not have a timestamp at all and these have a timestamp but no year. To still test that we convert the timestamps correctly it would be nice if the if statement would just replace the year and not ignore the timestamp completely if possible.

jsoriano pushed a commit to jsoriano/beats that referenced this pull request Jan 15, 2019
…ebeat module (elastic#9855) (elastic#9858)

* Ignore timestamp in redis, haproxy and system filebeat module (elastic#9855)

* Ignore timestamp in redis, haproxy and system filebeat module

* Fixing formatting

(cherry picked from commit b9a20f7)

* Use event.dataset in if statement

* Update golden files for icinga/startup test to exclude timestamp (elastic#9506)

The icing/startup log file does not contain a timestamp. Because of this the timestamp from filebeat is taken. During the generation of golden files still a timestamp was added but every skipped on comparison. Instead now the timestamp is not added to the generated file anymore to now show a diff each time GENERATE is run.

* Regenerate expectations for logs without timestamps (elastic#9862)

* Rerun GENERATE=1 to update expected json files

(cherry picked from commit 6defdd4)
@jsoriano jsoriano added v6.5.5 and removed needs_backport PR is waiting to be backported to other branches. labels Jan 15, 2019
jsoriano added a commit that referenced this pull request Jan 17, 2019
…ebeat module (#9855) (#9858) (#10072)

(cherry picked from commit b9a20f7)

The icing/startup log file does not contain a timestamp. Because of this the timestamp from filebeat is taken. During the generation of golden files still a timestamp was added but every skipped on comparison. Instead now the timestamp is not added to the generated file anymore to now show a diff each time GENERATE is run.

(cherry picked from commit 6defdd4)

Co-authored-by: Nicolas Ruflin <spam@ruflin.com>
Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…ebeat module (elastic#9855) (elastic#9858)

* Ignore timestamp in redis, haproxy and system filebeat module (elastic#9855)

* Ignore timestamp in redis, haproxy and system filebeat module

* Fixing formatting

(cherry picked from commit a5f0323)

* Use event.dataset in if statement

* Update golden files for icinga/startup test to exclude timestamp (elastic#9506)

The icing/startup log file does not contain a timestamp. Because of this the timestamp from filebeat is taken. During the generation of golden files still a timestamp was added but every skipped on comparison. Instead now the timestamp is not added to the generated file anymore to now show a diff each time GENERATE is run.

* Regenerate expectations for logs without timestamps (elastic#9862)

* Rerun GENERATE=1 to update expected json files
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…ebeat module (elastic#9855) (elastic#9858) (elastic#10072)

(cherry picked from commit a5f0323)

The icing/startup log file does not contain a timestamp. Because of this the timestamp from filebeat is taken. During the generation of golden files still a timestamp was added but every skipped on comparison. Instead now the timestamp is not added to the generated file anymore to now show a diff each time GENERATE is run.

(cherry picked from commit 6defdd4)

Co-authored-by: Nicolas Ruflin <spam@ruflin.com>
Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Filebeat Filebeat Team:Integrations Label for the Integrations team v6.5.5
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants