Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter_parser: Add remove_key_name_field parameter #2012

Merged
merged 1 commit into from
Jun 8, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions lib/fluent/plugin/filter_parser.rb
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ class ParserFilter < Filter
config_param :reserve_data, :bool, default: false
desc 'Keep original event time in parsed result.'
config_param :reserve_time, :bool, default: false
desc 'Remove "key_name" field from the record when parsing is succeeded'
config_param :remove_key_name_field, :bool, default: false
desc 'Store parsed values with specified key name prefix.'
config_param :inject_key_prefix, :string, default: nil
desc 'If true, invalid string is replaced with safe characters and re-parse it.'
Expand Down Expand Up @@ -75,6 +77,7 @@ def filter_with_time(tag, time, record)
else
t.nil? ? time : t
end
@accessor.delete(record) if @remove_key_name_field
r = handle_parsed(tag, record, t, values)
return t, r
else
Expand Down
12 changes: 10 additions & 2 deletions test/plugin/test_filter_parser.rb
Original file line number Diff line number Diff line change
Expand Up @@ -209,10 +209,13 @@ def test_filter

end

def test_filter_with_reserved_data
data(:keep_key_name => false,
:remove_key_name => true)
def test_filter_with_reserved_data(remove_key_name)
d1 = create_driver(%[
key_name data
reserve_data yes
remove_key_name_field #{remove_key_name}
<parse>
@type regexp
expression /^(?<x>\\d)(?<y>\\d) (?<t>.+)$/
Expand All @@ -230,6 +233,7 @@ def test_filter_with_reserved_data
d2 = create_driver(%[
key_name data
reserve_data yes
remove_key_name_field #{remove_key_name}
<parse>
@type json
</parse>
Expand All @@ -244,7 +248,11 @@ def test_filter_with_reserved_data

first = filtered[0]
assert_equal_event_time time, first[0]
assert_equal '{"xxx":"first","yyy":"second"}', first[1]['data']
if remove_key_name
assert_not_include first[1], 'data'
else
assert_equal '{"xxx":"first","yyy":"second"}', first[1]['data']
end
assert_equal 'first', first[1]['xxx']
assert_equal 'second', first[1]['yyy']

Expand Down