Add flatten_json and flatten_key_delimiter option

[shou10152208]

Which issue(s) this PR fixes:
Fixes #1578 (Already closed)

What this PR does / why we need it:
It makes it easy to output nested json values as flat data formats such as csv, tsv and ltsv.

Example

base fluentd.conf

<source>
  @type tail
  path "/var/log/test.ltsv"
  tag "debug.test"
  <parse>
    @type "ltsv"
  </parse>
</source>
<filter debug.**>
  @type parser
  key_name "nested_json_parameters"
  reserve_data true
  hash_value_field "nested_json_parameters"
  <parse>
    @type "json"
  </parse>
</filter>
<match debug.**>
  @type stdout
  @id stdout_output
  output_type ltsv
  <format>
    output_type ltsv
  </format>
</match>

input

$ echo 'json:message  nested_json_parameters:{"hash_hash":{"a":{"a":0},"b":{"b":1,"c":2}},"array_array":[[3],[4,5],[6,7,8]],"hash_array":{"a":[[9],[10,11],[12,13,14]]},"array_hash":[{"a":15},{"b":16,"c":17}]}' >> /var/log/test.ltsv

fluentd.conf changes and output

1. Not flatten

changes

Nothing

output

2019-06-21 13:35:11.593512000 +0900 debug.test: hash_hash:{"a"=>{"a"=>0}, "b"=>{"b"=>1, "c"=>2}}	array_array:[[3], [4, 5], [6, 7, 8]]   [hash_array:{"a"=>[[9], [10, 11], [12, 13, 14]]}	array_hash:[{"a"=>15}, {"b"=>16, "c"=>17}]

2. Flatten

changes

  <filter debug.**>
    @type parser
    key_name "nested_json_parameters"
    <parse>
      @type "json"
      flatten_json true        ## Add this
    </parse>
  </filter>

output

2019-06-21 13:38:16.397746000 +0900 debug.test: hash_hash.a.a:0 hash_hash.b.b:1 hash_hash.b.c:2 array_array.0.0:3       array_array.1.0:4       array_array.1.1:5       array_array.2.0:6       array_array.2.1:7       array_array.2.2:8    hash_array.a.0.0:9	hash_array.a.1.0:10	hash_array.a.1.1:11	hash_array.a.2.0:12	hash_array.a.2.1:13	hash_array.a.2.2:14	array_hash.0.a:15	array_hash.1.b:16	array_hash.1.c:17

3. Set the delimiter to "_" and flatten

changes

  <filter debug.**>
    @type parser
    key_name "nested_json_parameters"
    <parse>
      @type "json"
      flatten_json true              ## Add this
      flatten_key_delimiter "_"      ## Add this
    </parse>
  </filter>

output

2019-06-21 13:40:04.244973000 +0900 debug.test: hash_hash_a_a:0	hash_hash_b_b:1	hash_hash_b_c:2	array_array_0_0:3	array_array_1_0:4dd-flatarray_array_1_1:5son]array_array_2_0:6	array_array_2_1:7	array_array_2_2:8	hash_array_a_0_0:9	hash_array_a_1_0:10	hash_array_a_1_1:11	hash_array_a_2_0:12	hash_array_a_2_1:13	hash_array_a_2_2:14	array_hash_0_a:15	array_hash_1_b:16	array_hash_1_c:17

Docs Changes:
Need to add option description and sample https://docs.fluentd.org/parser/json

Release Note:
Same as the title

Thank you

[shou10152208]

@repeatedly
This PR is a feature addition.
Failure of the automatic CI is not related to this change, but
Is there any reason not to merge?
I want to improve as much as possible.

[repeatedly]

This is similar to https://github.com/kazegusuri/fluent-plugin-flatten-hash
And we added record_accessor filter so almost cases don't need this feature, right?

[shou10152208]

I'm sorry.
Thank you for your kindness.