id_key should allow nested fields

[tom932]

Problem

I have several fields that are represented as json structures, of which i want to use one's value for the id.

Currently, I have to move the value of such a nested field to an own field via a record_transformer in order to use it in id_key.

Is there a possibility to use such a nested structure for the id_key command?

Steps to replicate

• have a json-structure as input with field a.b.c=val
• try to use nested field a.b.c for id_key
• the id setting doesn't work, a random id is generated.

Expected Behavior or What you need to ask

Key within nested structure is usable for the id

[cosmo0920]

This request for 2.x series?
If so, it should be possible to specify nested structured record's key with record_accessor plugin helper.
For 1.x, it is not easy to implement your request.

[tom932]

I'm currently using fluentd v0.12 (since I use another github project), therefore I used 1.13.0 of your plugin, since the documentation stated that v2.x.y doesn't work with fluentd 0.12.
If it is too hard to implement, I can use the work-around with writing the data to a seperate, non-nested filed.

[cosmo0920]

I've noticed that fluent-plugin-elasticsearch 1.x should work with Ruby 1.9.
So, it is too hard to implement nested id_key feature in v0.12 branch.
Do you want to use 1.x version of this plugin with Ruby 1.9?
Dropping Ruby 1.9 support and then targeting Ruby 2.0 or later will be possible to accept your request.

[cosmo0920]

I decided this feature request accept for v2.x and decline for 1.x.
Because, Fluentd v0.12 era, accessing nested record way had not been standardize:
fluent/fluentd#1578
Introduce nested record manipulation support on 1.x for Fluentd v0.12 version causes confuse about usability against other plugins.

[cosmo0920]

I've released v2.5.0 including this feature.