Issue with symlink_path and placeholder

[Erouan50]

Hello,

I notice the symlink_path option now supports placeholder in the latest 1.4.1 release #2254. However, it seems to need the whole arborescence present or Fluentd can't create the links.

Here is an example:

<source>
  @type forward
  @id forward_input
</source>

<match debug.**>
  @type file
  path /var/log/test/${key1}/${tag[1]}-%Y%m%d.log
  symlink_path /var/log/test/${key1}/${tag[1]}.log.current
  <buffer tag,key1,time>
    path /var/lib/fluentd/buffers
    timekey 1d
  </buffer>
</match>

If I try to send the following log, the symlink is not created, and I get a "No such file or directory" error in the fluentd log:

echo '{"json":"message","key1":"prod"}' | fluent-cat debug.test

The exception:

2019-03-26 15:40:59 -0400 [warn]: #0 fluent/log.rb:342:warn: emit transaction failed: error_class=Errno::ENOENT error="No such file or directory @ rb_file_s_symlink - (/var/lib/fluentd/buffers/buffer.b585048026018f6e083e1e95ae1b8f112.log, /var/log/test/prod/test.log.current)" location="/usr/share/ruby/fileutils.rb:323:in `symlink'" tag="debug.test"
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /usr/share/ruby/fileutils.rb:323:in `symlink'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /usr/share/ruby/fileutils.rb:323:in `block in ln_s'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /usr/share/ruby/fileutils.rb:1479:in `fu_each_src_dest0'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /usr/share/ruby/fileutils.rb:321:in `ln_s'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /usr/share/ruby/fileutils.rb:340:in `ln_sf'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/out_file.rb:90:in `generate_chunk'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/buffer.rb:585:in `block in write_once'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /usr/share/ruby/monitor.rb:226:in `mon_synchronize'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/buffer.rb:585:in `write_once'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/buffer.rb:311:in `block in write'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/buffer.rb:309:in `each'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/buffer.rb:309:in `write'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/output.rb:954:in `block in handle_stream_with_custom_format'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/output.rb:880:in `write_guard'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/output.rb:953:in `handle_stream_with_custom_format'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/output.rb:872:in `execute_chunking'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/output.rb:793:in `emit_buffered'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/event_router.rb:96:in `emit_stream'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/in_forward.rb:320:in `on_message'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/in_forward.rb:211:in `block in handle_connection'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/in_forward.rb:248:in `block (3 levels) in read_messages'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/in_forward.rb:247:in `feed_each'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/in_forward.rb:247:in `block (2 levels) in read_messages'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin/in_forward.rb:256:in `block in read_messages'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin_helper/server.rb:612:in `on_read_without_connection'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/cool.io-1.5.3/lib/cool.io/io.rb:123:in `on_readable'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/cool.io-1.5.3/lib/cool.io/io.rb:186:in `on_readable'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin_helper/event_loop.rb:93:in `block in start'
  2019-03-26 15:40:59 -0400 [warn]: #0 plugin/in_forward.rb:320:on_message: /home/erouan/.gem/ruby/gems/fluentd-1.4.1/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'

If I create the /var/log/test/prod path, it works.

Thanks!

Environment information:

• OS: Fedora release 29 (Twenty Nine)
• Kernel: Linux 4.20.15-200.fc29.x86_64 SMP Mon Mar 11 16:01:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Fluentd version: 1.4.1

[repeatedly]

Yeah. The problem is ln_sf doesn't create parent directories.
So need to call mkdir_p before it.

[repeatedly]

patch is here: #2353