The reload configuration with the worker tag failed

[xidiandb]

Describe the bug

I used a configuration with the worker tag, which runs on startup but not on reload.

To Reproduce

Use my configuration to start and reload

Expected behavior

fluent/log.rb:371:error: Failed to reload config file: specified worker_id<0> collisions is detected on directive. Available worker id(s): []

Your Environment

- Fluentd version: 1.11.4
- TD Agent version:
- Operating system: ubuntu 18.04
- Kernel version:5.4.61-050461-generic

Your Configuration

<system>
  workers 1
  rpc_endpoint "#{ENV['POD_IP']}:24444"
</system>
<label @FLUENT_LOG>
<match fluent.*>
  @type null
</match>
</label>   
<worker 0>
<source>
  @type sample
  sample {"hello borg ooo":"world"}
  rate 1
  tag sample.ni.hao
</source>
</worker>
<worker 0>
<match sample.*.*>
  @type stdout
</match>
</worker>

Your Error Log

fluent/log.rb:371:error: Failed to reload config file: specified worker_id<0> collisions is detected on <worker> directive. Available worker id(s): []

Additional context

No response

[kenhys]

Use the following, do not define multiple <worker 0>.

<system>
  workers 1
  rpc_endpoint "#{ENV['POD_IP']}:24444"
</system>
<label @FLUENT_LOG>
  <match fluent.*>
    @type null
  </match>
</label>   
<worker 0>
  <source>
    @type sample
    sample {"hello borg ooo":"world"}
    rate 1
    tag sample.ni.hao
  </source>
  <match sample.*.*>
    @type stdout
  </match>
</worker>

[xidiandb]

@kenhys But the first time it started, there was no problem, only on reload ，Moreover, my configuration is very complex, which is divided into multiple files, and some configurations cannot be written to a worker tag. I would like to know if this boot is ok and if it is a bug when it reloads

[xidiandb]

I want to add configuration dynamically by adding files, each with worker tags. Is that not supportive? But why is there no problem at startup, only on reload

[kenhys]

Hmm, I've overlooked it.

[kenhys]

https://github.com/fluent/fluentd/blob/master/lib/fluent/supervisor.rb#L290-L303
It seems that Fluent::Engine.reload_config raise it.

[ashie]

I cannot reproduce it by HUP signal, but can reproduce by USR2 signal.

[ashie]

There are 2 places which show such message:

fluentd/lib/fluent/static_config_analysis.rb

Line 81 in 1b46fe0

raise Fluent::ConfigError, "specified worker_id<#{id}> collisions is detected on <worker> directive. Available worker id(s): #{available_worker_ids}"

fluentd/lib/fluent/root_agent.rb

Line 95 in 1b46fe0

raise Fluent::ConfigError, "specified worker_id<#{worker_id}> collisions is detected on <worker> directive. Available worker id(s): #{available_worker_ids}"

and former one is used in this case.

[ashie]

Fluent::StaticConfigAnalysis is used only on graceful-reload.
It's added for implementing graceful-reload feature.

[ashie]

<worker> elements are parsed by the following code on start up.

fluentd/lib/fluent/root_agent.rb

Lines 69 to 124 in 8f990b8

	used_worker_ids = []
	available_worker_ids = (0..Fluent::Engine.system_config.workers - 1).to_a
	# initialize <worker> elements
	conf.elements(name: 'worker').each do |e|
	target_worker_id_str = e.arg
	if target_worker_id_str.empty?
	raise Fluent::ConfigError, "Missing worker id on <worker> directive"
	end
	target_worker_ids = target_worker_id_str.split("-")
	if target_worker_ids.size == 2
	first_worker_id = target_worker_ids.first.to_i
	last_worker_id = target_worker_ids.last.to_i
	if first_worker_id > last_worker_id
	raise Fluent::ConfigError, "greater first_worker_id<#{first_worker_id}> than last_worker_id<#{last_worker_id}> specified by <worker> directive is not allowed. Available multi worker assign syntax is <smaller_worker_id>-<greater_worker_id>"
	end
	target_worker_ids = []
	first_worker_id.step(last_worker_id, 1) do |worker_id|
	target_worker_id = worker_id.to_i
	target_worker_ids << target_worker_id
	if target_worker_id < 0 || target_worker_id > (Fluent::Engine.system_config.workers - 1)
	raise Fluent::ConfigError, "worker id #{target_worker_id} specified by <worker> directive is not allowed. Available worker id is between 0 and #{(Fluent::Engine.system_config.workers - 1)}"
	end
	available_worker_ids.delete(target_worker_id) if available_worker_ids.include?(target_worker_id)
	if used_worker_ids.include?(target_worker_id)
	raise Fluent::ConfigError, "specified worker_id<#{worker_id}> collisions is detected on <worker> directive. Available worker id(s): #{available_worker_ids}"
	end
	used_worker_ids << target_worker_id
	e.elements.each do |elem|
	unless ['source', 'match', 'filter', 'label'].include?(elem.name)
	raise Fluent::ConfigError, "<worker> section cannot have <#{elem.name}> directive"
	end
	end
	unless target_worker_ids.empty?
	e.set_target_worker_ids(target_worker_ids.uniq)
	end
	end
	else
	target_worker_id = target_worker_id_str.to_i
	if target_worker_id < 0 || target_worker_id > (Fluent::Engine.system_config.workers - 1)
	raise Fluent::ConfigError, "worker id #{target_worker_id} specified by <worker> directive is not allowed. Available worker id is between 0 and #{(Fluent::Engine.system_config.workers - 1)}"
	end
	e.elements.each do |elem|
	unless ['source', 'match', 'filter', 'label'].include?(elem.name)
	raise Fluent::ConfigError, "<worker> section cannot have <#{elem.name}> directive"
	end
	elem.set_target_worker_id(target_worker_id)
	end
	end
	conf += e
	end
	conf.elements.delete_if{|e| e.name == 'worker'}

Fluent::StaticConfigAnalysis seems be quit different than it.

[ashie]

I want to add configuration dynamically by adding files, each with worker tags. Is that not supportive? But why is there no problem at startup, only on reload

Hmm, it's ambiguous whether multiple <worker> for same ID is supported or not.
The duplication check is introduced at #2292 only for <worker n-m> syntax.
<worker n> syntax is allowed multiple both before & after it.

[ashie]

I think it would be better that check duplication for <worker n> syntax too, but show only warning and don't block loading to keep compatibility.