RabbitMQ plugin with tls enable error #13

meveno · 2019-10-03T12:35:11Z

Hi,

I'm setting up AMQP/TLS on my rabbitmq server but trying to connect fluentd to rabbitmq using TLS is not working.

fluentd-rabbit-vdev_1  | 2019-10-03 12:16:01 +0000 [info]: adding source type="rabbitmq"
fluentd-rabbit-vdev_1  | 2019-10-03 12:16:01 +0000 [warn]: #0 Using TLS but no client certificate is provided. If RabbitMQ is configured to require & verify peer
fluentd-rabbit-vdev_1  | certificate, connection will be rejected. Learn more at https://www.rabbitmq.com/ssl.html
fluentd-rabbit-vdev_1  |
fluentd-rabbit-vdev_1  | 2019-10-03 12:16:01 +0000 [info]: #0 starting fluentd worker pid=34 ppid=6 worker=0
fluentd-rabbit-vdev_1  | 2019-10-03 12:16:01 +0000 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `add' for #<Fluent::Log:0x0000564ea68845f0>"
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/bunny-2.14.3/lib/bunny/transport.rb:374:in `log_peer_certificate_info'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/bunny-2.14.3/lib/bunny/transport.rb:93:in `connect'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/bunny-2.14.3/lib/bunny/session.rb:315:in `start'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-rabbitmq-0.0.7/lib/fluent/plugin/in_rabbitmq.rb:113:in `start'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:203:in `block in start'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:191:in `each'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:178:in `each'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:178:in `lifecycle'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/root_agent.rb:202:in `start'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/engine.rb:274:in `start'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/engine.rb:219:in `run'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:808:in `run_engine'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:551:in `block in run_worker'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:733:in `main_process'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:546:in `run_worker'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/lib/fluent/command/fluentd.rb:30:in `<top (required)>'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.6.2/bin/fluentd:8:in `<top (require d)>'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/bin/fluentd:23:in `load'
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 /usr/bin/fluentd:23:in `<main>'
fluentd-rabbit-vdev_1  | 2019-10-03 12:16:01 +0000 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `add' for #<Fluent::Log:0x0000564ea68845f0>"
fluentd-rabbit-vdev_1  |   2019-10-03 12:16:01 +0000 [error]: #0 suppressed same stacktrace
fluentd-rabbit-vdev_1  | 2019-10-03 12:16:02 +0000 [info]: Worker 0 finished unexpectedly with status 1

Here is my configuration file :

  @type rabbitmq
  tag msg.business
  host "#{ENV['RABBITMQ_HOST']}"
  port "#{ENV['RABBITMQ_PORT']}"
  user "#{ENV['RABBITMQ_USER']}"
  pass "#{ENV['RABBITMQ_PWD']}"
  vhost /
  tls "#{ENV['RABBITMQ_ENABLE_TLS']}"
  verify_peer false
  exchange Business # not required. if specified, the queue will be bound to the exchange
  exchange_type topic # required: type of exchange e.g. topic, direct
  exchange_durable true
  create_exchange true
  queue business-fluentd-queue
  include_headers true
  routing_key "#" # if not specified, the tag is used
  heartbeat 10 # integer as seconds or :server (interval specified by server)
  <parse>
    @type json # or msgpack, ltsv, none
  </parse>
  <buffer> # to use in buffered mode
  </buffer>
</source>

<filter msg.**>
  @type record_transformer
  <record>
    contexte "#{ENV['CONTEXTE']}"
    eventType "${tag_parts[1]}"
    env "#{ENV['ENV_NAME']}"
  </record>
</filter>

<match msg.**>
  @type elasticsearch
  host elasticsearch
  include_tag_key true
  logstash_format true
  logstash_prefix "#{ENV['ENV_NAME']}-rabbitmq"
  flush_interval 10s # for testing
</match>

The text was updated successfully, but these errors were encountered:

meveno · 2019-10-03T15:39:54Z

Using fluentd docker image v1.6.2-1.0
with fluentd-rabbitmq-plugin: v0.0.7
And rabbitmq docker image rabbitmq:3.7.15-management

meveno · 2019-10-24T07:29:50Z

No one can help me ?

meveno · 2019-12-02T10:15:07Z

I've dig a few on this error, it seems that bunny tries to log certificate informations and fail on logger :
https://groups.google.com/forum/#!msg/ruby-amqp/lWrxJ0PZz34/Kfm76GHrBwAJ

Message :
Bunny's logger is not set up [1]. It is taken from the connection object [2] which Bunny
will initialize by default [3]. I don't know how the Fluentd client you use sets up logging
but extremely likely that it passes a nil for :logger.

I'm not a ruby pro ;)
How can we provide a logger ?

abstractvector · 2020-03-09T03:17:17Z

I'm running into this exact same error message while trying to use TLS for RabbitMQ. Also not a Ruby expert though....

mmasaki · 2020-03-09T03:38:09Z

Please check RabbitMQ configuration.

https://github.com/ruby-amqp/bunny/blob/master/lib/bunny/transport.rb#L467-L472
https://www.rabbitmq.com/ssl.html

You would need to provide a client certificate ( tls_cert ) and a key ( tls_key ) or disable verify_peer on RabbitMQ server.

abstractvector · 2020-03-09T03:39:48Z

@mmasaki yes, I've provided all the correct tls parameters - but no luck. Same error as @meveno above.

I've just switched to the fluentd-plugin-amqp plugin and it works fine.

mmasaki · 2020-03-09T03:47:34Z

Could you show me fluent.conf ?

abstractvector · 2020-03-09T03:57:58Z

Sure:

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<filter docker.**>
  @type parser
  key_name log
  <parse>
    @type json
  </parse>
</filter>

<match docker.**>
  @type copy

  <store>
    @type stdout
  </store>

  <store>
    @type rabbitmq
    host <REDACTED>
    user <REDACTED>
    pass <REDACTED>
    format json
    exchange logs
    exchange_type topic
    exchange_durable true
    heartbeat 10
    persistent true
    content_type application/json
    automatically_recover true

    tls true
    tls_cert <REDACTED>
    tls_key <REDACTED>
    tls_ca_certificates ["<REDACTED>"]
    verify_peer true

    <format>
      @type json
    </format>
  </store>

</match>

mmasaki · 2020-03-09T04:20:17Z

Thanks. I found Fluent::Log is not compatible with Bunny (it lacks #add method).
I've just removed the logger and released v0.0.9.

abstractvector · 2020-03-09T04:21:50Z

That’s awesome - thanks for the quick response. I’ll try it again tomorrow and report back!

abstractvector · 2020-03-09T15:43:27Z

@mmasaki I switched back to this plugin and can confirm all is working. For anyone else who runs across this, I can confirm that on the new version of the plugin, the following config is successful in connecting to a TLS-enabled RabbitMQ server including peer verification using a client certificate:

<match docker.**>
  @type copy

  <store>
    @type stdout
  </store>

  <store>
    @type rabbitmq
    host <REDACTED>
    user <REDACTED>
    pass <REDACTED>
    exchange <REDACTED>
    exchange_type topic
    exchange_durable true
    heartbeat 10
    persistent true
    content_type application/json
    automatically_recover true

    tls true
    tls_cert <REDACTED>
    tls_key <REDACTED>
    tls_ca_certificates [<REDACTED>]
    verify_peer true

    <format>
      @type json
    </format>
  </store>

</match>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RabbitMQ plugin with tls enable error #13

RabbitMQ plugin with tls enable error #13

meveno commented Oct 3, 2019

meveno commented Oct 3, 2019 •

edited

Loading

meveno commented Oct 24, 2019

meveno commented Dec 2, 2019

abstractvector commented Mar 9, 2020

mmasaki commented Mar 9, 2020

abstractvector commented Mar 9, 2020

mmasaki commented Mar 9, 2020

abstractvector commented Mar 9, 2020

mmasaki commented Mar 9, 2020

abstractvector commented Mar 9, 2020

abstractvector commented Mar 9, 2020

RabbitMQ plugin with tls enable error #13

RabbitMQ plugin with tls enable error #13

Comments

meveno commented Oct 3, 2019

meveno commented Oct 3, 2019 • edited Loading

meveno commented Oct 24, 2019

meveno commented Dec 2, 2019

abstractvector commented Mar 9, 2020

mmasaki commented Mar 9, 2020

abstractvector commented Mar 9, 2020

mmasaki commented Mar 9, 2020

abstractvector commented Mar 9, 2020

mmasaki commented Mar 9, 2020

abstractvector commented Mar 9, 2020

abstractvector commented Mar 9, 2020

meveno commented Oct 3, 2019 •

edited

Loading