Unhandled Exception: System.DllNotFoundException: Failed to load the librdkafka native library.

[Zuendelmeister]

Description

Hi, I'm trying to create a small kafka producer on my arm device. The code is pretty simple.

using Confluent.Kafka;
using System;

namespace KafkaTester
{
    class Program
    {
        static async System.Threading.Tasks.Task Main(string[] args)
        {
            string kafkaServer = "127.0.0.1:9092";
            var config = new ProducerConfig
            {
                BootstrapServers = kafkaServer
            };
            using (var p = new ProducerBuilder<Null, string>(config).Build())
            {
                try
                {
                    var dr = await p.ProduceAsync("first_topic", new Message<Null, string> { Value = "Message by C# testprogram" });
                    Console.WriteLine($"Delivered '{dr.Value}' to '{dr.TopicPartitionOffset}'");
                }
                catch (ProduceException<string, string> e)
                {
                    Console.WriteLine($"Delivery failed: {e.Error.Reason}");
                }
            }
        }
    }
}

More infos:
Nuget: Confluent.Kafka v1.1.0
Kafka Version: kafka_2.12-2.2.0

It works quite fine on linux and windows. Unfortunately I want to deploy it on an arm device and get this error:

Unhandled Exception: System.DllNotFoundException: Failed to load the librdkafka native library.
at Confluent.Kafka.Impl.Librdkafka.Initialize(String userSpecifiedPath)
at Confluent.Kafka.Producer2..ctor(ProducerBuilder2 builder)
at Confluent.Kafka.ProducerBuilder`2.Build()

How to reproduce

Several ways:
Create a dotnet core console application with the given code (I did this on windows).
Build it for arm self contained:

dotnet publish --self-contained -r ubuntu.16.04-arm

Then start the code (I called my Project KafkaTester)

sudo ./KafkaTester

Also I created the same project on my arm device

dotnet new console

Then I used the same code in the Program.cs with the same nuget package and built it.

dotnet restore
dotnet publish 

After running the excutable I get the same System.DllNotFoundException

Checklist

Please provide the following information:

• A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
• Confluent.Kafka nuget version.
• Apache Kafka version.
• Client configuration.
• Operating system.
• Provide logs (with "debug" : "..." as necessary in configuration).
• Provide broker log excerpts.
• Critical issue.

[mhowlett]

We don't provide a build of librdkafka for arm out-of-the-box (which Confluent.Kafka depends on). you'll need to build librdkafka yourself https://github.com/edenhill/librdkafka#build-from-source then use Confluent.Kafka.Library.Load in your code to use your custom build.

[Zuendelmeister]

Hi @mhowlett
thank you so much! I built the .so file and it works now. I could not use Library.Load though. I had to put the .so file to /lib/arm-linux-gnueabihf/ (maybe this helps another one with this problem).

Best regards,
Zuendelmeister

[ratri-git]

#778 (comment)