Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Message not delivered if header added. #2996

Closed
3 tasks done
harshadvichare opened this issue Jul 17, 2020 · 1 comment
Closed
3 tasks done

Message not delivered if header added. #2996

harshadvichare opened this issue Jul 17, 2020 · 1 comment

Comments

@harshadvichare
Copy link

Description

Message not delivered if the header added in PHP.

How to reproduce

<?php


$conf = new RdKafka\Conf();

$conf->set('debug', 'broker,topic,msg');
$conf->set('compression.type', 'snappy');

$conf->setErrorCb(function ($kafka, $err, $reason) {
    printf("Kafka error: %s (reason: %s)\n", rd_kafka_err2str($err), $reason);
});


$conf->setDrMsgCb(function ($kafka, $message) {
    if ($message->err) {
	echo $message->errstr();
    echo ' message permanently failed to be delivered'; 
    } else {
        echo 'message successfully delivered';
    }
});

$conf->setLogCb(function ($kafka, $level, $facility, $message) {
    printf("Kafka %s: %s (level: %d)\n", $facility, $message, $level);
});

$conf->set('metadata.broker.list', 'broker-list');

$producer = new RdKafka\Producer($conf);

$topic = $producer->newTopic("test-feed");

for ($i = 0; $i < 1; $i++) {
	$message = [
    'msg' => "Message $i",
    'id' => $i,
];

$headers = [
    'SomeKey' => 'SomeValue-' . $i,
    'AnotherKey' => 'AnotherValue-' . $i,
];

$payload = json_encode($message);

   $topic->producev(RD_KAFKA_PARTITION_UA, 0, $payload, $i, $headers );
   $producer->poll(0);
}

$result = $producer->flush(10000);
?>

Checklist

  • librdkafka version: librdkafka1-1.4.4
  • phprdkafka version: php-rdkafka-4.0.3-
  • Apache Kafka version:0.11.0

Log

Kafka BROKER: [thrd:app]: kafka01.test.com:50360/bootstrap: Added new broker with NodeId -1 (level: 7)
Kafka BRKMAIN: [thrd:kafka01.test.com:50360/bootstrap]: kafka01.test.com:50360/bootstrap: Enter main broker thread (level: 7)
Kafka BRKMAIN: [thrd::0/internal]: :0/internal: Enter main broker thread (level: 7)
Kafka BROKER: [thrd:app]: kafka02.test.com:50360/bootstrap: Added new broker with NodeId -1 (level: 7)
Kafka BROKER: [thrd:app]: kafka03.test.com:50360/bootstrap: Added new broker with NodeId -1 (level: 7)
Kafka BRKMAIN: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Enter main broker thread (level: 7)
Kafka CONNECT: [thrd:app]: kafka02.test.com:50360/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s)) (level: 7)
Kafka BRKMAIN: [thrd:kafka03.test.com:50360/bootstrap]: kafka03.test.com:50360/bootstrap: Enter main broker thread (level: 7)
Kafka CONNECT: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Received CONNECT op (level: 7)
Kafka STATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Broker changed state INIT -> TRY_CONNECT (level: 7)
Kafka CONNECT: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: broker in state TRY_CONNECT connecting (level: 7)
Kafka STATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Broker changed state TRY_CONNECT -> CONNECT (level: 7)
Kafka INIT: [thrd:app]: librdkafka v1.4.4 (0x10404ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,sasl_oauthbearer, GCC GXX PKGCONFIG INSTALL GNULD LDS LIBDL PLUGINS ZLIB SSL SASL_CYRUS HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x46) (level: 7)
Kafka TOPIC: [thrd:app]: New local topic: test-feed (level: 7)
Kafka TOPPARNEW: [thrd:app]: NEW test-feed [-1] 0x204cd20 (at rd_kafka_topic_new0:397) (level: 7)
Kafka CONNECT: [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: leader query (level: 7)
Kafka CONNECT: [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: leader query (level: 7)
Kafka CONNECT: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Connecting to ipv4#10.44.81.113:50360 (plaintext) with socket 11 (level: 7)
Kafka CONNECT: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Connected to ipv4#10.44.81.113:50360 (level: 7)
Kafka CONNECTED: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Connected (#1) (level: 7)
Kafka FEATURE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion (level: 7)
Kafka STATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY (level: 7)
Kafka PROTOERR: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Protocol parse failure for ApiVersion v3 at 3/6 (rd_kafka_handle_ApiVersion:1915) (incorrect broker.version.fallback?) (level: 7)
Kafka PROTOERR: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: ApiArrayCnt -1 out of range (level: 7)
Kafka APIVERSION: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0 (level: 7)
Kafka FEATURE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer (level: 7)
Kafka STATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: Broker changed state APIVERSION_QUERY -> UP (level: 7)
Kafka BROKER: [thrd:main]: kafka08.test.com:50360/8: Added new broker with NodeId 8 (level: 7)
Kafka BROKER: [thrd:main]: kafka05.test.com:50360/5: Added new broker with NodeId 5 (level: 7)
Kafka BRKMAIN: [thrd:kafka08.test.com:50360/8]: kafka08.test.com:50360/8: Enter main broker thread (level: 7)
Kafka BROKER: [thrd:main]: kafka04.test.com:50360/4: Added new broker with NodeId 4 (level: 7)
Kafka BRKMAIN: [thrd:kafka05.test.com:50360/5]: kafka05.test.com:50360/5: Enter main broker thread (level: 7)
Kafka BRKMAIN: [thrd:kafka04.test.com:50360/4]: kafka04.test.com:50360/4: Enter main broker thread (level: 7)
Kafka BROKER: [thrd:main]: kafka07.test.com:50360/7: Added new broker with NodeId 7 (level: 7)
Kafka BROKER: [thrd:main]: kafka09.test.com:50360/9: Added new broker with NodeId 9 (level: 7)
Kafka BROKER: [thrd:main]: kafka06.test.com:50360/6: Added new broker with NodeId 6 (level: 7)
Kafka BRKMAIN: [thrd:kafka07.test.com:50360/7]: kafka07.test.com:50360/7: Enter main broker thread (level: 7)
Kafka STATE: [thrd:main]: Topic test-feed changed state unknown -> exists (level: 7)
Kafka PARTCNT: [thrd:main]: Topic test-feed partition count changed from 0 to 10 (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [0] 0x7f19fc009f10 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [1] 0x7f19fc00a500 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka BRKMAIN: [thrd:kafka09.test.com:50360/9]: kafka09.test.com:50360/9: Enter main broker thread (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [2] 0x7f19fc00abb0 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka BRKMAIN: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Enter main broker thread (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [3] 0x7f19fc00b260 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [4] 0x7f19fc00b910 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [5] 0x7f19fc00bfc0 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [6] 0x7f19fc00c670 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [7] 0x7f19fc00cd20 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [8] 0x7f19fc00d3d0 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka TOPPARNEW: [thrd:main]: NEW test-feed [9] 0x7f19fc00da80 (at rd_kafka_topic_partition_cnt_update:713) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 0 Leader 6 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [0]: delegate to broker kafka06.test.com:50360/6 (rktp 0x7f19fc009f10, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [0]: delegating to broker kafka06.test.com:50360/6 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [0] 0x7f19fc009f10 from (none) to kafka06.test.com:50360/6 (sending PARTITION_JOIN to kafka06.test.com:50360/6) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 1 Leader 7 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [1]: delegate to broker kafka07.test.com:50360/7 (rktp 0x7f19fc00a500, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [1]: delegating to broker kafka07.test.com:50360/7 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [1] 0x7f19fc00a500 from (none) to kafka07.test.com:50360/7 (sending PARTITION_JOIN to kafka07.test.com:50360/7) (level: 7)
Kafka TOPBRK: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Topic test-feed [0]: joining broker (rktp 0x7f19fc009f10, 0 message(s) queued) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 2 Leader 8 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [2]: delegate to broker kafka08.test.com:50360/8 (rktp 0x7f19fc00abb0, term 0, ref 2) (level: 7)
Kafka FETCHADD: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Added test-feed [0] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [2]: delegating to broker kafka08.test.com:50360/8 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [2] 0x7f19fc00abb0 from (none) to kafka08.test.com:50360/8 (sending PARTITION_JOIN to kafka08.test.com:50360/8) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 3 Leader 9 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [3]: delegate to broker kafka09.test.com:50360/9 (rktp 0x7f19fc00b260, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [3]: delegating to broker kafka09.test.com:50360/9 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [3] 0x7f19fc00b260 from (none) to kafka09.test.com:50360/9 (sending PARTITION_JOIN to kafka09.test.com:50360/9) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 4 Leader 1 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [4]: delegate to broker (none) (rktp 0x7f19fc00b910, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [4]: delegating to broker :0/internal for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [4] 0x7f19fc00b910 from (none) to :0/internal (sending PARTITION_JOIN to :0/internal) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 5 Leader 2 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [5]: delegate to broker (none) (rktp 0x7f19fc00bfc0, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [5]: delegating to broker :0/internal for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [5] 0x7f19fc00bfc0 from (none) to :0/internal (sending PARTITION_JOIN to :0/internal) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 6 Leader 3 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [6]: delegate to broker (none) (rktp 0x7f19fc00c670, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [6]: delegating to broker :0/internal for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [6] 0x7f19fc00c670 from (none) to :0/internal (sending PARTITION_JOIN to :0/internal) (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 7 Leader 4 (level: 7)
Kafka TOPBRK: [thrd:kafka07.test.com:50360/7]: kafka07.test.com:50360/7: Topic test-feed [1]: joining broker (rktp 0x7f19fc00a500, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd:kafka07.test.com:50360/7]: kafka07.test.com:50360/7: Added test-feed [1] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka TOPBRK: [thrd:kafka08.test.com:50360/8]: kafka08.test.com:50360/8: Topic test-feed [2]: joining broker (rktp 0x7f19fc00abb0, 0 message(s) queued) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [7]: delegate to broker kafka04.test.com:50360/4 (rktp 0x7f19fc00cd20, term 0, ref 2) (level: 7)
Kafka FETCHADD: [thrd:kafka08.test.com:50360/8]: kafka08.test.com:50360/8: Added test-feed [2] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka TOPBRK: [thrd::0/internal]: :0/internal: Topic test-feed [4]: joining broker (rktp 0x7f19fc00b910, 0 message(s) queued) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [7]: delegating to broker kafka04.test.com:50360/4 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [7] 0x7f19fc00cd20 from (none) to kafka04.test.com:50360/4 (sending PARTITION_JOIN to kafka04.test.com:50360/4) (level: 7)
Kafka FETCHADD: [thrd::0/internal]: :0/internal: Added test-feed [4] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka TOPBRK: [thrd::0/internal]: :0/internal: Topic test-feed [5]: joining broker (rktp 0x7f19fc00bfc0, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd::0/internal]: :0/internal: Added test-feed [5] to active list (2 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka TOPBRK: [thrd:kafka09.test.com:50360/9]: kafka09.test.com:50360/9: Topic test-feed [3]: joining broker (rktp 0x7f19fc00b260, 0 message(s) queued) (level: 7)
Kafka TOPBRK: [thrd::0/internal]: :0/internal: Topic test-feed [6]: joining broker (rktp 0x7f19fc00c670, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd:kafka09.test.com:50360/9]: kafka09.test.com:50360/9: Added test-feed [3] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 8 Leader 5 (level: 7)
Kafka FETCHADD: [thrd::0/internal]: :0/internal: Added test-feed [6] to active list (3 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [8]: delegate to broker kafka05.test.com:50360/5 (rktp 0x7f19fc00d3d0, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [8]: delegating to broker kafka05.test.com:50360/5 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka TOPBRK: [thrd:kafka04.test.com:50360/4]: kafka04.test.com:50360/4: Topic test-feed [7]: joining broker (rktp 0x7f19fc00cd20, 0 message(s) queued) (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [8] 0x7f19fc00d3d0 from (none) to kafka05.test.com:50360/5 (sending PARTITION_JOIN to kafka05.test.com:50360/5) (level: 7)
Kafka FETCHADD: [thrd:kafka04.test.com:50360/4]: kafka04.test.com:50360/4: Added test-feed [7] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka METADATA: [thrd:main]: Topic test-feed partition 9 Leader 6 (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [9]: delegate to broker kafka06.test.com:50360/6 (rktp 0x7f19fc00da80, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:main]: test-feed [9]: delegating to broker kafka06.test.com:50360/6 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:main]: Migrating topic test-feed [9] 0x7f19fc00da80 from (none) to kafka06.test.com:50360/6 (sending PARTITION_JOIN to kafka06.test.com:50360/6) (level: 7)
Kafka TOPBRK: [thrd:kafka05.test.com:50360/5]: kafka05.test.com:50360/5: Topic test-feed [8]: joining broker (rktp 0x7f19fc00d3d0, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd:kafka05.test.com:50360/5]: kafka05.test.com:50360/5: Added test-feed [8] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka PARTCNT: [thrd:main]: Partitioning 1 unassigned messages in topic test-feed to 10 partitions (level: 7)
Kafka UAS: [thrd:main]: 1/1 messages were partitioned in topic test-feed (level: 7)
Kafka TOPBRK: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Topic test-feed [9]: joining broker (rktp 0x7f19fc00da80, 1 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Added test-feed [9] to active list (2 entries, opv 0, 1 messages queued): joining (level: 7)
Kafka TOPPAR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9] 1 message(s) queued but broker not up (level: 7)
Kafka STATE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Broker changed state INIT -> TRY_CONNECT (level: 7)
Kafka CONNECT: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: broker in state TRY_CONNECT connecting (level: 7)
Kafka UPDATE: [thrd:kafka03.test.com:50360/bootstrap]: kafka03.test.com:50360/bootstrap: NodeId changed from -1 to 3 (level: 7)
Kafka STATE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Broker changed state TRY_CONNECT -> CONNECT (level: 7)
Kafka UPDATE: [thrd:kafka03.test.com:50360/bootstrap]: kafka03.test.com:50360/3: Name changed from kafka03.test.com:50360/bootstrap to kafka03.test.com:50360/3 (level: 7)
Kafka METADATA: [thrd:main]: kafka02.test.com:50360/bootstrap: 1/1 requested topic(s) seen in metadata (level: 7)
Kafka UPDATE: [thrd:kafka01.test.com:50360/bootstrap]: kafka01.test.com:50360/bootstrap: NodeId changed from -1 to 1 (level: 7)
Kafka UPDATE: [thrd:kafka01.test.com:50360/bootstrap]: kafka01.test.com:50360/1: Name changed from kafka01.test.com:50360/bootstrap to kafka01.test.com:50360/1 (level: 7)
Kafka TOPICUPD: [thrd:kafka01.test.com:50360/bootstrap]: Topic test-feed [4]: migrating from broker -1 to 1 (leader is 1): broker node information updated (level: 7)
Kafka BRKDELGT: [thrd:kafka01.test.com:50360/bootstrap]: test-feed [4]: delegate to broker kafka01.test.com:50360/1 (rktp 0x7f19fc00b910, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:kafka01.test.com:50360/bootstrap]: test-feed [4]: no longer delegated to broker :0/internal (level: 7)
Kafka BRKDELGT: [thrd:kafka01.test.com:50360/bootstrap]: test-feed [4]: delegating to broker kafka01.test.com:50360/1 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:kafka01.test.com:50360/bootstrap]: Migrating topic test-feed [4] 0x7f19fc00b910 from :0/internal to kafka01.test.com:50360/1 (sending PARTITION_LEAVE to :0/internal) (level: 7)
Kafka LEADER: [thrd:kafka01.test.com:50360/bootstrap]: kafka01.test.com:50360/1: Mapped 1 partition(s) to broker (level: 7)
Kafka CLUSTERID: [thrd:main]: kafka02.test.com:50360/bootstrap: ClusterId update "" -> "Vet_TRy-QdqT2t63yDAfdA" (level: 7)
Kafka CONTROLLERID: [thrd:main]: kafka02.test.com:50360/bootstrap: ControllerId update -1 -> 3 (level: 7)
Kafka UPDATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/bootstrap: NodeId changed from -1 to 2 (level: 7)
Kafka UPDATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/2: Name changed from kafka02.test.com:50360/bootstrap to kafka02.test.com:50360/2 (level: 7)
Kafka TOPICUPD: [thrd:kafka02.test.com:50360/bootstrap]: Topic test-feed [5]: migrating from broker -1 to 2 (leader is 2): broker node information updated (level: 7)
Kafka BRKDELGT: [thrd:kafka02.test.com:50360/bootstrap]: test-feed [5]: delegate to broker kafka02.test.com:50360/2 (rktp 0x7f19fc00bfc0, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:kafka02.test.com:50360/bootstrap]: test-feed [5]: no longer delegated to broker :0/internal (level: 7)
Kafka BRKDELGT: [thrd:kafka02.test.com:50360/bootstrap]: test-feed [5]: delegating to broker kafka02.test.com:50360/2 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:kafka02.test.com:50360/bootstrap]: Migrating topic test-feed [5] 0x7f19fc00bfc0 from :0/internal to kafka02.test.com:50360/2 (sending PARTITION_LEAVE to :0/internal) (level: 7)
Kafka LEADER: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/2: Mapped 1 partition(s) to broker (level: 7)
Kafka STATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/2: Broker changed state UP -> UPDATE (level: 7)
Kafka STATE: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/2: Broker changed state UPDATE -> UP (level: 7)
Kafka TOPICUPD: [thrd:kafka03.test.com:50360/bootstrap]: Topic test-feed [6]: migrating from broker -1 to 3 (leader is 3): broker node information updated (level: 7)
Kafka BRKDELGT: [thrd:kafka03.test.com:50360/bootstrap]: test-feed [6]: delegate to broker kafka03.test.com:50360/3 (rktp 0x7f19fc00c670, term 0, ref 2) (level: 7)
Kafka BRKDELGT: [thrd:kafka03.test.com:50360/bootstrap]: test-feed [6]: no longer delegated to broker :0/internal (level: 7)
Kafka TOPBRK: [thrd::0/internal]: :0/internal: Topic test-feed [4]: leaving broker (0 messages in xmitq, next broker kafka01.test.com:50360/1, rktp 0x7f19fc00b910) (level: 7)
Kafka FETCHADD: [thrd::0/internal]: :0/internal: Removed test-feed [4] from active list (2 entries, opv 0): leaving (level: 7)
Kafka BRKDELGT: [thrd:kafka03.test.com:50360/bootstrap]: test-feed [6]: delegating to broker kafka03.test.com:50360/3 for partition with 0 messages (0 bytes) queued (level: 7)
Kafka BRKMIGR: [thrd:kafka03.test.com:50360/bootstrap]: Migrating topic test-feed [6] 0x7f19fc00c670 from :0/internal to kafka03.test.com:50360/3 (sending PARTITION_LEAVE to :0/internal) (level: 7)
Kafka LEADER: [thrd:kafka03.test.com:50360/bootstrap]: kafka03.test.com:50360/3: Mapped 1 partition(s) to broker (level: 7)
Kafka TOPBRK: [thrd::0/internal]: :0/internal: Topic test-feed [5]: leaving broker (0 messages in xmitq, next broker kafka02.test.com:50360/2, rktp 0x7f19fc00bfc0) (level: 7)
Kafka TOPBRK: [thrd:kafka01.test.com:50360/bootstrap]: kafka01.test.com:50360/1: Topic test-feed [4]: joining broker (rktp 0x7f19fc00b910, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd::0/internal]: :0/internal: Removed test-feed [5] from active list (1 entries, opv 0): leaving (level: 7)
Kafka FETCHADD: [thrd:kafka01.test.com:50360/bootstrap]: kafka01.test.com:50360/1: Added test-feed [4] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka TOPBRK: [thrd::0/internal]: :0/internal: Topic test-feed [6]: leaving broker (0 messages in xmitq, next broker kafka03.test.com:50360/3, rktp 0x7f19fc00c670) (level: 7)
Kafka FETCHADD: [thrd::0/internal]: :0/internal: Removed test-feed [6] from active list (0 entries, opv 0): leaving (level: 7)
Kafka TOPBRK: [thrd:kafka03.test.com:50360/bootstrap]: kafka03.test.com:50360/3: Topic test-feed [6]: joining broker (rktp 0x7f19fc00c670, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd:kafka03.test.com:50360/bootstrap]: kafka03.test.com:50360/3: Added test-feed [6] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka TOPBRK: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/2: Topic test-feed [5]: joining broker (rktp 0x7f19fc00bfc0, 0 message(s) queued) (level: 7)
Kafka FETCHADD: [thrd:kafka02.test.com:50360/bootstrap]: kafka02.test.com:50360/2: Added test-feed [5] to active list (1 entries, opv 0, 0 messages queued): joining (level: 7)
Kafka CONNECT: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Connecting to ipv4#10.44.81.118:50360 (plaintext) with socket 20 (level: 7)
Kafka TOPPAR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9] 1 message(s) queued but broker not up (level: 7)
Kafka TOPPAR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9] 1 message(s) queued but broker not up (level: 7)
Kafka CONNECT: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Connected to ipv4#10.44.81.118:50360 (level: 7)
Kafka CONNECTED: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Connected (#1) (level: 7)
Kafka FEATURE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Updated enabled protocol features +ApiVersion to ApiVersion (level: 7)
Kafka STATE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Broker changed state CONNECT -> APIVERSION_QUERY (level: 7)
Kafka TOPPAR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9] 1 message(s) queued but broker not up (level: 7)
Kafka PROTOERR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Protocol parse failure for ApiVersion v3 at 3/6 (rd_kafka_handle_ApiVersion:1915) (incorrect broker.version.fallback?) (level: 7)
Kafka PROTOERR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: ApiArrayCnt -1 out of range (level: 7)
Kafka APIVERSION: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0 (level: 7)
Kafka TOPPAR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9] 1 message(s) queued but broker not up (level: 7)
Kafka FEATURE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer (level: 7)
Kafka STATE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: Broker changed state APIVERSION_QUERY -> UP (level: 7)
Kafka PRODUCE: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9]: Produce MessageSet with 1 message(s) (142 bytes, ApiVersion 3, MsgVersion 2, MsgId 0, BaseSeq -1, PID{Invalid}, uncompressed) (level: 7)
Kafka REQERR: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: ProduceRequest failed: Unknown broker error: actions Permanent,MsgNotPersisted (level: 7)
Kafka MSGSET: [thrd:kafka06.test.com:50360/6]: kafka06.test.com:50360/6: test-feed [9]: MessageSet with 1 message(s) (MsgId 0, BaseSeq -1) encountered error: Unknown broker error (actions Permanent,MsgNotPersisted) (level: 7)
@edenhill
Copy link
Contributor

My guess would be that the topic is configured (on the broker) with an older (pre v0.11) message.format.version that does not support message headers.
Check the broker logs for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@edenhill @harshadvichare