Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] Add *Kafka sink connector guide* #5210

Merged
merged 4 commits into from
Sep 22, 2019
Merged

[Doc] Add *Kafka sink connector guide* #5210

merged 4 commits into from
Sep 22, 2019

Conversation

Anonymitaet
Copy link
Member

Fix #5015

| `keyDeserializationClass` | String|false | org.apache.kafka.common.serialization.StringSerializer | The serializer class for Kafka producers to serialize keys.
| `valueDeserializationClass` | String|false | org.apache.kafka.common.serialization.ByteArraySerializer | The serializer class for Kafka producers to serialize values.<br/><br/>The serializer is set by a specific implementation of [`KafkaAbstractSink`](https://github.com/apache/pulsar/blob/master/pulsar-io/kafka/src/main/java/org/apache/pulsar/io/kafka/KafkaAbstractSink.java).
|`producerConfigProperties`|Map||" " (empty string)|The producer configuration properties to be passed to producers. <br/><br/>**Note: other properties specified in the connector configuration file take precedence over this configuration**.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuteng Are batchsize (line 20), maxRequestSize (line 21) and producerConfigProperties (line 25) optional or required?
I don't find this information in KafkaSinkConfig.java.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional, The required is true, indicating that it is necessary, otherwise, it is optional.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuteng thank you, I've added.


The Kafka sink connector pulls messages from Pulsar topics and persists the messages
to Kafka topics.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuteng are line7 and line8 accurate?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Anonymitaet yeah, it is correct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@murong00 thanks for your help

sasl.mechanism: "GSSAPI"
sasl.kerberos.service.name: "kafka"
acks: "all"
```
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuteng Is the format of this YAML file correct? (line 54-65)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

Copy link
Member Author

@Anonymitaet Anonymitaet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuteng could you please help review? Thank you

@Anonymitaet
Copy link
Member Author

run c++ tests

@sijie sijie added component/connect doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. labels Sep 18, 2019
@sijie sijie added this to the 2.5.0 milestone Sep 18, 2019
@Anonymitaet
Copy link
Member Author

run c++ tests

1 similar comment
@Anonymitaet
Copy link
Member Author

run c++ tests

@tuteng
Copy link
Member

tuteng commented Sep 20, 2019

run cpp tests

1 similar comment
@Anonymitaet
Copy link
Member Author

run cpp tests

@sijie sijie merged commit c728bd5 into apache:master Sep 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connector doc Your PR contains doc changes, no matter whether the changes are in markdown or code files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Doc] Update Pulsar Connector Guide
5 participants