Skip to content

shipstation/kafka-connect-fs

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

88 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is a Shipstation fork of https://github.com/mmolimar/kafka-connect-fs

It patches up a couple things for use in our environment. See commits on master of this fork to see what. At the time of this writing, it was to fix up a few dependency versions and prevent OOMs in the connector when using it against large numbers of files in S3.

Packing it up for use in Kafka Connect (assuming you have the infrastructure repo checked out at ~/Projects/):

<change things>
mvn clean package
cp target/kafka-connect-fs-0.3-SNAPSHOT-package/share/java/kafka-connect-fs/* ~/Projects/infrastructure/kubernetes/servicedefs/confluent-platform/kafka-connect/jars/kafka-connect-fs/

...then go over to ~/Projects/infrastructure/kubernetes/servicedefs/confluent-platform/kafka-connect and run ./build-and-push.sh to get it into our Kafka Connect docker image.

About

Kafka Connect FileSystem Connector

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 100.0%