Add table of contents (#59)

* Add table of contents
memiiso · Feb 6, 2022 · 89cc123 · 89cc123
1 parent 8332107
commit 89cc123
Show file tree

Hide file tree

Showing 3 changed files with 20 additions and 7 deletions.
diff --git a/README.md b/README.md
@@ -2,11 +2,17 @@
 ![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)
 ![Java CI with Maven](https://github.com/memiiso/debezium-server-batch/workflows/Java%20CI%20with%20Maven/badge.svg?branch=master)
 
+# Table of contents
+* [Debezium Batch Consumers](#debezium-batch-consumers)
+  * [`sparkbatch` Consumer](#sparkbatch-consumer)
+  * [`sparkbigquerybatch` Consumer](#sparkbigquerybatch-consumer)
+  * [`bigquerybatch` Consumer](#bigquerybatch-consumer)
+* [Install from source](#install-from-source)
+
 # Debezium Batch Consumers
 
 This project adds batch consumers
-to [Debezium Server](https://debezium.io/documentation/reference/operations/debezium-server.html). Using batch consumers
-its possible to consume CDC events as mini batches
+to [Debezium Server](https://debezium.io/documentation/reference/operations/debezium-server.html). Using batch consumers its possible to consume CDC events as mini batches
 
 ## `sparkbatch` Consumer
 Consumes debezium events using spark
@@ -25,6 +31,13 @@ Consumes debezium events using spark
 | `debezium.sink.batch.destination-regexp-replace`    | ``                   | Regexp Replace part to modify destination                                           |
 | `debezium.sink.batch.batch-size-wait`               | `NoBatchSizeWait`    | Batch size wait strategy to optimize data files and upload interval. explained below. |
 
+## `sparkbigquerybatch` Consumer
+Slightly customized spark consumer that delivers the messages into Bigquery.
+
+| Config  | Default       | Description        |
+|---------|---------------|--------------------|
+| `@TODO` | `@TODO` | Destination bucket |
+
 ## `bigquerybatch` Consumer
 Consumes debezium events to Bigquery using Bigquery writer api.
 

diff --git a/...-sinks/src/main/java/io/debezium/server/batch/spark/BatchSparkBigqueryChangeConsumer.java b/...-sinks/src/main/java/io/debezium/server/batch/spark/BatchSparkBigqueryChangeConsumer.java
@@ -31,7 +31,7 @@
 import static org.apache.spark.sql.functions.*;
 
 /**
- * Implementation of the consumer that delivers the messages into Amazon S3 destination.
+ * Implementation of the consumer that delivers the messages into Bigquery destination using Spark.
  *
  * @author Ismail Simsek
  */

diff --git a/pom.xml b/pom.xml
@@ -27,7 +27,7 @@
         <skipITs>true</skipITs>
 
         <version.groovy>3.0.9</version.groovy>
-        <version.awssdk>2.15.79</version.awssdk>
+        <version.awssdk>2.17.123</version.awssdk>
         <version.assembly.plugin>3.1.1</version.assembly.plugin>
         <version.jackson>2.12.6</version.jackson>
         <version.spark>3.2.1</version.spark>
@@ -125,7 +125,7 @@
             <dependency>
                 <groupId>com.google.cloud.bigdataoss</groupId>
                 <artifactId>gcs-connector</artifactId>
-                <version>hadoop3-2.2.3</version>
+                <version>hadoop3-2.2.4</version>
             </dependency>
             <!-- Google Bom -->
             <dependency>
@@ -138,12 +138,12 @@
             <dependency>
                 <groupId>com.google.cloud</groupId>
                 <artifactId>google-cloud-bigquery</artifactId>
-                <version>2.6.2</version>
+                <version>2.8.0</version>
             </dependency>
             <dependency>
                 <groupId>com.google.cloud</groupId>
                 <artifactId>google-cloud-bigquerystorage</artifactId>
-                <version>2.8.0</version>
+                <version>2.8.4</version>
             </dependency>
             <!-- END Google -->