diff --git a/content/rdi/reference/cli/redis-di-add-context.md b/content/rdi/reference/cli/redis-di-add-context.md index 61238f7ee8b..093cd198c65 100644 --- a/content/rdi/reference/cli/redis-di-add-context.md +++ b/content/rdi/reference/cli/redis-di-add-context.md @@ -24,12 +24,12 @@ Usage: redis-di add-context [OPTIONS] CONTEXT_NAME - Default: `none` - Usage: `context-name` -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `cluster_host` (REQUIRED): @@ -119,7 +119,7 @@ Usage: redis-di add-context [OPTIONS] CONTEXT_NAME Adds a new context Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --cluster-host TEXT Host/IP of Redis Enterprise Cluster (service name in case of k8s) [required] diff --git a/content/rdi/reference/cli/redis-di-create.md b/content/rdi/reference/cli/redis-di-create.md index cc9d98d8eb0..83f9d28757c 100644 --- a/content/rdi/reference/cli/redis-di-create.md +++ b/content/rdi/reference/cli/redis-di-create.md @@ -18,12 +18,12 @@ Usage: redis-di create [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `silent`: @@ -33,14 +33,6 @@ Usage: redis-di create [OPTIONS] Silent install. Do not prompt to enter missing parameters -- `no_configure`: - - - Type: BOOL - - Default: `false` - - Usage: `--no-configure` - - Do not install RDI Engine to the RDI Database - - `cluster_host` (REQUIRED): - Type: STRING @@ -113,13 +105,13 @@ Usage: redis-di create [OPTIONS] In-memory database replication -- `redisgears_module`: +- `with_redisgears`: - - Type: STRING - - Default: `` - - Usage: `--redisgears-module` + - Type: BOOL + - Default: `false` + - Usage: `--with-redisgears` - RedisGears module file + Include RedisGears module in the RDI database - `with_rejson`: @@ -145,12 +137,10 @@ Usage: redis-di create [OPTIONS] Creates the RDI Database instance Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --silent Silent install. Do not prompt to enter missing parameters - --no-configure Do not install RDI Engine to the RDI - Database --cluster-host TEXT Host/IP of Redis Enterprise Cluster (service name in case of k8s) [required] --cluster-api-port INTEGER RANGE @@ -169,6 +159,5 @@ Options: --rdi-shards INTEGER RANGE Number of database server-side shards [x>=1] --replication In-memory database replication - --redisgears-module TEXT RedisGears module file --help Show this message and exit. ``` diff --git a/content/rdi/reference/cli/redis-di-delete-all-contexts.md b/content/rdi/reference/cli/redis-di-delete-all-contexts.md index f591440d437..0d76320a344 100644 --- a/content/rdi/reference/cli/redis-di-delete-all-contexts.md +++ b/content/rdi/reference/cli/redis-di-delete-all-contexts.md @@ -18,12 +18,12 @@ Usage: redis-di delete-all-contexts [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `force`: @@ -50,7 +50,7 @@ Usage: redis-di delete-all-contexts [OPTIONS] Deletes all contexts Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] -f, --force Force operation. skips verification prompts --help Show this message and exit. diff --git a/content/rdi/reference/cli/redis-di-delete-context.md b/content/rdi/reference/cli/redis-di-delete-context.md index 6f3bd840ef1..70dce67a01b 100644 --- a/content/rdi/reference/cli/redis-di-delete-context.md +++ b/content/rdi/reference/cli/redis-di-delete-context.md @@ -18,12 +18,12 @@ Usage: redis-di delete-context [OPTIONS] CONTEXT_NAME ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `context_name` (REQUIRED): @@ -56,7 +56,7 @@ Usage: redis-di delete-context [OPTIONS] CONTEXT_NAME Deletes a context Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] -f, --force Force operation. skips verification prompts --help Show this message and exit. diff --git a/content/rdi/reference/cli/redis-di-delete.md b/content/rdi/reference/cli/redis-di-delete.md index 54ac77e94e6..7e50b2ee477 100644 --- a/content/rdi/reference/cli/redis-di-delete.md +++ b/content/rdi/reference/cli/redis-di-delete.md @@ -18,12 +18,12 @@ Usage: redis-di delete [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `cluster_host` (REQUIRED): @@ -146,7 +146,7 @@ Usage: redis-di delete [OPTIONS] Deletes RDI database permanently Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --cluster-host TEXT Host/IP of Redis Enterprise Cluster (service name in case of k8s) [required] diff --git a/content/rdi/reference/cli/redis-di-deploy.md b/content/rdi/reference/cli/redis-di-deploy.md index 777f2e95f3b..7b2eba586bb 100644 --- a/content/rdi/reference/cli/redis-di-deploy.md +++ b/content/rdi/reference/cli/redis-di-deploy.md @@ -18,12 +18,12 @@ Usage: redis-di deploy [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -113,7 +113,7 @@ Usage: redis-di deploy [OPTIONS] Deploys the RDI configurations including target Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-describe-job.md b/content/rdi/reference/cli/redis-di-describe-job.md index 207e515966c..fe983a28044 100644 --- a/content/rdi/reference/cli/redis-di-describe-job.md +++ b/content/rdi/reference/cli/redis-di-describe-job.md @@ -18,12 +18,12 @@ Usage: redis-di describe-job [OPTIONS] JOB_NAME ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `job_name` (REQUIRED): @@ -111,7 +111,7 @@ Usage: redis-di describe-job [OPTIONS] JOB_NAME Describes a transformation engine's job Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-dump-support-package.md b/content/rdi/reference/cli/redis-di-dump-support-package.md index 00dcb8bc6d4..c7b0436e100 100644 --- a/content/rdi/reference/cli/redis-di-dump-support-package.md +++ b/content/rdi/reference/cli/redis-di-dump-support-package.md @@ -18,12 +18,12 @@ Usage: redis-di dump-support-package [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -119,7 +119,7 @@ Usage: redis-di dump-support-package [OPTIONS] - Default: `10` - Usage: `--max-change-records` - Maximum traced change records per shard + Maximum traced change records - `trace_only_rejected`: @@ -145,7 +145,7 @@ Usage: redis-di dump-support-package [OPTIONS] Dumps RDI support package Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; @@ -163,8 +163,7 @@ Options: --trace-timeout INTEGER RANGE Stops the trace after exceeding this timeout (in seconds) [1<=x<=600] --max-change-records INTEGER RANGE - Maximum traced change records per shard - [x>=1] + Maximum traced change records [x>=1] --trace-only-rejected Trace only rejected change records --help Show this message and exit. ``` diff --git a/content/rdi/reference/cli/redis-di-get-rejected.md b/content/rdi/reference/cli/redis-di-get-rejected.md index 7ef6fc255ae..678964f95cc 100644 --- a/content/rdi/reference/cli/redis-di-get-rejected.md +++ b/content/rdi/reference/cli/redis-di-get-rejected.md @@ -18,12 +18,12 @@ Usage: redis-di get-rejected [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -130,7 +130,7 @@ Usage: redis-di get-rejected [OPTIONS] Returns all the stored rejected entries Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-configure.md b/content/rdi/reference/cli/redis-di-install-collector.md similarity index 82% rename from content/rdi/reference/cli/redis-di-configure.md rename to content/rdi/reference/cli/redis-di-install-collector.md index d6c125c4e3b..23abbb83eed 100644 --- a/content/rdi/reference/cli/redis-di-configure.md +++ b/content/rdi/reference/cli/redis-di-install-collector.md @@ -1,29 +1,29 @@ --- -Title: redis-di configure -linkTitle: redis-di configure -description: Configures the RDI Database so it is ready to process data +Title: redis-di install-collector +linkTitle: redis-di install-collector +description: Installs RDI Redis Collector on Redis BDB with RedisGears weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- -Configures the RDI Database so it is ready to process data +Installs RDI Redis Collector on Redis BDB with RedisGears ## Usage ``` -Usage: redis-di configure [OPTIONS] +Usage: redis-di install-collector [OPTIONS] ``` ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -100,12 +100,12 @@ Usage: redis-di configure [OPTIONS] ## CLI help ``` -Usage: redis-di configure [OPTIONS] +Usage: redis-di install-collector [OPTIONS] - Configures the RDI Database so it is ready to process data + Installs RDI Redis Collector on Redis BDB with RedisGears Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-install-redisgears.md b/content/rdi/reference/cli/redis-di-install-redisgears.md new file mode 100644 index 00000000000..4ee79fd5115 --- /dev/null +++ b/content/rdi/reference/cli/redis-di-install-redisgears.md @@ -0,0 +1,97 @@ +--- +Title: redis-di install-redisgears +linkTitle: redis-di install-redisgears +description: Installs RedisGears module on Redis cluster +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Installs RedisGears module on Redis cluster + +## Usage + +``` +Usage: redis-di install-redisgears [OPTIONS] +``` + +## Options + +- `log_level`: + + - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) + - Default: `info` + - Usage: `--log-level +-l` + +- `cluster_host` (REQUIRED): + + - Type: STRING + - Default: `none` + - Usage: `--cluster-host` + + Host/IP of Redis Enterprise Cluster (service name in case of k8s) + +- `cluster_api_port` (REQUIRED): + + - Type: + - Default: `9443` + - Usage: `--cluster-api-port` + + API Port of Redis Enterprise Cluster + +- `cluster_user` (REQUIRED): + + - Type: STRING + - Default: `none` + - Usage: `--cluster-user` + + Redis Enterprise Cluster username with either DB Member, Cluster Member or Cluster Admin roles + +- `cluster_password`: + + - Type: STRING + - Default: `none` + - Usage: `--cluster-password` + + Redis Enterprise Cluster Password + +- `redisgears_module` (REQUIRED): + + - Type: STRING + - Default: `none` + - Usage: `--redisgears-module` + + RedisGears module file + +- `help`: + + - Type: BOOL + - Default: `false` + - Usage: `--help` + + Show this message and exit. + +## CLI help + +``` +Usage: redis-di install-redisgears [OPTIONS] + + Installs RedisGears module on Redis cluster + +Options: + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] + [default: INFO] + --cluster-host TEXT Host/IP of Redis Enterprise Cluster (service + name in case of k8s) [required] + --cluster-api-port INTEGER RANGE + API Port of Redis Enterprise Cluster + [default: 9443; 1000<=x<=65535; required] + --cluster-user TEXT Redis Enterprise Cluster username with + either DB Member, Cluster Member or Cluster + Admin roles [required] + --cluster-password TEXT Redis Enterprise Cluster Password + --redisgears-module TEXT RedisGears module file [required] + --help Show this message and exit. +``` diff --git a/content/rdi/reference/cli/redis-di-install.md b/content/rdi/reference/cli/redis-di-install.md new file mode 100644 index 00000000000..7f9e426c99a --- /dev/null +++ b/content/rdi/reference/cli/redis-di-install.md @@ -0,0 +1,58 @@ +--- +Title: redis-di install +linkTitle: redis-di install +description: Installs RDI +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Installs RDI + +## Usage + +``` +Usage: redis-di install [OPTIONS] +``` + +## Options + +- `log_level`: + + - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) + - Default: `info` + - Usage: `--log-level +-l` + +- `file`: + + - Type: + - Default: `none` + - Usage: `-f +--file` + + Path to a YAML configuration file for silent installation + +- `help`: + + - Type: BOOL + - Default: `false` + - Usage: `--help` + + Show this message and exit. + +## CLI help + +``` +Usage: redis-di install [OPTIONS] + + Installs RDI + +Options: + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] + [default: INFO] + -f, --file FILE Path to a YAML configuration file for silent + installation + --help Show this message and exit. +``` diff --git a/content/rdi/reference/cli/redis-di-list-jobs.md b/content/rdi/reference/cli/redis-di-list-jobs.md index 9e07c72f84c..e4ec64e01a2 100644 --- a/content/rdi/reference/cli/redis-di-list-jobs.md +++ b/content/rdi/reference/cli/redis-di-list-jobs.md @@ -18,12 +18,12 @@ Usage: redis-di list-jobs [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -105,7 +105,7 @@ Usage: redis-di list-jobs [OPTIONS] Lists transformation engine's jobs Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-monitor.md b/content/rdi/reference/cli/redis-di-monitor.md index ab700be77a8..966745ed5f1 100644 --- a/content/rdi/reference/cli/redis-di-monitor.md +++ b/content/rdi/reference/cli/redis-di-monitor.md @@ -18,12 +18,12 @@ Usage: redis-di monitor [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -121,7 +121,7 @@ Usage: redis-di monitor [OPTIONS] Monitors RDI by collecting metrics and exporting to Prometheus Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-reset.md b/content/rdi/reference/cli/redis-di-reset.md index c8fa5757d6f..7c31eaef27b 100644 --- a/content/rdi/reference/cli/redis-di-reset.md +++ b/content/rdi/reference/cli/redis-di-reset.md @@ -18,12 +18,12 @@ Usage: redis-di reset [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -98,6 +98,14 @@ Usage: redis-di reset [OPTIONS] Force operation. skips verification prompts +- `pause_for_confirmation`: + + - Type: BOOL + - Default: `false` + - Usage: `--pause-for-confirmation` + + Pause for user confirmation if manual shutdown of collector required + - `help`: - Type: BOOL @@ -114,7 +122,7 @@ Usage: redis-di reset [OPTIONS] Resets the pipeline into initial full sync mode Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; @@ -127,5 +135,7 @@ Options: --rdi-key-password TEXT Password for unlocking an encrypted private key -f, --force Force operation. skips verification prompts + --pause-for-confirmation Pause for user confirmation if manual + shutdown of collector required --help Show this message and exit. ``` diff --git a/content/rdi/reference/cli/redis-di-scaffold.md b/content/rdi/reference/cli/redis-di-scaffold.md index 03e550f8108..0da3bc33298 100644 --- a/content/rdi/reference/cli/redis-di-scaffold.md +++ b/content/rdi/reference/cli/redis-di-scaffold.md @@ -18,12 +18,12 @@ Usage: redis-di scaffold [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `db_type` (REQUIRED): @@ -76,7 +76,7 @@ Usage: redis-di scaffold [OPTIONS] Redis) Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --db-type [cassandra|db2|mongodb|mysql|oracle|postgresql|redis|sqlserver] DB type [required] diff --git a/content/rdi/reference/cli/redis-di-set-context.md b/content/rdi/reference/cli/redis-di-set-context.md index c475fa5385a..816eda1816e 100644 --- a/content/rdi/reference/cli/redis-di-set-context.md +++ b/content/rdi/reference/cli/redis-di-set-context.md @@ -18,12 +18,12 @@ Usage: redis-di set-context [OPTIONS] CONTEXT_NAME ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `context_name` (REQUIRED): @@ -47,7 +47,7 @@ Usage: redis-di set-context [OPTIONS] CONTEXT_NAME Sets a context to be the active one Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --help Show this message and exit. ``` diff --git a/content/rdi/reference/cli/redis-di-set-secret.md b/content/rdi/reference/cli/redis-di-set-secret.md index d6cb962be99..7b72132004b 100644 --- a/content/rdi/reference/cli/redis-di-set-secret.md +++ b/content/rdi/reference/cli/redis-di-set-secret.md @@ -1,141 +1,46 @@ --- Title: redis-di set-secret linkTitle: redis-di set-secret -description: Writes a secret to Redis secret store +description: Creates a secret of a specified key weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- -Writes a secret to Redis secret store +Creates a secret of a specified key ## Usage ``` -Usage: redis-di set-secret [OPTIONS] +Usage: redis-di set-secret [OPTIONS] {source-db-username|source-db- + password|target-db-username|target-db- + password|source-db-cacert|source-db-cert|source-db- + key|source-db-key-password|target-db-cacert|target- + db-cert|target-db-key|target-db-key-password} + [VALUE] ``` ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` -- `cluster_host` (REQUIRED): +- `key` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--cluster-host` - - Host/IP of Redis Enterprise Cluster (service name in case of k8s) - -- `cluster_api_port` (REQUIRED): - - - Type: - - Default: `9443` - - Usage: `--cluster-api-port` - - API Port of Redis Enterprise Cluster - -- `cluster_user` (REQUIRED): - - - Type: STRING - - Default: `none` - - Usage: `--cluster-user` - - Redis Enterprise Cluster username with either DB Member, Cluster Member or Cluster Admin roles - -- `cluster_password`: - - - Type: STRING - - Default: `none` - - Usage: `--cluster-password` - - Redis Enterprise Cluster Password - -- `rdi_host` (REQUIRED): - - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - - Type: STRING + - Type: Choice(['source-db-username', 'source-db-password', 'target-db-username', 'target-db-password', 'source-db-cacert', 'source-db-cert', 'source-db-key', 'source-db-key-password', 'target-db-cacert', 'target-db-cert', 'target-db-key', 'target-db-key-password']) - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with + - Usage: `key` -- `rdi_key_password`: +- `value`: - Type: STRING - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `secret_name` (REQUIRED): - - - Type: STRING - - Default: `none` - - Usage: `--secret-name` - - The name of the secret - -- `secret_value` (REQUIRED): - - - Type: STRING - - Default: `none` - - Usage: `--secret-value` - - The value of the secret + - Usage: `value` - `help`: @@ -148,33 +53,17 @@ Usage: redis-di set-secret [OPTIONS] ## CLI help ``` -Usage: redis-di set-secret [OPTIONS] +Usage: redis-di set-secret [OPTIONS] {source-db-username|source-db- + password|target-db-username|target-db- + password|source-db-cacert|source-db-cert|source-db- + key|source-db-key-password|target-db-cacert|target- + db-cert|target-db-key|target-db-key-password} + [VALUE] - Writes a secret to Redis secret store + Creates a secret of a specified key Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] - --cluster-host TEXT Host/IP of Redis Enterprise Cluster (service - name in case of k8s) [required] - --cluster-api-port INTEGER RANGE - API Port of Redis Enterprise Cluster - [default: 9443; 1000<=x<=65535; required] - --cluster-user TEXT Redis Enterprise Cluster username with - either DB Member, Cluster Member or Cluster - Admin roles [required] - --cluster-password TEXT Redis Enterprise Cluster Password - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --secret-name TEXT The name of the secret [required] - --secret-value TEXT The value of the secret [required] --help Show this message and exit. ``` diff --git a/content/rdi/reference/cli/redis-di-start.md b/content/rdi/reference/cli/redis-di-start.md index 7e60b79c90d..3209580a78d 100644 --- a/content/rdi/reference/cli/redis-di-start.md +++ b/content/rdi/reference/cli/redis-di-start.md @@ -18,12 +18,12 @@ Usage: redis-di start [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -105,7 +105,7 @@ Usage: redis-di start [OPTIONS] Starts the pipeline Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-status.md b/content/rdi/reference/cli/redis-di-status.md index 4427915bf97..3ff3fc889ff 100644 --- a/content/rdi/reference/cli/redis-di-status.md +++ b/content/rdi/reference/cli/redis-di-status.md @@ -18,12 +18,12 @@ Usage: redis-di status [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -141,7 +141,7 @@ Usage: redis-di status [OPTIONS] Displays the status of the pipeline end to end Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-stop.md b/content/rdi/reference/cli/redis-di-stop.md index cbdfafa569f..69165d9ae3f 100644 --- a/content/rdi/reference/cli/redis-di-stop.md +++ b/content/rdi/reference/cli/redis-di-stop.md @@ -18,12 +18,12 @@ Usage: redis-di stop [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -105,7 +105,7 @@ Usage: redis-di stop [OPTIONS] Stops the pipeline Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; diff --git a/content/rdi/reference/cli/redis-di-trace.md b/content/rdi/reference/cli/redis-di-trace.md index 977b311401b..e160d02f1c4 100644 --- a/content/rdi/reference/cli/redis-di-trace.md +++ b/content/rdi/reference/cli/redis-di-trace.md @@ -18,12 +18,12 @@ Usage: redis-di trace [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `rdi_host` (REQUIRED): @@ -95,7 +95,7 @@ Usage: redis-di trace [OPTIONS] - Default: `10` - Usage: `--max-change-records` - Maximum traced change records per shard + Maximum traced change records - `timeout` (REQUIRED): @@ -129,7 +129,7 @@ Usage: redis-di trace [OPTIONS] Starts a trace session for troubleshooting data transformation Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --rdi-host TEXT Host/IP of RDI Database [required] --rdi-port INTEGER RANGE Port of RDI Database [1000<=x<=65535; @@ -142,8 +142,7 @@ Options: --rdi-key-password TEXT Password for unlocking an encrypted private key --max-change-records INTEGER RANGE - Maximum traced change records per shard - [x>=1] + Maximum traced change records [x>=1] --timeout INTEGER RANGE Stops the trace after exceeding this timeout (in seconds) [default: 20; 1<=x<=600; required] diff --git a/content/rdi/reference/cli/redis-di-upgrade.md b/content/rdi/reference/cli/redis-di-upgrade.md index 7a55e1b2aa4..00ea8fbd7c6 100644 --- a/content/rdi/reference/cli/redis-di-upgrade.md +++ b/content/rdi/reference/cli/redis-di-upgrade.md @@ -18,12 +18,12 @@ Usage: redis-di upgrade [OPTIONS] ## Options -- `loglevel`: +- `log_level`: - Type: Choice(['DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']) - Default: `info` - - Usage: `--loglevel --log-level` + - Usage: `--log-level +-l` - `cluster_host` (REQUIRED): @@ -146,7 +146,7 @@ Usage: redis-di upgrade [OPTIONS] Upgrades RDI Engine without losing data or downtime Options: - -log-level, --loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL] + -l, --log-level [DEBUG|INFO|WARN|ERROR|CRITICAL] [default: INFO] --cluster-host TEXT Host/IP of Redis Enterprise Cluster (service name in case of k8s) [required] diff --git a/content/rdi/reference/cli/redis-di.md b/content/rdi/reference/cli/redis-di.md index a67629396a8..052e98c6868 100644 --- a/content/rdi/reference/cli/redis-di.md +++ b/content/rdi/reference/cli/redis-di.md @@ -47,7 +47,6 @@ Options: Commands: add-context Adds a new context - configure Configures the RDI Database so it is ready to... create Creates the RDI Database instance delete Deletes RDI database permanently delete-all-contexts Deletes all contexts @@ -56,13 +55,16 @@ Commands: describe-job Describes a transformation engine's job dump-support-package Dumps RDI support package get-rejected Returns all the stored rejected entries + install Installs RDI + install-collector Installs RDI Redis Collector on Redis BDB with... + install-redisgears Installs RedisGears module on Redis cluster list-contexts Lists all saved contexts list-jobs Lists transformation engine's jobs monitor Monitors RDI by collecting metrics and exporting... reset Resets the pipeline into initial full sync mode scaffold Generates configuration files for RDI and... set-context Sets a context to be the active one - set-secret Writes a secret to Redis secret store + set-secret Creates a secret of a specified key start Starts the pipeline status Displays the status of the pipeline end to end stop Stops the pipeline diff --git a/content/rdi/reference/config-yaml-reference.md b/content/rdi/reference/config-yaml-reference.md index 1609e35feb1..a1bc332f875 100644 --- a/content/rdi/reference/config-yaml-reference.md +++ b/content/rdi/reference/config-yaml-reference.md @@ -10,14 +10,14 @@ aliases: **Properties** -| Name | Type | Description | Required | -| ------------------------------------------------------------------------------------------ | ---------------- | ----------- | -------- | -| [**applier**](#applier)
(Configuration details of Redis Data Integration Applier Gear) | `object`, `null` | | | -| [**connections**](#connections) | `object` | | | +| Name | Type | Description | Required | +| ---------------------------------------------------------------------------------------------- | ---------------- | ----------- | -------- | +| [**processors**](#processors)
(Configuration details of Redis Data Integration Processors) | `object`, `null` | | | +| [**targets**](#targets)
(Target connections) | `object` | | | - + -## applier: Configuration details of Redis Data Integration Applier Gear +## processors: Configuration details of Redis Data Integration Processors **Properties** @@ -36,14 +36,33 @@ aliases: | **target_data_type**
(Target data type: hash/json \- RedisJSON module must be in use in the target DB) | `string` | Default: `"hash"`
Pattern: ``^\${.\*}$ | hash | json``
| | | **json_update_strategy**
(Target update strategy: replace/merge \- RedisJSON module must be in use in the target DB) | `string` | (DEPRECATED)
Property 'json_update_strategy' will be deprecated in future releases. Use 'on_update' job-level property to define the json update strategy.
Default: `"replace"`
Pattern: ``^\${.\*}$ | replace | merge``
| | | **initial_sync_processes**
(Number of processes RDI Engine creates to process the initial sync with the source) | `integer`, `string` | Default: `4`
Pattern: `^\${.*}$`
Minimum: `1`
Maximum: `32`
| | +| **idle_sleep_time_ms**
(Idle sleep time \(in milliseconds\) between batches) | `integer`, `string` | Default: `200`
Pattern: `^\${.*}$`
Minimum: `1`
Maximum: `999999`
| | +| **idle_streams_check_interval_ms**
(Interval \(in milliseconds\) for checking new streams when the stream processor is idling) | `integer`, `string` | Default: `1000`
Pattern: `^\${.*}$`
Minimum: `1`
Maximum: `999999`
| | +| **busy_streams_check_interval_ms**
(Interval \(in milliseconds\) for checking new streams when the stream processor is busy) | `integer`, `string` | Default: `5000`
Pattern: `^\${.*}$`
Minimum: `1`
Maximum: `999999`
| | | **wait_enabled**
(Checks if the data has been written to the replica shard) | `boolean` | Default: `false`
| | | **wait_timeout**
(Timeout in milliseconds when checking write to the replica shard) | `integer`, `string` | Default: `1000`
Pattern: `^\${.*}$`
Minimum: `1`
| | | **retry_on_replica_failure**
(Ensures that the data has been written to the replica shard and keeps retrying if not) | `boolean` | Default: `true`
| | **Additional Properties:** not allowed - + -## connections: Connections +## targets: Target connections + +**Properties** + +| Name | Type | Description | Required | +| -------------------------------------------------------------------- | -------- | ----------- | -------- | +| [**connection**](#targetsconnection)
(Target connection details) | `object` | | | + +**Example** + +```yaml +connection: {} +``` + + + +### targets\.connection: Target connection details **Properties (Pattern)** diff --git a/content/rdi/reference/debezium/cassandra.md b/content/rdi/reference/debezium/cassandra.md index a2c366f0b43..e5192b76015 100644 --- a/content/rdi/reference/debezium/cassandra.md +++ b/content/rdi/reference/debezium/cassandra.md @@ -11,72 +11,5 @@ aliases: ## application.properties ```properties -debezium.sink.type=redis -debezium.sink.redis.message.format=extended -debezium.sink.redis.address=: -# Uncomment the following line if you are not using a separate user for Redis Data Integration. -#debezium.sink.redis.user= -# If you are using the default user for authentication, please configure the password below. -# If you are using a custom user, make sure to specify both the username and password correctly. -debezium.sink.redis.password= -debezium.sink.redis.memory.limit.mb=80 -# Redis SSL/TLS -#debezium.sink.redis.ssl.enabled=true -# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica. -#debezium.sink.redis.wait.enabled=true -#debezium.sink.redis.wait.timeout.ms=1000 -#debezium.sink.redis.wait.retry.enabled=true -#debezium.sink.redis.wait.retry.delay.ms=1000 -#debezium.source.database.history.redis.ssl.enabled=true -# Location of the Java keystore file containing an application process' own certificate and private key. -#javax.net.ssl.keyStore= -# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password). -#javax.net.ssl.keyStorePassword= -# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). -#javax.net.ssl.trustStore= -# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore. -#javax.net.ssl.trustStorePassword= -debezium.source.connector.class=io.debezium.connector.cassandra.Cassandra4Connector -debezium.source.snapshot.consistency=ONE -debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore -debezium.source.topic.prefix= -debezium.source.cassandra.node.id= -debezium.source.cassandra.hosts= -debezium.source.cassandra.port= -debezium.source.cassandra.config= -debezium.source.commit.log.relocation.dir= -debezium.source.commit.log.real.time.processing.enabled=true -debezium.source.commit.marked.complete.poll.interval.ms=1000 -debezium.source.http.port=8040 - -# Whether to include the detailed schema information generated by Debezium in each record written to RDI. -# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets. -debezium.source.key.converter.schemas.enable=false -debezium.source.value.converter.schemas.enable=false -# When detailed schema information is excluded, handle decimal numeric types as strings. -debezium.source.decimal.handling.mode=string - -debezium.transforms=AddPrefix -debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter -debezium.transforms.AddPrefix.regex=.* -debezium.transforms.AddPrefix.replacement=data:$0 - -# Logging -# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized). -#quarkus.log.file.enable=true -#quarkus.log.file.path= -#quarkus.log.file.rotation.max-file-size=100M -#quarkus.log.file.rotation.rotate-on-boot=true -#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz -#quarkus.log.file.rotation.max-backup-index=3 - -# The default minimum log level for every log category, change only quarkus.log.level when needed. -quarkus.log.min-level=TRACE -# The default log level for every log category. -quarkus.log.level=INFO -# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. -quarkus.log.console.json=false -# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. -quarkus.http.port=8088 ``` diff --git a/content/rdi/reference/debezium/mongodb.md b/content/rdi/reference/debezium/mongodb.md index 2ea1c831c13..eea115a16a7 100644 --- a/content/rdi/reference/debezium/mongodb.md +++ b/content/rdi/reference/debezium/mongodb.md @@ -11,86 +11,5 @@ aliases: ## application.properties ```properties -debezium.sink.type=redis -debezium.sink.redis.message.format=extended -debezium.sink.redis.address=: -# Uncomment the following line if you are not using a separate user for Redis Data Integration. -#debezium.sink.redis.user= -# If you are using the default user for authentication, please configure the password below. -# If you are using a custom user, make sure to specify both the username and password correctly. -debezium.sink.redis.password= -debezium.sink.redis.memory.limit.mb=80 -# Redis SSL/TLS -#debezium.sink.redis.ssl.enabled=true -# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica. -#debezium.sink.redis.wait.enabled=true -#debezium.sink.redis.wait.timeout.ms=1000 -#debezium.sink.redis.wait.retry.enabled=true -#debezium.sink.redis.wait.retry.delay.ms=1000 -#debezium.source.database.history.redis.ssl.enabled=true -# Location of the Java keystore file containing an application process' own certificate and private key. -#javax.net.ssl.keyStore= -# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password). -#javax.net.ssl.keyStorePassword= -# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). -#javax.net.ssl.trustStore= -# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore. -#javax.net.ssl.trustStorePassword= -debezium.source.connector.class=io.debezium.connector.mongodb.MongoDbConnector -debezium.source.mongodb.hosts=/: -debezium.source.mongodb.connection.mode=replica_set -debezium.source.mongodb.user= -debezium.source.mongodb.password= -debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore -debezium.source.topic.prefix= - -debezium.source.offset.flush.interval.ms=1000 -debezium.source.tombstones.on.delete=false -debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory - -# Important: Do NOT use `include` and `exclude` database lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match database names to be monitored. -# By default, all databases are monitored. -#debezium.source.database.include.list=,... -# An optional, comma-separated list of regular expressions that match database names for which you do not want to capture changes. -#debezium.source.database.exclude.list=,... -# Important: Do NOT use `include` and `exclude` collection lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match collection names to be monitored. -#debezium.source.collection.include.list=,... -# An optional, comma-separated list of regular expressions that match collection names for which you do not want to capture changes. -#debezium.source.collection.exclude.list=,... - -#An optional, comma-separated list of regular expressions that match field names for which you do not want to capture changes. -#debezium.source.field_exclude_list=, - -# Whether to include the detailed schema information generated by Debezium in each record written to RDI. -# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets. -debezium.source.key.converter.schemas.enable=false -debezium.source.value.converter.schemas.enable=false -# When detailed schema information is excluded, handle decimal numeric types as strings. -debezium.source.decimal.handling.mode=string - -debezium.transforms=AddPrefix -debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter -debezium.transforms.AddPrefix.regex=.* -debezium.transforms.AddPrefix.replacement=data:$0 - -# Logging -# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized). -#quarkus.log.file.enable=true -#quarkus.log.file.path= -#quarkus.log.file.rotation.max-file-size=100M -#quarkus.log.file.rotation.rotate-on-boot=true -#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz -#quarkus.log.file.rotation.max-backup-index=3 - -# The default minimum log level for every log category, change only quarkus.log.level when needed. -quarkus.log.min-level=TRACE -# The default log level for every log category. -quarkus.log.level=INFO -# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. -quarkus.log.console.json=false -# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. -quarkus.http.port=8088 ``` diff --git a/content/rdi/reference/debezium/mysql.md b/content/rdi/reference/debezium/mysql.md index 493fc4358a2..0cbcb8e4ea8 100644 --- a/content/rdi/reference/debezium/mysql.md +++ b/content/rdi/reference/debezium/mysql.md @@ -11,103 +11,5 @@ aliases: ## application.properties ```properties -debezium.sink.type=redis -debezium.sink.redis.message.format=extended -debezium.sink.redis.address=: -# Uncomment the following line if you are not using a separate user for Redis Data Integration. -#debezium.sink.redis.user= -# If you are using the default user for authentication, please configure the password below. -# If you are using a custom user, make sure to specify both the username and password correctly. -debezium.sink.redis.password= -debezium.sink.redis.memory.limit.mb=80 -# Redis SSL/TLS -#debezium.sink.redis.ssl.enabled=true -# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica. -#debezium.sink.redis.wait.enabled=true -#debezium.sink.redis.wait.timeout.ms=1000 -#debezium.sink.redis.wait.retry.enabled=true -#debezium.sink.redis.wait.retry.delay.ms=1000 -#debezium.source.database.history.redis.ssl.enabled=true -# Location of the Java keystore file containing an application process' own certificate and private key. -#javax.net.ssl.keyStore= -# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password). -#javax.net.ssl.keyStorePassword= -# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). -#javax.net.ssl.trustStore= -# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore. -#javax.net.ssl.trustStorePassword= -debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector -# A numeric ID of this database client, which must be unique across all currently-running database processes in the MySQL cluster. -debezium.source.database.server.id=1 -debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore -debezium.source.topic.prefix= - -debezium.source.database.hostname= -debezium.source.database.port= -debezium.source.database.user= -debezium.source.database.password= -debezium.source.include.schema.changes=false -# Determines whether the connector should omit publishing change events when there are no modifications in the included columns. -debezium.source.skip.messages.without.change=true -debezium.source.offset.flush.interval.ms=1000 -debezium.source.tombstones.on.delete=false -debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory - -# Important: Do NOT use `include` and `exclude` database lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match database names to be monitored. -# By default, all databases are monitored. -#debezium.source.database.include.list=,... -# An optional, comma-separated list of regular expressions that match database names for which you do not want to capture changes. -#debezium.source.database.exclude.list=,... -# Important: Do NOT use `include` and `exclude` table lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture. -#debezium.source.table.include.list=,... -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers for tables whose changes you do not want to capture. -#debezium.source.table.exclude.list=,... - -# Important: Do NOT use `include` and `exclude` column lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values. -#debezium.source.column.include.list=,... -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to exclude from change event record values. -#debezium.source.column.exclude.list=,... - -# Data filtering during snapshot. -#debezium.source.snapshot.select.statement.overrides=schema.TABLE11,schema.TABLE2,schema.TABLE3 -# An optional query. Provide a valid query to override the default snapshot query. -#debezium.source.snapshot.select.statement.overrides..=select * from .
- -# Records only DDL statements that are relevant to tables whose changes are being captured by Debezium. -# In case of changing the captured tables, run `redis-di reset`. -debezium.source.schema.history.internal.store.only.captured.tables.ddl=true - -# Whether to include the detailed schema information generated by Debezium in each record written to RDI. -# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets. -debezium.source.key.converter.schemas.enable=false -debezium.source.value.converter.schemas.enable=false -# When detailed schema information is excluded, handle decimal numeric types as strings. -debezium.source.decimal.handling.mode=string - -debezium.transforms=AddPrefix -debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter -debezium.transforms.AddPrefix.regex=.* -debezium.transforms.AddPrefix.replacement=data:$0 - -# Logging -# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized). -#quarkus.log.file.enable=true -#quarkus.log.file.path= -#quarkus.log.file.rotation.max-file-size=100M -#quarkus.log.file.rotation.rotate-on-boot=true -#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz -#quarkus.log.file.rotation.max-backup-index=3 - -# The default minimum log level for every log category, change only quarkus.log.level when needed. -quarkus.log.min-level=TRACE -# The default log level for every log category. -quarkus.log.level=INFO -# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. -quarkus.log.console.json=false -# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. -quarkus.http.port=8088 ``` diff --git a/content/rdi/reference/debezium/oracle.md b/content/rdi/reference/debezium/oracle.md index 101a1960273..f951c94336c 100644 --- a/content/rdi/reference/debezium/oracle.md +++ b/content/rdi/reference/debezium/oracle.md @@ -11,112 +11,5 @@ aliases: ## application.properties ```properties -debezium.sink.type=redis -debezium.sink.redis.message.format=extended -debezium.sink.redis.address=: -# Uncomment the following line if you are not using a separate user for Redis Data Integration. -#debezium.sink.redis.user= -# If you are using the default user for authentication, please configure the password below. -# If you are using a custom user, make sure to specify both the username and password correctly. -debezium.sink.redis.password= -debezium.sink.redis.memory.limit.mb=80 -# Redis SSL/TLS -#debezium.sink.redis.ssl.enabled=true -# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica. -#debezium.sink.redis.wait.enabled=true -#debezium.sink.redis.wait.timeout.ms=1000 -#debezium.sink.redis.wait.retry.enabled=true -#debezium.sink.redis.wait.retry.delay.ms=1000 -#debezium.source.database.history.redis.ssl.enabled=true -# Location of the Java keystore file containing an application process' own certificate and private key. -#javax.net.ssl.keyStore= -# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password). -#javax.net.ssl.keyStorePassword= -# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). -#javax.net.ssl.trustStore= -# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore. -#javax.net.ssl.trustStorePassword= -debezium.source.connector.class=io.debezium.connector.oracle.OracleConnector -debezium.source.log.mining.strategy=online_catalog -debezium.source.log.mining.transaction.retention.ms=1800000 -# Specifies a comma-separated list of Oracle Real Application Clusters (RAC) SCAN VIP names. -# To obtain the list of configured SCAN VIP names, use the command: `srvctl config scan`. -#debezium.source.rac.nodes=,... -# This mode creates a JDBC query that filters not only operation types at the database level, but also schema, table, and username include/exclude lists. -debezium.source.log.mining.query.filter.mode=in -# The name of the Oracle Pluggable Database that the connector captures changes from. -# For non-CDB installation, do not specify this property. -#debezium.source.database.pdb.name=ORCLPDB1 -# Enables capturing and serialization of large object (CLOB, NCLOB, and BLOB) column values in change events. -#debezium.source.lob.enabled=true -# Specifies the constant that the connector provides to indicate that the original value is unchanged and not provided by the database. -#debezium.source.unavailable.value.placeholder=__debezium_unavailable_value -debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore -debezium.source.topic.prefix= -debezium.source.database.dbname= - -debezium.source.database.hostname= -debezium.source.database.port= -debezium.source.database.user= -debezium.source.database.password= -debezium.source.include.schema.changes=false -# Determines whether the connector should omit publishing change events when there are no modifications in the included columns. -debezium.source.skip.messages.without.change=true -debezium.source.offset.flush.interval.ms=1000 -debezium.source.tombstones.on.delete=false -debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory - -# Important: Do NOT use `include` and `exclude` table lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture. -#debezium.source.table.include.list=,... -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers for tables whose changes you do not want to capture. -#debezium.source.table.exclude.list=,... - -# Important: Do NOT use `include` and `exclude` column lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values. -#debezium.source.column.include.list=,... -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to exclude from change event record values. -#debezium.source.column.exclude.list=,... - -# Data filtering during snapshot. -# Table names must be in uppercase unless you have created a case-sensitive table name. -#debezium.source.snapshot.select.statement.overrides=schema.TABLE11,schema.TABLE2,schema.TABLE3 -# An optional query. Provide a valid query to override the default snapshot query. -# Table names must be in uppercase unless you have created a case-sensitive table name. -#debezium.source.snapshot.select.statement.overrides..
=select * from .
- -# Records only DDL statements that are relevant to tables whose changes are being captured by Debezium. -# In case of changing the captured tables, run `redis-di reset`. -debezium.source.schema.history.internal.store.only.captured.tables.ddl=true - -# Whether to include the detailed schema information generated by Debezium in each record written to RDI. -# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets. -debezium.source.key.converter.schemas.enable=false -debezium.source.value.converter.schemas.enable=false -# When detailed schema information is excluded, handle decimal numeric types as strings. -debezium.source.decimal.handling.mode=string - -debezium.transforms=AddPrefix -debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter -debezium.transforms.AddPrefix.regex=.* -debezium.transforms.AddPrefix.replacement=data:$0 - -# Logging -# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized). -#quarkus.log.file.enable=true -#quarkus.log.file.path= -#quarkus.log.file.rotation.max-file-size=100M -#quarkus.log.file.rotation.rotate-on-boot=true -#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz -#quarkus.log.file.rotation.max-backup-index=3 - -# The default minimum log level for every log category, change only quarkus.log.level when needed. -quarkus.log.min-level=TRACE -# The default log level for every log category. -quarkus.log.level=INFO -# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. -quarkus.log.console.json=false -# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. -quarkus.http.port=8088 ``` diff --git a/content/rdi/reference/debezium/postgresql.md b/content/rdi/reference/debezium/postgresql.md index 762e266a001..48a0601ce1f 100644 --- a/content/rdi/reference/debezium/postgresql.md +++ b/content/rdi/reference/debezium/postgresql.md @@ -11,94 +11,5 @@ aliases: ## application.properties ```properties -debezium.sink.type=redis -debezium.sink.redis.message.format=extended -debezium.sink.redis.address=: -# Uncomment the following line if you are not using a separate user for Redis Data Integration. -#debezium.sink.redis.user= -# If you are using the default user for authentication, please configure the password below. -# If you are using a custom user, make sure to specify both the username and password correctly. -debezium.sink.redis.password= -debezium.sink.redis.memory.limit.mb=80 -# Redis SSL/TLS -#debezium.sink.redis.ssl.enabled=true -# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica. -#debezium.sink.redis.wait.enabled=true -#debezium.sink.redis.wait.timeout.ms=1000 -#debezium.sink.redis.wait.retry.enabled=true -#debezium.sink.redis.wait.retry.delay.ms=1000 -#debezium.source.database.history.redis.ssl.enabled=true -# Location of the Java keystore file containing an application process' own certificate and private key. -#javax.net.ssl.keyStore= -# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password). -#javax.net.ssl.keyStorePassword= -# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). -#javax.net.ssl.trustStore= -# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore. -#javax.net.ssl.trustStorePassword= -debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector -debezium.source.plugin.name=pgoutput -debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore -debezium.source.topic.prefix= -debezium.source.database.dbname= - -debezium.source.database.hostname= -debezium.source.database.port= -debezium.source.database.user= -debezium.source.database.password= -debezium.source.include.schema.changes=false -# Determines whether the connector should omit publishing change events when there are no modifications in the included columns. -# This property takes effect when the `REPLICA IDENTITY` of the table is set to `FULL`. -debezium.source.skip.messages.without.change=true -debezium.source.offset.flush.interval.ms=1000 -debezium.source.tombstones.on.delete=false -debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory - -# Important: Do NOT use `include` and `exclude` table lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture. -#debezium.source.table.include.list=,... -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers for tables whose changes you do not want to capture. -#debezium.source.table.exclude.list=,... - -# Important: Do NOT use `include` and `exclude` column lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values. -#debezium.source.column.include.list=,... -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to exclude from change event record values. -#debezium.source.column.exclude.list=,... - -# Data filtering during snapshot. -#debezium.source.snapshot.select.statement.overrides=schema.TABLE11,schema.TABLE2,schema.TABLE3 -# An optional query. Provide a valid query to override the default snapshot query. -#debezium.source.snapshot.select.statement.overrides..
=select * from .
- -# Whether to include the detailed schema information generated by Debezium in each record written to RDI. -# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets. -debezium.source.key.converter.schemas.enable=false -debezium.source.value.converter.schemas.enable=false -# When detailed schema information is excluded, handle decimal numeric types as strings. -debezium.source.decimal.handling.mode=string - -debezium.transforms=AddPrefix -debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter -debezium.transforms.AddPrefix.regex=.* -debezium.transforms.AddPrefix.replacement=data:$0 - -# Logging -# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized). -#quarkus.log.file.enable=true -#quarkus.log.file.path= -#quarkus.log.file.rotation.max-file-size=100M -#quarkus.log.file.rotation.rotate-on-boot=true -#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz -#quarkus.log.file.rotation.max-backup-index=3 - -# The default minimum log level for every log category, change only quarkus.log.level when needed. -quarkus.log.min-level=TRACE -# The default log level for every log category. -quarkus.log.level=INFO -# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. -quarkus.log.console.json=false -# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. -quarkus.http.port=8088 ``` diff --git a/content/rdi/reference/debezium/sqlserver.md b/content/rdi/reference/debezium/sqlserver.md index 4202c748424..2062113322b 100644 --- a/content/rdi/reference/debezium/sqlserver.md +++ b/content/rdi/reference/debezium/sqlserver.md @@ -11,93 +11,5 @@ aliases: ## application.properties ```properties -debezium.sink.type=redis -debezium.sink.redis.message.format=extended -debezium.sink.redis.address=: -# Uncomment the following line if you are not using a separate user for Redis Data Integration. -#debezium.sink.redis.user= -# If you are using the default user for authentication, please configure the password below. -# If you are using a custom user, make sure to specify both the username and password correctly. -debezium.sink.redis.password= -debezium.sink.redis.memory.limit.mb=80 -# Redis SSL/TLS -#debezium.sink.redis.ssl.enabled=true -# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica. -#debezium.sink.redis.wait.enabled=true -#debezium.sink.redis.wait.timeout.ms=1000 -#debezium.sink.redis.wait.retry.enabled=true -#debezium.sink.redis.wait.retry.delay.ms=1000 -#debezium.source.database.history.redis.ssl.enabled=true -# Location of the Java keystore file containing an application process' own certificate and private key. -#javax.net.ssl.keyStore= -# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password). -#javax.net.ssl.keyStorePassword= -# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). -#javax.net.ssl.trustStore= -# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore. -#javax.net.ssl.trustStorePassword= -debezium.source.connector.class=io.debezium.connector.sqlserver.SqlServerConnector -debezium.source.database.names= -debezium.source.database.encrypt=false -debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore -debezium.source.topic.prefix= - -debezium.source.database.hostname= -debezium.source.database.port= -debezium.source.database.user= -debezium.source.database.password= -debezium.source.include.schema.changes=false -# Determines whether the connector should omit publishing change events when there are no modifications in the included columns. -debezium.source.skip.messages.without.change=true -debezium.source.offset.flush.interval.ms=1000 -debezium.source.tombstones.on.delete=false -debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory - -# Important: Do NOT use `include` and `exclude` table lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture. -#debezium.source.table.include.list=,... -# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers for tables whose changes you do not want to capture. -#debezium.source.table.exclude.list=,... - -# Important: Do NOT use `include` and `exclude` column lists at the same time, use either `include` or `exclude`. -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values. -#debezium.source.column.include.list=,... -# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to exclude from change event record values. -#debezium.source.column.exclude.list=,... - -# Data filtering during snapshot. -#debezium.source.snapshot.select.statement.overrides=schema.TABLE11,schema.TABLE2,schema.TABLE3 -# An optional query. Provide a valid query to override the default snapshot query. -#debezium.source.snapshot.select.statement.overrides..
=select * from .
- -# Whether to include the detailed schema information generated by Debezium in each record written to RDI. -# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets. -debezium.source.key.converter.schemas.enable=false -debezium.source.value.converter.schemas.enable=false -# When detailed schema information is excluded, handle decimal numeric types as strings. -debezium.source.decimal.handling.mode=string - -debezium.transforms=AddPrefix -debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter -debezium.transforms.AddPrefix.regex=.* -debezium.transforms.AddPrefix.replacement=data:$0 - -# Logging -# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized). -#quarkus.log.file.enable=true -#quarkus.log.file.path= -#quarkus.log.file.rotation.max-file-size=100M -#quarkus.log.file.rotation.rotate-on-boot=true -#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz -#quarkus.log.file.rotation.max-backup-index=3 - -# The default minimum log level for every log category, change only quarkus.log.level when needed. -quarkus.log.min-level=TRACE -# The default log level for every log category. -quarkus.log.level=INFO -# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. -quarkus.log.console.json=false -# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. -quarkus.http.port=8088 ```