Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Add String field parsing transforms #98

Open
yoniwg opened this issue Mar 13, 2023 · 1 comment
Open

[Enhancement] Add String field parsing transforms #98

yoniwg opened this issue Mar 13, 2023 · 1 comment

Comments

@yoniwg
Copy link

yoniwg commented Mar 13, 2023

Add a transform that parse a string field into other primitive values or into array, such as:
int, long, float, double, boolean (or an array of one of these).

Configuration

  • transforms.<name>.type - One of:
    • com.github.jcustenborder.kafka.connect.transform.common.ParseString$Key
    • com.github.jcustenborder.kafka.connect.transform.common.ParseString$Value
  • transforms.<name>.fields - field1,field2,...
  • transforms.<name>.type - One of: int, long, float, double, boolean
  • transforms.<name>.isArray - true or false (Default to false)

Examples

Example 1

If the source contains

"field1": "1" - Parse Integer

Then after applying the following transformation:

"transforms":"parseInt",
"transforms.parseInt.type":"com.github.jcustenborder.kafka.connect.transform.common.ParseString$Value",
"transforms.parseInt.fields":"field1",
"transforms.parseInt.type":"int",

The result will be:

"field1": 1

Example 2 - Array

If the source contains

"field1": "1,2,3"

Then after applying the following transformation:

"transforms":"parseIntArray",
"transforms.parseIntArray.type":"com.github.jcustenborder.kafka.connect.transform.common.ParseString$Value",
"transforms.parseIntArray.fields":"field1",
"transforms.parseIntArray.type":"int",
"transforms.parseIntArray.isArray":"true",

The result will be:

"field1": [1,2,3]

Example 3 - Multiple Fields

If the source contains

"field1": "true",
"field2": "false"

Then after applying the following transformation:

"transforms":"parseMultipleBooleans",
"transforms.parseMultipleBooleans.type":"com.github.jcustenborder.kafka.connect.transform.common.ParseString$Value",
"transforms.parseMultipleBooleans.fields":"field1,field2",
"transforms.parseMultipleBooleans.type":"boolean",

The result will be:

"field1": true,
"field2": false

Example 4 - Supporting Avro null values

If the source contains

"field1": {"string": "true"},
"field2": null

Then after applying the following transformation:

"transforms":"parseMultipleOptionalBooleans",
"transforms.parseMultipleOptionalBooleans.type":"com.github.jcustenborder.kafka.connect.transform.common.ParseString$Value",
"transforms.parseMultipleOptionalBooleans.fields":"field1,field2",
"transforms.parseMultipleOptionalBooleans.type":"boolean",

The result will be:

"field1": {"boolean": true},
"field2": null
@yoniwg
Copy link
Author

yoniwg commented Mar 13, 2023

If you think it is useful - Let me know and I may implement this transform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant