Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve secret replacement matching reg ex #18234

Merged
merged 2 commits into from
Oct 20, 2022
Merged

Improve secret replacement matching reg ex #18234

merged 2 commits into from
Oct 20, 2022

Conversation

jdpgrailsdev
Copy link
Contributor

What

  • Improve the masking of secrets so that the replaced version is valid JSON

How

  • Improve the regular expression used to find and replace secret values to better handle values with double quotes and commas in their value.

Recommended reading order

  1. MaskedDataInterceptor.java

Tests

  • All unit tests pass
  • Tested regular expression manually against a variety of strings, some which included double quotes and commas in the value to verify the replacement works.

@jdpgrailsdev jdpgrailsdev requested a review from gosusnp October 20, 2022 14:10
@jdpgrailsdev jdpgrailsdev temporarily deployed to more-secrets October 20, 2022 14:11 Inactive
gosusnp
gosusnp approved these changes Oct 20, 2022
View reviewed changes
airbyte-commons/src/main/java/io/airbyte/commons/logging/MaskedDataInterceptor.java
@@ -127,7 +127,7 @@ private String generatePattern(final Set<String> properties) {
builder.append("(?i)"); // case insensitive
builder.append("\"(");
builder.append(properties.stream().collect(Collectors.joining("|")));
builder.append(")\"\\s*:\\s*\"?((\\\\\"|[^\",}])*)\"?");
builder.append(")\"\\s*:\\s*(\"(?:[^\"\\\\]|\\\\.)*\"|\\[[^]\\[]*]|\\d+)");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Feels like it start to make sense to add the not escaped regex as a comment for readability.

@jdpgrailsdev jdpgrailsdev merged commit 2d32e9d into master Oct 20, 2022
@jdpgrailsdev jdpgrailsdev deleted the jonathan/mask-reg-ex-fix branch October 20, 2022 16:23
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Oct 20, 2022
Merged
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022
@jdpgrailsdev @jhammarstedt
Improve secret replacement matching reg ex (airbytehq#18234)
977351b 
* Improve secret replacement matching reg ex

* Remove System.out.println
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gosusnp gosusnp gosusnp approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jdpgrailsdev @gosusnp