🐛 Source Dynamodb: Fix reserved words in expression #20172

itaseskii · 2022-12-07T00:08:13Z

What

Fixes #19739

How

Using placeholders for attributes in the filter expression.

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

itaseskii · 2022-12-13T18:40:26Z

/test connector=connectors/source-dynamodb

🕑 connectors/source-dynamodb https://github.com/airbytehq/airbyte/actions/runs/3688353203
❌ connectors/source-dynamodb https://github.com/airbytehq/airbyte/actions/runs/3688353203
🐛

itaseskii · 2022-12-13T22:39:00Z

/test connector=connectors/source-dynamodb

🕑 connectors/source-dynamodb https://github.com/airbytehq/airbyte/actions/runs/3689956136
✅ connectors/source-dynamodb https://github.com/airbytehq/airbyte/actions/runs/3689956136
No Python unittests run

Build Passed

Test summary info:

All Passed

vincentkoc

Missing the docker version bump and notes

vincentkoc · 2022-12-15T09:53:33Z

@itaseskii can you please update the link in the PR to the exact issue that this is resolving for full context of the change?

piyushsingariya · 2023-01-11T07:01:54Z

@itaseskii can you please update the link in the PR to the exact issue that this is resolving for full context of the change?

@koconder the PR has been linked to the issue already, could you re-review the PR again?

vincentkoc · 2023-01-19T10:18:07Z

/publish connector=connectors/source-dynamodb

🕑 Publishing the following connectors:
connectors/source-dynamodb
https://github.com/airbytehq/airbyte/actions/runs/3957461279

Connector	Did it publish?	Were definitions generated?
connectors/source-dynamodb	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

vincentkoc · 2023-01-19T10:30:22Z

@itaseskii see publish image error logs, something is not right. Its not liking the code changes for some reason.

itaseskii · 2023-01-19T20:43:09Z

/test connector=connectors/source-dynamodb

🕑 connectors/source-dynamodb https://github.com/airbytehq/airbyte/actions/runs/3962228595
✅ connectors/source-dynamodb https://github.com/airbytehq/airbyte/actions/runs/3962228595
No Python unittests run

Build Passed

Test summary info:

All Passed

itaseskii · 2023-01-19T20:44:06Z

@itaseskii see publish image error logs, something is not right. Its not liking the code changes for some reason.

@koconder there were some changes in the protocol model which made the connector incompatible. the issue should be fixed now, please try again.

sh4sh · 2023-02-23T13:54:05Z

/publish connector=connectors/source-dynamodb

🕑 Publishing the following connectors:
connectors/source-dynamodb
https://github.com/airbytehq/airbyte/actions/runs/4253307965

Connector	Did it publish?	Were definitions generated?
connectors/source-dynamodb	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

sh4sh · 2023-02-23T14:52:21Z

Published but checks failing now... hang on

sh4sh · 2023-02-23T23:22:09Z

Something is amiss with connectors.md that is causing the build to fail. I'll take a look tomorrow morning, for now I have rolled back the above /publish.

Update 2/24: I wasn't able to resolve this today, but I will come back to it on Monday

sh4sh · 2023-02-27T21:41:08Z

The connectors.md issue was resolved, but now getting GitHub self-hosted runner registration error on the Connectors Base: Start Build EC2 Runner step which i think is a transient error. Fortunately the connectors base build passed before so I think we are probably in the clear. Re-running here: https://github.com/airbytehq/airbyte/actions/runs/4286881888

Oops, we still need to publish this too as that got rolled back. I should have disabled auto-merge. Reverting now.

This reverts commit b9b8cb0.

…#23515) This reverts commit b9b8cb0.

…20172)" (#23515)" This reverts commit a5cd384.

…20172)" (#23515)" (#23598) This reverts commit a5cd384.

sh4sh · 2023-03-02T16:32:59Z

/publish connector=connectors/source-dynamodb

🕑 Publishing the following connectors:
connectors/source-dynamodb
https://github.com/airbytehq/airbyte/actions/runs/4316142995

Connector	Did it publish?	Were definitions generated?
connectors/source-dynamodb	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

sh4sh · 2023-03-02T16:48:18Z

Please ignore the above message ^ these changes have been merged into source-dynamodb:0.1.2

murat-cetinkaya · 2023-03-02T17:43:47Z

Thanks @sh4sh 👏🏼

…20172)" (#23515)" (#23598) This reverts commit a5cd384.

* fix reserved words in expression * chore: bump version * fix state message * auto-bump connector version * fix reserved words in projection expression * bump dockerfile version * auto-bump connector version --------- Co-authored-by: Vincent Koc <vincentkoc@ieee.org> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Sunny <6833405+sh4sh@users.noreply.github.com>

…q#20172)" (airbytehq#23515) This reverts commit b9b8cb0.

…irbytehq#20172)" (airbytehq#23515)" (airbytehq#23598) This reverts commit a5cd384.

murat-cetinkaya · 2023-03-16T14:24:54Z

I upgraded to v0.42 which includes this fix but I'm still getting the same error (DynamoDbException: Invalid ProjectionExpression: Syntax error; token: "-", ).
I wonder about your experiences.

itaseskii · 2023-03-16T14:40:17Z

@murat-cetinkaya have you included the attribute name in the config property reserved_attribute_names?
https://github.com/airbytehq/airbyte/pull/20172/files#diff-bbb3f5b986d2e1b08e22436ca88d58dc55a6617addd6e2db46051ac8246db33fR58

murat-cetinkaya · 2023-03-16T15:30:50Z

Oh, I overlooked that. I tried again and it succeeded 🎉. Thanks again @itaseskii 👏🏼👏🏼

fix reserved words in expression

90cfce8

octavia-squidington-iv added connectors/source/dynamodb area/connectors Connector related issues labels Dec 7, 2022

itaseskii mentioned this pull request Dec 7, 2022

Source DynamoDB - Reserved Keyword Issue #19739

Closed

itaseskii changed the title ~~🐛 Source Dynamodb: Fix reserved words in expressions~~ 🐛 Source Dynamodb: Fix reserved words in expression Dec 7, 2022

Merge branch 'master' into dynamodb-fix-reserved-keyword

d2e1ff3

Merge branch 'master' into dynamodb-fix-reserved-keyword

c146292

vincentkoc requested review from vincentkoc and a team December 15, 2022 09:52

vincentkoc suggested changes Dec 15, 2022

View reviewed changes

Merge branch 'master' into dynamodb-fix-reserved-keyword

4290c28

vincentkoc self-assigned this Jan 15, 2023

vincentkoc added 2 commits January 19, 2023 18:59

chore: bump version

52bb5d2

Merge branch 'master' into dynamodb-fix-reserved-keyword

00ee718

vincentkoc previously approved these changes Jan 19, 2023

View reviewed changes

octavia-squidington-iv added the area/documentation Improvements or additions to documentation label Jan 19, 2023

Merge branch 'master' into dynamodb-fix-reserved-keyword

5b99519

sajarin added bounty bounty-S Maintainer program: claimable small bounty PR labels Jan 19, 2023

itaseskii added 3 commits January 19, 2023 20:55

Merge branch 'master' into dynamodb-fix-reserved-keyword

fe45003

fix state message

d76dca8

Merge branch 'master' into dynamodb-fix-reserved-keyword

8fc5a60

octavia-squidington-iii and others added 2 commits February 23, 2023 14:07

auto-bump connector version

3a41393

Merge branch 'master' into dynamodb-fix-reserved-keyword

68eb5ef

Merge branch 'master' into dynamodb-fix-reserved-keyword

bdf865b

sh4sh enabled auto-merge (squash) February 23, 2023 21:58

Merge branch 'master' into dynamodb-fix-reserved-keyword

0783931

sh4sh merged commit b9b8cb0 into airbytehq:master Feb 27, 2023

sh4sh added a commit that referenced this pull request Feb 27, 2023

Revert "🐛 Source Dynamodb: Fix reserved words in expression (#20172)"

26e3f95

This reverts commit b9b8cb0.

sh4sh mentioned this pull request Feb 27, 2023

Revert "🐛 Source Dynamodb: Fix reserved words in expression" #23515

Merged

sh4sh added a commit that referenced this pull request Feb 28, 2023

Revert "🐛 Source Dynamodb: Fix reserved words in expression (#20172)" (…

a5cd384

…#23515) This reverts commit b9b8cb0.

sh4sh added a commit that referenced this pull request Feb 28, 2023

Revert "Revert "🐛 Source Dynamodb: Fix reserved words in expression (#…

4f3b2e6

…20172)" (#23515)" This reverts commit a5cd384.

sh4sh mentioned this pull request Feb 28, 2023

Revert "Revert "🐛 Source Dynamodb: Fix reserved words in expression"" #23598

Merged

sh4sh added a commit that referenced this pull request Mar 2, 2023

Revert "Revert "🐛 Source Dynamodb: Fix reserved words in expression (#…

821fce1

…20172)" (#23515)" (#23598) This reverts commit a5cd384.

jbfbell pushed a commit that referenced this pull request Mar 6, 2023

Revert "Revert "🐛 Source Dynamodb: Fix reserved words in expression (#…

d311b14

…20172)" (#23515)" (#23598) This reverts commit a5cd384.

danielduckworth pushed a commit to danielduckworth/airbyte that referenced this pull request Mar 13, 2023

Revert "🐛 Source Dynamodb: Fix reserved words in expression (airbyteh…

2b6ec6d

…q#20172)" (airbytehq#23515) This reverts commit b9b8cb0.

danielduckworth pushed a commit to danielduckworth/airbyte that referenced this pull request Mar 13, 2023

Revert "Revert "🐛 Source Dynamodb: Fix reserved words in expression (a…

b54114e

…irbytehq#20172)" (airbytehq#23515)" (airbytehq#23598) This reverts commit a5cd384.

jbfbell mentioned this pull request Mar 15, 2023

Check With Catalog JDBC POC (feature flag default off) #24109

Closed

37 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Source Dynamodb: Fix reserved words in expression #20172

🐛 Source Dynamodb: Fix reserved words in expression #20172

itaseskii commented Dec 7, 2022 •

edited by sh4sh

Loading

itaseskii commented Dec 13, 2022 •

edited by github-actions bot

Loading

itaseskii commented Dec 13, 2022 •

edited by github-actions bot

Loading

vincentkoc left a comment

vincentkoc commented Dec 15, 2022

piyushsingariya commented Jan 11, 2023

vincentkoc commented Jan 19, 2023 •

edited by github-actions bot

Loading

vincentkoc commented Jan 19, 2023

itaseskii commented Jan 19, 2023 •

edited by github-actions bot

Loading

itaseskii commented Jan 19, 2023

sh4sh commented Feb 23, 2023 •

edited by github-actions bot

Loading

sh4sh commented Feb 23, 2023

sh4sh commented Feb 23, 2023 •

edited

Loading

sh4sh commented Feb 27, 2023 •

edited

Loading

sh4sh commented Mar 2, 2023 •

edited by github-actions bot

Loading

sh4sh commented Mar 2, 2023

murat-cetinkaya commented Mar 2, 2023

murat-cetinkaya commented Mar 16, 2023

itaseskii commented Mar 16, 2023

murat-cetinkaya commented Mar 16, 2023

🐛 Source Dynamodb: Fix reserved words in expression #20172

🐛 Source Dynamodb: Fix reserved words in expression #20172

Conversation

itaseskii commented Dec 7, 2022 • edited by sh4sh Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

Tests

itaseskii commented Dec 13, 2022 • edited by github-actions bot Loading

itaseskii commented Dec 13, 2022 • edited by github-actions bot Loading

Build Passed

vincentkoc left a comment

Choose a reason for hiding this comment

vincentkoc commented Dec 15, 2022

piyushsingariya commented Jan 11, 2023

vincentkoc commented Jan 19, 2023 • edited by github-actions bot Loading

vincentkoc commented Jan 19, 2023

itaseskii commented Jan 19, 2023 • edited by github-actions bot Loading

Build Passed

itaseskii commented Jan 19, 2023

sh4sh commented Feb 23, 2023 • edited by github-actions bot Loading

sh4sh commented Feb 23, 2023

sh4sh commented Feb 23, 2023 • edited Loading

sh4sh commented Feb 27, 2023 • edited Loading

sh4sh commented Mar 2, 2023 • edited by github-actions bot Loading

sh4sh commented Mar 2, 2023

murat-cetinkaya commented Mar 2, 2023

murat-cetinkaya commented Mar 16, 2023

itaseskii commented Mar 16, 2023

murat-cetinkaya commented Mar 16, 2023

itaseskii commented Dec 7, 2022 •

edited by sh4sh

Loading

itaseskii commented Dec 13, 2022 •

edited by github-actions bot

Loading

itaseskii commented Dec 13, 2022 •

edited by github-actions bot

Loading

vincentkoc commented Jan 19, 2023 •

edited by github-actions bot

Loading

itaseskii commented Jan 19, 2023 •

edited by github-actions bot

Loading

sh4sh commented Feb 23, 2023 •

edited by github-actions bot

Loading

sh4sh commented Feb 23, 2023 •

edited

Loading

sh4sh commented Feb 27, 2023 •

edited

Loading

sh4sh commented Mar 2, 2023 •

edited by github-actions bot

Loading