Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨Source Amazon SQS: Migrate Python CDK to Low code #41064

Open
wants to merge 32 commits into
base: master
Choose a base branch
from

Conversation

btkcodedev
Copy link
Collaborator

@btkcodedev btkcodedev commented Jul 9, 2024

What

Closes https://github.com/airbytehq/airbyte-internal-issues/issues/11159
Migrating Source Amazon SQS to Low-Code CDK

How

Developed using (Configuration Based Source) low-code CDK

Recommended reading order

  1. spec.yaml
  2. manifest.yaml
  3. schemas/*

🚨 User Impact 🚨

  • Migration to low-code

@btkcodedev btkcodedev self-assigned this Jul 9, 2024
Copy link

vercel bot commented Jul 9, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 17, 2024 3:54pm

@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/amazon-sqs labels Jul 9, 2024
@btkcodedev
Copy link
Collaborator Author

Awaiting Creds

@btkcodedev
Copy link
Collaborator Author

Blocked due to unavailable credentials

@natikgadzhi
Copy link
Contributor

@btkcodedev rebase this, let's dust it off. Then, set metadata.releases.isReleaseCandidate = true and set the version in metadata and in dockerImageTag and in pyproject.toml to be whatever-rc1.

This will trigger a release candidate publish, and we'll proceed with the merge.

@natikgadzhi
Copy link
Contributor

@btkcodedev this is still important, but I'd say let's go all the way to Manifest-Only, even if there are custom components.

Builder supports custom components when running in abctl.

Do you have everything you need to work on this, or blocked?

@btkcodedev
Copy link
Collaborator Author

Hi @natikgadzhi, I was waiting for the creds for completely testing the migration.
I'll dust and rebase!
Lets gooo

@btkcodedev
Copy link
Collaborator Author

Taking this as priority

@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Dec 5, 2024

This boto3 package is giving some headache.
I'll try to reinvent the wheel, because the previous custom component that i made somehow seems to be broken after 4 months :P
so that it could be completely configured with declarative manifest :)

@btkcodedev
Copy link
Collaborator Author

Yahoo!
Reinventing wheel is success!
Got the connection without any boto3 dependency
image

@btkcodedev
Copy link
Collaborator Author

Currently, the test account has 0 messages, I've updated some specs and documentation,
This PR would be a breaking change
image

@btkcodedev btkcodedev requested a review from DanyloGL December 6, 2024 23:02
@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Dec 6, 2024

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (2518269)

@octavia-squidington-iv octavia-squidington-iv requested a review from a team December 6, 2024 23:13
@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Dec 9, 2024

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (f31ebc8)

@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Dec 9, 2024

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (7dbe815)

@btkcodedev
Copy link
Collaborator Author

Connection successfull in local, Waiting for maintainers to update CI creds

@DanyloGL
Copy link
Collaborator

DanyloGL commented Dec 17, 2024

Creds updated.
@btkcodedev do you have task for this PR? Couldn't find on the board.

@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Dec 17, 2024

The "PR" was only updated on board, I will update it as an issue. Thanks for the reminder! @DanyloGL

@btkcodedev
Copy link
Collaborator Author

Done & Done, Connection passed and checks cleared. Note: The MessageQueue is currently empty in the production account.
A review is needed from breaking-change-reviewers. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/amazon-sqs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants