Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Migrate S3 Client from AWS SDK v2 to v3 #221

Open
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

vahidalizad
Copy link

Summary

This pull request migrates the S3 client implementation from AWS SDK v2 to v3.

Changes

  1. Initialization: Updated the S3 client initialization to use S3Client from @aws-sdk/client-s3 package.
  2. Commands: Replaced v2 commands (getObject, putObject, etc.) with their v3 equivalents (GetObjectCommand, PutObjectCommand, etc.).
  3. Response Handling: Modified response handling to accommodate changes in the v3 API, particularly in how streams and buffers are managed.
  4. Testing: Added unit tests to cover new functionality and ensure compatibility with the existing codebase. Implemented mocks for offline testing using Jasmine.

Release Dependency

This pull request is intended for after the release of the asynchronous getFileLocation function on the parse-server repository. Please ensure that this dependency is met before merging.

The relevant pull request for adding support of asynchronous getFileLocation in adapters can be found here: parse-server PR #9271.

References

Please review the changes and provide feedback. Thank you for your time and consideration.

Closes: #197

Copy link

Thanks for opening this pull request!

@mtrezza
Copy link
Member

mtrezza commented Oct 6, 2024

Continuing from #220 (comment).

I notice that this CI does not test against Parse Server, so with or without parse-community/parse-server#9271, the CI passes. That is not good, because w/o integration test we cannot really know whether this all works together. That has been less of an issue in the past, where this adapter just had to comply with the interface. But now that it depends on a specific implementation of the interface (async support for interface methods), it would be good to test. Since none of the parse server adapters does integration tests, this is not an exception, and we could go ahead and merge this, but someone should test this end-to-end and confirm this is working before we merge.

@vahidalizad
Copy link
Author

Continuing from #220 (comment).

I notice that this CI does not test against Parse Server, so with or without parse-community/parse-server#9271, the CI passes. That is not good, because w/o integration test we cannot really know whether this all works together. That has been less of an issue in the past, where this adapter just had to comply with the interface. But now that it depends on a specific implementation of the interface (async support for interface methods), it would be good to test. Since none of the parse server adapters does integration tests, this is not an exception, and we could go ahead and merge this, but someone should test this end-to-end and confirm this is working before we merge.

I completely agree that testing integrations is crucial, especially given the dependency on async support for interface methods. That said, I think we could merge this into a separate branch for now and make adding integration tests a separate task. We could raise an issue for that and have others contribute to the testing. Once the tests are complete and we’ve confirmed everything works end-to-end, we can merge it into the master branch.

@mtrezza
Copy link
Member

mtrezza commented Oct 9, 2024

Our CD automation setup currently does not allow maintaining feature branches. Hence we'll have to merge this into master.

I think it would be enough if you could test this out with the latest version of Parse Server that supports the async interface. Maybe @mman also has time to test this out, just to confirm it works. Then we can go ahead and merge this and open a Server e2e test as a separate issue for later on.

@mman
Copy link

mman commented Oct 9, 2024

@mtrezza I would love to test it but currently I am low-priority stuck on two conflicting issues:

const ParsePushAdapter = require('@parse/push-adapter').ParsePushAdapter;

that I used to work around push adapter version issues fails with: Error [ERR_REQUIRE_ESM]: require() of ES Module not supported..

And when I convert my code to ESM, parse server fails to start with (simplified):

Error [ERR_REQUIRE_ESM]: require() of ES Module ./cloud/main.js from ./node_modules/parse-server/lib/ParseServer.js not supported.

not sure what is the solution yet but will have a time to look into it next week...

@mtrezza
Copy link
Member

mtrezza commented Oct 9, 2024

@mman does this help you? parse-community/parse-server#9316 (comment)

@mman
Copy link

mman commented Oct 9, 2024

@mman does this help you? parse-community/parse-server#9316 (comment)

I think it's actually this one: parse-community/parse-server#7559 with workaround mentioned here: https://stackoverflow.com/questions/77638089/how-to-make-parse-server-detect-im-using-esm-now where instead of passing a cloud code path to Parse Server constructor, you actually have to import cloud code in your main .js file and pass the imported code to the constructor...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to AWS JS SDK v3 needed
3 participants