Parse Targeted Messaging data files #2033
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hectorgomezv
force-pushed
the
parse-outreach-definition-files
branch
from
37b491b to
461b614
Compare
iamacook
reviewed
Oct 21, 2024
src/datasources/targeted-messaging/targeted-messaging.datasource.ts
Outdated
Show resolved
Hide resolved
iamacook
reviewed
Oct 22, 2024
Comment on lines
+40
to
+56
| // Check for AWS_* fields in production and staging environments | ||
| ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY', 'AWS_REGION'].forEach( | ||
| (field) => { | ||
| if ( | ||
| config.CGW_ENV && | ||
| config instanceof Object && | ||
| ['production', 'staging'].includes(config.CGW_ENV) && | ||
| !(config as Record<string, unknown>)[field] | ||
| ) { | ||
| ctx.addIssue({ | ||
| code: z.ZodIssueCode.custom, | ||
| message: `is required in production and staging environments`, | ||
| path: [field], | ||
| }); | ||
| } | ||
| }, | ||
| ), |
There was a problem hiding this comment.
Why do we need to superRefine this? We could just remove optional() from the respective property.
There was a problem hiding this comment.
I thought it would be nice to avoid enforcing these on the env.sample file and in the local development environments, as we have a local FileStorageType, the need to set those AWS_* env variables when the local FileStorageType is selected would be a bit weird IMO.
Using this superRefine approach, we could even apply it to other environment variables, to stop requiring placeholders in development environments and simplify the safe-infrastrastructure repository management. Wdyt? (Of course, I'd tackle that in a separate PR if you agree).
src/datasources/targeted-messaging/entities/__tests__/outreach-file.builder.ts
Outdated
Show resolved
Hide resolved
iamacook
approved these changes
Oct 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces the functionality for parsing Targeted Messaging data files. These data files should be JSON files, and the source could be either Cloud Storage or Local Storage.
Changes
OutreachFileProcessor, which dumps the data file contents to the database.ITargetedMessagingDatasourceto allow the data file processing.AwsCloudStorageApiService(and@aws-sdk/client-s3package) which facilitates file management through AWS S3.targetedMessagingconfiguration tree.sample.jsonfile with randomsafe_addressesand mock data, for testing/local development purposes.