Proof of Concept - SQS + Lambda #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Provision SQS with 'arn' as output - Bind for SQS doesn't matter much (unless we want to limit who can put stuff into/out of queue - Provision Lambda function with no outputs + sqs 'arn' as input for event_source_mapping - Bind varies for Lambda function (multiple options for different hooks afterwards) - Parameterize SQS Provision with 'name' - Paramterize Lambda Provision with 'name', 'script', 'sqs_input' - (Maybe event_source_mapping could be the binding?) - Copy lambda code from the solr brokerpak - Tested manually with 'terraform init/apply' and the pipeline was successfully connected - Next steps: add more features + create test script
- For the first iteration, the plan options will be simple. So SQS and Lambda only have a small set of inputs/outputs - Currently, there is no 'binding' for sqs, so just update the code to output the sqs_arn as the only output - Cleanup unnecessary things
- No concept of 'binding' for each plan - Fix manifest.yml issues that prevented building - Cleanup dependencies
- Remove unnecessary kind k8s stuff - Remove aws provider version restriction from sqs provision - Update sqs binding input default - Cleanup makefile - TODO: fix sqs binding, the arn is not being output right now
- Cleanup sqs definition, there were weird parameters in weird spots - The main problem was that the BIND_PARAMS were not making the CSB happy, so I removed them - Add the basic test.sh file - Remove extra variable input to sqs binding - Test script creates S3 bucket and SQS instance
- Pass BIND_NAME as parameter to demo-up to make sure the 'error saving bind request details to database: error creating bind request details: UNIQUE constraint failed: bind_request_details.service_binding_id. Unbind operations will not be able to complete' error doesn't appear - Add template provider for lambda plan - Add code to parse sqs 'binding' for lambda 'provisioning'
- Hopefully the csb->terraform naming was the only reason why lambda provisioning failed, will update in next commit - Add option to destroy resources (helps on re-built of csb)
it's an env var, not a script parameter
There was a lot wrong with string manipulation from bash --> json --> csb --> terraform --> aws - I'm kinda burnt from figuring it out, but at least multiline string has been partially solved (the quotation needs another bash line, but I'll fix that in the next commit) - Remove ecs permissions, only need sqs permissions - Fix bash quotation of CLOUD_PROVISION_PARAMS
Accomodate for quotes + spaces.. lambda provisioning works
- Add four more variables for an s3 bucket - Update script to read input and then pass data to 'raw' or 'clean' based on prescence of word - Update test.sh with the new s3 arguments
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to
References: