Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: queue naming issues (#444) #552

Merged
merged 5 commits into from
May 13, 2022
Merged

fix: queue naming issues (#444) #552

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
83617d6
fix: ensure name uniqueness (#444)
NoxHarmonium May 2, 2022
b22944f
fix: allow physical ID to be omitted (#444)
NoxHarmonium May 2, 2022
8785e9d
chore: add tool-versions file (#444)
NoxHarmonium May 13, 2022
d8ec3bc
chore: bump the example project node version
NoxHarmonium May 13, 2022
da20719
chore: simplify the example stack to enable easier testing
NoxHarmonium May 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .tool-versions
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nodejs 14.19.0
15 changes: 15 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ functions:
- snsSqs:
name: TestEvent # Required - choose a name prefix for the event queue
topicArn: !Ref Topic # Required - SNS topic to subscribe to
omitPhysicalId: true # Optional - default value is false but recommended to be set to true for new deployments (see below)
batchSize: 2 # Optional - default value is 10
maximumBatchingWindowInSeconds: 10 # optional - default is 0 (no batch window)
maxRetryCount: 2 # Optional - default value is 5
Expand Down Expand Up @@ -89,6 +90,20 @@ plugins:
- "@agiledigital/serverless-sns-sqs-lambda"
```

### What is the omitPhysicalId option?

AWS allows you to omit the physical ID (queue name) for SQS resources.

In this case, the name is automatically generated by AWS based on the logical ID (the CloudFormation resource name).

This provides some benefits such as the ability to perform updates that require replacement, and not having to worry about the 80 character maximum length for queue names.

However, if you need to refer to the queue by name (which should be rare), rather than a CloudFormation reference, you might have to switch this off so that the name is stable.

This would be set to 'true' by default if this was the first version of the plugin. However, since the plugin is already in use and it could break existing stacks and may not be backwards compatible we have set this to 'false' by default. Switching this to 'true' on an existing stack has been tested and it seems to work OK, although it does create new queues and deletes the old ones. Switch the option on an existing stack at your own risk.

See also: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-name.html

### CloudFormation Overrides

If you would like to override a part of the CloudFormation template
Expand Down
13 changes: 7 additions & 6 deletions example-service/serverless.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ configValidationMode: error

provider:
name: aws
runtime: nodejs10.x
runtime: nodejs14.x
region: ap-southeast-2
stage: ${opt:stage, 'dev'}

Expand All @@ -21,17 +21,18 @@ functions:
- snsSqs:
name: ResourcePrefix
topicArn: ${self:custom.topicArn}
omitPhysicalId: true
batchSize: 2
maximumBatchingWindowInSeconds: 30
maxRetryCount: 2
kmsMasterKeyId: alias/aws/sqs
kmsDataKeyReusePeriodSeconds: 600
visibilityTimeout: 120
rawMessageDelivery: true
filterPolicy:
pet:
- dog
- cat
# filterPolicy:
# pet:
# - dog
# - cat

# Overrides for generated CloudFormation templates
# Mirrors the CloudFormation docs but uses camel case instead of title case
Expand All @@ -44,7 +45,7 @@ functions:
delaySeconds: 120
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html
eventSourceMappingOverride:
enabled: false
enabled: true
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sns-subscription.html
subscriptionOverride:
rawMessageDelivery: true
Expand Down
110 changes: 88 additions & 22 deletions lib/__snapshots__/serverless-sns-sqs-lambda.test.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-Test-functionsome-nameQueue.fifo",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-Test-functionsome-nameDeadLetterQueue.fifo",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -169,10 +175,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-Test-functionsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-Test-functionsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -363,10 +375,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:some prefixsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:some prefixsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -507,10 +525,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:some prefixsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:some prefixsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -639,10 +663,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-Test-functionsome-nameQueue.fifo",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-Test-functionsome-nameDeadLetterQueue.fifo",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -773,10 +803,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-Test-functionsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-Test-functionsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -905,10 +941,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:some prefixsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:some prefixsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -1039,10 +1081,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-config-Test-functionsome-nameQueue.fifo",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-config-Test-functionsome-nameDeadLetterQueue.fifo",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -1173,10 +1221,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-config-Test-functionsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-config-Test-functionsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -1305,10 +1359,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-provider-Test-functionsome-nameQueue.fifo",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-provider-Test-functionsome-nameDeadLetterQueue.fifo",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down Expand Up @@ -1439,10 +1499,16 @@ Object {
"Effect": "Allow",
"Resource": Array [
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-provider-Test-functionsome-nameQueue",
"Fn::GetAtt": Array [
"some-nameQueue",
"Arn",
],
},
Object {
"Fn::Sub": "arn:\${AWS::Partition}:sqs:\${AWS::Region}:\${AWS::AccountId}:test-service-dev-test-provider-Test-functionsome-nameDeadLetterQueue",
"Fn::GetAtt": Array [
"some-nameDeadLetterQueue",
"Arn",
],
},
],
},
Expand Down
Loading