(aws-iam): StringParameter.value_from_lookup's dummy value did not suffice

[ThomasSteinbach]

aws_iam.StringParameter.value_from_lookup(...)

returns a dummy-value-for-${parameterName} during synthesis (from #3654). This value did not suffice for use as ARN. The dummy value itself should represent a dummy ARN pattern to avoid errors.

Reproduction Steps

Here is a short (and stripped) example, which currently leads to an error:

aws_kms.Key.from_key_arn(
    self,
    id,
    key_arn=aws_ssm.StringParameter.value_from_lookup(
        self,
        parameter_name="/example/param",
    ),
)

Error Log

During synthesis this leads to an error:

jsii.errors.JSIIError: ARNs must have at least 6 components: dummy-value-for-/example/param

Workaround

_param = aws_ssm.StringParameter.value_from_lookup(self, parameter_name="/example/param")

if "dummy-value" in _param:
    _param = "arn:aws:service:eu-central-1:123456789012:entity/dummy-value"

aws_kms.Key.from_key_arn(
    self,
    id,
    key_arn=_param,
)

Solution Proposal

Instead of dummy-value-for-${parameterName} the method should return something like arn:aws:service:eu-central-1:123456789012:entity/dummy-value

This solution would also address/solve #7051

---

This is 🐛 Bug Report

[ThomasSteinbach]

Hi @eladb , is looks like you have created the original implementation of the "dummy-value" for ssm-StringParameters. If so, could you please have a look on my proposed solution code (#8700 ) for this bug report and "do it right", as I am no TypeScript programmer? Thank you :)

[MrArnoldPalmer]

@ThomasSteinbach I'm curious to know about the use case that is breaking with this right now. Can you provide details about what constructs/processes you're using that aren't supported currently?

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[rix0rrr]

The dummy return value should probably be a Token, this should suppress literal string validation in most of the CDK.

[ThomasSteinbach]

Sorry @MrArnoldPalmer for the late response. (I spent my last 14d in holidays). Regarding your questions:

I'm curious to know about the use case that is breaking with this right now.

The Reproduction Steps in the issues description will give you one use case ( 'aws_kms.Key` can't handle the current dummy values).

Without testing it, in my description I also statet that the solution, provided with this issue ( #8700 ), may resolve #7051 , which then is also a use case, as aws_s3.Bucket.from_bucket_arn will raise the exact same error on the current dummy value.

Can you provide details about what constructs/processes you're using that aren't supported currently?

As shown in the descriptions example, aws_kms.Key.from_key_arn() would not support the dummy-value-for-${parameterName} returned by aws_ssm.StringParameter.value_from_lookup() but would support arn:aws:service:eu-central-1:123456789012:entity/dummy-value as proposed in my solution https://github.com/aws/aws-cdk/pull/8700/files.

The proposed solution should work for all other resource methods requiring an ARN, like aws_s3.Bucket.from_bucket_arn.

We've tested workaround (for python) given in the description, and it worked well for us under every circumstances. The proposed solution would fix the issue natively in TypeScript. However a professional TypeScript programmer should review and "cleanup" the solution.

[MrArnoldPalmer]

@ThomasSteinbach thanks, I didn't derive all those details from the original post.

Since ssm params can obviously contain stuff other than Arns, and validations may be anything really, a more generic solution is required I believe. As @rix0rrr mentioned, we can possibly change the dummy value return type to be a Token, which makes sense since Tokens are stand ins for values that will resolve at some point in the future.

[ThomasSteinbach]

As long as it works, I am totally fine with any solution 😃

As Python-CDK user I feel not able to provide a clean solution for replacing the current dummy values by tokens. Could you please provide one or delegate it to a CDK TypeScript programmer? I am confident, that it should be no extravagant expense but would solve this as well as this #7051 issue.

Thanks in advance

[zxkane]

Another use case,

If putting the json string into SSM parameter store for externalizing the parameters of CDK app, the valueFromLookup always immediately returns the default value that breaks the JSON parsing.

[jishi]

I'm getting this problem when trying to "import" an SNS topic via ARN that I get from parameter store:

const cachePurgeTopicArn = StringParameter.valueFromLookup(
      this, '/infrastructure/topic/cache-purge');
const cachePurgeTopic = Topic.fromTopicArn(this, 'CachePurgeTopic', cachePurgeTopicArn);

Meaning, I can't synthesize this to validate it. This works fine with importing VPCs, ALB listeners, ECS cluster ecs and actually synthesize to the actual parameter store value, so I'm not sure why this becomes a dummy value in this example?

EDIT: OH wait, it only fails on first run? If I run the lookup first, then add the .fromTopicArn, it works? Because of resource caching I guess, that's even weirder...

[jishi]

This actually fails for deploy runs as well, not only synthesizing. Making it somewhat cumbersome to store ARNs on SSM without having to jump through hoops to create the cdk context first without actually using the values.

[ilko-rbi]

and another one:

asm.Secret.from_secret_attributes(
        scope=scope, id=secret_arn, secret_complete_arn=secret_arn, encryption_key=key
    )

leads to

 `secretCompleteArn` does not appear to be complete; missing 6-character suffix

CDK: 1.85.0

[infa-rasrinivasan]

With the following lines of code I am getting a similar error with 1.85.0. Is this a know bug ? If yes will be fixed in the version.
iam.Role.from_role_arn(self, id="SourceCodeRole", role_arn = source_role_arn, mutable = False)

Error: jsii.errors.JSIIError: ARNs must start with "arn:" and have at least 6 components: ""

[mspolitaev]

Hi team! Still planned to be fixed or waiting for better times?)

[iwt-kschoenrock]

I'm trying to load a JSON string from ParameterStore (StringParameter.valueFromLookup) and getting bit by this (I'm using CDK 1.128.0). cdk synth fails without updating the context if I try to parse the JSON, but works if I skip the parsing.

[adam-phillipps]

This is still happening to me using 1.132.0. I'm trying to do a Function.from_function_arn using an SSM lookup.
Basically hard coding ARNs is not my favorite way to go. Any idea if we can expect a solution or non-hard code solution to this?

[gshpychka]

The workaround is to check whether the value contains "dummy_value_for*", and replacing it with your own dummy value that adheres to the required pattern.

After the first synth, the looked-up value will be written to cdk.context.json and subsequent synths will work as expected.

[adam-phillipps]

That's essentially a hard coding, imho.
We can do string interpolation for most parts of the ARN but the name of a function is still needed so we have to create contracts in our code and/or manually encode at least some part of the ARN e.g.

f"arn:aws:lambda:{env.region}:{env.account}:function:{function_name}"

After that, we still have to add in a ternary or something to account for the problematic dummy-value-for-... format.

The workaround does work but I'm looking forward to that feature that "fixes" this one 👍
Any idea how far down the priorities list this is or an idea if/when it might see some love?

[sdhuang32]

I ran into this too, and found a bit more elegant solution recently, which was to use Lazy values.
To be specific, the following example I use the Lazy.string() of the CDK core module, which encodes your variable as a token and defer the calculation of the string value to synthesis time.

import * as cdk from '@aws-cdk/core';
import * as ssm from '@aws-cdk/aws-ssm';
import * as iam from '@aws-cdk/aws-iam';

const roleArn = ssm.StringParameter.valueFromLookup(this, "/param/testRoleArn");
const role = iam.Role.fromRoleArn(this, "role", cdk.Lazy.string({ produce: () => roleArn }));

It's more general because it can apply to many other use cases besides fromXXXArn() methods, and is in fact used in many places of internal CDK source to get values rendered at synthesis time, as long as the method you throw a lazy value to can handle tokens.

The following is an example that shows why I don't think making ssm.StringParameter.valueFromLookup() return a token is a good idea.

When importing a VPC using ec2.Vpc.fromLookup(), just give the return value of the
ssm.StringParameter.valueFromLookup() and it works.

import * as cdk from '@aws-cdk/core';
import * as ssm from '@aws-cdk/aws-ssm';
import * as ec2 from '@aws-cdk/aws-ec2';

const vpcId = ssm.StringParameter.valueFromLookup(this, "/param/vpcId");
const vpc = ec2.Vpc.fromLookup(this, "vpc", {
  vpcId: vpcId
});

This is because ec2.Vpc.fromLookup() does not check the format of vpcId input parameter,
so it won't cause the construction phase to fail.

However, this method demands your input vpcId to be a concrete string and not a token, so the statement
vpcId: cdk.Lazy.string({produce: () => vpcId }) will result in the error
All arguments to Vpc.fromLookup() must be concrete (no Tokens).
You never know how the callee will deal with the input parameter.

You can store any kind of strings, for different purposes in a SSM parameter,
so it's hard to demand a universal fix for every use cases from inside the valueFromLookup() method,
unless (the ideal fix in my mind) the CDK team does a big re-design and make it fetch the parameters at construction time.

Seems to me at the moment the best way is to understand your use case and have a look at the source code of the method
you're gonna pass your parameter to, and decide what approach you need to pre-process your parameter.

Anyone interested can have a look at my complete analysis.

[gshpychka]

@sdhuang32 thanks for this approach, this is very helpful. Although I often use valueFromLookup precisely in places that don't accept a token, like you mentioned, so this wouldn't work in those cases (and neither would having the lookup return a token).

[goosefraba]

any update on this issue?
Still getting the error when trying to load a SNS topic ARN from SSM parameter store and then do a Topic.fromTopicArn(...)

[zxkane]

I suffered this issue when synthesizing cdk application in ec2 with IAM role(using AWS_DEFAULT_PROFILE to another aws account).

Synthesizing same app in local env using AK/SK works, then updated the context.json in ec2 env as workaround.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.