Cross-App Resource Sharing

[ivawzh]

I found cross-cdk-repo resource importing is a massive pain point of using AWS CDK. Majority of the public static methods with the pattern of from...Atributes(scrope, id, attrs) (such as fromVpcAttributes, fromClusterAttributes, fromDomainNameAttributes, etc) are hard to work with because they require many attributes. For example, fromClusterAttributes requires the argument of securityGroups. I don't even know how to make CfnOutput of the array cluser.connection.securityGroups via CDK while documentation doesn't mention the securityGroups will always be in a fixed order.

Use case

I believe cross-repo resource sharing is a common scenario for enterprise, SOA, and micro-service architectures.

Proposed Solution

Instead of importing individual resources with attribute lookup and CfnOutputs, now my idea is to utilize the programming language and import the entire dependent CDK App as a library. And enhance the App class to better support in-memory children resource lookup.

For example,

At Infrastructure CDK app

const infrastructure = new cdk.App()

new VpcStack(infrastructure, 'VpcInfrastructure', ...)
new ClusterStack(infrastructure, 'ClusterInfrastructure', ...)
new KmsStack(infrastructure, 'KmsInfrastructure', ...)

export default infrastructure

At micro-service CDK app

import infrastructure from '@private-packages/infrastructure-app'

const microServiceA = new cdk.App()

const vpcInfrastructure = infrastructure.lookupByStackId('VpcInfrastructure')
const clusterInfrastructure = infrastructure.lookupByStackId('ClusterInfrastructure')

new EcsServiceStack(microServiceA, 'MicroServiceA', {
	cluster: clusterInfrastructure.cluster,
    vpc: vpcInfrastructure.vpc,
    ...
})

The only API change I need is the new method app.lookupByStackId(id: string): Stack. I can also build my own function lookupStack(app: App, stackId: string): Stack if you could provide me some guidelines on how to implement the lookup.

Other

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[ivawzh]

I tried to implement the idea and created a repo https://github.com/ivawzh/cdk-playground#plan-1---multiple-cdk-apps.

It is failing on deploying the service repo for error:

Error: Cannot reference across apps. Consuming and producing stacks must be defined within the same CDK app.
    at resolveValue (./infrastructure/node_modules/@aws-cdk/core/lib/private/refs.ts:48:11)
    at Object.resolveReferences (./infrastructure/node_modules/@aws-cdk/core/lib/private/refs.ts:29:24)
    at Object.prepareApp (./infrastructure/node_modules/@aws-cdk/core/lib/private/prepare-app.ts:26:5)
    at App.prepare (./infrastructure/node_modules/@aws-cdk/core/lib/app.ts:153:5)
    at App.onPrepare (./infrastructure/node_modules/@aws-cdk/core/lib/construct-compat.ts:111:10)
    at Node.prepare (./infrastructure/node_modules/constructs/lib/construct.ts:429:12)
    at Node.synthesize (./infrastructure/node_modules/constructs/lib/construct.ts:386:10)
    at Function.synth (./infrastructure/node_modules/@aws-cdk/core/lib/construct-compat.ts:231:22)
    at App.synth (./infrastructure/node_modules/@aws-cdk/core/lib/app.ts:142:36)
    at process.<anonymous> (./infrastructure/node_modules/@aws-cdk/core/lib/app.ts:121:45)

Is the cross-app reference limitation necessary on AWS CDK?

My assumption is the highest-level units store in AWS are CloudFormation stacks. AWS has no concept of app on server-side. If that is the case, as long as I can obtain stack information on my client-side, why would AWS CDK limit me from cross-app referencing?

[eladb]

@ivawzh this is an interesting proposal. I am moving this issue to the RFC repo. I would recommend creating an RFC and following the RFC process to get this explored a bit deeper.

[tqhoughton]

Has there been any movement on this? I would very much support a more intuitive way to split up CDK apps across different projects, as of now the CDK assumes you want to deploy all of your stacks in the same project and reference things by exposing public properties of each stack.

As soon as you try to split up CDK projects into smaller projects this all falls apart, and you are stuck with 1 of 3 options:

1. Hard code all of your id refs between stacks and use fromId functions to reference resources created by other projects
2. Use Fn::ImportValue and Fn::ExportValue to create explicit links between stacks. This sort of works but introduces issues when an exported value needs to be updated or removed and requires you to manage ordered deployments to get out of that state.
3. Save resource ids as outputs in SSM and reference those SSM parameters on deploy of stacks that need those resources. This is probably the easiest way to manage things currently, but still requires things to be deployed in a specific order if standing up stacks in a new account.
4. Use private packages to define your other cdk stacks, then import all of your stacks into a single project and use the CDK normally. This gets around needing to manage deploy orders but kind of goes against microservice architecture design and can be annoying to set up if you aren't already using a monorepo.

I don't know what a better solution would be, maybe this has to be functionality outside of current CloudFormation support, but some way to more easily detect deploy orders when referencing things across projects and be able to automatically deploy projects that are dependent on values in that project/stack would be a huge improvement.

Maybe this is all a cautionary tale against splitting up CDK stacks across multiple projects, but I wanted to jot some things down in case someone else comes across this issue so they can consider all of the options.

[evgenyka]

No plans to implement because of large effort required and there is not enough upvotes that this feature is beneficial to all users.

[benjaminpottier]

Could we reopen this? I think this is very much a desired feature and there really isn’t a good solution currently.