pipelines: additionalInputs documentation is lacking in detail

[HansFalkenberg-Visma]

Describe the issue

We have one repository with tooling and another repository with application code to be analyzed. This seems to fit with using toolingRepo as the main input and appRepo as an additionalInput. However, the documentation does not have enough detail to set this up correctly without trial and failure.

At first attempt I got the error message

additionalInputs: "../app-repo" must not exist yet. If you want to merge multiple artifacts, use a "cp" command.

Google led me to #17224 which is unhelpful, because the error message was actually true: The sibling directory app-repo did indeed exist.

It turns out CodePipeline action provider "GitHub (Version 2)" and CodePipeline action input artifacts works together to put the repository source code in a sibling directory named the same as the repository. So the documentation may mention something like:

The construct works by making the specified directory into a symlink to the output file set. Note that the output file set may already exist as a sibling directory. Eg. source code can exist with the repository name as a sibling directory name, in which case symlink creation will fail and you need to use a different sibling directory name.

Once I had understood the above, my next task failed because a file was not found. It appears that when there are multiple source repositories, any step that does not explicitly set its input will just get one of them, including ones used in additionalInputs. So the documentation might make it clear:

When you have specified a CodePipelineSource as an additional input, CDK will no longer know which source to use as the default for other steps. You must explicitly specify input for all steps in the pipeline.

Links

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.pipelines-readme.html#additional-inputs
https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.pipelines.ShellStepProps.html#additionalinputs

[HansFalkenberg-Visma]

Here's an example that showcases the two issues above. The documentation's example could be changed to incorporate elements from this example.

This example also illustrates that it is not necessary to have all source code repositories as input for the synth step. A CodePipelineSource used in any step will be part of the Source stage.

    const toolingRepo = pipelines.CodePipelineSource.connection('my-org/tooling-repo', 'main', {
      connectionArn: codeStarArn,
    });
    const appRepo = pipelines.CodePipelineSource.connection('my-org/app-repo', 'main', {
      connectionArn: codeStarArn,
    });

    const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
      synth: new pipelines.ShellStep('Synth', {
        input: toolingRepo,
        commands: ['npm ci', 'npm run synth'],
      }),
    });

    pipeline.addWave('Analyze', {
      post: [
        new pipelines.CodeBuildStep('AnalyzeStep', {
          input: toolingRepo,
          additionalInputs: {
            // NOT '../app-repo'
            ['../target']: appRepo,
          },
          commands: [
            'toolPath=$PWD/tool',
            // same as key in `additionalInputs`
            'cd ../target',
            'cat app-repo-file.json | "$toolPath"',
          ],
        }),
      ],
    });

    pipeline.addWave('AnotherStage', {
      post: [
        new pipelines.CodeBuildStep('AnotherStep', {
          // might default to `appRepo` if not specified
          input: toolingRepo,
          commands: [
            // might be obvious, but: ../app-repo does not exist
            'tool --whatever'
          ],
        }),
      ],
    });

[pahud]

Thank you for bringing this to our attention. Yes we should improve the doc to get it clarified. Would you like to submit a PR for this?

[dennisvang]

There also seems to be some issue with additionalInputs and powershell, on windows images.

[dennisvang]

Thank you for bringing this to our attention. Yes we should improve the doc to get it clarified. Would you like to submit a PR for this?

@pahud Wouldn't it be fitting for the AWS engineers responsible for additionalInputs to write their own documentation?

I assume they are the ones most familiar with the idiosyncrasies of their implementation.