feat(toolkit-lib)!: generic messages do not have a code (#526)

In the [original
RFC](https://github.com/aws/aws-cdk-rfcs/blob/main/text/0300-programmatic-toolkit.md)
we had introduced the concept of "standard" messages for which no
backwards compatibility rules won't apply. In the first developer
preview release of `toolkit-lib` this was implemented as message with a
code ending in `0000` being consider "standard" (or "default") and other
messages.

With the experience from the developer preview, we have found this way
to handle the distinction slightly confusing and putting comparatively
complicated work on the user (they need to know and check for the code
ending in `0000`). It was also easy to accidentally make a mistake by
copy and pasting the message code and not noticing it's specialness.

Instead, we are now proposing to change this implementation: Messages
and requests with a code will be subject to our backwards compatibility
guarantees and can be relied upon by integrators. Standard messages will
no longer have a code and must be treated purely informational. Note
that all requests and any messages with a payload will always have a
code. However some coded messages might not currently have a payload
(but may in future according to our backwards compatibility rules).

This PR reflects these changes and updates the message registry with the
respective documentation.

BREAKING CHANGE: The `code` field on messages is now optional. All
messages with a code are subject to our backwards compatibility
guarantees and can be relied upon by integrators. Messages that
previously had a "default code" ending in `0000` no longer have a code
and must be treated purely informational. Requests and messages with a
payload are always coded.

---
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache-2.0 license

Author: mrgrain

packages/@aws-cdk/toolkit-lib/docs/message-registry.md

@@ -1,11 +1,66 @@
 ---
-title: IoMessages Registry
+title: Messages and payloads
 group: Documents
 ---
-# IoMessages Registry
+# Messages and payloads
 
-| Code | Description | Level | Data Interface |
-|------|-------------|-------|----------------|
+The CDK Toolkit emits *messages* and *requests* to structure interactions.
+A *request* is a special *message* that allows the receiver to respond, if no response is returned the toolkit will continue with a default.
+Messages are unidirectional and always send from the CDK Toolkit to your integration.
+
+All messages include text that is suitable for display to an end-user or logging.
+Some messages also include a unique `code` and a additional payload `data`, providing you with structured information for your integration.
+*Requests* always have a `code` and can be addressed explicitly.
+See {@link IoMessage} and {@link IoRequest} for a complete list of available fields.
+
+## Levels
+
+Messages have a `level` assigned to them.
+Levels are ordered by their importance, with `error` being the most and `trace` being the least important.
+
+| Level      | Description                                    |
+| ---------- | ---------------------------------------------- |
+| `error`  | Error messages that may affect operation.      |
+| `result` | Primary message of an operation.               |
+| `warn`   | Warning messages that don't prevent operation. |
+| `info`   | General informational messages.                |
+| `debug`  | Detailed messages for troubleshooting.         |
+| `trace`  | Very detailed execution flow information.      |
+
+Attached levels are an informal recommendation of what *we* believe is the relevance of a specific message.
+Your integration will always receive all messages of all levels.
+It is up to you to filter out irrelevant messages.
+For standard operations, we recommend to display all messages with level `info` or above.
+
+## Backwards compatibility
+
+Messages and requests are an essential part of the CDK Toolkit's public contract.
+We recognize integrators will build critical workflows depending on these structured interactions.
+To help integrators build with confidence, we provide clear expectations with regards to backwards compatibility of messages.
+
+**Depend only on messages and requests with a `code`. Treat all other messages as informational only.**
+If a message does not have a code, it can change or disappear at any time without notice.
+
+**Only the `code` and `data` properties of a message are in scope for backwards compatibility.**
+Payload data can change, but we will only make type-compatible, additive changes.
+For example we may add new data, but will not remove information.
+
+For the avoidance of doubt, the following changes are explicitly not considered breaking:
+
+- a change to the message text or level,
+- a change to the default response of a request,
+- a change to the order messages and requests are emitted in,
+- the addition of new messages and requests, and
+- the removal of messages without a code
+
+## Registry
+
+This is the complete list of all currently available messages with codes and their respective payload interface.
+We are welcoming requests for additional coded messages and data.
+Please let us know by [opening an issue](https://github.com/aws/aws-cdk-cli/issues/new/choose).
+
+| Code | Description | Level | Payload data interface |
+|------|-------------|-------|------------------------|
 | `CDK_TOOLKIT_W0100` | Credential plugin warnings | `warn` | n/a |
 | `CDK_TOOLKIT_I1000` | Provides synthesis times. | `info` | {@link Duration} |
 | `CDK_TOOLKIT_I1001` | Cloud Assembly synthesis is starting | `trace` | {@link StackSelectionDetails} |

packages/@aws-cdk/toolkit-lib/lib/api/aws-auth/awscli-compatible.ts

@@ -156,7 +156,7 @@ export class AwsCliCompatible {
 
     if (!region) {
       const usedProfile = !profile ? '' : ` (profile: "${profile}")`;
-      await this.ioHelper.sdkDefaults.debug(
+      await this.ioHelper.defaults.debug(
         `Unable to determine AWS region from environment or AWS configuration${usedProfile}, defaulting to '${defaultRegion}'`,
       );
       return defaultRegion;
@@ -173,7 +173,7 @@ export class AwsCliCompatible {
    * @returns The region for the instance identity
    */
   private async regionFromMetadataService() {
-    await this.ioHelper.sdkDefaults.debug('Looking up AWS region in the EC2 Instance Metadata Service (IMDS).');
+    await this.ioHelper.defaults.debug('Looking up AWS region in the EC2 Instance Metadata Service (IMDS).');
     try {
       const metadataService = new MetadataService({
         httpOptions: {
@@ -185,7 +185,7 @@ export class AwsCliCompatible {
       const document = await metadataService.request('/latest/dynamic/instance-identity/document', {});
       return JSON.parse(document).region;
     } catch (e) {
-      await this.ioHelper.sdkDefaults.debug(`Unable to retrieve AWS region from IMDS: ${e}`);
+      await this.ioHelper.defaults.debug(`Unable to retrieve AWS region from IMDS: ${e}`);
     }
   }
 
@@ -223,7 +223,7 @@ export class AwsCliCompatible {
    * Result is send to callback function for SDK to authorize the request
    */
   private async tokenCodeFn(deviceArn: string): Promise<string> {
-    const debugFn = (msg: string, ...args: any[]) => this.ioHelper.sdkDefaults.debug(format(msg, ...args));
+    const debugFn = (msg: string, ...args: any[]) => this.ioHelper.defaults.debug(format(msg, ...args));
     await debugFn('Require MFA token from MFA device with ARN', deviceArn);
     try {
       const token: string = await this.ioHelper.requestResponse(IO.CDK_SDK_I1100.req(`MFA token for ${deviceArn}`, {

packages/@aws-cdk/toolkit-lib/lib/api/aws-auth/proxy-agent.ts

@@ -26,14 +26,14 @@ export class ProxyAgentProvider {
   private async tryGetCACert(bundlePath?: string) {
     const path = bundlePath || this.caBundlePathFromEnvironment();
     if (path) {
-      await this.ioHelper.sdkDefaults.debug(`Using CA bundle path: ${path}`);
+      await this.ioHelper.defaults.debug(`Using CA bundle path: ${path}`);
       try {
         if (!fs.pathExistsSync(path)) {
           return undefined;
         }
         return fs.readFileSync(path, { encoding: 'utf-8' });
       } catch (e: any) {
-        await this.ioHelper.sdkDefaults.debug(String(e));
+        await this.ioHelper.defaults.debug(String(e));
         return undefined;
       }
     }

packages/@aws-cdk/toolkit-lib/lib/api/aws-auth/sdk-provider.ts

@@ -173,10 +173,10 @@ export class SdkProvider {
       // feed the CLI credentials which are sufficient by themselves. Prefer to assume the correct role if we can,
       // but if we can't then let's just try with available credentials anyway.
       if (baseCreds.source === 'correctDefault' || baseCreds.source === 'plugin') {
-        await this.ioHelper.sdkDefaults.debug(err.message);
+        await this.ioHelper.defaults.debug(err.message);
 
         const level = quiet ? 'debug' : 'warn';
-        await this.ioHelper.sdkDefaults[level](
+        await this.ioHelper.defaults[level](
           `${fmtObtainedCredentials(baseCreds)} could not be used to assume '${options.assumeRoleArn}', but are for the right account. Proceeding anyway.`,
         );
         return {
@@ -252,13 +252,13 @@ export class SdkProvider {
         // they are complaining about if we fail 'cdk synth' on them. We loudly complain in order to show that
         // the current situation is probably undesirable, but we don't fail.
         if (e.name === 'ExpiredToken') {
-          await this.ioHelper.sdkDefaults.warn(
+          await this.ioHelper.defaults.warn(
             'There are expired AWS credentials in your environment. The CDK app will synth without current account information.',
           );
           return undefined;
         }
 
-        await this.ioHelper.sdkDefaults.debug(`Unable to determine the default AWS account (${e.name}): ${formatErrorMessage(e)}`);
+        await this.ioHelper.defaults.debug(`Unable to determine the default AWS account (${e.name}): ${formatErrorMessage(e)}`);
         return undefined;
       }
     });
@@ -320,7 +320,7 @@ export class SdkProvider {
     additionalOptions?: AssumeRoleAdditionalOptions,
     region?: string,
   ): Promise<SDK> {
-    await this.ioHelper.sdkDefaults.debug(`Assuming role '${roleArn}'.`);
+    await this.ioHelper.defaults.debug(`Assuming role '${roleArn}'.`);
 
     region = region ?? this.defaultRegion;
 
@@ -354,7 +354,7 @@ export class SdkProvider {
         throw err;
       }
 
-      await this.ioHelper.sdkDefaults.debug(`Assuming role failed: ${err.message}`);
+      await this.ioHelper.defaults.debug(`Assuming role failed: ${err.message}`);
       throw new AuthenticationError(
         [
           'Could not assume role in target account',

packages/@aws-cdk/toolkit-lib/lib/api/aws-auth/sdk.ts

@@ -593,7 +593,7 @@ export class SDK {
     ioHelper: IoHelper,
     logger?: ISdkLogger,
   ) {
-    const debugFn = async (msg: string) => ioHelper.sdkDefaults.debug(msg);
+    const debugFn = async (msg: string) => ioHelper.defaults.debug(msg);
     this.accountCache = new AccountAccessKeyCache(AccountAccessKeyCache.DEFAULT_PATH, debugFn);
     this.debug = debugFn;
     this.config = {

packages/@aws-cdk/toolkit-lib/lib/api/cloud-assembly/private/prepare-source.ts

@@ -60,7 +60,7 @@ export class ExecutionEnvironment implements AsyncDisposable {
   ) {
     this.ioHelper = services.ioHelper;
     this.sdkProvider = services.sdkProvider;
-    this.debugFn = (msg: string) => this.ioHelper.assemblyDefaults.debug(msg);
+    this.debugFn = (msg: string) => this.ioHelper.defaults.debug(msg);
     this.lock = lock;
     this.shouldClean = outDirIsTemporary;
   }
@@ -228,7 +228,7 @@ export class ExecutionEnvironment implements AsyncDisposable {
  * @param assembly the assembly to check
  */
 async function checkContextOverflowSupport(assembly: cxapi.CloudAssembly, ioHelper: IoHelper): Promise<void> {
-  const traceFn = (msg: string) => ioHelper.assemblyDefaults.trace(msg);
+  const traceFn = (msg: string) => ioHelper.defaults.trace(msg);
   const tree = await loadTree(assembly, traceFn);
   const frameworkDoesNotSupportContextOverflow = some(tree, node => {
     const fqn = node.constructInfo?.fqn;

packages/@aws-cdk/toolkit-lib/lib/api/cloud-assembly/stack-assembly.ts

@@ -148,7 +148,7 @@ async function includeDownstreamStacks(
   } while (madeProgress);
 
   if (added.length > 0) {
-    await ioHelper.assemblyDefaults.info(`Including depending stacks: ${chalk.bold(added.join(', '))}`);
+    await ioHelper.defaults.info(`Including depending stacks: ${chalk.bold(added.join(', '))}`);
   }
 }
 
@@ -180,6 +180,6 @@ async function includeUpstreamStacks(
   }
 
   if (added.length > 0) {
-    await ioHelper.assemblyDefaults.info(`Including dependency stacks: ${chalk.bold(added.join(', '))}`);
+    await ioHelper.defaults.info(`Including dependency stacks: ${chalk.bold(added.join(', '))}`);
   }
 }

packages/@aws-cdk/toolkit-lib/lib/api/io/io-message.ts

@@ -36,25 +36,21 @@ export interface IoMessage<T> {
   /**
    * A short message code uniquely identifying a message type using the format CDK_[CATEGORY]_[E/W/I][0000-9999].
    *
+   * Every code releates to a message with a specific payload.
+   * Messages without code are considered generic and do not have a payload.
+   *
    * The level indicator follows these rules:
    * - 'E' for error level messages
    * - 'W' for warning level messages
    * - 'I' for info/debug/trace level messages
    *
-   * Codes ending in 000 0 are generic messages, while codes ending in 0001-9999 are specific to a particular message.
-   * The following are examples of valid and invalid message codes:
-   * ```ts
-   * 'CDK_ASSETS_I0000'       // valid: generic assets info message
-   * 'CDK_TOOLKIT_E0002'      // valid: specific toolkit error message
-   * 'CDK_SDK_W0023'          // valid: specific sdk warning message
-   * ```
-   *
-   * @see https://github.com/aws/aws-cdk-cli/blob/main/packages/%40aws-cdk/toolkit-lib/CODE_REGISTRY.md
+   * @see https://docs.aws.amazon.com/cdk/api/toolkit-lib/message-registry/
    */
-  readonly code: IoMessageCode;
+  readonly code?: IoMessageCode;
 
   /**
    * The message text.
+   *
    * This is safe to print to an end-user.
    */
   readonly message: string;
@@ -83,4 +79,6 @@ export interface IoRequest<T, U> extends IoMessage<T> {
    * The default response that will be used if no data is returned.
    */
   readonly defaultResponse: U;
+
+  readonly code: IoMessageCode;
 }

packages/@aws-cdk/toolkit-lib/lib/api/io/private/io-default-messages.ts

@@ -1,6 +1,6 @@
 import * as util from 'util';
-import type { ActionLessMessage, ActionLessRequest, IoHelper } from './io-helper';
-import type { IoMessageCode, IoMessageLevel } from '../io-message';
+import type { ActionLessMessage, IoHelper } from './io-helper';
+import type { IoMessageLevel } from '../io-message';
 
 /**
  * Helper class to emit standard log messages to an IoHost
@@ -10,24 +10,18 @@ import type { IoMessageCode, IoMessageLevel } from '../io-message';
  */
 export class IoDefaultMessages {
   private readonly ioHelper: IoHelper;
-  private readonly category: 'TOOLKIT' | 'ASSEMBLY' | 'SDK';
 
-  constructor(ioHelper: IoHelper, category: 'TOOLKIT' | 'ASSEMBLY' | 'SDK') {
+  constructor(ioHelper: IoHelper) {
     this.ioHelper = ioHelper;
-    this.category = category;
   }
 
   public async notify(msg: Omit<ActionLessMessage<unknown>, 'code'>): Promise<void> {
     return this.ioHelper.notify({
       ...msg,
-      code: levelToCode(this.category, msg.level),
+      code: undefined,
     });
   }
 
-  public async requestResponse<T, U>(msg: ActionLessRequest<T, U>): Promise<U> {
-    return this.ioHelper.requestResponse(msg);
-  }
-
   public async error(input: string, ...args: unknown[]): Promise<void> {
     return this.emitMessage('error', input, ...args);
   }
@@ -65,7 +59,6 @@ export class IoDefaultMessages {
 
     return {
       time: new Date(),
-      code: levelToCode(this.category, level),
       level,
       message,
       data: undefined,
@@ -76,14 +69,3 @@ export class IoDefaultMessages {
     return this.ioHelper.notify(this.msg(level, input, ...args));
   }
 }
-
-function levelToCode(category: string, level: IoMessageLevel): IoMessageCode {
-  switch (level) {
-    case 'error':
-      return `CDK_${category}_E0000`;
-    case 'warn':
-      return `CDK_${category}_W0000`;
-    default:
-      return `CDK_${category}_I0000`;
-  }
-}

packages/@aws-cdk/toolkit-lib/lib/api/io/private/io-helper.ts

@@ -20,18 +20,14 @@ export class IoHelper implements IIoHost {
    * Simplified access to emit default messages.
    */
   public readonly defaults: IoDefaultMessages;
-  public readonly assemblyDefaults: IoDefaultMessages;
-  public readonly sdkDefaults: IoDefaultMessages;
 
   private readonly ioHost: IIoHost;
   private readonly action: ToolkitAction;
 
   private constructor(ioHost: IIoHost, action: ToolkitAction) {
     this.ioHost = ioHost;
     this.action = action;
-    this.defaults = new IoDefaultMessages(this, 'TOOLKIT');
-    this.assemblyDefaults = new IoDefaultMessages(this, 'ASSEMBLY');
-    this.sdkDefaults = new IoDefaultMessages(this, 'SDK');
+    this.defaults = new IoDefaultMessages(this);
   }
 
   /**