core: cdk diff --fail should return a unique exit code when there is a difference

[l0b0]

Describe the bug

Detecting a difference is very different class of error from running an invalid command or not being authorised to even run a diff at all. This should be clearly indicated in the exit code, to make CDK automation easier. With the current functionality, you must check standard error to see if there's a difference, which is much harder than checking the exit code, and even harder than just checking standard output.

Basically, with the current functionality you need to do something like this in Bash to keep the original output while detecting an actual diff:

cdk diff --fail 2> >(tee diff.log >&2)
exit_code=$?
if (( exit_code )); then
    if grep '^✨  Number of stacks with differences: 0$' diff.log; then
        echo 'Some other issue'
        exit "$exit_code"
    else
        echo 'Diff detected!'
    fi
else
    echo 'No diff detected'
fi

Expected Behavior

cdk diff --fail should return a specific exit code among one of the 254 exit codes other than 0 and 1 when a difference is detected.

Current Behavior

cdk diff --fail returns exit code 1 in case of a difference. This is the same exit code as a host of completely different classes of errors.

Reproduction Steps

Run cdk diff --fail against a stack with a difference.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.93.0 (build 724bd01)

Framework Version

No response

Node.js Version

v18.17.1

OS

NixOS

Language

Typescript

Language Version

5.2.2

Other information

No response

[SankyRed]

Hey @l0b0 after extensive team discussion regarding altering the exit codes, we collectively decided against implementing unique ones. Since the current exit codes 0 and 1 specify the use case within the description of --fail specifying option('fail', { type: 'boolean', desc: 'Fail with exit code 1 in case of diff' }). Furthermore, modifying the exit codes would constitute a breaking change, potentially disrupting existing CI/CD processes for our customers. Therefore, unless there is a specific bug that needs to be fixed, maybe not wise to change it.