Plugin error messages not straightforward for users

[Two-Hearts]

What is not working as expected?

As a user of Notation CLI, the following command:
notation sign --id $Id --plugin $my_plugin $my_image

could return the following error:
Error: describe-key command failed: VALIDATION_ERROR: {plugin specific error details}

This error message is not very user friendly:

1. What does describe-key command stand for?
2. What is been validated here that caused the VALIDATION_ERROR?

What did you expect to happen?

Move the details into verbose/debug logs and a more user-friendly error message might be:
Error: invalid plugin request: {plugin specific error details}

How can we reproduce it?

notation sign --id $Id --plugin $my_plugin $my_image with a misconfigured id.

The related error message is generated in notation-go/plugin/plugin.go.

Describe your environment

applies to all env

What is the version of your notation-go Library?

notation-go v1.1.0 with notation CLI v1.0.0+

[FeynmanZhou]

Thanks @Two-Hearts for reporting this issue.

In the short term, I suggest improving the error message to be more descriptive and actionable as follows:

Error: failed to sign with the plugin $my_plugin. Invalid input passed to "--id". 
Follow this format to input the ID "https://<vaultname>.vault.azure.net/<keys|certificate>/<name>/<version>"

In the long term, as this is a plugin-specific problem, we can improve the experience in the plugin side.
I created a GitHub issue in Notation Azure Key Vault plugin: Azure/notation-azure-kv#180. The error message will not be displayed after this issue has been resolved in Notation AKV plugin.

[JeyJeyGao]

Error: failed to sign with the plugin $my_plugin.

This part is generated in notation-go. We need a PR to fix it.

Invalid input passed to "--id". Follow this format to input the ID "https://{vault-name}.vault.azure.net/certificates/{certificate-name}"

I simplified the example in the second part to make it easier for users to understand. This will done in Azure/notation-azure-kv#181

@FeynmanZhou

[Two-Hearts]

@FeynmanZhou This issue is about the first half of the error message, Error: describe-key command failed: VALIDATION_ERROR:..., which is generated by notation-go library, the 2nd half is plugin specific.

[FeynmanZhou]

@Two-Hearts Yes. We just need to improve the first part (line) of the error message in notation-go. The second part (line) is an actionable recommendation for users which could be improved in the plugin side.

Does the first part look good to you?

[Two-Hearts]

@Two-Hearts Yes. We just need to improve the first part (line) of the error message in notation-go. The second part (line) is an actionable recommendation for users which could be improved in the plugin side.

Does the first part look good to you?

Yep, the first part LGTM. @FeynmanZhou