soroban-cli/rpc: future releasing, semver, api stability

[leighmcculloch]

This issue is some thoughts I shared internally on slack yesterday, but should have posted in a public space for discussion:

v20 is both the protocol the CLI and RPC support, as well as its major version, so from here we should avoid breaking the interfaces (command line / API) to both the RPC and CLI.

Two things comes to mind:

1. There's a few PRs open that are usability improvements in the CLI, but they break the interfaces. Such as moving the config subcommands. We should move ahead with them, but make the changes backwards compatible so that someone who has a script running a specific subcommand will continue to work even if the subcommand has moved?

   In 'clap' it's relatively easy to do this by keeping an old command that calls the new command, and then marking that old command as hidden or at least adding the word "deprecated" to its help text.

   We can then plan to remove all deprecated commands in the v21 release.

2. We currently test each CLI release with only the latest RPC. That's not really enough. The CLI is a client of the RPC and I think a v20 CLI should work with any v20 RPC. It won't always be easy for folks to have multiple CLIs installed if they interact with multiple RPCs which could be different versions. It's also a poor user experience to force people to upgrade CLIs for every minor or patch release of the RPC. This problem isn't unique to the CLI/RPC, as there is also the js-stellar-sdk and other clients, and it won't be realistic to coordinate updates and upgrades to all synchronously.

   We can maintain a compatibility guarantee for both tools by:

   1. Don't add new required params to existing RPC endpoints that would cause existing CLI requests to fail, or requests from any client (js-stellar-sdk, or custom). (Until soroban-rpc: adding optional params breaks clients using params array format stellar-rpc#13 is resolved don't add any new params.)

   2. Keep the CLI tolerant of new RPC response fields being missing. If we add a new response field to the RPC that CLI will consume, the CLI should assume its optional. This principle can be applied to any client in development, so the js-stellar-sdk, or typescript bindings.

   3. Test each CLI release with a range of RPC releases. Probably need to do same with other clients too.

   4. Invest in some static API semver checks so that (iii) is less necessary.

   5. Split up the CLI and RPC repos, so that we don't develop them in such a coupled manner, given it is just one of many clients of the RPC and the CLI needs to work with a range of RPC versions, not just the latest. Related discussion here: enable independent releases of soroban-rpc and soroban-cli #1115

[janewang]

1. Breaking changes in the CLI: imo it's the best practice for a tool to be backward compatible when possible for a least sometime to give developers the time to adopt a new command. I agree the idea of keeping the old commands which call the new commands. For breaking changes in tooling it could be good to also announce these changes on Community Dev Calls in addition to release notes.
2. An idea is the CLI could have a preferred version of RPC that's defined in the release. If the CLI is configured to a different RPC version that the dev needs to set a config param to --allow-mismatch so the dev is aware that she may run into issues. Allowing CLI to maintain compatibility guarantees to multiple versions of RPC also makes sense but unclear how much lift that adds to dev and testing. Devs ideally should be working with the latest stable release of RPC or an upcoming release if they want to test the newest features.

[leighmcculloch]

2. An idea is the CLI could have a preferred version of RPC that's defined in the release. If the CLI is configured to a different RPC version that the dev needs to set a config param to --allow-mismatch so the dev is aware that she may run into issues.

I think this shifts the responsibility too much onto the user. Other clients, such as the js-stellar-sdk, will have to work across multiple releases. It will be impractical to ask developers of applications to upgrade, or downgrade, to specific versions of the js-stellar-sdk before connecting to different RPC servers. If other clients need to be capable, I think the CLI can be too. The principles above are also not unique or extravagant. The development of any client/server software typically requires strategies for ensuring compatibility is maintained over time and there are standard patterns.

[leighmcculloch]

On other thing we need to ensure when adding optional params to the RPC, is that they always need to be appended to the list of params, not inserted earlier in the list. This assumes we continue to support array style param requests (stellar/stellar-rpc#13).

[tupui]

Breaking changes in the CLI: imo it's the best practice for a tool to be backward compatible when possible for a least sometime to give developers the time to adopt a new command. I agree the idea of keeping the old commands which call the new commands. For breaking changes in tooling it could be good to also announce these changes on Community Dev Calls in addition to release notes.

A drop by comment from some past experience with the Python community if I may 😃

I would suggest to be Cristal clear from the start on your policy. We got bitten by this with SciPy so many times by having something ambiguous around. Unless there is a clear message saying in version X or after the following Y date something will be drop, people really ignore deprecations and come raising issues and complain 😅

If that can help to give an example, what we do now is:

1. have a process to propose the deprecation openly so that the community can weigh in before doing anything (here as well, you need to set some rules on the "RFC")
2. raise a deprecation warning at runtime for the decided period (1 version to announce and then we remove, for us that's 1 year as we do 2 releases a year). I am still new to Rust, so not sure how that's playing out there.
3. Do pay attention to feedbacks if people really want something different. We had cases were we had to backport a lot of things and that's always painful both in terms of maintenance but also communication.

Some avenue to explore, the stripe way 😅 The simplified version is that they are always compatible and achieve this by writing migrations from version A to B. Then you can more or less always go from any version to any version by following the migrations.

Forgot to add: all the best for this amazing perspective 🚀

[janewang]

Blocked due to recent release issues. Will work on this again by Platform

[leighmcculloch]

@tupui Thank you for sharing your learnings and insights. I agree, there needs to be an explicit guarantee / compatibility statement, and I like the idea of being explicit with deprecations about when they will eventuate into actual removal, and how to provide feedback on them as well. We do have some commands in the CLI marked as deprecated, but no other plans beyond that documented.

This is something I'm working on. ❤️