Proposal: Implementing Versioning Across Application Serialization Structures

[ajnavarro]

Summary

We propose implementing versioning across various serialization structures within our application to enhance compatibility, maintainability, and future scalability. This entails versioning in three key areas:

Versioning RPC API:

Implementing versioning in RPC API endpoints by prefixing paths with version numbers (e.g., /api/v1, /api/v2) ensures backward compatibility while allowing for iterative updates and enhancements. This practice enables seamless transitions for clients using different versions of the API and facilitates the graceful evolution of the API over time.

Versioning Data Storage Format:

Introducing versioning to the storage format of our application's data enables smooth migrations when format changes occur. By incorporating version numbers into the data storage mechanism, we ensure that data remains accessible and usable across different versions of the application. This practice is essential for maintaining data integrity and ensuring uninterrupted operation during updates or migrations.

Versioning Data Structures for Inter-Node Communication:

Versioning complex data structures transmitted between nodes enhances interoperability and compatibility across distributed systems. By versioning these data structures, we can accommodate changes and updates without causing disruptions to the communication protocol.

Additional considerations

• Documentation and Communication: Alongside implementation, comprehensive documentation should be provided to guide users and developers on handling versioned structures effectively.
• Testing and Validation: Testing processes should be implemented to validate backward and forward compatibility across different versions of serialization structures. Automated testing suites can help ensure that versioning mechanisms function as intended and do not introduce regressions or compatibility issues.
• Version Compatibility: It's imperative to emphasize that versioning structures do not imply a carte blanche to break compatibility with each version increment. Each version should be carefully specified, with thorough consideration given to backward and forward compatibility.

[ajnavarro]

Future Versioning Strategy Proposal

Description

After attempting to add a VERSION field to various public interfaces, I realized this approach was making the code less readable and generally worse without any real benefit. I propose we consider versioning only after V1, leaving V1 as it is at the final release.

Proposal

To handle versioning in future major versions, we need to consider the following aspects:

RPC APIs: Version Through Path, Not by version Method

Using the path for versioning allows clients time to update to the latest version while the previous versions remain available in subsequent releases. Versioning via a method (as we are doing right now) will break clients until they update to be compatible with new changes.

Data Storage Formats: Version in Header

Add a VERSION value in the header after a magic word identifying the file format or follow guidelines for the specific storage format used. For example, follow Protobuf/(Amino?) best practices:

• Dos and Don'ts
• API Best Practices

Specifically, do not use the same schema for wire and storage.

Avoid Versioning Twice

Avoid having multiple versions for the same data. For instance, if an RPC endpoint is already versioned, adding an extra version for the returned object is unnecessary and discouraged.

Data Storage Migrations

Applications control their data storage systems. When a binary is updated, we can detect and initiate a migration process. This is a solved problem for various storage systems:

• SQL:
  • golang-migrate/migrate
  • go-gormigrate/gormigrate
• Key/Value Storages: It's even simpler. Create a new DB file with a VERSION prefix containing all the data in the new format. If successful, the old database file can be removed by the node management team to prevent data loss if issues arise.

Other aspects to take care about

• Documentation and Communication: Maintain detailed documentation for each API version, including migration guides and changelogs. Documentation is highly tied to code and must be versioned as the code is.
• Versioning Policy: Adopt a clear versioning policy such as semantic versioning.
• Monitoring and Analytics: Implement usage tracking for different API versions.
• Security Considerations: Apply security patches to all supported versions and regularly review security practices.
• Backward Compatibility Period: Define and communicate a standard period during which older versions will be supported post-release of a new version. This point is really important because if we decide to support too many versions, might be an important load when keeping up with bug fixes and security patches.

[ajnavarro]

Summarizing the final decision for versioning: YOLO

• RPC: we can make changes without moving to a major version (adding new methods, adding new fields, and so on)
• Storage and VM: META - Backward Compatibility #1896 (comment)