[Format][C] Introducing new API functions in a backwards compatible way

[lidavidm]

This issue proposes a way to manage API additions for C. (Go and Java are easier.)

• New structs and constants are OK without breaking compatibility.
• Existing structs, function prototypes, and constant values cannot be changed or renamed.
• New functions are OK but require adding a new driver struct.

That is, adding new functions will require us to do this:

struct AdbcDriver { /* ... */ };
/* ADBC API version 1.1.0 */
struct AdbcDriver110 {
  struct AdbcDriver base;

  /* New function prototypes go here */
};

When a driver is initialized, it is given a version ID and a pointer to one of these tables. A driver implementing 1.0.0 will not recognize the new version ID and will return an error. A driver implementing 1.1.0 should ideally be able to recognize both the version ID for 1.0.0 and 1.1.0.

The driver manager can manage this: it will try 1.1.0, failing that, it will try 1.0.0 and polyfill the new functions. The driver manager would always expose a 1.1.0 API.

We should aim to 'batch up' additions of new functions to avoid implementation complexity. Hence why I've created the milestone on GitHub. Also, we should figure out how to handle this with respect to voting (since this is a 'format') and merging. Possibly, new API specifications should get merged into a branch, which can be merged/cherry-picked into the main branch after the vote. This lets us prototype the new implementations alongside the new specification as well.