Shared library versioning scheme (SOVERSION)

[vt-alt]

As pointed there open-quantum-safe/openssh#111 (comment) liboqs would benefit from proper library versioning. (Sometimes called API version or SOVERSION). This is different thing than usual project versioning. It's purpose to maintain binary (ABI) compatibility between executable binaries and shared libraries. Currently, liboqs just copy project version into SOVERSION which does not reflect backward incompatible ABI changes (which may cause obscure program errors when binary that is linked to a newer library used on system with older library — current versioning allows this, because major version number (that signifying backward-incompatible ABI change) is not incremented (while library does have backward incompatible ABI changes).

References:

• General explanation https://stackoverflow.com/questions/12637841/what-is-the-soname-option-for-building-shared-libraries-for
• GNU tooling https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html (Note that their -version-info arguments order is different from what will actually appear in the library suffix - they say current:revision:age, but library will be named like liboqs.so..current.age.revision).
• Cmake property https://cmake.org/cmake/help/latest/prop_tgt/SOVERSION.html

[vt-alt]

I found you currently using SOVERSION 0 in src/CMakeLists.txt for oqs target. Perhaps, you just need to increment it sometimes. I tried already — it looked unusual in compare to what libtool does, but seems valid too. I tried to set SOVERSION to 1. It creates SONAME liboqs.so.1 with symlink to liboqs.so.0.7.0. In that case binaries will link to liboqs.so.1 which is good.

[vt-alt]

I think Dilithium ABI (that was caused heap corruption) is changed with changes like this: https://github.com/open-quantum-safe/liboqs/commits/main/src/sig/dilithium/sig_dilithium.h, for example when any _length_ define is increased. abipkgdiff cannot detect this, because (I think) it is change in define and buffers are passed using generic uint8_t * pointer of unspecified length.

So, any release after such change should have incremented SOVERSION.

[baentsch]

Thanks again for the explanation and tests.

So, any release after such change should have incremented SOVERSION.

One question related to this: In your opinion/experience, would a "regular" (read: couple of times a year) increase of SOVERSION not be "unusual" or even create problems for downstream users of the library? Reason for asking: liboqs basically is a collection of many different upstream algorithms -- each of which is separately enhanced (one example of which is Dilithium as you point out above) by a separate team.

Thus --short of manually analyzing each upstream code import in detail to judge the need for a version increase-- the one way to be sure such (possibly) "ABI-breaking" changes are detected/highlighted would be to increase SOVERSION each time a release is done after importing any new upstream algorithm version. The latter is typically done automatically via copy_from_upstream to minimize the need to inspect all upstream code.

In other words, would a simple ("preventative") SOVERSION++ be sufficient (advisable?) each time (i.e., for each release) after taking in any updated upstream code (or intentionally changing liboqs' common APIs)?

[vt-alt]

I think not increasing SOVERSION while having ABI change can cause more problems, than increasing SOVERSION while ABI is not changed.

Thus, I would recommend to increase SOVERSION for libqos if you are not sure that ABI changed or not, on every release (no need more frequent). And especially while it is not stabilized library (I conclude this from its version starting from 0).

This SOVERSION change will force all library clients to ne rebuild with a new library. This is OK if there are few clients. As seems to be case for liboqs. (In ALT Linux we have only OQS openssh.)

If there are many clients, thing will be getting much more complicated, and it's better to only increase SOVERSION when needed.

[dstebila]

We've discussed this further, and will take your suggested approach of increasing SOVERSION in future releases. We'll try to check for particular ABI changes, but when in doubt will increase. The plan would be to do this starting from our next release. What number should we use there? In particular, @vt-alt if you've used .1, would you prefer us to use .2 for the next one?

[vt-alt]

In particular, @vt-alt if you've used .1, would you prefer us to use .2 for the next one?

Yes, I used already 1, so it would be nice if you start incrementing from 2. Thanks!

[dstebila]

In particular, @vt-alt if you've used .1, would you prefer us to use .2 for the next one?

Yes, I used already 1, so it would be nice if you start incrementing from 2. Thanks!

Okay, will do. I'm going to close this issue; I've made a note in our release process wiki page to remember to do this upon our next release.

[dstebila]

Hi @vt-alt, we have a follow-up questions about SOVERSION numbering. In #1470 we're doing an algorithm update which changes algorithm interoperability, so we are planning to bump the SOVERSION. Should we bump the SOVERSION at the time of commit, or only at the time of release?

[vt-alt]

I think what is really important is to have soversion changed appropriately between releases (including pre-releases) and binary packages. Perhaps that's why libtool manual suggests to change soversioning only on releases:

Update the version information only immediately before a public release of your software. More frequent updates are unnecessary, and only guarantee that the current interface number gets larger faster.

But to change more frequently is not a bad thing (IMHO) and it only "wastes" numbers which are plenty. There maybe benefits to update soversion after/during actual ABI change -- for example if someone wishes to build a package from intermediate commit (their build will not break dependent packages), or you may document ABI change in a commit updating soversion.