Generalize PQC KEM KAT Runner + Minor Adaptions in Kyber

[reneme]

KAT Test Generalization

This generalizes the Known-Answer-Test runner for PQC KEMs. Kyber is currently the only consumer, but I'll rebase #3679 onto this and adapt the tests there as well. Also, it probably makes sense to generalize the signature KATs along the same line.

As a result, we can remove the heavy KAT vectors that we had used initially. Instead, the new vectors contain hashed versions of the KEM ciphertexts, and keys. That saves significant checkout space. Also, I reduced the set of KATs from 100 per algorithm parameterization to just 25. The latter is certainly debatable, but IMO 25 should be just good enough.

The general implementation (in src/tests/test_pubkey_pqc.h) is based on an "implementation delegate", that is passed in as a template parameter, rather than a class hierarchy. That way, the general implementation can directly use the public types (e.g. Kyber_PublicKey, Kyber_PrivateKey) instead of hiding key generation and serialization behind abstract methods.

Minor Kyber Adaptions

I've taken over the idea of having a KyberMode.is_available() from the FrodoKEM PR. Albeit, this method being deprecated from the start, I figured it would be good to keep things consistent. Also it helps with the generic KAT test implementation.

Additionally, Kyber::algo_name() reported the fully-qualified name (i.e. "Kyber-512-90s-r3") instead of just "Kyber". We don't do that anywhere else (e.g. ECDSA::algo_name() just reports "ECDSA"). I'm aware that this is somewhat an API break, though. What do you think, @randombit?

Finally, I removed the stream dependency of Kyber. That module was probably needed for the shake_cipher that was replaced by the shake_xof at some point. This must have been a leftover from this cleanup.

[coveralls]

coverage: 92.055% (-0.01%) from 92.065%
when pulling 7ca45d5 on Rohde-Schwarz:test/generic_kem_test
into 2977426 on randombit:master.

[randombit]

I'm aware that this is somewhat an API break, though. What do you think, @randombit?

This seems ok we should just make sure it's mentioned in the release notes.

[randombit]

Seems fine. Left a couple of nits.

One thing we should consider is that if we are deprecated 90s mode, we should make the modern mode mandatory. Then the options become modern+90s, or modern only, but no 90s-only.

[randombit]

This is really hard to read!

Maybe

#if defined(BOTAN_HAS_KYBER)
  if(is_modern())
     return true;
#endif
...
return false;

[randombit]

Use a ternary to get the hash name then a single call to create_or_throw

const std::string hash_name = m_mode.is_modern() ? "SHAKE-256(128)" : "SHA-256";
auto hash = Botan::HashFunction::create_or_throw(hash_name);

[reneme]

Addressed review comments and rebased to latest master.

we should make the modern mode mandatory. Then the options become modern+90s, or modern only, but no 90s-only.

Sounds reasonable. I'll look into that independently.

Edit: As far as I'm aware there's no concept of a "deprecated module", right? Hence, I don't see an obvious way to prohibit configuring like: --minimized-build --enable-modules=kyber_90s. At least not without a special case in ./configure.py, which I would find overkill, frankly.

[randombit]

As far as I'm aware there's no concept of a "deprecated module", right? Hence, I don't see an obvious way to prohibit configuring like: --minimized-build --enable-modules=kyber_90s. At least not without a special case in ./configure.py, which I would find overkill, frankly.

There is a <warning> block you can include in a module info.txt which will print a warning during configuration, but there's no explicit module deprecation annotation. That would be nice to have at least for the cases where our deprecations basically map onto a module 1:1 (eg for obsolete ciphers)