Do you have any guidance on when the six-beta branch gets merged?

I guess parts of it I'll merge relatively soon after IKEv2 identifiers for ML-KEM have been (pre-)allocated. Many changes are not technically dependent on it. For instance, multiple key exchanges also work with classic (EC)DH methods and the rekey collision refactoring is only in that branch because the code it depends on changed a lot. But we wanted these changes to be part of the next major release, so merging the branch to master means that the next release will be 6.0 and it would make sense to have at least one standardized KEM to go with it. However, that doesn't mean we already need to have all plugins ready by then. There will definitely be some time after the merge to get the multi-KE changes tested in the wild a bit more (we also have to sort out some stuff with changing the default plugin selection). I also hope FrodoKEM gets an identifier (via ISO and then IETF) before we eventually make the release as we have a plugin for it that doesn't depend on any third-party libraries.

...OpenSSL, I suppose, as EVP_PKEY_encapsulate() will just allocate the shared secret from some internal RNG (setting the private key from a static value might be possible via OSSL_PKEY_PARAM*, similar to what we do for DH, if they extend that for KEMs).

Yes, that will be the case for OpenSSL. I think this is why the Kyber authors specified the KATs with a seed and the specific DRBG algorithm. This would've been the only way to make the whole operation deterministic. We can construct an EVP_PKEY and related CTX from raw bytes, but we can't force the shared secret value. I took a look at the OpenSSL RNG's man page and they don't make it simple (and explicitly discourage) to override the library's RNG (EVP_RAND). I agree, that it'll be interesting to see what OSSL_PKEY_PARAMs upstream OpenSSL decides to support for PQ-KEMs. The OQS Provider only uses this interface to surface the keys of a hybrid contruction. Notably, they don't support overriding the RNG through the OpenSSL Provider interface.

To complement your investigation of other libraries, I can confirm that BoringSSL also doesn't support RNG overrides in the public interface. It has a private interface for testing KATs here. Similarly, we have this interface in AWS-LC which was created for a similar usecase of passing KATs in third party software, but it's really hacky and is planned for removal.

First, it's not actually optional at the moment. While it's documented in the header as such, the crypto tester will just call it anyway and cause a crash. You could provide a dummy implementation that does nothing (e.g. return_true()), but this will eventually cause the test vectors to fail anyway

Would it be possible to downgrade this test to a warning for certain libraries?

Now the problem with the latter is that algorithms can be disabled at runtime based on failing those tests (for our CI it's obviously not ideal either). So if we actually make the method optional, I'm not sure what to do with such implementations. Maybe this needs a new crypto_test option that causes KEM implementations that are skipped due to a missing set_seed() to get accepted.

How open is the project to such a workaround? From my perspective, I wouldn't mind losing this automatic self-test and unload functionality for AWS-LC. I am concerned about how this would be handled with upstream OpenSSL, that seems less acceptable.

It looks likely that we will be adding "derandomized" versions of the keygen and encapsulate APIs for ML-KEM in AWS-LC to support this use case and other efforts. Basically, these versions of the APIs will accept a buffer containing the 32-bytes of randomness needed for each of these operations. This means we can support strongSwan's testing by calling into the DRBG passed in by the test_vector plugin and passing the results into the derandomized APIs. Does that work?

I do suggest simplifying strongSwan's KEM vectors to just specify the random bytes to use instead of accepting a seed for a DRBG. The NIST CAVP vectors simply specify the 32-bytes of entropy directly which avoids the indirection (z is defined in the standard).

Does that work?

That sounds similar to what wolfSSL provides, right? (i.e. something like wc_KyberKey_MakeKeyWithRandom().) If so, that should work. But as mentioned above, it's not ideal to call different API functions during the tests and in regular use, so we'd probably always use that API and allocate randomness via an rng_t/drbg_t outside even when not testing.

I do suggest simplifying strongSwan's KEM vectors to just specify the random bytes to use instead of accepting a seed for a DRBG. The NIST CAVP vectors simply specify the 32-bytes of entropy directly which avoids the indirection (z is defined in the standard).

We need test vectors not for single operations (which the CAVP vectors seem to provide) but for the complete key exchange. So what's called "seed" in our test vectors has to be the source for all the secrets (for both ends of the exchange, not just z for the initiator).

For classic (EC)DH the seed is just the secret key for both ends that gets split in half before passing one halve to each instance. And with the KATs during the submission rounds, the seeded DRBG provided all the secrets when called in the right order (note that the same DRBG instance is passed to both KEM instances). The latter also worked for all algorithms the same way (which is particularly important for the oqs plugin that wraps multiple of them).

But we could theoretically work with static entropy values by providing a pseudo-DRBG that just hands it out as is (I feel like we had something like this at some point). So the seed could then just be the concatenated z and m values. If we continue to support HQC, we have to change the DRBG allocation anyway, so it might be possible to support such "static" vectors too (i.e. let the test vector specify the DRBG type and provide something like DRBG_STATIC for such vectors and the others could continue to select DRBG_CTR_AES256 or something else, like SHAKE256 for HQC, although that's currently not a DRBG type).

That sounds similar to what wolfSSL provides, right? (i.e. something like wc_KyberKey_MakeKeyWithRandom().) If so, that should work. But as mentioned above, it's not ideal to call different API functions during the tests and in regular use, so we'd probably always use that API and allocate randomness via an rng_t/drbg_t outside even when not testing.

Yes, that's exactly what it would look like and I agree that that's the most consistent approach here. Any specific preference for the "default" DRBG to use for the plugin? Otherwise, I'll see what the best option is that complies with the NIST standard.

We need test vectors not for single operations (which the CAVP vectors seem to provide) but for the complete key exchange. So what's called "seed" in our test vectors has to be the source for all the secrets (for both ends of the exchange, not just z for the initiator).

Ahh, that explains the disconnect between CAVP and the OQS/Crystals vectors. The approach you have now definitely works for your requirements and is easy enough to work with. Like you and your the concerns around HQC, I'm worried how well this generalizes across different algorithms and KAT schemes.

Any specific preference for the "default" DRBG to use for the plugin?

It does not have to be a DRBG, we can just use an rng_t with RNG_STRONG (which is often backed by a DRBG anyway). I was just thinking that using a drbg_t even when not testing might simplify the code. But we could also just check if a drbg_t was set via set_seed() and then either get the 32 bytes from there or create an rng_t and use that (which would be necessary to seed a drbg_t anyway, so not sure if this would really be a simplification).

I'm worried how well this generalizes across different algorithms and KAT schemes

Yeah, we'll have to see. It wouldn't be that much of a problem to add a new field to ke_test_vector_t that specifies the DRBG type and if not set default to DRBG_CTR_AES256 so we wouldn't have to change any of the existing vectors.