Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHA3 and SHAKE - New API Design #2098

Merged
merged 78 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
c6ed451
Introduce SHA3/SHAKE layered API design; Only SHA3/SHAKE files updates
manastasova Dec 30, 2024
a05d255
Add changes to ML-KEM based on SHA3/SHAKE new API Design
manastasova Dec 30, 2024
50cf7fa
Add changes to ML-DSA based on SHA3/SHAKE new API Design
manastasova Dec 30, 2024
4b0b92e
Update build files in generated-src
manastasova Dec 30, 2024
eb992ea
Update service indicator in SHA3_Final
manastasova Dec 30, 2024
d40fbec
Initialize |ctx->padded| to 0 for SHAKE inside SHAKE_Init
manastasova Dec 31, 2024
adb910d
Update service indicator at the end of SHAKE_Finalize; The XOF functi…
manastasova Dec 31, 2024
02b8085
Fix conflicts with MLDSA parameters renaming
manastasova Dec 31, 2024
e61be0d
Merge branch 'main' into sha3_absorb_squeeze
manastasova Dec 31, 2024
3008821
Merge branch 'aws:main' into sha3_absorb_squeeze
manastasova Jan 3, 2025
2a1622f
Update SHAKE single-shot and streaming APIs
manastasova Jan 3, 2025
c5d0afd
Update incremental block-wise SHAKE squeezes in MLKEM
manastasova Jan 3, 2025
b6a5590
Update incremental block-wise SHAKE squeezes in MLDSA
manastasova Jan 3, 2025
7ccaeba
Replace |keccak_st->padded| flag with |keccak_st->state| flag
manastasova Jan 3, 2025
7edb6c7
Update MLKEM and MLDSA
manastasova Jan 3, 2025
7386c1b
Update Keccak state flag in SHA3 functions
manastasova Jan 3, 2025
e424771
Address code review comments
manastasova Jan 4, 2025
6597af1
Add export macro to functions in the tests
manastasova Jan 4, 2025
7766425
Merge branch 'aws:main' into sha3_absorb_squeeze
manastasova Jan 6, 2025
ff3cbd8
Rename Absorb and Squeeze functions to Keccak1600_ layer specific
manastasova Jan 6, 2025
680dd43
Update build files in generated-src
manastasova Jan 6, 2025
872d368
Apply suggestions from code review
manastasova Jan 7, 2025
5780ee5
Move all common |ctx->state| flag checks in the FIPS202 layer
manastasova Jan 7, 2025
3f43dde
Merge branch 'sha3_absorb_squeeze' of github.com:manastasova/aws-lc i…
manastasova Jan 7, 2025
07bac7c
Update MLKEM and MLDSA
manastasova Jan 7, 2025
2973e4a
Merge branch 'main' of github.com:aws/aws-lc into sha3_absorb_squeeze
manastasova Jan 7, 2025
86fa4b0
Remove SHAKE_Squeeze service indicator update
manastasova Jan 8, 2025
36ab448
Merge branch 'main' of github.com:aws/aws-lc into sha3_absorb_squeeze
manastasova Jan 8, 2025
b2228b6
Bring back exports
manastasova Jan 8, 2025
14da500
Merge branch 'main' of github.com:aws/aws-lc into sha3_only_rename
manastasova Jan 8, 2025
97b02c6
Only add shanges to Init functions
manastasova Jan 8, 2025
95c7e26
add new line at the end of file
manastasova Jan 8, 2025
077ef78
Merge branch 'main' into sha3_absorb_squeeze
manastasova Jan 8, 2025
b4ce7b2
Merge branch 'main' into sha3_only_rename
manastasova Jan 9, 2025
5b18483
Merge branch 'main' into sha3_only_rename
manastasova Jan 13, 2025
d3bba6b
Merge branch 'main' into sha3_only_rename
manastasova Jan 13, 2025
f48fb78
merge with aws-lc main and sha3/shake_only_Init PR changes
manastasova Jan 13, 2025
6ce3a3b
Allow TLS PSK without server certificate (#2083)
WillChilds-Klein Jan 14, 2025
72373f3
Align guard macros for OPENSSL_cpuid_setup (#2111)
justsmth Jan 14, 2025
820394a
Init variable to avoid "may be used uninitialized" warning (#2114)
manastasova Jan 15, 2025
26544da
SCRUTINICE fixes (#2103)
smittals2 Jan 15, 2025
0239af3
Remove jent_read_entropy_safe usage from AWS-LC (main) (#2110)
smittals2 Jan 15, 2025
2ddcd83
CDK: Add scrutinice permissions (#2118)
justsmth Jan 15, 2025
94bc599
Address Scrutinice findings (#2121)
justsmth Jan 15, 2025
6933d45
Finalize ML-DSA asn.1 module (#2117)
jakemas Jan 16, 2025
9462df8
Align BN_bn2hex behavior with OpenSSL (#2122)
samuel40791765 Jan 16, 2025
9d433cb
Start making asserts constant-time too
davidben Jan 16, 2024
705f36c
Fix EVP_PKEY_CTX_dup with EC generation
davidben Mar 17, 2024
a03e160
Remove unused flags argument from trust handlers
botovq Mar 21, 2024
507ff49
Document that null STACK_OF(T) can be used with several functions
davidben Mar 22, 2024
863387d
Upstream rebase
manastasova Jan 30, 2025
5a4dc9c
Add a PrivacyInfo plist file
Mar 26, 2024
ebfb590
Switch EVP_CIPHERs to C99 initializers
davidben Mar 29, 2024
b80f99e
Document that our Unicode APIs reject noncharacters
davidben Mar 29, 2024
3001ac9
Rewrite RAND_enable_fork_unsafe_buffering documentation
davidben Mar 29, 2024
8d6b7f5
Disable `-Wcast-function-type-strict` for `BORINGSSL_DEFINE_STACK_OF_…
brianpl Apr 2, 2024
64f25eb
Increase DTLS window size from 64 to 256
nharper Mar 28, 2024
80260d9
Avoid strdup in crypto/err/err.c
davidben Nov 20, 2023
2d90347
Add more debug logging to channelID test failures (#2130)
andrewhop Jan 21, 2025
55742fa
Compress crypto_test_data.cc (#2123)
justsmth Jan 21, 2025
c91bd6a
Prepare AWS-LC v1.43.0 (#2133)
justsmth Jan 22, 2025
a66a1a5
Minor symbols to work with Ruby's mainline (#2132)
samuel40791765 Jan 22, 2025
7fb93e8
ACVP test harness for ML-DSA (#2127)
jakemas Jan 24, 2025
95a9e8a
Remove remaining support for Trusty and Fuchsia operating systems (#2…
torben-hansen Jan 24, 2025
0b9ef21
Avoid mixing SSE and AVX in XTS-mode AVX512 implementation (#2140)
torben-hansen Jan 27, 2025
df51857
Support for ML-DSA public key generation from private key (#2142)
jakemas Jan 28, 2025
9d3b72b
Ed25519ph and Ed25519ctx Support (#2120)
skmcgrail Jan 28, 2025
3b18668
Check for MIPSEB in target.h (#2143)
justsmth Jan 28, 2025
3cdcaa3
Optimize x86/aarch64 MD5 implementation (#2137)
olivergillespie Jan 28, 2025
cb6f877
Support keypair calculation for PQDSA PKEY (#2145)
jakemas Jan 29, 2025
96f8169
Upstream merge
manastasova Jan 30, 2025
7848023
fix
manastasova Jan 30, 2025
1fdb30e
Update comments and exports
manastasova Jan 30, 2025
9c559df
Merge branch 'main' into sha3_absorb_squeeze
manastasova Jan 31, 2025
0973fc2
Merge branch 'main' into sha3_absorb_squeeze
manastasova Feb 4, 2025
2616442
Address PR comments
manastasova Feb 5, 2025
85a0d6f
Address PR comments
manastasova Feb 5, 2025
873f017
Merge branch 'sha3_absorb_squeeze' of github.com:manastasova/aws-lc i…
manastasova Feb 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions crypto/fipsmodule/digest/digests.c
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,7 @@ static void shake128_init(EVP_MD_CTX *ctx) {
}

static void shake128_update(EVP_MD_CTX *ctx, const void *data, size_t count) {
CHECK(SHA3_Update(ctx->md_data, data, count));
CHECK(SHAKE_Absorb(ctx->md_data, data, count));
}

static void shake128_final(EVP_MD_CTX *ctx, uint8_t *md, size_t len) {
Expand All @@ -455,7 +455,7 @@ static void shake256_init(EVP_MD_CTX *ctx) {
}

static void shake256_update(EVP_MD_CTX *ctx, const void *data, size_t count) {
CHECK(SHA3_Update(ctx->md_data, data, count));
CHECK(SHAKE_Absorb(ctx->md_data, data, count));
}

static void shake256_finalXOF(EVP_MD_CTX *ctx, uint8_t *md, size_t len) {
Expand Down
20 changes: 11 additions & 9 deletions crypto/fipsmodule/ml_kem/ml_kem_ref/symmetric-shake.c
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ void kyber_shake128_absorb(KECCAK1600_CTX *ctx,
// SHAKE_Init always returns 1 when called with correct block size value
SHAKE_Init(ctx, SHAKE128_BLOCKSIZE);

// SHA3_Update always returns 1 on first call of sizeof(extseed) (34 bytes)
SHA3_Update(ctx, extseed, sizeof(extseed));
// SHAKE_Absorb always returns 1 on first call of sizeof(extseed) (34 bytes)
SHAKE_Absorb(ctx, extseed, sizeof(extseed));
}

/*************************************************
Expand All @@ -48,8 +48,9 @@ void kyber_shake128_absorb(KECCAK1600_CTX *ctx,
void kyber_shake128_squeeze(KECCAK1600_CTX *ctx, uint8_t *out, int nblocks)
{
// Return code checks can be omitted
// SHAKE_Final always returns 1
SHAKE_Final(out, ctx, nblocks * SHAKE128_BLOCKSIZE);
// SHAKE_Squeeze always returns 1 when |ctx->state| flag is different
// from |KECCAK1600_STATE_FINAL|
SHAKE_Squeeze(out, ctx, nblocks * SHAKE128_BLOCKSIZE);
}

/*************************************************
Expand Down Expand Up @@ -94,12 +95,13 @@ void kyber_shake256_rkprf(ml_kem_params *params, uint8_t out[KYBER_SSBYTES], con
// SHAKE_Init always returns 1 when called with correct block size value
SHAKE_Init(&ctx, SHAKE256_BLOCKSIZE);

// SHA3_Update always returns 1 on first call of KYBER_SYMBYTES (32 bytes)
SHA3_Update(&ctx, key, KYBER_SYMBYTES);
// SHAKE_Absorb always returns 1 on first call of KYBER_SYMBYTES (32 bytes)
SHAKE_Absorb(&ctx, key, KYBER_SYMBYTES);

// SHA3_Update always returns 1 processing all data blocks that don't need pad
SHA3_Update(&ctx, input, params->ciphertext_bytes);
// SHAKE_Absorb always returns 1 processing all data blocks that don't need pad
SHAKE_Absorb(&ctx, input, params->ciphertext_bytes);

// SHAKE_Final always returns 1
// SHAKE_Final always returns 1 when |ctx->state| flag is set to
// |KECCAK1600_STATE_ABSORB| (no previous calls to SHAKE_Final)
SHAKE_Final(out, &ctx, KYBER_SSBYTES);
}
61 changes: 41 additions & 20 deletions crypto/fipsmodule/sha/internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,16 @@ extern "C" {
// SHAKE128 has the maximum block size among the SHA3/SHAKE algorithms.
#define SHA3_MAX_BLOCKSIZE SHAKE128_BLOCKSIZE

// Define state flag values for Keccak-based functions
#define KECCAK1600_STATE_ABSORB 0
// KECCAK1600_STATE_SQUEEZE is set when |SHAKE_Squeeze| is called.
// It remains set while |SHAKE_Squeeze| is called repeatedly to output
// chunks of the XOF output.
#define KECCAK1600_STATE_SQUEEZE 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to put a comment here like
'''
// KECCAK1600_STATE_SQUEEZE is set when |SHAKE_Squeeze| is called.
// It remains set while |SHAKE_Squeeze| is called repeatedly to output chunks of the XOF output.
'''

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Nevine. Added!

// KECCAK1600_STATE_FINAL is set once |SHAKE_Final| is called
// so that |SHAKE_Squeeze| cannot be called anymore.
#define KECCAK1600_STATE_FINAL 2

typedef struct keccak_st KECCAK1600_CTX;

// The data buffer should have at least the maximum number of
Expand All @@ -82,7 +92,7 @@ struct keccak_st {
size_t buf_load; // used bytes in below buffer
uint8_t buf[SHA3_MAX_BLOCKSIZE]; // should have at least the max data block size bytes
uint8_t pad; // padding character
uint8_t padded; // denotes if padding has been performed
uint8_t state; // denotes the keccak phase (absorb, squeeze, final)
};

// Define SHA{n}[_{variant}]_ASM if sha{n}_block_data_order[_{variant}] is
Expand Down Expand Up @@ -396,32 +406,43 @@ OPENSSL_EXPORT uint8_t *SHAKE128(const uint8_t *data, const size_t in_len,
OPENSSL_EXPORT uint8_t *SHAKE256(const uint8_t *data, const size_t in_len,
uint8_t *out, size_t out_len);

// SHAKE_Init initializes |ctx| with specified |block_size|, returns 1 on
// success and 0 on failure. Calls SHA3_Init under the hood.
int SHAKE_Init(KECCAK1600_CTX *ctx, size_t block_size);

// SHAKE_Final writes |len| bytes of finalized digest to |md|, returns 1 on
// success and 0 on failure. Calls SHA3_Final under the hood.
int SHAKE_Final(uint8_t *md, KECCAK1600_CTX *ctx, size_t len);

// SHA3_Reset zeros the bitstate and the amount of processed input.
void SHA3_Reset(KECCAK1600_CTX *ctx);

// SHA3_Init initialises |ctx| fields and returns 1 on success and 0 on failure.
// SHA3_Init initialises |ctx| fields through |FIPS202_Init| and
// returns 1 on success and 0 on failure.
OPENSSL_EXPORT int SHA3_Init(KECCAK1600_CTX *ctx, size_t bitlen);

// SHA3_Update processes all data blocks that don't need pad through
// |Keccak1600_Absorb| and returns 1 and 0 on failure.
// SHA3_Update check |ctx| pointer and |len| value, calls |FIPS202_Update|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it not do what it used to do: "processes all full blocks of the input data"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHA3_Update technically does the same: "processes all data blocks that don't need pad through". However, now it is performed via call to FIPS202_Update, which internally "processes all data blocks that don't need pad through".

// and returns 1 on success and 0 on failure.
int SHA3_Update(KECCAK1600_CTX *ctx, const void *data, size_t len);

// SHA3_Final pads the last data block and processes it through |Keccak1600_Absorb|.
// It processes the data through |Keccak1600_Squeeze| and returns 1 and 0 on failure.
// SHA3_Final pads the last data block and absorbs it through |FIPS202_Finalize|.
// It then calls |Keccak1600_Squeeze| and returns 1 on success
// and 0 on failure.
int SHA3_Final(uint8_t *md, KECCAK1600_CTX *ctx);

// Keccak1600_Absorb processes the largest multiple of |r| out of |len| bytes and
// returns the remaining number of bytes.
// SHAKE_Init initialises |ctx| fields through |FIPS202_Init| and
// returns 1 on success and 0 on failure.
int SHAKE_Init(KECCAK1600_CTX *ctx, size_t block_size);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a historical reason why we chose to input bitlength to SHA3_Init and block_size to SHAKE_Init?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good observation!
I see it as more intuitive to consider the digest length as an identifier for the security level of the hashing algs rather than the block size (as it is for SHA2)
The block size for SHAKE servers as the SL identifier based on the lack of digest lengths.
I can change the SHA3_Init to take the block size as a parameter. However, the reverse, changing SHAKE_Init won't be possible, so I think it may be more confusing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to leave the bitlength in SHA3_Init because there isn't an equivalent function for SHA2 that we need to align with (e.g. using byte length). If we used bitlength with SHAKE_Init, it would identify 128 or 256. It's minor anyway and if it makes more sense to you this way, that's good.


// SHAKE_Absorb checks |ctx| pointer and |len| values. It updates and absorbs
// input blocks via |FIPS202_Update|.
int SHAKE_Absorb(KECCAK1600_CTX *ctx, const void *data,
size_t len);

// SHAKE_Squeeze pads the last data block and absorbs it through
// |FIPS202_Finalize| on first call. It writes |len| bytes of incremental
// XOF output to |md| and returns 1 on success and 0 on failure. It can be
// called multiple times.
int SHAKE_Squeeze(uint8_t *md, KECCAK1600_CTX *ctx, size_t len);

// SHAKE_Final writes |len| bytes of finalized extendible output to |md|, returns 1 on
// success and 0 on failure. It should be called once to finalize absorb and
// squeeze phases. Incremental XOF output should be generated via |SHAKE_Squeeze|.
int SHAKE_Final(uint8_t *md, KECCAK1600_CTX *ctx, size_t len);

// Keccak1600_Absorb processes the largest multiple of |r| (block size) out of
// |len| bytes and returns the remaining number of bytes.
size_t Keccak1600_Absorb(uint64_t A[KECCAK1600_ROWS][KECCAK1600_ROWS],
const uint8_t *data, size_t len, size_t r);
const uint8_t *data, size_t len, size_t r);

// Keccak1600_Squeeze generates |out| value of |len| bytes (per call). It can be called
// multiple times when used as eXtendable Output Function. |padded| indicates
Expand Down
Loading
Loading