Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All four SHA-2 variants from HACL* #2

Draft
wants to merge 12 commits into
base: cf-zeta
Choose a base branch
from
Draft

Conversation

karthikbhargavan
Copy link
Collaborator

No description provided.

crypto/sha2-hacl.c Outdated Show resolved Hide resolved
EXPORT_SYMBOL(hacl_sha512_finup);


static struct shash_alg sha2_hacl_algs[4] = { {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my suggestion is to break this file into two:

  1. module boilerplate code (written by hand), which would internally call into below
  2. autogenerated implementation code

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ignatk does it make sense to put the implementation code inside /lib/crypto/, and the module file here /crypto/.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for "pure C cross platform" implementations - yes

include/crypto/hacl_hash.h Outdated Show resolved Hide resolved
crypto/Makefile Outdated Show resolved Hide resolved
Copy link
Collaborator

@armfazh armfazh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crypto/hacl_hash.h Outdated Show resolved Hide resolved
crypto/hacl_hash.h Outdated Show resolved Hide resolved
crypto/hacl_hash.h Outdated Show resolved Hide resolved
crypto/hacl_hash.h Outdated Show resolved Hide resolved
crypto/hacl_hash.h Outdated Show resolved Hide resolved
crypto/sha2-hacl.c Outdated Show resolved Hide resolved
EXPORT_SYMBOL(hacl_sha512_finup);


static struct shash_alg sha2_hacl_algs[4] = { {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ignatk does it make sense to put the implementation code inside /lib/crypto/, and the module file here /crypto/.

crypto/tcrypt.c Outdated
@@ -775,14 +775,14 @@ static int test_ahash_cycles_digest(struct ahash_request *req, int blen,
int ret, i;

/* Warm-up run. */
for (i = 0; i < 4; i++) {
for (i = 0; i < 16; i++) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modifications to this file should be in another PR. Also not sure they are needed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undid these changes in this PR, but we note that without them the performance measurements are very unstable.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if passing sec=2 helps, this was added in the CI execution.

crypto/sha2-hacl-generated.c Outdated Show resolved Hide resolved
crypto/sha2-hacl-generated.c Outdated Show resolved Hide resolved
crypto/Makefile Outdated Show resolved Hide resolved
@armfazh armfazh changed the base branch from cf-linux-rolling-stable to cf-zeta October 16, 2023 23:58
@armfazh
Copy link
Collaborator

armfazh commented Dec 5, 2023

I've added three commits, one is about the license identifiers, multiple file compilation, and manually loading the new module.
This was necessary since manually loading the module had some issues, now it's fixed.

@fredlawl
Copy link
Collaborator

@franziskuskiefer Is it possible to tweak the tool that generates code? I'm beginning the process of at least putting together the SHA2 algos, and we have some initial formatting results. If you're able to, it would be nice to comb through these and update the styles for crypto/sha2-hacl.c and friends as well. Ignoring any ./cf-zeta & Signed-off-by's for now.

initial-commit-issues.txt

This was generated with the following:

$ git remote add cryptodev git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
$ git checkout -b fred/sha2
$ git reset --hard cryptodev/master && git cherry-pick 1dce3a13f7678c80a6a3ce99b8564918b59adb14^..376f49921e8e2c89196016b68ec813a4fa3e1cf1 &> initial-commit-issues.txt

I have this in my settings to automatically pickup commit style issues:

$ cat ~/.git-templates/hooks/post-commit
#!/bin/sh

[ -f ./scripts/checkpatch.pl ] && echo "executing checkpatch"  && exec git show --format=email HEAD | ./scripts/checkpatch.pl --strict --max-line-length=80 --codespell

I can translate this to a github worker/action to catch these style problems sooner.

@fredlawl
Copy link
Collaborator

I confirmed that make ARCH=um will set -Werror to off, but for x86_64 and arm64 it's at least on.

Are we intending for other modules to use this module? We need a header file in include/linux/crypto if we're going to export these functions. Otherwise, lets make these static and drop the EXPORT_SYMBOL().

$ grep -ri 'hacl_sha512_finup' .
./crypto/sha2-hacl.c:int hacl_sha512_finup(struct shash_desc *desc, const u8 *data, unsigned int len,
./crypto/sha2-hacl.c:EXPORT_SYMBOL(hacl_sha512_finup);
./crypto/sha2-hacl.c:                .finup = hacl_sha512_finup,
./crypto/sha2-hacl.c:                .finup = hacl_sha512_finup,
$ make -j$(nproc)
mkdir -p /home/fred/Work/upstream-linux/zeta/tools/objtool && make O=/home/fred/Work/upstream-linux/zeta subdir=tools/objtool --no-print-directory -C objtool 
  INSTALL libsubcmd_headers
  CALL    scripts/checksyscalls.sh
  CC      crypto/sha2-hacl-generated.o
  CC      crypto/sha2-hacl.o
  CC      crypto/aes_generic.o
  CC      crypto/crc32c_generic.o
  CC      crypto/authenc.o
crypto/sha2-hacl-generated.c:12:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha256_init’ [-Werror=missing-prototypes]
   12 | void Hacl_SHA2_Scalar32_sha256_init(uint32_t *hash)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC      crypto/authencesn.o
crypto/sha2-hacl-generated.c:114:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha256_update_nblocks’ [-Werror=missing-prototypes]
  114 | void Hacl_SHA2_Scalar32_sha256_update_nblocks(uint32_t len, uint8_t *b,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:125:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha256_update_last’ [-Werror=missing-prototypes]
  125 | void Hacl_SHA2_Scalar32_sha256_update_last(uint64_t totlen, uint32_t len,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:159:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha256_finish’ [-Werror=missing-prototypes]
  159 | void Hacl_SHA2_Scalar32_sha256_finish(uint32_t *st, uint8_t *h)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:167:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha224_init’ [-Werror=missing-prototypes]
  167 | void Hacl_SHA2_Scalar32_sha224_init(uint32_t *hash)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:180:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha224_update_last’ [-Werror=missing-prototypes]
  180 | void Hacl_SHA2_Scalar32_sha224_update_last(uint64_t totlen, uint32_t len,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:186:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha224_finish’ [-Werror=missing-prototypes]
  186 | void Hacl_SHA2_Scalar32_sha224_finish(uint32_t *st, uint8_t *h)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:502:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha512_init’ [-Werror=missing-prototypes]
  502 | void Hacl_SHA2_Scalar32_sha512_init(uint64_t *hash)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:604:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha512_update_nblocks’ [-Werror=missing-prototypes]
  604 | void Hacl_SHA2_Scalar32_sha512_update_nblocks(uint32_t len, uint8_t *b,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:615:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha512_update_last’ [-Werror=missing-prototypes]
  615 | void Hacl_SHA2_Scalar32_sha512_update_last(FStar_UInt128_uint128 totlen,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:651:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha512_finish’ [-Werror=missing-prototypes]
  651 | void Hacl_SHA2_Scalar32_sha512_finish(uint64_t *st, uint8_t *h)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:659:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha384_init’ [-Werror=missing-prototypes]
  659 | void Hacl_SHA2_Scalar32_sha384_init(uint64_t *hash)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:667:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha384_update_nblocks’ [-Werror=missing-prototypes]
  667 | void Hacl_SHA2_Scalar32_sha384_update_nblocks(uint32_t len, uint8_t *b,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:673:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha384_update_last’ [-Werror=missing-prototypes]
  673 | void Hacl_SHA2_Scalar32_sha384_update_last(FStar_UInt128_uint128 totlen,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
crypto/sha2-hacl-generated.c:680:6: error: no previous prototype for ‘Hacl_SHA2_Scalar32_sha384_finish’ [-Werror=missing-prototypes]
  680 | void Hacl_SHA2_Scalar32_sha384_finish(uint64_t *st, uint8_t *h)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC      crypto/rng.o
crypto/sha2-hacl.c:14:5: error: no previous prototype for ‘hacl_sha256_update’ [-Werror=missing-prototypes]
   14 | int hacl_sha256_update(struct shash_desc *desc, const u8 *data,
      |     ^~~~~~~~~~~~~~~~~~
crypto/sha2-hacl.c:42:5: error: no previous prototype for ‘hacl_sha256_finup’ [-Werror=missing-prototypes]
   42 | int hacl_sha256_finup(struct shash_desc *desc, const u8 *data, unsigned int len,
      |     ^~~~~~~~~~~~~~~~~
crypto/sha2-hacl.c:59:5: error: no previous prototype for ‘hacl_sha512_update’ [-Werror=missing-prototypes]
   59 | int hacl_sha512_update(struct shash_desc *desc, const u8 *data,
      |     ^~~~~~~~~~~~~~~~~~
crypto/sha2-hacl.c:87:5: error: no previous prototype for ‘hacl_sha512_finup’ [-Werror=missing-prototypes]
   87 | int hacl_sha512_finup(struct shash_desc *desc, const u8 *data, unsigned int len,
      |     ^~~~~~~~~~~~~~~~~
  CC      crypto/drbg.o
cc1: all warnings being treated as errors
  CC      crypto/jitterentropy.o
make[3]: *** [scripts/Makefile.build:243: crypto/sha2-hacl.o] Error 1
make[3]: *** Waiting for unfinished jobs....
cc1: all warnings being treated as errors
make[3]: *** [scripts/Makefile.build:243: crypto/sha2-hacl-generated.o] Error 1
make[2]: *** [scripts/Makefile.build:481: crypto] Error 2
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [/home/fred/Work/upstream-linux/zeta/Makefile:1917: .] Error 2
make: *** [Makefile:240: __sub-make] Error 2

@franziskuskiefer
Copy link
Collaborator

Is it possible to tweak the tool that generates code?

It's certainly possible to tweak things. But what exactly are the issues? Can you list some specific issues? Then we can look into it.

Skimming the file I see formatting issues. Formatting has been done as instructed. But that's not really a code issue and can be easily fixed by running the tools to get what you want.

@fredlawl
Copy link
Collaborator

Is it possible to tweak the tool that generates code?

It's certainly possible to tweak things. But what exactly are the issues? Can you list some specific issues? Then we can look into it.

This will be more obvious in the code samples below, but things like:

WARNING: please, no spaces at the start of a line
#59: FILE: crypto/hacl_hash.h:47:
+        uint32_t *block_state;$

and

CHECK: Prefer kernel type 'u8' over 'uint8_t'
#334: FILE: crypto/sha2-hacl-generated.c:20:
+static inline void hacl_sha256_update(uint8_t *b, uint32_t *hash)

Granted, there was some cleanup work in previous commits with clang-format such as: b0e0339 but mentioned below, it wont fix all checkpatch problems.

Skimming the file I see formatting issues. Formatting has been done as instructed. But that's not really a code issue and can be easily fixed by running the tools to get what you want.

Yes and no. I couldn't get astyle to work due to a bug (https://sourceforge.net/p/astyle/bugs/541/; and I didn't want to manually compile it for linux atm), but assuming it's supposed prepare clang-format (which is based off linux's style) we can see that some styling is fixed but not all:

➜  zeta git:(fred/sha2) ✗ git show diff --staged
diff --git a/crypto/sha2-hacl.c b/crypto/sha2-hacl.c
index d5339c93949e..50c52eb84742 100644
--- a/crypto/sha2-hacl.c
+++ b/crypto/sha2-hacl.c
@@ -9,11 +9,12 @@
 #include <crypto/sha512_base.h>
 
 #include "hacl_hash.h"
-#include "hacl_lib.h"
+                                               #include "hacl_lib.h"
 
 int hacl_sha256_update(struct shash_desc *desc, const u8 *data,
                        unsigned int len)
 {
+       uint32_t test;
         struct sha256_state *sctx = shash_desc_ctx(desc);
         struct Hacl_Streaming_MD_state_32_s st;
         st.block_state = sctx->state;
         
➜  zeta git:(fred/sha2) ✗ git commit -m 'bad format'
executing checkpatch
No codespell typos will be found - file '/usr/share/codespell/dictionary.txt': No such file or directory
WARNING: Missing commit description - Add an appropriate one

CHECK: Prefer kernel type 'u32' over 'uint32_t'
#21: FILE: crypto/sha2-hacl.c:17:
+	uint32_t test;

ERROR: Missing Signed-off-by: line(s)

total: 1 errors, 1 warnings, 1 checks, 13 lines checked

NOTE: For some of the reported defects, checkpatch may be able to
      mechanically convert to the typical style using --fix or --fix-inplace.

"[PATCH] bad format" has style problems, please review.

NOTE: If any of the errors are false positives, please report
      them to the maintainer, see CHECKPATCH in MAINTAINERS.
[fred/sha2 47aba1dbb9fb] bad format
 1 file changed, 2 insertions(+), 1 deletion(-)

➜  zeta git:(fred/sha2) git status
On branch fred/sha2
nothing to commit, working tree clean

➜  zeta git:(fred/sha2) ✗ clang-format -i -style=file:zeta/.clang-format ./crypto/sha2-hacl.c
diff --git a/crypto/sha2-hacl.c b/crypto/sha2-hacl.c
index 50c52eb84742..0cccab1753bb 100644
--- a/crypto/sha2-hacl.c
+++ b/crypto/sha2-hacl.c
@@ -9,12 +9,12 @@
 #include <crypto/sha512_base.h>
 
 #include "hacl_hash.h"
-                                               #include "hacl_lib.h"
+#include "hacl_lib.h"
 
 int hacl_sha256_update(struct shash_desc *desc, const u8 *data,
                        unsigned int len)
 {
-       uint32_t test;
+        uint32_t test;
         struct sha256_state *sctx = shash_desc_ctx(desc);
         struct Hacl_Streaming_MD_state_32_s st;
         st.block_state = sctx->state;

However, when it comes to /** etc... comments:

WARNING: Block comments use * on subsequent lines
#227: FILE: crypto/hacl_hash.h:215:
+/**
+Reset an existing state to the initial hash state with empty data.

That's not necessarily the case:

➜  zeta git:(fred/sha2) ✗ git commit --amend
executing checkpatch
No codespell typos will be found - file '/usr/share/codespell/dictionary.txt': No such file or directory
WARNING: Missing commit description - Add an appropriate one

WARNING: Block comments use * on subsequent lines
#16: FILE: crypto/hacl_hash.h:19:
+/**
+bad comment

ERROR: Missing Signed-off-by: line(s)

total: 1 errors, 2 warnings, 0 checks, 10 lines checked

NOTE: For some of the reported defects, checkpatch may be able to
      mechanically convert to the typical style using --fix or --fix-inplace.

"[PATCH] bad comment" has style problems, please review.

NOTE: If any of the errors are false positives, please report
      them to the maintainer, see CHECKPATCH in MAINTAINERS.
[fred/sha2 3275872255c3] bad comment
 Date: Mon Mar 18 12:29:59 2024 -0500
 1 file changed, 4 insertions(+)
➜  zeta git:(fred/sha2) git status
On branch fred/sha2
nothing to commit, working tree clean        
➜  zeta git:(fred/sha2) clang-format -i -style=file:zeta/.clang-format ./crypto/hacl_hash.h
➜  zeta git:(fred/sha2) git diff
➜  zeta git:(fred/sha2) git status
On branch fred/sha2
nothing to commit, working tree clean

Therefore, the comments were not handled by clang-format. So it would be nice if the code generator can fix somethings up.

I'll squash everything and then run through a more realistic checkpatch to see what is specifically left. My previous comment pointed out everything per commit due to the rebase, and that will have out-dated information. With that said, it does provide a very clear indication of what upstream is expecting with these changes.

@fredlawl
Copy link
Collaborator

@franziskuskiefer Sorry for the long wait (busy day). I went ahead and squashed the commits, and reran checkpatch on everything and the results are in:
checkpatch-output.txt

I then ran a clang-format -i -style=file:zeta/.clang-format ./crypto/sha2-hacl-generated.c ./crypto/hacl_hash.h ./crypto/sha2-hacl.c ./crypto/hacl_lib.h, and the only changes from the formater are:

diff --git a/crypto/hacl_hash.h b/crypto/hacl_hash.h
index 80d8dd4ef224..74746bc84b60 100644
--- a/crypto/hacl_hash.h
+++ b/crypto/hacl_hash.h
@@ -223,4 +223,4 @@ void Hacl_Streaming_SHA2_finish_384(struct Hacl_Streaming_MD_state_64_s *p,
 void Hacl_Streaming_SHA2_hash_384(uint8_t *input, uint32_t input_len,
                                   uint8_t *dst);
 
-#endif  // CRYPTO_HACL_HASH_H_
+#endif // CRYPTO_HACL_HASH_H_
diff --git a/crypto/hacl_lib.h b/crypto/hacl_lib.h
index 12a9a3c2fcac..de645982039f 100644
--- a/crypto/hacl_lib.h
+++ b/crypto/hacl_lib.h
@@ -231,4 +231,4 @@ static inline void store128_be(u8 *buf, u128 x)
 #define KRML_MAYBE_FOR16(i, z, n, k, x) KRML_ACTUAL_FOR(i, z, n, k, x)
 #endif
 
-#endif  // CRYPTO_HACL_LIB_H_
+#endif // CRYPTO_HACL_LIB_H_

Let me know if this helps clarify what's missing formatting/patch wise. If we can get most of these handled by the code-generator, we'd be in a good spot with the other PR's as well. When I have some more cycles, I'll think about how to incorporate this into the build pipeline.

I did try executing some of the automatic cleanup tools the kernel provides: ./scripts/cleanpatch, ./scripts/cleanfile, and --fix-inline with checkpatch. These didn't quite work out the way I was hoping. It covered some of the whitespace issues, but that's about it. It's probably better to address these issues in the code generator for repeatable builds.

Lastly, regarding #2 (comment) I'm about to merge in #9 which will cover the lingering compile cases. (And I'll be rebasing all the PRs too)

I'm going to make the assumption that we do want to export these functions to other modules, so I'll try to include a header file to hopefully fix the compile.

+cc: @karthikbhargavan @armfazh @ignatk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants