Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation of encryption key length and method #20

Merged
merged 1 commit into from
Oct 30, 2023

Conversation

acelyc111
Copy link
Member

@acelyc111 acelyc111 commented Oct 28, 2023

Before this patch, there is no validation of the encryption key length
and method, so if using an encryption method but with a not matched length
encryption key, it would cause encrypt and decrypt not match bug.

This patch fixes this bug and adds some related modifacations, including:

  • Add validation of encryption key length and method
  • Use RegisterOptions to initialize the AESEncryptionProvider objects
    instead of AddFactory
  • Still keep AddFactory to create AESEncryptionProvider objects
  • The AddFactory will only create AESEncryptionProvider objects successfully
    when the uri match AES
  • Remove the unit tests added in previous patched, we can use the enviroment
    variables instead, say TEST_FS_URI and ENCRYPTED_ENV
  • Remove the new add gflags in db_bench tool, we can use --fs_uri instead
  • Add more unit tests in encryption/encryption_test.cc
  • Add or update related CI jobs build-linux-encrypted_env-openssl*

The functionality can be checked by the following script as well.

#!/usr/bin/env bash

set -ex

# 1. build
mkdir build && cd build
cmake -DWITH_LZ4=1 -DCMAKE_BUILD_TYPE=Debug -DWITH_TESTS=1 -DROCKSDB_BUILD_SHARED=0 -DWITH_OPENSSL=1 ..
make -j32

uris=("provider={id=AES;hex_instance_key=0123456789ABCDEF0123456789ABCDEF;method=AES128CTR};id=EncryptedFileSystem"
      "provider={id=AES;hex_instance_key=0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF;method=AES192CTR};id=EncryptedFileSystem"
      "provider={id=AES;hex_instance_key=0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF;method=AES256CTR};id=EncryptedFileSystem"
      "provider={id=AES;hex_instance_key=0123456789ABCDEF0123456789ABCDEF;method=SM4CTR};id=EncryptedFileSystem")
for uri in ${uris[*]}; do
  echo "${uri}"

  # Set env (select 1 of the 4)
  export TEST_FS_URI=${uri}

  # Run unit tests
  ./env_basic_test --gtest_filter=*CustomEnv*
  ./env_test --gtest_filter=CreateEnvTest.CreateEncryptedFileSystem

  # Run benchmarks
  ./db_bench --fs_uri="${uri}" --benchmarks="fillseq,readrandom,readseq" --compression_type=lz4 --num=1000000

  # Run ldb tools
  ./tools/ldb --fs_uri="${uri}" --db=/tmp/rocksdbtest-1000/dbbench/ put k v
  ./tools/ldb --fs_uri="${uri}" --db=/tmp/rocksdbtest-1000/dbbench/ get k
  ls -l /tmp/rocksdbtest-1000/dbbench | grep "log" | awk '{print $NF}' | xargs -i ./tools/ldb --fs_uri="${uri}" dump_wal --walfile=/tmp/rocksdbtest-1000/dbbench/{} | head
  ./tools/ldb --fs_uri="${uri}" --db=/tmp/rocksdbtest-1000/dbbench/ scan --hex | head
  ./tools/ldb --fs_uri="${uri}" --db=/tmp/rocksdbtest-1000/dbbench/ dump --hex | head
  ./tools/ldb --fs_uri="${uri}" --db=/tmp/rocksdbtest-1000/dbbench/ manifest_dump | head
  ./tools/ldb --fs_uri="${uri}" --db=/tmp/rocksdbtest-1000/dbbench/ list_live_files_metadata | head
done

# Unset the env
unset TEST_FS_URI

@empiredan empiredan merged commit a3ad4de into pegasus-kv:v8.5.3-pegasus-encrypt Oct 30, 2023
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants