Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,9 @@ jobs:
env:
ARROW_HOME: /tmp/local
ARROW_AZURE: ON
ARROW_CMAKE_ARGS: -DPARQUET_BUILD_DBPS_LIBS=OFF
ARROW_CMAKE_ARGS: >-
-DARROW_BUILD_SHARED=ON
-DPARQUET_BUILD_DBPS_LIBS=OFF
ARROW_DATASET: ON
ARROW_FLIGHT: ON
ARROW_GANDIVA: ON
Expand Down Expand Up @@ -242,10 +244,8 @@ jobs:
ci/scripts/python_build.sh $(pwd) $(pwd)/build
- name: Test
shell: bash
env:
# Boost problems in this env prevent the DBPS library from being built.
PYTEST_ARGS: -k not(test_external_encryption)
run: ci/scripts/python_test.sh $(pwd) $(pwd)/build
run:
ci/scripts/python_test.sh "$(pwd)" "$(pwd)/build"

windows:
name: AMD64 Windows 2022 Python 3.13
Expand Down
1 change: 1 addition & 0 deletions ci/scripts/python_build.bat
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ cmake ^
-DCMAKE_UNITY_BUILD=%CMAKE_UNITY_BUILD% ^
-DMSVC_LINK_VERBOSE=ON ^
-DPARQUET_REQUIRE_ENCRYPTION=%PARQUET_REQUIRE_ENCRYPTION% ^
-DPARQUET_BUILD_DBPS_LIBS=OFF ^
-Dxsimd_SOURCE=BUNDLED ^
-G "%CMAKE_GENERATOR%" ^
%CPP_SOURCE_DIR% || exit /B 1
Expand Down
12 changes: 12 additions & 0 deletions cpp/src/parquet/encryption/external_dbpa_encryption.h
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,18 @@ void UpdateEncryptorMetadata(
/// operation is open, and is used to guarantee the lifetime of the encryptor.
class PARQUET_EXPORT ExternalDBPAEncryptorAdapterFactory {
public:
// Windows DLL construction tries to instantiate a copy constructor for this class,
// and runs into problem as it tries to copy a deleted pointer. Explicitly delete
// copy operations so MSVC doesn't try to instantiate a copy constructor.ßßß
ExternalDBPAEncryptorAdapterFactory() = default;
ExternalDBPAEncryptorAdapterFactory(const ExternalDBPAEncryptorAdapterFactory&) =
delete;
ExternalDBPAEncryptorAdapterFactory& operator=(
const ExternalDBPAEncryptorAdapterFactory&) = delete;
ExternalDBPAEncryptorAdapterFactory(ExternalDBPAEncryptorAdapterFactory&&) = default;
ExternalDBPAEncryptorAdapterFactory& operator=(ExternalDBPAEncryptorAdapterFactory&&) =
default;

Comment on lines +143 to +151
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? (again, just curiosity)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and maybe a comment is granted in the code?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Windows DLL construction tries to instantiate a copy constructor and runs into a deleted pointer.

ExternalDBPAEncryptorAdapter* GetEncryptor(
ParquetCipher::type algorithm,
const ColumnChunkMetaDataBuilder* column_chunk_metadata,
Expand Down
25 changes: 16 additions & 9 deletions python/pyarrow/tests/parquet/test_external_encryption.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,22 @@
import re


def get_agent_library_path():
# TODO: move this code to a common library
# See https://github.com/protegrity/arrow/issues/191
return os.environ.get(
'DBPA_LIBRARY_PATH',
'libDBPATestAgent.so'
if platform.system() == 'Linux' else 'libDBPATestAgent.dylib')


@pytest.fixture(scope="module", autouse=True)
def _skip_if_external_dbpa_agent_missing():
path = get_agent_library_path()
if not os.path.exists(path):
pytest.skip(f"External DBPA agent library not found in path [{path}]")


class FooKmsClient(ppe.KmsClient):

def __init__(self, kms_connection_config):
Expand Down Expand Up @@ -63,15 +79,6 @@ def kms_client_factory(kms_connection_config):
return FooKmsClient(kms_connection_config)


def get_agent_library_path():
# TODO: move this code to a common library
# See https://github.com/protegrity/arrow/issues/191
return os.environ.get(
'DBPA_LIBRARY_PATH',
'libDBPATestAgent.so'
if platform.system() == 'Linux' else 'libDBPATestAgent.dylib')


def get_kms_connection_config():
return ppe.KmsConnectionConfig(
custom_kms_conf={
Expand Down
Loading