[C++][Parquet] Enable external material and rotation for encryption keys

[asfimport]

Encryption key material keeps the information required to recover the keys by Parquet readers. Currently, only "internal storage mode" is supported, where the key material is stored inside Parquet file footers, and therefore can't be rotated (re-wrapped with refreshed master keys) in immutable files. The new mechanism in this Jira adds the advanced "external storage mode" for the key material, where the key material is stored in separate small files, that enable the key rotation. This mechanism will also implement the key rotation functionality.

Reporter: Gidon Gershinsky / @ggershinsky

Related issues:

• [C++][Dataset][Parquet] Support parquet modular encryption in the new Dataset API (Dependency)
• [C++][Parquet] Encryption key management tools (depends upon)
• [Python][Parquet] Enable usage of external key material and rotation for encryption keys in PyArrow (is depended upon by)

PRs and other links:

• GitHub Pull Request #10491
• GitHub Pull Request #8286
• Overall design of key management tools

_{Note: This issue was originally created as ARROW-9960. Please see the migration documentation for further details.}

[asfimport]

Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

[wjones127]

@adamreeve is working on this.

[wjones127]

@adamreeve could you comment "take" on this issue? Once you do, this issue should be assigned to you and I can merge the PR. (Also I'm looking into improving this process.)

[adamreeve]

take