[FEATURE]: Create uber-IAM profile for performing external table migration on AWS #879
Assignees
Labels
Comments
nfx
added
enhancement
nfx
changed the title
This was referenced Feb 5, 2024
1 task
5 tasks
nfx
pushed a commit
that referenced
this issue
Mar 11, 2024
…pal` command (#993) ## Changes Added CLI command `databricks labs ucx create-uber-principal` for creating uber-IAM profile for performing external table migration on AWS. Logic: * Stop if UCX migration cluster policy is not found * Collect paths of all locations/paths used in tables (call `external_location.snapshot`) * If cluster policy has an existing iam instance profile/role specified, then add/update migration policy providing access to the locations * If cluster policy does not have iam instance profile/role specified, then create new iam profile/role and migration policy, and add it to the cluster policy ### Linked issues Resolves #879 Related issues: - #976 - #693 ### Functionality - [x] added new CLI command ### Tests - [x] manually tested - [x] added unit tests ### TODO - [x] added integration tests - [x] verified on staging environment (screenshot attached) --------- Co-authored-by: Vuong <vuong.nguyen@databricks.com>
dmoore247
pushed a commit
that referenced
this issue
Mar 23, 2024
…pal` command (#993) ## Changes Added CLI command `databricks labs ucx create-uber-principal` for creating uber-IAM profile for performing external table migration on AWS. Logic: * Stop if UCX migration cluster policy is not found * Collect paths of all locations/paths used in tables (call `external_location.snapshot`) * If cluster policy has an existing iam instance profile/role specified, then add/update migration policy providing access to the locations * If cluster policy does not have iam instance profile/role specified, then create new iam profile/role and migration policy, and add it to the cluster policy ### Linked issues Resolves #879 Related issues: - #976 - #693 ### Functionality - [x] added new CLI command ### Tests - [x] manually tested - [x] added unit tests ### TODO - [x] added integration tests - [x] verified on staging environment (screenshot attached) --------- Co-authored-by: Vuong <vuong.nguyen@databricks.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is there an existing issue for this?
Problem statement
Related issues:
databricks labs ucx create-uber-principalcommand to automate the creation of SPN with storage access to all locations #693Proposed Solution
Create AWS IAM role/profile that has read access for all of the S3 buckets for external table migration and add it to UCX cluster policy:
Additional Context
No response
The text was updated successfully, but these errors were encountered: