Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure: test_running_real_remove_backup_groups_job #3471

Closed
github-actions bot opened this issue Dec 25, 2024 · 2 comments
Closed

Test failure: test_running_real_remove_backup_groups_job #3471

github-actions bot opened this issue Dec 25, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@github-actions
Copy link

❌ test_running_real_remove_backup_groups_job: TimeoutError: Timed out after 0:05:00 (8m33.877s) 
TimeoutError: Timed out after 0:05:00
[gw4] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
05:20 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.K5Cw/config.yml) doesn't exist.
05:20 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:20 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:20 INFO [databricks.labs.ucx.install] Fetching installations...
05:20 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:20 INFO [databricks.labs.ucx.install] Installing UCX v0.53.1+120241225052048
05:20 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:21 INFO [databricks.labs.ucx.install] Creating dashboards...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.K5Cw/README for the next steps.
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] fetching groups inventory
05:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group ucx-Vc7I-ra78a5a88b defined in configuration does not exist on the groups table. Consider checking if the group exist in the workspace or re-running the assessment.
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] crawling new set of snapshot data for groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 2 WorkspaceGroup
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 81 account groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] found 1 new records for groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 2 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] fetching groups inventory
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to rename 1 groups for migration...
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Renaming group: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename is assumed complete: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 62 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 613549174872695 (renamed to rename-K5Cw-ucx-Vc7I-ra78a5a88b)
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 62 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 613549174872695 (renamed to rename-K5Cw-ucx-Vc7I-ra78a5a88b)
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration showed all 1 renamed groups; assuming complete.
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 81 account groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 3 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] fetching groups inventory
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration...
05:21 ERROR [databricks.labs.ucx.workspace_access.groups] Skipping ucx-Vc7I-ra78a5a88b: group already exists in workspace
05:21 DEBUG [databricks.labs.ucx.installer.workflows] starting remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/273902891462920
05:21 INFO [databricks.labs.ucx.installer.workflows] Started remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/273902891462920/runs/1120135287232467
05:21 DEBUG [databricks.labs.ucx.installer.workflows] Waiting for completion of remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/273902891462920/runs/1120135287232467
05:22 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 1120135287232467 with state: RunResultState.SUCCESS
05:22 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 1120135287232467 duration: 0:00:42.167000 (2024-12-25 05:21:48.486000+00:00 thru 2024-12-25 05:22:30.653000+00:00)
05:22 DEBUG [databricks.labs.ucx.installer.workflows] Validating remove-workspace-local-backup-groups workflow: https://DATABRICKS_HOST#job/273902891462920
05:20 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.K5Cw/config.yml) doesn't exist.
05:20 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:20 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:20 INFO [databricks.labs.ucx.install] Fetching installations...
05:20 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:20 INFO [databricks.labs.ucx.install] Installing UCX v0.53.1+120241225052048
05:20 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:21 INFO [databricks.labs.ucx.install] Creating dashboards...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:21 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.K5Cw/README for the next steps.
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] fetching groups inventory
05:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group ucx-Vc7I-ra78a5a88b defined in configuration does not exist on the groups table. Consider checking if the group exist in the workspace or re-running the assessment.
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] crawling new set of snapshot data for groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 2 WorkspaceGroup
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 81 account groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] found 1 new records for groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 2 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] fetching groups inventory
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to rename 1 groups for migration...
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Renaming group: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename is assumed complete: ucx-Vc7I-ra78a5a88b (id=613549174872695) -> rename-K5Cw-ucx-Vc7I-ra78a5a88b
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 62 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 613549174872695 (renamed to rename-K5Cw-ucx-Vc7I-ra78a5a88b)
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 62 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 613549174872695 (renamed to rename-K5Cw-ucx-Vc7I-ra78a5a88b)
05:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration showed all 1 renamed groups; assuming complete.
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 81 account groups
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 3 WorkspaceGroup
05:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sstnc.groups] fetching groups inventory
05:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration...
05:21 ERROR [databricks.labs.ucx.workspace_access.groups] Skipping ucx-Vc7I-ra78a5a88b: group already exists in workspace
05:21 DEBUG [databricks.labs.ucx.installer.workflows] starting remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/273902891462920
05:21 INFO [databricks.labs.ucx.installer.workflows] Started remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/273902891462920/runs/1120135287232467
05:21 DEBUG [databricks.labs.ucx.installer.workflows] Waiting for completion of remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/273902891462920/runs/1120135287232467
05:22 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 1120135287232467 with state: RunResultState.SUCCESS
05:22 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 1120135287232467 duration: 0:00:42.167000 (2024-12-25 05:21:48.486000+00:00 thru 2024-12-25 05:22:30.653000+00:00)
05:22 DEBUG [databricks.labs.ucx.installer.workflows] Validating remove-workspace-local-backup-groups workflow: https://DATABRICKS_HOST#job/273902891462920
05:27 INFO [databricks.labs.ucx.install] Deleting UCX v0.53.1+120241225052048 from https://DATABRICKS_HOST
05:27 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sstnc
05:27 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=201209752004531, as it is no longer needed
05:27 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=277322751066345, as it is no longer needed
05:27 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=155478689379641, as it is no longer needed
05:27 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=273902891462920, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=401115446591880, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=834495705391230, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=88236829520983, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1124346520062289, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=365919224068655, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=93275775956364, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=245113666545104, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=181144854534641, as it is no longer needed
05:28 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=191241366234851, as it is no longer needed
05:28 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:28 INFO [databricks.labs.ucx.install] Deleting secret scope
05:28 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw4] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #327
@github-actions github-actions bot added the bug Something isn't working label Dec 25, 2024
@github-actions GitHub Actions
Copy link
Author

github-actions bot commented Jan 4, 2025

❌ test_running_real_remove_backup_groups_job: TimeoutError: Timed out after 0:05:00 (7m35.207s) 
TimeoutError: Timed out after 0:05:00
[gw9] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
05:46 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.1mh1/config.yml) doesn't exist.
05:46 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:46 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:46 INFO [databricks.labs.ucx.install] Fetching installations...
05:46 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:46 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:46 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:46 INFO [databricks.labs.ucx.install] Installing UCX v0.53.2+120250104054637
05:46 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:46 INFO [databricks.labs.ucx.install] Creating dashboards...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.1mh1/README for the next steps.
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] fetching groups inventory
05:46 WARNING [databricks.labs.ucx.workspace_access.groups] Group ucx-n9Md-ra78b334e7 defined in configuration does not exist on the groups table. Consider checking if the group exist in the workspace or re-running the assessment.
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] crawling new set of snapshot data for groups
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 1 WorkspaceGroup
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 80 account groups
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] found 1 new records for groups
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 1 WorkspaceGroup
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] fetching groups inventory
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Starting to rename 1 groups for migration...
05:46 DEBUG [databricks.labs.ucx.workspace_access.groups] Renaming group: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:46 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:46 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:46 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename is assumed complete: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 60 WorkspaceGroup
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 234335849391227 (renamed to rename-1mh1-ucx-n9Md-ra78b334e7)
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 60 WorkspaceGroup
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 234335849391227 (renamed to rename-1mh1-ucx-n9Md-ra78b334e7)
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration showed all 1 renamed groups; assuming complete.
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 80 account groups
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 1 WorkspaceGroup
05:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] fetching groups inventory
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration...
05:47 ERROR [databricks.labs.ucx.workspace_access.groups] Skipping ucx-n9Md-ra78b334e7: group already exists in workspace
05:47 DEBUG [databricks.labs.ucx.installer.workflows] starting remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/416393346583705
05:47 INFO [databricks.labs.ucx.installer.workflows] Started remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/416393346583705/runs/535324902805233
05:47 DEBUG [databricks.labs.ucx.installer.workflows] Waiting for completion of remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/416393346583705/runs/535324902805233
05:48 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 535324902805233 with state: RunResultState.SUCCESS
05:48 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 535324902805233 duration: 0:01:12.046000 (2025-01-04 05:47:19.224000+00:00 thru 2025-01-04 05:48:31.270000+00:00)
05:48 DEBUG [databricks.labs.ucx.installer.workflows] Validating remove-workspace-local-backup-groups workflow: https://DATABRICKS_HOST#job/416393346583705
05:46 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.1mh1/config.yml) doesn't exist.
05:46 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:46 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:46 INFO [databricks.labs.ucx.install] Fetching installations...
05:46 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:46 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:46 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:46 INFO [databricks.labs.ucx.install] Installing UCX v0.53.2+120250104054637
05:46 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:46 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:46 INFO [databricks.labs.ucx.install] Creating dashboards...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:46 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:46 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:46 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.1mh1/README for the next steps.
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] fetching groups inventory
05:46 WARNING [databricks.labs.ucx.workspace_access.groups] Group ucx-n9Md-ra78b334e7 defined in configuration does not exist on the groups table. Consider checking if the group exist in the workspace or re-running the assessment.
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] crawling new set of snapshot data for groups
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 1 WorkspaceGroup
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 80 account groups
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] found 1 new records for groups
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Found 1 WorkspaceGroup
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] fetching groups inventory
05:46 INFO [databricks.labs.ucx.workspace_access.groups] Starting to rename 1 groups for migration...
05:46 DEBUG [databricks.labs.ucx.workspace_access.groups] Renaming group: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:46 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:46 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:46 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 WARNING [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename is assumed complete: ucx-n9Md-ra78b334e7 (id=234335849391227) -> rename-1mh1-ucx-n9Md-ra78b334e7
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 60 WorkspaceGroup
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 234335849391227 (renamed to rename-1mh1-ucx-n9Md-ra78b334e7)
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 60 WorkspaceGroup
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 234335849391227 (renamed to rename-1mh1-ucx-n9Md-ra78b334e7)
05:47 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration showed all 1 renamed groups; assuming complete.
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 80 account groups
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Found 1 WorkspaceGroup
05:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_szq5k.groups] fetching groups inventory
05:47 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration...
05:47 ERROR [databricks.labs.ucx.workspace_access.groups] Skipping ucx-n9Md-ra78b334e7: group already exists in workspace
05:47 DEBUG [databricks.labs.ucx.installer.workflows] starting remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/416393346583705
05:47 INFO [databricks.labs.ucx.installer.workflows] Started remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/416393346583705/runs/535324902805233
05:47 DEBUG [databricks.labs.ucx.installer.workflows] Waiting for completion of remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/416393346583705/runs/535324902805233
05:48 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 535324902805233 with state: RunResultState.SUCCESS
05:48 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 535324902805233 duration: 0:01:12.046000 (2025-01-04 05:47:19.224000+00:00 thru 2025-01-04 05:48:31.270000+00:00)
05:48 DEBUG [databricks.labs.ucx.installer.workflows] Validating remove-workspace-local-backup-groups workflow: https://DATABRICKS_HOST#job/416393346583705
05:53 INFO [databricks.labs.ucx.install] Deleting UCX v0.53.2+120250104054637 from https://DATABRICKS_HOST
05:53 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_szq5k
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=591348994503004, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=469592962658859, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=416393346583705, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=138377352984211, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=593799006670999, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=489594756884295, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=16465336927669, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=386824194293547, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=657645476142651, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=947564663154173, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=524734114945218, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=395723644196246, as it is no longer needed
05:53 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=330547489320017, as it is no longer needed
05:53 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:53 INFO [databricks.labs.ucx.install] Deleting secret scope
05:53 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw9] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #337

@github-actions GitHub Actions
Copy link
Author

github-actions bot commented Jan 6, 2025

❌ test_running_real_remove_backup_groups_job: TimeoutError: Timed out after 0:05:00 (9m54.882s) 
TimeoutError: Timed out after 0:05:00
[gw5] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
13:21 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.d7rc/config.yml) doesn't exist.
13:21 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
13:21 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
13:21 INFO [databricks.labs.ucx.install] Fetching installations...
13:21 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
13:21 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:21 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:21 INFO [databricks.labs.ucx.install] Installing UCX v0.53.2+4820250106132124
13:21 INFO [databricks.labs.ucx.install] Creating ucx schemas...
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
13:21 INFO [databricks.labs.ucx.install] Creating dashboards...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.d7rc/README for the next steps.
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] fetching groups inventory
13:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group ucx-EgiU-ra78b335b7 defined in configuration does not exist on the groups table. Consider checking if the group exist in the workspace or re-running the assessment.
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] crawling new set of snapshot data for groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 83 account groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] found 1 new records for groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] fetching groups inventory
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to rename 1 groups for migration...
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Renaming group: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename is assumed complete: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 63 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 896347633854120 (renamed to rename-d7rc-ucx-EgiU-ra78b335b7)
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 63 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 896347633854120 (renamed to rename-d7rc-ucx-EgiU-ra78b335b7)
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration showed all 1 renamed groups; assuming complete.
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 83 account groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] fetching groups inventory
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration...
13:21 ERROR [databricks.labs.ucx.workspace_access.groups] Skipping ucx-EgiU-ra78b335b7: group already exists in workspace
13:21 DEBUG [databricks.labs.ucx.installer.workflows] starting remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/814135515278409
13:21 INFO [databricks.labs.ucx.installer.workflows] Started remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/814135515278409/runs/728042835245833
13:21 DEBUG [databricks.labs.ucx.installer.workflows] Waiting for completion of remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/814135515278409/runs/728042835245833
13:24 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 728042835245833 with state: RunResultState.SUCCESS
13:24 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 728042835245833 duration: 0:02:36.854000 (2025-01-06 13:21:53.753000+00:00 thru 2025-01-06 13:24:30.607000+00:00)
13:24 DEBUG [databricks.labs.ucx.installer.workflows] Validating remove-workspace-local-backup-groups workflow: https://DATABRICKS_HOST#job/814135515278409
13:21 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.d7rc/config.yml) doesn't exist.
13:21 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
13:21 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
13:21 INFO [databricks.labs.ucx.install] Fetching installations...
13:21 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
13:21 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:21 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:21 INFO [databricks.labs.ucx.install] Installing UCX v0.53.2+4820250106132124
13:21 INFO [databricks.labs.ucx.install] Creating ucx schemas...
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
13:21 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
13:21 INFO [databricks.labs.ucx.install] Creating dashboards...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
13:21 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
13:21 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:21 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.d7rc/README for the next steps.
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] fetching groups inventory
13:21 WARNING [databricks.labs.ucx.workspace_access.groups] Group ucx-EgiU-ra78b335b7 defined in configuration does not exist on the groups table. Consider checking if the group exist in the workspace or re-running the assessment.
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] crawling new set of snapshot data for groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 83 account groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Group listing provided, a subset of all groups will be migrated
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] found 1 new records for groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] fetching groups inventory
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to rename 1 groups for migration...
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Renaming group: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group still has old name; still waiting for rename to take effect: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename has possibly taken effect: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group rename is assumed complete: ucx-EgiU-ra78b335b7 (id=896347633854120) -> rename-d7rc-ucx-EgiU-ra78b335b7
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 63 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 896347633854120 (renamed to rename-d7rc-ucx-EgiU-ra78b335b7)
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 63 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration reflects renamed group: 896347633854120 (renamed to rename-d7rc-ucx-EgiU-ra78b335b7)
13:21 DEBUG [databricks.labs.ucx.workspace_access.groups] Group enumeration showed all 1 renamed groups; assuming complete.
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing account groups with id,displayName,externalId...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 83 account groups
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=Group) with id,displayName,externalId,meta ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 59 Group
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Listing workspace groups (resource_type=WorkspaceGroup) with id,displayName,meta,externalId,members,roles,entitlements ...
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Found 4 WorkspaceGroup
13:21 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_shqjr.groups] fetching groups inventory
13:21 INFO [databricks.labs.ucx.workspace_access.groups] Starting to reflect 1 account groups into workspace for migration...
13:21 ERROR [databricks.labs.ucx.workspace_access.groups] Skipping ucx-EgiU-ra78b335b7: group already exists in workspace
13:21 DEBUG [databricks.labs.ucx.installer.workflows] starting remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/814135515278409
13:21 INFO [databricks.labs.ucx.installer.workflows] Started remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/814135515278409/runs/728042835245833
13:21 DEBUG [databricks.labs.ucx.installer.workflows] Waiting for completion of remove-workspace-local-backup-groups job: https://DATABRICKS_HOST#job/814135515278409/runs/728042835245833
13:24 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 728042835245833 with state: RunResultState.SUCCESS
13:24 INFO [databricks.labs.ucx.installer.workflows] Completed remove-workspace-local-backup-groups job run 728042835245833 duration: 0:02:36.854000 (2025-01-06 13:21:53.753000+00:00 thru 2025-01-06 13:24:30.607000+00:00)
13:24 DEBUG [databricks.labs.ucx.installer.workflows] Validating remove-workspace-local-backup-groups workflow: https://DATABRICKS_HOST#job/814135515278409
13:29 INFO [databricks.labs.ucx.install] Deleting UCX v0.53.2+4820250106132124 from https://DATABRICKS_HOST
13:29 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_shqjr
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=920778459865920, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=74051058697192, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=572980669004156, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=346474460534051, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1096268726503876, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=814135515278409, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=423493561004496, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=796279940232084, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=987704185526473, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=33075280351049, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=338846077239782, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=322103637089870, as it is no longer needed
13:29 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=148415202430833, as it is no longer needed
13:29 INFO [databricks.labs.ucx.install] Deleting cluster policy
13:29 INFO [databricks.labs.ucx.install] Deleting secret scope
13:29 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw5] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #340

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JCZuurmond