Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix file already exists error in install.py #219

Closed
larsgeorge-db opened this issue Sep 19, 2023 · 0 comments · Fixed by #222
Closed

Fix file already exists error in install.py #219

larsgeorge-db opened this issue Sep 19, 2023 · 0 comments · Fixed by #222
Assignees
@larsgeorge-db
Labels
bug Something isn't working

Comments

@larsgeorge-db
Copy link
Contributor

This is caused by the code trying to upload the notebook again.

See this stacktrace:

(databricks-labs-ucx) ➜  ucx git:(main) ./install.sh
[i] found Python 3.9.6 -> /usr/bin/python3
[i] found Python 3.11.4 -> /opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
[i] found Python 3.11.4 -> /opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
[i] latest python is /opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
[+] making sure we have the latest pip version
[+] installing dependencies within ephemeral Virtualenv: /var/folders/qf/rb54syb53992k0qhj_y04_yr0000gp/T/tmp.yaPI2K55
Type 'yes' to open config file in the browser: no
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 395, in <module>
    installer.run()
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 72, in run
    self._create_jobs()
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 159, in _create_jobs
    settings = self._job_settings(step_name, remote_wheel)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 257, in _job_settings
    "tasks": [self._job_task(task, dbfs_path) for task in tasks],
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 257, in <listcomp>
    "tasks": [self._job_task(task, dbfs_path) for task in tasks],
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 267, in _job_task
    return self._job_notebook_task(jobs_task, task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 274, in _job_notebook_task
    self._ws.workspace.upload(remote_notebook, f)
  File "/private/var/folders/qf/rb54syb53992k0qhj_y04_yr0000gp/T/tmp.yaPI2K55/lib/python3.11/site-packages/databricks/sdk/mixins/workspace.py", line 90, in upload
    raise e
  File "/private/var/folders/qf/rb54syb53992k0qhj_y04_yr0000gp/T/tmp.yaPI2K55/lib/python3.11/site-packages/databricks/sdk/mixins/workspace.py", line 84, in upload
    return self._api.do('POST', '/api/2.0/workspace/import', files={'content': content}, data=data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/var/folders/qf/rb54syb53992k0qhj_y04_yr0000gp/T/tmp.yaPI2K55/lib/python3.11/site-packages/databricks/sdk/core.py", line 1011, in do
    raise self._make_nicer_error(status_code=response.status_code, **payload) from None
databricks.sdk.core.DatabricksError: Path (/Users/lars.george@databricks.com/.ucx/tables.scala) already exists.
@larsgeorge-db larsgeorge-db added the bug Something isn't working label Sep 19, 2023
@larsgeorge-db larsgeorge-db added this to the 1 week milestone Sep 19, 2023
@larsgeorge-db larsgeorge-db self-assigned this Sep 19, 2023
larsgeorge-db added a commit that referenced this issue Sep 19, 2023
@larsgeorge-db
Fix file already exists error in install.py (#219)
fcd8fb8
larsgeorge-db added a commit that referenced this issue Sep 19, 2023
@larsgeorge-db
Fix file already exists error in install.py (#219)
86ed0d1
@nfx nfx linked a pull request Sep 19, 2023 that will close this issue
Fix file already exists error in install.py (#219) #222
Merged
github-merge-queue bot pushed a commit that referenced this issue Sep 19, 2023
@larsgeorge-db
Fix file already exists error in install.py (#219) (#222)
d34726e
@nfx nfx closed this as completed in #222 Sep 19, 2023
FastLee pushed a commit that referenced this issue Sep 20, 2023
@larsgeorge-db @FastLee
Fix file already exists error in install.py (#219) (#222)
488b8dc
@nfx nfx mentioned this issue Sep 21, 2023
Merged
nfx added a commit that referenced this issue Sep 21, 2023
@nfx
Release v0.1.1 (#261)
c3173eb 
* Added batched iteration for `INSERT INTO` queries in
`StatementExecutionBackend` with default `max_records_per_batch=1000`
([#237](#237)).
* Added crawler for mount points
([#209](#209)).
* Added crawlers for compatibility of jobs and clusters, along with
basic recommendations for external locations
([#244](#244)).
* Added safe return on grants
([#246](#246)).
* Added ability to specify empty group filter in the installer script
([#216](#216))
([#217](#217)).
* Added ability to install application by multiple different users on
the same workspace ([#235](#235)).
* Added dashboard creation on installation and a requirement for
`warehouse_id` in config, so that the assessment dashboards are
refreshed automatically after job runs
([#214](#214)).
* Added reliance on rate limiting from Databricks SDK for listing
workspace ([#258](#258)).
* Fixed errors in corner cases where Azure Service Principal Credentials
were not available in Spark context
([#254](#254)).
* Fixed `DESCRIBE TABLE` throwing errors when listing Legacy Table ACLs
([#238](#238)).
* Fixed `file already exists` error in the installer script
([#219](#219))
([#222](#222)).
* Fixed `guess_external_locations` failure with `AttributeError:
as_dict` and added an integration test
([#259](#259)).
* Fixed error handling edge cases in `crawl_tables` task
([#243](#243))
([#251](#251)).
* Fixed `crawl_permissions` task failure on folder names containing a
forward slash ([#234](#234)).
* Improved `README` notebook documentation
([#260](#260),
[#228](#228),
[#252](#252),
[#223](#223),
[#225](#225)).
* Removed redundant `.python-version` file
([#221](#221)).
* Removed discovery of account groups from `crawl_permissions` task
([#240](#240)).
* Updated databricks-sdk requirement from ~=0.8.0 to ~=0.9.0
([#245](#245)).
larsgeorge-db pushed a commit that referenced this issue Sep 23, 2023
@nfx @larsgeorge-db
Release v0.1.1 (#261)
340fab4 
* Added batched iteration for `INSERT INTO` queries in
`StatementExecutionBackend` with default `max_records_per_batch=1000`
([#237](#237)).
* Added crawler for mount points
([#209](#209)).
* Added crawlers for compatibility of jobs and clusters, along with
basic recommendations for external locations
([#244](#244)).
* Added safe return on grants
([#246](#246)).
* Added ability to specify empty group filter in the installer script
([#216](#216))
([#217](#217)).
* Added ability to install application by multiple different users on
the same workspace ([#235](#235)).
* Added dashboard creation on installation and a requirement for
`warehouse_id` in config, so that the assessment dashboards are
refreshed automatically after job runs
([#214](#214)).
* Added reliance on rate limiting from Databricks SDK for listing
workspace ([#258](#258)).
* Fixed errors in corner cases where Azure Service Principal Credentials
were not available in Spark context
([#254](#254)).
* Fixed `DESCRIBE TABLE` throwing errors when listing Legacy Table ACLs
([#238](#238)).
* Fixed `file already exists` error in the installer script
([#219](#219))
([#222](#222)).
* Fixed `guess_external_locations` failure with `AttributeError:
as_dict` and added an integration test
([#259](#259)).
* Fixed error handling edge cases in `crawl_tables` task
([#243](#243))
([#251](#251)).
* Fixed `crawl_permissions` task failure on folder names containing a
forward slash ([#234](#234)).
* Improved `README` notebook documentation
([#260](#260),
[#228](#228),
[#252](#252),
[#223](#223),
[#225](#225)).
* Removed redundant `.python-version` file
([#221](#221)).
* Removed discovery of account groups from `crawl_permissions` task
([#240](#240)).
* Updated databricks-sdk requirement from ~=0.8.0 to ~=0.9.0
([#245](#245)).
@nfx nfx removed this from the 1 week milestone Sep 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@larsgeorge-db larsgeorge-db

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix file already exists error in install.py (#219) databrickslabs/ucx
2 participants
@nfx @larsgeorge-db