Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added collection of Databricks Runtime versions used together with Python SDK #287

Merged
merged 2 commits into from
Aug 17, 2023
Merged

Added collection of Databricks Runtime versions used together with Python SDK #287

merged 2 commits into from
Aug 17, 2023

Conversation

nfx
Copy link
Contributor

@nfx nfx commented Aug 15, 2023

This PR adds collection of DBR version if used from within Databricks Cluster. This will allow to improve on notebook-native auth.

@nfx
Added collection of Databricks Runtime versions used together with Py…
09bc2be 
…thon SDK

This PR adds collection of DBR version if used from within Databricks Cluster. This will allow to improve on notebook-native auth.
@nfx nfx requested a review from a team August 15, 2023 09:41
@codecov-commenter
Copy link

codecov-commenter commented Aug 15, 2023 •
edited
Loading

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.01% 🎉

Comparison is base (8ea491d) 52.61% compared to head (e28183a) 52.62%.
Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #287      +/-   ##
==========================================
+ Coverage   52.61%   52.62%   +0.01%     
==========================================
  Files          33       33              
  Lines       19874    19890      +16     
==========================================
+ Hits        10456    10467      +11     
- Misses       9418     9423       +5
Files Changed Coverage Δ
databricks/sdk/core.py 67.57% <100.00%> (+0.42%) ⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pietern
pietern reviewed Aug 16, 2023
View reviewed changes
databricks/sdk/core.py
if 'DATABRICKS_RUNTIME_VERSION' in os.environ:
runtime_version = os.environ['DATABRICKS_RUNTIME_VERSION']
if runtime_version != '':
ua.append(f'runtime/{runtime_version}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a check that this value conforms to an expected pattern, e.g. no spaces/slashes.

Can be a simple regex match.

I suspect you expect to see 13.3.x-scala2.12?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pietern it's 13.3

@nfx nfx requested a review from pietern August 17, 2023 08:43
@pietern pietern requested a review from mgyucht August 17, 2023 12:46
mgyucht
mgyucht approved these changes Aug 17, 2023
View reviewed changes
Copy link
Contributor

@mgyucht mgyucht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

@mgyucht
Copy link
Contributor

mgyucht commented Aug 17, 2023

As no good deed goes unpunished, can you make this change in Go/Java as well?

@mgyucht
Copy link
Contributor

mgyucht commented Aug 17, 2023

Actually, I guess it is not needed in Go, but Java would.

@nfx nfx added this pull request to the merge queue Aug 17, 2023
Merged via the queue into main with commit b0b3543 Aug 17, 2023
@nfx nfx deleted the feature/runtime-version-in-user-agent branch August 17, 2023 13:31
mgyucht added a commit that referenced this pull request Aug 17, 2023
@mgyucht
Release v0.6.0
2f4f9b5 
* Added collection of Databricks Runtime versions used together with Python SDK ([#287](#287)).
* Applied attribute transformer when reading in attributes from the environment ([#293](#293)).
* Made notebook-native auth work with more configurations of the Databricks Runtime ([#285](#285)).
* Added retry in `w.clusters.ensure_cluster_is_running(id)` when cluster is simultaneously started by two different processes. ([#283](#283)).
* Set necessary headers when authenticating via Azure CLI ([#290](#290)).
* Updated classifier to `Development Status :: 4 - Beta` ([#291](#291)).
* Introduced Artifact Allowlist, Securable Tags, and Subentity Tags services.
* Introduced DeleteRuns and RestoreRuns methods in the Experiments API.
* Introduced the GetSecret method in the Secrets API.
* Renamed Auto Maintenance to Predictive Optimization.

New Services:

 * Added [w.artifact_allowlists](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/artifact_allowlists.html) workspace-level service.
 * Added [w.securable_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/securable_tags.html) workspace-level service.
 * Added [w.subentity_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/subentity_tags.html) workspace-level service.
 * Added `databricks.sdk.service.catalog.ArtifactAllowlistInfo` dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactMatcher` dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactType` dataclass.
 * Added `databricks.sdk.service.catalog.GetArtifactAllowlistRequest` dataclass.
 * Added `databricks.sdk.service.catalog.ListSecurableTagsRequest` dataclass.
 * Added `databricks.sdk.service.catalog.ListSecurableType` dataclass.
 * Added `databricks.sdk.service.catalog.ListSubentityTagsRequest` dataclass.
 * Added `databricks.sdk.service.catalog.MatchType` dataclass.
 * Added `databricks.sdk.service.catalog.SetArtifactAllowlist` dataclass.
 * Added `databricks.sdk.service.catalog.TagChanges` dataclass.
 * Added `databricks.sdk.service.catalog.TagKeyValuePair` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurable` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurableAssignment` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurableAssignmentsList` dataclass.
 * Added `databricks.sdk.service.catalog.TagSubentity` dataclass.
 * Added `databricks.sdk.service.catalog.TagSubentityAssignmentsList` dataclass.
 * Added `databricks.sdk.service.catalog.TagsSubentityAssignment` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateSecurableType` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateTags` dataclass.

New APIs:

 * Added `delete_runs()` method for [w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html) workspace-level service.
 * Added `restore_runs()` method for [w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html) workspace-level service.
 * Added `databricks.sdk.service.ml.DeleteRuns` dataclass.
 * Added `databricks.sdk.service.ml.DeleteRunsResponse` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRuns` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRunsResponse` dataclass.
 * Added `get_secret()` method for [w.secrets](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/secrets.html) workspace-level service.
 * Added `databricks.sdk.service.workspace.GetSecretRequest` dataclass.
 * Added `databricks.sdk.service.workspace.GetSecretResponse` dataclass.

Service Renames:

 * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Added `effective_predictive_optimization_flag` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Added `enable_predictive_optimization` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Removed `databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlag` dataclass.
 * Removed `databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlagInheritedFromType` dataclass.
 * Removed `databricks.sdk.service.catalog.EnableAutoMaintenance` dataclass.
 * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Added `effective_predictive_optimization_flag` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Added `enable_predictive_optimization` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.TableInfo`.
 * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.TableInfo`.
 * Added `effective_predictive_optimization_flag` field for `databricks.sdk.service.catalog.TableInfo`.
 * Added `enable_predictive_optimization` field for `databricks.sdk.service.catalog.TableInfo`.
 * Added `databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlag` dataclass.
 * Added `databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlagInheritedFromType` dataclass.
 * Added `databricks.sdk.service.catalog.EnablePredictiveOptimization` dataclass.

OpenAPI SHA: beff621d7b3e1d59244e2e34fc53a496f310e130, Date: 2023-08-17
@mgyucht mgyucht mentioned this pull request Aug 17, 2023
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 17, 2023
@mgyucht
Release v0.6.0 (#296)
cef100c 
* Added collection of Databricks Runtime versions used together with
Python SDK
([#287](#287)).
* Applied attribute transformer when reading in attributes from the
environment
([#293](#293)).
* Made notebook-native auth work with more configurations of the
Databricks Runtime
([#285](#285)).
* Added retry in `w.clusters.ensure_cluster_is_running(id)` when cluster
is simultaneously started by two different processes.
([#283](#283)).
* Set necessary headers when authenticating via Azure CLI
([#290](#290)).
* Updated classifier to `Development Status :: 4 - Beta`
([#291](#291)).
* Introduced Artifact Allowlist, Securable Tags, and Subentity Tags
services.
* Introduced DeleteRuns and RestoreRuns methods in the Experiments API.
* Introduced the GetSecret method in the Secrets API.
* Renamed Auto Maintenance to Predictive Optimization.

New Services:

* Added
[w.artifact_allowlists](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/artifact_allowlists.html)
workspace-level service.
* Added
[w.securable_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/securable_tags.html)
workspace-level service.
* Added
[w.subentity_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/subentity_tags.html)
workspace-level service.
* Added `databricks.sdk.service.catalog.ArtifactAllowlistInfo`
dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactMatcher` dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactType` dataclass.
* Added `databricks.sdk.service.catalog.GetArtifactAllowlistRequest`
dataclass.
* Added `databricks.sdk.service.catalog.ListSecurableTagsRequest`
dataclass.
 * Added `databricks.sdk.service.catalog.ListSecurableType` dataclass.
* Added `databricks.sdk.service.catalog.ListSubentityTagsRequest`
dataclass.
 * Added `databricks.sdk.service.catalog.MatchType` dataclass.
* Added `databricks.sdk.service.catalog.SetArtifactAllowlist` dataclass.
 * Added `databricks.sdk.service.catalog.TagChanges` dataclass.
 * Added `databricks.sdk.service.catalog.TagKeyValuePair` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurable` dataclass.
* Added `databricks.sdk.service.catalog.TagSecurableAssignment`
dataclass.
* Added `databricks.sdk.service.catalog.TagSecurableAssignmentsList`
dataclass.
 * Added `databricks.sdk.service.catalog.TagSubentity` dataclass.
* Added `databricks.sdk.service.catalog.TagSubentityAssignmentsList`
dataclass.
* Added `databricks.sdk.service.catalog.TagsSubentityAssignment`
dataclass.
 * Added `databricks.sdk.service.catalog.UpdateSecurableType` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateTags` dataclass.

New APIs:

* Added `delete_runs()` method for
[w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html)
workspace-level service.
* Added `restore_runs()` method for
[w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html)
workspace-level service.
 * Added `databricks.sdk.service.ml.DeleteRuns` dataclass.
 * Added `databricks.sdk.service.ml.DeleteRunsResponse` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRuns` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRunsResponse` dataclass.
* Added `get_secret()` method for
[w.secrets](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/secrets.html)
workspace-level service.
 * Added `databricks.sdk.service.workspace.GetSecretRequest` dataclass.
 * Added `databricks.sdk.service.workspace.GetSecretResponse` dataclass.

Service Renames:

* Removed `effective_auto_maintenance_flag` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Removed `enable_auto_maintenance` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Added `effective_predictive_optimization_flag` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Added `enable_predictive_optimization` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Removed `databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlag`
dataclass.
* Removed
`databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlagInheritedFromType`
dataclass.
* Removed `databricks.sdk.service.catalog.EnableAutoMaintenance`
dataclass.
* Removed `effective_auto_maintenance_flag` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Removed `enable_auto_maintenance` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Added `effective_predictive_optimization_flag` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Added `enable_predictive_optimization` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Removed `effective_auto_maintenance_flag` field for
`databricks.sdk.service.catalog.TableInfo`.
* Removed `enable_auto_maintenance` field for
`databricks.sdk.service.catalog.TableInfo`.
* Added `effective_predictive_optimization_flag` field for
`databricks.sdk.service.catalog.TableInfo`.
* Added `enable_predictive_optimization` field for
`databricks.sdk.service.catalog.TableInfo`.
* Added
`databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlag`
dataclass.
* Added
`databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlagInheritedFromType`
dataclass.
* Added `databricks.sdk.service.catalog.EnablePredictiveOptimization`
dataclass.

OpenAPI SHA: beff621d7b3e1d59244e2e34fc53a496f310e130, Date: 2023-08-17
@shreyas-goenka shreyas-goenka mentioned this pull request Mar 11, 2024
Merged
github-merge-queue bot pushed a commit to databricks/databricks-sdk-go that referenced this pull request Mar 15, 2024
@shreyas-goenka
Add telemetry for SDK usage from DBR (#851)
750e4b6 
## Changes
This PR adds telemetry through the user agent for DBR usage from the
SDK. We use the same mechanism and key as the Python SDK to add this to
the user agent (PR:
databricks/databricks-sdk-py#287).

## Tests
Using unit tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mgyucht mgyucht mgyucht approved these changes

@pietern pietern pietern approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nfx @codecov-commenter @mgyucht @pietern