Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set necessary headers when authenticating via Azure CLI #136

Merged
merged 3 commits into from
Aug 18, 2023

Conversation

mgyucht
Copy link
Contributor

@mgyucht mgyucht commented Aug 17, 2023

Changes

The Java SDK request authentication logic is inconsistent between the Azure login types: for service principal auth, the SDK correctly adds the X-Databricks-Azure-Workspace-Resource-Id when configured, but this is missed for Azure CLI auth. Additionally, when logging in via Azure CLI using a service principal, the service management token must also be fetched from the CLI.

This PR fixes this by defining the logic to attach these header in a common function that is used by all Azure-specific authentication types.

See databricks/databricks-sdk-go#584 for the same change in the Go SDK.
See databricks/databricks-sdk-py#290 for the same changes in the Python SDK.

Tests

  • Unit tests to cover the two scenarios for Azure CLI w.r.t. management endpoint token fetching, and one to verify that X-Databricks-Azure-Workspace-Resource-Id is included when using Azure CLI.

@mgyucht mgyucht requested a review from pietern August 17, 2023 11:22
@mgyucht mgyucht changed the title Set necessary headers when authenticating via Azure CLI. Set necessary headers when authenticating via Azure CLI Aug 17, 2023
@mgyucht mgyucht marked this pull request as ready for review August 17, 2023 11:22
LOG.debug("Not including service management token in headers", e);
mgmtTokenSource = null;
}
CliTokenSource finalMgmtTokenSource = mgmtTokenSource;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is final about it? Seems like the alias is not necessary below.

Copy link
Contributor Author

@mgyucht mgyucht Aug 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Java compiler doesn't allow capturing non-final variables in closures. https://www.baeldung.com/java-lambda-effectively-final-local-variables#capturing-lambdas

@mgyucht mgyucht added this pull request to the merge queue Aug 18, 2023
Merged via the queue into main with commit 7bb4fd0 Aug 18, 2023
9 checks passed
@mgyucht mgyucht deleted the add-x-databricks-azure-workspace-resource-id-header branch August 18, 2023 07:22
mgyucht added a commit that referenced this pull request Aug 18, 2023
* Added support for `dbutils.secrets` ([#132](#132)).
* Added retry in `w.clusters().ensureClusterIsRunning(id)` when cluster is simultaneously started by two different processes ([#134](#134)).
* Set necessary headers when authenticating via Azure CLI ([#136](#136)).

New Services:

 * Added `workspaceClient.artifactAllowlists()` service.
 * Added `workspaceClient.securableTags()` service.
 * Added `workspaceClient.subentityTags()` service.
 * Added `com.databricks.sdk.service.catalog.ArtifactAllowlistInfo` class.
 * Added `com.databricks.sdk.service.catalog.ArtifactMatcher` class.
 * Added `com.databricks.sdk.service.catalog.ArtifactType` class.
 * Added `com.databricks.sdk.service.catalog.GetArtifactAllowlistRequest` class.
 * Added `com.databricks.sdk.service.catalog.ListSecurableTagsRequest` class.
 * Added `com.databricks.sdk.service.catalog.ListSecurableType` class.
 * Added `com.databricks.sdk.service.catalog.ListSubentityTagsRequest` class.
 * Added `com.databricks.sdk.service.catalog.MatchType` class.
 * Added `com.databricks.sdk.service.catalog.SetArtifactAllowlist` class.
 * Added `com.databricks.sdk.service.catalog.TagChanges` class.
 * Added `com.databricks.sdk.service.catalog.TagKeyValuePair` class.
 * Added `com.databricks.sdk.service.catalog.TagSecurable` class.
 * Added `com.databricks.sdk.service.catalog.TagSecurableAssignment` class.
 * Added `com.databricks.sdk.service.catalog.TagSecurableAssignmentsList` class.
 * Added `com.databricks.sdk.service.catalog.TagSubentity` class.
 * Added `com.databricks.sdk.service.catalog.TagSubentityAssignmentsList` class.
 * Added `com.databricks.sdk.service.catalog.TagsSubentityAssignment` class.
 * Added `com.databricks.sdk.service.catalog.UpdateSecurableType` class.
 * Added `com.databricks.sdk.service.catalog.UpdateTags` class.

New APIs:

 * Added `deleteRuns()` method for `workspaceClient.experiments()` service.
 * Added `restoreRuns()` method for `workspaceClient.experiments()` service.
 * Added `com.databricks.sdk.service.ml.DeleteRuns` class.
 * Added `com.databricks.sdk.service.ml.DeleteRunsResponse` class.
 * Added `com.databricks.sdk.service.ml.RestoreRuns` class.
 * Added `com.databricks.sdk.service.ml.RestoreRunsResponse` class.
 * Added `getSecret()` method for `workspaceClient.secrets()` service.
 * Added `com.databricks.sdk.service.workspace.GetSecretRequest` class.
 * Added `com.databricks.sdk.service.workspace.GetSecretResponse` class.

API Renames:

 * Renamed `effectiveAutoMaintenanceFlag` field to `effectivePredictiveOptimizationFlag` field for `com.databricks.sdk.service.catalog.CatalogInfo`.
 * Renamed `enableAutoMaintenance` field to `enablePredictiveOptimization` field for `com.databricks.sdk.service.catalog.CatalogInfo`.
 * Renamed `com.databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlag` class to `com.databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlag` class.
 * Renamed `com.databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlagInheritedFromType` class to `com.databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlagInheritedFromType`.
 * Renamed `com.databricks.sdk.service.catalog.EnableAutoMaintenance` class to `com.databricks.sdk.service.catalog.EnablePredictiveOptimization`.
 * Renamed `effectiveAutoMaintenanceFlag` field to `effectivePredictiveOptimizationFlag` field for `com.databricks.sdk.service.catalog.SchemaInfo`.
 * Renamed `enableAutoMaintenance` field to `enablePredictiveOptimization` field for `com.databricks.sdk.service.catalog.SchemaInfo`.
 * Renamed `effectiveAutoMaintenanceFlag` field to `effectivePredictiveOptimizationFlag` field for `com.databricks.sdk.service.catalog.TableInfo`.
 * Renamed `enableAutoMaintenance` field to `enablePredictiveOptimization` field for `com.databricks.sdk.service.catalog.TableInfo`.

OpenAPI SHA: beff621d7b3e1d59244e2e34fc53a496f310e130, Date: 2023-08-17
@mgyucht mgyucht mentioned this pull request Aug 18, 2023
github-merge-queue bot pushed a commit that referenced this pull request Aug 18, 2023
* Added support for `dbutils.secrets`
([#132](#132)).
* Added retry in `w.clusters().ensureClusterIsRunning(id)` when cluster
is simultaneously started by two different processes
([#134](#134)).
* Set necessary headers when authenticating via Azure CLI
([#136](#136)).

New Services:

 * Added `workspaceClient.artifactAllowlists()` service.
 * Added `workspaceClient.securableTags()` service.
 * Added `workspaceClient.subentityTags()` service.
* Added `com.databricks.sdk.service.catalog.ArtifactAllowlistInfo`
class.
 * Added `com.databricks.sdk.service.catalog.ArtifactMatcher` class.
 * Added `com.databricks.sdk.service.catalog.ArtifactType` class.
* Added `com.databricks.sdk.service.catalog.GetArtifactAllowlistRequest`
class.
* Added `com.databricks.sdk.service.catalog.ListSecurableTagsRequest`
class.
 * Added `com.databricks.sdk.service.catalog.ListSecurableType` class.
* Added `com.databricks.sdk.service.catalog.ListSubentityTagsRequest`
class.
 * Added `com.databricks.sdk.service.catalog.MatchType` class.
* Added `com.databricks.sdk.service.catalog.SetArtifactAllowlist` class.
 * Added `com.databricks.sdk.service.catalog.TagChanges` class.
 * Added `com.databricks.sdk.service.catalog.TagKeyValuePair` class.
 * Added `com.databricks.sdk.service.catalog.TagSecurable` class.
* Added `com.databricks.sdk.service.catalog.TagSecurableAssignment`
class.
* Added `com.databricks.sdk.service.catalog.TagSecurableAssignmentsList`
class.
 * Added `com.databricks.sdk.service.catalog.TagSubentity` class.
* Added `com.databricks.sdk.service.catalog.TagSubentityAssignmentsList`
class.
* Added `com.databricks.sdk.service.catalog.TagsSubentityAssignment`
class.
 * Added `com.databricks.sdk.service.catalog.UpdateSecurableType` class.
 * Added `com.databricks.sdk.service.catalog.UpdateTags` class.

New APIs:

* Added `deleteRuns()` method for `workspaceClient.experiments()`
service.
* Added `restoreRuns()` method for `workspaceClient.experiments()`
service.
 * Added `com.databricks.sdk.service.ml.DeleteRuns` class.
 * Added `com.databricks.sdk.service.ml.DeleteRunsResponse` class.
 * Added `com.databricks.sdk.service.ml.RestoreRuns` class.
 * Added `com.databricks.sdk.service.ml.RestoreRunsResponse` class.
 * Added `getSecret()` method for `workspaceClient.secrets()` service.
 * Added `com.databricks.sdk.service.workspace.GetSecretRequest` class.
 * Added `com.databricks.sdk.service.workspace.GetSecretResponse` class.

API Renames:

* Renamed `effectiveAutoMaintenanceFlag` field to
`effectivePredictiveOptimizationFlag` field for
`com.databricks.sdk.service.catalog.CatalogInfo`.
* Renamed `enableAutoMaintenance` field to
`enablePredictiveOptimization` field for
`com.databricks.sdk.service.catalog.CatalogInfo`.
* Renamed
`com.databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlag` class
to
`com.databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlag`
class.
* Renamed
`com.databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlagInheritedFromType`
class to
`com.databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlagInheritedFromType`.
* Renamed `com.databricks.sdk.service.catalog.EnableAutoMaintenance`
class to
`com.databricks.sdk.service.catalog.EnablePredictiveOptimization`.
* Renamed `effectiveAutoMaintenanceFlag` field to
`effectivePredictiveOptimizationFlag` field for
`com.databricks.sdk.service.catalog.SchemaInfo`.
* Renamed `enableAutoMaintenance` field to
`enablePredictiveOptimization` field for
`com.databricks.sdk.service.catalog.SchemaInfo`.
* Renamed `effectiveAutoMaintenanceFlag` field to
`effectivePredictiveOptimizationFlag` field for
`com.databricks.sdk.service.catalog.TableInfo`.
* Renamed `enableAutoMaintenance` field to
`enablePredictiveOptimization` field for
`com.databricks.sdk.service.catalog.TableInfo`.

OpenAPI SHA: beff621d7b3e1d59244e2e34fc53a496f310e130, Date: 2023-08-17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants