[DJM] Document custom host tags env variables #28577
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Preview links (active after the
|
warrierr
requested changes
Apr 7, 2025
content/en/data_jobs/databricks.md
Outdated
| @@ -71,6 +71,16 @@ This setup does not work on Databricks clusters in <strong>Standard</strong> (fo | |||
| 1. Click **Save Databricks Workspace** at the bottom of the browser window. | |||
| {{< img src="data_jobs/databricks/configure-data-jobs-monitoring-existing.png" alt="In the Datadog-Databricks integration tile, Datadog Agent Setup for a Databricks workspace already added to the integration. Datadog can install and manage a global init script." style="width:100%;" >}} | |||
|
|
|||
| Optionally, you can add custom host tags to your Databricks clusters by configuring the following environment variable in the Advanced Configuration section of your cluster in the Databricks UI or as [Spark env vars][2] with the Databricks API: | |||
There was a problem hiding this comment.
Suggested change
| Optionally, you can add custom host tags to your Databricks clusters by configuring the following environment variable in the Advanced Configuration section of your cluster in the Databricks UI or as [Spark env vars][2] with the Databricks API: | |
| Optionally, you can add tags to your Databricks cluster and Spark performance metrics by configuring the following environment variable in the Advanced Configuration section of your cluster in the Databricks UI or as [Spark env vars][2] with the Databricks API: |
There was a problem hiding this comment.
@aboitreaud what API are you referring to at the end: "with the Databricks API" ?
There was a problem hiding this comment.
https://docs.databricks.com/api/workspace/clusters/edit
It's the way StoneX are updating their cluster from what I understood in DJMS-46
content/en/data_jobs/databricks.md
Outdated
|
|
||
| | Variable | Description | Default | | ||
| |--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| | ||
| | DD_TAGS | Add host tags to Databricks clusters. Comma or space separated key:value pairs. Follow [Datadog tag conventions][1]. Example: `env:staging,team:data_engineering` | | |
There was a problem hiding this comment.
@aboitreaud do we actually want this as the method for showing how to set env? Isn't that DD_ENV ?
Co-authored-by: Ryan Warrier <ryan.warrier@datadoghq.com>
michaelcretzman
approved these changes
Apr 7, 2025
Co-authored-by: Michael Cretzman <58786311+michaelcretzman@users.noreply.github.com>
Co-authored-by: Ryan Warrier <ryan.warrier@datadoghq.com>
aboitreaud
force-pushed
the
adrien.boitreaud/host-tags-djm-doc
branch
from
d40a1af to
ea738e1
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
In the doc pages for the three DJM plarforms, document the possibility to add custom host tags with DD_TAGS and DD_EXTRA_TAGS.
Merge instructions
Merge readiness: Not ready
For Datadog employees:
Merge queue is enabled in this repo. Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass in CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
To have your PR automatically merged after it receives the required reviews, add the following PR comment:
Additional notes