Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAB deploy different resources to different targets #2364

Open
iam-meghana opened this issue Feb 21, 2025 · 0 comments
Open

DAB deploy different resources to different targets #2364

iam-meghana opened this issue Feb 21, 2025 · 0 comments

Comments

@iam-meghana
Copy link

iam-meghana commented Feb 21, 2025

Describe the issue

We have defined resources in individual yml files in the bundle that has all the jobs in one folder. However lets say we have two targets a and b, and want to include only resources that end with *_a.yml for target a and *_b.yml for target b plus any common resources, there is no way to include/exclude only relevant resources when deploying the targets.

└── some_sample_proj/
└── bundle_name/
|── resources/
│ ├── jobs/
│ │ ├── job_a.yml
│ │ ├── job_b.yml
│ │ └── job_common.yml
│ ├── pipelines/
│ │ ├── pipeline_a.yml
│ │ └── pipeline_b.yml
│ └── clusters/
│ └── shared_cluster.yml
├── databricks.yml
└── ...

In the databricks.yml file, it would be great to have some way of referencing resources in the targets to be included or excluded (similar to sync include/exclude) that also supports wildcards, like:

bundle:
   name: bundle_name

include:
  - resources/jobs/*.yml

targets:
  a:
     ....
     resources: 
        exclude: 
            - resources/jobs/*_b.yml

 b:
     ....
     resources: 
        exclude: 
            - resources/jobs/*_a.yml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant