-
Notifications
You must be signed in to change notification settings - Fork 384
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] Re-add Resource databricks_instance_profile
skip_validation
option
#762
Comments
databricks_instance_profile
skip_validation
optiondatabricks_instance_profile
skip_validation
option
How to guarantee that instance profile added would work, if we skip validation? Terraform's goal is to create a resource in a valid runnable state or fail with error and remove that underlying invalid resource. Can you add exceptions for dry runs? There were multiple complaints that clusters with attached profiles didn't work. |
That's the risk you run when using the To avoid this you would have the This would not impact current consumers of the resource while also giving the option for users who understand the potential impacts. This also brings the resource more inline with the 2.0 api functionality. |
@tylangesmith provider core team doesn't currently have cycles for this feature request. If you still would like to have this functionality, please send a pull request with the passing output of |
Hey, came across this issue recently too. We were trying to enforce tags on cluster creation to follow our organization standards using this documentation as reference: https://docs.databricks.com/clusters/configure.html#enforce-mandatory-tags Having the mandatory tag policy on the cross account role makes validation fail when a new instance profile is added. Seconding the request to skip validation if there's no other way to workaround this while having a tagging policy. |
The aws ec2 run-instances command also has a |
Hey @davehowell, small world 😆 Yeah I had this discussion with our Databricks SA, highlighting these exact issues i.e. the current validation implementation will not work for our AWS environment and likely more customers as they mature their cloud environments. Hence why we want to use the This sort of simple validation is something I've seen as a common theme across a few of the Databricks APIs. Hopefully with their latest round of funding they'll be able to further the maturity and address these types of issues. Unfortunately for this issue we're 1 abstraction layer higher than the API implementation so we won't be able to change the underlying implementation and how it actually performs the underlying validation logic. |
@tylangesmith I see, makes sense then. It should be consistent with the other Databricks APIs. If that feature was previously removed it might be easy to cherry pick it out of the history. |
What a chat here 😆 |
This is also something that would be great for us. Right now we have to add the instance profile manually to each workspace. |
It would be very valuable for us to have this. Otherwise, we have to do it manually via the UI. |
In the project I'm working on, was introduced a rule to not allow creating an EC2 instance without disk encryption and therefore the instance validation is failing. As we know the instance works for other projects using the same configuration, we consider it's safe to skip the instance validation. As this option is part of Databricks API, I think this should be available in Terraform as well. Sad to hear it got removed, we are adding instance profile manually and importing it to TF state. |
I believe this is what needs to be reverted: cdef6f9#diff-142ef36d5f14d2cdcd77cb2c75ba58375dcfc4209bc51065ab45835994c14c8fL44 |
Hey all,
As an enterprise customer we've recently started configuring all of our Databricks configuration using infrastructure as code principles. This has lead us to using this terraform databricks provider.
One of our current frustrations comes from the
databricks_instance_profile
resource. The 2.0 api specifies that the instance profile is validated using theaws ec2 run-instance --dry-run
command.In practice this sounds like a good idea. However for enterprise customers who have a mature cloud environment there are security postures that are enforced e.g. AWS SCPs that enforce a set of tags to be applied to EC2 instances.
By using the
aws ec2 run-instance --dry-run
command to validate without the ability to specific additional run options this gives us a false positive validation check. To get around this we can use theskip_validation
option provided by the 2.0 api.As of this provider's 0.3.0 release this
skip_validation
option was removed from thedatabricks_instance_profile
resource.Is the removal of this option able to be reconsidered? As it has a very legitimate use-case for us enterprise customers.
The text was updated successfully, but these errors were encountered: