Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add databricks_app resource #4099

Merged
merged 116 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
c8859e2
added `databricks_app` resource
nkvuong Oct 12, 2024
962d9f2
wip
nkvuong Oct 16, 2024
2e9137a
add doc
nkvuong Oct 29, 2024
91b5166
alias
nkvuong Oct 29, 2024
bcf2a97
Merge branch 'main' into feature/app
nkvuong Oct 29, 2024
35b85eb
fix
nkvuong Oct 29, 2024
9c7ad60
fix test
nkvuong Oct 30, 2024
863fa37
fix test
nkvuong Oct 30, 2024
2d7b38c
fix
nkvuong Oct 30, 2024
b733ee9
Merge branch 'main' into feature/app
nkvuong Nov 14, 2024
ff8bee8
feedback
nkvuong Nov 14, 2024
0daeea1
fix test
nkvuong Nov 15, 2024
7449b45
Merge branch 'main' into feature/app
nkvuong Nov 15, 2024
eda8a91
feedback
nkvuong Nov 15, 2024
f254176
feedback
nkvuong Nov 16, 2024
18b3520
Merge branch 'main' into feature/app
nkvuong Nov 19, 2024
be96f91
feedback
nkvuong Nov 21, 2024
8338d09
add app permission
nkvuong Nov 22, 2024
c6ee7ea
Merge branch 'main' into feature/app
nkvuong Nov 22, 2024
07ea8b7
fix test
nkvuong Nov 22, 2024
3b1f6e2
Merge branch 'feature/app' of https://github.com/databricks/terraform…
nkvuong Nov 22, 2024
9e41c69
more computed fields
nkvuong Nov 22, 2024
0ad4294
feedback
nkvuong Nov 26, 2024
8bdcda9
fix test
nkvuong Nov 26, 2024
2bac7aa
feedback
nkvuong Nov 27, 2024
17e1923
work
mgyucht Nov 28, 2024
731469e
Remove unused configuration from blocks
mgyucht Nov 29, 2024
7dd91f4
add test
mgyucht Nov 29, 2024
14fea49
fix
mgyucht Nov 29, 2024
6bd77aa
Add ConvertToAttribute() support
mgyucht Dec 2, 2024
25ae4ba
small typos
mgyucht Dec 2, 2024
7ba55dd
Merge branch 'main' into add-convert-to-attribute
mgyucht Dec 3, 2024
a62a65f
Merge branch 'main' into feature/app
mgyucht Dec 3, 2024
08d4111
Merge branch 'add-convert-to-attribute' into feature/app
mgyucht Dec 3, 2024
1628cb8
work
mgyucht Dec 3, 2024
e30e677
wip first commit
mgyucht Dec 4, 2024
b9f8e65
Merge branch 'use-tftypes-everywhere' into feature/app
mgyucht Dec 4, 2024
6e44632
maybe
mgyucht Dec 4, 2024
29d73dd
Merge branch 'use-tftypes-everywhere' into feature/app
mgyucht Dec 4, 2024
ff11182
fixes
mgyucht Dec 5, 2024
8af1c31
works
mgyucht Dec 5, 2024
dac1bab
Merge branch 'main' into use-tftypes-everywhere
mgyucht Dec 5, 2024
c4c9375
work
mgyucht Dec 5, 2024
3e26f55
some work
mgyucht Dec 5, 2024
7336447
working
mgyucht Dec 5, 2024
cfdec91
more work
mgyucht Dec 5, 2024
4b36d53
fix
mgyucht Dec 5, 2024
93763a8
more work
mgyucht Dec 5, 2024
37d3f1b
work
mgyucht Dec 5, 2024
3b5015b
go mod tidy
mgyucht Dec 5, 2024
123432c
comment
mgyucht Dec 5, 2024
b84854a
cleanup
mgyucht Dec 5, 2024
6222b90
add coverage for object types
mgyucht Dec 5, 2024
295c972
more comments
mgyucht Dec 5, 2024
6a39e1b
more comments
mgyucht Dec 5, 2024
e4aff40
clean up resource_sahre
mgyucht Dec 5, 2024
c5a1181
Merge branch 'use-tftypes-everywhere' into feature/app
mgyucht Dec 5, 2024
2c3ed47
fix
mgyucht Dec 5, 2024
5b52d77
fix
mgyucht Dec 5, 2024
edc1d4c
work
mgyucht Dec 5, 2024
30398da
it passes a test
mgyucht Dec 5, 2024
560ce27
work
mgyucht Dec 6, 2024
687996c
Merge branch 'use-tftypes-everywhere' into feature/app
mgyucht Dec 6, 2024
5779ece
works again
mgyucht Dec 6, 2024
b3634bd
work
mgyucht Dec 6, 2024
32cba97
fix
mgyucht Dec 6, 2024
4fdd72b
work
mgyucht Dec 6, 2024
0619e2e
Merge branch 'use-tftypes-everywhere' into feature/app
mgyucht Dec 6, 2024
a9f12b6
bugfix
mgyucht Dec 9, 2024
2feedf4
improve error message
mgyucht Dec 9, 2024
4feb053
fix go to tf conversion
mgyucht Dec 9, 2024
d10447f
fixes
mgyucht Dec 9, 2024
ed7246e
fmt
mgyucht Dec 9, 2024
0d49760
fix types.Object handling
mgyucht Dec 9, 2024
a0297e6
Merge branch 'use-tftypes-everywhere' into feature/app
mgyucht Dec 9, 2024
adb9746
Merge branch 'main' into feature/app
mgyucht Dec 11, 2024
5adec2e
revert sharing change
mgyucht Dec 11, 2024
6496607
remove extra file
mgyucht Dec 11, 2024
285c29f
work
mgyucht Dec 11, 2024
ed7b0ec
Panic if the provided path is invalid
mgyucht Dec 11, 2024
853622c
Merge branch 'panic-on-invalid-path' into feature/app
mgyucht Dec 11, 2024
9588149
fix
mgyucht Dec 11, 2024
07a77bc
Expose several integration test helpers for use in plugin framework i…
mgyucht Dec 11, 2024
c3ef933
more
mgyucht Dec 11, 2024
5da3a69
Merge branch 'expose-integration-test-helpers' into feature/app
mgyucht Dec 11, 2024
ecd7d29
tests
mgyucht Dec 11, 2024
f138239
Merge branch 'main' into feature/app
mgyucht Dec 11, 2024
b929583
first attempt
mgyucht Dec 11, 2024
5c4a1c7
Merge branch 'use-attributes-by-default' into feature/app
mgyucht Dec 11, 2024
ec950fb
fix
mgyucht Dec 11, 2024
eaf71eb
work
mgyucht Dec 11, 2024
7f8a845
work
mgyucht Dec 11, 2024
d16ec81
fixes
mgyucht Dec 11, 2024
29aa7e9
Merge branch 'readonly-and-list-validators' into use-attributes-by-de…
mgyucht Dec 11, 2024
7c539b8
some work
mgyucht Dec 11, 2024
e282b55
work
mgyucht Dec 11, 2024
223acf8
fix formatting
mgyucht Dec 11, 2024
92dbbe9
fix formatting
mgyucht Dec 11, 2024
3ba9694
Merge branch 'main' into use-attributes-by-default
mgyucht Dec 12, 2024
8d42b08
less unnecessary diff
mgyucht Dec 12, 2024
f0117c8
work
mgyucht Dec 12, 2024
28c6acc
fix
mgyucht Dec 12, 2024
361cec4
update contributing
mgyucht Dec 12, 2024
402f40a
docs
mgyucht Dec 12, 2024
c425508
docs
mgyucht Dec 12, 2024
fc9e4c8
tests and better error
mgyucht Dec 12, 2024
10c456e
rename
mgyucht Dec 12, 2024
86b8cce
fix test
mgyucht Dec 12, 2024
5f107cf
Merge branch 'use-attributes-by-default' into feature/app
mgyucht Dec 12, 2024
c414f8a
improve integration test
mgyucht Dec 12, 2024
a9e438c
Merge branch 'main' into feature/app
mgyucht Dec 12, 2024
66fe1b9
import
mgyucht Dec 12, 2024
59e16d5
add import test
mgyucht Dec 12, 2024
e0dd84d
sort
mgyucht Dec 12, 2024
9114ea1
data sources
mgyucht Dec 12, 2024
52e975c
fast app
mgyucht Dec 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 181 additions & 0 deletions apps/resource_app.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
package apps

import (
"context"
"log"
"time"

"github.com/databricks/databricks-sdk-go/service/apps"
"github.com/databricks/terraform-provider-databricks/common"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
)

const defaultAppProvisionTimeout = 10 * time.Minute
const deleteCallTimeout = 10 * time.Second
nkvuong marked this conversation as resolved.
Show resolved Hide resolved

var appAliasMap = map[string]string{
"resources": "resource",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rauchy Can you discuss with @nkvuong about adding this alias to our API spec?

}

type appStruct struct {
apps.App
// The mode of which the deployment will manage the source code.
Mode string `json:"mode,omitempty"`
// The workspace file system path of the source code used to create the app
// deployment. This is different from
// `deployment_artifacts.source_code_path`, which is the path used by the
// deployed app. The former refers to the original source code location of
// the app in the workspace during deployment creation, whereas the latter
// provides a system generated stable snapshotted source code path used by
// the deployment.
SourceCodePath string `json:"source_code_path,omitempty"`
}

type appCreateStruct struct {
apps.CreateAppRequest
}

func (appCreateStruct) Aliases() map[string]map[string]string {
return map[string]map[string]string{
"apps.appCreateStruct": appAliasMap,
}
}

func (appCreateStruct) CustomizeSchema(s *common.CustomizableSchema) *common.CustomizableSchema {
return s
}

type appUpdateStruct struct {
apps.UpdateAppRequest
}

func (appUpdateStruct) Aliases() map[string]map[string]string {
return map[string]map[string]string{
"apps.appUpdateStruct": appAliasMap,
}
}

func (appUpdateStruct) CustomizeSchema(s *common.CustomizableSchema) *common.CustomizableSchema {
return s
}

func (appStruct) Aliases() map[string]map[string]string {
return map[string]map[string]string{
"apps.appStruct": appAliasMap,
}
}

func (appStruct) CustomizeSchema(s *common.CustomizableSchema) *common.CustomizableSchema {

// Required fields & validation
s.SchemaPath("name").SetRequired().SetForceNew().SetValidateFunc(validation.StringLenBetween(2, 30))
nkvuong marked this conversation as resolved.
Show resolved Hide resolved
s.SchemaPath("mode").SetDefault("SNAPSHOT").SetValidateFunc(validation.StringInSlice([]string{"SNAPSHOT", "AUTO_SYNC"}, false))

// Computed fields
for _, p := range []string{"active_deployment", "app_status", "compute_status", "create_time", "creator",
"default_source_code_path", "pending_deployment", "service_principal_id", "service_principal_name",
"update_time", "updater", "url"} {
nkvuong marked this conversation as resolved.
Show resolved Hide resolved
s.SchemaPath(p).SetComputed()
}

// SuppressDiff
s.SchemaPath("source_code_path").SetCustomSuppressDiff(common.WorkspacePathPrefixDiffSuppress)

return s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resource types are mutually exclusive per resource element (secret, sql_warehouse, etc)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can add multiple resources to an app

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but within a resource you should only have one of (secret, sql_warehouse, etc).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add this constraint back, where you cannot have more than one defined per resource?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because the type is nested within resource, we cannot add constraints with sdkv2 - would need to rewrite this to framework to be able to add this validation rule

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's at least do the validation in the create/update. (Or at least test to verify what its behavior is.)

}

var appSchema = common.StructToSchema(appStruct{}, nil)

func ResourceApp() common.Resource {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW: Can we use the plugin framework for this new resource? Generally speaking, new resources should use the plugin framework, since it should be much more consistent for handling things like effective fields, dealing with lists, etc.

return common.Resource{
Create: func(ctx context.Context, d *schema.ResourceData, c *common.DatabricksClient) error {
var createApp appCreateStruct
var createAppDeployment apps.CreateAppDeploymentRequest
common.DataToStructPointer(d, appSchema, &createApp)
common.DataToStructPointer(d, appSchema, &createAppDeployment)
w, err := c.WorkspaceClient()
if err != nil {
return err
}

// create the app, which does not require the source code path yet
wait, err := w.Apps.Create(ctx, createApp.CreateAppRequest)
if err != nil {
return err
}
app, err := wait.GetWithTimeout(d.Timeout(schema.TimeoutCreate) - deleteCallTimeout)
pietern marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
log.Printf("[ERROR] Error waiting for app to be created: %s", err.Error())
_, nestedErr := w.Apps.DeleteByName(ctx, createApp.Name)
if nestedErr != nil {
log.Printf("[ERROR] Error cleaning up app: %s", nestedErr.Error())
}
return err
}
// now deploy the app, using the source code path
createAppDeployment.AppName = app.Name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we discussed it offline and agreed that the deployment won't be part of TF configuration. Instead, if an users want to deploy, they use CLI to do that. Otherwise it's confusing because we do the deployments only when source_code_path changed but we need to do deployments when source code is changed.

@pietern wdyt?

waitDeploy, err := w.Apps.Deploy(ctx, createAppDeployment)
if err != nil {
return err
}
_, err = waitDeploy.GetWithTimeout(d.Timeout(schema.TimeoutCreate) - deleteCallTimeout)
if err != nil {
log.Printf("[ERROR] Error waiting for app to be deployed: %s", err.Error())
_, nestedErr := w.Apps.DeleteByName(ctx, createApp.Name)
if nestedErr != nil {
log.Printf("[ERROR] Error cleaning up app: %s", nestedErr.Error())
}
return err
}
d.SetId(app.Name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update this to get it back into a healthy state? if not this is fine. If so, let's set the ID before waiting for ready, then subsequent applies can call update.

return nil
},
Read: func(ctx context.Context, d *schema.ResourceData, c *common.DatabricksClient) error {
w, err := c.WorkspaceClient()
if err != nil {
return err
}
app, err := w.Apps.GetByName(ctx, d.Id())
if err != nil {
return err
}
return common.StructToData(appStruct{App: *app}, appSchema, d)
},
Update: func(ctx context.Context, d *schema.ResourceData, c *common.DatabricksClient) error {
var update appUpdateStruct
common.DataToStructPointer(d, appSchema, &update)
w, err := c.WorkspaceClient()
if err != nil {
return err
}
_, err = w.Apps.Update(ctx, update.UpdateAppRequest)
if err != nil {
return err
}
if d.HasChanges("source_code_path", "mode") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment above, I think we should remove app deployment altogether for now

_, err = w.Apps.Deploy(ctx, apps.CreateAppDeploymentRequest{
nkvuong marked this conversation as resolved.
Show resolved Hide resolved
AppName: d.Id(),
Mode: apps.AppDeploymentMode(d.Get("mode").(string)),
SourceCodePath: d.Get("source_code_path").(string),
})
if err != nil {
return err
}
}
return nil
},
Delete: func(ctx context.Context, d *schema.ResourceData, c *common.DatabricksClient) error {
w, err := c.WorkspaceClient()
if err != nil {
return err
}
_, err = w.Apps.DeleteByName(ctx, d.Id())
return err
},
Schema: appSchema,
Timeouts: &schema.ResourceTimeout{
Create: schema.DefaultTimeout(defaultAppProvisionTimeout),
},
}
}
Loading
Loading