Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add caching of scim/me API to databricks_current_user data source and databricks_permissions resource. #2170

Merged
merged 6 commits into from
Mar 29, 2023

Conversation

kartikgupta-db
Copy link
Contributor

@kartikgupta-db kartikgupta-db commented Mar 29, 2023

Changes

  • cache scim/me API

Tests

  • manual
  • existing acceptance tests
    • gcp-prod
    • aws-prod
    • azure-prod
    • aws-prod-ucws
    • aws-prod-ucacct
    • azure-prod-acct
    • gcp-prod-acct

close #2120

@codecov-commenter
Copy link

codecov-commenter commented Mar 29, 2023

Codecov Report

Merging #2170 (576cd00) into master (9451a90) will decrease coverage by 0.20%.
The diff coverage is 18.75%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2170      +/-   ##
==========================================
- Coverage   88.99%   88.79%   -0.20%     
==========================================
  Files         137      137              
  Lines       11182    11209      +27     
==========================================
+ Hits         9951     9953       +2     
- Misses        827      849      +22     
- Partials      404      407       +3     
Impacted Files Coverage Δ
common/client.go 34.28% <0.00%> (-7.58%) ⬇️
jobs/resource_job.go 95.79% <ø> (ø)
exporter/context.go 85.51% <50.00%> (-0.41%) ⬇️
permissions/resource_permissions.go 90.25% <50.00%> (-1.49%) ⬇️

@kartikgupta-db kartikgupta-db marked this pull request as draft March 29, 2023 09:42
Copy link
Contributor

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you just create func (c *DatabricksClient) CurrentUser(ctx), that'll lock/unlock properly. just because there's only one instance of DatabricksClient per provider ;)

common/client.go Outdated Show resolved Hide resolved
common/client.go Outdated
if err != nil {
return user, err
}
*a.cachedUser = *user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not a.cachedUser = user?..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pointer points to the cached user in client. This is why we need to copy into the cachedUser value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just make it cachedUser *scim.User on the client.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would not work I think. If we do this, then a.cachedUser = user will lead to the cachedUser within CachedUserService to point to the new user, but the changes won't be propagated to the client.

common/client.go Outdated Show resolved Hide resolved
common/client.go Outdated Show resolved Hide resolved
common/client.go Show resolved Hide resolved
common/client.go Outdated Show resolved Hide resolved
@kartikgupta-db kartikgupta-db requested a review from nfx March 29, 2023 10:06
common/client.go Show resolved Hide resolved
common/client.go Outdated Show resolved Hide resolved
common/client.go Outdated Show resolved Hide resolved
common/client.go Outdated
if err != nil {
return user, err
}
*a.cachedUser = *user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just make it cachedUser *scim.User on the client.

common/client.go Show resolved Hide resolved
@kartikgupta-db kartikgupta-db requested a review from nfx March 29, 2023 10:21
Copy link
Contributor

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nfx nfx marked this pull request as ready for review March 29, 2023 10:57
@kartikgupta-db kartikgupta-db changed the title Cache scim/me API Add caching for scim/me API for databricks_current_user data source and databricks_permissions resource. Mar 29, 2023
@kartikgupta-db kartikgupta-db changed the title Add caching for scim/me API for databricks_current_user data source and databricks_permissions resource. Add caching of scim/me API to databricks_current_user data source and databricks_permissions resource. Mar 29, 2023
@kartikgupta-db kartikgupta-db merged commit 9e1eb13 into master Mar 29, 2023
@kartikgupta-db kartikgupta-db deleted the cache-scim-me branch March 29, 2023 11:30
nkvuong pushed a commit that referenced this pull request Mar 29, 2023
…nd `databricks_permissions` resource. (#2170)

* Cache scim/me API

* add mutext to protect cached user

* naming fix

* defer mu.unlock and remove unused code

* cache ws client

* inline method and rename
@nfx nfx mentioned this pull request Mar 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Reduce API calls associated with databricks_permissions to avoid rate limiting
3 participants