Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CBG-3791: change getAuthScopeHandleCreateDB method to not expand environment variables #6703

Merged
merged 4 commits into from
Mar 5, 2024

Conversation

gregns1
Copy link
Contributor

@gregns1 gregns1 commented Feb 26, 2024

CBG-3791

  • Removed the use of DecodeAndSanitiseConfig as it expands environment variables
  • As we only need bucket name from this function just kept much the functionality the same but with no expansion of environment variables.
  • Kept the use of Decode over Unmarshal as I feel it's more memory efficient and given we unmarshal the config later in readSanitizeDbConfigJSON I didn't want this done twice.
  • Evaluated the use of readSanitizeDbConfigJSON in place of what I have done in this PR but found it awkward putting the bytes back into a reader for use later.

Pre-review checklist

  • Removed debug logging (fmt.Print, log.Print, ...)
  • Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
  • Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Integration Tests

@gregns1 gregns1 requested a review from bbrks February 26, 2024 11:51
@bbrks bbrks assigned gregns1 and unassigned bbrks Feb 26, 2024
@@ -181,7 +181,14 @@ func getAuthScopeHandleCreateDB(ctx context.Context, h *handler) (bucketName str
Bucket string `json:"bucket"`
}
reader := bytes.NewReader(bodyJSON)
err = DecodeAndSanitiseConfig(ctx, reader, &dbConfigBody, false)

// read and decode json to pull bucket name from the config
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can call readSantizedJson here.

The thing that might happen is that you need to call ConvertBackQuotedStrings to make it valid json. The test should include a config with backquotes

},
"num_index_replicas": 0,
"enable_shared_bucket_access": true,
"use_views": false}`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might fail if SG_TEST_USE_XATTRS=false is set, can you check?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checked and no failure.

@gregns1 gregns1 assigned bbrks and unassigned gregns1 Feb 28, 2024
Comment on lines 183 to 193
reader := bytes.NewReader(bodyJSON)
err = DecodeAndSanitiseConfig(ctx, reader, &dbConfigBody, false)

// read and decode json to pull bucket name from the config
b, err := io.ReadAll(reader)
if err != nil {
return "", err
}

// Expand environment variables if needed
if base.BoolDefault(h.server.Config.Unsupported.AllowDbConfigEnvVars, true) {
b, err = expandEnv(h.ctx(), b)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're doing bytes->reader->bytes for no reason here with NewReader->ReadAll

We should be able to pass bodyJSON straight into expandEnv and ConvertBackQuotedStrings, since they don't modify the slice, they return a modified copy of it.

It does feels like we need to push those two pieces (expandEnv/ConvertBackQuotedStrings) into a shared function called by this, DecodeAndSanitiseConfig, and readSanitizeJSON.

We can pass a bool in to that function control env var behaviour and just have it take bytes and return bytes.

Copy link
Member

@bbrks bbrks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look fine. We need to make sure the new tests work with the different combinations of env vars we can use though.

// - Create CBS user to authenticate with over admin port to force auth scope callback call
// - Create db with sync function that calls env variable
// - Assert that db is created
func TestDatabaseCreationWithEnvVariable(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test doesn't work for SG_TEST_USE_GSI=false or SG_TEST_USE_DEFAULT_COLLECTION=true

Will need to build the db config to accommodate those modes.

Some existing functions to consider using or adapting:

  • dbConfigForTestBucket
  • makeDbConfig

Should be able to include string values like "${VAR}" for sync function before marshalling it to pass into the request.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow didn't even think about that, on it fixing it

@bbrks bbrks assigned gregns1 and unassigned bbrks Mar 1, 2024
@gregns1 gregns1 assigned bbrks and unassigned gregns1 Mar 1, 2024
@bbrks bbrks merged commit 5bfdb17 into master Mar 5, 2024
30 checks passed
@bbrks bbrks deleted the CBG-3791 branch March 5, 2024 12:34
gregns1 added a commit that referenced this pull request Mar 5, 2024
…ronment variables (#6703)

* CBG-3791: change getAuthScopeHandleCreateDB method to not expand environment variables

* updates after review

* updates off review

* fix test for default collection
gregns1 added a commit that referenced this pull request Mar 5, 2024
…ronment variables (#6703) (#6714)

* CBG-3791: change getAuthScopeHandleCreateDB method to not expand environment variables

* updates after review

* updates off review

* fix test for default collection
bbrks pushed a commit that referenced this pull request Mar 28, 2024
…ronment variables (#6703)

* CBG-3791: change getAuthScopeHandleCreateDB method to not expand environment variables

* updates after review

* updates off review

* fix test for default collection
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants