Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/clickhouse] Add create_schema option to config #32282

Merged
merged 5 commits into from
Jun 3, 2024

Conversation

SpencerTorres
Copy link
Member

Description:

Adds create_schema boolean to config. This flag will disable the automatic creation of the database and tables so that this can be manually managed by the user.

Link to tracking Issue:
#29443 (related)

Testing:
Ran integration tests, and added test for verifying undefined + true + false in for the value in the config.

Documentation:

Added config option to README along with new "schema management" section that contains some recommendations for production deployments as well as requirements for maintaining compatibility.

Also added examples/default_ddl/ which contains all of the default DDL that the exporter runs. These are to be used as a starting point for users managing their own schema.

@SpencerTorres SpencerTorres requested a review from dmitryax as a code owner April 9, 2024 20:42
@SpencerTorres SpencerTorres requested a review from a team April 9, 2024 20:42
@github-actions github-actions bot requested review from Frapschen and hanjm April 9, 2024 20:42
@@ -0,0 +1,42 @@
-- Default Histogram metrics table DDL

CREATE TABLE IF NOT EXISTS otel_metrics_histogram (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a unit test to verify the table DDL gennerate by code is as same as the example ones.

it can prevent the example ones allway up-to-date.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered reading the template strings from a file to avoid this completely, but ultimately this DDL is just an example.
As long as the column names and types are the same, the inserts should run correctly.

If we were to add unit tests then I would prefer to just read the SQL in from a file, or maybe embed it in the code at build time, or extract them into individual .go files. Let me know if any of this sounds like a better solution and I will update the PR. 👍

if err := createDatabase(ctx, e.cfg); err != nil {
return err
}

internal.SetLogger(e.logger)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to drop this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was moved to the top of the function so that it is called before the DDL statements (conditionally) run.

I also considered extracting all of the DDL to its own function outside of start so this separation would be more explicit instead of relying on an early return from ShouldCreateSchema. Let me know if I should extract it to a runDDL type function. 👍

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label May 14, 2024
@mx-psi
Copy link
Member

mx-psi commented May 14, 2024

@Frapschen can you give this another look and check @SpencerTorres's comments?

@github-actions github-actions bot removed the Stale label May 15, 2024
@Frapschen
Copy link
Contributor

@SpencerTorres We can put the work discussed in the comments into the next PR and get this one to merge.

@Frapschen
Copy link
Contributor

@mx-psi ready to merge

@mx-psi mx-psi merged commit ed0cf90 into open-telemetry:main Jun 3, 2024
162 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 3, 2024
dmitryax added a commit that referenced this pull request Jun 24, 2024
#### EXTRACTED FROM #33614 
#### DEPENDS ON #33693 

**Description:**

A follow-up to #32282 that changes `create_schema` from `*bool` to
`bool`, while also properly using the default config / factory.

**Testing:** 
- Updated tests

---------

Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
greatestusername-splunk pushed a commit to greatestusername/opentelemetry-collector-contrib that referenced this pull request Jun 24, 2024
[clickhouse/exporter] Update create schema config option (open-telemetry#33694)

**Description:**

A follow-up to open-telemetry#32282 that changes `create_schema` from `*bool` to
`bool`, while also properly using the default config / factory.

**Testing:**
- Updated tests

---------

Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
@SpencerTorres SpencerTorres deleted the create-schema-option branch July 2, 2024 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants