Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/splunkhecexporter] expose HTTPClientSettings on splunkhecexporter #16839

Merged

Conversation

atoulme
Copy link
Contributor

@atoulme atoulme commented Dec 10, 2022

Description:
Expose HTTPClientSettings on the config of the exporter
Link to tracking Issue:
#16838

Testing:
Unit tests.

Documentation:
Update to README with new documentation and deprecation of the MaxConnections setting.

@runforesight
Copy link

runforesight bot commented Dec 10, 2022

Foresight Summary

    
Major Impacts

build-and-test-windows duration(5 seconds) has decreased 41 minutes 22 seconds compared to main branch avg(41 minutes 27 seconds).
View More Details

⭕  build-and-test-windows workflow has finished in 5 seconds (41 minutes 22 seconds less than main branch avg.) and finished at 25th Jan, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  telemetrygen workflow has finished in 50 seconds (1 minute 35 seconds less than main branch avg.) and finished at 25th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  tracegen workflow has finished in 1 minute 3 seconds (1 minute 23 seconds less than main branch avg.) and finished at 25th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  check-links workflow has finished in 1 minute 47 seconds and finished at 25th Jan, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  changelog workflow has finished in 2 minutes 42 seconds and finished at 25th Jan, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 11 minutes 2 seconds (⚠️ 3 minutes 27 seconds more than main branch avg.) and finished at 25th Jan, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

 build-and-test workflow has finished in 8 minutes 59 seconds (43 minutes 9 seconds less than main branch avg.) and finished at 25th Jan, 2023.


Job Failed Steps Tests
correctness-metrics N/A  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces N/A  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, internal) N/A  ✅ 561  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, internal) N/A  ✅ 561  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, extension) N/A  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, extension) N/A  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, processor) N/A  ✅ 1471  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) N/A  ✅ 1471  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-0) N/A  ✅ 2567  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) N/A  ✅ 2567  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) N/A  ✅ 2411  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-1) N/A  ✅ 1944  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) N/A  ✅ 4631  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, exporter) N/A  ✅ 2411  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) N/A  ✅ 1944  ❌ 0  ⏭ 0    🔗 See Details
integration-tests N/A  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, other) N/A  ✅ 4631  ❌ 0  ⏭ 0    🔗 See Details

✅  load-tests workflow has finished in 19 minutes 27 seconds (⚠️ 5 minutes 9 seconds more than main branch avg.) and finished at 26th Jan, 2023.


Job Failed Steps Tests
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 19  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

Copy link
Contributor

@MovieStoreGuy MovieStoreGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes make sense to me but worth code owners also reviewing.

exporter/splunkhecexporter/client.go Outdated Show resolved Hide resolved
@fatsheep9146
Copy link
Contributor

@dmitryax Do you have time to quick review for this enhancement? The change is not that much, and also comes from an another owner of this component.

@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch from 555dda7 to 993b98b Compare December 14, 2022 07:13
@MovieStoreGuy
Copy link
Contributor

@atoulme once you get a chance, please rebase

@atoulme atoulme closed this Jan 6, 2023
@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch from 993b98b to bf8b664 Compare January 6, 2023 06:31
@atoulme atoulme reopened this Jan 6, 2023
@atoulme
Copy link
Contributor Author

atoulme commented Jan 6, 2023

I restarted this PR from scratch. I used the opportunity to clean up some constructs no longer needed with exporterhelper.

@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch 2 times, most recently from 08c6204 to 0005065 Compare January 11, 2023 17:03
@atoulme
Copy link
Contributor Author

atoulme commented Jan 11, 2023

Rebased after 0.69.0, needs a new review.

@atoulme atoulme requested a review from MovieStoreGuy January 11, 2023 17:05
@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch from 0005065 to cf3a472 Compare January 11, 2023 19:10
Copy link
Member

@dmitryax dmitryax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR now has many changes that are not always related to the original issue. Can you please keep the scope of the PR limited to the original issue and make any other refactoring in a separate follow-up PR?

exporter/splunkhecexporter/factory.go Outdated Show resolved Hide resolved
exporter/splunkhecexporter/factory.go Outdated Show resolved Hide resolved
exporter/splunkhecexporter/factory.go Outdated Show resolved Hide resolved
@atoulme
Copy link
Contributor Author

atoulme commented Jan 12, 2023

OK I will break it down into smaller PRs. Thanks!

@atoulme
Copy link
Contributor Author

atoulme commented Jan 12, 2023

Per code review, I have removed the errors and hopefully simplified the diff a bit more. Let me know.

@bogdandrutu
Copy link
Member

Please rebase

@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch from 80d03f3 to 168a424 Compare January 24, 2023 06:00
@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch from 168a424 to 7c4d0b4 Compare January 24, 2023 22:29
@atoulme atoulme force-pushed the splunkhecexporter_httpclientsettings branch from 7c4d0b4 to 83c0266 Compare January 24, 2023 23:02
@atoulme
Copy link
Contributor Author

atoulme commented Jan 24, 2023

Had one lint job time out, rerunning...

@atoulme
Copy link
Contributor Author

atoulme commented Jan 25, 2023

@bogdandrutu rebased


func buildHTTPClient(config *Config, host component.Host, telemetrySettings component.TelemetrySettings) (*http.Client, error) {
// we handle compression explicitly.
config.HTTPClientSettings.Compression = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't mutate the original config. We should make a copy instead. Can you please create an issue to follow up on this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. I think we do this in signalfx too. I'll make an issue. I guess those objects will eventually be immutable, is that right?

Copy link
Member

@dmitryax dmitryax Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forcing them to be immutable would be pretty complicated, but as a rule we've been trying to not mutate the original user-supplied config

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so would you agree a better fix is to validate the user doesn't set a compression algo then? That would take care of this, and might be more elegant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be better, but we also override MaxIdleConns and MaxIdleConnsPerHost

@atoulme atoulme mentioned this pull request Jan 26, 2023
@dmitryax dmitryax merged commit 395c731 into open-telemetry:main Jan 26, 2023
@atoulme atoulme deleted the splunkhecexporter_httpclientsettings branch January 26, 2023 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants