Upstream merge 0.102.0 #14201

steves-canva · 2024-06-11T23:18:30Z

Upstream merge of v0.102.0

No changes to Canva forked components.

**Description:** This is a cleanup to consolidate the constants used for prometheus translation in a single place. --------- Co-authored-by: Curtis Robert <crobert@splunk.com>

**Description:** <Describe what has changed.>  * remove suppport for carbon2 and graphite * add support for otlp format * do not support metadata attributes * do not support source headers * set otlp as default format This PR reduces size of open-telemetry#32315 **Link to tracking Issue:** open-telemetry#31479 **Testing:** - unit tests - manual tests **Documentation:** - Readme --------- Signed-off-by: Dominik Rosiek <drosiek@sumologic.com> Co-authored-by: Adam Boguszewski <aboguszewski@sumologic.com>

) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/aws/aws-sdk-go](https://togithub.com/aws/aws-sdk-go) | `v1.52.3` -> `v1.52.4` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2faws%2faws-sdk-go/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2faws%2faws-sdk-go/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2faws%2faws-sdk-go/v1.52.3/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2faws%2faws-sdk-go/v1.52.3/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>aws/aws-sdk-go (github.com/aws/aws-sdk-go)</summary> ### [`v1.52.4`](https://togithub.com/aws/aws-sdk-go/blob/HEAD/CHANGELOG.md#Release-v1524-2024-05-07) [Compare Source](https://togithub.com/aws/aws-sdk-go/compare/v1.52.3...v1.52.4) \=== ##### Service Client Updates - `service/b2bi`: Updates service documentation - `service/budgets`: Updates service API and documentation - This release adds tag support for budgets and budget actions. - `service/resiliencehub`: Updates service API, documentation, and paginators - `service/route53profiles`: Updates service API and documentation </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib).  --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com>

… OS (open-telemetry#32878) **Description:** <Describe what has changed.>  With the enhancement in open-telemetry#30297, the SQL Server receiver can now run on MacOS and Linux, since it no longer solely relies on Windows. The difference in functionality has (hopefully) been documented clearly between the different platforms, so the unsupported warning can be removed. I missed this in my previous PRs.

…23-772616cd8f5c (open-telemetry#32897) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/grafana/loki/pkg/push](https://togithub.com/grafana/loki) | `v0.0.0-20240506154431-a772ed705c65` -> `v0.0.0-20240507085123-772616cd8f5c` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240506154431-a772ed705c65/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240506154431-a772ed705c65/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib).  --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [go](https://go.dev/) ([source](https://togithub.com/golang/go)) | toolchain | minor | `1.21.9` -> `1.22.3` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>golang/go (go)</summary> ### [`v1.22.3`](https://togithub.com/golang/go/compare/go1.22.2...go1.22.3) ### [`v1.22.2`](https://togithub.com/golang/go/compare/go1.22.1...go1.22.2) ### [`v1.22.1`](https://togithub.com/golang/go/compare/go1.22.0...go1.22.1) ### [`v1.22.0`](https://togithub.com/golang/go/compare/go1.21.7...go1.22rc1) ### [`v1.21.10`](https://togithub.com/golang/go/compare/go1.21.9...go1.21.10) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib).  --------- Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com> Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de>

I wasn't able to reproduce the exact situation from CI, but the tests I changed weren't completing at all here. Looking closely, it looks like they never closed the stream, something the first test does. After closing it, the tests start passing locally for me. While I'm not confident this will fix the CI flaky failures, this does make the test work locally for me. Fixes open-telemetry#32877 Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>

…try#32902) Fixes open-telemetry#31628 Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> --------- Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>

…belonging to the same backend (open-telemetry#32032) **Description:** no need to reimplement that in an extremely allocation-inefficient fashion. I'm actually not sure why mergeTraces() and mergeMetrics() need to exist in the first place; all the other exporters coupled with the batch processor work just fine, not sure why loadbalancing would be special. open-telemetry#30141 seems to imply they were implemented to improve performance, but I don't really understand why batch processor would not have been sufficient for that improvement originally. benchmarks before: ``` goos: darwin goarch: arm64 pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter BenchmarkMergeTraces_X100-8 50214 23507 ns/op BenchmarkMergeTraces_X500-8 10000 113952 ns/op BenchmarkMergeTraces_X1000-8 5208 226062 ns/op BenchmarkMergeMetrics_X100-8 64933 18540 ns/op BenchmarkMergeMetrics_X500-8 12885 91418 ns/op BenchmarkMergeMetrics_X1000-8 6590 184584 ns/op PASS ok github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter 9.783s ``` and after: ``` goos: darwin goarch: arm64 pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter BenchmarkMergeTraces_X100-8 295886529 3.836 ns/op BenchmarkMergeTraces_X500-8 309865370 3.833 ns/op BenchmarkMergeTraces_X1000-8 310739948 3.800 ns/op BenchmarkMergeMetrics_X100-8 315567813 3.841 ns/op BenchmarkMergeMetrics_X500-8 310341650 3.849 ns/op BenchmarkMergeMetrics_X1000-8 314292003 3.830 ns/op PASS ok github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter 10.733s ``` **Link to tracking Issue:** n/a **Testing:** unit tests pass & cpu time for our collectors using loadbalancingexporter (12 replicas, total of 25k-40k spans/sec) went from 800ms-1400ms/sec down to <40msec/sec. **Documentation:** none --------- Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

I have been unable to provide this position the bandwidth that it deserves and it is time to formalize recognition of that fact. Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

…pen-telemetry#32361) **Description:** <Describe what has changed.>  This PR contains the following changes: 1. Add `Close` call to the receiver's GRPC client. Without this, goroutines were being leaked on shutdown. 2. Change `grpc.Dial` -> `grpc.NewClient`. They offer the same functionality, but `Dial` is being deprecated in favor of `NewClient`. 3. Enable `goleak` checks on this receiver to help ensure no goroutines are being leaked. 4. Change a couple `Assert.Nil` calls to `Assert.NoError`. The output of `NoError` includes the error message if hit, `Nil` simply includes the object's address, i.e. `&status.Error{s:(*status.Status)(0xc00007e158)}` **Link to tracking Issue:** <Issue number if applicable> open-telemetry#30438 **Testing:** <Describe what testing was performed and which tests were added.> All existing tests are passing, as well as added goleak check.

…ry#32855) Admittedly I tested this via docker-compose. I can expand to having a full docker-compose file with telemetry gen if that's preferred. --------- Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>

…source (open-telemetry#32687) **Description:** <Describe what has changed.> Removed the `vcenter.cluster.name` resource attribute from all Datastore resources. **Link to tracking Issue:** <Issue number if applicable> open-telemetry#32674 **Testing:** <Describe what testing was performed and which tests were added.> Unit/integration tests updated and tested. Local environment tested. **Documentation:** <Describe the documentation added.> New documentation generated based on the metadata. --------- Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com> Co-authored-by: Curtis Robert <crobert@splunk.com>

…telemetry#32930) **Description:** <Describe what has changed.> Adds new default disabled (with Warning log for default enabled on next release) metric `vcenter.vm.network.packet.drop.rate` for Virtual Machines. This metric makes use of the `droppedRx` and `droppedTx` Network performance metrics detailed [here](https://vdc-repo.vmware.com/vmwb-repository/dcr-public/d1902b0e-d479-46bf-8ac9-cee0e31e8ec0/07ce8dbd-db48-4261-9b8f-c6d3ad8ba472/network_counters.html) for Virtual machines. This would use the same metric attributes as the other VM packet metrics and closely match `vcenter.vm.network.packet.rate` in every other way. **Link to tracking Issue:** <Issue number if applicable> open-telemetry#32929 **Testing:** <Describe what testing was performed and which tests were added.> Unit/integration tests updated and tested. Local environment tested. **Documentation:** <Describe the documentation added.> New documentation generated based on the metadata.

…#32481) bug: The remotetapprocessor `limit` configure doesn't work. how to fix: use `time/rate` to limit traffic. Resolves open-telemetry#32385 --------- Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

…0 Release (open-telemetry#32913) **Description:** <Describe what has changed.> A number of configurations were disabled by default and had warnings that they were going to be enabled in v0.101.0 (1 metric had a warning that it was going to be removed). Now that v0.100.0 has been release, I have removed all of these warnings, and made the modifications that the warnings "warned" about. I have also updated the tests to reflect this. **Link to tracking Issue:** <Issue number if applicable> open-telemetry#32803 open-telemetry#32805 open-telemetry#32821 open-telemetry#32531 open-telemetry#32557 **Testing:** <Describe what testing was performed and which tests were added.> Unit/integration tests updated and tested. Local environment tested. **Documentation:** <Describe the documentation added.> New documentation generated based on the metadata.

…telemetry#32946) See open-telemetry#32715 This also adds a bit more debugging info for other tests which fail on the same expectation, since it's not very obvious what was expected vs actually found.

**Description:** <Describe what has changed.>  I found in open-telemetry#32948 that the label `Run ARM` has been added, but the `build-and-test-arm / arm-unittest-matrix (pull_request) ` workflow is still skipped. This is because the `label` action does not trigger a retry. From [documentation](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request): ``` if no activity types are specified, the workflow runs when a pull request is opened or reopened or when the head branch of the pull request is updated. ``` We need to specify that labelling issues should trigger the workflow to check to see if it needs to run again. I've copied the added section from the Windows workflow. I also added that we should only run on PRs against `main`. **Testing:** This PR shows it's working as it should now. Arm test was [originally skipped](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/9009218559/job/24753003216?pr=32955), but after adding the label, tests [have started](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/9009223570/job/24753017935?pr=32955)

…ry#32956) **Description:** <Describe what has changed.>  The readme for the Splunk Enterprise receiver does not currently have the autogenerated header. This was missing because `mdatagen` requires `` and `` to know where to insert the generated data.

…-docappender (open-telemetry#32359) **Description:** <Describe what has changed.>  Replace go-elasticsearch BulkIndexer with go-docappender BulkIndexer for Flush function in preparation for reliability fixes. Maintain similar interface and implementation to go-elasticsearch BulkIndexer. Further changes to expose individual `docappender.BulkIndexer` instances are needed down the road but it is out of the scope of this PR. Implications of this change: - flush timeout is now enforced on client side - oversize payload special handling is now removed - go-docappender uses bulk request filterPath which means bulk response is smaller, less JSON parsing and lower CPU usage - document level retry debug logging is removed as retries are done transparently ~~Blocked by open-telemetry#32585~~ **Link to tracking Issue:** <Issue number if applicable> Fixes open-telemetry#32378 **Testing:** Integration test is passing --------- Co-authored-by: Vishal Raj <vishal.raj@elastic.co>

…ntil the root cause is confirmed (open-telemetry#32937) **Description:** Remove flaky test around forcing collector re-registration until the root cause is confirmed **Link to tracking Issue:** open-telemetry#32785 **Testing:** Unit tests

…emetry#32795) **Description:** Graduate splunkenterprise receiver component to alpha **Link to tracking Issue:** **Testing:** Performed `make otelcontribcol` and ran resulting binary with the following config: ```yaml extensions: basicauth/indexer: client_auth: username: admin password: securityFirst basicauth/cluster_master: client_auth: username: admin password: securityFirst receivers: splunkenterprise: indexer: auth: authenticator: basicauth/indexer endpoint: "https://localhost:8089/" timeout: 45s cluster_master: auth: authenticator: basicauth/cluster_master endpoint: "https://localhost:8089/" timeout: 45s exporters: otlp: endpoint: 127.0.0.1:8000 service: extensions: [basicauth/indexer, basicauth/cluster_master] pipelines: metrics: receivers: [splunkenterprise] exporters: [otlp] ``` and received the following output: ``` sh ~> ./otelcontribcol_linux_amd64 --config=file:config.yaml 2024-05-08T17:34:33.032-0500 info service@v0.100.0/service.go:102 Setting up own telemetry... 2024-05-08T17:34:33.032-0500 info service@v0.100.0/telemetry.go:103 Serving metrics {"address": ":8888", "level": "Normal"} 2024-05-08T17:34:33.032-0500 info receiver@v0.100.0/receiver.go:310 Development component. May change in the future. {"kind": "receiver", "name": "splunkenterprise", "data_type": "metrics"} 2024-05-08T17:34:33.033-0500 info service@v0.100.0/service.go:169 Starting otelcontribcol... {"Version": "0.100.0-dev", "NumCPU": 16} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:34 Starting extensions... 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:37 Extension is starting... {"kind": "extension", "name": "basicauth/cluster_master"} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:52 Extension started. {"kind": "extension", "name": "basicauth/cluster_master"} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:37 Extension is starting... {"kind": "extension", "name": "basicauth/indexer"} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:52 Extension started. {"kind": "extension", "name": "basicauth/indexer"} 2024-05-08T17:34:33.033-0500 info service@v0.100.0/service.go:195 Everything is ready. Begin running and processing data. ``` indicating that the collector was able to successfully start with the component configured. **Documentation:** Documentation was updated to indicate change in status from development to alpha --------- Co-authored-by: Curtis Robert <crobert@splunk.com>

…ry#32479) **Description:** Added support for default timezone in Time converter. Timezone is optional and can be specified as so: `Time("2023-05-26 12:34:56", "%Y-%m-%d %H:%M:%S", "America/New_York")` **Link to tracking Issue:** open-telemetry#32140 **Testing:** Unit tests added **Documentation:** Documentation in ottl/Readme updated --------- Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com> Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>

**Description:** Add global conditions with where clause  **Link to tracking Issue:** Fixes open-telemetry#27830 **Testing:** Unit tests **Documentation:** TODO ~~The main objective is to extend the `ContextStatements` struct by adding a new `Conditions` parameter. By introducing `Conditions` to `ContextStatements`, we can now apply a global condition to all related statements in `WithStatementSequenceGlobalConditions` function.~~ Thanks in advance for your feedback! If this changes will be fine, I will add common where clause into another context `span`, `metrics`.

**Description:** <Describe what has changed.> In open-telemetry/opentelemetry-collector#10069, I am making Type an interface. This means the zero value of Type will be nil - which will cause this test to fail. Initializing ID instead of relying on the zero value fixes this --------- Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com> Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>

**Description:** Added a new component to the testbed called DataConnectors which allows connectors to be added to the testbed config. Also, added a sample connector correctness test using the routingconnector as an example of the usage. **Link to tracking Issue:** open-telemetry#30165 **Testing:** Sample correctness test using routingconnector. **Documentation:** Will update the testbed README with new addition. --------- Co-authored-by: Bryan Aguilar <bryaag@amazon.com>

…emetry#32957) **Description:** <Describe what has changed.>  This is two main changes: 1. Remove `goleak` section. It's now added by default by mdatagen, there's nothing required of users here. 2. Add information and reformat the `Last steps` section of adding a new component. - Move the directions to be bullet points - Add `make generate` to make sure the component's README is updated properly - Explicitly point out that stability and distribution needs to be updated in `metadata.yaml` - Add step for adding the component to the releases repo. My understanding is that if a component is `Alpha`, it should be included in the release, so I've made that a noted part of the steps here. (Let me know if we should word this in a more "optional" way.) --------- Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>

…pen-telemetry#32951) **Description:** <Describe what has changed.> * Breaks our the custom message interface to a separate module, so other components can use the interface without needing to import the `opampextension` module in its entirety. We could temporarily alias the old methods if we'd like, but I think that the CustomMessage stuff has been so short lived that, in addition to the alpha status of the opampextension component, it feels justified to just skip the deprecation process and move it to a new module. **Link to tracking Issue:** Closes open-telemetry#32950 **Testing:** * Covered by existing unit tests **Documentation:** * Added more documentation on usage in the new module. * Modified opampextension docs to point to the new module.

…ape targets (open-telemetry#33298) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

… and metrics exporter (open-telemetry#33291) `TraceAgent` is only called in the traces exporter and `consumeStatsPayload` is only called in the metrics exporter, so I think using the same `wg` field was causing conflicts with both exporters in use at the same time. This PR splits `wg` into separate fields for each use case. This PR also removes `wg.Wait` on the logs exporter since that wasn't applicable in the first place.

**Description:** <Describe what has changed.>  - Adds the shutdown scraper method. The implementation cancels the `containerEventLoop` go routine. Similar to the dockerstats: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/dockerstatsreceiver/receiver.go#L76 - Moves the configuration validation to the start method, in alignment with the dockerstats receiver: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/dockerstatsreceiver/receiver.go#L55 **Link to tracking Issue:** Related issue open-telemetry#29994 **Testing:** <Describe what testing was performed and which tests were added.> Use scraper instead of metrics receiver interface. **Documentation:** <Describe the documentation added.> --------- Co-authored-by: Curtis Robert <crobert@splunk.com>

…ic (open-telemetry#32295) **Description:** <Describe what has changed.>  At the moment. We calculate the `k8s.container.cpu_limit_utilization` as [a ratio of the container's limits](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/kubeletstatsreceiver/documentation.md#k8scontainercpu_limit_utilization) at https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/867d6700c31446172e6998e602c55fbf7351831f/receiver/kubeletstatsreceiver/internal/kubelet/cpu.go#L30. Similarly we can calculate the cpu utilization as ratio of the whole node's allocatable cpu, if we divide by the total number of node's cores. We can retrieve this information from the Node's `Status.Capacity`, for example: ```console $ k get nodes kind-control-plane -ojsonpath='{.status.capacity}' {"cpu":"8","ephemeral-storage":"485961008Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"32564732Ki","pods":"110"} ``` ## Performance concerns In order to get the Node's capacity we need an API call to the k8s API in order to get the Node object. Something to consider here is the performance impact that this extra API call would bring. We can always choose to have this metric as disabled by default and clearly specify in the docs that this metric comes with an extra API call to get the Node of the Pods. The good thing is that `kubeletstats` receiver target's only one node so I believe it's a safe assumption to only fetch the current node because all the observed Pods will belong to the one single local node. Correct me if I miss anything here. In addition, instead of performing the API call explicitly on every single `scrape` we can use an informer instead and leverage its cache. I can change this patch to this direction if we agree on this. Would love to hear other's opinions on this. ## Todos ✅ 1) Apply this change behind a feature gate as it was indicated at open-telemetry#27885 (comment) ✅ 2) Use an Informer instead of direct API calls. **Link to tracking Issue:** <Issue number if applicable> ref: open-telemetry#27885 **Testing:** <Describe what testing was performed and which tests were added.> I experimented with this approach and the results look correct. In order to verify this I deployed a stress Pod on my machine to consume a target cpu of 4 cores: ```yaml apiVersion: v1 kind: Pod metadata: name: cpu-stress spec: containers: - name: cpu-stress image: polinux/stress command: ["stress"] args: ["-c", "4"] ``` And then the collected `container.cpu.utilization` for that Pod's container was at `0,5` as exepcted, based that my machine-node comes with 8 cores in total: ![cpu-stress](https://github.com/open-telemetry/opentelemetry-collector-contrib/assets/11754898/3abe4a0d-6c99-4b4e-a704-da5789dde01b) Unit test is also included. **Documentation:** <Describe the documentation added.> Added: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/32295/files#diff-8ad3b506fb1132c961e8da99b677abd31f0108e3f9ed6999dd96ad3297b51e08 --------- Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

…placeUnsafeCharacters` (open-telemetry#33236) **Description:** Removes the feature flag according to schedule. **Link to tracking Issue:** - open-telemetry#3148 **Testing:** No changes. **Documentation:** Replaced the description of the feature gate with the description of the feature.

**Description:** Documentation only change **Documentation:** Small change to fix bad reference docs for splunk

due to undefined map iteration order, this test was flaky Fixes open-telemetry#33094 Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de> Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>

Signed-off-by: Pavol Loffay <p.loffay@gmail.com> Co-authored-by: Sean Marciniak <30928402+MovieStoreGuy@users.noreply.github.com>

) This is a first step towards a [Flatten/Unflatten workaround](open-telemetry#32080) for OTTL's data corruption issue, specifically this would support Unflatten. The workaround was discussed in a recent collector SIG and it sounded like it would be acceptable if available behind a feature gate and only for the transform processor. If this is accepted I'll work on a Flatten utility next, then integrate them into the tranform processor to prove end-to-end. Finally, I'll implement similar features for metrics and traces. This PR adds an internal utility package which simplifies grouping logs by resource and scope. I'm proposing this initially in `internal/pdatautil` but the functionality could eventually be merged into `pkg/pdatautil` if we find it useful elsewhere.

…pen-telemetry#33240) This PR configures internal metrics for the prometheus remote write exporter to use mdatagen. It additionally uses the utility functions to test emitted internal telemetry instead of using mock telemetry Follows open-telemetry#33239 --------- Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>

…ry#33237) **Description:** <Describe what has changed.>  This PR adds the logging part from open-telemetry#31256. With this addition every time that is identified that file is rotated either by move/create or copy/truncate, proper logging takes place. **Link to tracking Issue:** <Issue number if applicable> open-telemetry#31256 **Testing:** <Describe what testing was performed and which tests were added.> Updated existing unit tests ### How to test this manually Using the following config: ```yaml receivers: filelog: start_at: beginning poll_interval: 5s include: - /var/log/busybox/monitoring/stable*.log exporters: debug: verbosity: detailed service: telemetry: logs: level: info pipelines: logs: receivers: [filelog] exporters: [debug] processors: [] ``` #### Testing truncate (out of pattern) ```console echo "$(date '+%FT%H:%M:%S.%NZ') some line1" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line2" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line3" >> /var/log/busybox/monitoring/stable_trunc.log && sleep 6 && cp /var/log/busybox/monitoring/stable_trunc.log /var/log/busybox/monitoring/stable_trunc.log.1 && : > /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line new0" >> /var/log/busybox/monitoring/stable_trunc.log ``` #### Testing truncate (in pattern) ```console echo "$(date '+%FT%H:%M:%S.%NZ') some line1" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line2" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line3" >> /var/log/busybox/monitoring/stable_trunc.log && sleep 6 && cp /var/log/busybox/monitoring/stable_trunc.log /var/log/busybox/monitoring/stable_trunc_1.log && : > /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line new1" >> /var/log/busybox/monitoring/stable_trunc.log ``` #### Testing move/create (out of pattern) ```console echo "$(date '+%FT%H:%M:%S.%NZ') some line1" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line2" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line3" >> /var/log/busybox/monitoring/stable_trunc.log && sleep 6 && mv /var/log/busybox/monitoring/stable_trunc.log /var/log/busybox/monitoring/stable_trunc.log.1 && echo "$(date '+%FT%H:%M:%S.%NZ') some line new0" >> /var/log/busybox/monitoring/stable_trunc.log ``` #### Testing move/create (in pattern) ```console echo "$(date '+%FT%H:%M:%S.%NZ') some line1" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line2" >> /var/log/busybox/monitoring/stable_trunc.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line3" >> /var/log/busybox/monitoring/stable_trunc.log && sleep 6 && mv /var/log/busybox/monitoring/stable_trunc.log /var/log/busybox/monitoring/stable_trunc_1.log && echo "$(date '+%FT%H:%M:%S.%NZ') some line new0" >> /var/log/busybox/monitoring/stable_trunc.log ``` **Documentation:** <Describe the documentation added.> Add some extra notes in the `design.md` --------- Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

…etry#33320) This PR adds support for passing a traceID and spanID to be used as exemplars on the generated data points. This was verified with an e2e test using OTel Collector and the debug exporter in detailed verbosity, resulting in this: ``` 2024-05-31T11:41:51.378+0200 info ResourceMetrics #0 Resource SchemaURL: https://opentelemetry.io/schemas/1.13.0 ScopeMetrics #0 ScopeMetrics SchemaURL: InstrumentationScope Metric #0 Descriptor: -> Name: gen -> Description: -> Unit: -> DataType: Gauge NumberDataPoints #0 StartTimestamp: 1970-01-01 00:00:00 +0000 UTC Timestamp: 2024-05-31 09:41:51.374564535 +0000 UTC Value: 0 Exemplars: Exemplar #0 -> Trace ID: ae87dadd90e9935a4bc9660628efd569 -> Span ID: 5828fa4960140870 -> Timestamp: 2024-05-31 09:41:51.374432029 +0000 UTC -> Value: 1 ``` Given that I needed to do the validation of the traceID/spanID on the metrics config as well, I refactored the current validation done in the logging part of the command so that it's now part of the common package. Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> --------- Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

This file needed regeneration --------- Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com> Co-authored-by: Dan Jaglowski <jaglows3@gmail.com>

Followup to open-telemetry#33311. Adds the ability to flatten logs.

**Description:** Adds integration test for the rabbitmq exporter. It tests the happy path and the logic for creating a new connection when the current one becomes unhealthy

…metry#32282) **Description:** Adds `create_schema` boolean to config. This flag will disable the automatic creation of the database and tables so that this can be manually managed by the user. **Link to tracking Issue:** open-telemetry#29443 (related) **Testing:** Ran integration tests, and added test for verifying undefined + true + false in for the value in the config. **Documentation:** Added config option to README along with new "schema management" section that contains some recommendations for production deployments as well as requirements for maintaining compatibility. Also added `examples/default_ddl/` which contains all of the default DDL that the exporter runs. These are to be used as a starting point for users managing their own schema. --------- Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>

…telemetry#33321) **Description:** <Describe what has changed.> Fixing a bug - the calculation when converting pdata timestamps to `time` is off in both the `syslog` and `tcpudp` datasenders. Seconds are calculated correctly by dividing the timestamp by `10^9`, but in nanosecond calculation the remainder of `timestamp/10^8` is used, effectively always setting the first decimal in the resulting timestamp to `0`. Discovered this while writing some tests against the syslog receiver and receiving unexpected data in the log body, after identifying the issue a quick search revealed that the same thing was happening in the tcp/upd datasender. **Example:** `2024-05-31T11:53:54.612287004+02:00` becomes `2024-05-31T11:53:54.012287004+02:00` **Link to tracking Issue:** no issue as it's a fairly minor thing that only affects tests, but can open if required **Testing:** - manual tests - existing tests **Documentation:** no additional documentation added.

…3267) **Description:** <Describe what has changed.> Redacts primitive values in the effective config reported by the extension. Necessary for open-telemetry#31641 until open-telemetry/opentelemetry-collector#10139 is resolved. Relates to open-telemetry#32983

…-telemetry#33318) **Description:** Replace the `elasticsearchTracesExporter` and `elasticsearchLogsExporter` with a single `elasticsearchExporter` that handles both. Tests have been cleaned up to reduce repetition between traces and logs exporters, e.g. config validation tests have been moved to more specific `TestConfig*` tests. To enable this, some further validation has been added to `Config.Validate`, which was previously done only when reaching the go-elasticsearch code when constructing exporters. Various tests have been updated to use the public APIs rather than internal functions, so we're testing behaviour rather than the implementation; this will enable further refactoring without breaking tests. **Link to tracking Issue:** None **Testing:** Ran the unit tests. This is a non-functional change. **Documentation:** N/A, non-functional change. --------- Co-authored-by: Carson Ip <carsonip@users.noreply.github.com>

**Description:** make update-otel in support of open-telemetry/opentelemetry-collector/issues/10294 **Link to tracking Issue:** open-telemetry/opentelemetry-collector/issues/10294 --------- Signed-off-by: Pablo Baeyens <pablo.baeyens@datadoghq.com>

The following commands were run to prepare this release: - make chlog-update VERSION=v0.102.0 - sed -i.bak s/0.101.0/0.102.0/g versions.yaml - make multimod-prerelease - make multimod-sync

…ntrib/release/v0.102.x' into upstream-merge-0.102.0

renovate bot and others added 30 commits May 8, 2024 10:29

chore: add constants for prometheus translation (open-telemetry#32830)

1139675

**Description:** This is a cleanup to consolidate the constants used for prometheus translation in a single place. --------- Co-authored-by: Curtis Robert <crobert@splunk.com>

Move Aneurysm9 to emeritus status (open-telemetry#32943)

bbcbe3c

I have been unable to provide this position the bandwidth that it deserves and it is time to formalize recognition of that fact. Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

[remotetapprocessor] use 'time/rate' to limit traffic (open-telemetry…

497fed7

…#32481) bug: The remotetapprocessor `limit` configure doesn't work. how to fix: use `time/rate` to limit traffic. Resolves open-telemetry#32385 --------- Co-authored-by: Andrzej Stencel <astencel@sumologic.com>

[chore][fileconsumer] Skip flaky TestFlushPeriodEOF on windows (open-…

b1bbd58

…telemetry#32946) See open-telemetry#32715 This also adds a bit more debugging info for other tests which fail on the same expectation, since it's not very obvious what was expected vs actually found.

atoulme and others added 26 commits May 30, 2024 14:55

[chore] Remove unused stable feature gate (open-telemetry#33305)

51ca100

Allow to configure http client used by target allocator generated scr…

28ca05c

…ape targets (open-telemetry#33298) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

[chore] remove duplicated config line (open-telemetry#33314)

bf7cd57

Fix bad docs for splunk tls config (open-telemetry#33312)

5427c93

**Description:** Documentation only change **Documentation:** Small change to fix bad reference docs for splunk

Zipkin receiver document server options (open-telemetry#33289)

f1a07c9

Signed-off-by: Pavol Loffay <p.loffay@gmail.com> Co-authored-by: Sean Marciniak <30928402+MovieStoreGuy@users.noreply.github.com>

[chore] Remove use of deprecated ConfigProvider (open-telemetry#33323)

6005ae8

[chore] make generate (open-telemetry#33328)

288f5db

This file needed regeneration --------- Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com> Co-authored-by: Dan Jaglowski <jaglows3@gmail.com>

[chore] Adds internal pdatautil.FlattenLogs (open-telemetry#33322)

89bc3f6

Followup to open-telemetry#33311. Adds the ability to flatten logs.

Add integration test for rabbitmq exporter (open-telemetry#32856)

7cc24ff

**Description:** Adds integration test for the rabbitmq exporter. It tests the happy path and the logic for creating a new connection when the current one becomes unhealthy

[chore] Prepare release 0.102.0 (open-telemetry#33362)

6b1d3dd

The following commands were run to prepare this release: - make chlog-update VERSION=v0.102.0 - sed -i.bak s/0.101.0/0.102.0/g versions.yaml - make multimod-prerelease - make multimod-sync

Merge remote-tracking branch 'refs/remotes/opentelemetry-collector-co…

6aeec0e

…ntrib/release/v0.102.x' into upstream-merge-0.102.0

foadnh approved these changes Jun 12, 2024

View reviewed changes

steves-canva mentioned this pull request Jun 14, 2024

Upstream merge 0.100.x #14196

Closed

steves-canva merged commit 66623dd into main Jun 14, 2024
142 of 162 checks passed

steves-canva deleted the upstream-merge-0.102.0 branch June 14, 2024 00:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upstream merge 0.102.0 #14201

Upstream merge 0.102.0 #14201

steves-canva commented Jun 11, 2024

Upstream merge 0.102.0 #14201

Upstream merge 0.102.0 #14201

Conversation

steves-canva commented Jun 11, 2024