Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[correctnesstests/tracing] Flaky test: TestTracingGoldenData/opencensus-zipkin #34707

Open
crobert-1 opened this issue Aug 15, 2024 · 7 comments
Open

[correctnesstests/tracing] Flaky test: TestTracingGoldenData/opencensus-zipkin #34707

crobert-1 opened this issue Aug 15, 2024 · 7 comments
Labels
exporter/zipkin flaky test a test is flaky help wanted Extra attention is needed receiver/opencensus

Comments

@crobert-1
Copy link
Member

Component(s)

No response

Describe the issue you're reporting

Failing CI/CD link

Failure output (too long to paste all of it here):

panic: test timed out after 10m0s
running tests:
	TestTracingGoldenData (10m0s)
	TestTracingGoldenData/opencensus-zipkin (9m39s)
@crobert-1
Copy link
Member Author

Potentially related: #33715

@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners for receiver/opencensus: @open-telemetry/collector-approvers. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners for exporter/zipkin: @MovieStoreGuy @andrzej-stencel @crobert-1. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1 crobert-1 mentioned this issue Aug 15, 2024
Merged
@github-actions github-actions bot mentioned this issue Aug 20, 2024
Closed
This was referenced Aug 27, 2024
Closed
Closed
@songy23
Copy link
Member

songy23 commented Sep 3, 2024

+1 https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10684195286/job/29614311127?pr=34972

This was referenced Sep 10, 2024
Closed
Closed
This was referenced Sep 24, 2024
Closed
Closed
@github-actions github-actions bot mentioned this issue Oct 8, 2024
Closed
@atoulme atoulme added flaky test a test is flaky and removed needs triage New item requiring triage labels Oct 12, 2024
@dmitryax
Copy link
Member

One more https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11788131851/job/32834719070?pr=36297

@dmitryax dmitryax added the help wanted Extra attention is needed label Nov 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Jan 13, 2025
@andrzej-stencel
Copy link
Member

Here's the relevant part of the logs from https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11788131851/job/32834719070?pr=36297:

Log output 
=== RUN   TestTracingGoldenData/opencensus-zipkin
2024/11/12 00:09:41 
receivers:
  opencensus:
    endpoint: "127.0.0.1:37473"
exporters:
  zipkin:
    endpoint: http://127.0.0.1:43339/api/v2/spans
    format: json
processors:
  
  batch:
    send_batch_size: 1024



extensions:

service:
  telemetry:
    metrics:
      address: 127.0.0.1:41779
  extensions:
  pipelines:
    traces:
      receivers: [opencensus]
      processors: [batch]
      exporters: [zipkin]

2024/11/12 00:09:41 Starting mock backend...
2024-11-12T00:09:41.791Z	info	service@v0.113.0/service.go:166	Setting up own telemetry...
2024-11-12T00:09:41.791Z	warn	service@v0.113.0/service.go:221	service::telemetry::metrics::address is being deprecated in favor of service::telemetry::metrics::readers
2024-11-12T00:09:41.791Z	info	telemetry/metrics.go:70	Serving metrics	{"address": "127.0.0.1:41779", "metrics level": "Normal"}
2024-11-12T00:09:41.791Z	info	service@v0.113.0/service.go:238	Starting otelcol...	{"Version": "latest", "NumCPU": 4}
2024-11-12T00:09:41.791Z	info	extensions/extensions.go:39	Starting extensions...
2024-11-12T00:09:41.792Z	info	service@v0.113.0/service.go:261	Everything is ready. Begin running and processing data.
2024/11/12 00:09:42 Starting load generator at 1024 items/sec.
2024/11/12 00:09:44  | Sent:     3,907 traces (1,953/sec) | Received:     3,907 items (1,302/sec)
2024/11/12 00:09:44 Stopped generator. Sent:     3,907 traces (1,952/sec)
2024-11-12T00:09:44.791Z	info	otelcol@v0.113.0/collector.go:336	Received shutdown request
2024-11-12T00:09:44.791Z	info	service@v0.113.0/service.go:303	Starting shutdown...
2024/11/12 00:09:47  | Sent:     3,907 traces (781/sec) | Received:     3,907 items (651/sec)
2024/11/12 00:09:50  | Sent:     3,907 traces (488/sec) | Received:     3,907 items (434/sec)
2024/11/12 00:09:53  | Sent:     3,907 traces (355/sec) | Received:     3,907 items (325/sec)
2024/11/12 00:09:56  | Sent:     3,907 traces (279/sec) | Received:     3,907 items (260/sec)
2024/11/12 00:09:59  | Sent:     3,907 traces (229/sec) | Received:     3,907 items (217/sec)
2024/11/12 00:10:02  | Sent:     3,907 traces (195/sec) | Received:     3,907 items (186/sec)
2024/11/12 00:10:05  | Sent:     3,907 traces (169/sec) | Received:     3,907 items (162/sec)
2024/11/12 00:10:08  | Sent:     3,907 traces (150/sec) | Received:     3,907 items (144/sec)

... LOG OUTPUT TRUNCATED ...

2024/11/12 00:19:11  | Sent:     3,907 traces (6/sec) | Received:     3,907 items (6/sec)
2024/11/12 00:19:14  | Sent:     3,907 traces (6/sec) | Received:     3,907 items (6/sec)
2024/11/12 00:19:17  | Sent:     3,907 traces (6/sec) | Received:     3,907 items (6/sec)
panic: test timed out after 10m0s
running tests:
	TestTracingGoldenData (10m0s)
	TestTracingGoldenData/opencensus-zipkin (9m39s)

goroutine 846 [running]:
testing.(*M).startAlarm.func1()
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/opt/hostedtoolcache/go/1.22.8/x64/src/time/sleep.go:177 +0x2d

goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0xc001205520, {0x40b0926?, 0x0?}, 0x4234d78)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1750 +0x3ab
testing.runTests.func1(0xc001205520)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:2161 +0x37
testing.tRunner(0xc001205520, 0xc00107fbc0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1689 +0xfb
testing.runTests(0xc001760810, {0x6b17d00, 0x2, 0x2}, {0x10d?, 0x3?, 0x6c9c3c0?})
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:2159 +0x445
testing.(*M).Run(0xc00176c500)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:2027 +0x68b
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.DoTestMain(0xc00176c500, {0x49b1d60, 0x6c9b6a0})
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/test_bed.go:72 +0xef
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/traces.TestMain(...)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/traces/correctness_test.go:22
main.main()
	_testmain.go:51 +0x1a5

goroutine 23 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc001128180)
	/home/runner/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:292 +0x9f
created by go.opencensus.io/stats/view.init.0 in goroutine 1
	/home/runner/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x8d

goroutine 24 [sync.Cond.Wait, 10 minutes]:
sync.runtime_notifyListWait(0xc0011a5390, 0x0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0x0?)
	/opt/hostedtoolcache/go/1.22.8/x64/src/sync/cond.go:70 +0x85
github.com/cihub/seelog.(*asyncLoopLogger).processItem(0xc0011c02d0)
	/home/runner/go/pkg/mod/github.com/cihub/seelog@v0.0.0-20170130134532-f561c5e57575/behavior_asynclooplogger.go:50 +0x99
github.com/cihub/seelog.(*asyncLoopLogger).processQueue(0xc0011c02d0)
	/home/runner/go/pkg/mod/github.com/cihub/seelog@v0.0.0-20170130134532-f561c5e57575/behavior_asynclooplogger.go:63 +0x33
created by github.com/cihub/seelog.NewAsyncLoopLogger in goroutine 1
	/home/runner/go/pkg/mod/github.com/cihub/seelog@v0.0.0-20170130134532-f561c5e57575/behavior_asynclooplogger.go:40 +0xcf

goroutine 25 [sync.Cond.Wait, 10 minutes]:
sync.runtime_notifyListWait(0xc0011a5510, 0x0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0x0?)
	/opt/hostedtoolcache/go/1.22.8/x64/src/sync/cond.go:70 +0x85
github.com/cihub/seelog.(*asyncLoopLogger).processItem(0xc0011c03f0)
	/home/runner/go/pkg/mod/github.com/cihub/seelog@v0.0.0-20170130134532-f561c5e57575/behavior_asynclooplogger.go:50 +0x99
github.com/cihub/seelog.(*asyncLoopLogger).processQueue(0xc0011c03f0)
	/home/runner/go/pkg/mod/github.com/cihub/seelog@v0.0.0-20170130134532-f561c5e57575/behavior_asynclooplogger.go:63 +0x33
created by github.com/cihub/seelog.NewAsyncLoopLogger in goroutine 1
	/home/runner/go/pkg/mod/github.com/cihub/seelog@v0.0.0-20170130134532-f561c5e57575/behavior_asynclooplogger.go:40 +0xcf

goroutine 760 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7ff605bc6ea0, 0x72)
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/netpoll.go:345 +0x85
internal/poll.(*pollDesc).wait(0x8?, 0xb03?, 0x0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.22.8/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc001e26880)
	/opt/hostedtoolcache/go/1.22.8/x64/src/internal/poll/fd_unix.go:611 +0x2ac
net.(*netFD).accept(0xc001e26880)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc0071e4ae0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc0071e4ae0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/tcpsock.go:327 +0x30
net/http.(*Server).Serve(0xc00518c1e0, {0x49b2cf0, 0xc0071e4ae0})
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:3260 +0x33e
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinreceiver.(*zipkinReceiver).Start.func1()
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/zipkinreceiver/trace_receiver.go:110 +0x76
created by github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinreceiver.(*zipkinReceiver).Start in goroutine 757
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/zipkinreceiver/trace_receiver.go:107 +0x23c

goroutine 31 [chan receive, 9 minutes]:
testing.(*T).Run(0xc0012056c0, {0xc0057414a0?, 0xc0000b6f38?}, 0xc0071e3c50)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1750 +0x3ab
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/traces.TestTracingGoldenData(0xc0012056c0)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/traces/correctness_test.go:41 +0xda
testing.tRunner(0xc0012056c0, 0x4234d78)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 1
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1742 +0x390

goroutine 757 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc00562a840?)
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc0057414d0?)
	/opt/hostedtoolcache/go/1.22.8/x64/src/sync/waitgroup.go:116 +0x48
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*inProcessCollector).Stop(0xc0044fa900)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/in_process_collector.go:104 +0x4f
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*TestCase).StopAgent(0xc005789180)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/test_case.go:165 +0x24
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/traces.testWithTracingGoldenDataset(0xc00579a340, {0x7ff604664f28, 0xc0071e39b0}, {0x49cb380, 0xc0071ee2a0}, {0x2492f60?, 0xc0?, 0x54f213?, 0x4a176f?}, {0xc001762b40, ...})
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/traces/correctness_test.go:95 +0x77b
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/traces.TestTracingGoldenData.func1(0xc00579a340?)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/traces/correctness_test.go:42 +0x54
testing.tRunner(0xc00579a340, 0xc0071e3c50)
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 31
	/opt/hostedtoolcache/go/1.22.8/x64/src/testing/testing.go:1742 +0x390

goroutine 641 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc004784950, {0x49cb1c0, 0xc003539630})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 639
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 525 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc005413070, {0x49cb1c0, 0xc0026ac140})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 524
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 271 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0018c8100, {0x49cb1c0, 0xc002eecdc0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 268
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 774 [select]:
go.opentelemetry.io/collector/processor/batchprocessor.(*shard[...]).startLoop(0x49e8e20)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.113.0/batch_processor.go:199 +0x145
created by go.opentelemetry.io/collector/processor/batchprocessor.(*shard[...]).start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.113.0/batch_processor.go:185 +0x8d

goroutine 189 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc004bcb410, {0x49cb1c0, 0xc004b7caf0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 187
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 190 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc004bcb490, {0x49cb1c0, 0xc004b7cbe0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 187
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 270 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0018c80d0, {0x49cb1c0, 0xc002eecd70})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 268
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 188 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc004bcb3e0, {0x49cb1c0, 0xc004b7ca50})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 187
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 765 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc00398e360, 0xc0017def28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc63807?, {0x12cd340?, 0xc00722b2c0?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 794 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc007506e30, {0x49cb1c0, 0xc00750c3c0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 791
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 508 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc003a907a0, {0x49cb1c0, 0xc003a8c140})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 507
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 771 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc00131d748, 0xc00131d728?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc00389e150?, {0x0?, 0x6d0f120?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 76 [syscall, 10 minutes]:
os/signal.signal_recv()
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/sigqueue.go:152 +0x29
os/signal.loop()
	/opt/hostedtoolcache/go/1.22.8/x64/src/os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 66
	/opt/hostedtoolcache/go/1.22.8/x64/src/os/signal/signal.go:151 +0x1f

goroutine 792 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc007506dd0, {0x49cb1c0, 0xc00750c320})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 791
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 509 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc003a907d0, {0x49cb1c0, 0xc003a8c190})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 507
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 777 [chan send, 9 minutes]:
go.opentelemetry.io/collector/service/internal/graph.(*Host).NotifyComponentStatusChange(...)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/graph/host.go:83
go.opentelemetry.io/collector/service/internal/status.(*reporter).componentFSM.func1(0x37b1a80?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/status/status.go:149 +0x25
go.opentelemetry.io/collector/service/internal/status.(*fsm).transition(0xc00747e5a0, 0xc007237290)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/status/status.go:40 +0x8b
go.opentelemetry.io/collector/service/internal/status.(*reporter).ReportStatus(0xc007464ca0, 0x3afa4e0?, 0xc007237290)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/status/status.go:129 +0x8f
go.opentelemetry.io/collector/service/internal/graph.(*HostWrapper).Report(0x6aebb10?, 0x3bd60e0?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/graph/graph.go:616 +0x32
go.opentelemetry.io/collector/component/componentstatus.ReportStatus(...)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/component/componentstatus@v0.113.0/status.go:158
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusreceiver.(*ocReceiver).Start.func3()
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/opencensusreceiver/opencensus.go:162 +0x176
created by github.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusreceiver.(*ocReceiver).Start in goroutine 761
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/opencensusreceiver/opencensus.go:159 +0xa85

goroutine 269 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0018c80a0, {0x49cb1c0, 0xc002eecd20})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 268
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 772 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc002495f40, 0xc002495f28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc00389e150?, {0x0?, 0x106d0f120?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 550 [select, 9 minutes]:
google.golang.org/grpc.newClientStreamWithParams.func4()
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:393 +0x8c
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 512
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:392 +0xe08

goroutine 793 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc007506e00, {0x49cb1c0, 0xc00750c370})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 791
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 773 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc000879748, 0xc001341f28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0x0?, {0x0?, 0xc00722b2c0?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 763 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7ff605bc6bb8, 0x72)
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/netpoll.go:345 +0x85
internal/poll.(*pollDesc).wait(0xa?, 0x10?, 0x0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.22.8/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc00288db80)
	/opt/hostedtoolcache/go/1.22.8/x64/src/internal/poll/fd_unix.go:611 +0x2ac
net.(*netFD).accept(0xc00288db80)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc002299540)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc002299540)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/tcpsock.go:327 +0x30
net/http.(*Server).Serve(0xc00518c780, {0x49b2cf0, 0xc002299540})
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:3260 +0x33e
net/http.(*Server).ListenAndServe(0xc00518c780)
	/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:3189 +0x71
go.opentelemetry.io/collector/service/telemetry/internal/otelinit.InitPrometheusServer.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/telemetry/internal/otelinit/config.go:108 +0x74
created by go.opentelemetry.io/collector/service/telemetry/internal/otelinit.InitPrometheusServer in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/telemetry/internal/otelinit/config.go:106 +0x245

goroutine 551 [select, 9 minutes]:
google.golang.org/grpc.newClientStreamWithParams.func4()
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:393 +0x8c
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 512
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:392 +0xe08

goroutine 766 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0x6bd376, 0xc0017daf28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc63807?, {0xa42693?, 0xc00722b2c0?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 733 [select, 9 minutes]:
google.golang.org/grpc.newClientStreamWithParams.func4()
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:393 +0x8c
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 796
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:392 +0xe08

goroutine 761 [sync.Mutex.Lock, 9 minutes]:
sync.runtime_SemacquireMutex(0xc005095928?, 0x6c?, 0x7ff60494be40?)
	/opt/hostedtoolcache/go/1.22.8/x64/src/runtime/sema.go:77 +0x25
sync.(*Mutex).lockSlow(0xc007464ca0)
	/opt/hostedtoolcache/go/1.22.8/x64/src/sync/mutex.go:171 +0x15d
sync.(*Mutex).Lock(...)
	/opt/hostedtoolcache/go/1.22.8/x64/src/sync/mutex.go:90
go.opentelemetry.io/collector/service/internal/status.(*reporter).ReportStatus(0xc007464ca0, 0xc00746b940, 0xc007541320)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/status/status.go:126 +0x4c
go.opentelemetry.io/collector/service/internal/graph.(*Graph).ShutdownAll(0xc00721b140, {0x49cb020, 0x6d0f120}, {0x499def0, 0xc007464ca0})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/internal/graph/graph.go:474 +0x326
go.opentelemetry.io/collector/service.(*Service).Shutdown(0xc00722b040, {0x49cb020, 0x6d0f120})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/service@v0.113.0/service.go:309 +0xe5
go.opentelemetry.io/collector/otelcol.(*Collector).shutdown(0xc0071f0b40, {0x49cb020, 0x6d0f120})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/otelcol@v0.113.0/collector.go:358 +0xc5
go.opentelemetry.io/collector/otelcol.(*Collector).Run(0xc0071f0b40, {0x49cb020, 0x6d0f120})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/otelcol@v0.113.0/collector.go:344 +0x9b0
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*inProcessCollector).Start.func2()
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/in_process_collector.go:80 +0x68
created by github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*inProcessCollector).Start in goroutine 757
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/in_process_collector.go:78 +0x3a5

goroutine 770 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0x3738616638383037, 0xc001eb0f28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0x3738616638383037?, {0x0?, 0xc001eb0fd0?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 769 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc002494748, 0xc002494728?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0x0?, {0x0?, 0xc002494790?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 690 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc004784980, {0x49cb1c0, 0xc003539680})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 639
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 640 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc004784920, {0x49cb1c0, 0xc0035395e0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 639
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 764 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0x0, 0xc00133cf28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc63807?, {0xc0052fe260?, 0xc00722b2c0?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 759 [select]:
github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*TestCase).logStats(0xc005789180)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/test_case.go:303 +0xbc
created by github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.NewLoadGeneratorTestCase in goroutine 757
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/test_case.go:126 +0x767

goroutine 767 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc001ef4340, 0xc002494f28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc00529fb60?, {0xc002494f98?, 0xc003aa6900?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 510 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc003a90800, {0x49cb1c0, 0xc003a8c1e0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 507
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 768 [chan receive, 9 minutes]:
go.opentelemetry.io/collector/exporter/internal/queue.(*sizedChannel[...]).pop(0xc004097198, 0xc002493f28?)
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/sized_channel.go:75 +0x48
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Read(0xc002493ee8?, {0xc002493fd0?, 0x11dcfaf?})
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/bounded_memory_queue.go:44 +0x45
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1()
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:39 +0xcc
created by go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start in goroutine 761
	/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.113.0/internal/queue/consumers.go:35 +0x45

goroutine 732 [select, 9 minutes]:
google.golang.org/grpc.newClientStreamWithParams.func4()
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:393 +0x8c
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 796
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/stream.go:392 +0xe08

goroutine 527 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0054130d0, {0x49cb1c0, 0xc0026ac1e0})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 524
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 526 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0054130a0, {0x49cb1c0, 0xc0026ac190})
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:88 +0x115
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 524
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.67.1/internal/grpcsync/callback_serializer.go:52 +0x11a
exit status 2
FAIL	github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/traces	600.048s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
exporter/zipkin flaky test a test is flaky help wanted Extra attention is needed receiver/opencensus
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@atoulme @dmitryax @songy23 @andrzej-stencel @crobert-1