fix k8s e2e test

[fatsheep9146]

Description:
related #24223

[fatsheep9146]

captured one error with telemetrygen's log

2023-08-18T05:54:33.996Z	INFO	traces/traces.go:62	starting gRPC exporter
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:477	[core][Channel #1] Channel created	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1786	[core][Channel #1] original dial target is: "otelcol-3190295a:4317"	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1793	[core][Channel #1] parsed dial target is: {URL:{Scheme:otelcol-3190295a Opaque:4317 User: Host: Path: RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1807	[core][Channel #1] fallback to scheme "passthrough"	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1815	[core][Channel #1] parsed dial target is: {URL:{Scheme:passthrough Opaque: User: Host: Path:/otelcol-3190295a:4317 RawPath: OmitHost:false ForceQuery:false RawQu
ery: Fragment: RawFragment:}}	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1948	[core][Channel #1] Channel authority set to "otelcol-3190295a:4317"	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/resolver_conn_wrapper.go:238	[core][Channel #1] Resolver state updated: {
  "Addresses": [
    {
      "Addr": "otelcol-3190295a:4317",
      "ServerName": "",
      "Attributes": null,
      "BalancerAttributes": null,
      "Type": 0,
      "Metadata": null
    }
  ],
  "ServiceConfig": null,
  "Attributes": null
} (resolver returned new addresses)	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/balancer_conn_wrappers.go:192	[core][Channel #1] Channel switches to new LB policy "pick_first"	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/balancer_conn_wrappers.go:312	[core][Channel #1 SubChannel #2] Subchannel created	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33
.996Z	INFO	grpc@v1.57.0/clientconn.go:565	[core][Channel #1] Channel Connectivity change to CONNECTING	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1301	[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.996Z	INFO	grpc@v1.57.0/clientconn.go:1414	[core][Channel #1 SubChannel #2] Subchannel picks a new address "otelcol-3190295a:4317" to connect	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.998Z	INFO	grpc@v1.57.0/clientconn.go:1301	[core][Channel #1 SubChannel #2] Subchannel Connectivity change to READY	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/clientconn.go:565	[core][Channel #1] Channel Connectivity change to READY	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	traces/traces.go:122	generation of traces is limited	{"per-second": 1}
2023-08-18T05:54:33.999Z	ERROR	traces/traces.go:102	failed to execute the test sce
nario.	{"error": "expected `status-code` to be one of (Unset, Error, Ok) or (0, 1, 2), got \"Unset\" instead"}
github.com/open-telemetry/opentelemetry-collector-contrib/cmd/telemetrygen/internal/traces.Start
	/src/internal/traces/traces.go:102
main.glob..func1
	/src/config.go:40
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
main.Execute
	/src/config.go:84
main.main
	/src/main.go:8
runtime.main
	/usr/local/go/src/runtime/proc.go:250
2023-08-18T05:54:33.999Z	INFO	traces/traces.go:80	stop the batch span processor
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/clientconn.go:565	[core][Channel #1] Channel Connectivity change to SHUTDOWN	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/balancer_conn_wrappers.go:203	[core][Channel #1] ccB
alancerWrapper: closing	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/resolver_conn_wrapper.go:126	[core][Channel #1] Closing the name resolver	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/clientconn.go:1301	[core][Channel #1 SubChannel #2] Subchannel Connectivity change to SHUTDOWN	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/clientconn.go:1249	[core][Channel #1 SubChannel #2] Subchannel deleted	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	grpc@v1.57.0/clientconn.go:1251	[core][Channel #1] Channel deleted	{"system": "grpc", "grpc_log": true}
2023-08-18T05:54:33.999Z	INFO	traces/traces.go:70	stopping the exporter
Error: expected `status-code` to be one of (Unset, Error, Ok) or (0, 1, 2), got "Unset" instead
Usage:
  telemetrygen traces [flags]

Examples:
telemetrygen traces

Flags:
      --batch                               Whether to batch traces (default true)
      --duration durat
ion                   For how long to run the test
  -h, --help                                help for traces
      --interval duration                   Reporting interval (default 1 second) (default 1s)
      --marshal                             Whether to marshal trace context via HTTP headers
      --otlp-attributes map[string]string   Custom resource attributes to use. The value is expected in the format key="value".Flag may be repeated to set multiple attributes (e.g -otlp-attributes key1="value1" -otlp-attributes key2="value2")
      --otlp-endpoint string                Target to which the exporter is going to send metrics. (default "localhost:4317")
      --otlp-header map[string]string       Custom header to be passed along with each OTLP request. The value is expected in the format key=value.Flag may be repeated to set multiple headers (e.g -otlp-header key1=value1 -otlp-header key2=value2)
      --otlp-http                           Whether to use HTTP exporter rather than a gRPC one
      --otl
p-http-url-path string           Which URL path to write to (default /v1/metrics) (default "/v1/metrics")
      --otlp-insecure                       Whether to enable client transport security for the exporter's grpc or http connection
      --rate int                            Approximately how many metrics per second each worker should generate. Zero means no throttling.
      --service string                      Service name to use (default "telemetrygen")
      --size int                            Desired minimum size in MB of string data for each trace generated. This can be used to test traces with large payloads, i.e. when testing the OTLP receiver endpoint max receive size.
      --status-code string                  Status code to use for the spans, one of (Unset, Error, Ok) or the equivalent integer (0,1,2) (default "Unset")
      --traces int                          Number of traces to generate in each worker (ignored if duration is provided) (default 1)
      --workers int                    
     Number of workers (goroutines) to run (default 1)


panic: EOF

goroutine 74 [running]:
github.com/open-telemetry/opentelemetry-collector-contrib/internal/k8stest.WaitForTelemetryGenToStart.func1()
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/k8stest/k8s_telemetrygen.go:101 +0x756
github.com/stretchr/testify/assert.Eventually.func1()
	/home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.4/assert/assertions.go:1852 +0x29
created by github.com/stretchr/testify/assert.Eventually
	/home/runner/go/pkg/mod/github.com/stretchr/testify@v1.8.4/assert/assertions.go:1852 +0x22a
exit status 2

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.