OTLP metrics exporter get 404 response from collector

[CSGkeramidas]

When beyla is configured to send data to a collector otlp endpoint, the metrics endpoint responds with 404.

failed to upload metrics: failed to send metrics to http://metrics-collector-collector.monitoring.svc.cluster.local:4318/v1/metrics: 404 Not Found

I believe this is due to the fact that beyla does not set the correct headers when sending the metrics to collector.
You can find a similar case here: open-telemetry/opentelemetry-collector#3540

[mariomac]

Thank you for contacting us @CSGkeramidas. This bug could be a duplicate of #291

As a workaround while it is fixed, could you check whether providing the OTEL_EXPORTER_OTLP_METRICS_HEADERS environment variable allows you sending data to the OTEL endpoint?

[mariomac]

Also please try providing the OTEL_EXPORTER_OTLP_PROTOCOL value if the above does not work.

[jpkrohling]

Here's a config that works for me:

OTel Collector config:

receivers:
    otlp:
      protocols:
        http:
        grpc:

exporters:
  logging:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [logging]
    metrics:
      receivers: [otlp]
      exporters: [logging]

Beyla:

> set -xg OTEL_EXPORTER_OTLP_ENDPOINT http://localhost:4318
> set -xg OPEN_PORT 8080
> sudo -E ~/bin/beyla 

And once my application got a request, I saw traces being received:

2023-09-19T12:50:50.807-0300	info	TracesExporter	{"kind": "exporter", "data_type": "traces", "name": "logging", "resource spans": 1, "spans": 3}

[grcevski]

I believe traces work for @CSGkeramidas, but not metrics export.

[grcevski]

Hi @CSGkeramidas, can you please confirm that you have correctly configured the

service:
   pipelines:
      metrics:

section?

I took @jpkrohling's example above and I removed the metrics section from the pipelines. I immediately got:

http://otelcol:4318/v1/metrics: 404 Not Found

[CSGkeramidas]

You are correct. I have forgotten to declare the otlp on the metrics pipeline.
Thank you very much for your help