Add a sub-command to retrieve configuration

[gbbr]

I would like to propose adding a sub-command to the core which allows retrieving the configuration from a running collector. I propose this to be done by opening up an endpoint (which listens strictly on the local host) that reports the configuration.

Ideally, I would also like to explore an implementation where each each configuration item also reports its origin (e.g. env, file, remote, etc). This part however may deem tricky and may need to be addressed in a second milestone.

Am happy to start work on this if approved.

Usage

One option would be to simply add the config sub-command, just as we have the components sub-command today.

$ ./bin/otelcorecol_darwin_arm64 --help
Usage:
  otelcorecol [flags]
  otelcorecol [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  components  Outputs available components in this collector distribution
  config      Outputs a running collector's configuration.
  help        Help about any command

Alternative solution

One may also consider doing this as part of an extension. However, the debugging capabilities of a running Collector are reduced if the user is not running such an extension (ref).

[kentquirk]

The localhost constraint might be difficult to deal with in practice, depending on the way it's deployed.

What about instead specifying some sort of really simple auth model? It could be as simple as a configured token in a custom header that must be specified as part of the GET request.

[dmitryax]

I would like to propose adding a sub-command to the core which allows retrieving the configuration from a running collector. I propose this to be done by opening up an endpoint (which listens strictly on the local host) that reports the configuration.

I'm not sure I understand how you would get a configuration from a running collector. How'd you call an executable which is already running? And why do you need that? Wouldn't it be enough just to have a flag that validates and shows the configuration provided to the collector?

[kentquirk]

I understood the proposal to be that the command line switch would cause collector to open an endpoint that would be available to read back the fully hydrated configuration, but that the endpoint would only listen to requests that came from the same machine. If I'm wrong about that, then you can ignore my comment.

The use case I'm describing is most useful in a cluster containing multiple instances where you want to check how the machines are configured, especially in a situation where configs can be hot-reloaded.

[dmitryax]

I understood the proposal to be that the command line switch would cause collector to open an endpoint that would be available to read back the fully hydrated configuration, but that the endpoint would only listen to requests that came from the same machine. If I'm wrong about that, then you can ignore my comment.

That makes sense. It can be potentially added to https://github.com/open-telemetry/opentelemetry-collector/tree/main/extension/zpagesextension or implemented as another extension

This seems like a duplicate issue #5223

[gbbr]

Thanks for clarifying @kentquirk. I think you have definitely understood both the issue and the use case, and have described it well.

@dmitryax please allow me some time to study #5223 and make sure they are the same thing, and if so, close this one as a duplicate. An extension is indeed something that was on my mind as an option.

[seankhliao]

zpages has comments like

opentelemetry-collector/service/extensions/extensions.go

Line 104 in 14c039d

// TODO: Add config + status info.

but it doesn't seem like it has access to config since now it is only passed a builder for the components with the configs already hidden inside.

[atoulme]

Can we close in favor of #5223? FWIW the splunk distro exposes a zpages endpoint for effective config, so it can be done.

[mx-psi]

Closing as duplicate of #5223