[Discussion] Add deployment.environment support

[RassK]

What

Adds deployment.environment resource support via env SIGNALFX_ENV.
In OTel spec this is advised but for SingalFX APM this is required (for grouping and navigating purposes).

https://signalfuse.atlassian.net/browse/APMI-3667

OTel spec suggests OTEL_RESOURCE_ATTRIBUTES but this is not suitable in machine level.

Tests

• Unit tests added
• Manual integration test

https://app.signalfx.com/#/apm/traces/fc4620b36ee3c5759bd9f575397e0b1c

[pellared]

None of the distributions support SIGNALFX_ENV. It is legacy. I think we should close this PR.

[RassK]

None of the distributions support SIGNALFX_ENV. It is legacy. I think we should close this PR.

I agree, I would have used SPLUNK_ENV but this seems also not used.
OTEL_RESOURCE_ATTRIBUTES unfortunately is not suitable for global scope as more precise scope will overwrite the global env. To encounter it the more precise env needs to contain the content of global env, which means it is going to add multiple change (failure) points.

[theletterf]

All our other distros use resource attributes to deal with this, AFAIK.

[RassK]

All our other distros use resource attributes to deal with this, AFAIK.

I'm not sure how big is the issue for other languages, since world is moving towards containers / pods, where global scope generally equals process scope, 1 service per container / pod, etc.
When you use heavy environments like Windows Server and IIS, then you can expect multiple services per instance. I see even today loads of use cases of multiple IIS websites + windows services for background processing.
This means you would like to set some machine level environment vars to globally describe the environment. Even I would, for eg testing with Splunk APM, it's less overhead to make it automatically pick up machine level variable, so I know from which VM the trace was originated.

[theletterf]

I can see the positive effect there. Wouldn't this have to be added to the GDI specs first though? Could changes be rolled out before that?

[RassK]

I can see the positive effect there. Wouldn't this have to be added to the GDI specs first though? Could changes be rolled out before that?

I guess @pellared knowns the process much better.
@pellared any thoughts? - worth pushing or not?

[RassK]

Team meeting update: We will not bring any more SINGALFX_* vars.

[RassK]

Linking spec issue here > open-telemetry/opentelemetry-specification#3025

[Kielek]

Closing for now. See internal issue for details.