Provide access to the SchemaURL fields from the OTTL contexts

[bw972]

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

Currently, there is no way to access the SchemaURL fields from the OTTL contexts. Having access to the SchemaURL within the OTTL contexts would enhance the functionality and flexibility of using OTTL.

Describe the solution you'd like

Add the SchemaURL fields to the list of available fields that can be accessed within the OTTL contexts.

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• pkg/ottl: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[kernelpanic77]

Hi @TylerHelmuth! Can I take this up ?

[TylerHelmuth]

All yours!

[kernelpanic77]

@TylerHelmuth I am lacking a bit of clarity, on what needs to be done for this issue ? Could you point me to the exact piece of code i should be looking at.

[TylerHelmuth]

Take a look at pkg/ottl/contexts/internal/scope.go. You'll want to add a new case to the switch statement.

[kernelpanic77]

@TylerHelmuth Sorry for my delayed response. Here is my understanding till now.

opentelemetry-collector-contrib/pkg/ottl/contexts/internal/scope.go

Line 19 in 2047ee9

func ScopePathGetSetter[K InstrumentationScopeContext](path ottl.Path[K]) (ottl.GetSetter[K], error) {

I will have to add a field specifically for the "schemaURL" context for the as part of ScopePathGetSetter function.
Here are some doubts I am having.

1. How can I access the schemaURL from the InstrumentationScope ?
2. ScopeLogs, ScopeMetrics and ScopeSpans allow access to the SchemaURL function, should the InstrumentationScope be somehow casted to Scope Logs/Metrics/Spans?

Sorry for my silly questions, I couldn't get my head around the documentation till now 😅

[TylerHelmuth]

Oh I see the problem now, schema_url didn't live in the proto where I thought it did.

This makes it a lot harder to add :/ We'd need to update the TransformContext of each Context to have access to the value, which will be a lot of breaking changes. I don't want to expose all the entire ScopeLogs/ScopeMetrics/ScopeSpan object tho. We also have to provide a way to access both the resource's schemaurl and the scope's.

This problem is no longer easy (you're welcome to continue working on it anyways).

@evan-bradley how do you think we should expose these 2 fields from the proto? OTTL has always followed the proto to the letter, and schema_url lives in a section of the proto hierarchy we've never exposed. I don't really want to make a new contexts for all these scenarios. Could we get away with resource.schema_url and scope.schema_url since the schema_url is the same for all resources and all scopes?

[kernelpanic77]

I'll like to continue working on this issue. @evan-bradley any suggestions on how we should expose the schema_url from the proto.

[evan-bradley]

@evan-bradley how do you think we should expose these 2 fields from the proto? OTTL has always followed the proto to the letter, and schema_url lives in a section of the proto hierarchy we've never exposed. I don't really want to make a new contexts for all these scenarios. Could we get away with resource.schema_url and scope.schema_url since the schema_url is the same for all resources and all scopes?

I'm not sure the last statement will always hold true (each resource and scope can have it's own value for the field), but I would say we can just take the approach you're suggesting and expose the Schema URL like other fields. If enough users want to modify scopes (as opposed to just reading from them in a lower-level contexts) then we should add a context to make that more efficient.

[TylerHelmuth]

Lets start with that approach and see what happens. @kernelpanic77 let's start small by modifying only the pkg/ottl/contexts/ottlresource module for now. It'll be a breaking change to that package's NewTransformContext function.

[TylerHelmuth]

@kernelpanic77 the name of the structs that contain SchemaURL are (for logs): plog.ResourceLogs, plog.ScopeLogs. For traces they will be ptrace.ResourceSpans and ptrace.ScopeSpans. For metrics they will be pmetric.ResourceMetrics and pmetric.ScopeMetrics.

You're going to run into a challenge with the generic Resource and Scope OTTL contexts. Ignore those for now.

@evan-bradley this situation has highlighted that Resource and Scope contexts aren't perfectly generic between the signals. We're gonna need special cases to handle SchemaURL in those context, or just not allow it.

[evan-bradley]

Could we solve the fact that they're different concrete types with an interface like this?

type SchemaURLItem interface {
    SchemaUrl() string
    SetSchemaUrl(v string)
}

I think all of those types would conform to this interface.

[TylerHelmuth]

Ya something like that would work.

[kernelpanic77]

@TylerHelmuth @evan-bradley

From my understanding till now, the following changes can be added to pkg/ottl/contexts/ottlresource module.

type TransformContext struct {
	resource  pcommon.Resource
	cache     pcommon.Map
	schemaURL string
}

func NewTransformContext(resource pcommon.Resource, schemaURL string) TransformContext {
	return TransformContext{
		resource:  resource,
		cache:     pcommon.NewMap(),
		schemaURL: schemaURL,
	}
}

Here I am adding the schemaURL to the NewTransformContext method, along with that the corresponding getters and setters have to be added in the accessSchemaURL function.

I will also fix the breaking references of NewTransformContext inside the connector and processor pkgs.
Let me know if I am going in the correct direction.

[TylerHelmuth]

@kernelpanic77 passing only the string isn't enough. The TransformContext will need access to the ptrace/pmetric/plog structs I mentioned before so that context can correct set the value.

Dont start with Resource or Scope, those will be hard. Try implementing for ottllog.

[kernelpanic77]

@TylerHelmuth This is my attempt to implement for ottllog, have a look.
I was a bit confused on whether I should use scopeLog or resourceLog as part of newTransformContext func

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• pkg/ottl: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.