Mismatch Between Routing Processor Limitations Docs and Prior Issue?

[bdschaap]

Component(s)

routingprocessor

Describe the issue you're reporting

The routingprocessor documentation mentions that "OTTL statements can be applied to resource attributes". However, a previously filed issue may indicate that there may be support for routing based on log attributes. Though the fix may have been limited to resolving a crash and the functionality may not actually be supported.

Docs - https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/routingprocessor#limitations
Previous issue - #26462

[github-actions]

Pinging code owners for processor/routing: @jpkrohling. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Good catch @bdschaap, I agree the comment doesn't line up with functionality.

Currently the following contexts are used for routing each telemetry type:
ottlspan for traces - ottlspan context README
ottldatapoint for metrics - ottldatapoint context README
ottllog for logs - ottllog context README

As shown in each of the OTTL context READMEs, more things than just resource attributes are available for comparison and routing.

From git blame it looks like these contexts and the README comment have all been in place for multiple years, so I assume if this functionality was wrong it would have created issues by now. Also, if we ended up changing code to match the README it would be a very impacting and breaking change. For these reasons I believe the README should be updated to be more clear, but I'll have to defer to code owners to know if we're missing something here.

[github-actions]

Pinging code owners for pkg/ottl: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[bdschaap]

@crobert-1 Thank you. Though I've tried routing based on severity text of the log but have been unsuccessful. Though perhaps I was doing it wrong if the functionality does exist. It would be great to get confirmation as well as an example added to the docs if the functionality exists .

As a workaround, I've been using 1) the groupbyattrs processor to group logs by the log attribute, 2) the transform processor to create a new resource attribute from the log attribute, and 3) routing processor to route based on the resource attribute. It's a roundabout approach that complicates the configuration but it does work.

[crobert-1]

Happy to help debug routing failures, it's a pretty common challenge for users. However, it'd be better to open a new issue so that this one is kept for it's original purpose. Feel free to open another issue if you want further help getting it to work!

[bdschaap]

Gotcha. Just providing additional context that my experience seemed to align with the documentation and not the previous issue where the crash was fixed.

[TylerHelmuth]

The processor's readme is correct: the routingprocessor (and routingconnector) only make decisions based on resource telemetry. The processor was created before the ottlresource context was added and since it is deprecated it hasn't been updaed. You can see here how it only takes the resource into account:

opentelemetry-collector-contrib/processor/routingprocessor/logs.go

Lines 107 to 111 in bcc7163

	ltx := ottllog.NewTransformContext(
	plog.NewLogRecord(),
	pcommon.NewInstrumentationScope(),
	rlogs.Resource(),
	)

[crobert-1]

Thanks for the reference @TylerHelmuth. I was looking at metrics and should have seen that the transform context was being passed the nil datapoint and pmetric.NewMetric() rather than actual metrics coming in. 🤦

So the behavior here is that since these are valid OTTL statements they won't raise an error in initialization, but they'll just always fail to match incoming data since there are no attributes to compare against. Is that correct?

[TylerHelmuth]

Yes that is correct (and confusing). The routing connect does it correctly: it only uses ottlresource and therefore would allow a statement that uses a field from a log like serverity_text.

[crobert-1]

Closing as the README is correct here.