Safely truncate over-length string attributes

[jmacd]

This fixes invalid UTF-8 introduced by our truncation logic.
This does not address the problem of invalid UTF-8 being passed in by users, but it does at least ensure the SDK is not responsible for invalid UTF-8 itself.

Fixes #3021

[jmacd]

This needs at least one more test. ;)

[codecov]

Codecov Report

Merging #3156 (41af575) into main (7aba25d) will increase coverage by 0.0%.
The diff coverage is 94.7%.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #3156   +/-   ##
=====================================
  Coverage   76.3%   76.3%           
=====================================
  Files        180     180           
  Lines      11992   12009   +17     
=====================================
+ Hits        9153    9169   +16     
- Misses      2597    2598    +1     
  Partials     242     242           

Impacted Files	Coverage Δ
sdk/trace/span.go	88.2% <94.7%> (+0.2%)	⬆️

[jmacd]

PTAL. I had to rewrite the fallback case; when there is invalid UTF-8 it calls strings.ToValidUTF8 and then re-applies the same (safe) truncation logic.

[MadVikingGod]

The costly operation in this call is the ToValidUTF8, because it has to iterate over the entire string. So this means how you have it now we iterate over the string once, try to apply the limit, and iterate a second time.

I think an alternative but still valid interpretation would be to apply the limit, and then filter bad unicode. This would mean that if there bad points in the beginning of the string they would count toward the limit, even if they aren't in the output string.

[jmacd]

The way the code is organized, strings.ToValidUTF8() (which copies the data) will only be called when the user has supplied invalid code points. Scanning the input only happens when the input is over-limit, but it won't cause an allocation unless the attribute is over-limit and also contains invalid points. I believe the allocation is the expensive part of this, not the scanning part. This seems like a good compromise, although I think OTel may have to address the problem of users supplying invalid UTF8 anyway.

[jmacd]

Personally, I consider this an urgent matter and I would be willing to waive our ordinary wait period for PRs to merge after approvals. I suggest this because it's been about 4 weeks since the last release, and if we don't include this fix there will be immediate pressure to make another release. @open-telemetry/go-maintainers what do you think?

[MrAlias]

Personally, I consider this an urgent matter and I would be willing to waive our ordinary wait period for PRs to merge after approvals. I suggest this because it's been about 4 weeks since the last release, and if we don't include this fix there will be immediate pressure to make another release. @open-telemetry/go-maintainers what do you think?

I'm fine merging this early. That said, @MadVikingGod has decided to hold off the release until Monday to make sure this is included.