Add SearchValues<string>

[MihaZupan]

Closes #85573
Closes #86528
Contributes to #62447

As discussed during API review, only Ordinal and OrdinalIgnoreCase semantics are available.
Supports OrdinalIgnoreCase fully (including running under Invariant/NLS mode).
Includes Avx512 variants for vectorized paths.

This is an initial version of SearchValues<string>, which includes the following specialized implementations:

• Teddy-based approach for searching for length=2 or 3 prefixes. Includes a bucketized variant when dealing with more than 8 values.
• Rabin-Karp-based approach that is only used as a fallback for Teddy for very short inputs.
• Aho-Corasick-based approach that we use when dealing with many values (e.g. searching for a blocklist of words in a text), or as a fallback when we can't use Teddy for other reasons.
• SingleStringSearchValuesThreeChars is a variant of http://0x80.pl/articles/simd-strfind.html#algorithm-1-generic-simd that uses 3 precomputed anchor points. On average provides lower per-call overheads than calling text.IndexOf("foo").
• SingleStringSearchValuesFallback for when we don't have anything fancier available and falling back to Aho-Corasick would be noticeably slower.

These variants have additional generic specializations for different case sensitivity semantics of values: CaseSensitive, CaseInsensitiveAsciiLetters, CaseInsensitiveAscii, and CaseInsensitiveUnicode.

I'll share actual perf numbers when we're happier with the overall shape since there are a lot of interesting needle+haystack combinations to look at.

cc: @stephentoub @teo-tsirpanis @danmoseley

[dotnet-issue-labeler]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-buffers
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Closes #85573
Contributes to #86528
Contributes to #62447

As discussed during API review, only Ordinal and OrdinalIgnoreCase semantics are available.
Supports OrdinalIgnoreCase fully (including running under Invariant/NLS mode).
Includes Avx512 variants for vectorized paths.

This is an initial version of SearchValues<string>, which includes the following specialized implementations:

• Teddy-based approach for searching for length=2 or 3 prefixes. Includes a bucketized variant when dealing with more than 8 values.
• Rabin-Karp-based approach that is only used as a fallback for Teddy for very short inputs.
• Aho-Corasick-based approach that we use when dealing with many values (e.g. searching for a blocklist of words in a text), or as a fallback when we can't use Teddy for other reasons.
• SingleStringSearchValuesThreeChars is a variant of http://0x80.pl/articles/simd-strfind.html#algorithm-1-generic-simd that uses 3 precomputed anchor points. On average provides lower per-call overheads than calling text.IndexOf("foo").
• SingleStringSearchValuesFallback for when we don't have anything fancier available and falling back to Aho-Corasick would be noticeably slower. Has some length-based specializations as well.

These variants have additional generic specializations for different case sensitivity semantics of values: CaseSensitive, CaseInensitiveAsciiLetters, CaseInensitiveAscii, and CaseInensitiveUnicode.

There are still some less-critical TODOs left, and the code could use more comments. Opening this early in case we want to make any more significant changes.

I'll share actual perf numbers when we're happier with the overall shape since there are a lot of interesting needle+haystack combinations to look at.

cc: @stephentoub @teo-tsirpanis @danmoseley

Author:	MihaZupan
Assignees:	MihaZupan
Labels:	area-System.Buffers
Milestone:	8.0.0

[danmoseley]

do we need anything added to the THIRD-PARTY-NOTICES.txt such as something for http://0x80.pl/articles/simd-strfind.html#algorithm-1-generic-simd or existing implementations like https://github.com/jneem/teddy if they were particularly helpful?

[MihaZupan]

do we need anything added to the THIRD-PARTY-NOTICES.txt

If we don't have anything for http://0x80.pl/articles/simd-strfind.html#algorithm-1-generic-simd yet, then yes, we should add that as we're already using it in our string IndexOf implementations.

It would also be fair to add a mention of https://github.com/BurntSushi/aho-corasick as I've spent a lot of time digging through its code initially to understand how the Teddy algorithm works.

[danmoseley]

It would also be fair to add a mention of https://github.com/BurntSushi/aho-corasick as I've spent a lot of time digging through its code initially to understand how the Teddy algorithm works.

kudos to @BurntSushi here.

[stephentoub]

@tannergooding, this PR feels like a really good candidate for exploring ways of reducing duplication between Vector128/256/512.