Skip to content

feat(parser): enhance user agent parsing logic and add tests for inva… #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Aug 25, 2025
Merged

feat(parser): enhance user agent parsing logic and add tests for inva… #76

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
99d5f6d
feat(parser): enhance user agent parsing logic and add tests for inva…
BenjaminAbt Aug 24, 2025
5f56311
fix(parser): improve browser detection logic and clean up tests
BenjaminAbt Aug 24, 2025
9c7ad02
add updated bench
BenjaminAbt Aug 25, 2025
af6d579
re-add deleted comment
BenjaminAbt Aug 25, 2025
942b99b
fix duplicate ua span
BenjaminAbt Aug 25, 2025
f8d2e26
remove old code
BenjaminAbt Aug 25, 2025
a54935d
remove old comment
BenjaminAbt Aug 25, 2025
bb953fc
remove old comment
BenjaminAbt Aug 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,21 +122,21 @@ AMD Ryzen 9 9950X, 1 CPU, 32 logical and 16 physical cores
Job=ShortRun IterationCount=3 LaunchCount=1
WarmupCount=3

| Method | Categories | Data | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio |
|------------------- |----------- |------------- |----------------:|-----------------:|---------------:|----------:|--------:|---------:|---------:|---------:|-----------:|------------:|
| MyCSharp | Basic | Chrome Win10 | 936.44 ns | 131.253 ns | 7.194 ns | 1.00 | 0.01 | 0.0029 | - | - | 48 B | 1.00 |
| UAParser | Basic | Chrome Win10 | 9,512,347.40 ns | 3,961,045.109 ns | 217,118.249 ns | 10,158.42 | 211.89 | 656.2500 | 546.8750 | 109.3750 | 11523315 B | 240,069.06 |
| DeviceDetector.NET | Basic | Chrome Win10 | 5,428,530.73 ns | 5,276,988.556 ns | 289,249.550 ns | 5,797.23 | 270.29 | 296.8750 | 125.0000 | 31.2500 | 5002239 B | 104,213.31 |
| | | | | | | | | | | | | |
| MyCSharp | Basic | Google-Bot | 165.66 ns | 21.926 ns | 1.202 ns | 1.00 | 0.01 | - | - | - | - | NA |
| UAParser | Basic | Google-Bot | 9,737,403.12 ns | 2,336,698.462 ns | 128,082.328 ns | 58,781.92 | 764.74 | 671.8750 | 656.2500 | 109.3750 | 11877003 B | NA |
| DeviceDetector.NET | Basic | Google-Bot | 6,331,960.42 ns | 1,602,716.199 ns | 87,850.283 ns | 38,224.23 | 518.30 | 500.0000 | 62.5000 | - | 8817013 B | NA |
| | | | | | | | | | | | | |
| MyCSharp | Cached | Chrome Win10 | 26.75 ns | 3.749 ns | 0.205 ns | 1.00 | 0.01 | - | - | - | - | NA |
| UAParser | Cached | Chrome Win10 | 250,039.55 ns | 6,502.182 ns | 356.407 ns | 9,346.54 | 63.39 | 2.1973 | - | - | 37488 B | NA |
| | | | | | | | | | | | | |
| MyCSharp | Cached | Google-Bot | 19.66 ns | 4.312 ns | 0.236 ns | 1.00 | 0.01 | - | - | - | - | NA |
| UAParser | Cached | Google-Bot | 184,991.85 ns | 46,235.986 ns | 2,534.350 ns | 9,408.77 | 148.82 | 2.6855 | - | - | 45857 B | NA |
| Method | Categories | Data | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio |
|------------------- |----------- |------------- |----------------:|-----------------:|---------------:|----------:|---------:|---------:|---------:|---------:|-----------:|------------:|
| MyCSharp | Basic | Chrome Win10 | 871.85 ns | 132.008 ns | 7.236 ns | 1.00 | 0.01 | 0.0029 | - | - | 48 B | 1.00 |
| UAParser | Basic | Chrome Win10 | 8,901,909.90 ns | 3,411,259.484 ns | 186,982.644 ns | 10,210.80 | 199.60 | 656.2500 | 578.1250 | 109.3750 | 11523310 B | 240,068.96 |
| DeviceDetector.NET | Basic | Chrome Win10 | 5,391,412.50 ns | 8,253,446.769 ns | 452,399.269 ns | 6,184.14 | 451.58 | 296.8750 | 125.0000 | 31.2500 | 5002239 B | 104,213.31 |
| | | | | | | | | | | | | |
| MyCSharp | Basic | Google-Bot | 158.80 ns | 19.584 ns | 1.073 ns | 1.00 | 0.01 | - | - | - | - | NA |
| UAParser | Basic | Google-Bot | 9,666,739.32 ns | 7,566,085.041 ns | 414,722.653 ns | 60,873.62 | 2,289.43 | 671.8750 | 656.2500 | 109.3750 | 11876998 B | NA |
| DeviceDetector.NET | Basic | Google-Bot | 6,106,666.41 ns | 593,634.990 ns | 32,539.137 ns | 38,455.05 | 285.97 | 539.0625 | 117.1875 | 23.4375 | 8817078 B | NA |
| | | | | | | | | | | | | |
| MyCSharp | Cached | Chrome Win10 | 26.43 ns | 0.132 ns | 0.007 ns | 1.00 | 0.00 | - | - | - | - | NA |
| UAParser | Cached | Chrome Win10 | 177,417.99 ns | 24,390.139 ns | 1,336.906 ns | 6,713.66 | 43.84 | 2.1973 | - | - | 37488 B | NA |
| | | | | | | | | | | | | |
| MyCSharp | Cached | Google-Bot | 17.03 ns | 1.835 ns | 0.101 ns | 1.00 | 0.01 | - | - | - | - | NA |
| UAParser | Cached | Google-Bot | 129,445.13 ns | 21,319.059 ns | 1,168.570 ns | 7,599.76 | 70.93 | 2.6855 | - | - | 45857 B | NA |
```

## Disclaimer
Expand Down
71 changes: 48 additions & 23 deletions src/HttpUserAgentParser/HttpUserAgentParser.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ namespace MyCSharp.HttpUserAgentParser;
/// Parser logic for user agents
/// </summary>
public static class HttpUserAgentParser

{
/// <summary>
/// Parses given <param name="userAgent">user agent</param>
Expand Down Expand Up @@ -78,6 +77,7 @@ public static bool TryGetPlatform(string userAgent, [NotNullWhen(true)] out Http
public static (string Name, string? Version)? GetBrowser(string userAgent)
{
ReadOnlySpan<char> ua = userAgent.AsSpan();

foreach ((string Name, string DetectToken, string? VersionToken) browserRule in HttpUserAgentStatics.s_browserRules)
{
if (!TryIndexOf(ua, browserRule.DetectToken, out int detectIndex))
Expand All @@ -86,7 +86,20 @@ public static (string Name, string? Version)? GetBrowser(string userAgent)
}

// Version token may differ (e.g., Safari uses "Version/")
int versionSearchStart = detectIndex;
// Keep full span immutable across iterations

ReadOnlySpan<char> uaFull = userAgent.AsSpan();
int versionSearchStart;
// For rules without a specific version token, ensure pattern Token/<digits>
if (string.IsNullOrEmpty(browserRule.VersionToken))
{
int afterDetect = detectIndex + browserRule.DetectToken.Length;
if (afterDetect >= uaFull.Length || uaFull[afterDetect] != '/')
{
// Likely a misspelling or partial token (e.g., Edgg, Oprea, Chromee)
continue;
}
}
if (!string.IsNullOrEmpty(browserRule.VersionToken))
{
if (TryIndexOf(ua, browserRule.VersionToken!, out int vtIndex))
Expand All @@ -104,14 +117,22 @@ public static (string Name, string? Version)? GetBrowser(string userAgent)
versionSearchStart = detectIndex + browserRule.DetectToken.Length;
}

string? version = null;
ua = ua.Slice(versionSearchStart);
if (TryExtractVersion(ua, out Range range))
// Work on a local slice to avoid mutating the main span for following rules
if (versionSearchStart < 0 || versionSearchStart >= uaFull.Length)
{
// Nothing to search; try next rule
continue;
}

ReadOnlySpan<char> search = uaFull.Slice(versionSearchStart);
if (TryExtractVersion(search, out Range range))
{
version = ua[range].ToString();
string? version = search[range].ToString();
return (browserRule.Name, version);
}

return (browserRule.Name, version);
// If we didn't find a version for this rule, try next rule
continue;
}

return null;
Expand Down Expand Up @@ -198,39 +219,43 @@ private static bool TryExtractVersion(ReadOnlySpan<char> haystack, out Range ran

// Limit search window to avoid scanning entire UA string unnecessarily
const int Window = 128;
if (haystack.Length >= Window)
if (haystack.Length > Window)
{
haystack = haystack.Slice(0, Window);
}

int i = 0;
for (; i < haystack.Length; ++i)
// Find first digit
int start = -1;
for (int i = 0; i < haystack.Length; i++)
{
char c = haystack[i];
if (char.IsBetween(c, '0', '9'))
if (c >= '0' && c <= '9')
Copy link
Contributor

@gfoidl gfoidl Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is two comparisons now, instead of just one. Either use the uint cast directly, or use char.IsBetween which does the same thing but more readable IMO.

Only the JIT for .NET 10+ is able to optimize that to one comparison (by using the uint cast).

{
start = i;
break;
}
}

int s = i;
haystack = haystack.Slice(i + 1);
for (i = 0; i < haystack.Length; ++i)
if (start < 0)
{
char c = haystack[i];
if (!(char.IsBetween(c, '0', '9') || c == '.'))
{
break;
}
// No digit found => no version
return false;
}
i += s + 1; // shift back the previous domain

if (i == s)
// Consume digits and dots after first digit
int end = start + 1;
while (end < haystack.Length)
{
return false;
char c = haystack[end];
if (!((c >= '0' && c <= '9') || c == '.'))
Copy link
Contributor

@gfoidl gfoidl Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

{
break;
}
end++;
}

range = new Range(s, i);
// Create exclusive end range
range = new Range(start, end);
return true;
}
}
2 changes: 1 addition & 1 deletion src/HttpUserAgentParser/HttpUserAgentStatics.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ internal static readonly (string Name, string DetectToken, string? VersionToken)
("Opera", "OPR", null),
("Flock", "Flock", null),
("Edge", "Edge", null),
("Edge", "EdgiOS", null),
("Edge", "EdgA", null),
("Edge", "Edg", null),
("Vivaldi", "Vivaldi", null),
Expand All @@ -208,7 +209,6 @@ internal static readonly (string Name, string DetectToken, string? VersionToken)
("Netscape", "Netscape", null),
("OmniWeb", "OmniWeb", null),
("Safari", "Version/", "Version/"),
("Mozilla", "Mozilla", null),
("Konqueror", "Konqueror", null),
("iCab", "icab", null),
("Lynx", "Lynx", null),
Expand Down
50 changes: 50 additions & 0 deletions tests/HttpUserAgentParser.UnitTests/HttpUserAgentParserTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,4 +173,54 @@ public void BotTests(string ua, string name)
Assert.False(uaInfo.IsMobile());
Assert.True(uaInfo.IsRobot());
}

[Theory]
[InlineData("")]
[InlineData("???")]
[InlineData("NotAUserAgent")]
[InlineData("Mozilla")]
[InlineData("Mozilla/")]
[InlineData("()")]
[InlineData("UserAgent/")]
[InlineData("Bot/123 (")]
[InlineData("123456")]
[InlineData("curl")]
[InlineData("invalid/useragent")]
[InlineData("Mozilla (Windows)")]
[InlineData("Chrome/ABC")]
[InlineData(";;!!##")]
[InlineData("Safari/ ")]
[InlineData("Opera( )")]
[InlineData("Mozilla/5.0 (X11; ) Gecko")]
[InlineData("FakeUA/1.0 (Test)???")]
[InlineData("Mozilla/ (iPhone; U; CPU iPhone OS like Mac OS X) AppleWebKit/ (KHTML, like Gecko) Version/ Mobile/ Safari/")]
[InlineData("Mozzila/5.0 (Windows NT 10.0; Win64; x64)")]
[InlineData("Chorme/91.0.4472.124 (Windows NT 10.0; Win64; x64)")]
[InlineData("FireFoxx/89.0 (Macintosh; Intel Mac OS X 10_15_7)")]
[InlineData("Safarii/14.1 (iPhone; CPU iPhone OS 14_6 like Mac OS X)")]
[InlineData("InternetExploder/11.0 (Windows NT 6.1; WOW64)")]
[InlineData("Bravee/1.25.72 (Windows NT 10.0; Win64; x64)")]
[InlineData("Mozzila/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0)")]
[InlineData("Chromee/99.0.4758.102 (X11; Linux x86_64)")]
[InlineData("FirreFox/100.0 (Windows NT 10.0; rv:100.0)")]
[InlineData("Saffari/605.1.15 (iPad; CPU OS 14_6 like Mac OS X)")]
[InlineData("Edgg/103.0.1264.37 (Macintosh; Intel Mac OS X 11_5_2)")]
[InlineData("Chorome/91.0.4472.124 (Linux; Android 10; SM-G973F)")]
[InlineData("Edgee/18.18363 (Windows 10 1909; Win64; x64)")]
public void InvalidUserAgent(string userAgent)
{
HttpUserAgentInformation info = HttpUserAgentInformation.Parse(userAgent);

// Invalid or malformed UAs must be classified as Unknown
Assert.Equal(HttpUserAgentType.Unknown, info.Type);
Assert.Null(info.Name);
Assert.Null(info.Version);

// Parser trims input via Cleanup, so compare to trimmed UA
Assert.Equal(userAgent.Trim(), info.UserAgent);

// Should not be considered a browser or a robot
Assert.False(info.IsBrowser());
Assert.False(info.IsRobot());
}
}