perf: improve to_file_path() #1018

dsherret · 2025-01-27T21:26:56Z

Reduces some heap allocations and pre-allocates the vector for building the string.

dsherret · 2025-01-27T21:27:21Z

url/src/lib.rs

-    } else {
-        Vec::new()
-    };
+    let mut bytes = Vec::with_capacity(estimated_capacity);


Before 1,000 iterations within an iteration:

test url_to_file_path ... bench: 202,125 ns/iter (+/- 12,207)

After:

test url_to_file_path ... bench: 127,257 ns/iter (+/- 4,782)

dsherret · 2025-01-27T21:27:55Z

url/src/lib.rs

    host: Option<&str>,
    segments: str::Split<char>,
 ) -> Result<PathBuf, ()> {
-    file_url_segments_to_pathbuf_windows(host, segments)
+    file_url_segments_to_pathbuf_windows(estimated_capacity, host, segments)
 }

 // Build this unconditionally to alleviate https://github.com/servo/rust-url/issues/102
 #[cfg(feature = "std")]
 #[cfg_attr(not(windows), allow(dead_code))]
 fn file_url_segments_to_pathbuf_windows(


Before 1000 iterations within an iteration:

test url_to_file_path ... bench: 437,555 ns/iter (+/- 11,519)

After:

test url_to_file_path ... bench: 119,461 ns/iter (+/- 5,927)

url/src/lib.rs

codecov · 2025-01-27T21:32:53Z

Codecov Report

Attention: Patch coverage is 21.73913% with 18 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@ffca1ef). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
url/src/lib.rs	21.73%	18 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1018   +/-   ##
=======================================
  Coverage        ?   79.97%           
=======================================
  Files           ?       24           
  Lines           ?     4274           
  Branches        ?        0           
=======================================
  Hits            ?     3418           
  Misses          ?      856           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lucacasonato

LGTM

lucacasonato · 2025-02-06T18:22:04Z

url/src/lib.rs

@@ -2720,7 +2720,7 @@ impl Url {
                _ => return Err(()),
            };

-            return file_url_segments_to_pathbuf(host, segments);
+            return file_url_segments_to_pathbuf(self.as_str().len(), host, segments);


Please split the estimated_len into its own temp variable so it's cleaner to read what this is.

I think this over-alloc 7 bytes in 99% of cases on unix (file://). On redox, just 2 (//). On Windows, even 8 (file:///). Windows also has hostname support though, so to ensure no-realloc in most cases, you should probably do -5 there too (file: - the extra // is for the leading \\ in hostname cases).

The cost of doing this calculation may be too much perf cost though - so please check if this minor memory improvement wouldn't regress perf too much.

It adds a few nanoseconds. Worth it to use less memory IMO.

lucacasonato · 2025-02-06T21:13:29Z

url/src/lib.rs

@@ -2720,7 +2720,18 @@ impl Url {
                _ => return Err(()),
            };

-            return file_url_segments_to_pathbuf(host, segments);
+            let estimated_capacity = self.as_str().len()
+                - if cfg!(target_os = "redox") {


Can you make this a clamped subtract so that it doesn't wrap around in release when someone calls this with something other than file scheme URLs?

Good catch. I think the logic needs to be a bit smarter here because urls could be like a:/path? I updated the code to at least be a fast path for file://-like urls and maybe in some weird cases like a:/path it might under allocate. Probalby not worth the effort to make it smarter than it is now.

lucacasonato · 2025-02-07T10:00:05Z

url/src/lib.rs

+                let file_scheme_len = "file".len();
+                // remove only // because it still has file:
+                if scheme_len < file_scheme_len {
+                    let scheme_diff = file_scheme_len - scheme_len;


This still wraps around when scheme is longer than 4.

There's a check above for this:

if scheme_len < file_scheme_len { // if 5 < 4 {

(or maybe I'm misreading)

lucacasonato

LGTM, very nice improvement.

Bumps url from 2.5.4 to 2.5.5. Release notes Sourced from url's releases. v2.5.5 What's Changed ci: downgrade crates when building for Rust 1.67.0 by @mxinden in servo/rust-url#1003 ci: run unit tests with sanitizers by @mxinden in servo/rust-url#1002 fix small typo by @hkBst in servo/rust-url#1011 chore: fix clippy errors on main by @dsherret in servo/rust-url#1019 perf: remove heap allocation in parse_query by @dsherret in servo/rust-url#1020 perf: slightly improve parsing a port by @dsherret in servo/rust-url#1022 perf: improve to_file_path() by @dsherret in servo/rust-url#1018 perf: make parse_scheme slightly faster by @dsherret in servo/rust-url#1025 update LICENSE-MIT by @wmjae in servo/rust-url#1029 perf: url encode path segments in longer string slices by @dsherret in servo/rust-url#1026 Disable the default features on serde by @rilipco in servo/rust-url#1033 docs: base url relative join by @tisonkun in servo/rust-url#1013 perf: remove heap allocation in parse_host by @dsherret in servo/rust-url#1021 Update tests to Unicode 16.0 by @hsivonen in servo/rust-url#1045 Add some some basic functions to Mime by @mrobinson in servo/rust-url#1047 ran cargo clippy --fix -- -Wclippy::use_self by @mrobinson in servo/rust-url#1048 Fix MSRV and clippy CI by @Manishearth in servo/rust-url#1058 Update Url::domain docs to show that it includes subdomain by @supercoolspy in servo/rust-url#1057 set_hostname should error when encountering colon ':' by @edgul in servo/rust-url#1060 version bump to 2.5.5 by @edgul in servo/rust-url#1061 New Contributors @mxinden made their first contribution in servo/rust-url#1003 @hkBst made their first contribution in servo/rust-url#1011 @wmjae made their first contribution in servo/rust-url#1029 @rilipco made their first contribution in servo/rust-url#1033 @tisonkun made their first contribution in servo/rust-url#1013 @supercoolspy made their first contribution in servo/rust-url#1057 Full Changelog: servo/rust-url@v2.5.4...v2.5.5 Commits a40f904 version bump to 2.5.5 (#1061) cf305db set_hostname should error when encountering colon ':' (#1060) 88826bd Update Url::domain docs to show that it includes subdomain (#1057) c3bbf66 Fix MSRV and clippy CI (#1058) dbd5261 ran cargo clippy --fix -- -Wclippy::use_self (#1048) 9f6e92e Add some some basic functions to Mime (#1047) 68f151c Update tests to Unicode 16.0 (#1045) 7cff874 perf: remove heap allocation in parse_host (#1021) 968e862 docs: base url relative join (#1013) 2ce2e12 Disable the default features on serde. (#1033) Additional commits viewable in compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase. Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: @dependabot rebase will rebase this PR @dependabot recreate will recreate this PR, overwriting any edits that have been made to it @dependabot merge will merge this PR after your CI passes on it @dependabot squash and merge will squash and merge this PR after your CI passes on it @dependabot cancel merge will cancel a previously requested merge and block automerging @dependabot reopen will reopen this PR if it is closed @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

perf: improve to_file_path()

fbaf6df

dsherret commented Jan 27, 2025

View reviewed changes

url/src/lib.rs Outdated Show resolved Hide resolved

fix

93682a6

This comment was marked as outdated.

Sign in to view

Merge branch 'main' into perf_improve_url_to_file_path

313bde0

lucacasonato approved these changes Feb 6, 2025

View reviewed changes

make estimate more exact

3e1aeb4

lucacasonato reviewed Feb 6, 2025

View reviewed changes

dsherret added 2 commits February 6, 2025 17:00

use saturating_sub

51bc795

better to try_reserve

22873d0

lucacasonato reviewed Feb 7, 2025

View reviewed changes

dsherret requested a review from lucacasonato February 7, 2025 19:22

lucacasonato approved these changes Feb 8, 2025

View reviewed changes

lucacasonato added this pull request to the merge queue Feb 8, 2025

Merged via the queue into servo:main with commit 267afa5 Feb 8, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf: improve to_file_path() #1018

perf: improve to_file_path() #1018

Uh oh!

dsherret commented Jan 27, 2025 •

edited

Loading

Uh oh!

dsherret Jan 27, 2025 •

edited

Loading

Uh oh!

dsherret Jan 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

codecov bot commented Jan 27, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

lucacasonato left a comment

Uh oh!

lucacasonato Feb 6, 2025 •

edited

Loading

Uh oh!

dsherret Feb 6, 2025

Uh oh!

lucacasonato Feb 6, 2025

Uh oh!

lucacasonato Feb 6, 2025

Uh oh!

dsherret Feb 6, 2025

Uh oh!

lucacasonato Feb 7, 2025

Uh oh!

dsherret Feb 7, 2025 •

edited

Loading

Uh oh!

lucacasonato left a comment

Uh oh!

Uh oh!

Uh oh!

perf: improve to_file_path() #1018

perf: improve to_file_path() #1018

Uh oh!

Conversation

dsherret commented Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dsherret Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsherret Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

lucacasonato left a comment

Choose a reason for hiding this comment

Uh oh!

lucacasonato Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsherret Feb 6, 2025

Choose a reason for hiding this comment

Uh oh!

lucacasonato Feb 6, 2025

Choose a reason for hiding this comment

Uh oh!

lucacasonato Feb 6, 2025

Choose a reason for hiding this comment

Uh oh!

dsherret Feb 6, 2025

Choose a reason for hiding this comment

Uh oh!

lucacasonato Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

dsherret Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucacasonato left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dsherret commented Jan 27, 2025 •

edited

Loading

dsherret Jan 27, 2025 •

edited

Loading

dsherret Jan 27, 2025 •

edited

Loading

codecov bot commented Jan 27, 2025 •

edited

Loading

lucacasonato Feb 6, 2025 •

edited

Loading

dsherret Feb 7, 2025 •

edited

Loading