Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making DateTime data zero-copy #1550

Merged
merged 3 commits into from
Jan 27, 2022
Merged

Conversation

robertbastian
Copy link
Member

No description provided.

@robertbastian robertbastian linked an issue Jan 27, 2022 that may be closed by this pull request
Copy link
Member

@Manishearth Manishearth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, probably need a data regen for zeromap

@robertbastian
Copy link
Member Author

I did, which is why testdata.postcard is +81kb, which I find a bit strange.

@Manishearth
Copy link
Member

the zeromaps are a little bit less efficient than litemaps, serialization-wise, but i'm surprised it adds up to that much

@robertbastian robertbastian merged commit 405b9fd into unicode-org:main Jan 27, 2022
Copy link
Member

@sffc sffc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-submit review (please follow the CONTRIBUTING.md please and hold off on merging until there are no more pending reviews)

@@ -56,24 +59,28 @@ macro_rules! symbols {
($name: ident { $element: ident: Option<$ty: ty>, $($tokens: tt)+ } -> ($($members:tt)*)) => {
symbols!($name { $($tokens)* } -> (
$($members)*
#[cfg_attr(feature = "provider_serde", serde(borrow))]
pub $element: Option<$ty>,
Copy link
Member

@sffc sffc Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue: serde(borrow) does not always work through options. You may need deserialize_with.

This is a known issue that dtolnay does not intend to fix (see serde-rs/serde#2016)

Copy link
Member

@sffc sffc Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add tests to verify that the data being deserialized is actually being borrowed. #[serde(borrow)] is weird and it sometimes works and sometimes doesn't work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh i was looking for Option but didn't notice this one since it's in a macro

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably get a clippy lint that warns about serde(borrow) on option fields

@sffc
Copy link
Member

sffc commented Jan 27, 2022

I would also like to know where the 81 KB comes from.

This is a destabilizing change and it should not have been merged when 0.5 is days-to-hours away.

robertbastian added a commit that referenced this pull request Jan 27, 2022
Manishearth pushed a commit that referenced this pull request Jan 31, 2022
* Fixing Option borrows

* Custom Option<Cow<str>> deserializer

* Revert "Custom Option<Cow<str>> deserializer"

This reverts commit a69941c.

* Serde utils in icu_provider

* Removing serde_with dep

* fmt and fix

* Update provider/core/src/serde/borrow_de_utils.rs

Co-authored-by: Shane F. Carr <shane@unicode.org>

Co-authored-by: Shane F. Carr <shane@unicode.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate DateSymbolsV1 to zero-copy
3 participants