Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZeroCopy aliases #1777

Merged
merged 14 commits into from
Apr 8, 2022
Merged

ZeroCopy aliases #1777

merged 14 commits into from
Apr 8, 2022

Conversation

robertbastian
Copy link
Member

Closes #1034

@robertbastian robertbastian removed request for a team, dminor and nciric April 6, 2022 02:29
@robertbastian
Copy link
Member Author

@Manishearth could you please take a look at the make_varule error

@Manishearth
Copy link
Member

@robertbastian I have a fix, along with code making the rest of your stuff compile; mind if I push to the same branch?

@Manishearth
Copy link
Member

Commits are at https://github.com/Manishearth/icu4x/tree/pr-1777, either way. I plan to extract some of it to a separate PR, for one the trait implementation situation is annoying and will likely get worse

Copy link
Member

@sffc sffc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job: This is a challenging PR to write that leverages a large portion of the zerovec machinery!

@robertbastian robertbastian requested a review from sffc April 6, 2022 21:02
Manishearth
Manishearth previously approved these changes Apr 7, 2022
Manishearth
Manishearth previously approved these changes Apr 7, 2022
@robertbastian robertbastian requested a review from sffc April 7, 2022 21:39
.iter()
.map(zerofrom::ZeroFrom::zero_from)
{
if let Ok(from) = raw_from.parse::<LanguageIdentifier>() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I think this is still parsing each time for all canonicalize operations

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ Please fix if this causes us to allocate memory for every operation. If there is no memory being allocated, this is a nice-to-fix but not must-fix

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is, but this is the fallback field which we're trying to keep empty. If you have a way to check uts35_rule_matches on a string I'm all ears, but I don't see it without parsing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To do this without allocation we could make Variants a Cow<'data, str>. The only functionality it has is to iterate through the variants, which would be very cheap with something like self.0.split('-').map(Variant::from_raw_unchecked). The downside would be that this adds a lifetime to LanguageIdentifier and Locale.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, if this is for fallback data, and fallback data is currently empty, I have no concerns.

Modifying the data model of LanguageIdentifier or Locale is understood to be off the table with much discussion in #831.

@robertbastian robertbastian merged commit 6590b5d into unicode-org:main Apr 8, 2022
@robertbastian robertbastian deleted the zc branch May 23, 2022 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate LocaleCanonicalizer data structs to zero-copy
3 participants