Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add icu_properties feature to icu_normalizer #5551

Merged
merged 13 commits into from
Sep 23, 2024

Conversation

Manishearth
Copy link
Member

Fixes #5121

sffc
sffc previously approved these changes Sep 18, 2024
components/normalizer/src/lib.rs Outdated Show resolved Hide resolved
@sffc
Copy link
Member

sffc commented Sep 18, 2024

error[E0599]: no method named `get32` found for struct `CanonicalCombiningClassMapBorrowed` in the current scope
  --> ffi/capi/src/normalizer_properties.rs:54:34
   |
54 |             self.0.as_borrowed().get32(ch).0
   |                                  ^^^^^ help: there is a method with a similar name: `get32_u8`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `icu_capi` (lib) due to previous error

@hsivonen
Copy link
Member

hsivonen commented Sep 18, 2024

Thanks. This needs testing with Pernosco to see if this breaks Pernosco's special knowledge about the ICU4X types involved here. I'll test and will report back.

@hsivonen
Copy link
Member

I created a new crate that depends on icu_normalizer by path, and building the new crate ends up building icu_properties anyway. Is this changeset accomplishing what's intended?

Copy link
Member

@hsivonen hsivonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changeset with or without the feature enabled breaks Pernosco's rendering of the CharacterAndClass type.

Without this PR:
self ::CharacterAndClass*@0x7285c15ff1b8={char=‘⁨̧⁩’ 0x327, ccc=(202 ‘Ê’)}

With this PR:
self ::CharacterAndClass*@0x7eacb49ff1b8=(3388998439)

I think this should not be merged as-is.

I'm not quite sure what precisely Pernosco wants to see. I suggest putting the accessors on CharacterAndClass that have CanonicalCombiningClass in the return value behind the feature and introducing differently-named u8 accessors. (I haven't yet tested if that would work.)

@hsivonen
Copy link
Member

hsivonen commented Sep 18, 2024

I created a new crate that depends on icu_normalizer by path, and building the new crate ends up building icu_properties anyway. Is this changeset accomplishing what's intended?

I pushed a change to restore Pernosco compat. (That took way more rounds of experimentation than I expected!) However, the above observation still applies: icu_properties gets compiled when the icu_properties feature isn't set. Why?

components/normalizer/Cargo.toml Outdated Show resolved Hide resolved
components/normalizer/src/lib.rs Outdated Show resolved Hide resolved
@@ -180,12 +196,12 @@ fn decomposition_starts_with_non_starter(trie_value: u32) -> bool {
///
/// The trie value must not be one that signifies a special non-starter
/// decomposition. (Debug-only)
fn ccc_from_trie_value(trie_value: u32) -> CanonicalCombiningClass {
fn ccc_from_trie_value(trie_value: u32) -> u8 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: define type CanonicalCombiningClass = u8 or a newtype, and then qualify the properties one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea, I did one better, I wrote a seamless shim type CanonicalCombiningClass(pub u8) that gets imported in non-properties mode.

components/normalizer/src/lib.rs Outdated Show resolved Hide resolved
@Manishearth
Copy link
Member Author

However, the above observation still applies: icu_properties gets compiled when the icu_properties feature isn't set. Why?

It's because I didn't switch to ? deps. I fixed it.

robertbastian
robertbastian previously approved these changes Sep 19, 2024
sffc
sffc previously approved these changes Sep 23, 2024
@younies younies self-requested a review September 23, 2024 20:02
@Manishearth Manishearth merged commit cc3d7f5 into unicode-org:main Sep 23, 2024
28 checks passed
@Manishearth Manishearth deleted the normalizer-ccc branch September 23, 2024 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move CanonicalCombiningClass type out of icu_properties
5 participants