Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change(core): bifurcate normalize_nfd() to use JS native call instead of ICU on wasm 🙀 #11520

Merged
merged 3 commits into from
May 27, 2024

Conversation

srl295
Copy link
Member

@srl295 srl295 commented May 23, 2024

  • does not hit all ICU usage yet, so no removing ICU yet
  • predictably, wasm build fun was the major challenge here.

#9467

@keymanapp-test-bot skip

@srl295 srl295 self-assigned this May 23, 2024
@keymanapp-test-bot
Copy link

keymanapp-test-bot bot commented May 23, 2024

User Test Results

Test specification and instructions

User tests are not required

@keymanapp-test-bot keymanapp-test-bot bot added this to the A18S2 milestone May 23, 2024
@github-actions github-actions bot added core/ Keyman Core feat labels May 23, 2024
@srl295 srl295 marked this pull request as ready for review May 23, 2024 23:49
EM_JS(char*, NormalizeNFD, (const char* input), {
if (!input) return input; // pass through null
const instr = Module.UTF8ToString(input);
const nfd = instr.normalize("NFD");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what is happening here. Is ln 21 equivalent to icu::Normalizer2::getNFDInstance(status);
where is the ln 21 normalised defined?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it's the JS idiom for the same thing. Ends up calling icu..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So... this line is doing the JS call? I don't see a definition for instr, and I'd like to learn more about whatever interface there is that would allow WASM to make native-JS calls.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire EM_JS block is Javascript.

I don't see a definition for instr

It's on line 20.

I'd like to learn more about whatever interface there is that would allow WASM to make native-JS calls.

https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html#interacting-with-code-call-javascript-from-native

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Emscripten, beware, it's a deep rabbit hole.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a definition for instr

It's on line 20.

Kinda passes the buck off to Module, though. It's at least referenced through the link you provided - thanks! Kinda odd how it just assumes Module exists and doesn't talk about it, though.

In case I return to this comment in the future, or someone else wants a reference on it... https://emscripten.org/docs/api_reference/module.html#other-methods

I'm still not clear on where exactly UTF8ToString comes from, but I'm willing to handwave that tidbit away for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's all part of the Emscripten tooling. It's a little magical when you first encounter it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Module is a JavaScript global at that point.

Yeah it's mostly glue and one call. But it will eventually save us a lot in linked libraries.

Copy link
Contributor

@rc-swag rc-swag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@srl295 srl295 requested a review from jahorton May 24, 2024 04:34
@mcdurdin mcdurdin modified the milestones: A18S2, A18S3 May 24, 2024
Base automatically changed from feat/core/9467-devolve-regex-to-js-epic-ldml to master May 27, 2024 15:25
@srl295 srl295 merged commit 7d9b1a1 into master May 27, 2024
3 checks passed
@srl295 srl295 deleted the feat/core/9467-devolve-norm-to-js2-epic-ldml branch May 27, 2024 15:26
@srl295 srl295 changed the title feat(core): change normalize_nfd() to use JS native call instead of ICU on wasm 🙀 change(core): bifurcate normalize_nfd() to use JS native call instead of ICU on wasm 🙀 May 27, 2024
@srl295
Copy link
Member Author

srl295 commented May 27, 2024

updated title after merge 🤦

@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

6 similar comments
@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core/ Keyman Core feat
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants