-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make UTS 46 normalization non-experimental #4712
Merged
Merged
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
d18887c
Make UTS 46 normalization non-experimental
hsivonen 1c14305
Merge branch 'main' into uts46bis
hsivonen 7d7ef86
Regenerate normalization TOML files with the copyright flag
hsivonen dbe1066
Merge branch 'main' into uts46bis
hsivonen 0c492e4
Fix UTS 46 tests
hsivonen 0707223
Merge branch 'main' into uts46bis
hsivonen 947ca28
Linkify URL in docs
hsivonen 15a084e
Mark UTS 46 as not in FFI
hsivonen f773c22
Remove the special case for 5 compatibility ideographs and tweak docs
hsivonen e791f64
Merge branch 'nocompatideographspecialcase' into uts46bis
hsivonen 625031d
Fix a failed merge
hsivonen ef76004
Remove unused "use"
hsivonen f2c5cec
Merge branch 'main' into uts46bis
hsivonen 7c4a132
Merge branch 'main' into uts46bis
hsivonen 6b57b49
Merge branch 'main' into uts46bis
hsivonen 3a07fe7
Merge branch 'main' into uts46bis
hsivonen 608fdfe
Merge branch 'main' into uts46bis
hsivonen 0e9ef52
Per clippy impl Default for Uts46Mapper
hsivonen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
question: why is the default value for behavior of handling an ignored character not to ignore it (
IgnorableBehavior::Ignored
)?IgnorableBehavior::Ignored
is what UTS 46 would do by default, right?Or do you default to
Unsupported
because this is in the underlyingDecomposition
struct, which can be used independently of UTS 46, and thus would want to be kept separate by default unless otherwise specified?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The underlying struct can be used for non-UTS46 purposes: the usual normalizations. For those, we expose contiguous-buffer APIs, and the contiguous-buffer versions currently can't deal with ignorables, and since there is currently no use case for them to deal with ignorables, I think it's better to have a state that they can
debug_assert!
against than to add support for ignorables is the contiguous-buffer entry points.