Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document invariants in LocaleFallbacker #4414

Merged
merged 2 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion components/locid_transform/src/fallback/algorithms.rs
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ mod tests {
expected_region_chain: &["en-US-u-sd-usca", "en-US", "und-US-u-sd-usca", "und-US"],
},
TestCase {
// NOTE: -u-rg is not yet supported; when it is, this test should be updated
// TODO(#4413): -u-rg is not yet supported; when it is, this test should be updated
input: "en-u-rg-gbxxxx",
requires_data: false,
extension_key: None,
Expand Down
70 changes: 35 additions & 35 deletions components/locid_transform/src/fallback/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,40 +4,6 @@

//! Tools for locale fallback, enabling arbitrary input locales to be mapped into the nearest
//! locale with data.
//!
//! The algorithm implemented in this module is called [Flexible Vertical Fallback](
//! https://docs.google.com/document/d/1Mp7EUyl-sFh_HZYgyeVwj88vJGpCBIWxzlCwGgLCDwM/edit).
//! Watch [#2243](https://github.com/unicode-org/icu4x/issues/2243) to track improvements to
//! this algorithm and steps to enshrine the algorithm in CLDR.
//!
//! # Examples
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example was duplicated below.

//!
//! ```
//! use icu_locid::locale;
//! use icu_locid_transform::LocaleFallbacker;
//!
//! // Set up a LocaleFallbacker with data.
//! let fallbacker = LocaleFallbacker::new();
//!
//! // Create a LocaleFallbackerIterator with a default configuration.
//! // By default, uses language priority with no additional extension keywords.
//! let mut fallback_iterator = fallbacker
//! .for_config(Default::default())
//! .fallback_for(locale!("hi-Latn-IN").into());
//!
//! // Run the algorithm and check the results.
//! assert_eq!(fallback_iterator.get(), &locale!("hi-Latn-IN").into());
//! fallback_iterator.step();
//! assert_eq!(fallback_iterator.get(), &locale!("hi-Latn").into());
//! fallback_iterator.step();
//! assert_eq!(fallback_iterator.get(), &locale!("en-IN").into());
//! fallback_iterator.step();
//! assert_eq!(fallback_iterator.get(), &locale!("en-001").into());
//! fallback_iterator.step();
//! assert_eq!(fallback_iterator.get(), &locale!("en").into());
//! fallback_iterator.step();
//! assert_eq!(fallback_iterator.get(), &locale!("und").into());
//! ```

use crate::provider::*;
use icu_locid::extensions::unicode::Value;
Expand All @@ -52,9 +18,17 @@ mod algorithms;
/// Implements the algorithm defined in *[UTS #35: Locale Inheritance and Matching]*.
///
/// Note that this implementation performs some additional steps compared to the *UTS #35*
/// algorithm, see *[the design doc]* for a detailed description, and [#2243](
/// algorithm. See *[the design doc]* for a detailed description and [#2243](
/// https://github.com/unicode-org/icu4x/issues/2243) to track aligment with *UTS #35*.
///
/// If running fallback in a loop, use [`DataLocale::is_und()`] to break from the loop.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about cases where und-Script is distinct from und?

///
/// # Algorithm Invariants
///
/// 1. The [language identifier] will eventually reach `und`.
/// 2. The Unicode extension keywords will eventually be removed.
/// 3. The fallback chain requires context from the original input.
///
/// # Examples
///
/// ```
Expand Down Expand Up @@ -84,8 +58,34 @@ mod algorithms;
/// assert_eq!(fallback_iterator.get(), &locale!("und").into());
/// ```
///
/// Unicode extension keywords take part in fallback, but [auxiliary keys]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not put this here until aux keys are stabilized?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll merge this PR because I think we agree that it is a step in the right direction given the current state of how aux keys work. Users can click the link and read that aux keys are experimental. I think we'll need to change the machinery in 2.0.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We decided to not merge this in #4412 (comment).

/// are not modified:
///
/// ```
/// use icu_locid::locale;
/// use icu_locid_transform::LocaleFallbacker;
///
/// let fallbacker = LocaleFallbacker::new();
/// let mut fallback_iterator = fallbacker
/// .for_config(Default::default())
/// .fallback_for("en-US-u-sd-usca-x-aux".parse().unwrap());
///
/// assert_eq!(fallback_iterator.get().to_string(), "en-US-u-sd-usca-x-aux");
/// fallback_iterator.step();
/// assert_eq!(fallback_iterator.get().to_string(), "en-US-x-aux");
/// fallback_iterator.step();
/// assert_eq!(fallback_iterator.get().to_string(), "en-u-sd-usca-x-aux");
/// fallback_iterator.step();
/// assert_eq!(fallback_iterator.get().to_string(), "en-x-aux");
/// fallback_iterator.step();
/// assert_eq!(fallback_iterator.get().to_string(), "und-x-aux");
/// assert!(fallback_iterator.get().is_und());
/// ```
///
/// [UTS #35: Locale Inheritance and Matching]: https://www.unicode.org/reports/tr35/#Locale_Inheritance
/// [the design doc]: https://docs.google.com/document/d/1Mp7EUyl-sFh_HZYgyeVwj88vJGpCBIWxzlCwGgLCDwM/edit
/// [auxiliary keys]: icu_provider::AuxiliaryKeys
/// [language identifier]: icu_locid::LanguageIdentifier
#[doc(hidden)]
#[derive(Debug, Clone, PartialEq)]
pub struct LocaleFallbacker {
Expand Down