Skip to content

Commit

Permalink
Never use fallback for baked segmentation data (#4510)
Browse files Browse the repository at this point in the history
Fixes #4489
  • Loading branch information
robertbastian committed Jan 4, 2024
1 parent e8d80bf commit 0c9f657
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 14 deletions.
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
- `icu_datagen`
- Datagen shows elapsed time for keys that are slow to generate (https://github.com/unicode-org/icu4x/pull/4469)
- Datagen performance improvement by caching supported locales (https://github.com/unicode-org/icu4x/pull/4470)
- Never use fallback for baked segmentation data (https://github.com/unicode-org/icu4x/pull/4510)
- `icu_provider`
- (Small breakage) `DataPayload::new_owned()` is no longer `const`, this was a mistake (https://github.com/unicode-org/icu4x/pull/4456)
- `icu_provider_blob`
Expand All @@ -28,7 +29,7 @@
- Changed file extensions for better compatibility with nodejs modules (https://github.com/rust-diplomat/diplomat/pull/387)
- C++
- Fixed a bug where a result header defines a struct that shadows the class' name (https://github.com/rust-diplomat/diplomat/pull/394)
- Add `expclicit` keyword to internal constructors (https://github.com/rust-diplomat/diplomat/pull/386)
- Add `explicit` keyword to internal constructors (https://github.com/rust-diplomat/diplomat/pull/386)
- Utilities
- `calendrical_calculations`:
- Add Keviyah/Four Gates based optimized calculations module for the Hebrew calendar.
Expand Down
4 changes: 3 additions & 1 deletion provider/datagen/src/driver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -438,7 +438,9 @@ impl DatagenDriver {

let transform_duration = instant1.elapsed();

if fallback == FallbackMode::Runtime {
// segmenter uses hardcoded locales internally, so fallback is not necessary.
// TODO(#4511): Use auxiliary keys for segmenter
if fallback == FallbackMode::Runtime && !key.path().get().starts_with("segmenter") {
sink.flush_with_built_in_fallback(key, BuiltInFallbackMode::Standard)
} else {
sink.flush(key)
Expand Down
20 changes: 8 additions & 12 deletions tools/bakeddata-scripts/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -131,18 +131,14 @@ fn main() {
.unwrap();
}

if component == "segmenter" {
// segmenter uses hardcoded locales internally, so fallback is not necessary.
driver.clone().with_fallback_mode(FallbackMode::Hybrid)
} else {
driver.clone()
}
.with_keys(keys.iter().copied())
.export(
&source,
BakedExporter::new(path.join("data"), options).unwrap(),
)
.unwrap();
driver
.clone()
.with_keys(keys.iter().copied())
.export(
&source,
BakedExporter::new(path.join("data"), options).unwrap(),
)
.unwrap();

for file in ["data/any.rs", "data/mod.rs"] {
std::fs::remove_file(path.join(file)).unwrap();
Expand Down

0 comments on commit 0c9f657

Please sign in to comment.