Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Locale.fromSubtags and support for scriptCode. #6518

Merged
merged 27 commits into from
Oct 29, 2018
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e042526
Add Locale.fromComponents.
hugovdm Oct 12, 2018
4013600
Change toString from underscores to dashes. Expand the unit tests.
hugovdm Oct 12, 2018
813998d
Rename 'fromComponents' to 'create'. Change variants from String to L…
hugovdm Oct 15, 2018
fe5e110
Use default for language parameter. Use hashCode/hashList.
hugovdm Oct 15, 2018
fb72be0
Have toString() stick with old (underscore) behaviour.
hugovdm Oct 15, 2018
c6a8b96
Demonstrate empty-list bug in assert code.
hugovdm Oct 15, 2018
af40b72
Fix empty-list assert bug.
hugovdm Oct 15, 2018
488cf31
Add ignores for lint issues. Unsure about 71340 though.
hugovdm Oct 15, 2018
829af35
Fix operator== via _listEquals.
hugovdm Oct 16, 2018
c749663
Remove length-checking asserts: we're anyway not checking characters …
hugovdm Oct 16, 2018
04c52e9
Documentation update.
hugovdm Oct 16, 2018
58bfdc6
Change reasoning for ignore:prefer_initializing_formals.
hugovdm Oct 16, 2018
bc4cc07
Try 'fromSubtags' as new constructor name.
hugovdm Oct 17, 2018
a8b2797
Documentation improvements based on Pull Request review.
hugovdm Oct 17, 2018
d6f06f5
Assert-fail for invalid-length subtags and drop bad subtags in produc…
hugovdm Oct 18, 2018
8948502
Revert "Assert-fail for invalid-length subtags and drop bad subtags i…
hugovdm Oct 19, 2018
20e8bd7
Re-fix Locale.toString() for variants=[].
hugovdm Oct 19, 2018
9911dfd
Tear out variants, in case we want to have one fewer pointer in the f…
hugovdm Oct 19, 2018
cb81b55
Make named parameters' names consistent with member names.
hugovdm Oct 19, 2018
5e0def8
Also remove _listEquals: no longer in use.
hugovdm Oct 22, 2018
9f6e3e0
Merge branch 'master' into fromComponents
hugovdm Oct 23, 2018
97ab4fe
Lint fix.
hugovdm Oct 23, 2018
5f625ae
Fix code review nits.
hugovdm Oct 24, 2018
3b87068
Lint fix for assert, and a couple more not-zero-length-string asserts.
hugovdm Oct 24, 2018
df46762
Code Review: two of three nits addressed...
hugovdm Oct 26, 2018
57b068e
Review fix: change 'should' to 'must' for subtag prescriptions.
hugovdm Oct 26, 2018
6f07e8d
Assert-check that countryCode is never ''.
hugovdm Oct 29, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 94 additions & 25 deletions lib/ui/window.dart
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,11 @@ class WindowPadding {
}
}

/// An identifier used to select a user's language and formatting preferences,
/// consisting of a language and a country. This is a subset of locale
/// identifiers as defined by BCP 47.
/// An identifier used to select a user's language and formatting preferences.
///
/// This represents a [Unicode Language
hugovdm marked this conversation as resolved.
Show resolved Hide resolved
/// Identifier](https://www.unicode.org/reports/tr35/#Unicode_language_identifier)
/// (i.e. without Locale extensions), except variants are not supported.
///
/// Locales are canonicalized according to the "preferred value" entries in the
/// [IANA Language Subtag
Expand All @@ -145,16 +147,57 @@ class Locale {
/// The primary language subtag must not be null. The region subtag is
/// optional.
///
/// The values are _case sensitive_, and should match the case of the relevant
/// subtags in the [IANA Language Subtag
/// Registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry).
/// Typically this means the primary language subtag should be lowercase and
/// the region subtag should be uppercase.
const Locale(this._languageCode, [ this._countryCode ]) : assert(_languageCode != null), assert(_languageCode != '');
/// The subtag values are _case sensitive_ and must be one of the valid
/// subtags according to CLDR supplemental data:
/// [language](http://unicode.org/cldr/latest/common/validity/language.xml),
/// [region](http://unicode.org/cldr/latest/common/validity/region.xml). The
/// primary language subtag must be at least two and at most eight lowercase
/// letters, but not four letters. The region region subtag must be two
/// uppercase letters or three digits. See the [Unicode Language
/// Identifier](https://www.unicode.org/reports/tr35/#Unicode_language_identifier)
/// specification.
///
/// Validity is not checked by default, but some methods may throw away
/// invalid data.
///
/// See also:
///
/// * [new Locale.fromSubtags], which also allows a [scriptCode] to be
/// specified.
const Locale(
this._languageCode, [
this._countryCode,
]) : assert(_languageCode != null),
assert(_languageCode != ''),
scriptCode = null;

/// Creates a new Locale object.
///
/// The keyword arguments specify the subtags of the Locale.
///
/// The subtag values are _case sensitive_ and must be valid subtags according
/// to CLDR supplemental data:
/// [language](http://unicode.org/cldr/latest/common/validity/language.xml),
/// [script](http://unicode.org/cldr/latest/common/validity/script.xml) and
/// [region](http://unicode.org/cldr/latest/common/validity/region.xml) for
/// each of languageCode, scriptCode and countryCode respectively.
///
/// Validity is not checked by default, but some methods may throw away
/// invalid data.
const Locale.fromSubtags({
String languageCode = 'und',
this.scriptCode,
String countryCode,
}) : assert(languageCode != null),
assert(languageCode != ''),
_languageCode = languageCode,
assert(scriptCode != ''),
assert(countryCode != ''),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add this assert to the other constructor too while we're at it

Copy link
Contributor Author

@hugovdm hugovdm Oct 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this I then manage to bring us back to the unit test that I disliked:

expect(const Locale('en'), isNot(new Locale('en', '')));

The existing Locale constructor is being tested to support both null and '', and furthermore being tested to treat these differently. Mark mentioned to me yesterday "in other languages it's been found most convenient to return empty strings for omitted subtags rather than null". I suggest Dart is quite null-happy and thus might not have the same concerns. Either way, that data point and the idea to forbid '' both speak in favour of, at the very least, not treating '' and null differently?

So: in-scope or out of scope for this Pull Request to change existing behaviour? I'm happy to consider this via a follow-up change too...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine to remove support for '' as a value here IMHO (i.e. fix the test to verify that the empty string throws). No valid use of this API could have done that before. Having multiple ways to express the same value seems like a likely source of bugs (e.g. they'd have different hash codes presumably, unless we went out of our way to have them have the same hash code, at which point we're just wasting cycles on something that handled essentially for free by just banning the empty string).

null meaning "absent" is idiomatic in Dart. The empty string in Dart doesn't represent absence, it represents presence with a particular value (which happens to be the empty string, but could as easily be any other string).

Copy link
Contributor Author

@hugovdm hugovdm Oct 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the assert and removed what should have been a failing unit test, though I'm not sure how to actually trigger a failure. Looks like asserts are dropped (frontend_server.dart.snapshot requires no-asserts).

From an attempt to add --enable-asserts to the dart invocation for locale_test.dart:

+ run_test flutter/testing/dart/locale_test.dart
+ out/host_debug_unopt/dart --enable-asserts out/host_debug_unopt/gen/frontend_server.dart.snapshot --sdk-root out/host_debug_unopt/flutter_patched_sdk --incremental --strong --target=flutter --packages flutter/testing/dart/.packages --output-dill out/host_debug_unopt/engine_test.dill flutter/testing/dart/locale_test.dart
Snapshot not compatible with the current VM configuration: the snapshot requires 'release strong no-type_checks no-asserts no-error_on_bad_type sync_async reify_generic_functions use_field_guards use_osr x64-sysv' but the VM has 'release strong no-type_checks asserts no-error_on_bad_type sync_async reify_generic_functions use_field_guards use_osr x64-sysv'
Isolate creation failed

I'd need to look into how one might tweak the snapshot later.

_countryCode = countryCode;

/// The primary language subtag for the locale.
///
/// This must not be null.
/// This must not be null. It may be 'und', representing 'undefined'.
///
/// This is expected to be string registered in the [IANA Language Subtag
/// Registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry)
Expand All @@ -166,10 +209,19 @@ class Locale {
/// Locale('he')` and `const Locale('iw')` are equal, and both have the
/// [languageCode] `he`, because `iw` is a deprecated language subtag that was
/// replaced by the subtag `he`.
String get languageCode => _canonicalizeLanguageCode(_languageCode);
///
/// This must be a valid Unicode Language subtag as listed in [Unicode CLDR
/// supplemental
/// data](http://unicode.org/cldr/latest/common/validity/language.xml).
hugovdm marked this conversation as resolved.
Show resolved Hide resolved
///
/// See also:
///
/// * [new Locale.fromSubtags], which describes the conventions for creating
/// [Locale] objects.
String get languageCode => _replaceDeprecatedLanguageSubtag(_languageCode);
final String _languageCode;

static String _canonicalizeLanguageCode(String languageCode) {
static String _replaceDeprecatedLanguageSubtag(String languageCode) {
// This switch statement is generated by //flutter/tools/gen_locale.dart
// Mappings generated for language subtag registry as of 2018-08-08.
switch (languageCode) {
Expand Down Expand Up @@ -255,9 +307,23 @@ class Locale {
}
}

/// The script subtag for the locale.
///
/// This may be null, indicating that there is no specified script subtag.
///
/// This must be a valid Unicode Language Identifier script subtag as listed
/// in [Unicode CLDR supplemental
/// data](http://unicode.org/cldr/latest/common/validity/script.xml).
///
/// See also:
///
/// * [new Locale.fromSubtags], which describes the conventions for creating
/// [Locale] objects.
final String scriptCode;

/// The region subtag for the locale.
///
/// This can be null.
/// This may be null, indicating that there is no specified region subtag.
///
/// This is expected to be string registered in the [IANA Language Subtag
/// Registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry)
Expand All @@ -269,10 +335,15 @@ class Locale {
/// 'DE')` and `const Locale('de', 'DD')` are equal, and both have the
/// [countryCode] `DE`, because `DD` is a deprecated language subtag that was
/// replaced by the subtag `DE`.
String get countryCode => _canonicalizeRegionCode(_countryCode);
///
/// See also:
///
/// * [new Locale.fromSubtags], which describes the conventions for creating
/// [Locale] objects.
String get countryCode => _replaceDeprecatedRegionSubtag(_countryCode);
final String _countryCode;

static String _canonicalizeRegionCode(String regionCode) {
static String _replaceDeprecatedRegionSubtag(String regionCode) {
// This switch statement is generated by //flutter/tools/gen_locale.dart
// Mappings generated for language subtag registry as of 2018-08-08.
switch (regionCode) {
Expand All @@ -294,23 +365,21 @@ class Locale {
return false;
final Locale typedOther = other;
return languageCode == typedOther.languageCode
&& scriptCode == typedOther.scriptCode
&& countryCode == typedOther.countryCode;
}

@override
int get hashCode {
int result = 373;
result = 37 * result + languageCode.hashCode;
if (_countryCode != null)
result = 37 * result + countryCode.hashCode;
return result;
}
int get hashCode => hashValues(languageCode, scriptCode, countryCode);

@override
String toString() {
hugovdm marked this conversation as resolved.
Show resolved Hide resolved
if (_countryCode == null)
return languageCode;
return '${languageCode}_$countryCode';
final StringBuffer out = StringBuffer(languageCode);
if (scriptCode != null)
out.write('_$scriptCode');
if (_countryCode != null)
out.write('_$countryCode');
return out.toString();
}
}

Expand Down
26 changes: 26 additions & 0 deletions testing/dart/locale_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,30 @@ void main() {
expect(const Locale('iw', 'DD').toString(), 'he_DE');
expect(const Locale('iw', 'DD'), const Locale('he', 'DE'));
});

test('Locale.fromSubtags', () {
expect(const Locale.fromSubtags().languageCode, 'und');
expect(const Locale.fromSubtags().scriptCode, null);
expect(const Locale.fromSubtags().countryCode, null);

expect(const Locale.fromSubtags(languageCode: 'en').toString(), 'en');
expect(const Locale.fromSubtags(languageCode: 'en').languageCode, 'en');
expect(const Locale.fromSubtags(scriptCode: 'Latn').toString(), 'und_Latn');
expect(const Locale.fromSubtags(scriptCode: 'Latn').scriptCode, 'Latn');
expect(const Locale.fromSubtags(countryCode: 'US').toString(), 'und_US');
expect(const Locale.fromSubtags(countryCode: 'US').countryCode, 'US');

expect(Locale.fromSubtags(languageCode: 'es', countryCode: '419').toString(), 'es_419');
expect(Locale.fromSubtags(languageCode: 'es', countryCode: '419').languageCode, 'es');
expect(Locale.fromSubtags(languageCode: 'es', countryCode: '419').countryCode, '419');

expect(Locale.fromSubtags(languageCode: 'zh', scriptCode: 'Hans', countryCode: 'CN').toString(), 'zh_Hans_CN');
});

test('Locale equality', () {
expect(Locale.fromSubtags(languageCode: 'en'),
isNot(Locale.fromSubtags(languageCode: 'en', scriptCode: 'Latn')));
expect(Locale.fromSubtags(languageCode: 'en').hashCode,
isNot(Locale.fromSubtags(languageCode: 'en', scriptCode: 'Latn').hashCode));
});
}