Feature request: add 'language' fields for assets

[billgeo]

Is your feature request related to a problem? Please describe.

So that a user has some additional information about the assets, we want to add the language used in the dataset, and also the character set encoding of the data

Note: after discussion on Slack decided to leave out character set info for now as it is assumed all LINZ data will be in UTF-8. Enforcing this will be done by the Data Management COP / practice lead. Also decided to use IETF codes for languages/locales rather than ISO standards.

Describe the solution you'd like

Language

Location: Assets
Name: linz:language
Description: "The language code and local of the language used in the data. For example, en_NZ for New Zealand English."
Type: "Recommended for internal data. Mandatory for published data."
Restrictions: "Use IETF rfc 5646 for enum list. Precompiled lists 1 and 2"

Character set

~~Location: inside the asset object (can either be in item or collection)
Name: linz:character_set
Description: "Character encoding standard used for the data asset."
Type: "Recommended for internal data. Mandatory for published data."
Restrictions: UTF-8 only ~~

Describe alternatives you've considered
[A clear and concise description of any alternative solutions or features you've considered.]

Additional context
[Add any other context or screenshots about the feature request here.]

[l0b0]

Why those specific standards?

• In my experience IETF language codes are much more common than ISO 639-2.
• IETF language codes allow (but does not mandate) specifying languages as written within different locales, so you can use "en" for English or "en_NZ" for New Zealand English. It doesn't look like ISO 639-2 supports this.
• UTF-8 is much more common than ISO/IEC 10646:2020.
• ISO/IEC 10646:2020 has a full seven encoding schemes.
• ISO standards are expensive and not publicly available.
• "JSON exchange in an open ecosystem must be encoded in UTF-8."

[l0b0]

If we really want to support non-portable JSON we might want to use the word "encoding" rather than "character set".

[billgeo]

@l0b0 just noticed that your PR for this had this as a required property. It's supposed to be optional. Can you please do another PR to resolve that?

[billgeo]

Closing