From 7bde155605766cc243f91595891511481637464c Mon Sep 17 00:00:00 2001 From: Daniel Loreto <279789+loreto@users.noreply.github.com> Date: Wed, 10 Apr 2024 13:13:28 -0500 Subject: [PATCH] [typeid] Allow '_' in type prefix, update to v0.3 --- typeid/typeid/README.md | 80 +++++++++++++++++---------------- typeid/typeid/spec/README.md | 43 +++++++++--------- typeid/typeid/spec/invalid.json | 15 ++++--- typeid/typeid/spec/invalid.yml | 19 ++++++-- typeid/typeid/spec/valid.json | 6 +++ typeid/typeid/spec/valid.yml | 9 +++- 6 files changed, 102 insertions(+), 70 deletions(-) diff --git a/typeid/typeid/README.md b/typeid/typeid/README.md index c4adebe..8530556 100644 --- a/typeid/typeid/README.md +++ b/typeid/typeid/README.md @@ -11,7 +11,7 @@ in Stripe's APIs. TypeIDs are canonically encoded as lowercase strings consisting of three parts: -1. A type prefix (at most 63 characters in all lowercase ASCII [a-z]) +1. A type prefix (at most 63 characters in all lowercase snake_case ASCII [a-z_]). 2. An underscore '\_' separator 3. A 128-bit UUIDv7 encoded as a 26-character string using a modified base32 encoding. @@ -27,50 +27,52 @@ A [formal specification](./spec) defines the encoding in more detail. ## Benefits -- **Type-safe:** you can't accidentally use a `user` ID where a `post` ID is expected. When debugging, you can - immediately understand what type of entity a TypeID refers to thanks to the type prefix. -- **Compatible with UUIDs:** TypeIDs are a superset of UUIDs. They are based on the upcoming [UUIDv7 standard](https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html#name-uuid-version-7). If you decode the TypeID and remove the type information, you get a valid UUIDv7. -- **K-Sortable**: TypeIDs are K-sortable and can be used as the primary key in a database while ensuring good - locality. Compare to entirely random global ids, like UUIDv4, that generally suffer from poor database locality. -- **Thoughtful encoding**: the base32 encoding is URL safe, case-insensitive, avoids ambiguous characters, can be - selected for copy-pasting by double-clicking, and is a more compact encoding than the traditional hex encoding used by UUIDs (26 characters vs 36 characters). +- **Type-safe:** you can't accidentally use a `user` ID where a `post` ID is expected. When debugging, you can + immediately understand what type of entity a TypeID refers to thanks to the type prefix. +- **Compatible with UUIDs:** TypeIDs are a superset of UUIDs. They are based on the upcoming [UUIDv7 standard](https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html#name-uuid-version-7). If you decode the TypeID and remove the type information, you get a valid UUIDv7. +- **K-Sortable**: TypeIDs are K-sortable and can be used as the primary key in a database while ensuring good + locality. Compare to entirely random global ids, like UUIDv4, that generally suffer from poor database locality. +- **Thoughtful encoding**: the base32 encoding is URL safe, case-insensitive, avoids ambiguous characters, can be + selected for copy-pasting by double-clicking, and is a more compact encoding than the traditional hex encoding used by UUIDs (26 characters vs 36 characters). ## Implementations Implementations should adhere to the formal [specification](./spec). +Latest spec version: v0.3.0 + ### Official Implementations by `jetify` -| Language | Status | -| ----------------------------------------------------- | ------------- | -| [Go](https://github.com/jetify-com/typeid-go) | ✓ Implemented | -| [SQL](https://github.com/jetify-com/typeid-sql) | ✓ Implemented | -| [TypeScript](https://github.com/jetify-com/typeid-js) | ✓ Implemented | +| Language | Status | Spec Version | +| ----------------------------------------------------- | ------------- | ------------ | +| [Go](https://github.com/jetify-com/typeid-go) | ✓ Implemented | v0.2 | +| [SQL](https://github.com/jetify-com/typeid-sql) | ✓ Implemented | v0.2 | +| [TypeScript](https://github.com/jetify-com/typeid-js) | ✓ Implemented | v0.2 | ### Community Provided Implementations -| Language | Author | Validated Against Spec? | -| ------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------- | -| [C# (.Net)](https://github.com/TenCoKaciStromy/typeid-dotnet) | [@TenCoKaciStromy](https://github.com/TenCoKaciStromy) | Yes, on 2023-06-30 | -| [C# (.Net Standard 2.1)](https://github.com/cbuctok/typeId) | [@cbuctok](https://github.com/cbuctok) | Yes, on 2023-07-03 | -| [C# (.NET)](https://github.com/firenero/TypeId) | [@firenero](https://github.com/firenero) | Yes, on 2023-07-15 | -| [Dart](https://github.com/TBD54566975/typeid-dart) | [@mistermoe](https://github.com/mistermoe) [@tbd54566975](https://github.com/tbd54566975) | Yes, on 2024-03-25 | -| [Elixir](https://github.com/sloanelybutsurely/typeid-elixir) | [@sloanelybutsurely](https://github.com/sloanelybutsurely) | Yes, on 2023-07-02 | -| [Haskell](https://github.com/MMZK1526/mmzk-typeid) | [@MMZK1526](https://github.com/MMZK1526) | Yes, on 2023-07-07 | -| [Java](https://github.com/fxlae/typeid-java) | [@fxlae](https://github.com/fxlae) | Yes, on 2023-07-02 | -| [Java](https://github.com/softprops/typeid-java) | [@softprops](https://github.com/softprops) | Yes, on 2023-07-04 | -| [OCaml](https://github.com/titouancreach/typeid-ocaml) | [@titouancreach](https://github.com/titouancreach) | Yes, on 2024-03-07 | -| [PHP](https://github.com/BombenProdukt/typeid) | [@BombenProdukt](https://github.com/BombenProdukt) | Yes, on 2023-07-03 | -| [Python](https://github.com/akhundMurad/typeid-python) | [@akhundMurad](https://github.com/akhundMurad) | Yes, on 2023-06-30 | -| [Ruby](https://github.com/broothie/typeid-ruby) | [@broothie](https://github.com/broothie) | Yes, on 2023-06-30 | -| [Rust](https://github.com/conradludgate/type-safe-id) | [@conradludgate](https://github.com/conradludgate) | Yes, on 2023-07-01 | -| [Rust](https://github.com/johnnynotsolucky/strong_id) | [@johnnynotsolucky](https://github.com/johnnynotsolucky) | Yes, on 2023-07-13 | -| [Scala](https://github.com/ant8e/uuid4cats-effect) | [@ant8e](https://github.com/ant8e) | Yes, on 2023-07-14 | -| [Scala](https://github.com/guizmaii-opensource/zio-uuid) | [@guizmaii](https://github.com/guizmaii) | Not yet | -| [Swift](https://github.com/Frizlab/swift-typeid) | [@Frizlab](https://github.com/Frizlab) | Yes, on 2023-07-07 | -| [T-SQL](https://github.com/uniteeio/typeid_tsql) | [@uniteeio](https://github.com/uniteeio) | Yes, on 2023-08-25 | -| [TypeScript](https://github.com/ongteckwu/typeid-ts) | [@ongteckwu](https://github.com/ongteckwu) | Yes, on 2023-06-30 | -| [Zig](https://github.com/tensorush/zig-typeid) | [@tensorush](https://github.com/tensorush) | Yes, on 2023-07-05 | +| Language | Author | Spec Version | +| ------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ------------------ | +| [C# (.Net)](https://github.com/TenCoKaciStromy/typeid-dotnet) | [@TenCoKaciStromy](https://github.com/TenCoKaciStromy) | v0.2 on 2023-06-30 | +| [C# (.Net Standard 2.1)](https://github.com/cbuctok/typeId) | [@cbuctok](https://github.com/cbuctok) | v0.2 on 2023-07-03 | +| [C# (.NET)](https://github.com/firenero/TypeId) | [@firenero](https://github.com/firenero) | v0.2 on 2023-07-15 | +| [Dart](https://github.com/TBD54566975/typeid-dart) | [@mistermoe](https://github.com/mistermoe) [@tbd54566975](https://github.com/tbd54566975) | v0.2 on 2024-03-25 | +| [Elixir](https://github.com/sloanelybutsurely/typeid-elixir) | [@sloanelybutsurely](https://github.com/sloanelybutsurely) | v0.2 on 2023-07-02 | +| [Haskell](https://github.com/MMZK1526/mmzk-typeid) | [@MMZK1526](https://github.com/MMZK1526) | v0.2 on 2023-07-07 | +| [Java](https://github.com/fxlae/typeid-java) | [@fxlae](https://github.com/fxlae) | v0.2 on 2023-07-02 | +| [Java](https://github.com/softprops/typeid-java) | [@softprops](https://github.com/softprops) | v0.2 on 2023-07-04 | +| [OCaml](https://github.com/titouancreach/typeid-ocaml) | [@titouancreach](https://github.com/titouancreach) | v0.2 on 2024-03-07 | +| [PHP](https://github.com/BombenProdukt/typeid) | [@BombenProdukt](https://github.com/BombenProdukt) | v0.2 on 2023-07-03 | +| [Python](https://github.com/akhundMurad/typeid-python) | [@akhundMurad](https://github.com/akhundMurad) | v0.2 on 2023-06-30 | +| [Ruby](https://github.com/broothie/typeid-ruby) | [@broothie](https://github.com/broothie) | v0.2 on 2023-06-30 | +| [Rust](https://github.com/conradludgate/type-safe-id) | [@conradludgate](https://github.com/conradludgate) | v0.2 on 2023-07-01 | +| [Rust](https://github.com/johnnynotsolucky/strong_id) | [@johnnynotsolucky](https://github.com/johnnynotsolucky) | v0.2 on 2023-07-13 | +| [Scala](https://github.com/ant8e/uuid4cats-effect) | [@ant8e](https://github.com/ant8e) | v0.2 on 2023-07-14 | +| [Scala](https://github.com/guizmaii-opensource/zio-uuid) | [@guizmaii](https://github.com/guizmaii) | Not validated yet | +| [Swift](https://github.com/Frizlab/swift-typeid) | [@Frizlab](https://github.com/Frizlab) | v0.2 on 2023-07-07 | +| [T-SQL](https://github.com/uniteeio/typeid_tsql) | [@uniteeio](https://github.com/uniteeio) | v0.2 on 2023-08-25 | +| [TypeScript](https://github.com/ongteckwu/typeid-ts) | [@ongteckwu](https://github.com/ongteckwu) | v0.2 on 2023-06-30 | +| [Zig](https://github.com/tensorush/zig-typeid) | [@tensorush](https://github.com/tensorush) | v0.2 on 2023-07-05 | We are looking for community contributions to implement TypeIDs in other languages. @@ -106,10 +108,10 @@ prefix_01h2xcejqtf2nbrexx3vqjhp41 ## Related Work -- [UUIDv7](https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html#name-uuid-version-7) - The upcoming UUID standard that TypeIDs are based on. +- [UUIDv7](https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html#name-uuid-version-7) - The upcoming UUID standard that TypeIDs are based on. Alternatives to UUIDv7 that are also worth considering (but not type-safe like TypeIDs): -- [xid](https://github.com/rs/xid) -- [ulid](https://github.com/ulid) -- [ksuid](https://github.com/segmentio/ksuid) +- [xid](https://github.com/rs/xid) +- [ulid](https://github.com/ulid) +- [ksuid](https://github.com/segmentio/ksuid) diff --git a/typeid/typeid/spec/README.md b/typeid/typeid/spec/README.md index 61b8003..7f298b7 100644 --- a/typeid/typeid/spec/README.md +++ b/typeid/typeid/spec/README.md @@ -1,4 +1,4 @@ -# TypeID Specification (Version 0.2.0) +# TypeID Specification (Version 0.3.0) ## Overview @@ -19,23 +19,26 @@ This document formalizes the specification for TypeIDs. A typeid consists of three parts: 1. A **type prefix**: a string denoting the type of the ID. The prefix should be - at most 63 characters in all lowercase ASCII [a-z]. -1. A **separator**: an underscore '\_' character. + at most 63 characters in all lowercase snake_case ASCII [a-z_]. +1. A **separator**: an underscore '\_' character. The separator is omitted if the prefix is empty. 1. A **UUID suffix**: a 128-bit UUIDv7 encoded as a 26-character string in base32. ### Type Prefix -A type prefix is a string denoting the type of the ID. The prefix should be at most -63 characters in all lowercase ASCII [a-z]. Valid prefixes should match the following -regex: `[a-z]{0,63}`. +A type prefix is a string denoting the type of the ID. +The prefix must: + +- Contain at most 63 characters. +- May be empty. If it's not empty, it must contain only lowercase alphabetic ASCII characters [a-z], or an underscore '\_'. +- If the prefix is not empty, it must start and end with an alphabetic character [a-z] (underscores are not allowed at the beginning or end of the string). + +Valid prefixes match the following +regex: `^([a-z][a-z_]*[a-z])?$`. The empty string is a valid prefix, it's there for very specific use cases in which applications need to encode a typeid but elide the type information. In general though, applications should use a prefix that is at least 3 characters long. -> Note: [There's a proposal](https://github.com/jetify-com/typeid/issues/7) to add `_` as -> an allowed separator within type prefixes. - ### Separator The separator is a single underscore character `_`. If the prefix is empty, the separator @@ -96,20 +99,18 @@ other UUID variants like UUIDv1 or UUIDv4 at their discretion. This spec uses semantic versioning: `MAJOR.MINOR.PATCH`. The version is incremented when the spec changes in a way that is not backwards compatible. -Libraries that implement this spec should also use semantic versioning, and their -MAJOR and MINOR versions should match the version of the spec they implement. -The PATCH version is up to the discretion of the library author. +Libraries that implement this spec should also use semantic versioning. ## Validating Implementations To assist library authors in validating their implementations, we provide: -- A reference implementation in [Go](https://github.com/jetify-com/typeid-go) - with extensive testing. -- A [valid.yml](valid.yml) file containing a list of valid typeids along - with their corresponding decoded UUIDs. For convienience, we also provide - a [valid.json](valid.json) file containing the same data in JSON format. -- An [invalid.yml](invalid.yml) file containing a list of strings that are - invalid typeids and should fail to parse/decode. For convienience, we also - provide a [invalid.json](invalid.json) file containing the same data in - JSON format. +- A reference implementation in [Go](https://github.com/jetify-com/typeid-go) + with extensive testing. +- A [valid.yml](valid.yml) file containing a list of valid typeids along + with their corresponding decoded UUIDs. For convienience, we also provide + a [valid.json](valid.json) file containing the same data in JSON format. +- An [invalid.yml](invalid.yml) file containing a list of strings that are + invalid typeids and should fail to parse/decode. For convienience, we also + provide a [invalid.json](invalid.json) file containing the same data in + JSON format. diff --git a/typeid/typeid/spec/invalid.json b/typeid/typeid/spec/invalid.json index aa1ca6b..0f021c7 100644 --- a/typeid/typeid/spec/invalid.json +++ b/typeid/typeid/spec/invalid.json @@ -14,11 +14,6 @@ "typeid": "pre.fix_00000000000000000000000000", "description": "The prefix can't have symbols, it needs to be alphabetic" }, - { - "name": "prefix-underscore", - "typeid": "pre_fix_00000000000000000000000000", - "description": "The prefix can't have symbols, it needs to be alphabetic" - }, { "name": "prefix-non-ascii", "typeid": "préfix_00000000000000000000000000", @@ -88,5 +83,15 @@ "name": "suffix-overflow", "typeid": "prefix_8zzzzzzzzzzzzzzzzzzzzzzzzz", "description": "The suffix should encode at most 128-bits" + }, + { + "name": "prefix-underscore-start", + "typeid": "_prefix_00000000000000000000000000", + "description": "The prefix can't start with an underscore" + }, + { + "name": "prefix-underscore-end", + "typeid": "prefix__00000000000000000000000000", + "description": "The prefix can't end with an underscore" } ] diff --git a/typeid/typeid/spec/invalid.yml b/typeid/typeid/spec/invalid.yml index 3f288e7..1e19c33 100644 --- a/typeid/typeid/spec/invalid.yml +++ b/typeid/typeid/spec/invalid.yml @@ -4,7 +4,7 @@ # Each example contains an invalid TypeID string. Implementations are expected # to throw an error when attempting to parse/validate these strings. # -# Last updated: 2023-07-05 +# Last updated: 2024-04-10 (for version 0.3.0 of the spec) - name: prefix-uppercase typeid: "PREFIX_00000000000000000000000000" @@ -18,9 +18,10 @@ typeid: "pre.fix_00000000000000000000000000" description: "The prefix can't have symbols, it needs to be alphabetic" -- name: prefix-underscore - typeid: "pre_fix_00000000000000000000000000" - description: "The prefix can't have symbols, it needs to be alphabetic" +# Test removed in v0.3.0 – we now allow underscores in the prefix +# - name: prefix-underscore +# typeid: "pre_fix_00000000000000000000000000" +# description: "The prefix can't have symbols, it needs to be alphabetic" - name: prefix-non-ascii typeid: "préfix_00000000000000000000000000" @@ -86,3 +87,13 @@ # This is the first suffix that overflows into 129 bits typeid: "prefix_8zzzzzzzzzzzzzzzzzzzzzzzzz" description: "The suffix should encode at most 128-bits" + +# Tests below were added in v0.3.0 when we started allowing '_' within the +# type prefix. +- name: prefix-underscore-start + typeid: "_prefix_00000000000000000000000000" + description: "The prefix can't start with an underscore" + +- name: prefix-underscore-end + typeid: "prefix__00000000000000000000000000" + description: "The prefix can't end with an underscore" diff --git a/typeid/typeid/spec/valid.json b/typeid/typeid/spec/valid.json index 9a4e1a3..fff1de9 100644 --- a/typeid/typeid/spec/valid.json +++ b/typeid/typeid/spec/valid.json @@ -46,5 +46,11 @@ "typeid": "prefix_01h455vb4pex5vsknk084sn02q", "prefix": "prefix", "uuid": "01890a5d-ac96-774b-bcce-b302099a8057" + }, + { + "name": "prefix-underscore", + "typeid": "pre_fix_00000000000000000000000000", + "prefix": "pre_fix", + "uuid": "00000000-0000-0000-0000-000000000000" } ] diff --git a/typeid/typeid/spec/valid.yml b/typeid/typeid/spec/valid.yml index 8f63250..47c5328 100644 --- a/typeid/typeid/spec/valid.yml +++ b/typeid/typeid/spec/valid.yml @@ -23,7 +23,7 @@ # note that not all of them are UUIDv7s. When *generating* new random typeids, # implementations should always use UUIDv7s. # -# Last updated: 2023-07-05 +# Last updated: 2024-04-10 (for version 0.3.0 of the spec) - name: nil typeid: "00000000000000000000000000" @@ -64,3 +64,10 @@ typeid: "prefix_01h455vb4pex5vsknk084sn02q" prefix: "prefix" uuid: "01890a5d-ac96-774b-bcce-b302099a8057" + +# Tests below were added in v0.3.0 when we started allowing '_' within the +# type prefix. +- name: prefix-underscore + typeid: "pre_fix_00000000000000000000000000" + prefix: "pre_fix" + uuid: "00000000-0000-0000-0000-000000000000"