feat: add p/uuid #2076

DIGIX666 · 2024-05-12T15:39:06Z

I suggest adding a UUID package because I think it could be useful to have it available. I think it can further facilitate traceability and compatibility between different systems and apps as they would have a common standard for identifying entities.

To improve UUID generation, I wanted to add a resolver like Snowflake, but from what I've searched, sync/atomic is not supported on Gno. Is there another similar package that I might have missed ?

codecov · 2024-05-12T15:45:05Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.10%. Comparing base (af05780) to head (3a18918).
Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2076      +/-   ##
==========================================
- Coverage   63.32%   63.10%   -0.22%     
==========================================
  Files         548      548              
  Lines       78511    81134    +2623     
==========================================
+ Hits        49719    51203    +1484     
- Misses      25438    26511    +1073     
- Partials     3354     3420      +66

Flag	Coverage Δ
contribs/gnodev	`61.11% <ø> (+0.53%)`	⬆️
contribs/gnofaucet	`14.82% <ø> (ø)`
gno.land	`67.37% <ø> (+0.18%)`	⬆️
gnovm	`67.88% <ø> (ø)`
misc/genstd	`79.72% <ø> (ø)`
tm2	`62.42% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

notJoon · 2024-05-13T01:15:05Z

To improve UUID generation, I wanted to add a resolver like Snowflake, but from what I've searched, sync/atomic is not supported on Gno. Is there another similar package that I might have missed ?

There is no other way to guarantee atomicity besides using time in environments where atomic is not supported. If we consider it further, it seems possible to generate a random value (using crypto/rand or not merged yet, but maybe later) like a machine ID, assign it, and then combine it. Of course, this is slightly different from the implementation of Snowflake.

examples/gno.land/p/demo/uuid/uuid.gno

examples/gno.land/p/demo/uuid/uuid_test.gno

DIGIX666 · 2024-05-13T02:10:35Z

To improve UUID generation, I wanted to add a resolver like Snowflake, but from what I've searched, sync/atomic is not supported on Gno. Is there another similar package that I might have missed ?

There is no other way to guarantee atomicity besides using time in environments where atomic is not supported. If we consider it further, it seems possible to generate a random value (using crypto/rand or not merged yet, but #1933 later) like a machine ID, assign it, and then combine it. Of course, this is slightly different from the implementation of Snowflake.

Thanks for your explanation. I will check the progress of crypto/rand to try to use it.

zivkovicmilos

This is super cool, I can already see myself using it :)

I've left a few comments, otherwise it looks good 💯

examples/gno.land/p/demo/uuid/uuid.gno

zivkovicmilos · 2024-08-15T11:10:35Z

examples/gno.land/p/demo/uuid/uuid.gno

+
+func NewUUID() *UUID {
+	u := &UUID{
+		StartTime: time.Date(2008, 11, 10, 23, 0, 0, 0, time.UTC).Unix() * 1000,


If this is fixed, why not just use the constant unix time for this value?

Also, what is this value?

Indeed you are right it would be better to provide the data directly in Unix which would avoid recalculating them at each function call.
The value used is the one used in snowflake because it is based on the fact that go playground starts with the temp of 2009-11-10 23:00:00 UTC.
So I took the same as that of snowflake because I did not find the time.Now() of gno playground and I believe that in gno time.Now() is based on the return of the block time rather than the system time.

Do you have more information on this subject?

in gno time.Now() is based on the return of the block time rather than the system time.

Yes, it's the block execution context time, and same for all transactions within a block

Can you add a comment, and change this value to unix?

in gno time.Now() is based on the return of the block time rather than the system time.

Yes, it's the block execution context time, and same for all transactions within a block

Can you add a comment, and change this value to unix?

Yep, I had already made this modification in this commit :)
353beb1

examples/gno.land/p/demo/uuid/uuid.gno

examples/gno.land/p/demo/uuid/uuid_test.gno

zivkovicmilos · 2024-08-15T11:15:48Z

examples/gno.land/p/demo/uuid/uuid.gno

+	return buf
+}
+
+func bytesToHex(bs []byte) string {


What do you think about implementing this as:

return "0x + hex.EncodeToString(bs)`

?

Thanks for the idea of using hex.EncodeToString() I hadn't thought of it.
However I think it's better to stick to the standard generation of UUIDs because 0x indicates that the value is in hexadecimal but is not used in UUID display standards. For the structure you have to keep the - to delimit the 5 groups and keep a quick visibility.

What do you think of this implementation?

hexStr := hex.EncodeToString(bytes) return ufmt.Sprintf("%s-%s-%s-%s-%s", hexStr[0:8], hexStr[8:12], hexStr[12:16], hexStr[16:20], hexStr[20:], )

examples/gno.land/p/demo/uuid/uuid.gno

thehowl · 2024-09-13T15:09:16Z

examples/gno.land/p/demo/uuid/uuid.gno

+	TimestampLength uint8  = 32
+	MachineIDLength uint64 = 10
+	SequenceLength  uint64 = 12


more than "length" (which I understand immediately as bytes), it's bits (ie. TimestampBits).

also, why typed? they could just be untyped constants.

Thank for the clarification, I thinked length it is a correct name for length of bits.

I chose to keep these constants typed for several reasons:

Clarity: Explicit typing makes it clear that these constants are used in bit-related operations.

Error prevention: Ensure that operations follow expected constraints and avoid compilation errors, especially for bit shifts.

Performance: Typing optimizes performance and avoids implicit conversions.

Scalability: Code compatibility if the logic needs to evolve.

If untyped constants are more appropriate in this case, I'm open to hearing :)

examples/gno.land/p/demo/uuid/uuid.gno

thehowl · 2024-09-13T15:10:47Z

examples/gno.land/p/demo/uuid/uuid.gno

+	machineIDMoveLength = SequenceLength
+	timestampMoveLength = MachineIDLength + SequenceLength
+
+	startTime uint32 = 1226354400 // 10 Nov 2008 23:00:00 UTC in seconds (Snowflake epoch)


is this for some kind of compatibility, or could it just be 1/1/24?

i see no reason to use epochs from other formats unless it's to ensure some kind of sequencing / compatiblity from existing IDs.

I understand your opinion and yes it's possible to just use 1/1/24 but can you check my comment with @zivkovicmilos on this subject ? 🙏🏼
#2076 (comment)

thehowl · 2024-09-13T15:24:52Z

examples/gno.land/p/demo/uuid/uuid.gno

+	// Copy transformed ID into the second half of the array
+	copy(bytes[8:], uint64ToBytes(id))
+
+	// Use bits from the Snowflake ID to generate the first half of the UUID
+	bytes[0] = byte(id >> 60)
+	bytes[1] = byte(id >> 52)
+	bytes[2] = byte(id >> 44)
+	bytes[3] = byte(id >> 36)
+	bytes[4] = byte(id >> 28)
+	bytes[5] = byte(id >> 20)
+	bytes[6] = byte(id >> 12)
+	bytes[7] = byte(id >> 4)
+
+	// set the version and variant bits according to UUID specification.
+	bytes[6] = (bytes[6] & 0x0f) | 0x40 // version 4 (random)
+	bytes[8] = (bytes[8] & 0x3f) | 0x80 // variant 1 (RFC 4122)


Note, we're using the same id both in uint64ToBytes, and then copying it over in a slightly modified manner later in this function

I don't particularly like this. We're restricting the number of bits allotted to a timestamp to 32 bits, so we can generate 64-bit integers; when really UUIDs are 122-bits, so we could reasonably have timestamps of at the very least 40 bits (which IMO is more reasonable).

Additionally, this means that there are not 2^122 possible IDs out of this package, but only 2^64.

There needs to be a decision: is this a Snowflake ID generator or is this a UUID generator? Either is fine, but understanding the target is important; and if we use UUIDs we should try to use all 122 bits at our disposal to try to create really unique identifiers.

I understand what you're saying. Originally, I had opted for a 64-bit range, but I ran into a uniqueness problem with the entropy value, which was in uint32.

So, to use the full 64-bit range, I changed TimestampBits to 42 bits. Let me know what you think ;)
859cae9

This is a Snowflake ID generator because I want to keep the Timestamp present.

@DIGIX666 - as @thehowl said, why not call it snowflakeID and implement it like that then?

Because I thought keeping the name UUID was more explicit, but in light of your feedback and my desire to stick with a Snowflake ID generator system, I will change the names to maintain consistency throughout 👍🏼
3a18918

Co-authored-by: Morgan <git@howl.moe>

examples/gno.land/p/demo/uuid/uuid.gno

moul · 2024-09-17T13:22:20Z

examples/gno.land/p/demo/uuid/uuid.gno

+	current := u.getEntropy()
+	elapsedTime := current - u.startTime
+	u.lastTimestamp = current


why are you making math stuff between random and timestamp?
it looks like just waste CPU.

shouldn't "current" be the current time?

Because at first I used classic incrementation for current because time.Now() didn't work properly. So I replaced it with entropy once merge to decrease the chances of prediction. But now I'm testing to use time.Now() and entropy

Output with entropy :

examples/gno.land/p/demo/uuid/uuid.gno

moul

seems not "gno-ified" enough :)

can you also provide some examples, like a screenshot or something so that we can see various generated output examples just by reviewing the PR, please?

DIGIX666 · 2024-09-17T21:06:42Z

seems not "gno-ified" enough :)

can you also provide some examples, like a screenshot or something so that we can see various generated output examples just by reviewing the PR, please?

Output this my last commit 485ce8d:

DIGIX666

Finish corrective review

DIGIX666 · 2024-09-14T14:47:30Z

examples/gno.land/p/demo/uuid/uuid.gno

+	TimestampLength uint8  = 32
+	MachineIDLength uint64 = 10
+	SequenceLength  uint64 = 12


Thank for the clarification, I thinked length it is a correct name for length of bits.

I chose to keep these constants typed for several reasons:

Clarity: Explicit typing makes it clear that these constants are used in bit-related operations.

Error prevention: Ensure that operations follow expected constraints and avoid compilation errors, especially for bit shifts.

Performance: Typing optimizes performance and avoids implicit conversions.

Scalability: Code compatibility if the logic needs to evolve.

If untyped constants are more appropriate in this case, I'm open to hearing :)

DIGIX666 · 2024-09-17T13:17:56Z

examples/gno.land/p/demo/uuid/uuid.gno

+	// Copy transformed ID into the second half of the array
+	copy(bytes[8:], uint64ToBytes(id))
+
+	// Use bits from the Snowflake ID to generate the first half of the UUID
+	bytes[0] = byte(id >> 60)
+	bytes[1] = byte(id >> 52)
+	bytes[2] = byte(id >> 44)
+	bytes[3] = byte(id >> 36)
+	bytes[4] = byte(id >> 28)
+	bytes[5] = byte(id >> 20)
+	bytes[6] = byte(id >> 12)
+	bytes[7] = byte(id >> 4)
+
+	// set the version and variant bits according to UUID specification.
+	bytes[6] = (bytes[6] & 0x0f) | 0x40 // version 4 (random)
+	bytes[8] = (bytes[8] & 0x3f) | 0x80 // variant 1 (RFC 4122)


I understand what you're saying. Originally, I had opted for a 64-bit range, but I ran into a uniqueness problem with the entropy value, which was in uint32.

So, to use the full 64-bit range, I changed TimestampBits to 42 bits. Let me know what you think ;)
859cae9

This is a Snowflake ID generator because I want to keep the Timestamp present.

examples/gno.land/p/demo/uuid/uuid.gno

DIGIX666 · 2024-09-17T21:03:17Z

examples/gno.land/p/demo/uuid/uuid.gno

+	current := u.getEntropy()
+	elapsedTime := current - u.startTime
+	u.lastTimestamp = current


Because at first I used classic incrementation for current because time.Now() didn't work properly. So I replaced it with entropy once merge to decrease the chances of prediction. But now I'm testing to use time.Now() and entropy

Output with entropy :

DIGIX666 · 2024-09-18T11:32:03Z

examples/gno.land/p/demo/uuid/uuid.gno

+	machineIDMoveLength = SequenceLength
+	timestampMoveLength = MachineIDLength + SequenceLength
+
+	startTime uint32 = 1226354400 // 10 Nov 2008 23:00:00 UTC in seconds (Snowflake epoch)


I understand your opinion and yes it's possible to just use 1/1/24 but can you check my comment with @zivkovicmilos on this subject ? 🙏🏼
#2076 (comment)

zivkovicmilos · 2024-11-03T12:52:42Z

Hey @DIGIX666, can you please update this branch with master? 🙏

examples/gno.land/p/demo/uuid/uuid.gno

leohhhn · 2024-11-05T10:33:21Z

examples/gno.land/p/demo/uuid/uuid.gno

+	// Copy transformed ID into the second half of the array
+	copy(bytes[8:], uint64ToBytes(id))
+
+	// Use bits from the Snowflake ID to generate the first half of the UUID
+	bytes[0] = byte(id >> 60)
+	bytes[1] = byte(id >> 52)
+	bytes[2] = byte(id >> 44)
+	bytes[3] = byte(id >> 36)
+	bytes[4] = byte(id >> 28)
+	bytes[5] = byte(id >> 20)
+	bytes[6] = byte(id >> 12)
+	bytes[7] = byte(id >> 4)
+
+	// set the version and variant bits according to UUID specification.
+	bytes[6] = (bytes[6] & 0x0f) | 0x40 // version 4 (random)
+	bytes[8] = (bytes[8] & 0x3f) | 0x80 // variant 1 (RFC 4122)


@DIGIX666 - as @thehowl said, why not call it snowflakeID and implement it like that then?

add package UUID

c59a3ba

DIGIX666 requested review from a team as code owners May 12, 2024 15:39

DIGIX666 requested review from harry-hov and piux2 and removed request for a team May 12, 2024 15:39

github-actions bot assigned DIGIX666 May 12, 2024

github-actions bot added the 🧾 package/realm Tag used for new Realms or Packages. label May 12, 2024

théo dub added 2 commits May 12, 2024 17:50

make fmt imports

dbcacc9

change uint16 to uint64

bdb8b96

DIGIX666 changed the title ~~feat: initial p/uuid dapp~~ feat: add p/uuid May 12, 2024

notJoon reviewed May 13, 2024

View reviewed changes

examples/gno.land/p/demo/uuid/uuid.gno Outdated Show resolved Hide resolved

notJoon reviewed May 13, 2024

View reviewed changes

examples/gno.land/p/demo/uuid/uuid_test.gno Outdated Show resolved Hide resolved

delete a var and change comment

c0b93f0

michelleellen mentioned this pull request May 24, 2024

Minutes: Public Dev Calls [every Wednesday] gnolang/meetings#37

Open

DIGIX666 mentioned this pull request May 28, 2024

Zone01 - Student Program Journey gnolang/hackerspace#59

Open

DIGIX666 requested review from jaekwon, moul, deelawn, thehowl, mvertes, gfanton, zivkovicmilos and a team as code owners June 2, 2024 16:18

github-actions bot added 📦 🤖 gnovm Issues or PRs gnovm related 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Jun 2, 2024

DIGIX666 force-pushed the dev/thox/uuid branch from ba605ce to d86f906 Compare June 2, 2024 16:22

zivkovicmilos approved these changes Aug 15, 2024

View reviewed changes

DIGIX666 added 2 commits August 20, 2024 17:22

update for use Unix and hex.EncodeToString()

91cdfe8

make tidy

68efeff

leohhhn mentioned this pull request Aug 21, 2024

Minutes: SCP Updates [every Monday] gnolang/meetings#39

Open

change uint64 for uint32

e9a5953

DIGIX666 requested a review from zivkovicmilos August 21, 2024 12:19

DIGIX666 and others added 2 commits August 21, 2024 14:20

Merge branch 'master' into dev/thox/uuid

1611c05

Merge branch 'master' into dev/thox/uuid

a8184d0

thehowl reviewed Sep 13, 2024

View reviewed changes

Apply suggestions from code review

8ea2368

Co-authored-by: Morgan <git@howl.moe>

moul reviewed Sep 17, 2024

View reviewed changes

examples/gno.land/p/demo/uuid/uuid.gno Outdated Show resolved Hide resolved

moul reviewed Sep 17, 2024

View reviewed changes

examples/gno.land/p/demo/uuid/uuid.gno Outdated Show resolved Hide resolved

moul requested changes Sep 17, 2024

View reviewed changes

DIGIX666 and others added 5 commits September 17, 2024 15:34

change for 42 bits

859cae9

correct the owerflow for elaspedTime

cc38c09

change startTime second to ms

353beb1

Merge branch 'master' into dev/thox/uuid

335c5f2

add time.Now

485ce8d

DIGIX666 requested review from moul and thehowl September 17, 2024 21:08

DIGIX666 mentioned this pull request Sep 18, 2024

Digix666 Builder Journey gnolang/hackerspace#84

Open

add output comment in test

0169b68

DIGIX666 commented Sep 18, 2024

View reviewed changes

leohhhn mentioned this pull request Oct 2, 2024

Minutes: Solo Grantees Updates [every Wednesday] gnolang/meetings#40

Open

Merge branch 'master' into dev/thox/uuid

ad62314

leohhhn reviewed Nov 5, 2024

View reviewed changes

change name to snowflake

3a18918

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add p/uuid #2076

feat: add p/uuid #2076

DIGIX666 commented May 12, 2024

codecov bot commented May 12, 2024 •

edited

Loading

notJoon commented May 13, 2024 •

edited

Loading

DIGIX666 commented May 13, 2024

zivkovicmilos left a comment

zivkovicmilos Aug 15, 2024

DIGIX666 Aug 20, 2024

zivkovicmilos Nov 3, 2024

DIGIX666 Nov 3, 2024

zivkovicmilos Aug 15, 2024

DIGIX666 Aug 20, 2024

zivkovicmilos Nov 3, 2024

thehowl Sep 13, 2024

DIGIX666 Sep 14, 2024

thehowl Sep 13, 2024

DIGIX666 Sep 18, 2024

thehowl Sep 13, 2024

DIGIX666 Sep 17, 2024

leohhhn Nov 5, 2024 •

edited

Loading

DIGIX666 Nov 5, 2024

moul Sep 17, 2024

moul Sep 17, 2024

DIGIX666 Sep 17, 2024

moul left a comment

DIGIX666 commented Sep 17, 2024

DIGIX666 left a comment

DIGIX666 Sep 14, 2024

DIGIX666 Sep 17, 2024

DIGIX666 Sep 17, 2024

DIGIX666 Sep 18, 2024

zivkovicmilos commented Nov 3, 2024

leohhhn Nov 5, 2024 •

edited

Loading

feat: add p/uuid #2076

Are you sure you want to change the base?

feat: add p/uuid #2076

Conversation

DIGIX666 commented May 12, 2024

codecov bot commented May 12, 2024 • edited Loading

Codecov Report

notJoon commented May 13, 2024 • edited Loading

DIGIX666 commented May 13, 2024

zivkovicmilos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leohhhn Nov 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moul left a comment

Choose a reason for hiding this comment

DIGIX666 commented Sep 17, 2024

DIGIX666 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zivkovicmilos commented Nov 3, 2024

leohhhn Nov 5, 2024 • edited Loading

Choose a reason for hiding this comment

codecov bot commented May 12, 2024 •

edited

Loading

notJoon commented May 13, 2024 •

edited

Loading

leohhhn Nov 5, 2024 •

edited

Loading

leohhhn Nov 5, 2024 •

edited

Loading