From a4daaccacb1c6e2f93ddec37451fbf781a178000 Mon Sep 17 00:00:00 2001 From: Novus Nota <68142933+novusnota@users.noreply.github.com> Date: Mon, 22 Jul 2024 15:01:24 +0200 Subject: [PATCH 1/4] feat: explanation of `remaining` serialization format --- pages/book/cells.mdx | 43 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 39 insertions(+), 4 deletions(-) diff --git a/pages/book/cells.mdx b/pages/book/cells.mdx index 1db35995..498f47fc 100644 --- a/pages/book/cells.mdx +++ b/pages/book/cells.mdx @@ -126,8 +126,7 @@ While you may use them for [manual parsing](#cnp-manually) of the cells, it's st Similar to serialization options of [`Int{:tact}`](/book/integers) type, `Cell{:tact}`, `Builder{:tact}` and `Slice{:tact}` also have various representations for encoding their values in the following cases: * as fields of [contracts](/book/contracts) and [traits](/book/types#traits), -* as fields of [Structs](/book/structs-and-messages#structs) and [Messages](/book/structs-and-messages#messages), -* and as key/value types of [maps](/book/maps). +* and as fields of [Structs](/book/structs-and-messages#structs) and [Messages](/book/structs-and-messages#messages). ```tact contract SerializationExample { @@ -136,11 +135,47 @@ contract SerializationExample { } ``` -### `remaining` [#serialization-bytes64] +### `remaining` [#serialization-remaining] + +By default, when it comes to storing values of [`Cell{:tact}`](#cells), [`Builder{:tact}`](#builders) and [`Slice{:tact}`](#slices) types in persistent storage of loading them from the received messages or assigned values, Tact's auto-layout algorithm uses references (refs): + +```tact +contract ReReReF { + c: Cell; // ^cell in TL-B notation + b: Builder; // ^builder in TL-B notation + s: Slice; // ^slice in TL-B notation +} +``` + +However, using `remaining{:tact}` makes values assigned to those fields to be stored directly as a [`Slice{:tact}`](#slices): + +```tact +contract RemRemRem { + c: Cell as remaining; // remainder in TL-B notation, + // storing and loading Slices directly in this Cell + + b: Builder as remaining; // remainder in TL-B notation, + // storing and loading Slices directly in this Builder + + s: Slice as remaining; // remainder in TL-B notation, + // storing and loading Slices directly +} +``` + +That's like using [`Builder.storeSlice(){:tact}`][b-5] and [`Slice.loadSlice(){:tact}`][s-5] over [`Builder.storeRef(){:tact}`][b-8] and [`Slice.loadRef(){:tact}`][s-8], which are to be used by default. In practice, this makes using `Slice as remaining{:tact}` the most [gas](https://docs.ton.org/develop/smart-contracts/fees#gas)-efficient option for working with [`Slice{:tact}`](#slices) values, since it uses the least amount of operations and bits. - To be resolved by [#26](https://github.com/tact-lang/tact-docs/issues/26). + Note, that the field serialized `as remaining{:tact}` cannot be [optional](/book/optional). That is, specifying something like `Cell? as remaining{:tact}` would cause a compilation error. + + Also note, that as of now, specifying `remaining{:tact}` for `Cell{:tact}` as the [map](/book/maps) value type doesn't do anything — the cell would still be stored as a reference: + + ```tact + struct Nope { + m: map; // dict in TL-B notation, + // despite the `remaining` + } + ``` From 1ef11fd6bc28ef1b7bde09ea63dbf2d2e4f6569c Mon Sep 17 00:00:00 2001 From: Novus Nota <68142933+novusnota@users.noreply.github.com> Date: Tue, 23 Jul 2024 22:26:19 +0200 Subject: [PATCH 2/4] feat: simplified the simplification to make things simpler than the simplest simplification can ever be --- pages/book/cells.mdx | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/pages/book/cells.mdx b/pages/book/cells.mdx index 498f47fc..867ef0dd 100644 --- a/pages/book/cells.mdx +++ b/pages/book/cells.mdx @@ -137,43 +137,38 @@ contract SerializationExample { ### `remaining` [#serialization-remaining] -By default, when it comes to storing values of [`Cell{:tact}`](#cells), [`Builder{:tact}`](#builders) and [`Slice{:tact}`](#slices) types in persistent storage of loading them from the received messages or assigned values, Tact's auto-layout algorithm uses references (refs): +To illustrate what `remaining{:tact}` serialization type does, let's take a look at [cell manipulation primitives](#cells-immutability) and their [TL-B][tlb] representation produced by Tact: ```tact -contract ReReReF { - c: Cell; // ^cell in TL-B notation - b: Builder; // ^builder in TL-B notation - s: Slice; // ^slice in TL-B notation +contract SerializationExample { + // By default + cRef: Cell; // ^cell in TL-B + bRef: Builder; // ^builder in TL-B + sRef: Slice; // ^slice in TL-B + + // With `remaining` + cRem: Cell as remaining; // remainder in TL-B + bRem: Builder as remaining; // remainder in TL-B + sRem: Slice as remaining; // remainder in TL-B } ``` -However, using `remaining{:tact}` makes values assigned to those fields to be stored directly as a [`Slice{:tact}`](#slices): - -```tact -contract RemRemRem { - c: Cell as remaining; // remainder in TL-B notation, - // storing and loading Slices directly in this Cell - - b: Builder as remaining; // remainder in TL-B notation, - // storing and loading Slices directly in this Builder +There, `^cell`, `^builder` and `^slice` in [TL-B][tlb] syntax mean the reference to [`Cell{:tact}`](#cells), [`Builder{:tact}`](#builders) and [`Slice{:tact}`](#slices) values respectively, while the `remainder<…>` of `cell`, `builder` or `slice` tells that the given value would be stored as a `Slice{:tact}` directly and not as a reference. - s: Slice as remaining; // remainder in TL-B notation, - // storing and loading Slices directly -} -``` +To draw parallels with [cell manipulation instructions](#cells-immutability), specifying `remaining{:tact}` is like using [`Builder.storeSlice(){:tact}`][b-5] and [`Slice.loadSlice(){:tact}`][s-5] instead of [`Builder.storeRef(){:tact}`][b-8] and [`Slice.loadRef(){:tact}`][s-8], which are to be used by default. -That's like using [`Builder.storeSlice(){:tact}`][b-5] and [`Slice.loadSlice(){:tact}`][s-5] over [`Builder.storeRef(){:tact}`][b-8] and [`Slice.loadRef(){:tact}`][s-8], which are to be used by default. In practice, this makes using `Slice as remaining{:tact}` the most [gas](https://docs.ton.org/develop/smart-contracts/fees#gas)-efficient option for working with [`Slice{:tact}`](#slices) values, since it uses the least amount of operations and bits. +That's why you often see something like `forwardPayload: Slice as remaining{:tact}` in the context of [Jettons](https://docs.ton.org/develop/dapps/asset-processing/jettons) — to store the forwarded payload `Slice{:tact}` directly at the end of the message body cell, rather than as a reference. Note, that the field serialized `as remaining{:tact}` cannot be [optional](/book/optional). That is, specifying something like `Cell? as remaining{:tact}` would cause a compilation error. - Also note, that as of now, specifying `remaining{:tact}` for `Cell{:tact}` as the [map](/book/maps) value type doesn't do anything — the cell would still be stored as a reference: + Also note, that as of now, specifying `remaining{:tact}` for the `Cell{:tact}` as the [map](/book/maps) value type does nothing — the cell would still be stored as a reference: ```tact struct Nope { m: map; // dict in TL-B notation, - // despite the `remaining` + // despite the `remaining` specified } ``` From 0b9b2a049b26bd343c58b3debc266b49cbb77443 Mon Sep 17 00:00:00 2001 From: Novus Nota <68142933+novusnota@users.noreply.github.com> Date: Wed, 24 Jul 2024 23:32:11 +0200 Subject: [PATCH 3/4] fix: applied some suggestions from a code review --- pages/book/cells.mdx | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/pages/book/cells.mdx b/pages/book/cells.mdx index 867ef0dd..0de34a23 100644 --- a/pages/book/cells.mdx +++ b/pages/book/cells.mdx @@ -128,10 +128,17 @@ Similar to serialization options of [`Int{:tact}`](/book/integers) type, `Cell{: * as fields of [contracts](/book/contracts) and [traits](/book/types#traits), * and as fields of [Structs](/book/structs-and-messages#structs) and [Messages](/book/structs-and-messages#messages). -```tact +```tact {2-3} contract SerializationExample { someCell: Cell as remaining; someSlice: Slice as bytes32; + + // Constructor function, + // necessary for this example contract to compile + init() { + self.someCell = emptyCell(); + self.someSlice = emptySlice(); + } } ``` @@ -139,7 +146,7 @@ contract SerializationExample { To illustrate what `remaining{:tact}` serialization type does, let's take a look at [cell manipulation primitives](#cells-immutability) and their [TL-B][tlb] representation produced by Tact: -```tact +```tact {3-5, 8-10} contract SerializationExample { // By default cRef: Cell; // ^cell in TL-B @@ -150,6 +157,17 @@ contract SerializationExample { cRem: Cell as remaining; // remainder in TL-B bRem: Builder as remaining; // remainder in TL-B sRem: Slice as remaining; // remainder in TL-B + + // Constructor function, + // necessary for this example contract to compile + init() { + self.cRef = emptyCell(); + self.bRef = beginCell(); + self.sRef = emptySlice(); + self.cRem = emptyCell(); + self.bRem = beginCell(); + self.sRem = emptySlice(); + } } ``` From ed623719914ed75b80525f00936215c9397b67e5 Mon Sep 17 00:00:00 2001 From: Novus Nota <68142933+novusnota@users.noreply.github.com> Date: Tue, 13 Aug 2024 22:42:32 +0200 Subject: [PATCH 4/4] feat: re-written and significantly enhanced! --- pages/book/cells.mdx | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/pages/book/cells.mdx b/pages/book/cells.mdx index 0de34a23..895ef675 100644 --- a/pages/book/cells.mdx +++ b/pages/book/cells.mdx @@ -125,7 +125,7 @@ While you may use them for [manual parsing](#cnp-manually) of the cells, it's st Similar to serialization options of [`Int{:tact}`](/book/integers) type, `Cell{:tact}`, `Builder{:tact}` and `Slice{:tact}` also have various representations for encoding their values in the following cases: -* as fields of [contracts](/book/contracts) and [traits](/book/types#traits), +* as [storage variables](/book/contracts#variables) of [contracts](/book/contracts) and [traits](/book/types#traits), * and as fields of [Structs](/book/structs-and-messages#structs) and [Messages](/book/structs-and-messages#messages). ```tact {2-3} @@ -137,14 +137,18 @@ contract SerializationExample { // necessary for this example contract to compile init() { self.someCell = emptyCell(); - self.someSlice = emptySlice(); + self.someSlice = beginCell().storeUint(42, 256).asSlice(); } } ``` ### `remaining` [#serialization-remaining] -To illustrate what `remaining{:tact}` serialization type does, let's take a look at [cell manipulation primitives](#cells-immutability) and their [TL-B][tlb] representation produced by Tact: +The `remaining{:tact}` serialization option can be applied to values of [`Cell{:tact}`](#cells), [`Builder{:tact}`](#builders) and [`Slice{:tact}`](#slices) types. + +It affects the process of constructing and parsing cell values by causing them to be stored and loaded directly rather than as a reference. To draw parallels with [cell manipulation instructions](#cells-immutability), specifying `remaining{:tact}` is like using [`Builder.storeSlice(){:tact}`][b-5] and [`Slice.loadSlice(){:tact}`][s-5] instead of [`Builder.storeRef(){:tact}`][b-8] and [`Slice.loadRef(){:tact}`][s-8], which are to be used by default. + +In addition, the [TL-B][tlb] representation produced by Tact changes too: ```tact {3-5, 8-10} contract SerializationExample { @@ -173,13 +177,34 @@ contract SerializationExample { There, `^cell`, `^builder` and `^slice` in [TL-B][tlb] syntax mean the reference to [`Cell{:tact}`](#cells), [`Builder{:tact}`](#builders) and [`Slice{:tact}`](#slices) values respectively, while the `remainder<…>` of `cell`, `builder` or `slice` tells that the given value would be stored as a `Slice{:tact}` directly and not as a reference. -To draw parallels with [cell manipulation instructions](#cells-immutability), specifying `remaining{:tact}` is like using [`Builder.storeSlice(){:tact}`][b-5] and [`Slice.loadSlice(){:tact}`][s-5] instead of [`Builder.storeRef(){:tact}`][b-8] and [`Slice.loadRef(){:tact}`][s-8], which are to be used by default. +Now, to give a real-world example, imagine that you need to notice and react to inbound [jetton][jetton] transfers in your smart contract. The appropriate [Message][message] structure for doing so would look something like this: + +```tact /remaining/ +message(0x7362d09c) JettonTransferNotification { + queryId: Int as uint64; // arbitrary request number to prevent replay attacks + amount: Int as coins; // amount of jettons transferred + sender: Address; // address of the sender of the jettons + forwardPayload: Slice as remaining; // optional custom payload +} +``` + +And the [receiver][recv] in the contract would look like this: + +```tact +receive(msg: JettonTransferNotification) { + // ... you do you ... +} +``` + +Upon receiving a [jetton][jetton] transfer notification message, its cell body is converted into a `Slice{tact}` and then parsed as a `JettonTransferNotification{:tact}` [Message][message]. At the end of this process, the `forwardPayload` will have all the remaining data of the original message cell. + +If we were to violate the [jetton][jetton] standard and place the `forwardPayload: Slice as remaining` field in any other position in the `JettonTransferNotification{:tact}` [Message][message], the `forwardPayload` would have the remaining data of the message cell at the moment of parsing it, which would include all the subsequent fields of the [Message][message]. -That's why you often see something like `forwardPayload: Slice as remaining{:tact}` in the context of [Jettons](https://docs.ton.org/develop/dapps/asset-processing/jettons) — to store the forwarded payload `Slice{:tact}` directly at the end of the message body cell, rather than as a reference. +Therefore, to prevent misuse of the contract storage and reduce gas consumption, make sure to specify `remaining{:tact}` serialization option only on the last field of the given [Message][message], as it will store all the remaining data of the [`Slice{:tact}`](#slices) at the moment it was parsed into said [Message][message] in [receiver functions][recv]. - Note, that the field serialized `as remaining{:tact}` cannot be [optional](/book/optional). That is, specifying something like `Cell? as remaining{:tact}` would cause a compilation error. + Note, that the cell serialized via `as remaining{:tact}` cannot be [optional](/book/optional). That is, specifying something like `Cell? as remaining{:tact}`, `Builder{:tact}` or `Slice? as remaining{:tact}` would cause a compilation error. Also note, that as of now, specifying `remaining{:tact}` for the `Cell{:tact}` as the [map](/book/maps) value type does nothing — the cell would still be stored as a reference: @@ -421,9 +446,11 @@ let areSlicesNotEqual = aSlice.hash() != bSlice.hash(); // false [p]: /book/types#primitive-types [struct]: /book/structs-and-messages#structs [message]: /book/structs-and-messages#messages +[recv]: /book/contracts#receiver-functions [tvm]: https://docs.ton.org/learn/tvm-instructions/tvm-overview [tlb]: https://docs.ton.org/develop/data-formats/tl-b-language +[jetton]: https://docs.ton.org/develop/dapps/asset-processing/jettons [sha-2]: https://en.wikipedia.org/wiki/SHA-2#Hash_standard [quadtree]: https://en.wikipedia.org/wiki/Quadtree