Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CBOR support to transaction build and transaction build-raw commands #3483

Merged
merged 2 commits into from
Jan 24, 2022

Conversation

newhoggy
Copy link
Contributor

@newhoggy newhoggy commented Jan 8, 2022

Note, this deprecates options like --tx-in-datum-file, replacing them with the more descriptive --tx-in-datum-json-file

@newhoggy
Copy link
Contributor Author

newhoggy commented Jan 8, 2022

The PR introduces the following changes to the transaction build command:

diff -u a.txt b.txt
--- a.txt	2022-01-08 22:00:48.000000000 +1100
+++ b.txt	2022-01-08 22:00:42.000000000 +1100
@@ -1,4 +1,4 @@
-Usage: cardano-cli-master transaction build
+Usage: cardano-cli transaction build
             [ --byron-era
             | --shelley-era
             | --allegra-era
@@ -14,33 +14,52 @@
             [--witness-override WORD]
             (--tx-in TX-IN
               [--tx-in-script-file FILE
-                [(--tx-in-datum-file FILE | --tx-in-datum-value JSON VALUE)
-                  ( --tx-in-redeemer-file FILE
+                [
+                  ( --tx-in-datum-cbor-file CBOR FILE
+                  | --tx-in-datum-json-file JSON FILE
+                  | --tx-in-datum-file JSON FILE
+                  | --tx-in-datum-value JSON VALUE
+                  )
+                  ( --tx-in-redeemer-cbor-file CBOR FILE
+                  | --tx-in-redeemer-json-file JSON FILE
+                  | --tx-in-redeemer-file JSON FILE
                   | --tx-in-redeemer-value JSON VALUE
                   )]])
             [--required-signer FILE | --required-signer-hash HASH]
             [--tx-in-collateral TX-IN]
             [--tx-out ADDRESS VALUE
               [ --tx-out-datum-hash HASH
-              | --tx-out-datum-hash-file FILE
+              | --tx-out-datum-hash-cbor-file CBOR FILE
+              | --tx-out-datum-hash-json-file JSON FILE
+              | --tx-out-datum-hash-file JSON FILE
               | --tx-out-datum-hash-value JSON VALUE
-              | --tx-out-datum-embed-file FILE
+              | --tx-out-datum-embed-cbor-file CBOR FILE
+              | --tx-out-datum-embed-json-file JSON FILE
+              | --tx-out-datum-embed-file JSON FILE
               | --tx-out-datum-embed-value JSON VALUE
               ]]
             --change-address ADDRESS
             [--mint VALUE
               (--mint-script-file FILE
-                [--mint-redeemer-file FILE | --mint-redeemer-value JSON VALUE])]
+                [ --mint-redeemer-cbor-file CBOR FILE
+                | --mint-redeemer-json-file JSON FILE
+                | --mint-redeemer-file JSON FILE
+                | --mint-redeemer-value JSON VALUE
+                ])]
             [--invalid-before SLOT]
             [--invalid-hereafter SLOT]
             [--certificate-file CERTIFICATEFILE
               [--certificate-script-file FILE
-                [ --certificate-redeemer-file FILE
+                [ --certificate-redeemer-cbor-file CBOR FILE
+                | --certificate-redeemer-json-file JSON FILE
+                | --certificate-redeemer-file JSON FILE
                 | --certificate-redeemer-value JSON VALUE
                 ]]]
             [--withdrawal WITHDRAWAL
               [--withdrawal-script-file FILE
-                [ --withdrawal-redeemer-file FILE
+                [ --withdrawal-redeemer-cbor-file CBOR FILE
+                | --withdrawal-redeemer-json-file JSON FILE
+                | --withdrawal-redeemer-file JSON FILE
                 | --withdrawal-redeemer-value JSON VALUE
                 ]]]
             [--json-metadata-no-schema | --json-metadata-detailed-schema]
@@ -79,15 +98,30 @@
   --tx-in TX-IN            TxId#TxIx
   --tx-in-script-file FILE The file containing the script to witness the
                            spending of the transaction input.
-  --tx-in-datum-file FILE  The script datum, in the given JSON file. The file
+  --tx-in-datum-cbor-file CBOR FILE
+                           The script datum, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --tx-in-datum-json-file JSON FILE
+                           The script datum, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --tx-in-datum-file JSON FILE
+                           The script datum, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --tx-in-datum-value JSON VALUE
                            The script datum, in JSON syntax. There is no schema:
                            (almost) any JSON value is supported, including
                            top-level strings and numbers.
-  --tx-in-redeemer-file FILE
+  --tx-in-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --tx-in-redeemer-json-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --tx-in-redeemer-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --tx-in-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,
@@ -104,19 +138,36 @@
                            Lovelace).
   --tx-out-datum-hash HASH The script datum hash for this tx output, as the raw
                            datum hash (in hex).
-  --tx-out-datum-hash-file FILE
+  --tx-out-datum-hash-cbor-file CBOR FILE
                            The script datum hash for this tx output, by hashing
                            the script datum in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --tx-out-datum-hash-json-file JSON FILE
+                           The script datum hash for this tx output, by hashing
+                           the script datum in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --tx-out-datum-hash-file JSON FILE
+                           The script datum hash for this tx output, by hashing
+                           the script datum in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --tx-out-datum-hash-value JSON VALUE
                            The script datum hash for this tx output, by hashing
                            the script datum given here in JSON syntax. There is
                            no schema: (almost) any JSON value is supported,
                            including top-level strings and numbers.
-  --tx-out-datum-embed-file FILE
+  --tx-out-datum-embed-cbor-file CBOR FILE
+                           The script datum to embed in the tx for this output,
+                           in the given JSON file. The file must follow the
+                           special JSON schema for script data.
+  --tx-out-datum-embed-json-file JSON FILE
                            The script datum to embed in the tx for this output,
                            in the given JSON file. The file must follow the
                            special JSON schema for script data.
+  --tx-out-datum-embed-file JSON FILE
+                           The script datum to embed in the tx for this output,
+                           in the given JSON file. The file must follow the
+                           special JSON schema for script data. (DEPECRATED)
   --tx-out-datum-embed-value JSON VALUE
                            The script datum to embed in the tx for this output,
                            given here in JSON syntax. There is no schema:
@@ -127,9 +178,16 @@
                            syntax. You must specifiy a script witness.
   --mint-script-file FILE  The file containing the script to witness the minting
                            of assets for a particular policy Id.
-  --mint-redeemer-file FILE
+  --mint-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --mint-redeemer-json-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --mint-redeemer-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --mint-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,
@@ -144,9 +202,16 @@
   --certificate-script-file FILE
                            The file containing the script to witness the use of
                            the certificate.
-  --certificate-redeemer-file FILE
+  --certificate-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --certificate-redeemer-json-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --certificate-redeemer-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --certificate-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,
@@ -158,9 +223,16 @@
   --withdrawal-script-file FILE
                            The file containing the script to witness the
                            withdrawal of rewards.
-  --withdrawal-redeemer-file FILE
+  --withdrawal-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --withdrawal-redeemer-json-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --withdrawal-redeemer-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --withdrawal-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,

And the following changes to the build-raw command:

--- a.txt	2022-01-08 22:07:21.000000000 +1100
+++ b.txt	2022-01-08 22:07:29.000000000 +1100
@@ -1,4 +1,4 @@
-Usage: cardano-cli-master transaction build-raw
+Usage: cardano-cli transaction build-raw
             [ --byron-era
             | --shelley-era
             | --allegra-era
@@ -8,8 +8,15 @@
             [--script-valid | --script-invalid]
             (--tx-in TX-IN
               [--tx-in-script-file FILE
-                [(--tx-in-datum-file FILE | --tx-in-datum-value JSON VALUE)
-                  ( --tx-in-redeemer-file FILE
+                [
+                  ( --tx-in-datum-cbor-file CBOR FILE
+                  | --tx-in-datum-json-file JSON FILE
+                  | --tx-in-datum-file JSON FILE
+                  | --tx-in-datum-value JSON VALUE
+                  )
+                  ( --tx-in-redeemer-cbor-file CBOR FILE
+                  | --tx-in-redeemer-json-file JSON FILE
+                  | --tx-in-redeemer-file JSON FILE
                   | --tx-in-redeemer-value JSON VALUE
                   )
                   --tx-in-execution-units (INT, INT)]])
@@ -17,14 +24,23 @@
             [--required-signer FILE | --required-signer-hash HASH]
             [--tx-out ADDRESS VALUE
               [ --tx-out-datum-hash HASH
-              | --tx-out-datum-hash-file FILE
+              | --tx-out-datum-hash-cbor-file CBOR FILE
+              | --tx-out-datum-hash-json-file JSON FILE
+              | --tx-out-datum-hash-file JSON FILE
               | --tx-out-datum-hash-value JSON VALUE
-              | --tx-out-datum-embed-file FILE
+              | --tx-out-datum-embed-cbor-file CBOR FILE
+              | --tx-out-datum-embed-json-file JSON FILE
+              | --tx-out-datum-embed-file JSON FILE
               | --tx-out-datum-embed-value JSON VALUE
               ]]
             [--mint VALUE
               (--mint-script-file FILE
-                [(--mint-redeemer-file FILE | --mint-redeemer-value JSON VALUE)
+                [
+                  ( --mint-redeemer-cbor-file CBOR FILE
+                  | --mint-redeemer-json-file JSON FILE
+                  | --mint-redeemer-file JSON FILE
+                  | --mint-redeemer-value JSON VALUE
+                  )
                   --mint-execution-units (INT, INT)])]
             [--invalid-before SLOT]
             [--invalid-hereafter SLOT]
@@ -32,14 +48,18 @@
             [--certificate-file CERTIFICATEFILE
               [--certificate-script-file FILE
                 [
-                  ( --certificate-redeemer-file FILE
+                  ( --certificate-redeemer-cbor-file CBOR FILE
+                  | --certificate-redeemer-json-file JSON FILE
+                  | --certificate-redeemer-file JSON FILE
                   | --certificate-redeemer-value JSON VALUE
                   )
                   --certificate-execution-units (INT, INT)]]]
             [--withdrawal WITHDRAWAL
               [--withdrawal-script-file FILE
                 [
-                  ( --withdrawal-redeemer-file FILE
+                  ( --withdrawal-redeemer-cbor-file CBOR FILE
+                  | --withdrawal-redeemer-json-file JSON FILE
+                  | --withdrawal-redeemer-file JSON FILE
                   | --withdrawal-redeemer-value JSON VALUE
                   )
                   --withdrawal-execution-units (INT, INT)]]]
@@ -67,15 +87,30 @@
   --tx-in TX-IN            TxId#TxIx
   --tx-in-script-file FILE The file containing the script to witness the
                            spending of the transaction input.
-  --tx-in-datum-file FILE  The script datum, in the given JSON file. The file
+  --tx-in-datum-cbor-file CBOR FILE
+                           The script datum, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --tx-in-datum-json-file JSON FILE
+                           The script datum, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --tx-in-datum-file JSON FILE
+                           The script datum, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --tx-in-datum-value JSON VALUE
                            The script datum, in JSON syntax. There is no schema:
                            (almost) any JSON value is supported, including
                            top-level strings and numbers.
-  --tx-in-redeemer-file FILE
+  --tx-in-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --tx-in-redeemer-json-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --tx-in-redeemer-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --tx-in-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,
@@ -94,19 +129,36 @@
                            Lovelace).
   --tx-out-datum-hash HASH The script datum hash for this tx output, as the raw
                            datum hash (in hex).
-  --tx-out-datum-hash-file FILE
+  --tx-out-datum-hash-cbor-file CBOR FILE
+                           The script datum hash for this tx output, by hashing
+                           the script datum in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --tx-out-datum-hash-json-file JSON FILE
                            The script datum hash for this tx output, by hashing
                            the script datum in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --tx-out-datum-hash-file JSON FILE
+                           The script datum hash for this tx output, by hashing
+                           the script datum in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --tx-out-datum-hash-value JSON VALUE
                            The script datum hash for this tx output, by hashing
                            the script datum given here in JSON syntax. There is
                            no schema: (almost) any JSON value is supported,
                            including top-level strings and numbers.
-  --tx-out-datum-embed-file FILE
+  --tx-out-datum-embed-cbor-file CBOR FILE
                            The script datum to embed in the tx for this output,
                            in the given JSON file. The file must follow the
                            special JSON schema for script data.
+  --tx-out-datum-embed-json-file JSON FILE
+                           The script datum to embed in the tx for this output,
+                           in the given JSON file. The file must follow the
+                           special JSON schema for script data.
+  --tx-out-datum-embed-file JSON FILE
+                           The script datum to embed in the tx for this output,
+                           in the given JSON file. The file must follow the
+                           special JSON schema for script data. (DEPECRATED)
   --tx-out-datum-embed-value JSON VALUE
                            The script datum to embed in the tx for this output,
                            given here in JSON syntax. There is no schema:
@@ -116,9 +168,16 @@
                            syntax. You must specifiy a script witness.
   --mint-script-file FILE  The file containing the script to witness the minting
                            of assets for a particular policy Id.
-  --mint-redeemer-file FILE
+  --mint-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --mint-redeemer-json-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --mint-redeemer-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --mint-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,
@@ -136,9 +195,16 @@
   --certificate-script-file FILE
                            The file containing the script to witness the use of
                            the certificate.
-  --certificate-redeemer-file FILE
+  --certificate-redeemer-cbor-file CBOR FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+  --certificate-redeemer-json-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --certificate-redeemer-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --certificate-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,
@@ -152,9 +218,16 @@
   --withdrawal-script-file FILE
                            The file containing the script to witness the
                            withdrawal of rewards.
-  --withdrawal-redeemer-file FILE
+  --withdrawal-redeemer-cbor-file CBOR FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --withdrawal-redeemer-json-file JSON FILE
+                           The script redeemer, in the given JSON file. The file
+                           must follow the special JSON schema for script data.
+  --withdrawal-redeemer-file JSON FILE
                            The script redeemer, in the given JSON file. The file
                            must follow the special JSON schema for script data.
+                           (DEPECRATED)
   --withdrawal-redeemer-value JSON VALUE
                            The script redeemer, in JSON syntax. There is no
                            schema: (almost) any JSON value is supported,

@newhoggy
Copy link
Contributor Author

newhoggy commented Jan 8, 2022

This resolves #3472.

Notes about implementation are within the issue.

@newhoggy newhoggy force-pushed the newhoggy/cbor-support branch from 2b0262b to a4c1358 Compare January 8, 2022 11:06
<> Opt.help (helpTextForFile ++ " The file must follow the special \
\JSON schema for script data.")
)

pScriptDataValue =
pScriptDataFile = ScriptDataFile (Text.pack ("--" ++ dataFlagPrefix ++ "-file option is deprecated")) <$>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this being deprecated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think --tx-in-redeemer-json-file is clearer than --tx-in-redeemer-file, although I'm not sure if our deprecation policy allows this.

If we prefer to keep the original option name then I can revert that portion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say keep the original option name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@JaredCorduan
Copy link
Contributor

this is going to be very helpful, thank you @newhoggy !

@newhoggy newhoggy force-pushed the newhoggy/cbor-support branch 2 times, most recently from 6be6754 to fa61bb0 Compare January 12, 2022 14:01
@newhoggy
Copy link
Contributor Author

@JaredCorduan Do you have a recommended way of getting CBOR files for testing?

Copy link
Contributor

@Jimbo4350 Jimbo4350 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I think some of the lines go over the 100 char limit though.

@newhoggy newhoggy force-pushed the newhoggy/cbor-support branch 2 times, most recently from 3473089 to b631c9c Compare January 13, 2022 14:59
@newhoggy
Copy link
Contributor Author

Added some newlines to keep under the 100 char limit.

@newhoggy newhoggy force-pushed the newhoggy/cbor-support branch from b631c9c to 33121d1 Compare January 14, 2022 05:00
@JaredCorduan
Copy link
Contributor

@JaredCorduan Do you have a recommended way of getting CBOR files for testing?

@newhoggy in the ledger tests, we use this ruby program to generate random CBOR from a CDDL spec: https://rubygems.org/gems/cddl

The CDDL for datums is here: https://github.com/input-output-hk/cardano-ledger/blob/6fccece6415df6e8864985ddefdc30e0c1a6b8b6/eras/alonzo/test-suite/cddl-files/alonzo.cddl#L271-L276

If you want something simpler, I would suggest make a few examples conforming to that CDDL that use a variety of definite and indefinite lists and maps.

stuff

Use serialiser from plutus instead

Use serialiser from plutus instead
@newhoggy
Copy link
Contributor Author

Added some cbor datum files:

$ xxd scripts/plutus/data/42.datum.cbor
00000000: 182a                                     .*
$ xxd scripts/plutus/data/typed-42.datum.cbor
00000000: d879 9f18 2aff                           .y..*.

Which correspond to the JSON datum files:

$ cat scripts/plutus/data/42.datum
{"int":42}%
$ cat scripts/plutus/data/typed-42.datum
{"constructor":0,"fields":[{"int":42}]}%

Which according to https://cbor.nemo157.com/ are annotated as follows:

18 2a # unsigned(42)
d8 79       # tag(121)
   9f       #   array(*)
      18 2a #     unsigned(42)
      ff    #     break

@newhoggy
Copy link
Contributor Author

newhoggy commented Jan 24, 2022

Tested by hand editing example-txin-locking-plutus-script.sh to use the cbor flag instead.

@newhoggy newhoggy force-pushed the newhoggy/cbor-support branch from 33121d1 to 2d48896 Compare January 24, 2022 11:14
@newhoggy
Copy link
Contributor Author

bors merge

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jan 24, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants