CIP-0118? | Validation Zones

[polinavino]

We propose a set of changes that revolve around validation zones, a construct for allowing certain kinds of underspecified transactions. In particular, for the Babel-fees usecase we discuss here, we allow transactions that specify part of a swap request. A validation zone is a list of transactions such that earlier transactions in the list may be underspecified, while later transactions must complete all partial specifications. In the Babel-fees usecase, the completion of a specification is the fulfillment of a swap request. We discuss how validation zones for the Babel fees usecase can be generalized to a template for addressing a number of use cases from CPS-15.

---

📄 Rendered Proposal

[rphair]

thanks @polinavino ... really happy to see the continuation of this work. I'm marking the title with the obligatory ? because until merged the number (or its assignment at all) cannot be certain. Also I'm marking the prior version Likely Deprecated and will close as such with your confirmation:

• CIP-0118? | Validation zones #780

cc (for continuing review from the old proposal) @Quantumplation @fallen-icarus - p.s. cc (re: Rationale ["towards better design"]) @AndrewWestberg

[rphair]

I believe it is proper for this to be a separate PR from the first one, though that should be confirmed by other CIP editors today (https://hackmd.io/@cip-editors/93 - cc @Ryun1 @Crypto2099). Given the significance of the revision, the commit history in this case I think is more important than the discussion history & hopefully any prior discussion points will be summarised by previous reviewers here (@fallen-icarus @Quantumplation).

[rphair]

For consistency with other CIPs (mainly for review in parallel with 100+ others) this section needs to be broken into Acceptance and Implementation ... I guess since it refers to testing functionality then it would be on the Implementation path.

When done sifting material around in this section it will also help for these items to be GitHub formatted tickboxes (- [ ]) but I am not going to be pedantic about it especially at this stage.

[polinavino]

Sounds good, will do that later today!

[polinavino]

The new README-nested.md is a first draft of Nested Transactions.
It is based on (hopefully) best of #880 and previous versions Validation zones.

• No changes to block structure are required,
• Instead of a zone, we have a top-level transaction that contains sub-transactions (1 level deep only)
• An example of an additional kind of intent this design allows is given, which can be used to implement light client protocols (this is a demo of a potential approach to CPS-0015? | Intents for Cardano #779 )
• It is possible (but not mandatory) for transactions to see all other transactions in the batch

@rphair @Quantumplation @fallen-icarus @WhatisRT
@lehins since this is heavily inspired by Swaps and VZ criticism, would you like to be a coauthor?

There are still a couple of TODO's, and a Haskell prototype (built on the real ledger codebase) will be available shortly.

[Quantumplation]

I'm happy to weigh in on the proposal, but I definitely can't commit to being a coauthor 😅

[polinavino]

This new design makes it possible to define a script which (via conditioning on transactions fixed by requiredTx), checks that a specific other script (hash) must be run by another transaction in the same batch, or that payments made by other transactions satisfy some property.

Such conditioning can be done in two ways

• directly specifying requiredTxs for a given transaction, and also
• (as requested) by allowing your transaction to see all others in the batch and condition on those. This second option requires you to not provide your own ExUnits data, similar to the CIP-0131? | Transaction swaps #880 design, since you cannot locally check this without knowing all other transactions in the batch (if you could, you would be back to option 1)

[polinavino]

After some discussion with @willjgould and @lehins , we have come up with a way for sub-transactions to constrain the batches they are in - batch observers, see the updated README-nested.md. This is very similar to script observers and both can/will? be implemented together , however

• batch observers can be required by any transaction (using requireBatchObservers), but run only by a top-level transaction (it runs all the ones required by itself and its sub-txs)
• batch observers get special TxInfo versions that contain otherInfos : List TxInfo , specifying the data of all sub-transactions
• the TxInfo for all other script purposes does not contain otherInfos
• new script purpose BatchObs Ix

It is essentially impossible to run (calculate exUnits, etc.) batch observer scripts unless you are building the top-level tx, so it probably makes sense for off-chain code to give special instructions to the top-level tx builder in some high-level language about how to build it in a way that will satisfy the script (simpler than Plutus).

[fallen-icarus]

I don't understand what this means. The use of the terms "inputs" and "outputs" is confusing to me. In my vocabulary, "inputs" are UTxOs that a transaction intends to spend while "outputs" are UTxOs that would be created by the transaction. So I don't understand what it means to "specify outputs it intends to spend" or "[specify] the inputs that point to those outputs in the UTxO set". Can you explain what you mean by this?

[Quantumplation]

My interpretation is that "output" is the contents (address/datum/value), while "input" is the out_ref pointing to a specific output.

5. a new intent we call "spend-by-output", wherein a sub-transaction may specify the contents of the outputs it intends to spend, and the top-level transaction specifies the transaction references to those output in the UTxO set

[polinavino]

suppose tx has inputs txin1, txin2 and output o1

also, in the UTxO we have

txin1 |-> o
txin2 |-> o'\

then, we can build tx' that has spendOuts containing o, o' and output o1. A top-level transaction builder can then complete the batch with txTop, with (possibly some inputs), and

corInputs containing txin1, txin2

Then, in some sense, tx and tx'; txTop do a similar thing - remove entries corresponding to txin1, txin2 from the UTxO, and add o, o' to the UTxO. tx' would likely include a payment to whoever builds txTop for their tx-building services.

The reason we are proposing this is that if a light client (LC) is not following the chain, they would be unaware of the txins they would want to spend. If an LC requests a service provider to build a transaction for them that satisfies some specification (e.g. "pay key k the amount x of ada from key k'), the SP should be able to respond in a way that the LC does not use the data in this response to construct a new valid transaction that excludes payment to the SP. Obscuring inputs that SP can supply later makes it so that the transaction is not valid unless it comes with a top-level tx that provides the missing inputs.

[polinavino]

This design answers the question of "how can a user approve payment from their wallet without knowing anything about the chain except what the transaction spending their money is doing with their money"?

[fallen-icarus]

Can you possibly be a bit more explicit with this example? This CIP talks about underspecified transactions, but doesn't really give a user-friendly example of one. I believe this new version has adopted the approach from my Transaction Pieces, right? So it would be accurate to say:

• tx has inputs with 10 Ada and an output with 5 myT, and is left unbalanced like this.
• tx' has inputs with 5 myT and an output with 10 Ada.
• They are submitted together as a balanced batch with tx' as the top-level transaction.

I think I am biased by the original Validation Zones CIP that had unresolved holes, so I would personally find this example helpful to show there are no unresolved holes in this version.

[polinavino]

yes, this is exactly right.

[michele-nuzzi]

Hey everyone.

I didn't have a clear idea on this CIP for a while but recently I'm thinking that it doesn't bring a lot of value especially considering the cost of implementation (impact on the ledger etc...)

Most of what validations zone could achieve could instead be achieved leveraging utxo contention.

We can have multiple transactions spending different utxos of a contract, but the same utxos of a given user.
All these transactions are mutually exclusive and the outcome is indeterministic until one of the transactions makes it to a block.

If someone is worried about contention on the contract, the more utxos the contract can spend (and the user can use in mutually exclusive transactions), the less chance of contention by the contract.

This is effectively moving the load from the ledger to the consensus, that in theory should already be able to handle, without hardfork.

[michele-nuzzi]

BTW, I see and understand how validation zones could turn useful, and I not entirely opposed to it.

Just wanted to let you know of potential alternative possibilities

[AndrewWestberg]

@michele-nuzzi Signing multiple transactions is a UX nightmare. Also... how much are you willing to waste in min-utxo fractionalizing liquidity across enough utxos to avoid contention. The fact of the matter is that pooling liquidity will give a better UX.

Take for example a musician minting their 100m stream tokens on NEWM. They want to list these tokens for sale. They can put them into a pooled AMM with ~1.3 ada, or they can take your approach and fractionalize it into 1000 chunks of 100k tokens. This costs 1,300 ada to list it.

As you fractionalize further, you don't necessarily avoid utxo contention. It actually increases as you'd have to pull more utxos into a given transaction to perform the swap and even 1 of them conflicting is a failure of the whole transaction.

If you've guessed wrongly and either over-fractionalized or under-fractionalized based on end user demand for the amount of tokens they want to acquire, contention becomes more likely. As the fractions run out as people buy, you can either continue to fractionalize, which leads to contention when people are trying to grab amounts larger than the fraction size and need to take a high number of the fractions to do their swap... contention. Or, you don't continue to fractionalize and users are left fighting over the remaining fractions... contention.

I do look forward to what your catalyst proposal produces and how you plan to have users signing 2..3...20 duplicate transactions on their ledger device to "solve" the utxo contention issue. As you can tell from my response, I do not believe embracing utxo contention and fractionalizing liquidity is a viable approach.

[michele-nuzzi]

@AndrewWestberg I think we are basing our perspectives on different assumptions.

There would ideally be a sweet spot to fractionalize liquidity. I don't think it is realistic to assume 1000 orders can be executed in the same block, so over ~30 orders there would be no benefit.

A well designed contract would also make sure that some given utxos can only be spent together to further reduce contention (if we must spend two utxos together is like spending only one).

I would also point out that CIP 103 is currently active, allowing users to sign multiple transactions without impacting UX.

[michele-nuzzi]

Additionally relying on consensus rather than the ledger allows us more control over the transactions.

We can build many transactions that we directly control and look good enough for us rather than praying and hoping the nodes will pick favourable utxos for the transaction.

[AndrewWestberg]

If around 30 orders go into a block, but there are 1000s of users trying to swap, they will need to sign transactions against future state that is un-knowable. You're never going to get away from utxo contention being an extremely poor UX if there's any type of large demand. If I'm not in the first group of 30, there's no way for me to sign a tx(s) and come back and have my swap actually work. The utxos I signed against are long gone in any type of load scenario.

This CIP elegantly solves the problem.

[michele-nuzzi]

Once again we are basing our statements on different assumptions.

Assuming the 30 orders will change the part of the state that future transactions depend on is the side-effect of the single-threaded mentality that prevents us to fully leverage what the utxo model can offer.

In my opinion, this CIP incentivizes this single-threaded approach that in the long term is counterproductive for the utxo model.

[polinavino]

In my opinion, this CIP incentivizes this single-threaded approach that in the long term is counterproductive for the utxo model.

There will likely be cases for both single- and multi- threaded approaches, however, i can imagine many cases in which multi-threading would help also being situations where threads that are created by a single (or very few) entities. So, these same situations would benefit from that single entity instead setting up a Babel service which relieves the UTxO contention, removing the need for multi-threading.

[lehins]

This is redundant. NonEmpty subTxs already sets this flag to True, while empty means False

[lehins]

This was an old comment that I forgot to submit weeks ago 😄

[lehins]

How do you include the top level transaction? You can't because that would be recursive. Which means sub transactions should either never see the top level tx or always see it.

Moreover, making ability for scripts to see other txs as an option is a bad idea IMHO. It should be all or nothing!

[lehins]

Also old comment. Somehow I couldn't figure out on github to delete comments on the code that has disappeared

[lehins]

This will lead to a lot of duplication!!! Think of outputs with reference scripts or inline datums.

[lehins]

This goes back to making sub-txs in script context optional. This is totally unnecessary

[lehins]

I think this will be better than the latter below, since sub transactions are morally part of a transaction and LEDGER historically was a rule that validated a single transaction.

[lehins]

Why not include subTxs in the TxBody, instead of putting them into Tx and having hashes here?

One less indirection that we would need to check

[polinavino]

The short answer is that Agda made recursive types difficult in certain cases (including ones where there is a set of Tx's inside of Tx). In the Haskell prototype, it is done as you suggest I believe.

[lehins]

People hate script integrity hash, so we should go in the opposite direction and just include sub transaction in the body

[lehins]

You can instead link to this CIP: #749

[fallen-icarus]

@polinavino Can you update the Rendered Proposal link in the top comment? It is still linked to the first README. Some people are reading that one instead of the latest README-nested, and this is causing confusion among people trying to learn about this CIP.

[lehins]

@polinavino I'd would suggest getting rid of all other versions of README in this PR and only keep the single README-nested.md (renamed to README.md), since other approaches have no chance of being implemented for technical reasons and they only lead to confusion, as @fallen-icarus rightfully pointed out.