Non-normalized Values can get to the ScriptContext when in the emulator

[VictorCMiraldo]

Summary

IIUC, the cardano ledger uses Data.Map for values, hence, all the values in a ScriptContext are sorted and normalized by the time the validator gets them. It seems like the emulator doesn't perform this normalization/sorting and some library functions will actively generate "bad" values. One such example is adjustUnbalancedTx, which uses adjustTxOut, which in turn might return a txOut { txOutValue = txOutValue txOut <> Ada.toValue missingLovelace }. Note how the missingLovelace is added at the tail of the value, where it should have been added at the head or the values should be sorted to conform to how the ledger creates script contexts.

This is related to:

• Discrepancy between plutus-apps and cardano-ledger on handling values. #604
• Why doesn't EmulatorTrace use the ledger rules? #611
• Undocumented invariants in types with exposed constructors plutus#4476

Steps to reproduce the behavior

1. write a validator that returns whether its continuing output Value is sorted; that script is equivalent to const True in the actual chain.
2. create a transaction which is adjusted with adjustUnbalancedTx and submit it through the emulator, said script will now fail to validate.

Actual Result

see above

Expected Result

The validator shouldn't be able to distinguish if it is running from the emulator or from the ledger.

Describe the approach you would take to fix this

At this point, it would be good to have a concrete list of assumptions on the invariants that the different datatypes defined by plutus-apps need to respect and how these should be enforced. Is it up to us to sort these off-chain? Which of those invariants
are actually a specification? Is the validator even supposed to assume that values are sorted?

System info

n/a

[peter-mlabs]

@VictorCMiraldo:

At this point, it would be good to have a concrete list of assumptions on the invariants that the different datatypes defined by plutus-apps need to respect and how these should be enforced. (...) Which of those invariants are actually a specification?

You can find a more in-depth discussion on the list of invariants ensured by the ledger and/or serialization rules here. The bottom line is that these aren't very straight-forward, and you'll likely need to be intimately familiar with the spec and CDDL to get a full understanding.

Adding more documentation was discussed here.

(...) Is it up to us to sort these off-chain? (...) Is the validator even supposed to assume that values are sorted?

The it is up to developers to ensure the invariants both exist and are upheld when building up values (and I mean this in the "Value" sense, and "inhabitant of a type" sense). Values constructed within scripts do not "magically" get normalized, but the values presented to scripts via the script context will be normalized. (If this distinction isn't clear, I can elaborate).

However:

• the emulator does not ensure that values are normalized
• it is trivial to construct non-normalized values via plutus-ledger-api
• the plutus team considers this out-of-scope for the project

If you would like to avoid these pitfalls, utilities you may find helpful are:

• plutus-simple-model, which uses functions from cardano-ledger to ensure that invariants are uphled
• plutip, which runs transactions (either from PAB or CTL) on an actual local testnet, so you'll be getting as-close-as-possible to real-world behavior.

---

On another note, if you're working with Value, be aware that the Show instance will print sorted even if the underlying value is not. I burned a lot of time earlier today dealing with this, because the values "looked" sorted:

-- unsorted association list: "ddef" currency symbol comes first
ghci> unsorted
Map {unMap = [("ddef",Map {unMap = [("",1)]}),("abcd",Map {unMap = [("",1)]})]}

-- printed as a map, "abcd" comes first
ghci> Value unsorted
Value (Map [(abcd,Map [("",1)]),(ddef,Map [("",1)])])

-- round-tripping shows its still unsorted
ghci> getValue $ Value unsorted
Map {unMap = [(ddef,Map {unMap = [("",1)]}),(abcd,Map {unMap = [("",1)]})]}

[VictorCMiraldo]

Thanks for the answer @peter-mlabs! I believe that the efforts and discussions you linked are paramount for the Plutus ecosystem!

I'm no longer involved with Plutus, however. I'll tag @mmontin here, who will probably benefit from this information!