Cost models for `LookupCoin`, `ValueContains`, `ValueData`, `UnValueData` builtins #7344

Unisay · 2025-09-18T08:01:22Z

Summary

This PR implements cost modeling for Value-related builtins: lookupCoin, valueContains, valueData, and unValueData.

Implementation

Complete cost modeling pipeline:

Cost model infrastructure and parameter definitions
Benchmarking framework with realistic Cardano constraints
Statistical analysis with R models (linear/constant based on performance characteristics)
Updated JSON cost model configurations across all versions

Cost models:

valueData: Uses constant cost model based on uniform performance analysis
lookupCoin: Linear cost model with dimension reduction for 3+ parameters
valueContains: Linear cost model for container/contained size dependency
unValueData: Linear cost model for size-dependent deserialization

All functions now have proper cost models instead of unimplemented placeholders.

github-actions · 2025-09-18T11:17:16Z

PR Preview Action v1.6.2
🚀 View preview at https://IntersectMBO.github.io/plutus/pr-preview/pr-7344/
Built to branch `gh-pages` at 2025-09-19 08:01 UTC. Preview will be ready when the GitHub Pages deployment is complete.

kwxm

Here are some initial comments. I'll come back and add some more later. I need to look at the benchmarks properly though.

plutus-core/plutus-core/src/PlutusCore/Evaluation/Machine/ExMemoryUsage.hs

zliu41 · 2025-10-09T23:34:07Z

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs

+
+-- | Generate random key as ByteString (for lookup arguments)
+generateKeyBS :: (StatefulGen g m) => g -> m ByteString
+generateKeyBS = uniformByteStringM Value.maxKeyLen


If the keys are completely random, then lookupCoin will probably never hit an existing entry, right?

lookupCoin will probably never hit an existing entry,

Maybe that's what we want? Do we know if finding out that something's not in the map is the worst case? Naively you might think that the time taken to discover that some key is not in the map is always greater or equal to the time taken to find a key that is in the map.

I don't agree. I think we should actively include both the case when the map contains the key and when it doesn't. Otherwise we're not really measuring this case, and that's the whole point of benchmarking, right? Otherwise we would just use the, analytically discovered, worst-time complexity of the algorithm and pick a function from that category for its cost, right?

Also, as I mentioned above, you won't have a good idea of the actual size of the Value if you don't enforce uniqueness of the keys.

Unisay · 2025-10-10T11:35:47Z

I have simplified the generators (less fixed values, more randomly generated samples, quantities are all maxBound :: Int64)

After that I've re-benchmarked and re-generated cost models. This is how I view them:

LookupCoin

ValueContains

ValueData

UnValueData

CC: @kwxm

zliu41

In order to benchmark the worst case, I think you should also ensure that lookupCoin always hits the largest inner map (or at least, such cases should be well-represented).

Also, we'll need to re-run benchmarking for unValueData after adding the enforcement of integer range.

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

kwxm · 2025-10-10T17:40:57Z

plutus-core/cost-model/data/benching-conway.csv

@@ -12094,203 +12094,710 @@ IndexArray/42/1,1.075506579052359e-6,1.0748433439930302e-6,1.0762684407023462e-6
 IndexArray/46/1,1.0697135554442532e-6,1.0690902192698813e-6,1.0704133377013816e-6,2.2124820728450233e-9,1.8581237858977844e-9,2.6526943923047553e-9
 IndexArray/98/1,1.0700747499373992e-6,1.0693842628239684e-6,1.070727062396803e-6,2.2506114869928674e-9,1.9376849028666025e-9,2.7564941558204088e-9
 IndexArray/82/1,1.0755056682976695e-6,1.0750405368241111e-6,1.076102212770973e-6,1.8355219893844098e-9,1.5161640335164335e-9,2.4443625958006994e-9
-Bls12_381_G1_multiScalarMul/1/1,8.232134704712041e-5,8.228195390475752e-5,8.23582682466318e-5,1.224261187989977e-7,9.011720721178711e-8,1.843107342917502e-7


GitHub seeems to think that the data for all of the BLS functions has changed, but I don't think they have.

The file on master contains Windows-style line terminators (\r\n) for BLS lines:

git show master:plutus-core/cost-model/data/benching-conway.csv | grep "Bls12_381_G1_multiScalarMul/1/1" | od -c | grep -C1 "\r" 0000000 B l s 1 2 _ 3 8 1 _ G 1 _ m u l 0000020 t i S c a l a r M u l / 1 / 1 , 0000040 8 . 2 3 2 1 3 4 7 0 4 7 1 2 0 4 -- 0000200 8 7 1 1 e - 8 , 1 . 8 4 3 1 0 7 0000220 3 4 2 9 1 7 5 0 2 e - 7 \r \n

This PR changes \r\n to \n .

plutus-core/cost-model/data/builtinCostModelA.json

plutus-core/cost-model/data/models.R

Extends the cost modeling framework to support lookupCoin, valueContains, valueData, and unValueData builtins. Adds parameter definitions, arity specifications, and integrates with the cost model generation system. Establishes foundation for accurate costing of Value operations in Plutus Core execution.

Creates Values.hs benchmark module with systematic test generation for lookupCoin, valueContains, valueData, and unValueData operations. Includes value generation utilities, individual benchmark functions, and edge case testing with empty values. Enables data collection for accurate cost model parameter fitting.

Implements optimal statistical models for Value operations based on performance characteristics: linear models for lookupCoin and valueContains (size-dependent), constant model for valueData (uniform performance), and linear model for unValueData. Provides accurate cost parameters across all builtin cost model configurations and updates test expectations.

Removes unimplementedCostingFun placeholders for Value builtins and connects them to their respective cost model parameters (paramLookupCoin, paramValueContains, paramValueData, paramUnValueData). Enables accurate execution cost calculation for Value operations in Plutus Core scripts.

Includes extensive benchmark results covering various input sizes and edge cases for lookupCoin, valueContains, valueData, and unValueData. Data validates the chosen statistical models and cost parameters. Provides empirical foundation confirming model accuracy across different operation profiles.

Add a new Logarithmic newtype wrapper in ExMemoryUsage that transforms size measures logarithmically. This enables linear cost models to effectively capture O(log n) runtime behavior by measuring log(size) instead of size directly. The wrapper computes max(1, floor(log2(size) + 1)) from any wrapped ExMemoryUsage instance, making it composable with existing size measures like ValueOuterOrMaxInner for operations with logarithmic complexity. This infrastructure supports proper costing of Value builtins like lookupCoin which has O(log max(m, k)) complexity.

Refactor the Value benchmarking suite to use Cardano-compliant key sizes (32-byte max) and leverage the new Logarithmic wrapper for accurate modeling of logarithmic operations. Key changes: - Apply Logarithmic wrapper to lookupCoin and valueContains benchmarks for proper O(log n) cost modeling - Consolidate key generators from 4 functions to 2, eliminating duplication - Remove obsolete key size parameters throughout (keys always maxKeyLen) - Extract withSearchKeys pattern to eliminate repetitive code - Simplify test generation by removing arbitrary key size variations - Clean up lookupCoinArgs structure for better readability The refactoring reduces the module from 359 to 298 lines while improving clarity and ensuring all generated Values comply with Cardano's 32-byte key length limit.

Simplify the R model definitions for Value-related builtins by replacing custom linear model implementation with standard linearInY wrapper for valueContains. This maintains the same statistical behavior while improving code maintainability. Add inline comments documenting the parameter wrapping strategy used for each model (Logarithmic wrapping for lookupCoin/valueContains, ValueTotalSize for contains operand, unwrapped for valueData/unValueData). Clean up formatting inconsistencies in model definitions.

Refreshed benchmarking data for lookupCoin, valueContains, valueData, and unValueData with improved statistical coverage and sampling. This data serves as the foundation for the refined cost model parameters applied in the subsequent commit.

Updated cost parameters based on fresh benchmark data analysis: - lookupCoin: Adjusted intercept (284421→179661) and slope (1→7151) to better reflect actual performance with varying currency counts - valueContains: Changed from added_sizes to linear_in_y model with refined parameters (intercept 42125119→1000, slope 30→130383) - valueData: Reduced constant cost (205465→153844) based on updated profiling results - unValueData: Switched to linear_in_x model with refined parameters (intercept 10532326261→1000, slope 431→33094) All three cost model variants (A, B, C) updated for consistency.

Modernize logarithm calculation in the Logarithmic ExMemoryUsage instance by switching from the compatibility module GHC.Integer.Logarithms to the modern GHC.Num.Integer API. Changes: - Replace integerLog2# (unboxed, from GHC.Integer.Logarithms) with integerLog2 (boxed, from GHC.Num.Integer) - Simplify code by removing unboxing boilerplate: I# (integerLog2# x) becomes integerLog2 x - Keep other imports (GHC.Integer.Logarithms, GHC.Exts) as they are still used elsewhere in the file (memoryUsageInteger function) This addresses code review feedback to use the modern ghc-bignum API instead of the legacy compatibility module, while maintaining the same computational semantics. Cost model regeneration verified no regression in derived parameters.

Address Kenneth's review comment by ensuring builtins use the same size measure wrappers as their budgeting benchmarks. Changes: - Add LogValueOuterOrMaxInner newtype combining logarithmic transformation with outer/max inner size measurement - Update lookupCoin and valueContains to use size measure wrappers - Add KnownTypeAst instances for ValueTotalSize and LogValueOuterOrMaxInner - Update benchmarks to use new combined wrapper type This ensures the cost model accurately reflects runtime behavior by using identical size measures in both denotations and benchmarks.

Regenerate cost model parameters based on fresh benchmark runs for the four Value-related built-in functions: lookupCoin, valueContains, valueData, and unValueData. New cost models: - lookupCoin: linear_in_z (intercept: 209937, slope: 7181) - valueContains: linear_in_y (intercept: 1000, slope: 131959) - valueData: constant_cost (182815) - unValueData: linear_in_x (intercept: 1000, slope: 33361) The benchmark data includes 350 measurement points across varying input sizes to ensure accurate cost estimation. All three cost model variants (A, B, C) have been updated consistently with identical parameters.

Document the regeneration of benchmark data and cost model parameters for the four Value-related built-in functions following fresh benchmark measurements.

…verhead Regenerate cost model parameters based on fresh benchmark runs after rebasing on master. This accounts for the negative amount validation added to valueContains in commit 531f1b8. Updated cost models: - lookupCoin: linear_in_z (intercept: 203599, slope: 7256) - valueContains: linear_in_y (intercept: 1000, slope: 130720) - valueData: constant_cost (156990) - unValueData: linear_in_x (intercept: 1000, slope: 36194) The benchmark data includes 350 measurement points across varying input sizes. All three cost model variants (A, B, C) have been updated consistently with identical parameters.

Replace local benchmark data with results from GitHub Actions remote execution and regenerate cost model parameters for the four Value-related builtins: lookupCoin, valueContains, valueData, and unValueData. Remote benchmarking provides more consistent and reliable measurements by running on standardized infrastructure, eliminating local environment variations that could affect cost model accuracy. Updated parameters across all cost model versions (A, B, C): - lookupCoin: intercept 203599→210606, slope 7256→8019 - valueContains: slope 130720→94161 - valueData: constant 156990→162241 - unValueData: slope 36194→15417

Reformat builtin cost model JSON files to use consistent 4-space indentation instead of 2-space indentation. This improves readability and aligns with common JSON formatting conventions for configuration files. No semantic changes - only whitespace formatting updated. Files affected: - builtinCostModelA.json - builtinCostModelB.json - builtinCostModelC.json

Co-authored-by: Kenneth MacKenzie <kenneth.mackenzie@iohk.io>

Unisay · 2025-10-13T11:04:09Z

plutus-core/cost-model/data/builtinCostModelA.json

        "cpu": {
            "arguments": {
-               "intercept": 107878,
+                "intercept": 107878,


This was a 3-space indent, inconsistent with 4-space used in all other lines.

Apply the same optimization used in the Logarithmic instance to memoryUsageInteger, using integerLog2 directly instead of unboxed integerLog2# and quotInt# operations. This allows us to remove: - MagicHash language extension - GHC.Exts imports (Int (I#), quotInt#) - GHC.Integer and GHC.Integer.Logarithms imports The refactoring maintains identical functionality while making the code more consistent and simpler.

Simplifies the memory usage measurement by consolidating three separate types (Logarithmic, ValueOuterOrMaxInner, LogValueOuterOrMaxInner) into a single ValueLogOuterOrMaxInner type. This reduces complexity while maintaining the same functionality for measuring logarithmic Value sizes. The new type directly encodes the intended semantics: size = log(max(outer, maxInner)), making the code more maintainable and producing clearer type signatures in builtin function definitions.

Replaces unsafe fromJust usage with explicit error messages and HasCallStack constraint in costModelParamsForTesting. This provides better debugging context when cost model parameter extraction fails, including stack traces that pinpoint the exact call site.

Adds cost model parameter names for LookupCoin, ValueContains, ValueData, and UnValueData builtins (11 new parameters per ledger API version). Updates parameter count expectations to reflect the expanded parameter set. Updates golden type signatures and conformance test budget expectations to reflect the refined ValueLogOuterOrMaxInner type signature, ensuring accurate cost accounting for Value-based operations.

ana-pantilie

I'm only just understanding the philosophy behind costing, so I might be wrong. I think it's important we both generate random inputs and inputs which hit the various edge-cases the algorithm has. The benchmarking data we generate should be a sample which describes the algorithm's behavior as completely as possible.

ana-pantilie · 2025-10-14T12:15:16Z

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs

+  commonWithKeys <- mapM (withSearchKeys g . pure) testValues
+
+  -- Additional tests specific to lookupCoin
+  let valueSizes = [(100, 10), (500, 20), (1_000, 50), (2_000, 100)]


What is the reasoning behind picking these specific sizes?

Most importantly, however, you're not actually generating Values of these sizes, because you're not checking whether the keys you generate are unique per map or not.

ana-pantilie · 2025-10-14T12:18:24Z

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs

+
+-- | Generate random key as ByteString (for lookup arguments)
+generateKeyBS :: (StatefulGen g m) => g -> m ByteString
+generateKeyBS = uniformByteStringM Value.maxKeyLen


I don't agree. I think we should actively include both the case when the map contains the key and when it doesn't. Otherwise we're not really measuring this case, and that's the whole point of benchmarking, right? Otherwise we would just use the, analytically discovered, worst-time complexity of the algorithm and pick a function from that category for its cost, right?

ana-pantilie · 2025-10-14T12:21:26Z

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs

+
+-- | Generate random key as ByteString (for lookup arguments)
+generateKeyBS :: (StatefulGen g m) => g -> m ByteString
+generateKeyBS = uniformByteStringM Value.maxKeyLen


Also, as I mentioned above, you won't have a good idea of the actual size of the Value if you don't enforce uniqueness of the keys.

kwxm · 2025-10-14T16:54:52Z

I think it's important we both generate random inputs and inputs which hit the various edge-cases the algorithm has. The benchmarking data we generate should be a sample which describes the algorithm's behavior as completely as possible.

In general we want the costing function to describe the worst-case behaviour of the builtin, which means that the benchmarks should be run with inputs which produce worst-case behaviour. If we feed them totally random inputs then we'll fit a costing function which describes the average-case behaviour, and this could be dangerous. For example if a function takes 1ms on average but there are particular inputs which cause it to take 20ms then the cost model has to charge 20ms for all inputs so that we don't undercharge for scripts that do actually exercise the worst case (perhaps repeatedly) [An example of this kind of behaviour would be for equalsByteString, where we only benchmark the case where the two inputs are equal: if they're not equal (in particular if they have different lengths) then the function can return very quickly, but it they are equal the builtin will have to examine every byte of the input]. For some builtins the worst case won't be significantly worse than the average case, and then we miight be able to get away with random inputs, especially if it's hard to generate worst case inputs.

In the case of lookupCoin I'm not sure what the worst case is. It may well be when the thing you're looking up isn't present in the map, but we should check whether this is actually true.

Addendum. It's often necessary to do a lot of exploratory benchmarking to check that the builtin behaves as you're expecting, and this isn't generally reflected in the final costing benchmarks and costing function (for example, see the costing branches for expModInteger here and here, where I ran dozens of different benchmarks with different inputs before settling on the final version). So we're not just choosing a costing function and b,lindly fitting branhcmark results to it, we're checking that our initial assumptions are correct and modifying them if necessary. It might be useful to do this with the lookup/insertion/union costs as well.

zliu41 · 2025-10-22T15:39:20Z

In the case of lookupCoin I'm not sure what the worst case is. It may well be when the thing you're looking up isn't present in the map, but we should check whether this is actually true.

Like what I said earlier, for lookupCoin the worst case would be: the currency hits the largest inner map, and the token does not exist in the inner map. Because this involves lookups in both the outer map and inner map. @Unisay Let's make sure that the case where the currency hits the largest inner map is well represented in the data.

kwxm · 2025-10-22T22:12:06Z

In the case of lookupCoin I'm not sure what the worst case is. It may well be when the thing you're looking up isn't present in the map, but we should check whether this is actually true.

Like what I said earlier, for lookupCoin the worst case would be: the currency hits the largest inner map, and the token does not exist in the inner map. Because this involves lookups in both the outer map and inner map. @Unisay Let's make sure that the case where the currency hits the largest inner map is well represented in the data.

What about the case when the outer map is really big compared to the inner ones? Also, we could make all of the inner maps the same size. We probably don't want to be benchmarking anything that isn't the worst case.

zliu41 · 2025-10-22T22:59:14Z

What about the case when the outer map is really big compared to the inner ones?

Isn't the worst case in that case still performing lookup in both the outer map and inner map? If you are concerned that the outer map key is not at the leaf, you can use either the smallest key or the largest key.

By the way, to make key comparisons more expensive, it would also be useful to fix first 30 or 31 bytes, and only vary the last byte.

Unisay temporarily deployed to github-pages September 18, 2025 08:01 — with GitHub Actions Inactive

Unisay self-assigned this Sep 18, 2025

Unisay temporarily deployed to github-pages September 19, 2025 08:00 — with GitHub Actions Inactive

Unisay force-pushed the yura/costing-builtin-value branch 6 times, most recently from 528ebcd to 69f1d6f Compare September 24, 2025 16:06

Unisay changed the title ~~WIP: Add costing for lookupCoin and valueContains builtins~~ Cost models for LookupCoin, ValueContains, ValueData, UnValueData builtins Sep 24, 2025

Unisay marked this pull request as ready for review September 24, 2025 16:24

Unisay requested review from ana-pantilie and kwxm September 24, 2025 16:41

Unisay force-pushed the yura/costing-builtin-value branch 3 times, most recently from 53d9ea1 to 5b60cfc Compare September 30, 2025 10:15

ana-pantilie mentioned this pull request Oct 1, 2025

BuiltinValue: insertCoin and unionValue costing #7337

Closed

Unisay force-pushed the yura/costing-builtin-value branch from 5b60cfc to 7eebe28 Compare October 2, 2025 09:43

kwxm reviewed Oct 7, 2025

View reviewed changes

Unisay force-pushed the yura/costing-builtin-value branch from b1a6bf1 to 6afef50 Compare October 9, 2025 14:11

Unisay requested a review from zliu41 October 9, 2025 14:20

zliu41 reviewed Oct 9, 2025

View reviewed changes

Unisay force-pushed the yura/costing-builtin-value branch from 3cee663 to 86d645a Compare October 10, 2025 10:26

zliu41 reviewed Oct 10, 2025

View reviewed changes

kwxm reviewed Oct 10, 2025

View reviewed changes

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs Outdated Show resolved Hide resolved

kwxm reviewed Oct 10, 2025

View reviewed changes

plutus-core/cost-model/data/builtinCostModelA.json Outdated Show resolved Hide resolved

kwxm reviewed Oct 10, 2025

View reviewed changes

plutus-core/cost-model/data/models.R Outdated Show resolved Hide resolved

Unisay and others added 18 commits October 13, 2025 12:24

docs: add changelog entry for Value builtin cost model updates

109f444

Document the regeneration of benchmark data and cost model parameters for the four Value-related built-in functions following fresh benchmark measurements.

feat: update constant costs for builtin value operations

49f24a0

Unisay force-pushed the yura/costing-builtin-value branch from 680af99 to bff4bf2 Compare October 13, 2025 10:42

chore: space

071c8e4

Co-authored-by: Kenneth MacKenzie <kenneth.mackenzie@iohk.io>

Unisay commented Oct 13, 2025

View reviewed changes

Unisay added 4 commits October 13, 2025 13:09

ana-pantilie requested changes Oct 14, 2025

View reviewed changes

Cost models for LookupCoin, ValueContains, ValueData, UnValueData builtins #7344

Are you sure you want to change the base?

Cost models for LookupCoin, ValueContains, ValueData, UnValueData builtins #7344

Uh oh!

Conversation

Unisay commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implementation

Uh oh!

github-actions bot commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Built to branch gh-pages at 2025-09-19 08:01 UTC. Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

kwxm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Unisay commented Oct 10, 2025

LookupCoin

ValueContains

ValueData

UnValueData

Uh oh!

zliu41 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kwxm Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Unisay Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ana-pantilie left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kwxm commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zliu41 commented Oct 22, 2025

Uh oh!

kwxm commented Oct 22, 2025

Uh oh!

zliu41 commented Oct 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Cost models for `LookupCoin`, `ValueContains`, `ValueData`, `UnValueData` builtins #7344

Cost models for `LookupCoin`, `ValueContains`, `ValueData`, `UnValueData` builtins #7344

Unisay commented Sep 18, 2025 •

edited

Loading

github-actions bot commented Sep 18, 2025 •

edited

Loading

Built to branch `gh-pages` at 2025-09-19 08:01 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

kwxm Oct 10, 2025 •

edited

Loading

Unisay Oct 13, 2025 •

edited

Loading

ana-pantilie left a comment •

edited

Loading

kwxm commented Oct 14, 2025 •

edited

Loading