Skip to content

Commit

Permalink
Kwxm/provisional conway cost model (#6087)
Browse files Browse the repository at this point in the history
* Rename semantics variants

* Change some names

* Get uplc to use correct cost model for semvar

* Dumping cost models

* Dumping cost models

* Tidy up exports and tests

* Move a function

* More updates

* Fix some typos

* Update models.R

* Add updated cost models

* Add updated cost models

* Update test results

* Remove obsolete entries from V3.ParamName

* Fix cost model interface test

* Fix (sort of) plutus-ledger-api tests

* Various small fixes

* Forgot to update parameters

* Add changelog entries

* Fix the changelog entries

* Address PR review comments

* Add costs for new builtins to V1 and V2 cost model files

* Fix some dependencies in plutus-benchmark

* Fix partial reversion of #6086 caused by merge

* Remove old cost model files

* Update/rename benchmark CSV files
  • Loading branch information
Kenneth MacKenzie authored May 29, 2024
1 parent d7a5c76 commit 00beff4
Show file tree
Hide file tree
Showing 726 changed files with 13,000 additions and 1,266 deletions.
146 changes: 73 additions & 73 deletions plutus-benchmark/bls12-381-costs/test/9.6/bls12-381-costs.golden
Original file line number Diff line number Diff line change
Expand Up @@ -2,144 +2,144 @@ Hash n bytestrings onto G1 and add points

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 115 (0.7%) 17294189 (0.2%) 3918 (0.0%)
10 224 (1.4%) 693350079 (6.9%) 37278 (0.3%)
20 334 (2.0%) 1369405969 (13.7%) 70638 (0.5%)
30 444 (2.7%) 2045461859 (20.5%) 103998 (0.7%)
40 554 (3.4%) 2721517749 (27.2%) 137358 (1.0%)
50 664 (4.1%) 3397573639 (34.0%) 170718 (1.2%)
60 774 (4.7%) 4073629529 (40.7%) 204078 (1.5%)
70 884 (5.4%) 4749685419 (47.5%) 237438 (1.7%)
80 994 (6.1%) 5425741309 (54.3%) 270798 (1.9%)
90 1104 (6.7%) 6101797199 (61.0%) 304158 (2.2%)
100 1214 (7.4%) 6777853089 (67.8%) 337518 (2.4%)
110 1324 (8.1%) 7453908979 (74.5%) 370878 (2.6%)
120 1434 (8.8%) 8129964869 (81.3%) 404238 (2.9%)
130 1544 (9.4%) 8806020759 (88.1%) 437598 (3.1%)
140 1654 (10.1%) 9482076649 (94.8%) 470958 (3.4%)
150 1764 (10.8%) 10158132539 (101.6%) 504318 (3.6%)
0 115 (0.7%) 53556222 (0.5%) 3918 (0.0%)
10 224 (1.4%) 593877682 (5.9%) 37278 (0.3%)
20 334 (2.0%) 1134199142 (11.3%) 70638 (0.5%)
30 444 (2.7%) 1674520602 (16.7%) 103998 (0.7%)
40 554 (3.4%) 2214842062 (22.1%) 137358 (1.0%)
50 664 (4.1%) 2755163522 (27.6%) 170718 (1.2%)
60 774 (4.7%) 3295484982 (33.0%) 204078 (1.5%)
70 884 (5.4%) 3835806442 (38.4%) 237438 (1.7%)
80 994 (6.1%) 4376127902 (43.8%) 270798 (1.9%)
90 1104 (6.7%) 4916449362 (49.2%) 304158 (2.2%)
100 1214 (7.4%) 5456770822 (54.6%) 337518 (2.4%)
110 1324 (8.1%) 5997092282 (60.0%) 370878 (2.6%)
120 1434 (8.8%) 6537413742 (65.4%) 404238 (2.9%)
130 1544 (9.4%) 7077735202 (70.8%) 437598 (3.1%)
140 1654 (10.1%) 7618056662 (76.2%) 470958 (3.4%)
150 1764 (10.8%) 8158378122 (81.6%) 504318 (3.6%)


Hash n bytestrings onto G2 and add points

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 163 (1.0%) 33886964 (0.3%) 3936 (0.0%)
10 272 (1.7%) 2105540474 (21.1%) 37656 (0.3%)
20 382 (2.3%) 4177193984 (41.8%) 71376 (0.5%)
30 492 (3.0%) 6248847494 (62.5%) 105096 (0.8%)
40 602 (3.7%) 8320501004 (83.2%) 138816 (1.0%)
50 712 (4.3%) 10392154514 (103.9%) 172536 (1.2%)
60 822 (5.0%) 12463808024 (124.6%) 206256 (1.5%)
70 932 (5.7%) 14535461534 (145.4%) 239976 (1.7%)
80 1042 (6.4%) 16607115044 (166.1%) 273696 (2.0%)
90 1152 (7.0%) 18678768554 (186.8%) 307416 (2.2%)
100 1262 (7.7%) 20750422064 (207.5%) 341136 (2.4%)
110 1372 (8.4%) 22822075574 (228.2%) 374856 (2.7%)
120 1482 (9.0%) 24893729084 (248.9%) 408576 (2.9%)
130 1592 (9.7%) 26965382594 (269.7%) 442296 (3.2%)
140 1702 (10.4%) 29037036104 (290.4%) 476016 (3.4%)
150 1812 (11.1%) 31108689614 (311.1%) 509736 (3.6%)
0 163 (1.0%) 75306572 (0.8%) 3936 (0.0%)
10 272 (1.7%) 1769766432 (17.7%) 37656 (0.3%)
20 382 (2.3%) 3464226292 (34.6%) 71376 (0.5%)
30 492 (3.0%) 5158686152 (51.6%) 105096 (0.8%)
40 602 (3.7%) 6853146012 (68.5%) 138816 (1.0%)
50 712 (4.3%) 8547605872 (85.5%) 172536 (1.2%)
60 822 (5.0%) 10242065732 (102.4%) 206256 (1.5%)
70 932 (5.7%) 11936525592 (119.4%) 239976 (1.7%)
80 1042 (6.4%) 13630985452 (136.3%) 273696 (2.0%)
90 1152 (7.0%) 15325445312 (153.3%) 307416 (2.2%)
100 1262 (7.7%) 17019905172 (170.2%) 341136 (2.4%)
110 1372 (8.4%) 18714365032 (187.1%) 374856 (2.7%)
120 1482 (9.0%) 20408824892 (204.1%) 408576 (2.9%)
130 1592 (9.7%) 22103284752 (221.0%) 442296 (3.2%)
140 1702 (10.4%) 23797744612 (238.0%) 476016 (3.4%)
150 1812 (11.1%) 25492204472 (254.9%) 509736 (3.6%)


Uncompress n G1 points and add the results

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 112 (0.7%) 17294189 (0.2%) 3918 (0.0%)
10 661 (4.0%) 196953159 (2.0%) 35278 (0.3%)
20 1211 (7.4%) 376612129 (3.8%) 66638 (0.5%)
30 1761 (10.7%) 556271099 (5.6%) 97998 (0.7%)
40 2311 (14.1%) 735930069 (7.4%) 129358 (0.9%)
50 2861 (17.5%) 915589039 (9.2%) 160718 (1.1%)
60 3411 (20.8%) 1095248009 (11.0%) 192078 (1.4%)
70 3961 (24.2%) 1274906979 (12.7%) 223438 (1.6%)
80 4511 (27.5%) 1454565949 (14.5%) 254798 (1.8%)
90 5061 (30.9%) 1634224919 (16.3%) 286158 (2.0%)
100 5611 (34.2%) 1813883889 (18.1%) 317518 (2.3%)
110 6161 (37.6%) 1993542859 (19.9%) 348878 (2.5%)
120 6711 (41.0%) 2173201829 (21.7%) 380238 (2.7%)
130 7261 (44.3%) 2352860799 (23.5%) 411598 (2.9%)
140 7811 (47.7%) 2532519769 (25.3%) 442958 (3.2%)
150 8361 (51.0%) 2712178739 (27.1%) 474318 (3.4%)
0 112 (0.7%) 53556222 (0.5%) 3918 (0.0%)
10 661 (4.0%) 597620792 (6.0%) 35278 (0.3%)
20 1211 (7.4%) 1141685362 (11.4%) 66638 (0.5%)
30 1761 (10.7%) 1685749932 (16.9%) 97998 (0.7%)
40 2311 (14.1%) 2229814502 (22.3%) 129358 (0.9%)
50 2861 (17.5%) 2773879072 (27.7%) 160718 (1.1%)
60 3411 (20.8%) 3317943642 (33.2%) 192078 (1.4%)
70 3961 (24.2%) 3862008212 (38.6%) 223438 (1.6%)
80 4511 (27.5%) 4406072782 (44.1%) 254798 (1.8%)
90 5061 (30.9%) 4950137352 (49.5%) 286158 (2.0%)
100 5611 (34.2%) 5494201922 (54.9%) 317518 (2.3%)
110 6161 (37.6%) 6038266492 (60.4%) 348878 (2.5%)
120 6711 (41.0%) 6582331062 (65.8%) 380238 (2.7%)
130 7261 (44.3%) 7126395632 (71.3%) 411598 (2.9%)
140 7811 (47.7%) 7670460202 (76.7%) 442958 (3.2%)
150 8361 (51.0%) 8214524772 (82.1%) 474318 (3.4%)


Uncompress n G2 points and add the results

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 160 (1.0%) 33886964 (0.3%) 3936 (0.0%)
10 1189 (7.3%) 392603584 (3.9%) 35656 (0.3%)
20 2219 (13.5%) 751320204 (7.5%) 67376 (0.5%)
30 3249 (19.8%) 1110036824 (11.1%) 99096 (0.7%)
40 4279 (26.1%) 1468753444 (14.7%) 130816 (0.9%)
50 5309 (32.4%) 1827470064 (18.3%) 162536 (1.2%)
60 6339 (38.7%) 2186186684 (21.9%) 194256 (1.4%)
70 7369 (45.0%) 2544903304 (25.4%) 225976 (1.6%)
80 8399 (51.3%) 2903619924 (29.0%) 257696 (1.8%)
90 9429 (57.6%) 3262336544 (32.6%) 289416 (2.1%)
100 10459 (63.8%) 3621053164 (36.2%) 321136 (2.3%)
110 11489 (70.1%) 3979769784 (39.8%) 352856 (2.5%)
120 12519 (76.4%) 4338486404 (43.4%) 384576 (2.7%)
130 13549 (82.7%) 4697203024 (47.0%) 416296 (3.0%)
140 14579 (89.0%) 5055919644 (50.6%) 448016 (3.2%)
150 15609 (95.3%) 5414636264 (54.1%) 479736 (3.4%)
0 160 (1.0%) 75306572 (0.8%) 3936 (0.0%)
10 1189 (7.3%) 847209652 (8.5%) 35656 (0.3%)
20 2219 (13.5%) 1619112732 (16.2%) 67376 (0.5%)
30 3249 (19.8%) 2391015812 (23.9%) 99096 (0.7%)
40 4279 (26.1%) 3162918892 (31.6%) 130816 (0.9%)
50 5309 (32.4%) 3934821972 (39.3%) 162536 (1.2%)
60 6339 (38.7%) 4706725052 (47.1%) 194256 (1.4%)
70 7369 (45.0%) 5478628132 (54.8%) 225976 (1.6%)
80 8399 (51.3%) 6250531212 (62.5%) 257696 (1.8%)
90 9429 (57.6%) 7022434292 (70.2%) 289416 (2.1%)
100 10459 (63.8%) 7794337372 (77.9%) 321136 (2.3%)
110 11489 (70.1%) 8566240452 (85.7%) 352856 (2.5%)
120 12519 (76.4%) 9338143532 (93.4%) 384576 (2.7%)
130 13549 (82.7%) 10110046612 (101.1%) 416296 (3.0%)
140 14579 (89.0%) 10881949692 (108.8%) 448016 (3.2%)
150 15609 (95.3%) 11653852772 (116.5%) 479736 (3.4%)


Apply pairing to two pairs of points in G1 x G2 and run finalVerify on the results

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 342 (2.1%) 1292150444 (12.9%) 4654 (0.0%)
- 342 (2.1%) 1097919597 (11.0%) 4654 (0.0%)


Groth16 verification example

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 779 (4.8%) 2312779174 (23.1%) 10904 (0.1%)
- 779 (4.8%) 1996880342 (20.0%) 10904 (0.1%)

VRF example

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 712 (4.3%) 1303023563 (13.0%) 47949 (0.3%)
- 712 (4.3%) 1220773236 (12.2%) 47949 (0.3%)

G1 Verify

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 332 (2.0%) 1463720946 (14.6%) 5754 (0.0%)
- 332 (2.0%) 1190332196 (11.9%) 5754 (0.0%)

G2 Verify

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 380 (2.3%) 1342066427 (13.4%) 5754 (0.0%)
- 380 (2.3%) 1097700554 (11.0%) 5754 (0.0%)

Aggregate Single Key

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 777 (4.7%) 3336634422 (33.4%) 70002 (0.5%)
- 777 (4.7%) 2720886359 (27.2%) 70002 (0.5%)

Aggregate Multi Key

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 1704 (10.4%) 3674821887 (36.7%) 421586 (3.0%)
- 1704 (10.4%) 3446243236 (34.5%) 421586 (3.0%)

Schnorr Signature G1

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 370 (2.3%) 248136411 (2.5%) 13796 (0.1%)
- 370 (2.3%) 320182564 (3.2%) 13796 (0.1%)

Schnorr Signature G2

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 514 (3.1%) 493212089 (4.9%) 13964 (0.1%)
- 514 (3.1%) 552393171 (5.5%) 13964 (0.1%)

Groth16Verify succeeded
Simple Verify succeeded
Expand Down
8 changes: 4 additions & 4 deletions plutus-benchmark/common/PlutusBenchmark/Common.hs
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ getConfig limit = do
-- | Evaluate a script and return the CPU and memory costs (according to the cost model)
getCostsCek :: UPLC.Program UPLC.NamedDeBruijn DefaultUni DefaultFun () -> (Integer, Integer)
getCostsCek (UPLC.Program _ _ prog) =
case Cek.runCekDeBruijn PLC.defaultCekParameters Cek.tallying Cek.noEmitter prog of
case Cek.runCekDeBruijn PLC.defaultCekParametersForTesting Cek.tallying Cek.noEmitter prog of
(_res, Cek.TallyingSt _ budget, _logs) ->
let ExBudget (ExCPU cpu)(ExMemory mem) = budget
in (fromSatInt cpu, fromSatInt mem)
Expand All @@ -89,14 +89,14 @@ getCostsCek (UPLC.Program _ _ prog) =
-- deliberately not including it in the benchmarks.
mkEvalCtx :: LedgerApi.EvaluationContext
mkEvalCtx =
case PLC.defaultCostModelParams of
case PLC.defaultCostModelParamsForTesting of
Just p ->
let errOrCtx =
-- The validation benchmarks were all created from PlutusV1 scripts
LedgerApi.mkDynEvaluationContext
LedgerApi.PlutusV1
[DefaultFunSemanticsVariant1]
(const DefaultFunSemanticsVariant1)
[DefaultFunSemanticsVariantB]
(const DefaultFunSemanticsVariantB)
p
in case errOrCtx of
Right ec -> ec
Expand Down
32 changes: 16 additions & 16 deletions plutus-benchmark/ed25519-costs/test/9.6/ed25519-costs.golden
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 481 (2.9%) 5848112 (0.1%) 21522 (0.2%)
10 2208 (13.5%) 732286512 (7.3%) 516352 (3.7%)
20 3935 (24.0%) 1458724912 (14.6%) 1011182 (7.2%)
30 5662 (34.6%) 2185163312 (21.9%) 1506012 (10.8%)
40 7389 (45.1%) 2911601712 (29.1%) 2000842 (14.3%)
50 9115 (55.6%) 3638040112 (36.4%) 2495672 (17.8%)
60 10842 (66.2%) 4364478512 (43.6%) 2990502 (21.4%)
70 12569 (76.7%) 5090916912 (50.9%) 3485332 (24.9%)
80 14295 (87.2%) 5817355312 (58.2%) 3980162 (28.4%)
90 16022 (97.8%) 6543793712 (65.4%) 4474992 (32.0%)
100 17749 (108.3%) 7270232112 (72.7%) 4969822 (35.5%)
110 19476 (118.9%) 7996670512 (80.0%) 5464652 (39.0%)
120 21202 (129.4%) 8723108912 (87.2%) 5959482 (42.6%)
130 22929 (139.9%) 9449547312 (94.5%) 6454312 (46.1%)
140 24656 (150.5%) 10175985712 (101.8%) 6949142 (49.6%)
150 26383 (161.0%) 10902424112 (109.0%) 7443972 (53.2%)
0 481 (2.9%) 4458441 (0.0%) 21522 (0.2%)
10 2208 (13.5%) 646843741 (6.5%) 516352 (3.7%)
20 3935 (24.0%) 1289229041 (12.9%) 1011182 (7.2%)
30 5662 (34.6%) 1931614341 (19.3%) 1506012 (10.8%)
40 7389 (45.1%) 2573999641 (25.7%) 2000842 (14.3%)
50 9115 (55.6%) 3216384941 (32.2%) 2495672 (17.8%)
60 10842 (66.2%) 3858770241 (38.6%) 2990502 (21.4%)
70 12569 (76.7%) 4501155541 (45.0%) 3485332 (24.9%)
80 14295 (87.2%) 5143540841 (51.4%) 3980162 (28.4%)
90 16022 (97.8%) 5785926141 (57.9%) 4474992 (32.0%)
100 17749 (108.3%) 6428311441 (64.3%) 4969822 (35.5%)
110 19476 (118.9%) 7070696741 (70.7%) 5464652 (39.0%)
120 21202 (129.4%) 7713082041 (77.1%) 5959482 (42.6%)
130 22929 (139.9%) 8355467341 (83.6%) 6454312 (46.1%)
140 24656 (150.5%) 8997852641 (90.0%) 6949142 (49.6%)
150 26383 (161.0%) 9640237941 (96.4%) 7443972 (53.2%)

Off-chain version succeeded on 100 inputs
2 changes: 1 addition & 1 deletion plutus-benchmark/lists/exe/Main.hs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import UntypedPlutusCore.Evaluation.Machine.Cek qualified as Cek
getBudgetUsage :: Term -> Maybe Integer
getBudgetUsage term =
case (\ (fstT,sndT,_) -> (fstT,sndT) ) $
Cek.runCekDeBruijn PLC.defaultCekParameters Cek.counting Cek.noEmitter term
Cek.runCekDeBruijn PLC.defaultCekParametersForTesting Cek.counting Cek.noEmitter term
of
(Left _, _) -> Nothing
(Right _, Cek.CountingSt c) ->
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 278230764
({cpu: 196086904
| mem: 837952})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 23433064554
({cpu: 16341428194
| mem: 69498232})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 82807834
({cpu: 58837074
| mem: 253492})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 5975766004
({cpu: 4173000144
| mem: 17753632})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 184072620
({cpu: 113745780
| mem: 538480})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 14151996300
({cpu: 8662950900
| mem: 39539800})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 58810710
({cpu: 36645190
| mem: 178640})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 3652106700
({cpu: 2238212500
| mem: 10264400})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 167321654
({cpu: 123874594
| mem: 533932})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 361242389
({cpu: 272684232
| mem: 1083930})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 125205800
({cpu: 82856900
| mem: 454900})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 174221654
({cpu: 128674594
| mem: 563932})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 368142389
({cpu: 277484232
| mem: 1113930})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 132105800
({cpu: 87656900
| mem: 484900})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 247988561
({cpu: 185634074
| mem: 876490})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 373215336
({cpu: 274904136
| mem: 1373904})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 267781649
({cpu: 200204188
| mem: 956454})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 213732463
({cpu: 160863938
| mem: 747222})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 242342730
({cpu: 181505946
| mem: 850332})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 223274662
({cpu: 168188754
| mem: 778272})
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
({cpu: 221459085
({cpu: 167027055
| mem: 771670})
Loading

1 comment on commit 00beff4

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Plutus Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.05.

Benchmark suite Current: 00beff4 Previous: d7a5c76 Ratio
validation-decode-future-pay-out-1 247.6 μs 224.1 μs 1.10
validation-decode-future-settle-early-1 253.1 μs 223.8 μs 1.13
validation-decode-prism-3 243.4 μs 225.2 μs 1.08
validation-decode-stablecoin_2-2 167.5 μs 158.3 μs 1.06
validation-decode-uniswap-1 254.3 μs 225.9 μs 1.13
nofib-knights/6x6 45620 μs 43190 μs 1.06
nofib-knights/8x8 80610 μs 75370 μs 1.07

This comment was automatically generated by workflow using github-action-benchmark.

CC: @input-output-hk/plutus-core

Please sign in to comment.