Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize string migration: only return new value if needed #3173

Merged
merged 1 commit into from
Mar 13, 2024

Conversation

turbolent
Copy link
Member

Work towards #3096

Description

Great suggestion from @bluesign: Currently, the string migration always returns a new string/character value, even if the value was already normalized.

Only return a new value if the old value was not normalized.
Only compute the normalized form once, and use it both for comparison and actual use in the value.

Given the majority of the strings are likely do not need any normalization, this should reduce a lot of unnecessary overhead.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

Copy link

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit aa14837
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
DecodeBatchEventsCCF-4128ms ± 0%128ms ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-4403ms ± 0%395ms ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-42.76µs ± 0%2.78µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.TokensWithdrawn-42.10µs ± 0%2.10µs ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-42.83µs ± 0%2.86µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-43.07µs ± 0%3.08µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-42.12µs ± 0%2.12µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.RewardsPaid-42.53µs ± 0%2.53µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensDeposited-42.55µs ± 0%2.58µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-42.46µs ± 0%2.46µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensMinted-42.09µs ± 0%2.09µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensWithdrawn-42.57µs ± 0%2.57µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowFees.FeesDeducted-49.87µs ± 0%9.78µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowFees.TokensWithdrawn-45.69µs ± 0%5.68µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-48.89µs ± 0%9.28µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-412.2µs ± 0%12.2µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-45.82µs ± 0%5.78µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.RewardsPaid-47.49µs ± 0%7.52µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensDeposited-47.80µs ± 0%7.84µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-47.08µs ± 0%7.18µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensMinted-45.71µs ± 0%5.72µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensWithdrawn-47.80µs ± 0%7.99µs ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsCCF-490.4ms ± 0%89.7ms ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-4105ms ± 0%105ms ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-41.98µs ± 0%1.96µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.TokensWithdrawn-41.55µs ± 0%1.55µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-41.92µs ± 0%1.90µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-42.25µs ± 0%2.25µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-41.60µs ± 0%1.60µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.RewardsPaid-41.73µs ± 0%1.73µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensDeposited-41.84µs ± 0%1.83µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-41.81µs ± 0%1.80µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensMinted-41.56µs ± 0%1.56µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensWithdrawn-41.84µs ± 0%1.83µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowFees.FeesDeducted-42.47µs ± 0%2.50µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowFees.TokensWithdrawn-41.41µs ± 0%1.45µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-42.43µs ± 0%2.23µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-43.11µs ± 0%3.10µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-41.45µs ± 0%1.46µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.RewardsPaid-41.89µs ± 0%1.88µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensDeposited-42.18µs ± 0%2.21µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-41.74µs ± 0%1.69µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensMinted-41.44µs ± 0%1.43µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensWithdrawn-42.15µs ± 0%2.15µs ± 0%~(p=1.000 n=1+1)
ExportType/composite_type-4280ns ± 0%280ns ± 0%~(p=1.000 n=1+1)
ExportType/simple_type-470.3ns ± 0%70.4ns ± 0%~(p=1.000 n=1+1)
InterpretRecursionFib-42.16ms ± 0%2.14ms ± 0%~(p=1.000 n=1+1)
NewInterpreter/new_interpreter-4906ns ± 0%903ns ± 0%~(p=1.000 n=1+1)
NewInterpreter/new_sub-interpreter-4451ns ± 0%452ns ± 0%~(p=1.000 n=1+1)
ParseArray-45.98ms ± 2%5.91ms ± 2%~(p=0.667 n=2+2)
ParseDeploy/byte_array-48.89ms ± 1%8.79ms ± 1%~(p=0.667 n=2+2)
ParseDeploy/decode_hex-41.00ms ± 2%0.98ms ± 0%~(p=0.333 n=2+2)
ParseFungibleToken/With_memory_metering-4154µs ± 5%147µs ± 4%~(p=0.667 n=2+2)
ParseFungibleToken/Without_memory_metering-4114µs ± 3%117µs ± 1%~(p=0.667 n=2+2)
ParseInfix-45.17µs ± 0%5.19µs ± 0%~(p=0.667 n=2+2)
QualifiedIdentifierCreation/One_level-42.18ns ± 0%2.17ns ± 0%~(p=1.000 n=1+1)
QualifiedIdentifierCreation/Three_levels-484.5ns ± 0%84.5ns ± 0%~(p=1.000 n=1+1)
RuntimeScriptNoop-43.63µs ± 0%3.65µs ± 0%~(p=1.000 n=1+1)
SuperTypeInference/arrays-4237ns ± 0%236ns ± 0%~(p=1.000 n=1+1)
SuperTypeInference/composites-498.1ns ± 0%100.4ns ± 0%~(p=1.000 n=1+1)
SuperTypeInference/integers-4310ns ± 0%310ns ± 0%~(p=1.000 n=1+1)
ValueIsSubtypeOfSemaType-469.9ns ± 0%69.3ns ± 0%~(p=1.000 n=1+1)
 
alloc/opdelta
DecodeBatchEventsCCF-466.5MB ± 0%66.5MB ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-4244MB ± 0%244MB ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-41.40kB ± 0%1.40kB ± 0%~(all equal)
DecodeCCF/FlowFees.TokensWithdrawn-41.21kB ± 0%1.21kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-41.48kB ± 0%1.48kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-41.49kB ± 0%1.49kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-41.26kB ± 0%1.26kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.RewardsPaid-41.38kB ± 0%1.38kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited-41.32kB ± 0%1.32kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-41.30kB ± 0%1.30kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensMinted-41.21kB ± 0%1.21kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensWithdrawn-41.33kB ± 0%1.33kB ± 0%~(all equal)
DecodeJSON/FlowFees.FeesDeducted-46.00kB ± 0%6.00kB ± 0%~(all equal)
DecodeJSON/FlowFees.TokensWithdrawn-43.60kB ± 0%3.60kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-45.43kB ± 0%5.43kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-47.35kB ± 0%7.35kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-43.64kB ± 0%3.64kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.RewardsPaid-44.54kB ± 0%4.54kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited-44.88kB ± 0%4.88kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-44.46kB ± 0%4.46kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensMinted-43.60kB ± 0%3.60kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensWithdrawn-44.88kB ± 0%4.88kB ± 0%~(all equal)
EncodeBatchEventsCCF-462.4MB ± 0%62.4MB ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-439.1MB ± 0%39.1MB ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-41.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeCCF/FlowFees.TokensWithdrawn-41.17kB ± 0%1.17kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-41.44kB ± 0%1.44kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-41.41kB ± 0%1.41kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-41.34kB ± 0%1.34kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-41.42kB ± 0%1.42kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited-41.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-41.20kB ± 0%1.20kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensMinted-41.17kB ± 0%1.17kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensWithdrawn-41.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeJSON/FlowFees.FeesDeducted-4864B ± 0%864B ± 0%~(all equal)
EncodeJSON/FlowFees.TokensWithdrawn-4504B ± 0%504B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-4888B ± 0%888B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-41.08kB ± 0%1.08kB ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-4552B ± 0%552B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.RewardsPaid-4752B ± 0%752B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited-4776B ± 0%776B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-4640B ± 0%640B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensMinted-4512B ± 0%512B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensWithdrawn-4768B ± 0%768B ± 0%~(all equal)
ExportType/composite_type-4120B ± 0%120B ± 0%~(all equal)
ExportType/simple_type-40.00B 0.00B ~(all equal)
InterpretRecursionFib-41.00MB ± 0%1.00MB ± 0%~(all equal)
NewInterpreter/new_interpreter-4944B ± 0%944B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-4200B ± 0%200B ± 0%~(all equal)
ParseArray-42.70MB ± 0%2.71MB ± 2%~(p=1.000 n=2+2)
ParseDeploy/byte_array-44.22MB ± 0%4.22MB ± 0%~(p=0.667 n=2+2)
ParseDeploy/decode_hex-4214kB ± 0%214kB ± 0%~(p=0.333 n=2+2)
ParseFungibleToken/With_memory_metering-429.6kB ± 0%29.6kB ± 0%~(p=1.000 n=2+2)
ParseFungibleToken/Without_memory_metering-429.6kB ± 0%29.6kB ± 0%~(p=1.000 n=2+2)
ParseInfix-41.92kB ± 0%1.92kB ± 0%~(p=1.000 n=2+2)
QualifiedIdentifierCreation/One_level-40.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-464.0B ± 0%64.0B ± 0%~(all equal)
RuntimeScriptNoop-43.21kB ± 0%3.21kB ± 0%~(all equal)
SuperTypeInference/arrays-496.0B ± 0%96.0B ± 0%~(all equal)
SuperTypeInference/composites-40.00B 0.00B ~(all equal)
SuperTypeInference/integers-40.00B 0.00B ~(all equal)
ValueIsSubtypeOfSemaType-448.0B ± 0%48.0B ± 0%~(all equal)
 
allocs/opdelta
DecodeBatchEventsCCF-41.48M ± 0%1.48M ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-44.70M ± 0%4.70M ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-430.0 ± 0%30.0 ± 0%~(all equal)
DecodeCCF/FlowFees.TokensWithdrawn-426.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-430.0 ± 0%30.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-432.0 ± 0%32.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-426.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.RewardsPaid-429.0 ± 0%29.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited-431.0 ± 0%31.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-429.0 ± 0%29.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensMinted-426.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensWithdrawn-431.0 ± 0%31.0 ± 0%~(all equal)
DecodeJSON/FlowFees.FeesDeducted-4126 ± 0%126 ± 0%~(all equal)
DecodeJSON/FlowFees.TokensWithdrawn-471.0 ± 0%71.0 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-4102 ± 0%102 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-4159 ± 0%159 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-470.0 ± 0%70.0 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.RewardsPaid-487.0 ± 0%87.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited-495.0 ± 0%95.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-486.0 ± 0%86.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensMinted-471.0 ± 0%71.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensWithdrawn-495.0 ± 0%95.0 ± 0%~(all equal)
EncodeBatchEventsCCF-4950k ± 0%950k ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-4853k ± 0%853k ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowFees.TokensWithdrawn-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited-420.0 ± 0%20.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-420.0 ± 0%20.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensMinted-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensWithdrawn-420.0 ± 0%20.0 ± 0%~(all equal)
EncodeJSON/FlowFees.FeesDeducted-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeJSON/FlowFees.TokensWithdrawn-412.0 ± 0%12.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-416.0 ± 0%16.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-425.0 ± 0%25.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-412.0 ± 0%12.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.RewardsPaid-415.0 ± 0%15.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-414.0 ± 0%14.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensMinted-413.0 ± 0%13.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensWithdrawn-418.0 ± 0%18.0 ± 0%~(all equal)
ExportType/composite_type-43.00 ± 0%3.00 ± 0%~(all equal)
ExportType/simple_type-40.00 0.00 ~(all equal)
InterpretRecursionFib-418.9k ± 0%18.9k ± 0%~(all equal)
NewInterpreter/new_interpreter-415.0 ± 0%15.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-44.00 ± 0%4.00 ± 0%~(all equal)
ParseArray-459.6k ± 0%59.6k ± 0%~(p=0.667 n=2+2)
ParseDeploy/byte_array-489.4k ± 0%89.4k ± 0%~(p=1.000 n=2+2)
ParseDeploy/decode_hex-463.0 ± 0%63.0 ± 0%~(all equal)
ParseFungibleToken/With_memory_metering-4773 ± 1%773 ± 1%~(p=1.000 n=2+2)
ParseFungibleToken/Without_memory_metering-4773 ± 1%773 ± 1%~(p=1.000 n=2+2)
ParseInfix-448.0 ± 0%48.0 ± 0%~(all equal)
QualifiedIdentifierCreation/One_level-40.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-42.00 ± 0%2.00 ± 0%~(all equal)
RuntimeScriptNoop-451.0 ± 0%51.0 ± 0%~(all equal)
SuperTypeInference/arrays-43.00 ± 0%3.00 ± 0%~(all equal)
SuperTypeInference/composites-40.00 0.00 ~(all equal)
SuperTypeInference/integers-40.00 0.00 ~(all equal)
ValueIsSubtypeOfSemaType-41.00 ± 0%1.00 ± 0%~(all equal)
 

@turbolent turbolent merged commit e3776ae into master Mar 13, 2024
11 of 12 checks passed
@turbolent turbolent deleted the bastian/optimize-string-migration branch March 13, 2024 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants