Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix optional type ID ambiguity and migrate types #3272

Merged
merged 14 commits into from
Apr 23, 2024

Conversation

dsainati1
Copy link
Contributor

Closes #3248


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

Copy link

github-actions bot commented Apr 19, 2024

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit 052db5e
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
DecodeBatchEventsCCF-4128ms ± 0%128ms ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-4394ms ± 0%418ms ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-42.80µs ± 0%2.82µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.TokensWithdrawn-42.25µs ± 0%2.12µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-42.83µs ± 0%2.80µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-43.09µs ± 0%3.09µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-42.20µs ± 0%2.15µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.RewardsPaid-42.50µs ± 0%2.48µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensDeposited-42.58µs ± 0%2.54µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-42.48µs ± 0%2.44µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensMinted-42.12µs ± 0%2.10µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensWithdrawn-42.59µs ± 0%2.57µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowFees.FeesDeducted-49.87µs ± 0%9.96µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowFees.TokensWithdrawn-45.75µs ± 0%5.75µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-48.89µs ± 0%8.96µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-412.2µs ± 0%12.2µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-45.84µs ± 0%5.96µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.RewardsPaid-47.54µs ± 0%7.61µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensDeposited-48.02µs ± 0%7.91µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-47.10µs ± 0%7.14µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensMinted-45.85µs ± 0%5.88µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensWithdrawn-47.82µs ± 0%7.91µs ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsCCF-490.2ms ± 0%89.3ms ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-4105ms ± 0%106ms ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-41.99µs ± 0%2.00µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.TokensWithdrawn-41.57µs ± 0%1.57µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-41.95µs ± 0%1.94µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-42.24µs ± 0%2.37µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-41.61µs ± 0%1.58µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.RewardsPaid-41.75µs ± 0%1.75µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensDeposited-41.84µs ± 0%1.81µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-41.81µs ± 0%1.81µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensMinted-41.56µs ± 0%1.57µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensWithdrawn-41.84µs ± 0%1.80µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowFees.FeesDeducted-42.51µs ± 0%2.48µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowFees.TokensWithdrawn-41.42µs ± 0%1.44µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-42.22µs ± 0%2.23µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-43.11µs ± 0%3.09µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-41.44µs ± 0%1.45µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.RewardsPaid-41.89µs ± 0%1.90µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensDeposited-42.18µs ± 0%2.17µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-41.70µs ± 0%1.72µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensMinted-41.46µs ± 0%1.44µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensWithdrawn-42.15µs ± 0%2.14µs ± 0%~(p=1.000 n=1+1)
ExportType/composite_type-4283ns ± 0%287ns ± 0%~(p=1.000 n=1+1)
ExportType/simple_type-473.7ns ± 0%71.3ns ± 0%~(p=1.000 n=1+1)
InterpretRecursionFib-42.23ms ± 0%2.23ms ± 0%~(p=1.000 n=1+1)
NewInterpreter/new_interpreter-4910ns ± 0%934ns ± 0%~(p=1.000 n=1+1)
NewInterpreter/new_sub-interpreter-4461ns ± 0%464ns ± 0%~(p=1.000 n=1+1)
ParseArray-46.11ms ± 1%6.00ms ± 0%~(p=0.333 n=2+2)
ParseDeploy/byte_array-48.90ms ± 1%8.96ms ± 1%~(p=0.667 n=2+2)
ParseDeploy/decode_hex-41.00ms ± 0%1.00ms ± 0%~(p=0.333 n=2+2)
ParseFungibleToken/With_memory_metering-4147µs ± 4%153µs ± 4%~(p=0.667 n=2+2)
ParseFungibleToken/Without_memory_metering-4116µs ± 4%121µs ± 6%~(p=0.667 n=2+2)
ParseInfix-45.20µs ± 0%5.23µs ± 0%~(p=0.333 n=2+2)
QualifiedIdentifierCreation/One_level-42.17ns ± 0%2.16ns ± 0%~(p=1.000 n=1+1)
QualifiedIdentifierCreation/Three_levels-483.5ns ± 0%87.8ns ± 0%~(p=1.000 n=1+1)
RuntimeScriptNoop-43.68µs ± 0%3.70µs ± 0%~(p=1.000 n=1+1)
SuperTypeInference/arrays-4244ns ± 0%237ns ± 0%~(p=1.000 n=1+1)
SuperTypeInference/composites-496.5ns ± 0%98.1ns ± 0%~(p=1.000 n=1+1)
SuperTypeInference/integers-4309ns ± 0%311ns ± 0%~(p=1.000 n=1+1)
ValueIsSubtypeOfSemaType-474.9ns ± 0%71.0ns ± 0%~(p=1.000 n=1+1)
 
alloc/opdelta
DecodeBatchEventsCCF-466.5MB ± 0%66.5MB ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-4244MB ± 0%244MB ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-41.40kB ± 0%1.40kB ± 0%~(all equal)
DecodeCCF/FlowFees.TokensWithdrawn-41.21kB ± 0%1.21kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-41.48kB ± 0%1.48kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-41.49kB ± 0%1.49kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-41.26kB ± 0%1.26kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.RewardsPaid-41.38kB ± 0%1.38kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited-41.32kB ± 0%1.32kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-41.30kB ± 0%1.30kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensMinted-41.21kB ± 0%1.21kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensWithdrawn-41.33kB ± 0%1.33kB ± 0%~(all equal)
DecodeJSON/FlowFees.FeesDeducted-46.00kB ± 0%6.00kB ± 0%~(all equal)
DecodeJSON/FlowFees.TokensWithdrawn-43.60kB ± 0%3.60kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-45.43kB ± 0%5.43kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-47.35kB ± 0%7.35kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-43.64kB ± 0%3.64kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.RewardsPaid-44.54kB ± 0%4.54kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited-44.88kB ± 0%4.88kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-44.46kB ± 0%4.46kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensMinted-43.60kB ± 0%3.60kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensWithdrawn-44.88kB ± 0%4.88kB ± 0%~(all equal)
EncodeBatchEventsCCF-462.4MB ± 0%62.4MB ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-439.1MB ± 0%39.1MB ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-41.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeCCF/FlowFees.TokensWithdrawn-41.17kB ± 0%1.17kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-41.44kB ± 0%1.44kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-41.41kB ± 0%1.41kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-41.34kB ± 0%1.34kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-41.42kB ± 0%1.42kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited-41.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-41.20kB ± 0%1.20kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensMinted-41.17kB ± 0%1.17kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensWithdrawn-41.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeJSON/FlowFees.FeesDeducted-4864B ± 0%864B ± 0%~(all equal)
EncodeJSON/FlowFees.TokensWithdrawn-4504B ± 0%504B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-4888B ± 0%888B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-41.08kB ± 0%1.08kB ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-4552B ± 0%552B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.RewardsPaid-4752B ± 0%752B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited-4776B ± 0%776B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-4640B ± 0%640B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensMinted-4512B ± 0%512B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensWithdrawn-4768B ± 0%768B ± 0%~(all equal)
ExportType/composite_type-4120B ± 0%120B ± 0%~(all equal)
ExportType/simple_type-40.00B 0.00B ~(all equal)
InterpretRecursionFib-41.19MB ± 0%1.19MB ± 0%~(p=1.000 n=1+1)
NewInterpreter/new_interpreter-4944B ± 0%944B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-4200B ± 0%200B ± 0%~(all equal)
ParseArray-42.65MB ± 0%2.73MB ± 1%~(p=0.333 n=2+2)
ParseDeploy/byte_array-44.15MB ± 2%4.22MB ± 0%~(p=0.333 n=2+2)
ParseDeploy/decode_hex-4214kB ± 0%214kB ± 0%~(p=1.000 n=2+2)
ParseFungibleToken/With_memory_metering-429.6kB ± 0%29.6kB ± 0%~(p=0.667 n=2+2)
ParseFungibleToken/Without_memory_metering-429.6kB ± 0%29.6kB ± 0%~(p=1.000 n=2+2)
ParseInfix-41.92kB ± 0%1.92kB ± 0%~(p=0.667 n=2+2)
QualifiedIdentifierCreation/One_level-40.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-464.0B ± 0%64.0B ± 0%~(all equal)
RuntimeScriptNoop-43.37kB ± 0%3.37kB ± 0%~(all equal)
SuperTypeInference/arrays-496.0B ± 0%96.0B ± 0%~(all equal)
SuperTypeInference/composites-40.00B 0.00B ~(all equal)
SuperTypeInference/integers-40.00B 0.00B ~(all equal)
ValueIsSubtypeOfSemaType-448.0B ± 0%48.0B ± 0%~(all equal)
 
allocs/opdelta
DecodeBatchEventsCCF-41.48M ± 0%1.48M ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-44.70M ± 0%4.70M ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-430.0 ± 0%30.0 ± 0%~(all equal)
DecodeCCF/FlowFees.TokensWithdrawn-426.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-430.0 ± 0%30.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-432.0 ± 0%32.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-426.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.RewardsPaid-429.0 ± 0%29.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited-431.0 ± 0%31.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-429.0 ± 0%29.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensMinted-426.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensWithdrawn-431.0 ± 0%31.0 ± 0%~(all equal)
DecodeJSON/FlowFees.FeesDeducted-4126 ± 0%126 ± 0%~(all equal)
DecodeJSON/FlowFees.TokensWithdrawn-471.0 ± 0%71.0 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-4102 ± 0%102 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-4159 ± 0%159 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-470.0 ± 0%70.0 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.RewardsPaid-487.0 ± 0%87.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited-495.0 ± 0%95.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-486.0 ± 0%86.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensMinted-471.0 ± 0%71.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensWithdrawn-495.0 ± 0%95.0 ± 0%~(all equal)
EncodeBatchEventsCCF-4950k ± 0%950k ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-4853k ± 0%853k ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowFees.TokensWithdrawn-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited-420.0 ± 0%20.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-420.0 ± 0%20.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensMinted-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensWithdrawn-420.0 ± 0%20.0 ± 0%~(all equal)
EncodeJSON/FlowFees.FeesDeducted-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeJSON/FlowFees.TokensWithdrawn-412.0 ± 0%12.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-416.0 ± 0%16.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-425.0 ± 0%25.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-412.0 ± 0%12.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.RewardsPaid-415.0 ± 0%15.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited-419.0 ± 0%19.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-414.0 ± 0%14.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensMinted-413.0 ± 0%13.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensWithdrawn-418.0 ± 0%18.0 ± 0%~(all equal)
ExportType/composite_type-43.00 ± 0%3.00 ± 0%~(all equal)
ExportType/simple_type-40.00 0.00 ~(all equal)
InterpretRecursionFib-418.9k ± 0%18.9k ± 0%~(all equal)
NewInterpreter/new_interpreter-415.0 ± 0%15.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-44.00 ± 0%4.00 ± 0%~(all equal)
ParseArray-459.6k ± 0%59.6k ± 0%~(p=1.000 n=2+2)
ParseDeploy/byte_array-489.4k ± 0%89.4k ± 0%~(p=1.000 n=2+2)
ParseDeploy/decode_hex-463.0 ± 0%63.0 ± 0%~(all equal)
ParseFungibleToken/With_memory_metering-4773 ± 1%773 ± 1%~(p=1.000 n=2+2)
ParseFungibleToken/Without_memory_metering-4773 ± 1%773 ± 1%~(p=1.000 n=2+2)
ParseInfix-448.0 ± 0%48.0 ± 0%~(all equal)
QualifiedIdentifierCreation/One_level-40.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-42.00 ± 0%2.00 ± 0%~(all equal)
RuntimeScriptNoop-451.0 ± 0%51.0 ± 0%~(all equal)
SuperTypeInference/arrays-43.00 ± 0%3.00 ± 0%~(all equal)
SuperTypeInference/composites-40.00 0.00 ~(all equal)
SuperTypeInference/integers-40.00 0.00 ~(all equal)
ValueIsSubtypeOfSemaType-41.00 ± 0%1.00 ± 0%~(all equal)
 

Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

migrations/legacy_optional_type.go Outdated Show resolved Hide resolved
runtime/sema/type.go Show resolved Hide resolved
Copy link
Member

@SupunS SupunS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please also add a "Rehash test", similar to:

// TestIntersectionTypeRehash stores a dictionary in storage,
// which has a key that is a type value with a restricted type that has two interface types,
// runs the migration, and ensures the dictionary is still usable
func TestIntersectionTypeRehash(t *testing.T) {
.

The idea is to test, if there is an optional-typed TypeValue is being used as a key in a dictionary, then retrieving (using the old hash/typeID) and updating that key during the migration should work.

@dsainati1
Copy link
Contributor Author

Can you please also add a "Rehash test", similar to

I think TestRehash in entitlements/migration_test which was updated by this PR serves to test this? If not let me know and I'll add another test.

@dsainati1 dsainati1 requested review from turbolent and SupunS April 22, 2024 14:28
@SupunS
Copy link
Member

SupunS commented Apr 22, 2024

hmm, yes it does (didn't realize it earlier). However, that test is for testing entitlements rehashing, and the optional-type seems more like a coincident/side-effect? It would be good to have a separate test for optionals, so that any change to the entitlement test won't accidentally remove the use of optional type.

@dsainati1
Copy link
Contributor Author

So while I was writing these tests, I realized that I am not actually sure the migration itself is strictly necessary. We had to migrated restricted/intersection and reference static types because how they are encoded and decoded from storage changed, but with optional types even though the TypeID itself is changing how it's printed, the encoding of these values is not changing. So any optional static types that were created and encoded with the old IDs are just going to be loaded from storage the same way, and there is no migration that actually needs to be performed on the data. I realized this while trying to write the test @SupunS was talking about above because I couldn't actually get the migration to run on the test data, as when the data was loaded from storage to be migrated, it was already "migrated" so to speak, in that it was already converted to the new format by virtue of having been decoded.

Am I misunderstanding something about how this works, or do we just not need to do anything here?

@turbolent
Copy link
Member

@dsainati1 Do you need help here with the rehash test case? I can finish this up

@dsainati1
Copy link
Contributor Author

If you have time that would be super helpful! Otherwise I can get to it after the security stuff

@turbolent
Copy link
Member

  • Simplified the legacy optional type implementation in fe757c3
  • Forced re-storing/encoding of optional types, even if the inner type has not changed in 80ede1e
  • Add a rehashing test in e94d7e0. Showed that forcing the re-storage/encoding was still missing

Should be ready now. @SupunS could you PTAL?

Copy link
Member

@SupunS SupunS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@turbolent turbolent requested a review from SupunS April 23, 2024 19:08
Copy link

codecov bot commented Apr 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.81%. Comparing base (a664a1d) to head (5504380).
Report is 16 commits behind head on master.

❗ Current head 5504380 differs from pull request most recent head 37e3c26. Consider uploading reports for the commit 37e3c26 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3272      +/-   ##
==========================================
+ Coverage   80.80%   80.81%   +0.01%     
==========================================
  Files         380      381       +1     
  Lines       93643    93724      +81     
==========================================
+ Hits        75666    75743      +77     
- Misses      15276    15279       +3     
- Partials     2701     2702       +1     
Flag Coverage Δ
unittests 80.81% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@turbolent
Copy link
Member

@dsainati1 @SupunS Could you PTAL?

@turbolent turbolent merged commit 5131b0a into master Apr 23, 2024
10 of 12 checks passed
@turbolent turbolent added the Storage Breaking Change Breaks existing stored data (needs a storage migration) label Apr 23, 2024
@turbolent turbolent deleted the sainati/optional-type-id branch April 23, 2024 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bugfix Storage Breaking Change Breaks existing stored data (needs a storage migration)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optional Reference Types and References to Optional Types are represented with the same TypeID
3 participants