[WIP]Lotus 1.23.3

[arajasek]

Lotus 1.23.3 Release

What will be in the release

🚢 Estimated shipping date

The estimated shipping dates for the Lotus v1.23.3 release are as follows:

2023-07-04: v1.23..3-rc1
2023-08-10: v1.23.3

🔦 Highlights

< See Changelog>

✅ Release Checklist

Note for whomever is owning the release: please capture notes as comments in this issue for anything you noticed that could be improved for future releases. There is a Post Release step below for incorporating changes back into the RELEASE_ISSUE_TEMPLATE, and this is easier done by collecting notes from along the way rather than just thinking about it at the end.

First steps:

• Fork a new branch (release/vX.Y.Z) from master and make any further release related changes to this branch. If any "non-trivial" changes get added to the release, uncheck all the checkboxes and return to this stage.
• Bump the version in build/version.go in the master branch to vX.Y.(Z+1)-dev (bump from feature release) or vX.(Y+1).0-dev (bump from mandatory release). Run make gen and make docsgen-cli before committing changes

Prepping an RC:

• version string in build/version.go has been updated (in the release/vX.Y.Z branch)
• run make gen && make docsgen-cli
• Generate changelog using the script at scripts/mkreleaselog
• Add contents of generated text to lotus/CHANGELOG.md in addition to other details
• tag commit with vX.Y.Z-rcN
• cut a pre-release here

Testing an RC:

• Stage 0 - Automated Testing

  • Automated Testing
    • CI: Ensure that all tests are passing.
    • Testground tests

• Stage 1 - Internal Testing

  • Binaries
    • Ensure the RC release has downloadable binaries
  • Upgrade our testnet infra
    • Wait 24 hours, confirm nodes stay in sync
  • Upgrade our mainnet infra
  • Subset of development full archival nodes
  • Subset of bootstrappers (1 per region)
  • Confirm nodes stay in sync
  • Metrics report
    • Block validation time
    • Memory / CPU usage
    • Number of goroutines
    • IPLD block read latency
    • Bandwidth usage
  • If anything has worsened significantly, investigate + fix
  • Confirm the following work (some combination of Testground / Calibnet / Mainnet / beta users)
    • Seal a sector
    • make a deal
    • Submit a PoSt
    • (optional) let a sector go faulty, and see it be recovered

• Stage 2 - Community Testing

  • Test with SPX fellows
  • Work on documentations for new features, configuration changes and so on.

• Stage 3 - Community Prod Testing

  • Update the CHANGELOG.md to the state that can be used as release note.
  • Invite the wider community through (link to the release issue)

• Stage 4 - Stable Release

  • Final preparation
    • Verify that version string in version.go has been updated.
    • Verify that codegen is up to date (make gen && make docsgen-cli)
    • Ensure that CHANGELOG.md is up to date
    • Merge release-vX.Y.Z into the releases branch.
    • Tag this merge commit (on the releases branch) with vX.Y.Z
    • Cut the release here.

• Post-Release

  • Merge the releases branch back into master, ignoring the changes to version.go (keep the -dev version from master). Do NOT delete the releases branch when doing so!
  • Update RELEASE_ISSUE_TEMPLATE.md with any improvements determined from this latest release iteration.
  • Create an issue using RELEASE_ISSUE_TEMPLATE.md for the next release.

❤️ Contributors

See the final release notes!

⁉️ Do you have questions?

Leave a comment in this ticket!

[arajasek]

Important testing cases for v1.23.3

• Something for Parallel Reads?
• Performance: Query ChainGetTipsetByHeight twice with an early height (say, 5), and compare performance between 1.23.2 and 1.23.3. The second query is expected to be faster in both cases thanks to the warmed up cache.
• Run the compact partitions CLI cmd
  •  https://filfox.info/en/message/bafy2bzacedru3cecw64na25fib3sql6krkqxjulwncef66euw4bh3ytav4hm6
• EthEvents: Pick 10k random epochs between 2.7M and 3M, query events from them on a 1.23.2 node, rerun against 1.23.3. Confirm correctness and performance.
• Run Lotus-Worker with only dataCID

General test cases that we might want to cover:

• Create BLS / secp256k1 / secp256k1-ledger/delegated:
• A send transaction:
• Pledge a CC:
• Terminate sector(s):
• Compact partition(s):
• Withdraw from actor:
• Renew sector(s):
• ChangeWorkerAddress:
• ChangeOwnerAddress:
• Add ControlAddresses:
• Propose and change beneficiary
• Create msig:
• Msig propose send transaction & approve:
• Msig propose transaction & cancel:
• Msig swap signers:
• Msig add a signer:
• Msig change propose threshold:
• Msig lock balance:
• Send a unverified storage deal:
• Assign a Notary:
• Notary allocate DataCap:
• Send a verified storage deal:
• SnapUp a sector:
• PreCommit Batch Sectors:
• CommitAggregate Sectors:
• Create a SnapShot:
• Run the disputer & send invalid windowPoSt:
• Generate and send consensus fault:
• Let a single sector fail in a windowPoSt
  • Successfully recover that sector in next windowPoSt
• Add balance to market-actor:
• Withdraw funds from the market-actor:
• Perform a paid retrieval:
• Settle payment-channel:
• Collect payment from payment-channel:
• Request multi address change:
• Change PeerID:
• Get slashed:
• Repay Debt:

[arajasek]

Some performance comparison numbers

1.23.3-rc2

aayush@aayush-desktop ~/p/lotus ((v1.23.3-rc2)) [1]> time ./lotus chain list --count=3 --height=200 && time ./lotus chain list --count=3 --height=200
198: (Aug 24 19:39:00) [ bafy2bzacear4yqnylpchlufj5nccvdv56zi7655jtich2chbk2w7oci4fv4b6: f01002, ]
199: (Aug 24 19:39:30) [ bafy2bzacecrlj6vujyqwm5daygo27ksaibr37wyklcby6ihb4kiejlfludzyi: f01002, ]
200: (Aug 24 19:40:00) [ bafy2bzacedgt32ubhpx4hu3hrbu663djcykuplt4jc5mnyii2wlzx5ovsj7cs: f01000,bafy2bzaceb6ksh75v3m6dydsutfrcaoe542d44rotznniq5poi2bfirthl6se: f01002, ]

________________________________________________________
Executed in  556.81 secs      fish           external
   usr time  116.03 millis  604.00 micros  115.42 millis
   sys time   25.24 millis    0.00 micros   25.24 millis

198: (Aug 24 19:39:00) [ bafy2bzacear4yqnylpchlufj5nccvdv56zi7655jtich2chbk2w7oci4fv4b6: f01002, ]
199: (Aug 24 19:39:30) [ bafy2bzacecrlj6vujyqwm5daygo27ksaibr37wyklcby6ihb4kiejlfludzyi: f01002, ]
200: (Aug 24 19:40:00) [ bafy2bzacedgt32ubhpx4hu3hrbu663djcykuplt4jc5mnyii2wlzx5ovsj7cs: f01000,bafy2bzaceb6ksh75v3m6dydsutfrcaoe542d44rotznniq5poi2bfirthl6se: f01002, ]

________________________________________________________
Executed in  142.98 millis    fish           external
   usr time   57.46 millis    0.00 micros   57.46 millis
   sys time   16.70 millis  317.00 micros   16.39 millis

1.23.2


aayush@aayush-desktop ~/p/lotus ((v1.23.3-rc2))> time ./lotus chain list --count=3 --height=200 && time ./lotus chain list --count=3 --height=200
198: (Aug 24 19:39:00) [ bafy2bzacear4yqnylpchlufj5nccvdv56zi7655jtich2chbk2w7oci4fv4b6: f01002, ]
199: (Aug 24 19:39:30) [ bafy2bzacecrlj6vujyqwm5daygo27ksaibr37wyklcby6ihb4kiejlfludzyi: f01002, ]
200: (Aug 24 19:40:00) [ bafy2bzacedgt32ubhpx4hu3hrbu663djcykuplt4jc5mnyii2wlzx5ovsj7cs: f01000,bafy2bzaceb6ksh75v3m6dydsutfrcaoe542d44rotznniq5poi2bfirthl6se: f01002, ]

________________________________________________________
Executed in  569.42 secs      fish           external
   usr time   94.66 millis    0.00 micros   94.66 millis
   sys time   45.43 millis  630.00 micros   44.80 millis

198: (Aug 24 19:39:00) [ bafy2bzacear4yqnylpchlufj5nccvdv56zi7655jtich2chbk2w7oci4fv4b6: f01002, ]
199: (Aug 24 19:39:30) [ bafy2bzacecrlj6vujyqwm5daygo27ksaibr37wyklcby6ihb4kiejlfludzyi: f01002, ]
200: (Aug 24 19:40:00) [ bafy2bzacedgt32ubhpx4hu3hrbu663djcykuplt4jc5mnyii2wlzx5ovsj7cs: f01000,bafy2bzaceb6ksh75v3m6dydsutfrcaoe542d44rotznniq5poi2bfirthl6se: f01002, ]

________________________________________________________
Executed in  174.93 millis    fish           external
   usr time   57.87 millis   58.00 micros   57.81 millis
   sys time   19.52 millis  299.00 micros   19.22 millis