Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: mempool: lower priority optimizations #10693

Merged
merged 7 commits into from
May 3, 2023

Conversation

snissn
Copy link
Contributor

@snissn snissn commented Apr 19, 2023

Related Issues

#10597

Proposed Changes

  1. https://github.com/filecoin-project/lotus/pull/10561/files#r1148322934 in chain/messagepool/check.go pre-allocate the size of the array msgs array using msgs = make([]*types.Message, 0, len(mset.msgs))
  2. https://github.com/filecoin-project/lotus/pull/10561/files#r1152024072 remove the requirement of mp.keyCache.Add(ka, ka) by checking that the address is a robust address
  3. https://github.com/filecoin-project/lotus/pull/10561/files#r1150875872 chain/messagepool/repub.go - optimize the Lock timing and confirm if all tests pass with a read lock -- also ensure that mp.curTsLk.RLock() is not held during mp.api.ChainComputeBaseFee
  4. https://github.com/filecoin-project/lotus/pull/10561/files#r1150137317 chain/messagepool/selection.go change to read lock
  5. in chain/messagepool/check.go pending has a giant warning saying do NOT use it directly -- we should probably be calling getPendingMset here for address resolution?

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

@snissn snissn marked this pull request as ready for review April 19, 2023 22:00
@snissn snissn requested a review from a team as a code owner April 19, 2023 22:00
@snissn snissn requested a review from arajasek April 26, 2023 15:37
Copy link
Contributor

@arajasek arajasek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do a second pass tomorrow just to squint at the locks, but LGTM.

return mp.resolveToKeyFromID(ctx, addr)
}

func (mp *MessagePool) resolveToKeyFromID(ctx context.Context, addr address.Address) (address.Address, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need a new method here, just add the "is not ID" check to resolveToKey and leave it as before?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer the way I have it. I think it's easier to read! Definitely happy to defer to your suggestion if you have a preference

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong preference!

mset, ok, err := mp.getPendingMset(ctx, from)
if err != nil {
log.Warnf("errored while getting pending mset: %w", err)
return nil, err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please chain these errors (return nil, xerrors.Errorf("....%w", err)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(it makes debugging a lot easier when things go wrong)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YES, that code is a lot cleaner! Appreciate the tip!!

return mp.resolveToKeyFromID(ctx, addr)
}

func (mp *MessagePool) resolveToKeyFromID(ctx context.Context, addr address.Address) (address.Address, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong preference!

return mp.resolveToKeyFromID(ctx, addr)
}

func (mp *MessagePool) resolveToKeyFromID(ctx context.Context, addr address.Address) (address.Address, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien In what cases can an actor have both an f4 and an f2 address? Is that just EVM actors (not including EthAccounts)?

I wanna make sure you can never have an actor have both an f4 and an f2 address AND serving as a top-level sender, because that would make things funky here.

(Not really related to changes in this PR)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just EVM actors, at least for now. Account abstraction may someday change this, but that's a whole can of worms.

Copy link
Member

@Stebalien Stebalien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks correct. I'm slightly concerned about the dance with mp.republished (we're assuming that nothing changes while we drop the lock), but that's existing behavior.

@shrenujbansal shrenujbansal merged commit 742062f into master May 3, 2023
@shrenujbansal shrenujbansal deleted the mikers/messagePoolLowerPriorityOptimizations branch May 3, 2023 20:31
@simlecode simlecode mentioned this pull request May 23, 2023
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants