Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Upgrade to tokio 0.3 #1839

Closed
wants to merge 54 commits into from
Closed

Conversation

AgeManning
Copy link
Member

@AgeManning AgeManning commented Oct 29, 2020

Description

This PR updates Lighthouse to tokio 0.3. It includes a number of dependency updates and some structural changes as to how we create and spawn tasks.

This also brings with it a number of various improvements:

  • Discv5 update
  • Libp2p update
  • Fix for recompilation issues
  • Improved UPnP port mapping handling
  • Futures dependency update
  • Log downgrade to traces for rejecting peers when we've reached our max

@AgeManning
Copy link
Member Author

This compiles but requires compatibility layers on any dependencies that require tokio 0.2.

We can add compatibility layers, but the two main deps we need upgraded are warp and rust-libp2p.

It seems rust-libp2p has a PR pending. I'm going to hold off until we fix a few issues in rust-libp2p and it gets updated to 0.3.

@paulhauner paulhauner added the A0 label Nov 9, 2020
@AgeManning
Copy link
Member Author

Without us updating the entire ecosystem, this is now out of our hands, so I'm going to set this to a1

@AgeManning AgeManning added A1 and removed A0 labels Nov 17, 2020
* update dependencies

* update warp dependency repository

* cargo fmt
@AgeManning
Copy link
Member Author

Thanks to Ben, we're back on for A0

@AgeManning AgeManning added A0 and removed A1 labels Nov 17, 2020
@paulhauner
Copy link
Member

paulhauner commented Nov 25, 2020

I unblocked this since we've already released v1.0.0.

@paulhauner
Copy link
Member

All our Pyrmont nodes are now running the 420 commit, the latest at the time of this comment.

@AgeManning
Copy link
Member Author

bors r+

bors bot pushed a commit that referenced this pull request Nov 27, 2020
## Description

This PR updates Lighthouse to tokio 0.3. It includes a number of dependency updates and some structural changes as to how we create and spawn tasks.

This also brings with it a number of various improvements:

- Discv5 update
- Libp2p update
- Fix for recompilation issues
- Improved UPnP port mapping handling
- Futures dependency update
- Log downgrade to traces for rejecting peers when we've reached our max



Co-authored-by: blacktemplar <blacktemplar@a1.net>
@AgeManning
Copy link
Member Author

bors r-

@bors
Copy link

bors bot commented Nov 27, 2020

Canceled.

@AgeManning
Copy link
Member Author

bors r+

bors bot pushed a commit that referenced this pull request Nov 28, 2020
## Description

This PR updates Lighthouse to tokio 0.3. It includes a number of dependency updates and some structural changes as to how we create and spawn tasks.

This also brings with it a number of various improvements:

- Discv5 update
- Libp2p update
- Fix for recompilation issues
- Improved UPnP port mapping handling
- Futures dependency update
- Log downgrade to traces for rejecting peers when we've reached our max



Co-authored-by: blacktemplar <blacktemplar@a1.net>
@bors bors bot changed the title Upgrade to tokio 0.3 [Merged by Bors] - Upgrade to tokio 0.3 Nov 28, 2020
@bors bors bot closed this Nov 28, 2020
bors bot pushed a commit that referenced this pull request Dec 8, 2020
## Issue Addressed

Closes #1264

## Proposed Changes

* Milagro BLS: tweak the feature flags so that Milagro doesn't get compiled if we're using BLST. Profiling showed that it was consuming about 1 minute of CPU time out of 60 minutes of CPU time (real time ~15 mins). A 1.6% saving.
* Reduce monomorphization: compiling for 3 different `EthSpec` types causes a heck of a lot of generic functions to be instantiated (monomorphized). Removing 2 of 3 cuts the LLVM+linking step from around 250 seconds to 180 seconds, a saving of 70 seconds (real time!). This applies only to `make` and not the CI build, because we test with the minimal spec on CI.
* Update `web3` crate to v0.13. This is perhaps the most controversial change, because it requires axing some deposit contract tools from `lcli`. I suspect these tools weren't used much anyway, and could be maintained separately, but I'm also happy to revert this change. However, it does save us a lot of compile time. With #1839, we now have 3 versions of Tokio (and all of Tokio's deps). This change brings us down to 2 versions, but 1 should be achievable once web3 (and reqwest) move to Tokio 0.3.
* Remove `lcli` from the Docker image. It's a dev tool and can be built from the repo if required.
bors bot pushed a commit that referenced this pull request Dec 8, 2020
## Issue Addressed

Closes #1264

## Proposed Changes

* Milagro BLS: tweak the feature flags so that Milagro doesn't get compiled if we're using BLST. Profiling showed that it was consuming about 1 minute of CPU time out of 60 minutes of CPU time (real time ~15 mins). A 1.6% saving.
* Reduce monomorphization: compiling for 3 different `EthSpec` types causes a heck of a lot of generic functions to be instantiated (monomorphized). Removing 2 of 3 cuts the LLVM+linking step from around 250 seconds to 180 seconds, a saving of 70 seconds (real time!). This applies only to `make` and not the CI build, because we test with the minimal spec on CI.
* Update `web3` crate to v0.13. This is perhaps the most controversial change, because it requires axing some deposit contract tools from `lcli`. I suspect these tools weren't used much anyway, and could be maintained separately, but I'm also happy to revert this change. However, it does save us a lot of compile time. With #1839, we now have 3 versions of Tokio (and all of Tokio's deps). This change brings us down to 2 versions, but 1 should be achievable once web3 (and reqwest) move to Tokio 0.3.
* Remove `lcli` from the Docker image. It's a dev tool and can be built from the repo if required.
bors bot pushed a commit that referenced this pull request Dec 9, 2020
## Issue Addressed

Closes #1264

## Proposed Changes

* Milagro BLS: tweak the feature flags so that Milagro doesn't get compiled if we're using BLST. Profiling showed that it was consuming about 1 minute of CPU time out of 60 minutes of CPU time (real time ~15 mins). A 1.6% saving.
* Reduce monomorphization: compiling for 3 different `EthSpec` types causes a heck of a lot of generic functions to be instantiated (monomorphized). Removing 2 of 3 cuts the LLVM+linking step from around 250 seconds to 180 seconds, a saving of 70 seconds (real time!). This applies only to `make` and not the CI build, because we test with the minimal spec on CI.
* Update `web3` crate to v0.13. This is perhaps the most controversial change, because it requires axing some deposit contract tools from `lcli`. I suspect these tools weren't used much anyway, and could be maintained separately, but I'm also happy to revert this change. However, it does save us a lot of compile time. With #1839, we now have 3 versions of Tokio (and all of Tokio's deps). This change brings us down to 2 versions, but 1 should be achievable once web3 (and reqwest) move to Tokio 0.3.
* Remove `lcli` from the Docker image. It's a dev tool and can be built from the repo if required.
@AgeManning AgeManning deleted the tokio-0.3 branch January 7, 2021 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A1 ready-for-review The code is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants