London Timing

[timbeiko]

Given that the difficulty bomb is expected to go off sometime in July, we will want London to hit mainnet then.

If we want a reasonable delay between picking blocks, going live on testnets, and going live on mainnet, this means we need to roughly follow the following timeline:

• July: Mainnet
• June: Testnets
• May: Blocks chosen, London testing done
• April: Select final EIPs + Reference Testing + Wrap up ephemeral network testing
• March: Select initial EIPs + Start ephemeral network testing

In other words, given we are already late-February, we should start thinking about what we want to see included in the London YOLO-nets (along with the difficulty bomb pushback (see EIP-3238)) if we want to not be rushed in implementation and deployment. This will be a bit of a change from our current process, as we'll need to start working on London before Berlin is live.

[q9f]

Timeline LGTM but who ensures we stick to it?

Also, we might want to do testnets as soon as possible rather than waiting till May to choose blocks.

[timbeiko]

@q9f

Timeline LGTM but who ensures we stick to it?

In this case, the difficulty bomb 😅 I do think agreeing to it now makes it easier to have hard deadlines (ex: no more new EIPs past X), which was an issue with Berlin, where we debated the pros/cons of various approaches for months.

Also, we might want to do testnets as soon as possible rather than waiting till May to choose blocks.

Good point, but that needs we need to have a final selection of EIPs and testing done even sooner.

[q9f]

no more new EIPs past X

that's what we did for Istanbul and apparently that worked out fairly well, a feature freeze at a given deadline and after that only implementation/testing of accepted/reviewed proposals. everything after that deadline simply goes into the next fork (Shanghai?)

[timbeiko]

Yep, that's what I'd go for @q9f. I think we need to select the "large" EIPs in March, have a "soft close" then, and at the latest add any "small" EIP in April, and everything else gets pushed to Shanghai (assuming they are still the highest priority when that fork comes).

[q9f]

commit to a feature freeze early (now) and communicate it right away

[jochem-brouwer]

I'd like to raise the point that if we go for this planning, we essentially end up having to plan two HFs in parallel (also Berlin #248). This is uncovered ground. This would also imply that (assuming that the scheduled Berlin HF time is accepted) we would have a gap of about 2.5 months between the two forks. This is very little time. In the past, we have seen that, due to circumstances, we usually end up delaying these HFs. I'd therefore say that this planning is very optimistic.

I'm a bit afraid that in practice, this is going to happen:

(1) We schedule Berlin around the time when the difficulty bomb kicks in (I also consider 2.5 months before it "around" this time).
(2) Due to parallel HF planning, we end up that London gets delayed a lot (assuming London has additional EIPs besides the difficulty bomb delay which is likely to be in this HF). Due to the difficulty bomb kicking in, we end up that London would be a MuirGlacier-like fork (only delay the bomb), which would be unfortunate.

[q9f]

In that case we should have the difficulty bomb delay in Berlin and relax the schedule for London a bit.

[timbeiko]

Proposed another schedule for Berlin which would give us a bit more slack, see #248 (comment)

[CryptoBlockchainTechnologies]

I'd like to raise the point that if we go for this planning, we essentially end up having to plan two HFs in parallel (also Berlin #248). This is uncovered ground. This would also imply that (assuming that the scheduled Berlin HF time is accepted) we would have a gap of about 2.5 months between the two forks. This is very little time. In the past, we have seen that, due to circumstances, we usually end up delaying these HFs. I'd therefore say that this planning is very optimistic.

I'm a bit afraid that in practice, this is going to happen:

(1) We schedule Berlin around the time when the difficulty bomb kicks in (I also consider 2.5 months before it "around" this time).
(2) Due to parallel HF planning, we end up that London gets delayed a lot (assuming London has additional EIPs besides the difficulty bomb delay which is likely to be in this HF). Due to the difficulty bomb kicking in, we end up that London would be a MuirGlacier-like fork (only delay the bomb), which would be unfortunate.

I agree to focus on Berlin and combine it with difficulty bomb reset

[timbeiko]

@CryptoBlockchainTechnologies it was decided on the call that Berlin's EIP list is final and the difficulty bomb will be pushed back this summer.

[holgerd77]

As with Berlin, I think it would be valuable to keep this issue open until past-London.

[q9f]

Given we discussed the scope of London being centered around 1559 today, I would propose the following dates:

• Writing this comment today: Mar/5 2021
• (buffer 6 weeks/3 calls)
• Last call: Fri Apr/16 2021
  • last chance to propose EIPs for London
  • announce feature freeze for London in 4 weeks
• (buffer 4 weeks/2 calls)
• Feature freeze: Fri May/14 2021 (+4 weeks)
  • last chance to discuss inclusion of proposals in London
  • setting final block numbers
  • announcing testnets in 3-5 weeks, mainnet in 12 weeks
• (buffer 2.71 weeks/1.36 calls)
• Ropsten testnet: Wed Jun/2 2021
• Goerli testnet: Wed Jun/9 2021
• Rinkeby testnet: Wed Jun/16 2021
• (buffer 6 weeks/3 calls)
• Ethereum mainnet: Wed Jul/28 2021

And we could consider Shanghai as early as November in case any other proposals beyond 1559 are considered for inclusion.

[timbeiko]

@q9f just in case the difficulty bomb shows up earlier than expected, or that we end up being slower than The Plan ™️, I think we should plan all of this ~2 weeks earlier, and instead do something like:

• London EIPs: Mar 19
• Last Call: Apr 2
• (buffer 4 weeks/2 calls)
• Feature Freeze: Apr 30
• Ropsten: May 12
• Goerli: May 19
• Rinkeby: May 26
• (buffer 6 weeks/3 calls)
• Mainnet: July 14

WDYT?

[q9f]

WDYT?

We should actually run the numbers before speculating. I can look into that next week.

[timbeiko]

(Apologies in advance for the long comment!)

London timing was brought up extensively again in ACD 110 today.

There are a few factors to consider here, namely:

1. The difficulty bomb going off;
2. The fact that there may not be another feature fork post-London;
3. The desire to keep London slim to get 1559 deployed ASAP and ship London before (1) happens;
4. The community support we've seen for some proposals (e.g. 3074) and the time it would take to implement/audit them.

@vbuterin helped roughly estimate when the difficulty bomb would go off, using the following code:

>>> current_blknum = 12252201
>>> blocks_per_month = (86400 * 30) // 13
>>> future_blknum = current_blknum + blocks_per_month * 4
>>> diff_adjustment = 2 ** ((future_blknum - 9000000) // 100000 - 2)
>>> current_difficulty = 6467102703670090 
>>> diff_adjust_coeff = diff_adjustment / current_difficulty * 2048
>>> diff_adjust_coeff
0.0870482469842078

This means that 4 months from today (120 days), on August 14th, we are likely going to see a ~9% increase in difficulty due to the bomb. At 3 months (90 days, July 15th), that value is ~2%, and for 5 months (150 days, Sept 13), it is ~34%. This implies that the latest we realistically want to fork mainnet would be around mid-August: beyond that, the bomb's effects worsen rapidly.

Assuming mid-week forks, this is what timelines could look like for a July vs. August London:

Date	July London	August London
Aug 11		Mainnet Fork
July 21		Rinkeby Fork
July 14	Mainnet Fork	Gorli Fork
July 7		Ropsten Fork
June 23	Rinkeby Fork	Clients Released
June 23	Gorli Fork
June 16	Ropsten Fork	Client Freeze
June 2	Clients Released
May 19	Clients Freeze

It seems to me that if we want to ship London in July, then we need to agree to a final set of EIPs in the next ~2 weeks. On the other hand, if we want more time to decide (and get the results of the pending 3074 audit, for example) because we believe that this is the last change to include potentially valuable changes for a long time due to the merge, then we can aim for London to happen mid-August and give ourselves a bit more breathing room.

[timbeiko]

We agreed on ACD111 to the July London schedule, but did not set fork blocks. Here is a table people can copy to propose fork blocks:

Date	London Event	Block Number
July 14	Mainnet Fork	TBA
June 23	Rinkeby Fork	TBA
June 23	Gorli Fork	TBA
June 16	Ropsten Fork	TBA
June 2	Clients Released
May 19	Clients Freeze

[timbeiko]

Note, the last comment's table had two forks on June 23. To avoid this, we can move back the first fork to June 9th, which seemed to be the preferred option on discord.

Using this, here are tentative blocks we could use for London:

Date	London Event	Block Number
July 14	Mainnet Fork	12833000
June 23	Rinkeby Fork	8813188
June 16	Goerli Fork	4979794
June 9	Ropsten Fork	10399301
June 2	Clients Released
May 19	Clients Freeze

Calculations for the block numbers can be found here. Note, the time will be outdated, but should still work when updating values.

[timbeiko]

On ACD 113, we agreed to the general schedule above, but due to some concern about testing time, only wanted to commit to the testnet fork blocks, which have been added to the London spec.

When the first testnet fork happens successfully, we will confirm the mainnet block.

[timbeiko]

On ACD 114, given that we identified an issue in EIP-1559, we decided to launch an additional devnet, Calaveras, to test the fix. Assuming everything goes smoothly, we will set fork blocks in ACD115.

[timbeiko]

On ACD115 we agreed to the following testnet blocks:

Date	London Event	Block Number
July 7	Rinkeby Fork	8897988
June 30	Goerli Fork	5062605
June 24	Ropsten Fork	10499401

[timbeiko]

Difficulty bomb update, courtesy of @tjayrush of TrueBlocks:

TL;DR: we can expect the difficulty bomb to become noticeable in ~300k blocks, or in ~45 days at around block 12900000. Blocks 1 (July 28) & 2 (August 4) proposed for mainnet would fall in this period, while blocks 3 & 4 would happen after the next difficulty bomb increase.

---

From @tjayrush:

You can see from the first epoch that the green line bends down when the block start slowing down. I'm not see any lowering, but I expect it will start to bend soon.

This chart shows that the bomb is basically flat until it starts to explode and then it explode more and more quickly. You can just barely see it starting to appear at the very right hand side. You can judge it compared to the other explosions. Each step is a 100,000 block period.

This one shows the various bombs laid on top of each other. The vertical line is where the previous bombs started appearing. You can see that as the height of the bubbles goes lower (slower blocks). Different bombs gets slower depending on hash rate, but it's not noticeable until it gets further into the bomb.

[timbeiko]

We agreed on ACD 116 to wait until Goerli forks to pick a block. We will do so async before the next ACD and ideally have client versions ready for the next call.

[tjayrush]

while blocks 3 & 4 would happen after the next difficulty bomb increase.

Even after that, the effect won't be that large. You can see this in the first chart above. It usually takes a few periods after crossing the horizontal dashed line before its effect becomes observable.

The other two charts show the same thing. You can summarize the way it behaves like this: "Blocks take 14 seconds and are perfectly predictable until they're not" and "You have enough time once the bomb starts becoming apparent to respond."

One of the things that I'm trying to dispel with these charts is the thought that predicting when the bomb will happen is hard to predict. It's not. It's quite easy.

It's more difficult to predict how quickly it will get bad once it starts appearing, but it's easy to see it coming.

[timbeiko]

The mainnet block has been chosen: 12965000. PR here: ethereum/execution-specs#223

Closing this issue now 👋