3.5 release next steps: Code freeze Monday 5/17 #12970

hexfusion · 2021-05-14T14:15:02Z

etcd 3.5 has seen a tremendous amount of new features bug fixes and performance and stability improvements. At this time I would like to thank everyone for the hard work and dedication. As per the community meeting on May 6th we have outlined Monday, May 17th as code freeze for the 3.5 release. Below are the proposed series of events for input.

Steps:

cut release-3.5 branch from master (now main). All future merges to main will be 3.6 moving forward (Monday AM 5/17).
cut new v3.5.0-beta.3
cut new modularized release v3.5.0-beta.3 (Monday AM 5/17).
begin correctness and performance/stress testing (ideas, ownership?)

After testing and validation of the beta, we can consider a sequential beta release or move towards a v3.5 release candidate.

Your input is greatly appreciated.

cc @xiang90 @wenjiaswe @ptabor @gyuho @jingyih @lilic @jpbetz @spzala

The text was updated successfully, but these errors were encountered:

wenjiaswe · 2021-05-14T20:48:31Z

Thank you very much @hexfusion !

begin correctness and performance/stress testing (ideas, ownership?)

Would you mind putting together a "gift registry" so we could plan and start pick up?

gyuho · 2021-05-14T22:21:28Z

cut release-3.5 branch from master (now main). All future merges to main will be 3.6 moving forward (Monday AM 5/17).

As we discussed, @hexfusion @lilic will take ownership on the release management, right?

begin correctness and performance/stress testing (ideas, ownership?)

We have some in-house component that does this, but I believe our continuous functional tests are good enough.

Please chime in if you have better ideas.

gyuho · 2021-05-14T22:22:19Z

@hexfusion Please feel free to email and ping other maintainers via etcd-dev mailing list.

ptabor · 2021-05-15T15:05:56Z

Try to stabilize (official versions) our dependencies:
- Release bbolt 1.3.6 (bbolt 1.3.6 release plan bbolt#278)
- Zap (1.17 release uber-go/zap#942)
Verify: Migrate cluster attributes to use v3 backend #11427 (comment)
When beta is out we can ask k8s to incorporate update etcd, grpc, protobuf dependencies kubernetes/kubernetes#100488 (@liggitt ).
This will allow us to get CI feedback from k8s.
When branch --release-3.5 is tagged, we will run our in-house tests
Functional tests are a little flaky (not more than release-3.4), but if that's the main signal, someone should dig in
Blog-post was started by @gyuho (https://docs.google.com/document/d/1F2p2LVPo4gj3n8XFMd4QrlarbCdJrbd-pNrT3XLYXmM/edit?usp=sharing)
Investigate the deadlock in example: integration/clientv3/examples test flakes frequently #12983

hexfusion · 2021-05-15T17:03:29Z

As we discussed, @hexfusion @lilic will take ownership on the release management, right?

correct

lilic · 2021-05-17T08:21:46Z

Functional tests are a little flaky (not more than release-3.4), but if that's the main signal, someone should dig in

If no one is looking into this, I can have a look end of this week. @ptabor do you have a list of these? Otherwise I can have a look at previous CI runs.

serathius · 2021-05-17T08:44:34Z

I would be happy to help

hexfusion · 2021-05-17T10:25:52Z

Red Hat team was preparing to cut the release-3.5 branch at approximately 8:00am EDT. If we have a need to hold this please reach out.

cc @ptabor @gyuho @lilic

ptabor · 2021-05-17T11:05:43Z

@hexfusion: I think we can tag the branch.

I suspect Migrate cluster attributes to use v3 backend #11427 (comment) will require a patch, but its OK.
I still have some hope to merge Split etcdctl into etcdctl (public API access) & etcdutl (direct surgery on files) #12971 into. But this requires your approval.

ptabor · 2021-05-17T11:08:20Z

Functional tests are a little flaky (not more than release-3.4), but if that's the main signal, someone should dig in

If no one is looking into this, I can have a look end of this week. @ptabor do you have a list of these? Otherwise I can have a look at previous CI runs.

I've seen them on CI, but unfortunately I didn't collected the links and its not easy to search by 'jobs' currently. Running in a forloop locally mightbe an option as well.

hexfusion · 2021-05-17T12:08:29Z

I still have some hope to merge #12971 into. But this requires your approval.

complete

hexfusion · 2021-05-17T12:18:13Z

@ptabor thoughts on remaining nice to have[1]?

[1] https://github.com/etcd-io/etcd/pulls?q=is%3Apr+is%3Aopen+label%3Arel%2F3.5-nice-to-have

server: set multiple concurrentReadTx instances share one txReadBuffer area/performance rel/3.5-nice-to-have server: set multiple concurrentReadTx instances share one txReadBuffer #12933
backend: fix buffer range bug area/bug lgtm rel/3.5-nice-to-have backend: fix buffer range bug #12587
backend: set seq flag for each bucket buffer area/performance rel/3.5-nice-to-have backend: set seq flag for each bucket buffer #12568
server: make raft read only option configurable rel/3.5-nice-to-have server: make raft read only option configurable #12486
etcdserver: add ability to auto-promote learners to voters rel/3.5-nice-to-have etcdserver: add ability to auto-promote learners to voters #10887

lilic · 2021-05-17T12:20:56Z

There is also #12872. But I am not sure we need to do this with code freeze and it can be done afterwards, correct?

ptabor · 2021-05-17T12:28:10Z

I think we are good.

#12933 server: set multiple concurrentReadTx instances share one txReadBuffer area/performance

seems be very impactful, and I wish it could made eventually into 3.5.
as it touches very critical logic so we are very demanding with testing coverage.

We might make a call to include this one to beta/beta.2 later.

Small tweaks without public behavior change. We used to incorporate such changes in patch releases.

#12587 backend: fix buffer range bug area/bug lgtm
#12568 backend: set seq flag for each bucket buffer area/performance
#12486 server: make raft read only option configurable

Features: -> probably for 3.6

#10887 etcdserver: add ability to auto-promote learners to voters rel/3.5-nice-to-have
- We didn't help the contributor with reviews. Probably too late to start it now. Afraid 3.6.

hexfusion · 2021-05-17T12:28:55Z

@ptabor We are proposing to delay the branch cut until 1:00 EDT to allow the West coast teams final input and a last review of the above. WDYT?

ptabor · 2021-05-17T12:35:06Z

Cutting a brunch alone is not a 'point of no return'. We can decide to backport some PRs from main to release-3.5 after it happens - just the toil/bar will be a little higher.

Do you assume cutting a brunch is going atomically with releasing/tagging "etcd-3.5.0-beta.0" or this will be a follow up e.g. tomorrow ?

hexfusion · 2021-05-17T12:41:37Z

My initial thoughts were to create a separation of the beta code by branching. The downside with this approach is that we will need to backport changes. So if we feel there is still a reasonable amount of work to do (appears this way) we could consider holding the branch until we are ready for a release candidate? Thus create a beta release from main this morning.

hexfusion · 2021-05-17T12:43:34Z

The downside of the above is a more manual vetting of merges so it's a trade-off I suppose.

ptabor · 2021-05-17T13:06:56Z

I think we can branch 'release-3.5' today. On our end it will unblock onsite automation that understands 3.5 to be coming soon.

If we make a call to backport some changes from 'main' to 'release-3.5' and we consider it too toilish using github PR per change, we can do it in a single PR.

serathius · 2021-05-17T13:14:42Z

When beta is out we can ask k8s to incorporate kubernetes/kubernetes#100488 (Jordan Liggitt ).
This will allow us to get CI feedback from k8s.

I can help with that

lilic · 2021-05-17T15:23:52Z

There is also this PR #12979 that seems ready for review and can be merged if we want it.

hexfusion · 2021-05-17T15:35:05Z

@mitake could you possibly take a quick peek at #12979?

hexfusion · 2021-05-17T17:05:04Z

Hi team it is 1:00 EDT as promised I am about to create a new branch for release-3.5 do we have anything blocking this action? I will take any blocking input until quarter past (1:15 PM EDT) at which point the branch will be pushed.

cc @xiang90 @wenjiaswe @ptabor @gyuho @jingyih @lilic @jpbetz @spzala

hexfusion · 2021-05-17T17:19:54Z

branch is pushed we will begin in the process of new beta release.

serathius · 2021-06-01T15:27:30Z

Both fixes have been backported to 3.5 branch, we should be ready to cut another beta release.

lilic · 2021-06-03T06:33:49Z

Should we replace the alpha to latest beta in all the go.mod files? Example:

etcd/go.mod

Lines 23 to 32 in 932d42b

    
           go.etcd.io/etcd/api/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/client/pkg/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/client/v2 v2.305.0-alpha.0 
        
           go.etcd.io/etcd/client/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/etcdctl/v3 v3.0.0-00010101000000-000000000000 
        
           go.etcd.io/etcd/etcdutl/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/pkg/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/raft/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/server/v3 v3.5.0-alpha.0 
        
           go.etcd.io/etcd/tests/v3 v3.5.0-alpha.0

hexfusion · 2021-06-03T10:32:57Z

Both fixes have been backported to 3.5 branch, we should be ready to cut another beta release.

Catching up from holidays will cut new release today.

gyuho · 2021-06-03T11:47:44Z

@hexfusion Can we also update bbolt to v1.3.6?

hexfusion · 2021-06-03T12:28:25Z

yes plans are to bump deps and roll rc.0 will get to that tonight.

ptabor · 2021-06-03T13:39:34Z

@lilic @hexfusion

Should we replace the alpha to latest beta in all the go.mod files? Example:

Can we also update bbolt to v1.3.6?

Both changes and also zap to 1.17.0 waiting in:

hexfusion · 2021-06-03T16:20:31Z

both are now merged, thanks for the assist. now pending rc.0

hexfusion · 2021-06-04T02:09:50Z

etcd v3.5.0-rc.0 is now available for testing.

liggitt · 2021-06-04T02:41:43Z

kubernetes/kubernetes#100488 updated

gyuho · 2021-06-08T19:27:56Z

@hexfusion Any outstanding item to close out? The target date is June 15, 2021.

lilic · 2021-06-09T14:06:00Z

I believe we need a new release candidate that includes some changes since the 3.5.0-rc.0. Before that we also need this #13049 PR to land in release-3.5 branch.

serathius · 2021-06-09T15:01:43Z

kubernetes/kubernetes#102062 updated and passing all the tests.

hexfusion · 2021-06-09T18:16:50Z

@hexfusion Any outstanding item to close out? The target date is June 15, 2021.

#13095 is up once merged I can cut rc.1 tonight. Then pending any issues from k8s move towards a formal release.

hexfusion · 2021-06-09T22:42:30Z

etcd v3.5.0-rc.1 is now available for testing.

cc @liggitt @serathius

serathius · 2021-06-11T10:11:20Z

Kubernetes has been using v3.5.0-rc.0 in their CI for last 24h without any issues reported.

gyuho · 2021-06-14T13:59:42Z

@hexfusion Are we good with tomorrow release (June 15, 2021)?

@nate-double-u @chalin Can we coordinate with blog posting?

hexfusion · 2021-06-14T14:03:10Z

@gyuho yes, nothing blocking at this point shooting for 6PM EDT.

gyuho · 2021-06-14T15:05:24Z

@chalin @nate-double-u Can we make blog post public, right after

6PM EDT

?

nate-double-u · 2021-06-14T16:27:33Z

We can do that -- to be clear: blog release June 15, 6PM EDT

Also, I have an open PR for renaming the next folder to v3.5: etcd-io/website#363

nate-double-u · 2021-06-14T16:52:11Z

(also, please note that @chalin is out this week)

mrueg · 2021-06-15T08:54:42Z

I found these TODOs in the default branch, that might be worth to be cleaned up before release:

./server/mvcc/metrics_txn.go:   rangeCounterDebug.Add(ranges) // TODO: remove in 3.5 release
./tests/e2e/v3_curl_test.go:// TODO: remove /v3beta tests in 3.5 release
./tests/e2e/v3_curl_lease_test.go:// TODO remove /kv/lease/timetolive, /kv/lease/revoke, /kv/lease/leases tests in 3.5 release

nate-double-u · 2021-06-15T17:22:16Z

Website v3.5 release prep PR: etcd-io/website#379

nate-double-u · 2021-06-15T22:07:23Z

Standing by. I can merge the announcement PR (etcd-io/website#379) when you approve it @hexfusion, or when I see v3.5.0 on the Releases page.

hexfusion · 2021-06-15T22:31:40Z

Congratulations to everyone for an amazing team effort. etcd team proudly announces GA for v3.5.0

nate-double-u · 2021-06-15T22:35:20Z

Here's the blog announcement: https://etcd.io/blog/2021/announcing-etcd-3.5/
Congratulations everyone!

hexfusion · 2021-06-29T02:00:26Z

/close

hexfusion added the area/community label May 14, 2021

lilic mentioned this issue May 14, 2021

What's the plan for version 3.5? When will it be released? #12330

Closed

ptabor pinned this issue May 15, 2021

ptabor mentioned this issue May 15, 2021

1.17 release uber-go/zap#942

Closed

This was referenced May 17, 2021

[WIP] Testing etcd 3.5.0 client + server kubernetes/kubernetes#102061

Closed

Update Etcd server image to 3.5.0-rc.0-0 kubernetes/kubernetes#102062

Merged

spzala mentioned this issue Jun 14, 2021

Solution for CMD parameters #6088 #13105

Closed

nate-double-u mentioned this issue Jun 14, 2021

Publish v3.5 release announcement etcd-io/website#376

Closed

gyuho unpinned this issue Jun 21, 2021

gyuho closed this as completed Jun 29, 2021

3.5 release next steps: Code freeze Monday 5/17 #12970

3.5 release next steps: Code freeze Monday 5/17 #12970

Comments

hexfusion commented May 14, 2021 • edited Loading

wenjiaswe commented May 14, 2021

gyuho commented May 14, 2021

gyuho commented May 14, 2021

ptabor commented May 15, 2021 • edited Loading

hexfusion commented May 15, 2021 • edited by ptabor Loading

lilic commented May 17, 2021

serathius commented May 17, 2021

hexfusion commented May 17, 2021

ptabor commented May 17, 2021

ptabor commented May 17, 2021

hexfusion commented May 17, 2021 • edited Loading

hexfusion commented May 17, 2021 • edited by ptabor Loading

lilic commented May 17, 2021

ptabor commented May 17, 2021 • edited Loading

hexfusion commented May 17, 2021

ptabor commented May 17, 2021

hexfusion commented May 17, 2021

hexfusion commented May 17, 2021

ptabor commented May 17, 2021

serathius commented May 17, 2021

lilic commented May 17, 2021

hexfusion commented May 17, 2021

hexfusion commented May 17, 2021 • edited Loading

hexfusion commented May 17, 2021

serathius commented Jun 1, 2021

lilic commented Jun 3, 2021

hexfusion commented Jun 3, 2021

gyuho commented Jun 3, 2021

hexfusion commented Jun 3, 2021

ptabor commented Jun 3, 2021

hexfusion commented Jun 3, 2021

hexfusion commented Jun 4, 2021 • edited Loading

liggitt commented Jun 4, 2021

gyuho commented Jun 8, 2021

lilic commented Jun 9, 2021

serathius commented Jun 9, 2021 • edited Loading

hexfusion commented Jun 9, 2021 • edited Loading

hexfusion commented Jun 9, 2021

serathius commented Jun 11, 2021

gyuho commented Jun 14, 2021

hexfusion commented Jun 14, 2021 • edited Loading

gyuho commented Jun 14, 2021

nate-double-u commented Jun 14, 2021

nate-double-u commented Jun 14, 2021

mrueg commented Jun 15, 2021

nate-double-u commented Jun 15, 2021 • edited Loading

nate-double-u commented Jun 15, 2021 • edited Loading

hexfusion commented Jun 15, 2021

nate-double-u commented Jun 15, 2021

hexfusion commented Jun 29, 2021

hexfusion commented May 14, 2021 •

edited

Loading

ptabor commented May 15, 2021 •

edited

Loading

hexfusion commented May 15, 2021 •

edited by ptabor

Loading

hexfusion commented May 17, 2021 •

edited

Loading

hexfusion commented May 17, 2021 •

edited by ptabor

Loading

ptabor commented May 17, 2021 •

edited

Loading

hexfusion commented May 17, 2021 •

edited

Loading

hexfusion commented Jun 4, 2021 •

edited

Loading

serathius commented Jun 9, 2021 •

edited

Loading

hexfusion commented Jun 9, 2021 •

edited

Loading

hexfusion commented Jun 14, 2021 •

edited

Loading

nate-double-u commented Jun 15, 2021 •

edited

Loading

nate-double-u commented Jun 15, 2021 •

edited

Loading