Check we're on the right branch before tagging, and on the right tag before uploading #12556

DMRobertson · 2022-04-26T14:39:13Z

As part of deploying to matrix.org, I switched to the matrix-org-hotfixes branch. I then ran release.py tag and didn't notice until it was too late that I had tagged v1.58rc1 as the merge commit which merges release-1.58 onto matrix-org-hotfixes. I then naughtily retagged and force pushed which caused a bit of pain elsewhere. (Cancelling CI jobs, RSS feed consumers getting the tag early...)

Let's try to avoid this by forcing the releaser to acknowledge the commit that's about to be tagged.

erikjohnston · 2022-04-26T15:33:26Z

scripts-dev/release.py

- if tag_name in repo.tags:
- raise click.ClickException(f"Tag {tag_name} already exists!\n")
+ # if tag_name in repo.tags:
+ #  raise click.ClickException(f"Tag {tag_name} already exists!\n")


Was this intentional, or just part of you testing?.

Oops. Just testing.

erikjohnston · 2022-04-26T15:35:48Z

scripts-dev/release.py

+ f"by {commit.author} <{commit.author.email}>, "
+ f"committed at {commit.committed_datetime}"
+ )
+ click.confirm(f"Tag this commit as {tag_name}?", default=False, abort=True)


I'd be tempted to fail if we're not on the right branch, maybe? We already check the right branch in another command:

https://github.com/matrix-org/synapse/blob/706456de1f63a4b9796592fd452da7ce30735274/scripts-dev/release.py

Do you have a line number for me, sorry? I think that link is missing one

Bah, sorry, it got stripped, its:

synapse/scripts-dev/release.py

Lines 171 to 174 in 706456d

release_branch_name = (

f"release-v{parsed_new_version.major}.{parsed_new_version.minor}"

)

release_branch = find_ref(repo, release_branch_name)

erikjohnston

Woot! Are there any other commands where it would make sense to do the same check?

DMRobertson · 2022-04-26T17:19:18Z

Woot! Are there any other commands where it would make sense to do the same check?

publish checks the repo isn't dirty, but doesn't actually change anything in the repo or interact with the filesystem as far as I can see.

Might be worth adding a check to upload though.

DMRobertson · 2022-04-26T17:41:56Z

Might be worth adding a check to upload though.

What do you make of the last three commits?

more changes soz

DMRobertson · 2022-04-27T13:33:00Z

sytest failure appears to be matrix-org/sytest#1044

erikjohnston · 2022-05-03T07:57:31Z

scripts-dev/release.py

+ tag = repo.tag(f"refs/tags/{tag_name}")
+ if repo.head.commit != tag.commit:
+ click.echo("Tag {tag_name} (tag.commit) is not currently checked out!")
+ click.get_current_context().abort()


I don't think this is actually necessary for upload as we don't interact with the repo at all, we just download and reupload files.

Though might make sense to keep the check purely for sanity reasons.

Oh, I didn't realise we downloaded the wheel and sdist; I assumed we'd build them locally.

I guess I'll leave it in. (The line echoes an instruction to do a git merge, so it can't hurt)

DMRobertson · 2022-05-03T10:22:35Z

Going to retarget this at develop and resolve merge fun.

@SimonBrandner

Synapse 1.59.0rc1 (2022-05-10) ============================== This release makes several changes that server administrators should be aware of: - Device name lookup over federation is now disabled by default. ([\#12616](#12616)) - The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](#12452), [\#12654](#12654)) See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590) for more details. Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](#12597)) Features -------- - Support [MSC3266](matrix-org/matrix-spec-proposals#3266) room summaries over federation. ([\#11507](#11507)) - Implement [changes](matrix-org/matrix-spec-proposals@4a77139) to [MSC2285 (hidden read receipts)](matrix-org/matrix-spec-proposals#2285). Contributed by @SimonBrandner. ([\#12168](#12168), [\#12635](#12635), [\#12636](#12636), [\#12670](#12670)) - Extend the [module API](https://github.com/matrix-org/synapse/blob/release-v1.59/synapse/module_api/__init__.py) to allow modules to change actions for existing push rules of local users. ([\#12406](#12406)) - Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ([\#12452](#12452)) - Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. ([\#12654](#12654)) - Add new `enable_registration_token_3pid_bypass` configuration option to allow registrations via token as an alternative to verifying a 3pid. ([\#12526](#12526)) - Implement [MSC3786](matrix-org/matrix-spec-proposals#3786): Add a default push rule to ignore `m.room.server_acl` events. ([\#12601](#12601)) - Add new `mau_appservice_trial_days` configuration option to specify a different trial period for users registered via an appservice. ([\#12619](#12619)) Bugfixes -------- - Fix a bug introduced in Synapse 1.48.0 where the latest thread reply provided failed to include the proper bundled aggregations. ([\#12273](#12273)) - Fix a bug introduced in Synapse 1.22.0 where attempting to send a large amount of read receipts to an application service all at once would result in duplicate content and abnormally high memory usage. Contributed by Brad & Nick @ Beeper. ([\#12544](#12544)) - Fix a bug introduced in Synapse 1.57.0 which could cause `Failed to calculate hosts in room` errors to be logged for outbound federation. ([\#12570](#12570)) - Fix a long-standing bug where status codes would almost always get logged as `200!`, irrespective of the actual status code, when clients disconnect before a request has finished processing. ([\#12580](#12580)) - Fix race when persisting an event and deleting a room that could lead to outbound federation breaking. ([\#12594](#12594)) - Fix a bug introduced in Synapse 1.53.0 where bundled aggregations for annotations/edits were incorrectly calculated. ([\#12633](#12633)) - Fix a long-standing bug where rooms containing power levels with string values could not be upgraded. ([\#12657](#12657)) - Prevent memory leak from reoccurring when presence is disabled. ([\#12656](#12656)) Updates to the Docker image --------------------------- - Explicitly opt-in to using [BuildKit-specific features](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md) in the Dockerfile. This fixes issues with building images in some GitLab CI environments. ([\#12541](#12541)) - Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Contributed by @henryclw. ([\#12573](#12573)) Improved Documentation ---------------------- - Update SQL statements and replace use of old table `user_stats_historical` in docs for Synapse Admins. ([\#12536](#12536)) - Add missing linebreak to `pipx` install instructions. ([\#12579](#12579)) - Add information about the TCP replication module to docs. ([\#12621](#12621)) - Fixes to the formatting of `README.rst`. ([\#12627](#12627)) - Fix docs on how to run specific Complement tests using the `complement.sh` test runner. ([\#12664](#12664)) Deprecations and Removals ------------------------- - Remove unstable identifiers from [MSC3069](matrix-org/matrix-spec-proposals#3069). ([\#12596](#12596)) - Remove the unspecified `m.login.jwt` login type and the unstable `uk.half-shot.msc2778.login.application_service` from [MSC2778](matrix-org/matrix-spec-proposals#2778). ([\#12597](#12597)) - Synapse now requires at least Python 3.7.1 (up from 3.7.0), for compatibility with the latest Twisted trunk. ([\#12613](#12613)) Internal Changes ---------------- - Use supervisord to supervise Postgres and Caddy in the Complement image to reduce restart time. ([\#12480](#12480)) - Immediately retry any requests that have backed off when a server comes back online. ([\#12500](#12500)) - Use `make_awaitable` instead of `defer.succeed` for return values of mocks in tests. ([\#12505](#12505)) - Consistently check if an object is a `frozendict`. ([\#12564](#12564)) - Protect module callbacks with read semantics against cancellation. ([\#12568](#12568)) - Improve comments and error messages around access tokens. ([\#12577](#12577)) - Improve docstrings for the receipts store. ([\#12581](#12581)) - Use constants for read-receipts in tests. ([\#12582](#12582)) - Log status code of cancelled requests as 499 and avoid logging stack traces for them. ([\#12587](#12587), [\#12663](#12663)) - Remove special-case for `twisted` logger from default log config. ([\#12589](#12589)) - Use `getClientAddress` instead of the deprecated `getClientIP`. ([\#12599](#12599)) - Add link to documentation in Grafana Dashboard. ([\#12602](#12602)) - Reduce log spam when running multiple event persisters. ([\#12610](#12610)) - Add extra debug logging to federation sender. ([\#12614](#12614)) - Prevent remote homeservers from requesting local user device names by default. ([\#12616](#12616)) - Add a consistency check on events which we read from the database. ([\#12620](#12620)) - Remove use of the `constantly` library and switch to enums for `EventRedactBehaviour`. Contributed by @andrewdoh. ([\#12624](#12624)) - Remove unused code related to receipts. ([\#12632](#12632)) - Minor improvements to the scripts for running Synapse in worker mode under Complement. ([\#12637](#12637)) - Move `pympler` back in to the `all` extras. ([\#12652](#12652)) - Fix spelling of `M_UNRECOGNIZED` in comments. ([\#12665](#12665)) - Release script: confirm the commit to be tagged before tagging. ([\#12556](#12556)) - Fix a typo in the announcement text generated by the Synapse release development script. ([\#12612](#12612)) - Fix scripts-dev to pass typechecking. ([\#12356](#12356)) - Add some type hints to datastore. ([\#12485](#12485)) - Remove unused `# type: ignore`s. ([\#12531](#12531)) - Allow unused `# type: ignore` comments in bleeding edge CI jobs. ([\#12576](#12576)) - Remove redundant lines of config from `mypy.ini`. ([\#12608](#12608)) - Update to mypy 0.950. ([\#12650](#12650)) - Use `Concatenate` to better annotate `_do_execute`. ([\#12666](#12666)) - Use `ParamSpec` to refine type hints. ([\#12667](#12667)) - Fix mypy against latest pillow stubs. ([\#12671](#12671))

David Robertson added 2 commits April 26, 2022 13:57

Confirm commit to be tagged before tagging

27284ca

Changelog

4a3669b

DMRobertson marked this pull request as ready for review April 26, 2022 14:40

DMRobertson requested a review from a team as a code owner April 26, 2022 14:40

erikjohnston reviewed Apr 26, 2022

View reviewed changes

David Robertson added 2 commits April 26, 2022 16:41

Undo commented bit that shouldn't have been, argh

0e3c017

Instead, abort if we're on the wrong branch

34105e3

DMRobertson requested a review from erikjohnston April 26, 2022 16:08

erikjohnston previously approved these changes Apr 26, 2022

View reviewed changes

David Robertson added 3 commits April 26, 2022 18:24

factor out a get_repo function

bc23719

Check we have the right tag checked out

6895e89

Clarify that publish only releases to GitHub

f3d9985

DMRobertson changed the title ~~Confirm commit to be tagged before tagging~~ Check we're on the right branch before tagging, and on the right tag before uploading Apr 26, 2022

DMRobertson requested a review from erikjohnston April 27, 2022 11:17

David Robertson added 2 commits April 27, 2022 12:19

Appease flake8

a646562

Better name for get_repo

042dc35

erikjohnston approved these changes May 3, 2022

View reviewed changes

DMRobertson changed the base branch from release-v1.58 to develop May 3, 2022 10:22

Merge branch 'develop' into dmr/confirm-commit-to-be-tagged

5d24d26

DMRobertson enabled auto-merge (squash) May 3, 2022 10:24

DMRobertson merged commit bf2fea8 into develop May 3, 2022

DMRobertson deleted the dmr/confirm-commit-to-be-tagged branch May 3, 2022 10:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check we're on the right branch before tagging, and on the right tag before uploading #12556

Check we're on the right branch before tagging, and on the right tag before uploading #12556

DMRobertson commented Apr 26, 2022

erikjohnston Apr 26, 2022

DMRobertson Apr 26, 2022

erikjohnston Apr 26, 2022

DMRobertson Apr 26, 2022

erikjohnston Apr 26, 2022

erikjohnston left a comment

DMRobertson commented Apr 26, 2022

DMRobertson commented Apr 26, 2022

DMRobertson commented Apr 27, 2022

erikjohnston May 3, 2022

DMRobertson May 3, 2022

DMRobertson commented May 3, 2022

	release_branch_name = (
	f"release-v{parsed_new_version.major}.{parsed_new_version.minor}"
	)
	release_branch = find_ref(repo, release_branch_name)

Check we're on the right branch before tagging, and on the right tag before uploading #12556

Check we're on the right branch before tagging, and on the right tag before uploading #12556

Conversation

DMRobertson commented Apr 26, 2022

erikjohnston Apr 26, 2022

Choose a reason for hiding this comment

DMRobertson Apr 26, 2022

Choose a reason for hiding this comment

erikjohnston Apr 26, 2022

Choose a reason for hiding this comment

DMRobertson Apr 26, 2022

Choose a reason for hiding this comment

erikjohnston Apr 26, 2022

Choose a reason for hiding this comment

erikjohnston left a comment

Choose a reason for hiding this comment

DMRobertson commented Apr 26, 2022

DMRobertson commented Apr 26, 2022

DMRobertson commented Apr 27, 2022

erikjohnston May 3, 2022

Choose a reason for hiding this comment

DMRobertson May 3, 2022

Choose a reason for hiding this comment

DMRobertson commented May 3, 2022