Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beeper 1.114.0 #43

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
873dade
Add prometheus histograms for room shutdown/delete/purge
Fizzadar Mar 12, 2024
b74698c
Install shared secret authenticator in Docker image
Fizzadar Mar 8, 2024
06a5eb3
Update Beeper base rules
Fizzadar Mar 8, 2024
f767ff9
Add JWT UI auth flow
Fizzadar Mar 8, 2024
4aaaa8c
Update email copy for Beeper
Fizzadar Mar 8, 2024
00f1432
Add Beeper log formatter
Fizzadar Mar 8, 2024
b1fbed2
Add Beeper previews
Fizzadar Mar 8, 2024
bea2a98
Add Beeper inbox endpoints
Fizzadar Mar 8, 2024
7a8d680
Remove any bridge bot users from ignored user account data
Fizzadar Mar 8, 2024
0b291dd
Add Beeper notification counts
Fizzadar Aug 5, 2024
cdebea3
Include stream ordering in the unsigned event dict
Fizzadar Mar 8, 2024
81e8233
Allow clients to add extra content to receipts
Fizzadar Mar 8, 2024
e2616d2
Include empty identity server in well known
Fizzadar Oct 25, 2024
e0010ad
Allow checking username availability when registration disabled
Fizzadar Mar 8, 2024
e850cfd
Change default DM room redaction power level to 100
Fizzadar Mar 8, 2024
c532542
Disable transaction timeout when purging rooms
Fizzadar Mar 8, 2024
f257305
Send all notifications as high priority
Fizzadar Mar 8, 2024
8ab2bf0
Add Beeper specific test cases
Fizzadar Mar 8, 2024
5c5dad0
Add CI
Fizzadar Mar 8, 2024
04c8b86
Remove dependabot config
Fizzadar Mar 8, 2024
ce45f60
Add Beeper readme
Fizzadar Mar 8, 2024
16f975d
Add Beeper release flow scripts
Fizzadar Mar 25, 2024
c7b3f96
Add env var to disable writing to the `user_ips` table
Fizzadar Mar 28, 2024
fa59875
Increase last seen granularity to 1h
Fizzadar Mar 28, 2024
cc32545
Bump sytest/complement commits
Fizzadar May 21, 2024
d70d1b8
Add at_order property to inbox_state endpoint
bradtgmurray Aug 13, 2024
fb3f15b
Look for done.at_order in the correct place in inbox_state endpoint
bradtgmurray Aug 13, 2024
3b6ee83
Bump sytest + complement commits
Fizzadar Sep 9, 2024
0bbfe31
Add note to readme about real fork
Fizzadar Sep 9, 2024
2f2d3c7
Enable admin APIs on synapse workers
Fizzadar Oct 4, 2024
d10e5e3
Bump CI action
Fizzadar Oct 4, 2024
e6d4fc8
Register the admin user servelet in generic worker
Fizzadar Oct 4, 2024
7549fdb
Enable correct admin servelet
Fizzadar Oct 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/dependabot.yml

This file was deleted.

138 changes: 138 additions & 0 deletions .github/workflows/beeper-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
name: Beep

on:
push:
branches: ["beeper", "beeper-*"]
pull_request:


jobs:
lint-style:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install poetry
- run: poetry install
- run: poetry run isort --check synapse
- run: poetry run black --check synapse

lint-types:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install poetry
- run: poetry install --extras all
- run: poetry run mypy

# Tests

test-trial:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install poetry
- run: poetry install --extras all
- run: poetry run trial -j4 tests

test-sytest:
runs-on: ubuntu-latest
container:
image: matrixdotorg/sytest-synapse:focal
volumes:
- ${{ github.workspace }}:/src
env:
SYTEST_BRANCH: 91a3ab32fcb3ada91423d661125c428e6b2b7abe
TOP: ${{ github.workspace }}
POSTGRES: 1
MULTI_POSTGRES: 1
WOKRERS: 1
steps:
- uses: actions/checkout@v2
- name: Run SyTest
run: /bootstrap.sh synapse
working-directory: /src
- name: Summarise results.tap
if: ${{ always() }}
run: /sytest/scripts/tap_to_gha.pl /logs/results.tap
- name: Upload SyTest logs
uses: actions/upload-artifact@v4
if: ${{ always() }}
with:
name: Sytest Logs - ${{ job.status }} - (${{ join(matrix.*, ', ') }})
path: |
/logs/results.tap
/logs/**/*.log*

test-complement:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v2
with:
repository: matrix-org/complement
path: complement
ref: 39733c1b2f8314800776748cc7164f9a34650686
- name: Install complement dependencies
run: |-
sudo apt-get -qq update
sudo apt-get install -qqy libolm3 libolm-dev
go install -v github.com/gotesttools/gotestfmt/v2/cmd/gotestfmt@latest
- name: Run Complement
run: ./scripts-dev/complement.sh
env:
COMPLEMENT_DIR: complement

# Builds

build-python:
runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
steps:
- uses: actions/checkout@v2
- uses: docker/setup-qemu-action@v2
- uses: docker/setup-buildx-action@v2
with:
# TEMPORARY, see: https://github.com/docker/build-push-action/issues/761
driver-opts: |
image=moby/buildkit:v0.10.6
- uses: docker/login-action@v2
with:
registry: ${{ secrets.CI_REGISTRY }}
username: ${{ secrets.CI_REGISTRY_USER }}
password: ${{ secrets.CI_REGISTRY_PASSWORD }}
- run: |-
if [ "${{ github.ref_name }}" = "beeper" ]; then
tag=$(cat pyproject.toml | grep -E "^version =" | sed -E 's/^version = "(.+)"$/\1/')
else
tag="${{ github.head_ref || github.ref_name }}"
fi

docker buildx build \
--push \
--platform linux/amd64 \
--tag ${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }} \
-f docker/Dockerfile \
.

if [ "${{ github.ref_name }}" = "beeper" ]; then
docker pull ${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }}
docker tag \
${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }} \
${{ secrets.CI_REGISTRY }}/synapse:latest
docker push ${{ secrets.CI_REGISTRY }}/synapse:latest
fi

# Ensure the image works properly
docker run \
--entrypoint '' \
${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }} \
python -m synapse.app.homeserver --help

echo "Pushed image: synapse:$tag-${{ github.sha }}"
if [ "${{ github.ref_name }}" = "beeper" ]; then
echo "Pushed image: synapse:latest"
fi
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Synapse: Beeper Edition

This is Beeper's custom version of synapse, we rebase roughly 25 commits on top of each upstream release with a few Beeper specific modifications. We also have an actual Synapse fork here: [**beeper/synapse-fork**](https://github.com/beeper/synapse-fork) which is where we make changes we expect to merge into upstream.

## Rebase flow

### Create PR

Here we're upgrading to `v1.96.1`:

```
# Make a new branch from the upstream release, we do this so we can create a PR
# of Beeper -> upstream to run tests/confirm we're happy.
git checkout -f v1.96.1
git checkout -b upstream-1.96.1
git push -u beeper upstream-1.96.1

# Check out the base branch, pull any changes
git checkout beeper
git pull

# Now create a new branch to rebase
git checkout -b beeper-1.96.1
# And do the rebase
git rebase v1.96.1
# fix any conflicts...

# Push and make a PR from this branch to the upstream one created above
git push -u beeper beeper-1.96.1
```

### Make release

Once it's ready we just overwrite the `beeper` branch with the new one:

```
git checkout beeper-1.96.1
git push --force beeper beeper
```
31 changes: 31 additions & 0 deletions beeper/complete_release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env bash

set -euo pipefail
source $(realpath $(dirname $0))/utils.sh

BEEPER_REMOTE=$(get_beeper_remote)

VERSION=${1:-}

if [ -z "$VERSION" ]; then
echo >&2 "Must specify version!"
exit 1
fi

echo "Completing Synapse: Beeper Edition version $VERSION"
echo "WARNING: this script will DELETE the branch called: beeper"
read -p "Press enter to continue"

UPSTREAM_BRANCH=upstream-$VERSION
BEEPER_BRANCH=beeper-$VERSION

git checkout $BEEPER_BRANCH
git branch -D beeper
git checkout -b beeper
git push --force $BEEPER_REMOTE beeper

# Cleanup
git branch -D $BEEPER_BRANCH
git push $BEEPER_REMOTE --delete $BEEPER_BRANCH
git branch -D $UPSTREAM_BRANCH
git push $BEEPER_REMOTE --delete $UPSTREAM_BRANCH
43 changes: 43 additions & 0 deletions beeper/prepare_release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/usr/bin/env bash

set -euo pipefail
source $(realpath $(dirname $0))/utils.sh

BEEPER_REMOTE=$(get_beeper_remote)

VERSION=${1:-}

if [ -z "$VERSION" ]; then
echo >&2 "Must specify version!"
exit 1
fi

STARTING_BRANCH=$(git branch --show-current)

echo "Preparing Synapse: Beeper Edition version $VERSION"
echo "WARNING: this script will rebase on top of the CURRENT BRANCH: $STARTING_BRANCH"
read -p "Press enter to continue"

TAG=v$VERSION
UPSTREAM_BRANCH=upstream-$VERSION
BEEPER_BRANCH=beeper-$VERSION

# Checkout the tag, create upstream branch, push it
echo "Setup branch $UPSTREAM_BRANCH"
git checkout -f $TAG
git checkout -b $UPSTREAM_BRANCH
git push -u $BEEPER_REMOTE $UPSTREAM_BRANCH

# Switch back to our starting branch, create new version branch from it
echo "Setup branch $BEEPER_BRANCH"
git checkout $STARTING_BRANCH
git checkout -b $BEEPER_BRANCH

# And rebase against upstream, applying only our Beeper commits
echo "Initiate rebase..."
git rebase $UPSTREAM_BRANCH || read -p "Rebase was a mess, press enter once you fix it"

git push -u $BEEPER_REMOTE $BEEPER_BRANCH

echo "OK we done!"
echo "Go HERE and make the PR: https://github.com/beeper/synapse/compare/upstream-$VERSION...beeper-$VERSION?expand=1"
23 changes: 23 additions & 0 deletions beeper/utils.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
function get_upstream_remote() {
for remote in $(git remote); do
url=$(git remote get-url $remote)
if [ "$url" = "git@github.com:element-hq/synapse.git" ]; then
echo $remote
return 0
fi
done
echo >&2 "No upstream remote found (looking for URL: git@github.com:element-hq/synapse.git)"
return 1
}

function get_beeper_remote() {
for remote in $(git remote); do
url=$(git remote get-url $remote)
if [ "$url" = "git@github.com:beeper/synapse.git" ]; then
echo $remote
return 0
fi
done
echo >&2 "No upstream remote found (looking for URL: git@github.com:beeper/synapse.git)"
return 1
}
4 changes: 4 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,10 @@ COPY --from=requirements /synapse/requirements.txt /synapse/
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --prefix="/install" --no-deps --no-warn-script-location -r /synapse/requirements.txt

# Beeper: install shared secret authenticator
RUN pip install --prefix="/install" --no-deps --no-warn-script-location \
'git+https://github.com/devture/matrix-synapse-shared-secret-auth@e178353ec87c56e0169dd04466d4769da5ed9c46#egg=shared_secret_authenticator'

# Copy over the rest of the synapse source code.
COPY synapse /synapse/synapse/
COPY rust /synapse/rust/
Expand Down
Loading
Loading