Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Commit

Permalink
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/pas…
Browse files Browse the repository at this point in the history
…sword_reset_confirmation

* 'develop' of github.com:matrix-org/synapse: (85 commits)
  Use the default templates when a custom template file cannot be found (#8037)
  Changelog changes
  1.19.0
  Convert stream database to async/await. (#8074)
  Add a shadow-banned flag to users. (#8092)
  Convert pusher databases to async/await. (#8075)
  Convert receipts and events databases to async/await. (#8076)
  Add type hints to synapse.handlers.room (#8090)
  Remove some unused database functions. (#8085)
  Convert misc database code to async (#8087)
  Remove a space at the start of a changelog entry.
  More changelog tweaks
  More changelog tweaks
  Remove unwanted changelog line
  1.19.0rc1
  Run `remove_push_actions_from_staging` in foreground (#8081)
  Drop federation transmission queues during a significant remote outage. (#7864)
  Fix unawaited coroutine error in tests. (#8072)
  Convert the roommember database to async/await. (#8070)
  Convert devices database to async/await. (#8069)
  ...
  • Loading branch information
anoadragon453 committed Aug 17, 2020
2 parents 129903e + e04e465 commit 8895af3
Show file tree
Hide file tree
Showing 496 changed files with 5,482 additions and 5,009 deletions.
6 changes: 2 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,16 @@ jobs:
machine: true
steps:
- checkout
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} -t matrixdotorg/synapse:${CIRCLE_TAG}-py3 .
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} .
- run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
- run: docker push matrixdotorg/synapse:${CIRCLE_TAG}
- run: docker push matrixdotorg/synapse:${CIRCLE_TAG}-py3
dockerhubuploadlatest:
machine: true
steps:
- checkout
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest -t matrixdotorg/synapse:latest-py3 .
- run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest .
- run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
- run: docker push matrixdotorg/synapse:latest
- run: docker push matrixdotorg/synapse:latest-py3

workflows:
version: 2
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/BUG_REPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ about: Create a report to help us improve

---

<!--
**THIS IS NOT A SUPPORT CHANNEL!**
**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**,
please ask in **#synapse:matrix.org** (using a matrix.org account if necessary)
<!--
If you want to report a security issue, please see https://matrix.org/security-disclosure-policy/
This is a bug report template. By following the instructions below and
Expand Down
74 changes: 74 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,77 @@
Synapse 1.19.0 (2020-08-17)
===========================

No significant changes since 1.19.0rc1.

Removal warning
---------------

As outlined in the [previous release](https://github.com/matrix-org/synapse/releases/tag/v1.18.0), we are no longer publishing Docker images with the `-py3` tag suffix. On top of that, we have also removed the `latest-py3` tag. Please see [the announcement in the upgrade notes for 1.18.0](https://github.com/matrix-org/synapse/blob/develop/UPGRADE.rst#upgrading-to-v1180).


Synapse 1.19.0rc1 (2020-08-13)
==============================

Features
--------

- Add option to allow server admins to join rooms which fail complexity checks. Contributed by @lugino-emeritus. ([\#7902](https://github.com/matrix-org/synapse/issues/7902))
- Add an option to purge room or not with delete room admin endpoint (`POST /_synapse/admin/v1/rooms/<room_id>/delete`). Contributed by @dklimpel. ([\#7964](https://github.com/matrix-org/synapse/issues/7964))
- Add rate limiting to users joining rooms. ([\#8008](https://github.com/matrix-org/synapse/issues/8008))
- Add a `/health` endpoint to every configured HTTP listener that can be used as a health check endpoint by load balancers. ([\#8048](https://github.com/matrix-org/synapse/issues/8048))
- Allow login to be blocked based on the values of SAML attributes. ([\#8052](https://github.com/matrix-org/synapse/issues/8052))
- Allow guest access to the `GET /_matrix/client/r0/rooms/{room_id}/members` endpoint, according to MSC2689. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#7314](https://github.com/matrix-org/synapse/issues/7314))


Bugfixes
--------

- Fix a bug introduced in Synapse v1.7.2 which caused inaccurate membership counts in the room directory. ([\#7977](https://github.com/matrix-org/synapse/issues/7977))
- Fix a long standing bug: 'Duplicate key value violates unique constraint "event_relations_id"' when message retention is configured. ([\#7978](https://github.com/matrix-org/synapse/issues/7978))
- Fix "no create event in auth events" when trying to reject invitation after inviter leaves. Bug introduced in Synapse v1.10.0. ([\#7980](https://github.com/matrix-org/synapse/issues/7980))
- Fix various comments and minor discrepencies in server notices code. ([\#7996](https://github.com/matrix-org/synapse/issues/7996))
- Fix a long standing bug where HTTP HEAD requests resulted in a 400 error. ([\#7999](https://github.com/matrix-org/synapse/issues/7999))
- Fix a long-standing bug which caused two copies of some log lines to be written when synctl was used along with a MemoryHandler logger. ([\#8011](https://github.com/matrix-org/synapse/issues/8011), [\#8012](https://github.com/matrix-org/synapse/issues/8012))


Updates to the Docker image
---------------------------

- We no longer publish Docker images with the `-py3` tag suffix, as [announced in the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/UPGRADE.rst#upgrading-to-v1180). ([\#8056](https://github.com/matrix-org/synapse/issues/8056))


Improved Documentation
----------------------

- Document how to set up a client .well-known file and fix several pieces of outdated documentation. ([\#7899](https://github.com/matrix-org/synapse/issues/7899))
- Improve workers docs. ([\#7990](https://github.com/matrix-org/synapse/issues/7990), [\#8000](https://github.com/matrix-org/synapse/issues/8000))
- Fix typo in `docs/workers.md`. ([\#7992](https://github.com/matrix-org/synapse/issues/7992))
- Add documentation for how to undo a room shutdown. ([\#7998](https://github.com/matrix-org/synapse/issues/7998), [\#8010](https://github.com/matrix-org/synapse/issues/8010))


Internal Changes
----------------

- Reduce the amount of whitespace in JSON stored and sent in responses. Contributed by David Vo. ([\#7372](https://github.com/matrix-org/synapse/issues/7372))
- Switch to the JSON implementation from the standard library and bump the minimum version of the canonicaljson library to 1.2.0. ([\#7936](https://github.com/matrix-org/synapse/issues/7936), [\#7979](https://github.com/matrix-org/synapse/issues/7979))
- Convert various parts of the codebase to async/await. ([\#7947](https://github.com/matrix-org/synapse/issues/7947), [\#7948](https://github.com/matrix-org/synapse/issues/7948), [\#7949](https://github.com/matrix-org/synapse/issues/7949), [\#7951](https://github.com/matrix-org/synapse/issues/7951), [\#7963](https://github.com/matrix-org/synapse/issues/7963), [\#7973](https://github.com/matrix-org/synapse/issues/7973), [\#7975](https://github.com/matrix-org/synapse/issues/7975), [\#7976](https://github.com/matrix-org/synapse/issues/7976), [\#7981](https://github.com/matrix-org/synapse/issues/7981), [\#7987](https://github.com/matrix-org/synapse/issues/7987), [\#7989](https://github.com/matrix-org/synapse/issues/7989), [\#8003](https://github.com/matrix-org/synapse/issues/8003), [\#8014](https://github.com/matrix-org/synapse/issues/8014), [\#8016](https://github.com/matrix-org/synapse/issues/8016), [\#8027](https://github.com/matrix-org/synapse/issues/8027), [\#8031](https://github.com/matrix-org/synapse/issues/8031), [\#8032](https://github.com/matrix-org/synapse/issues/8032), [\#8035](https://github.com/matrix-org/synapse/issues/8035), [\#8042](https://github.com/matrix-org/synapse/issues/8042), [\#8044](https://github.com/matrix-org/synapse/issues/8044), [\#8045](https://github.com/matrix-org/synapse/issues/8045), [\#8061](https://github.com/matrix-org/synapse/issues/8061), [\#8062](https://github.com/matrix-org/synapse/issues/8062), [\#8063](https://github.com/matrix-org/synapse/issues/8063), [\#8066](https://github.com/matrix-org/synapse/issues/8066), [\#8069](https://github.com/matrix-org/synapse/issues/8069), [\#8070](https://github.com/matrix-org/synapse/issues/8070))
- Move some database-related log lines from the default logger to the database/transaction loggers. ([\#7952](https://github.com/matrix-org/synapse/issues/7952))
- Add a script to detect source code files using non-unix line terminators. ([\#7965](https://github.com/matrix-org/synapse/issues/7965), [\#7970](https://github.com/matrix-org/synapse/issues/7970))
- Log the SAML session ID during creation. ([\#7971](https://github.com/matrix-org/synapse/issues/7971))
- Implement new experimental push rules for some users. ([\#7997](https://github.com/matrix-org/synapse/issues/7997))
- Remove redundant and unreliable signature check for v1 Identity Service lookup responses. ([\#8001](https://github.com/matrix-org/synapse/issues/8001))
- Improve the performance of the register endpoint. ([\#8009](https://github.com/matrix-org/synapse/issues/8009))
- Reduce less useful output in the newsfragment CI step. Add a link to the changelog section of the contributing guide on error. ([\#8024](https://github.com/matrix-org/synapse/issues/8024))
- Rename storage layer objects to be more sensible. ([\#8033](https://github.com/matrix-org/synapse/issues/8033))
- Change the default log config to reduce disk I/O and storage for new servers. ([\#8040](https://github.com/matrix-org/synapse/issues/8040))
- Add an assertion on `prev_events` in `create_new_client_event`. ([\#8041](https://github.com/matrix-org/synapse/issues/8041))
- Add a comment to `ServerContextFactory` about the use of `SSLv23_METHOD`. ([\#8043](https://github.com/matrix-org/synapse/issues/8043))
- Log `OPTIONS` requests at `DEBUG` rather than `INFO` level to reduce amount logged at `INFO`. ([\#8049](https://github.com/matrix-org/synapse/issues/8049))
- Reduce amount of outbound request logging at `INFO` level. ([\#8050](https://github.com/matrix-org/synapse/issues/8050))
- It is no longer necessary to explicitly define `filters` in the logging configuration. (Continuing to do so is redundant but harmless.) ([\#8051](https://github.com/matrix-org/synapse/issues/8051))
- Add and improve type hints. ([\#8058](https://github.com/matrix-org/synapse/issues/8058), [\#8064](https://github.com/matrix-org/synapse/issues/8064), [\#8060](https://github.com/matrix-org/synapse/issues/8060), [\#8067](https://github.com/matrix-org/synapse/issues/8067))


Synapse 1.18.0 (2020-07-30)
===========================

Expand Down
1 change: 0 additions & 1 deletion changelog.d/7736.feature

This file was deleted.

1 change: 1 addition & 0 deletions changelog.d/7864.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix a memory leak by limiting the length of time that messages will be queued for a remote server that has been unreachable.
1 change: 0 additions & 1 deletion changelog.d/7899.doc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7902.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7936.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7952.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7964.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7965.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7970.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7971.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7973.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7975.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7976.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7978.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7979.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7980.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7981.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7990.doc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/7992.doc

This file was deleted.

1 change: 1 addition & 0 deletions changelog.d/8037.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Use the default template file when its equivalent is not found in a custom template directory.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
1 change: 1 addition & 0 deletions changelog.d/8081.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix `Re-starting finished log context PUT-nnnn` warning when event persistence failed.
1 change: 1 addition & 0 deletions changelog.d/8085.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove some unused database functions.
File renamed without changes.
1 change: 1 addition & 0 deletions changelog.d/8090.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add type hints to `synapse.handlers.room`.
1 change: 1 addition & 0 deletions changelog.d/8092.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add support for shadow-banning users (ignoring any message send requests).
6 changes: 3 additions & 3 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
matrix-synapse-py3 (1.xx.0) stable; urgency=medium
matrix-synapse-py3 (1.19.0) stable; urgency=medium

[ Synapse Packaging team ]
* New synapse release 1.xx.0.
* New synapse release 1.19.0.

[ Aaron Raimist ]
* Fix outdated documentation for SYNAPSE_CACHE_FACTOR

-- Synapse Packaging team <packages@matrix.org> XXXXX
-- Synapse Packaging team <packages@matrix.org> Mon, 17 Aug 2020 14:06:42 +0100

matrix-synapse-py3 (1.18.0) stable; urgency=medium

Expand Down
6 changes: 0 additions & 6 deletions docker/conf/log.config
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,10 @@ formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'

filters:
context:
(): synapse.logging.context.LoggingContextFilter
request: ""

handlers:
console:
class: logging.StreamHandler
formatter: precise
filters: [context]

loggers:
synapse.storage.SQL:
Expand Down
29 changes: 28 additions & 1 deletion docs/admin_api/shutdown_room.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ You will need to authenticate with an access token for an admin user.
* `message` - Optional. A string containing the first message that will be sent as
`new_room_user_id` in the new room. Ideally this will clearly convey why the
original room was shut down.

If not specified, the default value of `room_name` is "Content Violation
Notification". The default value of `message` is "Sharing illegal content on
othis server is not permitted and rooms in violation will be blocked."
Expand Down Expand Up @@ -72,3 +72,30 @@ Response:
"new_room_id": "!newroomid:example.com",
},
```

## Undoing room shutdowns

*Note*: This guide may be outdated by the time you read it. By nature of room shutdowns being performed at the database level,
the structure can and does change without notice.

First, it's important to understand that a room shutdown is very destructive. Undoing a shutdown is not as simple as pretending it
never happened - work has to be done to move forward instead of resetting the past. In fact, in some cases it might not be possible
to recover at all:

* If the room was invite-only, your users will need to be re-invited.
* If the room no longer has any members at all, it'll be impossible to rejoin.
* The first user to rejoin will have to do so via an alias on a different server.

With all that being said, if you still want to try and recover the room:

1. For safety reasons, shut down Synapse.
2. In the database, run `DELETE FROM blocked_rooms WHERE room_id = '!example:example.org';`
* For caution: it's recommended to run this in a transaction: `BEGIN; DELETE ...;`, verify you got 1 result, then `COMMIT;`.
* The room ID is the same one supplied to the shutdown room API, not the Content Violation room.
3. Restart Synapse.

You will have to manually handle, if you so choose, the following:

* Aliases that would have been redirected to the Content Violation room.
* Users that would have been booted from the room (and will have been force-joined to the Content Violation room).
* Removal of the Content Violation room if desired.
7 changes: 7 additions & 0 deletions docs/reverse_proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,10 @@ client IP addresses are recorded correctly.
Having done so, you can then use `https://matrix.example.com` (instead
of `https://matrix.example.com:8448`) as the "Custom server" when
connecting to Synapse from a client.


## Health check endpoint

Synapse exposes a health check endpoint for use by reverse proxies.
Each configured HTTP listener has a `/health` endpoint which always returns
200 OK (and doesn't get logged).
27 changes: 24 additions & 3 deletions docs/sample_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,10 @@ log_config: "CONFDIR/SERVERNAME.log.config"
# - one for ratelimiting redactions by room admins. If this is not explicitly
# set then it uses the same ratelimiting as per rc_message. This is useful
# to allow room admins to deal with abuse quickly.
# - two for ratelimiting number of rooms a user can join, "local" for when
# users are joining rooms the server is already in (this is cheap) vs
# "remote" for when users are trying to join rooms not on the server (which
# can be more expensive)
#
# The defaults are as shown below.
#
Expand All @@ -771,6 +775,14 @@ log_config: "CONFDIR/SERVERNAME.log.config"
#rc_admin_redaction:
# per_second: 1
# burst_count: 50
#
#rc_joins:
# local:
# per_second: 0.1
# burst_count: 3
# remote:
# per_second: 0.01
# burst_count: 3


# Ratelimiting settings for incoming federation
Expand Down Expand Up @@ -1565,6 +1577,17 @@ saml2_config:
#
#grandfathered_mxid_source_attribute: upn

# It is possible to configure Synapse to only allow logins if SAML attributes
# match particular values. The requirements can be listed under
# `attribute_requirements` as shown below. All of the listed attributes must
# match for the login to be permitted.
#
#attribute_requirements:
# - attribute: userGroup
# value: "staff"
# - attribute: department
# value: "sales"

# Directory in which Synapse will try to find the template files below.
# If not set, default templates from within the Synapse package will be used.
#
Expand Down Expand Up @@ -1979,9 +2002,7 @@ email:
# Directory in which Synapse will try to find the template files below.
# If not set, default templates from within the Synapse package will be used.
#
# DO NOT UNCOMMENT THIS SETTING unless you want to customise the templates.
# If you *do* uncomment it, you will need to make sure that all the templates
# below are in the directory.
# Do not uncomment this setting unless you want to customise the templates.
#
# Synapse will look for the following templates in this directory:
#
Expand Down
46 changes: 35 additions & 11 deletions docs/sample_log_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,33 +11,57 @@ formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'

filters:
context:
(): synapse.logging.context.LoggingContextFilter
request: ""

handlers:
file:
class: logging.handlers.RotatingFileHandler
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /var/log/matrix-synapse/homeserver.log
maxBytes: 104857600
backupCount: 10
filters: [context]
when: midnight
backupCount: 3 # Does not include the current log file.
encoding: utf8

# Default to buffering writes to log file for efficiency. This means that
# will be a delay for INFO/DEBUG logs to get written, but WARNING/ERROR
# logs will still be flushed immediately.
buffer:
class: logging.handlers.MemoryHandler
target: file
# The capacity is the number of log lines that are buffered before
# being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
capacity: 10
flushLevel: 30 # Flush for WARNING logs as well

# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
filters: [context]

loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: INFO

twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false

root:
level: INFO
handlers: [file, console]

# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuation for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]

disable_existing_loggers: false
Loading

0 comments on commit 8895af3

Please sign in to comment.