Don't load the media repo when configured to use an external media repo #5754

hawkowl · 2019-07-24T13:54:57Z

This checks if directories exist, which does not matter when the media repo is used.

codecov · 2019-07-24T14:04:30Z

Codecov Report

Merging #5754 into develop will decrease coverage by 0.07%.
The diff coverage is 29.41%.

@@             Coverage Diff             @@
##           develop    #5754      +/-   ##
===========================================
- Coverage    63.39%   63.31%   -0.08%     
===========================================
  Files          331      331              
  Lines        36594    36438     -156     
  Branches      6049     6022      -27     
===========================================
- Hits         23197    23070     -127     
+ Misses       11748    11720      -28     
+ Partials      1649     1648       -1

…dia-repo-options

richvdh

some initial thoughts, mostly: can we not hang the new stuff off enable_media_repo rather than introduce a second option which does something similar but subtly different?

synapse/config/repository.py

…dia-repo-options

old

docs/sample_config.yaml

Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

fixed

synapse/app/media_repository.py

richvdh · 2019-08-12T15:53:21Z

synapse/config/repository.py

+
+        # Enable the media store service inside Synapse.
+        #
+        # enable_media_repo: False


Suggested change

# enable_media_repo: False

#enable_media_repo: false

richvdh · 2019-08-12T15:54:23Z

synapse/config/repository.py

@@ -202,6 +215,12 @@ def generate_config_section(self, data_dir_path, **kwargs):

        return (
            r"""
+        ## Media Store ##
+
+        # Enable the media store service inside Synapse.


Suggested change

# Enable the media store service inside Synapse.

# Enable the media store service in the Synapse master. Uncomment the following if you are using a separate

# media store worker.

synapse/rest/admin/__init__.py

richvdh · 2019-08-12T16:08:08Z

synapse/rest/admin/__init__.py

@@ -822,10 +746,18 @@ def register_servlets(hs, http_server):
    VersionServlet(hs).register(http_server)


+def register_servlets_for_media_repo(hs, http_server):


suggestion: could move this into admin/media.py?

richvdh

lgtm modulo nits

…dia-repo-options

Synapse 1.3.0 (2019-08-15) ========================== Bugfixes -------- - Fix 500 Internal Server Error on `publicRooms` when the public room list was cached. ([\#5851](#5851)) Synapse 1.3.0rc1 (2019-08-13) ========================== Features -------- - Use `M_USER_DEACTIVATED` instead of `M_UNKNOWN` for errcode when a deactivated user attempts to login. ([\#5686](#5686)) - Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. ([\#5732](#5732)) - Synapse will no longer serve any media repo admin endpoints when `enable_media_repo` is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. ([\#5754](#5754), [\#5848](#5848)) - Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. ([\#5783](#5783)) - Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. ([\#5807](#5807)) Bugfixes -------- - Fix UISIs during homeserver outage. ([\#5693](#5693), [\#5789](#5789)) - Fix stack overflow in server key lookup code. ([\#5724](#5724)) - start.sh no longer uses deprecated cli option. ([\#5725](#5725)) - Log when we receive an event receipt from an unexpected origin. ([\#5743](#5743)) - Fix debian packaging scripts to correctly build sid packages. ([\#5775](#5775)) - Correctly handle redactions of redactions. ([\#5788](#5788)) - Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. ([\#5798](#5798)) - Fix check that tombstone is a state event in push rules. ([\#5804](#5804)) - Fix error when trying to login as a deactivated user when using a worker to handle login. ([\#5806](#5806)) - Fix bug where user `/sync` stream could get wedged in rare circumstances. ([\#5825](#5825)) - The purge_remote_media.sh script was fixed. ([\#5839](#5839)) Deprecations and Removals ------------------------- - Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration. ([\#5678](#5678), [\#5729](#5729)) - Remove non-functional 'expire_access_token' setting. ([\#5782](#5782)) Internal Changes ---------------- - Make Jaeger fully configurable. ([\#5694](#5694)) - Add precautionary measures to prevent future abuse of `window.opener` in default welcome page. ([\#5695](#5695)) - Reduce database IO usage by optimising queries for current membership. ([\#5706](#5706), [\#5738](#5738), [\#5746](#5746), [\#5752](#5752), [\#5770](#5770), [\#5774](#5774), [\#5792](#5792), [\#5793](#5793)) - Improve caching when fetching `get_filtered_current_state_ids`. ([\#5713](#5713)) - Don't accept opentracing data from clients. ([\#5715](#5715)) - Speed up PostgreSQL unit tests in CI. ([\#5717](#5717)) - Update the coding style document. ([\#5719](#5719)) - Improve database query performance when recording retry intervals for remote hosts. ([\#5720](#5720)) - Add a set of opentracing utils. ([\#5722](#5722)) - Cache result of get_version_string to reduce overhead of `/version` federation requests. ([\#5730](#5730)) - Return 'user_type' in admin API user endpoints results. ([\#5731](#5731)) - Don't package the sytest test blacklist file. ([\#5733](#5733)) - Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. ([\#5736](#5736)) - Blacklist some flakey tests in worker mode. ([\#5740](#5740)) - Fix some error cases in the caching layer. ([\#5749](#5749)) - Add a prometheus metric for pending cache lookups. ([\#5750](#5750)) - Stop trying to fetch events with event_id=None. ([\#5753](#5753)) - Convert RedactionTestCase to modern test style. ([\#5768](#5768)) - Allow looping calls to be given arguments. ([\#5780](#5780)) - Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. ([\#5785](#5785)) - Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. ([\#5787](#5787)) - Remove some spurious exceptions from the logs where we failed to talk to a remote server. ([\#5790](#5790)) - Improve performance when making `.well-known` requests by sharing the SSL options between requests. ([\#5794](#5794)) - Disable codecov GitHub comments on PRs. ([\#5796](#5796)) - Don't allow clients to send tombstone events that reference the room it's sent in. ([\#5801](#5801)) - Deny redactions of events sent in a different room. ([\#5802](#5802)) - Deny sending well known state types as non-state events. ([\#5805](#5805)) - Handle incorrectly encoded query params correctly by returning a 400. ([\#5808](#5808)) - Handle pusher being deleted during processing rather than logging an exception. ([\#5809](#5809)) - Return 502 not 500 when failing to reach any remote server. ([\#5810](#5810)) - Reduce global pauses in the events stream caused by expensive state resolution during persistence. ([\#5826](#5826)) - Add a lower bound to well-known lookup cache time to avoid repeated lookups. ([\#5836](#5836)) - Whitelist history visbility sytests in worker mode tests. ([\#5843](#5843))

…po (#5754)

hawkowl added 2 commits July 24, 2019 23:46

disable media repo if it's disabled and not an option

108bc6c

changelog

997fd94

hawkowl requested a review from a team July 24, 2019 13:54

hawkowl added 3 commits July 24, 2019 23:56

config sync

bbffae3

config sync

4f2caf1

config sync

c463960

hawkowl mentioned this pull request Jul 24, 2019

Turning off the media repo doesn't stop the background updates from running #5517

Closed

hawkowl added 8 commits July 25, 2019 00:15

reorder config too

081e102

reorder config too

cd70e82

fix

3ff9935

fix

85f5cbd

Merge remote-tracking branch 'origin/develop' into hawkowl/disable-me…

1763a73

…dia-repo-options

fix worker mode just exploding

f528371

unmove this about

c7561c2

Update and rename 5754.misc to 5754.feature

f695e0b

hawkowl changed the title ~~When the media repo is disabled, don't load the config options~~ Don't load the media repo when configured to use an external media repo Jul 29, 2019

hawkowl self-assigned this Jul 29, 2019

richvdh previously requested changes Jul 30, 2019

View reviewed changes

hawkowl added 4 commits August 9, 2019 22:17

Merge remote-tracking branch 'origin/develop' into hawkowl/disable-me…

22c1d80

…dia-repo-options

fix

fe5d337

cleanup config

5f6b687

fix import

7eeed4b

hawkowl added 3 commits August 9, 2019 22:50

lol oops

74232f5

reduce diff

7c2149f

doc

3a23f6a

anoadragon453 previously requested changes Aug 9, 2019

View reviewed changes

docs/sample_config.yaml Outdated Show resolved Hide resolved

hawkowl and others added 2 commits August 9, 2019 23:58

Update docs/sample_config.yaml

afb31e9

Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

fix

4a0380b

hawkowl requested a review from a team August 9, 2019 16:27

richvdh suggested changes Aug 12, 2019

View reviewed changes

richvdh reviewed Aug 12, 2019

View reviewed changes

richvdh approved these changes Aug 12, 2019

View reviewed changes

hawkowl added 2 commits August 13, 2019 21:22

Merge remote-tracking branch 'origin/develop' into hawkowl/disable-me…

94ec373

…dia-repo-options

review cleanups

80d6c48

hawkowl merged commit 0b6fbb2 into develop Aug 13, 2019

hawkowl deleted the hawkowl/disable-media-repo-options branch August 13, 2019 11:49

turt2live mentioned this pull request Sep 2, 2019

Ensure the list media admin API is always available #5966

Merged

3 tasks

babolivier pushed a commit that referenced this pull request Sep 1, 2021

Don't load the media repo when configured to use an external media re…

57c2254

…po (#5754)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't load the media repo when configured to use an external media repo #5754

Don't load the media repo when configured to use an external media repo #5754

hawkowl commented Jul 24, 2019 •

edited

Loading

codecov bot commented Jul 24, 2019 •

edited

Loading

richvdh left a comment

richvdh Aug 12, 2019

richvdh Aug 12, 2019

richvdh Aug 12, 2019

richvdh left a comment

	# Enable the media store service inside Synapse.
	# Enable the media store service in the Synapse master. Uncomment the following if you are using a separate
	# media store worker.

		@@ -822,10 +746,18 @@ def register_servlets(hs, http_server):
		VersionServlet(hs).register(http_server)


		def register_servlets_for_media_repo(hs, http_server):

Don't load the media repo when configured to use an external media repo #5754

Don't load the media repo when configured to use an external media repo #5754

Conversation

hawkowl commented Jul 24, 2019 • edited Loading

codecov bot commented Jul 24, 2019 • edited Loading

Codecov Report

richvdh left a comment

Choose a reason for hiding this comment

richvdh Aug 12, 2019

Choose a reason for hiding this comment

richvdh Aug 12, 2019

Choose a reason for hiding this comment

richvdh Aug 12, 2019

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

hawkowl commented Jul 24, 2019 •

edited

Loading

codecov bot commented Jul 24, 2019 •

edited

Loading