matrix.org's sentry is full of logcontext errors

[richvdh]

See https://sentry.matrix.org/sentry/synapse-matrixorg/issues/15805, for example. They mostly seem to be due to varanustra, though that should have been fixed in https://gitlab.matrix.org/new-vector/varunastra/-/merge_requests/20, so we need to investigate why it's still full of spam.

[callahad]

Link to contextmanager issues in Sentry: https://sentry.matrix.org/sentry/synapse-matrixorg/issues/?query=logger%3Asynapse.logging.scopecontextmanager

Seems to have changed profile recently; needs further investigation

[squahtx]

The errors we're seeing in Sentry are:

• "Tried to activate scope outside of loggingcontext", which stopped on September 1st
• "Tried to close a non-active scope!", which started on September 1st

These could very well be caused by logging contexts being lost / not being correctly restored.

@richvdh has pointed out that we have a known issue where this can happen during federation send timeouts: #5604

@anoadragon453 has found that synapse-spamcheck-badlist causes logging contexts to be lost, which we have a fix for: matrix-org/synapse-spamcheck-badlist#24

I've opened a PR to try to prevent logging contexts going missing during federation send timeouts: #10810

[squahtx]

The spam in Sentry has disappeared now that matrix-org/synapse-spamcheck-badlist#24 and #10810 have been deployed.

Thanks to @anoadragon453 and @reivilibre for pointing me in the right direction!

[richvdh]

yay, thanks for sorting this!