Merge Main into Production for May 2024 Release #58
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- custom storages - tar.gz within bucket using s3_tar - slightly changes export directory structure - major problems still outstanding re delivering s3 files to end users
- use signed url for s3 downloads - re-arrange tar.gz file to match original - delete all working files after tarring - import from s3 export TODO - check local export and import - fix error when avatar missing - deal with multiple s3 storage options (e.g. Azure)
pulls Mouse's fix for imagefile serialization
As of 0.1.13, the s3-tar library uses an environment variable (`S3_ENDPOINT_URL`) to determine the AWS endpoint. See: https://github.com/xtream1101/s3-tar/blob/0.1.13/s3_tar/utils.py#L25-L29. To save BookWyrm admins from having to set it (e.g., through `.env`) when they are already setting `AWS_S3_ENDPOINT_URL`, we create a Session class that unconditionally uses that URL, and feed it to S3Tar.
also undoes a line space change in settings.py to make the PR cleaner
Subclass boto3.Session to use AWS_S3_ENDPOINT_URL
Explicitly give codeql-analysis action the security-events: write permission so it still works even when the default GitHub Actions token is set to read-only.
* Bump versions * Bump back Pillow due to test failure * Bump setuptools
The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-DJANGO-5880505 - https://snyk.io/vuln/SNYK-PYTHON-DJANGO-5932095 - https://snyk.io/vuln/SNYK-PYTHON-DJANGO-6041515 - https://snyk.io/vuln/SNYK-PYTHON-DJANGO-6230369 - https://snyk.io/vuln/SNYK-PYTHON-DJANGO-6370660 - https://snyk.io/vuln/SNYK-PYTHON-OPENTELEMETRYINSTRUMENTATION-5926995 - https://snyk.io/vuln/SNYK-PYTHON-PILLOW-5918878 - https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6043904 - https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6182918 - https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6219984 - https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6219986 - https://snyk.io/vuln/SNYK-PYTHON-SETUPTOOLS-3180412
Reapplying the additional permission added to this fork in commit aaa31b4, which was reverted as part of our last sync to main. We want to keep this permission so we can get security alerts from the CodeQL Action.
The Pillow version bump recommended here is going to require other code changes, and shouldn't be handled as part of a bulk maintenance-task merge. I'd like to see if we can take the rest of these security upgrades without it, though.
Adding opentelemetry-instrumentation>=0.41b0 as snyk recommended broke out opentelemetry-instrumentation-* packages, so this time I removed the dependency snyk had added and instead bumped the versions on our opentelemetry-instrumentation-*s
The opentelemetry-sdk at 1.16.0 had a dependency that conflicted with the versions required by opentelemetry-instrumentation-* packages at versions we'd like to upgrade to for security reasons, so we'll need to upgrade opentelemetry-sdk too.
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.2 to 3.9.4. - [Release notes](https://github.com/aio-libs/aiohttp/releases) - [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst) - [Commits](aio-libs/aiohttp@v3.9.2...v3.9.4) --- updated-dependencies: - dependency-name: aiohttp dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
As outlined in bookwyrm-social#3354, a use `Move` fails if the user is moving from a BookWyrm server to another BookWrym server. This is because: 1. the original code did not announce changes to alsoKnownAs; 2. the original code always checked the locally saved profile rather than refetching the remote data; This commit fixes both these problems by forcing `MoveUser` to always perform a "refresh" of the local data from the remote, and by saving the user with broadcast=True when updating alsoKnownAs ids.
Support AWS_S3_URL_PROTOCOL
refactor Move for more redundancy
Otherwise the quotes end up in the strings.
…ot/pip/aiohttp-3.9.4 Bump aiohttp from 3.9.2 to 3.9.4
Correctly handle serving BookWyrm on custom port
feat: add OAuth authentication
Explicitly give codeql-analysis action the security-events: write permission so it still works even when the default GitHub Actions token is set to read-only.
* Bump versions * Bump back Pillow due to test failure * Bump setuptools
Catch Us Up to Upstream
…2009743df [Snyk] Security upgrade sqlparse from 0.4.4 to 0.5.0
…5bf2cc504 [Snyk] Fixes for Opentelemetry-related vulnerabilities: opentelemetry-api==1.22.0 opentelemetry-exporter-otlp-proto-grpc==1.22.0 opentelemetry-instrumentation-celery==0.43b0 opentelemetry-instrumentation-django==0.43b0 opentelemetry-instrumentation-psycopg2==0.43b0 opentelemetry-sdk==1.22.0
|
Note that I hand-resolved 190 merge conflicts here, so the odds that I broke something are extremely high. |
The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-REQUESTS-6928867
…ca9cf0824 [Snyk] Security upgrade requests from 2.31.0 to 2.32.0
|
Closing this; the issues we were running into were due to discrepancies between main and production in upstream. After discussion and debugging with @phildini , we're pretty sure that our previous merge strategy kept hitting issues due to these discrepancies. Work is underway in the upstream repo to simplify branch structure, but until that is finished we will pull to our main and production from the respective upstreams, and will not merge our main to production at all. Our security upgrades will go directly into our production so we can try them in our prod before recommending upstream, and feature development work will go into our main, from our main to upstream main, from upstream main to upstream production, and from upstream production to our production. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is to merge main into production, both in prep for the May 2024 release and because Snyk appears to be scanning our production branch instead of main, so vulnerabilities are not appearing to be resolved.