Merge tag 'v4.1.3' into foresdon-v4.1.3

CHANGELOG.md

@@ -3,6 +3,54 @@ Changelog
 
 All notable changes to this project will be documented in this file.
 
+## [4.1.3] - 2023-07-06
+
+### Added
+
+- Add fallback redirection when getting a webfinger query `LOCAL_DOMAIN@LOCAL_DOMAIN` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23600))
+
+### Changed
+
+- Change OpenGraph-based embeds to allow fullscreen ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25058))
+- Change AccessTokensVacuum to also delete expired tokens ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24868))
+- Change profile updates to be sent to recently-mentioned servers ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24852))
+- Change automatic post deletion thresholds and load detection ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24614))
+- Change `/api/v1/statuses/:id/history` to always return at least one item ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25510))
+- Change auto-linking to allow carets in URL query params ([renchap](https://github.com/mastodon/mastodon/pull/25216))
+
+### Removed
+
+- Remove invalid `X-Frame-Options: ALLOWALL` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25070))
+
+### Fixed
+
+- Fix wrong view being displayed when a webhook fails validation ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25464))
+- Fix soft-deleted post cleanup scheduler overwhelming the streaming server ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25519))
+- Fix incorrect pagination headers in `/api/v2/admin/accounts` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/25477))
+- Fix multiple inefficiencies in automatic post cleanup worker ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24607), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24785), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24840))
+- Fix performance of streaming by parsing message JSON once ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25278), [ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25361))
+- Fix CSP headers when `S3_ALIAS_HOST` includes a path component ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25273))
+- Fix `tootctl accounts approve --number N` not aproving N earliest registrations ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24605))
+- Fix reports not being closed when performing batch suspensions ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24988))
+- Fix being able to vote on your own polls ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25015))
+- Fix race condition when reblogging a status ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25016))
+- Fix “Authorized applications” inefficiently and incorrectly getting last use date ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25060))
+- Fix “Authorized applications” crashing when listing apps with certain admin API scopes ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25713))
+- Fix multiple N+1s in ConversationsController ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25134), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25399), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25499))
+- Fix user archive takeouts when using OpenStack Swift ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24431))
+- Fix searching for remote content by URL not working under certain conditions ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25637))
+- Fix inefficiencies in indexing content for search ([VyrCossont](https://github.com/mastodon/mastodon/pull/24285), [VyrCossont](https://github.com/mastodon/mastodon/pull/24342))
+
+### Security
+
+- Add finer permission requirements for managing webhooks ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25463))
+- Update dependencies
+- Add hardening headers for user-uploaded files ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25756))
+- Fix verified links possibly hiding important parts of the URL (CVE-2023-36462)
+- Fix timeout handling of outbound HTTP requests (CVE-2023-36461)
+- Fix arbitrary file creation through media processing (CVE-2023-36460)
+- Fix possible XSS in preview cards (CVE-2023-36459)
+
 ## [4.1.2] - 2023-04-04
 
 ### Fixed

Gemfile.lock

@@ -10,40 +10,40 @@ GIT
 GEM
  remote: https://rubygems.org/
  specs:
- actioncable (6.1.7.2)
- actionpack (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ actioncable (6.1.7.4)
+ actionpack (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  nio4r (~> 2.0)
  websocket-driver (>= 0.6.1)
- actionmailbox (6.1.7.2)
- actionpack (= 6.1.7.2)
- activejob (= 6.1.7.2)
- activerecord (= 6.1.7.2)
- activestorage (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ actionmailbox (6.1.7.4)
+ actionpack (= 6.1.7.4)
+ activejob (= 6.1.7.4)
+ activerecord (= 6.1.7.4)
+ activestorage (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  mail (>= 2.7.1)
- actionmailer (6.1.7.2)
- actionpack (= 6.1.7.2)
- actionview (= 6.1.7.2)
- activejob (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ actionmailer (6.1.7.4)
+ actionpack (= 6.1.7.4)
+ actionview (= 6.1.7.4)
+ activejob (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  mail (~> 2.5, >= 2.5.4)
  rails-dom-testing (~> 2.0)
- actionpack (6.1.7.2)
- actionview (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ actionpack (6.1.7.4)
+ actionview (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  rack (~> 2.0, >= 2.0.9)
  rack-test (>= 0.6.3)
  rails-dom-testing (~> 2.0)
  rails-html-sanitizer (~> 1.0, >= 1.2.0)
- actiontext (6.1.7.2)
- actionpack (= 6.1.7.2)
- activerecord (= 6.1.7.2)
- activestorage (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ actiontext (6.1.7.4)
+ actionpack (= 6.1.7.4)
+ activerecord (= 6.1.7.4)
+ activestorage (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  nokogiri (>= 1.8.5)
- actionview (6.1.7.2)
- activesupport (= 6.1.7.2)
+ actionview (6.1.7.4)
+ activesupport (= 6.1.7.4)
  builder (~> 3.1)
  erubi (~> 1.4)
  rails-dom-testing (~> 2.0)
@@ -54,22 +54,22 @@ GEM
  case_transform (>= 0.2)
  jsonapi-renderer (>= 0.1.1.beta1, < 0.3)
  active_record_query_trace (1.8)
- activejob (6.1.7.2)
- activesupport (= 6.1.7.2)
+ activejob (6.1.7.4)
+ activesupport (= 6.1.7.4)
  globalid (>= 0.3.6)
- activemodel (6.1.7.2)
- activesupport (= 6.1.7.2)
- activerecord (6.1.7.2)
- activemodel (= 6.1.7.2)
- activesupport (= 6.1.7.2)
- activestorage (6.1.7.2)
- actionpack (= 6.1.7.2)
- activejob (= 6.1.7.2)
- activerecord (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ activemodel (6.1.7.4)
+ activesupport (= 6.1.7.4)
+ activerecord (6.1.7.4)
+ activemodel (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
+ activestorage (6.1.7.4)
+ actionpack (= 6.1.7.4)
+ activejob (= 6.1.7.4)
+ activerecord (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  marcel (~> 1.0)
  mini_mime (>= 1.1.0)
- activesupport (6.1.7.2)
+ activesupport (6.1.7.4)
  concurrent-ruby (~> 1.0, >= 1.0.2)
  i18n (>= 1.6, < 2)
  minitest (>= 5.1)
@@ -173,7 +173,7 @@ GEM
  cocoon (1.2.15)
  coderay (1.1.3)
  color_diff (0.1)
- concurrent-ruby (1.2.0)
+ concurrent-ruby (1.2.2)
  connection_pool (2.3.0)
  cose (1.2.1)
  cbor (~> 0.5.9)
@@ -206,7 +206,7 @@ GEM
  docile (1.4.0)
  domain_name (0.5.20190701)
  unf (>= 0.0.5, < 1.0.0)
- doorkeeper (5.6.4)
+ doorkeeper (5.6.6)
  railties (>= 5)
  dotenv (2.8.1)
  dotenv-rails (2.8.1)
@@ -388,7 +388,7 @@ GEM
  loofah (2.19.1)
  crass (~> 1.0.2)
  nokogiri (>= 1.5.9)
- mail (2.8.0.1)
+ mail (2.8.1)
  mini_mime (>= 0.1.1)
  net-imap
  net-pop
@@ -405,12 +405,12 @@ GEM
  mime-types-data (~> 3.2015)
  mime-types-data (3.2022.0105)
  mini_mime (1.1.2)
- mini_portile2 (2.8.1)
+ mini_portile2 (2.8.2)
  minitest (5.17.0)
  msgpack (1.6.0)
  multi_json (1.15.0)
  multipart-post (2.1.1)
- net-imap (0.3.4)
+ net-imap (0.3.6)
  date
  net-protocol
  net-ldap (0.17.1)
@@ -423,8 +423,8 @@ GEM
  net-smtp (0.3.3)
  net-protocol
  net-ssh (7.0.1)
- nio4r (2.5.8)
- nokogiri (1.14.1)
+ nio4r (2.5.9)
+ nokogiri (1.14.5)
  mini_portile2 (~> 2.8.0)
  racc (~> 1.4)
  nsa (0.2.8)
@@ -497,7 +497,7 @@ GEM
  activesupport (>= 3.0.0)
  raabro (1.4.0)
  racc (1.6.2)
- rack (2.2.6.2)
+ rack (2.2.7)
  rack-attack (6.6.1)
  rack (>= 1.0, < 3)
  rack-cors (1.1.1)
@@ -512,20 +512,20 @@ GEM
  rack
  rack-test (2.0.2)
  rack (>= 1.3)
- rails (6.1.7.2)
- actioncable (= 6.1.7.2)
- actionmailbox (= 6.1.7.2)
- actionmailer (= 6.1.7.2)
- actionpack (= 6.1.7.2)
- actiontext (= 6.1.7.2)
- actionview (= 6.1.7.2)
- activejob (= 6.1.7.2)
- activemodel (= 6.1.7.2)
- activerecord (= 6.1.7.2)
- activestorage (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ rails (6.1.7.4)
+ actioncable (= 6.1.7.4)
+ actionmailbox (= 6.1.7.4)
+ actionmailer (= 6.1.7.4)
+ actionpack (= 6.1.7.4)
+ actiontext (= 6.1.7.4)
+ actionview (= 6.1.7.4)
+ activejob (= 6.1.7.4)
+ activemodel (= 6.1.7.4)
+ activerecord (= 6.1.7.4)
+ activestorage (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  bundler (>= 1.15.0)
- railties (= 6.1.7.2)
+ railties (= 6.1.7.4)
  sprockets-rails (>= 2.0.0)
  rails-controller-testing (1.0.5)
  actionpack (>= 5.0.1.rc1)
@@ -541,9 +541,9 @@ GEM
  railties (>= 6.0.0, < 7)
  rails-settings-cached (0.6.6)
  rails (>= 4.2.0)
- railties (6.1.7.2)
- actionpack (= 6.1.7.2)
- activesupport (= 6.1.7.2)
+ railties (6.1.7.4)
+ actionpack (= 6.1.7.4)
+ activesupport (= 6.1.7.4)
  method_source
  rake (>= 12.2)
  thor (~> 1.0)
@@ -688,9 +688,9 @@ GEM
  unicode-display_width (>= 1.1.1, < 3)
  terrapin (0.6.0)
  climate_control (>= 0.0.3, < 1.0)
- thor (1.2.1)
+ thor (1.2.2)
  tilt (2.0.11)
- timeout (0.3.1)
+ timeout (0.3.2)
  tpm-key_attestation (0.11.0)
  bindata (~> 2.4)
  openssl (> 2.0, < 3.1)
@@ -753,7 +753,7 @@ GEM
  xorcist (1.1.3)
  xpath (3.2.0)
  nokogiri (~> 1.8)
- zeitwerk (2.6.6)
+ zeitwerk (2.6.8)
 
 PLATFORMS
  ruby

app/controllers/admin/webhooks_controller.rb

@@ -20,6 +20,7 @@ def create
  authorize :webhook, :create?
 
  @webhook = Webhook.new(resource_params)
+ @webhook.current_account = current_account
 
  if @webhook.save
  redirect_to admin_webhook_path(@webhook)
@@ -39,10 +40,12 @@ def edit
  def update
  authorize @webhook, :update?
 
+ @webhook.current_account = current_account
+
  if @webhook.update(resource_params)
  redirect_to admin_webhook_path(@webhook)
  else
- render :show
+ render :edit
  end
  end
 

app/controllers/api/v1/conversations_controller.rb

@@ -11,7 +11,7 @@ class Api::V1::ConversationsController < Api::BaseController
 
  def index
  @conversations = paginated_conversations
- render json: @conversations, each_serializer: REST::ConversationSerializer
+ render json: @conversations, each_serializer: REST::ConversationSerializer, relationships: StatusRelationshipsPresenter.new(@conversations.map(&:last_status), current_user&.account_id)
  end
 
  def read
@@ -32,6 +32,19 @@ def set_conversation
 
  def paginated_conversations
  AccountConversation.where(account: current_account)
+ .includes(
+ account: :account_stat,
+ last_status: [
+ :media_attachments,
+ :preview_cards,
+ :status_stat,
+ :tags,
+ {
+ active_mentions: [account: :account_stat],
+ account: :account_stat,
+ },
+ ]
+ )
  .to_a_paginated_by_id(limit_param(LIMIT), params_slice(:max_id, :since_id, :min_id))
  end
 

app/controllers/api/v1/statuses/histories_controller.rb

@@ -7,11 +7,15 @@ class Api::V1::Statuses::HistoriesController < Api::BaseController
  before_action :set_status
 
  def show
- render json: @status.edits.includes(:account, status: [:account]), each_serializer: REST::StatusEditSerializer
+ render json: status_edits, each_serializer: REST::StatusEditSerializer
  end
 
  private
 
+ def status_edits
+ @status.edits.includes(:account, status: [:account]).to_a.presence || [@status.build_snapshot(at_time: @status.edited_at || @status.created_at)]
+ end
+
  def set_status
  @status = Status.find(params[:status_id])
  authorize @status, :show?

app/controllers/api/v1/statuses/reblogs_controller.rb

@@ -2,6 +2,8 @@
 
 class Api::V1::Statuses::ReblogsController < Api::BaseController
  include Authorization
+ include Redisable
+ include Lockable
 
  before_action -> { doorkeeper_authorize! :write, :'write:statuses' }
  before_action :require_user!
@@ -10,7 +12,9 @@ class Api::V1::Statuses::ReblogsController < Api::BaseController
  override_rate_limit_headers :create, family: :statuses
 
  def create
- @status = ReblogService.new.call(current_account, @reblog, reblog_params)
+ with_lock("reblog:#{current_account.id}:#{@reblog.id}") do
+ @status = ReblogService.new.call(current_account, @reblog, reblog_params)
+ end
 
  render json: @status, serializer: REST::StatusSerializer
  end

app/controllers/api/v2/admin/accounts_controller.rb

@@ -18,6 +18,14 @@ class Api::V2::Admin::AccountsController < Api::V1::Admin::AccountsController
 
  private
 
+ def next_path
+ api_v2_admin_accounts_url(pagination_params(max_id: pagination_max_id)) if records_continue?
+ end
+
+ def prev_path
+ api_v2_admin_accounts_url(pagination_params(min_id: pagination_since_id)) unless @accounts.empty?
+ end
+
  def filtered_accounts
  AccountFilter.new(translated_filter_params).results
  end

app/controllers/backups_controller.rb

@@ -13,7 +13,7 @@ def download
  when :s3
  redirect_to @backup.dump.expiring_url(10)
  when :fog
- if Paperclip::Attachment.default_options.dig(:storage, :fog_credentials, :openstack_temp_url_key).present?
+ if Paperclip::Attachment.default_options.dig(:fog_credentials, :openstack_temp_url_key).present?
  redirect_to @backup.dump.expiring_url(Time.now.utc + 10)
  else
  redirect_to full_asset_url(@backup.dump.url)