Improve account counters handling

[ClearlyClaire]

This is kind of a proof of concept for now, using upserts to avoid the convoluted and broken ruby-based handling of race conditions.

Before this change, Mastodon would load and instantiate the AccountStat object associated with an Account whenever changing any of the account stats, resulting in at least 2 SQL queries with a Ruby round-trip in the best-case scenario, and handling conflicts by doing more SQL queries and Ruby round-trips. Furthermore, this conflict resolution mechanism was broken in Rails 6.

This code ensures that updating the counters always take exactly one SQL query, plus a second in the rare case where account_stats was already loaded (and thus needs reloading). It also fixes these methods on Rails 6.

• review lock_version-based locking
• update last_status_at during the upsert
• disallow dirty records
• rewrite the upsert query in a saner way (Rails 6 has an upsert method but it's not flexible enough, nor stock ARel either apparently)

[ClearlyClaire]

Not too happy with the raw SQL, but ActiveRecord isn't expressive enough. ARel doesn't seem to be either (although it could be extended for our purposes, but that would overcomplicate the whole thing quite a bit).

[Gargron]

Upsert requires postgres 9.5. Do we have anyone stuck on 9.4 still?

SQLi is not possible with the way the values are chosen but string evaluation is still a risky code practice

[ClearlyClaire]

Upsert requires postgres 9.5. Do we have anyone stuck on 9.4 still?

Very good point. I'd say that everyone should use 9.5 at least by now, but I'm pretty sure one could find people using older versions of Postgres.

Bumping the Postgres version requirement might be worth it, but I'll look into whether people are using 9.4, or if we've already bumped the requirement somehow (I doubt it, but maybe).

SQLi is not possible with the way the values are chosen but string evaluation is still a risky code practice

Yeah, though there is no way to write that query in plain ActiveRecord or even unpatched ARel (the closest I could find is https://rubygems.org/gems/active_record_upsert which I'm not even sure exactly covers the query here). In this case, I far prefer two easy-to-read SQL queries than convoluted use of ARel or confidential gems.

EDIT: I edited the code a bit to build part of the query with Rails' sanitize_sql method, but that leaves the #{key} interpolation, as only sanitize_sql_hash_for_assignment seem to come close, and it would require stitching bits of SQL in a way that isn't any easier to read and check safety of.

[ClearlyClaire]

As far as current requirements go, I think the current required minimum PostgreSQL version for Mastodon is 9.4, since Mastodon 3.3.0, with the inclusion of the scenic gem and the use of the concurrently option. It might have required 9.4 at least for a longer while, and it may require 9.5 but I don't think it actually does.

Regarding people still using PostgreSQL 9.4, I checked a few common distributions:

• the oldest supported Ubuntu release, Ubuntu 16.04 LTS (support ends next month), ships with PostgreSQL 9.5
• Debian stable has PostgreSQL 11, oldstable has PostgreSQL 9.6
• the oldest supported Fedora release, Fedora 32, ships with PostgreSQL 12

In addition to that, I made a quick Mastodon poll. So far, it seems that out of 10 voters, 1 is still using PostgreSQL 9.4 or older.

Please also note that both PostgreSQL 9.4 and 9.5 are currently EOL: https://www.postgresql.org/support/versioning/

I think bumping the required version to 9.5 at least is very reasonable, although it'd probably warrant a note in the release notes, and maybe a database migration hook or something to inform users running older PostgreSQL versions of it.

EDIT: Added a hook to abort before running db:migrate if running PostgreSQL < 9.5.