Avoid unnecessary CMS DB hits for pages that will 404

[stevejalim]

This changeset adds a lookahead check before we call wagtail.views.serve to know whether it's worth asking the CMS to serve a page.

The idea is that if the page isn't in the lookahead's data source, we can avoid touching the DB just to ultimately return a 404 earlier.

This will save us DB load, particularly when we get drive-by scans that pepper the site with irrelevant URLs.

• I used an AI to write some of this code - I bounced some ideas around with ChatGPT, and reworked some of the code suggestions to fit with Bedrock

Significant changes and points to review

Please be sceptical about this, particularly around cache invalidation/cache updating - e.g. when a page is published, unpublished or moved in the page tree.

Important: this change will need to go hand in hand with an infra update that does give us a networked cache (Redis or Memcached) - if we stick with LocMemCache, then while the pods can and will build their own lookahead in local cache, that cache will not be invalidated when a page is published, unpublished or moved.

Issue / Bugzilla link

#15505

Testing

Details to come

[codecov]

Codecov Report

Attention: Patch coverage is 83.89831% with 19 lines in your changes missing coverage. Please review.

Project coverage is 79.34%. Comparing base (25b38f7) to head (131d341).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
bedrock/cms/signal_handlers.py	51.35%	18 Missing ⚠️
bedrock/cms/apps.py	87.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #15628      +/-   ##
==========================================
+ Coverage   79.28%   79.34%   +0.06%     
==========================================
  Files         159      161       +2     
  Lines        8343     8460     +117     
==========================================
+ Hits         6615     6713      +98     
- Misses       1728     1747      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[stevejalim]

Talking with Pmac, we're gonna try usign postgres as a DB-backed networked cache, possibly/probably with a locmem cache on each pod.