[#3045] Add option to enable/disable indexing by search engines

[pi-sigma]

Taiga: https://taiga.maykinmedia.nl/project/open-inwoner/us/3011?milestone=523

[swrichards]

LGTM, but two points:

1. We also have a base_pdf.html which might also avail itself of the flag.
2. We also have a bunch of error templates (403.html, etc). While we're doing this, it would make sense to me to simply add a static noindex to those (search engines should never pick those up, regardless of the environment).

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.20%. Comparing base (ad1cb55) to head (e0d3fa3).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1623   +/-   ##
========================================
  Coverage    94.20%   94.20%           
========================================
  Files         1081     1083    +2     
  Lines        39958    39985   +27     
========================================
+ Hits         37642    37669   +27     
  Misses        2316     2316           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.