Environment variable for spiders needing proxy

[jpmckinney]

I hardcoded Montreal as using a proxy. Since the need for a proxy depends on the server, we can instead implement the list of spiders needing a proxy as an environment variable, and configure the custom settings on the base spider.

Would need to look into how/when Scrapy reads the custom_settings class attribute, as we’ll want subclasses to be able to add to it.

[jpmckinney]

Paraguay needs a proxy:

2024-09-26 12:05:02,592 WARNING [data_registry.process_manager:101] Irrecoverable error during task #9649(process) (Paraguay: Dirección Nacional de Contrataciones Públicas (DNCP) (63)): Collection is empty