Add multiprocess support

[VladVolchkov]

relates to #15

[deepminimal]

@savar @mbarouski Hello! Can you check it please?

[savar]

@savar @mbarouski Hello! Cat you check it please?

Hey, I am not experienced in this multiprocess part of the prometheus client.
I am trying to read the doc but I am not sure that this Collector stuff can be done only once in the initializer as the example in https://prometheus.github.io/client_python/multiprocess/ is doing this every time the scrape is happening.

Also as far as I can tell the multiprocess would require a properly set PROMETHEUS_MULTIPROC_DIR environment variable to work. Maybe we need to test for this envrionment variable to be set and only if, then we start the multiprocess part, otherwise we stick to the existing setup?

Do you have a working example deployment where this is being used, so that we could have a better look onto it?

[savar]

It looks like it could work but we will lose the python_* and process_* as well as the *_created metrics. Also I would like to not use this when there is only a single process being used, specifically because we do not execute the CollectorRegistry+MultiProcessCollector on each scrape request but only once initially. The documentation states it has a risk, but I cannot say right now if this applies to us here.

I just cannot figure out how we could detect the multiprocess situation. Thumbor is not providing this information in the config it passes into the initializer and relying on the environment variable would only mean that people do not realize that they didn't configure it correctly. 🤔

[sboisson]

Maybe only use CollectorRegistry and MultiProcessCollector when PROMETHEUS_MULTIPROC_DIR env var is set ?

[savar]

Let's try it like that. I am still not 100% sure that it always works perfectly but it looks like it can. But without the environment variable it won't work, so not doing it without it.

I will change some more stuff and add some documentation when this is merged.