Multiprocessing\Multithreading

[diegocorradini]

Hi all,

I have been facing the following problem without finding a solution since a while.
I am developing a Django webapp which starts a Luigi pipeline with let s say N workers.
Every worker of the pipeline run queries.

If I am running just 1 worker everything fine.
Otherwise, if I am running more workers, I get random database errors.
It seems that Django and Postgres have problem to run queries from different processes

Is it possible to spawn workers as threads?
Otherwise could you suggest a workaround?

Hope the problem is clear.

The framework is great but without run queries in parallel it becomes useless.

Many thanks.

[kwilcox]

Is this the same issue as #2782? Can you continue the discussion there?

[diegocorradini]

Yes sorry.
I will close this issue.