Postgres Queries: non deterministic: Relation does not exist

[flobaader]

Issue Summary

We host Redash with docker compose and run a simple Postgres Docker container on the same host. When we run the queries, Redash returns the postgres error "relation xy does not exist" for at least two-thirds of all queries. The connection check in the source settings always works.
The behavior did not occur with Redash 5.

Steps to Reproduce

1. Run redash server and workers in default network and a "redash" network
2. Conntect to postgresql in the "redash" network

Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead?

Technical details:

• Redash Version: 7.0.0.b18042
• Browser/OS: All browsers and OS
• How did you install Redash: Docker-Compose

[justinclift]

Did you get this figured out?

That behaviour kind of sounds like there's more than one PostgreSQL instance operating in the system, and *somehow* queries are being distributed across them instead of going to the intended one.

Don't suppose you have Docker Swarm, kubernetes, or some other layer of container orchestration running? Just thinking there may be some unintended interaction happening. (guessing wildly!)

[flobaader]

Hello justin,

no, I did not find a solution to this strange issue. There are multiple postgres instances on the system and I just use plain docker networking, but the hostname should be unique. I can't really debug this issue, since redash only throws the postgres error but does not show information about the connected postgres instance.

[justinclift]

Yeah, it sounds like a weird one. For a potential way to diagnose if it really is Redash going to the wrong PG instance... maybe turn on SQL query logging in the Redash PostgreSQL?

With that turned on, new entries should be added to the SQL query log each time Redash connects to that database. If there are occasions when the Redash query fails, but no entry is appearing in the log, that would probably mean it's another PG instance receiving it.

That's a first thought anyway. 😄

[flobaader]

You were totally right. There was an old container dangling in the network, which had the same name in the network despite the actual service name has been changed. Then the requests were load-balanced between those two instances. Thanks for your work and have a nice day!

[justinclift]

Awesome, glad you got it sorted. 😄