Scatterplots render incorrect data since 24.04.0-dev-b8828206596.40

[BowlesCR]

Issue Summary

Scatter plots are rendering incorrect data. Last working version is 24.04.0-dev-b8815742641.39, first broken version 24.04.0-dev-b8828206596.40 and still broken as of 24.12.0-dev. The tabular data returned by the query looks reasonable even when the chart is broken.

I'm unclear what the hash in the image tag correlates to, but based on timing it looks like the issue was introduced with the merge of #6908, but the fix merged in #7022 for the 24.07.0-dev snapshot/image doesn't seem to fix this case.

Curiously, the user who spotted this issue reports that swapping the X and Y axis results in a sane plot (although not as intuitive to read).

Expected plot, as rendered by 24.04.0-dev-b8815742641.39:

Broken plot, as rendered by 24.12.0-dev:

Technical details:

• Redash Version: 24.04.0-dev-b8828206596.40 thru at least 24.12.0-dev
• Browser/OS: Firefox 133 on Fedora 41 for sure, also manifests for others likely using Chrome on Windows.
• How did you install Redash: Helm install of contrib-helm-chart version 3.1.0-alpha9, with image.tag: 24.04.0-dev-b8828206596.40

[eradman]

@BowlesCR can you show the visualization editor? Also what the sample data points look like

I tried reproducing this and seemed to get the same results using 24.04.0-dev and 24.11.0-dev

Scan_Times_2024_12_13.csv

[BowlesCR]

Visualization editor:

I guessed this is what you meant by sample data points:

As you can see, the max point is only ~103, so chart points rendering in the thousands is quite off.

[eradman]

I see, the difference is that after #6908 values (in this case on the Y axis) are summed. We need to figure out if this behavior should be considered to be an improvement, but it certainly is incompatible with previous queries.

[BowlesCR]

Summation probably makes sense for anything that isn't a scatterplot -- multiple Y values make no sense at all on a bar chart for example... but its perfectly valid for scatters to have multiple datapoints for a give X value.

Perhaps condition the summation logic off for scatterplots? Add a toggle?

[justinclift]

multiple Y values make no sense at all on a bar chart for example

Maybe I'm misunderstanding what you mean, but that doesn't sound correct. It's entirely possible (and useful) to have Y values on a bar chart stack on top of each other.

For example, the 1st and 3rd charts on this page: https://sqlitebrowser.org/stats/

I've seen other charts where the Y values are labelled and coloured differently (ie "foo" is blue, "bar" is red) and each Y column shows both next to each other

[BowlesCR]

Sorry, yes. Stacked bar very much a thing, but to my understanding that's using multiple series.

The weirdness here is what to do with (X, Y) datapoints in a single series where X is nonunique. Requiring uniqueness, summing them (as it is now), or displaying on the highest (old behavior) are all probably reasonable approaches for bar charts and the like where you can't really plot multiple values for a single X.