Add support for Float64 for Clickhouse (SaaS)

[ilongin]

The problem is that if number that can only be represented with double precision, e.g 0.3813476562 is inserted using CH database (in SaaS), it's actually inserted as single precision (Float32) as python float is saved as Float32 column type in database by default. This way precision of that value is lost for user.

One solution is to support datachain.sql.types when defining output schema (currently only python basic values are supported together with Feature type).

Example that works for SQLite but fails for Clickhouse:

import pandas as pd
from datachain.lib.dc import DataChain

num = 0.3813476562   # number that requires 64 bits to not loose precision
chain = DataChain.from_pandas(pd.DataFrame([[num]], columns=['number']))
assert chain.collect_one("number")[0] == num

[skshetry]

@ilongin, would it be possible to change the test assumptions to reduce precision (eg: with pytest.approx)?

[ilongin]

I will fix the tests for now (using 0.5 value everywhere for float) but I also added TODO with link on this issue to change values to something more realistic when this is fixed
#13

[skshetry]

pinging @dmpetrov for changing default size of float.

[dmpetrov]

It looks like we have to move to float64 by default since it's python default type. Related to iterative/dvcx#1697

[dmpetrov]

assert chain.collect_one("number")[0] == num

That's not a usualy way of comparing floats. Please use pytest.approx(x) or asb(val - target) < 1e-6

Otherwise, moving to float64 makes sense. We should do that everywhere.

[ilongin]

assert chain.collect_one("number")[0] == num

That's not a usualy way of comparing floats. Please use pytest.approx(x) or asb(val - target) < 1e-6

Otherwise, moving to float64 makes sense. We should do that everywhere.

That was just an example to show that values are different between Clickhouse and SQLite so this is not a real test or something.

[dmpetrov]

so this is not a real test or something.

oh, now got it!

[dmpetrov]

@ilongin is it still relevant or we can close?
Based on my understanding we are working with float64 only rn. Other types are not supported yet.

[ilongin]

@dmpetrov yes, looks like example in description works now so I'm closing this issue.

[skshetry]

I don't think we have made any changes to this. @ilongin, can you point me the commit that fixed this please?

[ilongin]

@skshetry I made a mistake testing this on SQLite DB instead of Clickhouse. Reopening it agan.

[dreadatour]

Let's discuss this approach, please? https://github.com/iterative/studio/pull/10528 🙏👀

[dreadatour]

Should be fixed now by related Studio issue.