Fix to allow integer division to be cast as double

[yyie]

I found a bug that generates an error when you try to divide to fields in metabase.

[Simba]AthenaJDBC An error has been thrown from the AWS Athena client. SYNTAX_ERROR: line 2:71: Unknown type: float [Execution ID: 2088bd6d-2f44-4b6c-8088-1f76c47b715d]

This error is generated because in the following pull request Metabase adds a Cast to Float to every division. However Athena doesn't support Float. It should be cast as double.

metabase/metabase#11116

[yyie]

Hi, I report the same situation to AWS and this is their answer:

---

I have tested CAST to FLOAT and faced the same issue as well using the below query:

SELECT CAST(1000 AS FLOAT);

As you might already know, the Athena query engine is based on Presto. FLOAT data type is not supported according to Presto documentation[1]. But, it says FLOAT data type is supported in Athena documentation[2]. That is a documentation issue. We will update the documentation. Thanks for bringing up this issue.

Presto supports REAL and DOUBLE types for float-point values and you can use REAL type to cast as you mentioned.

We apologize for any inconvenience caused by using AWS. If you have any further questions, please feel free to reach out to me.

References:
[1] https://prestodb.io/docs/current/language/types.html
[2] https://docs.aws.amazon.com/athena/latest/ug/data-types.html

[dacort]

@yyie I'd like to add a test for this if possible. Can you tell me what the original data types were of the fields on the Athena side?

I'll likely add something to https://github.com/dacort/metabase-athena-driver/blob/master/test/metabase/driver/athena_test.clj that replicates the Metabase behavior.

[yyie]

In my database both fields are double. However Metabase always add the Cast to Float any division. That's why it's necessary to include the fix in the driver

[koe-n]

Any idea when this fix will be released? I'm looking forward to it as I am experiencing this behaviour as well.

[dacort]

Apologies for the delays here folks - I was out due to a job transition. I'm prepping a release for 0.38 so hope to be able to get this in.

[haboshi]

I ran into the same problem.
I'm looking forward to the release.
Thank you

[dacort]

I tested this locally and, while I'd still like to add a test for it, I'm opting to get this into today's release as it's a pretty important but straightforward fix.

Thanks @yyie for contributing this!!

[koe-n]

Unfortunately, I am still receiving this error after upgrading to the newest version of the driver. Anybody else who is also still experiencing this?

[Simba]AthenaJDBC An error has been thrown from the AWS Athena client. SYNTAX_ERROR: line 2:177: Unknown type: float [Execution ID: b7d73507-aa44-4d44-8f68-03eb5f76ad9f]

[dacort]

@koe-n Can you provide some more details on what is breaking? A step-by-step reproduction would be great or the following information:

1. What does your schema look like?
2. What is the query you're sending to Athena and/or what steps are you taking in Metabase to get the error?

[raps83]

I'm also experiencing this error. Updated to the latest versions of Metabase & this Athena-plugin but still the same error. In my dataset I'm trying a division of two doubles. My steps are:
Ask a question > Simple question > Select the table > when the table is loaded, go to: Show editor > Add custom column > add in the calculation fieldA / fieldB and click on 'Preview results' or go to visualization.., both give the same error.

[Simba]AthenaJDBC An error has been thrown from the AWS Athena client. SYNTAX_ERROR: line 2:11288: Unknown type: float [Execution ID: 8f7feabf-0cc2-470a-a32b-334bdcfef72a]

[silberistgold]

We are also still experiencing the same error.

[dacort]

Thanks @silberistgold and @raps83 - I'll see if I can try to create a fix for this in the next week or two.

I don't use Metabase daily anymore (but I should 😁) so it takes me a while to spin up an instance to repro. :)

[raps83]

@dacort had you already a change to take a look at this? We'd really like to add some insightful KPI's to our reports. Thank you very much in advance! 🙏

Does anyone perhaps know a workaround for the time being..?

[dacort]

I'm looking into this today. Thanks @raps83 for the repro, trying to validate now with the latest version of Metabase.

[dacort]

Repro steps:

1. Create a new table with some sample float data (note that float is used in the DDL, but real is used in the SQL.)

CREATE TABLE mb_floats (
  id int, 
  float1 float,
  float2 float
)
LOCATION 's3://<BUCKET>/metabase-athena/tests/mb_floats/'

INSERT INTO mb_floats
VALUES (1, CAST(2.0 AS REAL), CAST(3.0 AS REAL)), (2, CAST(3.14 AS REAL), CAST(5.159 AS REAL))

2. Add a new custom column

Ask a question > Simple question > Select the table > when the table is loaded, go to: Show editor > Add custom column > add in the calculation fieldA / fieldB and click on 'Preview results' or go to visualization..,

3. Receive a failure.

This is the generated SQL query:

SELECT *
FROM (-- Metabase:: userID: 1 queryType: MBQL queryHash: ec46f87b7c97b6fd1499fd15561a03580eeb5c53d1b51d61f2a8abdc1f96e9f6
SELECT "source"."id" AS "id",
         "source"."float1" AS "float1",
         "source"."float2" AS "float2",
         "source"."Float3" AS "Float3"
FROM 
    (SELECT "default"."mb_floats"."float1" AS "float1",
         "default"."mb_floats"."float2" AS "float2",
         "default"."mb_floats"."id" AS "id",
         (CAST("default"."mb_floats"."float1" AS float) /
        CASE
        WHEN "default"."mb_floats"."float2" = 0 THEN
        NULL
        ELSE "default"."mb_floats"."float2" END) AS "Float3"
    FROM "default"."mb_floats") "source" LIMIT 2000) T LIMIT 0

Per Athena documentation, I think the problem is that the generated SQL is using floats. :\

In Athena, use FLOAT in DDL statements like CREATE TABLE and REAL in SQL functions like SELECT CAST.

[dacort]

Apologies, all, I think this was user error. 🤦

It doesn't look like the 1.20 release actually included the fix. Please give v1.2.1 a try.

[raps83]

Yes, it works now! (tested Metabase with 0.39.3)

Thank you very much! 🙏