Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Commit

Permalink
[NSE-635] Add document to clarify incompatibility issues in expressio…
Browse files Browse the repository at this point in the history
…ns (#657)

* Initial commit

* Clarify incompatibility cases in casting string to int/bigint/float4/float8

* Update the table

* Rename the doc

* Update the doc
  • Loading branch information
PHILO-HE authored and zhouyuan committed Jan 12, 2022
1 parent 434f168 commit 42780a3
Showing 1 changed file with 16 additions and 0 deletions.
16 changes: 16 additions & 0 deletions docs/Expressions-Compatibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
## Gazelle Expression Compatibility with Apache Spark

There are some cases that Gazelle behaves differently from Apache Spark. Here, we list the compatibility issues we have not addressed so far.


| No. | Expression | Incompatibility |
| --- | ----------------------------------------|---------------------------------------------------------------------|
| 1 | all expressions | Incompatibility issue when ANSI is on (throw exceptions at runtime instead of return null). |
| 2 | get_json_object | Single quote mark is not supported, but only support regular double quote mark. <br/>|
| | | If multiple same keys are contained, null will be returned. But vanilla spark returns the value for the firstly emerged key.|
| 3 | from_unixtime | Specifying timezone is not supported. By default, return date for <br/>UTC, not for local timezone like vanilla spark. | |
| 4 | date/time related expressions | Incompatible behaviors for different LEGACY_TIME_PARSER_POLICY <br/>(corrected, exception, legacy). |
| 5 | expressions with date format provided. | Parsing user-specified date format is not well supported. |
| 6 | castINT/castBIGINT/castFLOAT4/castFLOAT8| Return digital part leading in strings like "123abc" in WSCG, but vanilla spark return null. |


0 comments on commit 42780a3

Please sign in to comment.