Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Spotless SQL DBeaver formatter produces INVALID spark sql syntax with whitespaces added inside backticks and other places #2250

Open
6 tasks done
esobolievv opened this issue Sep 6, 2024 · 1 comment
Labels

Comments

@esobolievv
Copy link

esobolievv commented Sep 6, 2024

If you are submitting a bug, please include the following:

Hi Team,

We have a big side-effect of using DBeaver SQL formatter within spotless "as is".

When I run this formatter on valid Spark SQL code where column name is wrapped with backtick symbol, it puts whitespaces around column name making .sql valid INVALID for spark to run within
spark.sql("sql-query-from-file")

It's easy to reproduce. Just create a simple SQL file within columns given below with backticks (see: Expected example) and you'll see it's formatter to "Current behavior" that is not valid syntax.

Versions:

Gradle v8+. No matter what version, it doesn't work!
Spotless v.6.25.0. No matter what version, it doesn't work!
spotless {
        sql {
            dbeaver().configFile(rootProject.file(".dbeaver.properties"))

            target("<my-folder-with-sqls>/**/*.sql")
       }
}

.dbeaver.properties

sql.formatter.keyword.case=UPPER
sql.formatter.statement.delimiter=;
sql.formatter.indent.type=space
sql.formatter.indent.size=2

Current behavior (formatter gives not valid spark sql):

     ` MyCol.1 ` AS MY_COLUMN_1, -- whitespace inside backticks
     ` MyCol.2 ` AS MY_COLUMN_2 -- whitespace inside backticks

Expected (valid spark sql):

     `MyCol.1` AS MY_COLUMN_1, -- no whitespace inside backticks
     `MyCol.2` AS MY_COLUMN_2 -- no whitespace inside backticks

UPD:
Other problems with whitespace were discovered that makes it IMPOSSIBLE to use Spotless anymore :(

  • Adding spaces when we have columns wrapped with backtick `` (known and shown above)
  • Adding spaces when we have columns which start with number. e.g 3p_dummy -> 3 p_dummy
  • Adding spaces when accessing arrays in SQL. e.g arr[0] -> arr [ 0 ]
  • Adding spaces to some literal columns: e.g 0L as dummy -> 0 L as dummy

It seem like so popular Spark SQL syntax isn't supported here. Would appreciate it addressing asap or propose a possible workaround

  • summary of problem
  • Gradle or Maven version
  • spotless version
  • operating system and version
  • copy-paste your full Spotless configuration block(s), and a link to a public git repo that reproduces the problem if possible
  • copy-paste the full content of any console errors emitted by gradlew spotless[Apply/Check] --stacktrace

If you're just submitting a feature request or question, no need for the above.

@esobolievv esobolievv changed the title Spotless SQL dbeaver formatter produced NON valid syntax for query with backticks Spotless SQL dbeaver formatter produces INVALID sql syntax containing backticks Sep 6, 2024
@esobolievv esobolievv changed the title Spotless SQL dbeaver formatter produces INVALID sql syntax containing backticks Spotless SQL dbeaver formatter produces INVALID sql syntax with whitespaces inside backticks Sep 6, 2024
@esobolievv esobolievv changed the title Spotless SQL dbeaver formatter produces INVALID sql syntax with whitespaces inside backticks Spotless SQL dbeaver formatter produces INVALID spark sql syntax with whitespaces inside backticks Sep 6, 2024
@esobolievv esobolievv changed the title Spotless SQL dbeaver formatter produces INVALID spark sql syntax with whitespaces inside backticks Spotless SQL dbeaver formatter produces INVALID spark sql syntax with whitespaces added inside backticks Sep 6, 2024
@esobolievv esobolievv changed the title Spotless SQL dbeaver formatter produces INVALID spark sql syntax with whitespaces added inside backticks BUG: Spotless SQL dbeaver formatter produces INVALID spark sql syntax with whitespaces added inside backticks Sep 6, 2024
@nedtwigg nedtwigg added the bug label Sep 8, 2024
@esobolievv
Copy link
Author

esobolievv commented Sep 16, 2024

HI @nedtwigg thank you for marking as the BUG. I added "UPD" section and described that it's impossible for me to use Spotless SQL formatter anymore within Spark SQL syntax as I can't control how formatter puts a whitespace. It breaks SQL easily that is hard to track. Are there any possible workarounds?

@esobolievv esobolievv changed the title BUG: Spotless SQL dbeaver formatter produces INVALID spark sql syntax with whitespaces added inside backticks BUG: Spotless SQL DBeaver formatter produces INVALID spark sql syntax with whitespaces added inside backticks and other places Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants