Language Server for Dataform?

[LeonAndrade]

Hello @magal1337! I also use dataform at work, and been working on such nvim extension myself when I stumbled across your project.

I've been thinking about implementing a simple Language Server to allow better syntax highlight and autocomplete.

What are your thoughts?

Just cloned your project to take a look and maybe contribute to join forces ;)

cheers!

[magal1337]

Hey @LeonAndrade how are you? Actually thats a really great idea! @andres-lowrie did some basic implementation (which helps a lot) of syntax highlight but will be amazing to have a more complete language server for dataform (and also bigquery!). Feel free to submit the PR. =)

[LeonAndrade]

Nice! yeah, i'd love to have a better way to hint completions for BQ functions and modules defined as includes/ and javascript writen inside sqlx files.

At first I thought about writing a tree-sitter grammar for dataform based on the simple lexer they have in the dataform-core source, but it would improve mostly just parsing a file to extract relevant context.

Since there are tree-sitter bindins for node, python, lua and go, that's when I began pondering about building an actual LS.

Dataform-core also has a simple LS they use to power VScode extension and the GCP console environment, but it's quite limited.

The main Idea would be to move most of the responsibility below to the LS:

• compiling and parsing the compiled graph
• validating on BQ and maybe a static validation for SQL syntax before calling BQ api.
• go to definition
• ast generation

And have in the neovim extension only things concerning the editor, such as:

• context aware highlighting
• integration with other plugins
• redering compiled result.
• setting the filetype for dataform assets (since .SQLX is a valid extension outside the context of dataform projects, i've been using a "dataform" custom filetype to differentiate between general SQLX and the specific use dataform makes of this extension).

I'm currently exploring this Ideas because I never wrote an LS or a tree-sitter grammar before, but seem viable.