SQL Parsing

[ieaves]

Many existing integrations like dbt extract SQL transformations encoded in the edges between nodes. For example,

Column A -- (SQL 1) --> Column B -- (SQL 2) --> Column C

SQL parsing would allow us to statically analyze that pipeline of transformations and incorporate those results into the existing test suite. It might also be used to infer lineage based on query log history in various databases.

[jordandakota]

SQLGlot is the defacto pythonic method of parsing SQL. Works for most systems now. The guys over at Tobiko are doing great work in this area.

https://github.com/tobymao/sqlglot

They also use it in their SQLMesh product which has column level lineage that utilizes the ast from SQLGlot.