Upgrade to sqlparser 0.55.0

[alamb]

Is your feature request related to a problem or challenge?

The new version of sqlparser has been released:

https://crates.io/crates/sqlparser/0.55.0

Datafusion is still on sqlparser 0.54.0:

datafusion/Cargo.toml

Line 164 in 2458693

sqlparser = { version = "0.54.0", features = ["visitor"] }

Describe the solution you'd like

I would like to upgrade datafusion to 0.55.0 of sqlparser

This one may be tricky due to the change in ObjectName

Describe alternatives you've considered

Here is an example showing how it was done last time

• chore(deps): Update sqlparser to 0.54.0 #14255

Additional context

No response

[jonahgao]

take

[PokIsemaine]

@alamb @jonahgao
I have already made some progress on this part. If you don't mind, I would like to finish it. I want to use this update to become more familiar with DataFusion's code, and I would greatly appreciate any guidance you can offer.

Here are some sqlparser Pull Requests that involve modifications to DataFusion, which you can use as references:

Changelog: https://github.com/apache/datafusion-sqlparser-rs/blob/main/changelog/0.55.0.md

OrderBy: apache/datafusion-sqlparser-rs#1724
SQLExpr::Case: apache/datafusion-sqlparser-rs#1733
ColumnDef Collation: apache/datafusion-sqlparser-rs#1731
DataType: apache/datafusion-sqlparser-rs#1739
EXECUTE: apache/datafusion-sqlparser-rs#1717
CreateTable: apache/datafusion-sqlparser-rs#1664
QualifiedWildcard apache/datafusion-sqlparser-rs#1680
TypedString: apache/datafusion-sqlparser-rs#1739
SelectFlavor: apache/datafusion-sqlparser-rs#1713
StartTransaction: apache/datafusion-sqlparser-rs#1718

[jonahgao]

Thank you @PokIsemaine for your work. Assigned this issue to you ❤️

[PokIsemaine]

I want to discuss JoinType:

JoinOperator

• Allow plain JOIN without turning it into INNER datafusion-sqlparser-rs#1692
• Differentiate LEFT JOIN from LEFT OUTER JOIN datafusion-sqlparser-rs#1726

Do we need to distinguish between Inner/Join, Left/LeftOuter, and Right/RightOuter in JoinType, like this?

pub enum JoinType {
    Inner,
    Join,
    Left,
    LeftOuter,
    Right,
    RightOuter,
    // other...
}

Since sqlparser now differentiates between INNER JOIN and JOIN, the parsed result no longer matches the original test cases. For example:

TestStatementWithDialect {
    sql: "
        SELECT
          j1_string,
          j2_string
        FROM
          (
            SELECT
              distinct j1_id,
              j1_string,
              j2_string
            from
              j1
              INNER join j2 ON j1.j1_id = j2.j2_id
            order by
              j1.j1_id desc
            limit
              10
          ) abc
        ORDER BY
          abc.j2_string",
    expected: r#"SELECT abc.j1_string, abc.j2_string FROM (SELECT DISTINCT j1.j1_id, j1.j1_string, j2.j2_string FROM j1 JOIN j2 ON (j1.j1_id = j2.j2_id) ORDER BY j1.j1_id DESC NULLS FIRST LIMIT 10) AS abc ORDER BY abc.j2_string ASC NULLS LAST"#,
    parser_dialect: Box::new(GenericDialect {}),
    unparser_dialect: Box::new(UnparserDefaultDialect {}),
},

Should the JOIN in expected be changed to INNER JOIN?

If we make this distinction, it will affect printing and the unparser, which will indirectly impact many test cases. Do you have any good suggestions?

[jonahgao]

Do we need to distinguish between Inner/Join, Left/LeftOuter, and Right/RightOuter in JoinType, like this?

I think we don't need to distinguish them as they are equivalent. Introducing new join type variants would result in more significant changes.