[CT-2644] [Feature] Remove dependency on sqlparse

[emmyoop]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Remove the dependency on sqlparse comp;letely.

Currently we only use sqlparse here. Remove the sqlparse dependency and replace the logic with our own manual version.

Describe alternatives you've considered

#7515

[lukehsiao]

While the sqlparse vuln is only considered moderate by GitHub, it is considered high by NIST: https://nvd.nist.gov/vuln/detail/CVE-2023-30608

Curious if there is any estimate on when this removal might be implemented?

[kovaacs]

Is this really a "feature?" As far as I'm concerned, this is a fix for a high vulnerability in a pinned dependency.

[jtcohen6]

@lukehsiao @kovaacs Heard! This is high priority for us, and we're going to try tackling in our upcoming sprint.

[bdashrad]

Any idea when could we expect a fix for this to go out? The high severity CVE-2023-30608 is fixed by it.

[jtcohen6]

After spiking what would be required for a full replacement (#7919), and also endeavoring to reproduce the error we were seeing (only in CI, only on Ubuntu) — we've decided that supporting sqlparse==0.4.4 is the right move for now (#7929).

As such, I'm going to close this issue in favor of reopening #7515. We'll be including that change in v1.6, and in the next patch release of v1.5.x.