sqlite: Add split module

[lann]

let stmts = split::split_sql("SELECT 1;SELECT * FROM my_table;").collect::<Result<Vec<_>>>()?;
assert_eq!(stmts, ["SELECT 1;", "SELECT * FROM my_table;"]);

[itowlson]

I thought it was only used by the libsql implementation, so would vote for having it in the libsql crate as a private implementation detail, but that's a nit, and easy to move after merge.

This isn't currently hooked up to the libsql code that requires splitting; is that because this was primarily about socialising and agreeing the approach rather than plumbing it in? (It's fine if it is, I or others can do the plumbing - just checking.)

[lann]

Yeah it was more of a proof of concept that got out of hand.

[itowlson]

I did a quick test with this to make sure it approved Karthik's vexing SQL CREATE virtual TABLE vss_blog_posts3 USING vss0(embedding(384)); (which was the motivation for all this in the first place) - I wanted to check in case libsql and rusqlite disagreed on parsing. The good news is it worked!

The less good news is that this seemed to struggle with parsing files where one statement depended on a previous one. It happened that my test file for Karthik's SQL included some of my old database CREATE/INSERT setup - and that failed. Minimal repro:

CREATE TABLE fiteme23 (num INT);
INSERT INTO fiteme23(num) VALUES(55);

failed with "no such table: fiteme23". Could the embedded connection be doing more than parsing and be actually validating each statement in the context of the connection itself? Although if that were the case I don't get why it would allow the virtual table SQL which rusqlite presumably wouldn't be able to process without extensions.

(for the unit tests, the happy entry is ("CREATE TABLE fiteme23 (num INT); INSERT INTO fiteme23(num) VALUES(55);", &["CREATE TABLE fiteme23 (num INT);", "INSERT INTO fiteme23(num) VALUES(55);"]))

[lann]

Ugh I meant to test that but lost track of it in my unit testing fervour. I have another way to tackle this that I can try tomorrow.

[lann]

OK, this version doesn't validate statements quite so vigorously.

[itowlson]

A couple of test cases I'd like to throw in (they pass, just to have them locked in), but otherwise LGTM. Thanks!