Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlite functions json and jsonb #4388

Merged
merged 6 commits into from
Dec 13, 2024

Conversation

xuehaonan27
Copy link
Contributor

@xuehaonan27 xuehaonan27 commented Dec 11, 2024

Add SQLite functions json and jsonb according to issue #4366 .

Following the implementation of PostgreSQL bindings, module declaration statements were added and modified in files such as diesel/src/sqlite/mod.rs and diesel/src/sqlite/expression/mod.rs to facilitate easier coding and testing for future contributors.

NOTE: Testing on SQLite 3.47.2 with the libsqlite-sys 0.30.1 dependency failed due to the error: no such function: jsonb. Since SQLite 3.47.2 already supports JSONB, the issue lies within the SQLite binding crate. To resolve this, I updated the libsqlite3-sys dependency of diesel by switching the feature from bundled_bindings to bundled.

`bundled-bindings` to `bundled` to support jsonb.
@xuehaonan27 xuehaonan27 force-pushed the add_sqlite_json_functions branch from bc47130 to 287c9f6 Compare December 12, 2024 06:56
@xuehaonan27
Copy link
Contributor Author

Branch updated by force pushing: libsqlite3-sys feature bundled-bindings updated to bundled to support SQLite JSONB. No longer using rusqlite to keep dependency tree clean.

Copy link
Member

@weiznich weiznich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. This is really welcome ❤️

I've left a bunch of comments on details I would like to see solved differently to ensure that this works similar to other parts of diesel and does not break existing diesel users.

diesel/src/sqlite/expression/functions.rs Show resolved Hide resolved
/// # Ok(())
/// # }
/// ```
fn json<E: JsonOrNullableJsonOrJsonbOrNullableJsonb + SingleValue + MaybeNullableValue<Text>>(e: E) -> E::Out;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the correct signature here would be fn json<E: TextOrNullableText + MaybeNullableValue<Json>>(e: E) -> E::Out; as this function essentially turns text into json, not the other way around.

/// # Ok(())
/// # }
/// ```
fn jsonb<E: JsonOrNullableJsonOrJsonbOrNullableJsonb + SingleValue + MaybeNullableValue<Jsonb>>(e: E) -> E::Out;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the correct signature here would be fn json<E: BinaryOrNullableBinary + MaybeNullableValue<Json>>(e: E) -> E::Out; as this function essentially turns a blob into json, not the other way around.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you want to say fn **jsonb**<E: BinaryOrNullableBinary + MaybeNullableValue<**Jsonb**>>(e: E) -> E::Out; ? I tried out and found MaybeNullableValue<Jsonb> works but MaybeNullableValue<Json> not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that should be jsonb and not json. I just copied the other commend and failed to adjust that part, sorry for this.

@@ -24,7 +24,7 @@ include = [
byteorder = { version = "1.0", optional = true }
chrono = { version = "0.4.20", optional = true, default-features = false, features = ["clock", "std"] }
libc = { version = "0.2.0", optional = true }
libsqlite3-sys = { version = ">=0.17.2, <0.31.0", optional = true, features = ["bundled_bindings"] }
libsqlite3-sys = { version = ">=0.17.2, <0.31.0", optional = true, features = ["bundled"] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not to enable the bundled feature here as that breaks anyone that requires dynamically linking libsqlite3.

I would rather prefer to have a SELECT sqlite_version(); query in the doc tests and just skip the test if the local version is too old.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, this has nothing to do with the version of sqlite3, but with the feature switch of the libsqlite3-sys crate that diesel depends on. If we continue to use the bundled_bindings feature instead of bundled, even if the sqlite3 version is higher than 3.45.0 (mine is 3.47.2), it will still report an error of no such function: jsonb. This means that we cannot test jsonb at all, and (possibly) all JSONB-related functions mentioned in that issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have raised an issue under rusqlite (the repository where libsqlite3-sys is located) and received a relevant answer. rusqlite/rusqlite#1614

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested that locally and I believe it really just depends on which libsqlite3 (system library, not rust crate) version is linked. If that version is newer than 3.45 it works, if it's older it doesn't work. At least that's the case with my local 3.46 version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you are right, I rechecked my libsqlite and found that it was linked to the system library provided by MacOS instead of the library provided by homebrew. 😭

@xuehaonan27
Copy link
Contributor Author

Change made

@xuehaonan27 xuehaonan27 requested a review from weiznich December 13, 2024 06:03
Copy link
Member

@weiznich weiznich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing

diesel/src/sqlite/expression/expression_methods.rs Outdated Show resolved Hide resolved
@weiznich weiznich enabled auto-merge December 13, 2024 08:09
@weiznich weiznich added this pull request to the merge queue Dec 13, 2024
Merged via the queue into diesel-rs:master with commit b705023 Dec 13, 2024
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants