Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlite functions json and jsonb #4388

Merged
merged 6 commits into from
Dec 13, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion diesel/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ include = [
byteorder = { version = "1.0", optional = true }
chrono = { version = "0.4.20", optional = true, default-features = false, features = ["clock", "std"] }
libc = { version = "0.2.0", optional = true }
libsqlite3-sys = { version = ">=0.17.2, <0.31.0", optional = true, features = ["bundled_bindings"] }
libsqlite3-sys = { version = ">=0.17.2, <0.31.0", optional = true, features = ["bundled"] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not to enable the bundled feature here as that breaks anyone that requires dynamically linking libsqlite3.

I would rather prefer to have a SELECT sqlite_version(); query in the doc tests and just skip the test if the local version is too old.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, this has nothing to do with the version of sqlite3, but with the feature switch of the libsqlite3-sys crate that diesel depends on. If we continue to use the bundled_bindings feature instead of bundled, even if the sqlite3 version is higher than 3.45.0 (mine is 3.47.2), it will still report an error of no such function: jsonb. This means that we cannot test jsonb at all, and (possibly) all JSONB-related functions mentioned in that issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have raised an issue under rusqlite (the repository where libsqlite3-sys is located) and received a relevant answer. rusqlite/rusqlite#1614

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested that locally and I believe it really just depends on which libsqlite3 (system library, not rust crate) version is linked. If that version is newer than 3.45 it works, if it's older it doesn't work. At least that's the case with my local 3.46 version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you are right, I rechecked my libsqlite and found that it was linked to the system library provided by MacOS instead of the library provided by homebrew. 😭

mysqlclient-sys = { version = ">=0.2.5, <0.5.0", optional = true }
mysqlclient-src = { version = "0.1.0", optional = true }
pq-sys = { version = ">=0.4.0, <0.7.0", optional = true }
Expand Down
3 changes: 3 additions & 0 deletions diesel/src/expression/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ pub(crate) mod dsl {
#[cfg(feature = "postgres_backend")]
pub use crate::pg::expression::dsl::*;

#[cfg(feature = "sqlite")]
pub use crate::sqlite::expression::dsl::*;

/// The return type of [`count(expr)`](crate::dsl::count())
pub type count<Expr> = super::count::count<SqlTypeOf<Expr>, Expr>;

Expand Down
26 changes: 26 additions & 0 deletions diesel/src/sqlite/expression/expression_methods.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
//! Sqlite specific expression methods.

pub(in crate::sqlite) use self::private::{
JsonOrNullableJsonOrJsonbOrNullableJsonb, MaybeNullableValue,
};
use super::operators::*;
use crate::dsl;
use crate::expression::grouped::Grouped;
Expand Down Expand Up @@ -82,3 +85,26 @@ pub trait SqliteExpressionMethods: Expression + Sized {
}

impl<T: Expression> SqliteExpressionMethods for T {}

pub(in crate::sqlite) mod private {
use crate::sql_types::{Json, Jsonb, MaybeNullableType, Nullable, SingleValue};

pub trait JsonOrNullableJsonOrJsonbOrNullableJsonb {}
impl JsonOrNullableJsonOrJsonbOrNullableJsonb for Json {}
impl JsonOrNullableJsonOrJsonbOrNullableJsonb for Nullable<Json> {}
impl JsonOrNullableJsonOrJsonbOrNullableJsonb for Jsonb {}
impl JsonOrNullableJsonOrJsonbOrNullableJsonb for Nullable<Jsonb> {}

pub trait MaybeNullableValue<T>: SingleValue {
type Out: SingleValue;
}

impl<T, O> MaybeNullableValue<O> for T
where
T: SingleValue,
T::IsNull: MaybeNullableType<O>,
<T::IsNull as MaybeNullableType<O>>::Out: SingleValue,
{
type Out = <T::IsNull as MaybeNullableType<O>>::Out;
}
}
114 changes: 114 additions & 0 deletions diesel/src/sqlite/expression/functions.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
//! SQLite specific functions
use crate::expression::functions::define_sql_function;
use crate::sql_types::*;
use crate::sqlite::expression::expression_methods::JsonOrNullableJsonOrJsonbOrNullableJsonb;
use crate::sqlite::expression::expression_methods::MaybeNullableValue;

#[cfg(feature = "sqlite")]
define_sql_function! {
/// Verifies that its argument is a valid JSON string or JSONB blob and returns a minified
/// version of that JSON string with all unnecessary whitespace removed.
///
/// # Example
///
/// ```rust
/// # include!("../../doctest_setup.rs");
/// #
/// # fn main() {
/// # #[cfg(feature = "serde_json")]
/// # run_test().unwrap();
/// # }
/// #
/// # #[cfg(feature = "serde_json")]
/// # fn run_test() -> QueryResult<()> {
/// # use diesel::dsl::json;
/// # use serde_json::{json, Value};
/// # use diesel::sql_types::{Text, Json, Jsonb, Nullable};
/// # let connection = &mut establish_connection();
///
/// let result = diesel::select(json::<Json, _>(json!({"a": "b", "c": 1})))
/// .get_result::<String>(connection)?;
///
/// assert_eq!(r#"{"a":"b","c":1}"#, result);
///
/// let result = diesel::select(json::<Json, _>(json!({ "this" : "is", "a": [ "test" ] })))
/// .get_result::<String>(connection)?;
///
/// assert_eq!(r#"{"a":["test"],"this":"is"}"#, result);
///
/// let result = diesel::select(json::<Nullable<Json>, _>(None::<Value>))
/// .get_result::<Option<String>>(connection)?;
///
/// assert!(result.is_none());
///
/// let result = diesel::select(json::<Jsonb, _>(json!({"a": "b", "c": 1})))
/// .get_result::<String>(connection)?;
///
/// assert_eq!(r#"{"a":"b","c":1}"#, result);
///
/// let result = diesel::select(json::<Jsonb, _>(json!({ "this" : "is", "a": [ "test" ] })))
/// .get_result::<String>(connection)?;
///
/// assert_eq!(r#"{"a":["test"],"this":"is"}"#, result);
///
/// let result = diesel::select(json::<Nullable<Jsonb>, _>(None::<Value>))
/// .get_result::<Option<String>>(connection)?;
///
/// assert!(result.is_none());
///
///
/// # Ok(())
/// # }
/// ```
fn json<E: JsonOrNullableJsonOrJsonbOrNullableJsonb + SingleValue + MaybeNullableValue<Text>>(e: E) -> E::Out;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the correct signature here would be fn json<E: TextOrNullableText + MaybeNullableValue<Json>>(e: E) -> E::Out; as this function essentially turns text into json, not the other way around.

}

#[cfg(feature = "sqlite")]
define_sql_function! {
///
weiznich marked this conversation as resolved.
Show resolved Hide resolved
/// # Example
///
/// ```rust
/// # include!("../../doctest_setup.rs");
/// #
/// # fn main() {
/// # #[cfg(feature = "serde_json")]
/// # run_test().unwrap();
/// # }
/// #
/// # #[cfg(feature = "serde_json")]
/// # fn run_test() -> QueryResult<()> {
/// # use diesel::dsl::jsonb;
/// # use serde_json::{json, Value};
/// # use diesel::sql_types::{Text, Json, Jsonb, Nullable};
/// # let connection = &mut establish_connection();
///
/// let result = diesel::select(jsonb::<Json, _>(json!({"a": "b", "c": 1})))
/// .get_result::<Value>(connection)?;
///
/// assert_eq!(json!({"a": "b", "c": 1}), result);
/// println!("json abc1");
///
/// let result = diesel::select(jsonb::<Jsonb, _>(json!({"a": "b", "c": 1})))
/// .get_result::<Value>(connection)?;
///
/// assert_eq!(json!({"a": "b", "c": 1}), result);
/// println!("jsonb abc1");
///
/// let result = diesel::select(jsonb::<Nullable<Json>, _>(None::<Value>))
/// .get_result::<Option<Value>>(connection)?;
///
/// assert!(result.is_none());
/// println!("json null");
///
/// let result = diesel::select(jsonb::<Nullable<Jsonb>, _>(None::<Value>))
/// .get_result::<Option<Value>>(connection)?;
///
/// assert!(result.is_none());
/// println!("jsonb null");
///
/// # Ok(())
/// # }
/// ```
fn jsonb<E: JsonOrNullableJsonOrJsonbOrNullableJsonb + SingleValue + MaybeNullableValue<Jsonb>>(e: E) -> E::Out;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the correct signature here would be fn json<E: BinaryOrNullableBinary + MaybeNullableValue<Json>>(e: E) -> E::Out; as this function essentially turns a blob into json, not the other way around.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you want to say fn **jsonb**<E: BinaryOrNullableBinary + MaybeNullableValue<**Jsonb**>>(e: E) -> E::Out; ? I tried out and found MaybeNullableValue<Jsonb> works but MaybeNullableValue<Json> not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that should be jsonb and not json. I just copied the other commend and failed to adjust that part, sorry for this.

}
12 changes: 11 additions & 1 deletion diesel/src/sqlite/expression/helper_types.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,18 @@
use crate::dsl::AsExpr;
use crate::dsl::{AsExpr, SqlTypeOf};
use crate::expression::grouped::Grouped;

/// The return type of `lhs.is(rhs)`.
pub type Is<Lhs, Rhs> = Grouped<super::operators::Is<Lhs, AsExpr<Rhs, Lhs>>>;

/// The return type of `lhs.is_not(rhs)`.
pub type IsNot<Lhs, Rhs> = Grouped<super::operators::IsNot<Lhs, AsExpr<Rhs, Lhs>>>;

/// Return type of [`json(json)`](super::functions::json())
#[allow(non_camel_case_types)]
#[cfg(feature = "sqlite")]
pub type json<E> = super::functions::json<SqlTypeOf<E>, E>;

/// Return type of [`jsonb(json)`](super::functions::jsonb())
#[allow(non_camel_case_types)]
#[cfg(feature = "sqlite")]
pub type jsonb<E> = super::functions::jsonb<SqlTypeOf<E>, E>;
11 changes: 11 additions & 0 deletions diesel/src/sqlite/expression/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,16 @@
//! kept separate purely for documentation purposes.

pub(crate) mod expression_methods;
pub mod functions;
pub(crate) mod helper_types;
mod operators;

/// SQLite specific expression DSL methods.
///
/// This module will be glob imported by
/// [`diesel::dsl`](crate::dsl) when compiled with the `feature =
/// "postgres"` flag.
pub mod dsl {
#[doc(inline)]
pub use super::functions::*;
}
2 changes: 1 addition & 1 deletion diesel/src/sqlite/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

pub(crate) mod backend;
mod connection;
pub(crate) mod expression;
pub mod expression;

pub mod query_builder;

Expand Down
22 changes: 22 additions & 0 deletions diesel_derives/tests/auto_type.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ table! {
}
}

#[cfg(feature = "sqlite")]
table! {
sqlite_extras {
id -> Integer,
text -> Text,
json -> Json,
jsonb -> Jsonb,
}
}

joinable!(posts -> users(user_id));
joinable!(posts2 -> users(user_id));
joinable!(posts3 -> users(user_id));
Expand Down Expand Up @@ -465,6 +475,18 @@ fn postgres_functions() -> _ {
)
}

#[cfg(feature = "sqlite")]
#[auto_type]
fn sqlite_functions() -> _ {
use diesel::sqlite::expression::functions::{json, jsonb};
(
json(sqlite_extras::json),
json(sqlite_extras::jsonb),
jsonb(sqlite_extras::json),
jsonb(sqlite_extras::jsonb),
)
}

#[auto_type]
fn with_lifetime<'a>(name: &'a str) -> _ {
users::table.filter(users::name.eq(name))
Expand Down
Loading