Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move JS objects to Rust JSValues without deserialization #21

Open
ghost opened this issue Aug 12, 2019 · 5 comments
Open

Move JS objects to Rust JSValues without deserialization #21

ghost opened this issue Aug 12, 2019 · 5 comments

Comments

@ghost
Copy link

ghost commented Aug 12, 2019

I'm wondering is it possible to store JavaScript objects as Rust values without deserialization, so that objects with richer structure can be passed back to call_function. For example, this code

let val = context.eval("new Date()").unwrap();                                                            
context.eval("display = x => x.toString()").unwrap();                                                    
let result = context.call_function("display", vec![val]).unwrap();                                       
println!("result: {:?}", result);                                                                         

outputs now result: String("[object Object]"), but I would like it to output result: String("Mon Aug 12 2019 15:55:15 GMT+0000").

@theduke
Copy link
Owner

theduke commented Aug 12, 2019

This is actually how it works internally.
If you check bindings::ContextWrapper, eval() and call_function return a wrapper around the original quickjs Value (OwnedValueRef) and the serialization is done in the higher level Context.

I considered making this publicly accessible but decided against it because I wanted a simple API without footguns and complexity.

What's your use case for this?
I'd suggest wrapping this in a simple function that does some serialization on the JS side if required.

@ghost
Copy link
Author

ghost commented Aug 12, 2019

My use case is vectordotdev/vector#721, where I want to move objects containing Dates like

{
    timestamp: new Date(),
    host: 'hostname',
    message: 'some text'
}

between Rust and JavaScript code. The structure of the objects is not fixed and it is not known in advance how many fields would be there and which ones of them would be dates.

Right now it is implemented with serialization of the entire object to JSON. Because it is not known in advance which fields contain dates and which don't, in order to keep on-to-one correspondence between JavaScript and Rust dates I replace each Date to {"$date": "<ISO string>"} during serialization and then provide custom reviver and replacer functions to JSON.parse/JSON.stringify in JavaScript. The same encoding is applied when moving the object from JavaScript to Rust.

However, this approach affects performance and I want to pass the objects without serialization in order to reduce the overhead of moving data between Rust and JavaScript. Ideally I would like to be able to construct JsValue containing date in Rust and then be able to check if returned from call_function JsValue is date and convert it to DateTime<Utc> without calling additional JavaScript functions on the way.

@theduke
Copy link
Owner

theduke commented Aug 12, 2019

I see, this is an interesting problem.
A couple of notes:

  • I just merged object deserialization to JsValue::Object, this might be of interest to you
    I also hope to add serde support until the weekend

  • have you though about using f64 timestamps for the de/serialization? that would be a lot faster than toISOString() and parsing into chrono::DateTime
    you could for example convert Date to something like {_t: 1, v: 1231234.34}, that way it would be recognizable as a date from the Rust side (with JsValue::Object)

  • there is no C API to interact with (construct or read) Date objects, so conversion will incur the f64/String overhead either way

  • we could potentially add a chrono feature and add a JsValue::Date(DateTime<Utc>) variant, but conversion will still need to pass through timestamps

We can also consider making the lower level ContextWrapper API public but I'm really hesitant to do that if it can be avoided.

@ghost
Copy link
Author

ghost commented Aug 12, 2019

I just merged object deserialization to JsValue::Object, this might be of interest to you
I also hope to add serde support until the weekend

I've tried it, it works great.

have you though about using f64 timestamps for the de/serialization? that would be a lot faster than toISOString() and parsing into chrono::DateTime

I like this idea, it should improve the performance.

we could potentially add a chrono feature and add a JsValue::Date(DateTime) variant, but conversion will still need to pass through timestamps

I think introducing JsValue::Date(DateTime<Utc>) might be the way to go, combined with numeric timestamps it can be efficient.

@Bindernews
Copy link

I'd like to second this as a feature, because I'd like to be able to pass an object that contains a function back into JS.

My use-case is for a game scripting engine. I know I could store a reference in a global object and use indexing or similar techniques, but that seems like a hacky workaround instead of a good solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants