Expose a faster way to create objects with properties for Node-API

[devongovett]

What is the problem this feature will solve?

At the moment, there are a few ways to create objects using the napi C APIs.

• napi_create_object, and then napi_set_named_property for each property.
• napi_create_object, and use napi_define_properties to set all properties at once (under the hood it still sets them one by one though).
• use napi_define_class + napi_new_instance, and then add properties using one of the above methods. You can define accessors on the prototype, or primitive values but instance properties must be set the same way as above.

In my use case I need to create a lot of small objects of the same shape. I've noticed in profiling that v8 functions under napi_set_named_property are slow, specifically v8::internal::JSObject::MigrateToMap. From my research in the v8 source code, this indicates that the object is transitioning from a fast struct/class-like object to a slower dictionary representation (I could be wrong here).

I think v8 exposes ObjectTemplate and FunctionTemplate->InstanceTemplate which are meant to help with this by defining the instance properties up front. This way, the object property slots can be allocated in the object itself rather than in a separate hash map. I found some useful info about v8 object representations here. I see that Node makes use of ObjectTemplate for internal objects, and a previous issue nodejs/node-addon-api#1074 also found this to be the fastest way to create objects (though still not as fast as doing it in JS).

My problem is that napi does not expose a way to create an object together with its properties, or a way to define a template for class instance properties. This means, as far as I can tell, all objects constructed through napi will end up in the slow dictionary mode, leading to slower perf when both constructing and accessing properties.

What is the feature you are proposing to solve the problem?

It would be awesome if node exposed a new napi_create_object_with_properties function, which would accept a list of property descriptors like napi_define_properties and allocate an object and assign properties all at once. This could potentially allow using faster v8 methods to create the object and assign properties so that it doesn't go into hashmap mode when adding properties one by one.

Alternatively, a way to define an instance property template for classes would also work for me. For example, napi_property_attributes could be extended with a napi_instance attribute for defining properties on the instance template rather than the prototype template.

What alternatives have you considered?

I'm not a node or v8 internal expert, just noticed that setting properties on objects seemed slower than expected, so I could be totally wrong about everything above. Opening this issue to get a conversation started. Totally open to other suggestions!

[bnoordhuis]

This means, as far as I can tell, all objects constructed through napi will end up in the slow dictionary mode

Dictionary mode happens when an object has too many properties. In that respect ObjectTemplate vs. adding properties one by one doesn't matter.

Instantiating an ObjectTemplate is marginally faster because you're not going through the JS engine's "add property to object" code path repeatedly but it won't be noticeable unless you create millions of objects or your objects have many properties.

As a practical concern: support for JS engines other than V8 is a design goal of n-api and I don't think other engines have a concept that maps cleanly to ObjecTemplate. Spidermonkey's JSClass for example isn't a 1-to-1 fit.

[devongovett]

Dictionary mode happens when an object has too many properties

Hmm, in my benchmark all the objects I created have exactly 2 properties. Did I misinterpret the profile (what MigrateToMap does)?

I don't think other engines have a concept that maps cleanly to ObjecTemplate

They must have some way of constructing an object with properties upfront through, similar to creating an object literal. And if not, the API could always fall back to constructing an object and assigning properties one by one internally.

[bnoordhuis]

what MigrateToMap does

"Map" in V8 nomenclature is what other JS engines call "object shape" or "hidden class", it's a piece of metadata that describes the object's layout.

It's used every time an object's layout changes (at least in C++ code), not just when switching from fast to slow mode.

They must have some way of constructing an object with properties upfront through, similar to creating an object literal.

Oh yeah, they probably do.

I was thinking of lifetime issues. A JSClass has a static lifetime whereas an ObjectTemplate only lives as long as its isolate. Other engines tie such magic objects to the lifetime of the current execution context (shorter still.)

It's probably not insurmountable but some upfront research is necessary. (I'm not volunteering!)

[devongovett]

I created a simple benchmark in this repo showing the difference between various methods. Each creates 100000 objects with 2 properties and appends them to an array.

napi: 48.992ms
v8 tmpl: 16.973ms
js: 2.663ms

Even with ObjectTemplate, JS object literals are still significantly faster.

[kjvalencik]

I'm also encountering this while working on a serde (serilization) implementation for Neon (Node-API bindings for Rust). I'm having difficulty getting my direct Rust<->JS implementation to outperform JSON (JSON.parse(serde_json::to_string(data))).

I setup a benchmark with criterion serializing the pokedex data set and profiled with pprof and JSON is 2x faster.

Most time is spent in napi_set_property (~62%) with that time spent pretty evenly in ObjectSet between Object::AddDataProperty and PropertyKey::PropertyKey.

Looking at the V8 JSON parsing code, it looks like a lot of work goes into optimizing the construction of a JSON object from a list of properties.

It would be great if these optimizations could be brought over to a Node-API method. Perhaps something like:

node_api_create_object_with_properties(
    napi_env env,
    size_t property_count,
    const napi_property_descriptor* properties,
    napi_value* result
)

[devongovett]

Sounds like such an API could be made faster in JSC as well according to @Jarred-Sumner of Bun:

https://twitter.com/jarredsumner/status/1605566356895801344?s=20&t=ZlICNZzNZotl6gtaOtBULA

[bnoordhuis]

It would be great if these optimizations could be brought over to a Node-API method.

Someone would need to add the corresponding API to V8 first.

[devongovett]

v8 does have a version of v8::Object::New which accepts a list of names and values: https://v8.github.io/api/head/classv8_1_1Object.html#afdb58efc2065a4e7a012db394c302159.

But I noticed in the implementation it calls NewSlowJSObjectWithPropertiesAndElements so I don't think it's optimized. I benchmarked that and it's slower than ObjectTemplate. But seems like maybe that existing API could be made faster?

[mhdawson]

We discussed a bit in the Node-api team meeting. @vmoroz is going to take a look and do some experimentation.

[vmoroz]

My hope was that we can improve perf for setting properties by adding an API to create internalized strings (NewStringType::kInternalized). V8 like other JS engines internalizes all string keys, and we can help it by caching them. We must also allow napi_ref to keep strings in a cache (#45715 or #42557). Unfortunately, effect from such change is almost not visible in this case because we have only two properties per object.

It would be great to add an API like node_api_create_object_with_properties. The only question is how to make it work fast. Using ObjectTemplate does not help - it only shows good results when we reuse it multiple times. Any single use of ObjectTemplate makes the perf worse. If we want to expose ObjectTemplate, then we will have a lot of other issues such as lifetime issues, incompatibility with other JS engines, complex use scenarios, etc.

So far it seems that Node-API or V8 API cannot create JS objects as fast as the JS parser does from object literals. It makes sense: the parser has the full information upfront and can optimize the creation unlike anything we can do through the API. Adding a special V8 API as for JSON object might address this issue. But frankly, it all feels a bit wrong: on one side we have C++ or Rust that can be extremely efficient with handling large amount of data, and on the other side we have JavaScript that was never meant to be that efficient, and still when we use C++ we lose. The reason we lose is because we make C++ to play by JavaScript rules. Since the interop has an overhead, we theoretically cannot achieve the same perf as in JavaScript. The only way to win is to avoid playing by JavaScript rules. It means avoiding creation a lot of small objects from C++ and instead explore other techniques.

My understanding is that these small objects are needed to represent some UI scene graphs. Games written in C++ often avoid creation of many small objects and instead use arrays for better cache locality. Thus, the proposal is to rethink the JS code that works with the native APIs in a way that it keeps data mostly in big arrays and is not "chatty".

• The data that is kept in native arrays can be either completely native or be exposed to JS code through TypedArrays or Buffers.
• The API must not be "chatty": any call to the native code must result in some "juicy action" - a non-trivial work done by native code to process the data structures. This way any interop overhead will be negligible comparing with the useful work.

I realize that I am not helping to address the issue as it was stated, but I am quite interested to find a work around to this type of problems. It would be great to create a Node-API example that shows these techniques.

[devongovett]

Thanks for the investigation! In my case, I'm building a compiler in Rust which allows custom JavaScript visitor plugins. This consists of JS functions that are called when a node in the AST is visited. For a given file, the functions may be called hundreds or thousands of times, with small objects as arguments. It would be pretty hard to avoid creating all these objects in this case I think. If you're curious, the docs have more info.

Do you think it would be worth filing a ticket with V8 asking to optimize v8::Object::New similar to the JSON parser? Seems to me that if they know all the properties and values up front, it could work similarly. Internally, they probably have a way to do this for implementing language builtins or maybe even things like DOM APIs?

[vmoroz]

It is a very cool project! And it is completely different scenario from the scene graph... TypedArray or "less chatty" API will not work.

Based on the spec and examples it seems that each argument type may have several properties.
I wonder if the following approach would work:

• Each argument type must have a predefined prototype object with accessor (get/set) properties.
• Each new argument instance will use the corresponding prototype object and wrap associated native object pointer. No properties must be set per argument instance. They will be all "inherited" from the prototype.
• The property accessor methods (native functions) will take this argument, unwrap the associated native object pointer, and then access the required data.

Hopefully, this way we can reduce amount of work done per each argument object and mostly reuse all the code as prototypes.
EDIT: it all can be done by napi_define_class.

As an alternative, it is possible to use Proxy object. This example simulates ObjectTemplate behavior by using Proxy. I would expect the prototype to work better than Proxy, but I did not measure it.

As for asking V8 team for the new API, it should be worth doing it. We must be able to wrap it in a Node-API function if they implement it.

[kjvalencik]

In my use case, key names are known upfront and interning strings (with String objects and napi_ref) did have a significant improvement. However, it was still 2-3x slower than JSON with most of the time spent in napi_set_property.

Since I'm writing a Rust serializer, in most cases, keys and even types are known and can be aggregated up front, similar to the optimizations in the V8 JSON parser.

Interestingly, these optimizations appear to only exist in the JSON parser. I also tried writing a v8.Serializer implementation in Rust to improve performance, but v8.deserialize was many times slower than JSON.parse.

[Jarred-Sumner]

I don't know how it works in V8, but in JSC there is an explicit JSC::Structure class which holds the properties defined for an object and the structure can be reused for multiple objects.

To create new objects with the same properties quickly in JSC, we can:

1. define the structure once with all the properties added
2. put the properties onto the objects in the same order the properties were defined

It has a special fast path when there are under 64 properties and then a fast but not as fast path for objects with more than 63 properties

The particular types of the properties in the JSC::Structure don't matter much. It only needs to know how many, what are the property names, and what are the attributes

It's fastest to persistently hold on to a Structure if it's going to be reused, but then it needs to be visited by the garbage collector and that's tricky because who owns it? In bun's case, we use this optimization in a couple places and we lazily mark it as owned by the global object which means once created, it's never freed. This is okay because we are selective of when we do this optimization

I recommend against going the approach of using getter/setters because it will make the runtime performance of using the objects worse due to lack of inlining.

[vmoroz]

@kjvalencik

In my use case, key names are known upfront and interning strings (with String objects and napi_ref) did have a significant improvement. However, it was still 2-3x slower than JSON with most of the time spent in napi_set_property.

It sounds very promising for the interning strings case!
I also wonder if you tried using the v8::Object::CreateDataProperty instead of napi_set_property that uses v8::Object::Set. The comments in v8-object.h say that this is the fastest way to create a value property.

Since I'm writing a Rust serializer, in most cases, keys and even types are known and can be aggregated up front, similar to the optimizations in the V8 JSON parser.

V8 source code is included in Node.JS. I wonder if we can prototype and measure the JSON-like approach.

@Jarred-Sumner

I don't know how it works in V8, but in JSC there is an explicit JSC::Structure class which holds the properties defined for an object and the structure can be reused for multiple objects.

It would be interesting to learn more about such JS engine-specific optimizations and see if we can have a new Node-API function that takes advantage of them. Could you share a link to the related JSC source code or examples for it?

[kjvalencik]

@vmoroz

I also wonder if you tried using the v8::Object::CreateDataProperty instead of napi_set_property that uses v8::Object::Set

I did not. When I get a chance I can setup a local copy of nodejs for hacking.

I also took a bit of a shortcut on string interning and used thread-local-storage instance of instance data. However, I doubt that will have much of a performance impact.

So far, my best performance is using Node-API calls directly for simple types and switching to JSON for objects.

[kjvalencik]

@vmoroz I tried CreateDataProperty provided a small improvement. The improvement itself was offset by needing to call napi_coerce_to_string in order to use the interned string objects with it; leaving almost no measurable speed-up.

[vmoroz]

@kjvalencik, thank you for testing it! Now we have two possible "knobs" to do slight improvements for the property settings: interned strings and CreateDataProperty.
I am not sure why the napi_coerce_to_string is needed. I was hoping that we can add a new function like node_api_create_name_xxx similar to napi_create_string_xxx that internally creates a V8 string with the NewStringType::kInternalized parameter out of the native string. This call must be not more expensive than the regular string creation.

[kjvalencik]

napi_coerce_to_string is needed in my test because interning requires a long lived reference and napi_ref only works with objects. However, a String object is not a v8::Name--that's only string and Symbol.

Let me know if you can think of a better way to hold those references for testing purposes.

[vmoroz]

Ah, I see. If you are hacking Node.JS code, then you could enable napi_ref for strings by removing the restriction in the napi_create_reference. The rest of the code works just fine.
As an alternative to napi_coerce_to_string you could try storing strings in an Array and access them by index. I hope it can be more efficient because no new allocations needed. Unless the element access kills it again...

[kjvalencik]

@vmoroz Thanks for the tip! Making that change to napi_ref as well as using CreateDataProperty, performance is getting closer to JSON (844 µs vs 594 µs).

Benchmarking Serialize/direct
Benchmarking Serialize/direct: Warming up for 3.0000 s
Benchmarking Serialize/direct: Collecting 100 samples in estimated 13.027 s (15k iterations)
Benchmarking Serialize/direct: Analyzing
Serialize/direct        time:   [839.44 µs 844.67 µs 850.85 µs]
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

Benchmarking Serialize/json
Benchmarking Serialize/json: Warming up for 3.0000 s
Benchmarking Serialize/serde_json: Collecting 100 samples in estimated 12.055 s (20k iterations)
Benchmarking Serialize/json: Analyzing
Serialize/json          time:   [592.06 µs 594.63 µs 597.57 µs]
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  5 (5.00%) high severe

Setting the property is still the bottleneck.

[vmoroz]

@kjvalencik , @devongovett , it looks like V8 already has a function to create Object literals :)
https://github.com/nodejs/node/blob/main/deps/v8/include/v8-object.h#L673

  /**
   * Creates a JavaScript object with the given properties, and
   * a the given prototype_or_null (which can be any JavaScript
   * value, and if it's null, the newly created object won't have
   * a prototype at all). This is similar to Object.create().
   * All properties will be created as enumerable, configurable
   * and writable properties.
   */
  static Local<Object> New(Isolate* isolate, Local<Value> prototype_or_null,
                           Local<Name>* names, Local<Value>* values,
                           size_t length);

[devongovett]

Yeah that's the one I linked to above.

But I noticed in the implementation it calls NewSlowJSObjectWithPropertiesAndElements so I don't think it's optimized. I benchmarked that and it's slower than ObjectTemplate. But seems like maybe that existing API could be made faster?

[vmoroz]

Hmm... In that case the important part is that at least the API exists. So, instead of asking V8 team to create a new API, we can ask for improving the perf.
I wonder what the performance would be if we combine this Object.New with the interned strings and Object.Clone().

EDIT: You are right: the perf for this Object.New and Object.Clone() is much worse than for the ObjectTemplate... The fastest way is still using the CreateDataProperty, but it is ~25% slower than ObjectTemplate.

EDIT 2: @devongovett, I re-read the thread and I am sorry that I was repeating the same points you made before. You already pointed at the v8::Object::New function signature and that we can ask to make it faster.

[kjvalencik]

Using ObjectTemplate didn't seem to yield much of a speed-up for me. It was slower than CreateDataProperty.

[vmoroz]

@kjvalencik , I am playing with the benchmark sample created by @devongovett. So far, I cannot beat the perf offered by the ObjectTemplate there. This benchmark creates 100,000 objects out of the same template that clearly offers huge advantage to the ObjectTemplate comparing to creation objects from scratch. On my machine it was like:

• 5 ms for JS
• 40 ms for ObjectTemplate
• 50 ms for Object::New() + CreateDataProperty
• 100 ms for napi

If I put the ObjectTemplate inside of the loop, then I get something like 150 ms.

I bet your perf scenario is a bit more balanced and thus produces different results.

[kjvalencik]

In many scenarios, there would need to be a persistent reference to the v8::ObjectTemplate which adds some overhead. Also, I'm using larger objects (16 keys, pokemon.json), which might be contributing to the difference.

Edit
I accidentally removed string interning when testing ObjectTemplate. I do see a measurable improvement.

[Jarred-Sumner]

It would be interesting to learn more about such JS engine-specific optimizations and see if we can have a new Node-API function that takes advantage of them. Could you share a link to the related JSC source code or examples for it?

Usage:

https://github.com/oven-sh/bun/blob/ff3f5aa154fd4ee390c87733b6ed61d68a750c33/src/bun.js/bindings/webcore/JSTextEncoder.cpp#L298-L301

Creating it:

https://github.com/oven-sh/bun/blob/ff3f5aa154fd4ee390c87733b6ed61d68a750c33/src/bun.js/bindings/ZigGlobalObject.cpp#L2571-L2583

[vmoroz]

@devongovett, it would be nice to have the whole the end-to-end prototype first. Just to make sure that we ask for the right thing.
My current thinking is that we must add three different API sets to the Node-API:

• Fast object creation.
• Fast array creation.
• Create internalized strings.

The fast object creation can be achieved by

• Providing existing object that can be used as a template. Hopefully we can reuse its hidden class.
• Providing the full set of property name-value pairs when no template is given. Otherwise, it will provide only the delta for the template.

The API will essentially combine JavaScript Object initializer, Object.assign(), and Object.fromEntries() behavior.

This API will have one significant drawback - it will not support property descriptors and focus only on the writable, enumerable, and configurable value properties. Is it a big disadvantage? Should we use the property descriptors as @kjvalencik proposed above?

The fast array creation should mimic the JavaScript Array literals. There can be three overloads: for Integers, Doubles, and general napi_value. The hope is that knowing the size and type of values we can improve the initial array creation speed.

The string internalizing can give advantage for the V8 and other JS engines. The JS engines that do not support it can just return a regular string.

Thus, the full list of new APIs may look like:

napi_status node_api_create_object_from_entries(
  napi_env env,
  napi_value object_tempate, // optional object to clone
  napi_value prototype, // either object or null to use when no object_tempate given
  size_t property_count, // number of names and values entries
  napi_value* names,
  napi_value* values,
  napi_value* result);

napi_status node_api_create_array_int32(napi_env env, size_t count, int32_t* values, napi_value* result);
napi_status node_api_create_array_double(napi_env env, size_t count, double* values, napi_value* result);
napi_status node_api_create_array_from_values(napi_env env, size_t count, napi_value* values, napi_value* result);

napi_status node_api_create_property_key_utf8(napi_env env, const char* str, size_t length, napi_value* result);
napi_status node_api_create_property_key_utf16(napi_env env, const char16_t* str, size_t length, napi_value* result);
napi_status node_api_create_property_key(napi_env env, napi_value str_or_sym, napi_value* result);

[devongovett]

Looks good to me!

Should we use the property descriptors

I think this is a much more rare use case, so it's ok if it's done through a different API that isn't as optimized. This would be similar to creating an object literal in JS vs Object.create.

[kjvalencik]

Should we use the property descriptors

I wasn't proposing it, just benchmarking. I don't think this is necessary.

There can be three overloads: for Integers, Doubles, and general napi_value

I think only napi_value is necessary. Numeric values are already handled well by typed arrays which can usually be created with a simple memcpy.

[loynoir]

Any news?

napi_set_named_property and napi_define_properties are incredibly slow when ~20 properties.

17575 / 703391 = 0.02498610303515399

Only ~2.5% ops of C addon without create jsobj.

(index)	Task Name	ops/sec	Average Time (ns)	Margin	Samples
0	'js_decode'	'210,892'	4741.746690232202	'±0.46%'	105447
1	'c_decode_without_create_jsobj'	'703,391'	1421.6827947166248	'±0.35%'	351696
2	'c_decode_create_jsobj_with_napi_set_named_property'	'23,221'	43064.05215451868	'±0.25%'	11611
3	'c_decode_create_jsobj_with_napi_define_properties'	'17,575'	56897.14262819765	'±5.52%'	8816
4	'c_decode_create_jsobj_with_napi_call_function'	'117,885'	8482.791340440293	'±0.21%'	58943
5	'c_decode_create_jsobj_with_napi_call_function_arrow'	'119,532'	8365.921667058692	'±0.66%'	59767

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[devongovett]

Not stale