Skip to content

Commit

Permalink
perf: improve performance of bulk child entity updates and avoid hitt…
Browse files Browse the repository at this point in the history
…ing the AQL limit of 500 nesting limit doing it

Child entity updates generate a lot of AQL because each entity can have
different kinds of updates, and some of them refer to old values. This
is still the case after the optimizations.

However, previously, we performed multiple child entity updates by
nesting conditional expressions like this:

items = items.map(item =>
  item.id == 2 ? update2(item) : (item.id == 1 ? update1(item) : item)
);

This generated one level of nesting per update. ArangoDB 3.11 now limits
the AQL nesting to 500, which means we would be limited to a little
under 500 item updates in one mutation. In addition, a few hundred
updates had really bad performance.

Now, we convert the list into a dictionary (id -> entity), so we can
more efficiently look up items and construct the new list.

This optimization is only applied after a threshold of 3 (configurable)
because it involves some steps, and doing a single or two updates is
probably still faster with the conditionals.
  • Loading branch information
Yogu committed Nov 7, 2023
1 parent 1ca6cee commit 7228851
Show file tree
Hide file tree
Showing 15 changed files with 830 additions and 37 deletions.
3 changes: 2 additions & 1 deletion spec/regression/logistics/default-context.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{
"authRoles": ["allusers"]
"authRoles": ["allusers"],
"childEntityUpdatesViaDictStrategyThreshold": 2
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"childEntityUpdatesViaDictStrategyThreshold": 1,
"authRoles": ["allusers"]
}
120 changes: 120 additions & 0 deletions spec/regression/logistics/tests/update-child-entities-dict.graphql
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# in the .context.json, childEntityUpdatesViaDictStrategyThreshold is set to 1
# so we will always use the dict strategy to update child entities

mutation updateOne {
updateDelivery(
input: {
id: "@{ids/Delivery/1}"
updateItems: [{ id: "id_init_0000", itemNumber: "updated1" }]
}
) {
items {
id
itemNumber
}
}
}

# to verify it is updated in the db
query afterUpdateOne {
Delivery(id: "@{ids/Delivery/1}") {
items {
id
itemNumber
}
}
}

mutation addSome {
updateDelivery(
input: {
id: "@{ids/Delivery/1}"
addItems: [
{ itemNumber: "added00" }
{ itemNumber: "added01" }
{ itemNumber: "added02" }
{ itemNumber: "added03" }
{ itemNumber: "added04" }
{ itemNumber: "added05" }
{ itemNumber: "added06" }
{ itemNumber: "added07" }
{ itemNumber: "added08" }
{ itemNumber: "added09" }
{ itemNumber: "added10" }
]
}
) {
items {
id
itemNumber
}
}
}

mutation updateMultiple {
updateDelivery(
input: {
id: "@{ids/Delivery/1}"
updateItems: [
{ id: "id_test_0003", itemNumber: "updated03" }
{ id: "id_test_0005", itemNumber: "updated05" }
{ id: "id_test_0007", itemNumber: "updated07" }
{ id: "id_test_0008", itemNumber: "updated08" }
{ id: "id_test_0009", itemNumber: "updated09" }
]
}
) {
items {
id
itemNumber
}
}
}

# to verify it is updated in the db
query afterUpdateMultiple {
Delivery(id: "@{ids/Delivery/1}") {
items {
id
itemNumber
}
}
}

mutation addUpdateAndDelete {
updateDelivery(
input: {
id: "@{ids/Delivery/1}"
addItems: [
{ itemNumber: "finalNew1" }
{ itemNumber: "finalNew2" }
{ itemNumber: "finalNew3" }
]
updateItems: [
{ id: "id_test_0004", itemNumber: "finalUpdated04" }
# this is finalNew2
{ id: "id_test_0012", itemNumber: "finalUpdated02" }
]
removeItems: [
"id_test_0007"
# this is finalNew3
"id_test_0013"
]
}
) {
items {
id
itemNumber
}
}
}

# to verify it is updated in the db
query end {
Delivery(id: "@{ids/Delivery/1}") {
items {
id
itemNumber
}
}
}
Loading

0 comments on commit 7228851

Please sign in to comment.