-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(schema) no deep copy on select on process auto fields #8796
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Soft 👍 My knowledge of the dao/schema modules does not go very deep, but after looking through some of the code this seems relatively safe/sensible.
One note; the plugin doc string states that a new table is returned. It would be good to update this to reflect the new behavior, that the table is modified in-place for select
:
-- @return A new table, with the auto fields containing
-- appropriate updated values.
Yes, last time I tried this there was some tests failing, but it might be because of other changes. But now it seems tests are green. The biggest issue to me is things like (I think we rarely feed this thing with https://github.com/Kong/kong/blob/master/kong/runloop/handler.lua#L766 And especially on dbless: I have been testing this with some 64k routes, and deep copying tables takes around 1/4th of the rebuild time. I will adjust the description there. |
4da2ddd
to
84ea2e9
Compare
The description was adjusted and tests were added (so this is now "official", and not undefined behavior). |
84ea2e9
to
eee3867
Compare
39be720
to
8802f44
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - unless @flrgh has other concerns.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Failures in this PR (and other perf
PRs) are unrelated; as the perf testing framework picks a package for the latest official Kong release, and Kong master requires a newer OpenResty version, the older OpenResty isn't recognizing a new Nginx directive that is being used.
8802f44
to
dba3fef
Compare
### Summary It is inefficient to create deep copies of tables when e.g. looping through database rows. In my testing with uploading some 64k routes with dbless (which calls process auto fields twice), this can cut the time looping the data by 1/4th. It also generates much less garbage. I searched our code bases where we use "select" context, and could not find anything that might break because of this.
dba3fef
to
196fb3f
Compare
Summary
It is inefficient to create deep copies of tables when e.g. looping through database rows (and with records in them we recursively deep copy them). In my testing with uploading some 64k routes with dbless (which calls process auto fields twice), this can cut the time looping the data by 1/4th. It also generates much less garbage.
I searched our code bases where we use "select" context, and could not find anything that might break because of this.