-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Creating devices takes an inordinate amount of time #11775
Comments
I tested reindexing with and with an index on object_id and it seems like there's no difference. With index: Without index: |
Wonder if we could push changelogging to the RQWorker (maybe include the option to spin up additional workers as well) for certain execution paths (anything in a RQ worker already can be completed there, anything web/api can go through the RQ). This might also offer additional performance improvements, you just wouldn't get real-time change logging. |
Thanks for tackling this @kkthxbye-code! As we discussed earlier today, we can sidestep the root issue here and avoid clearing any existing cache entries altogether when a new object is created (for which it's impossible to have any pre-existing entries). Of course, we should still add the index to serve cases where we do need to clear them.
I'm not able to reproduce such a stark reduction, but it's likely my cache just isn't large enough to impose that much of a delay. Can you say roughly how many object your test cache contains?
I opened #11202 to explore that idea recently, but haven't done any work on it yet. |
@kkthxbye-code can you test PR #11785 when you get a chance? I haven't added the index yet, but it should result in the same improvement as we're sidestepping the call to If this is deemed acceptable to resolve this specific issue, I'd like to do an audit to add indexes everywhere we use generic foreign keys. |
I noted it in my first reply, but around 800k entries. Important to note that the 60% reduction is without DEBUG enabled. What does the reduction look like for you when just looking at the SQL tab in the debug toolbar? I'll try the PR out! |
Sorry, missed that. I'll need to generate a bunch of objects to increase the cache to test properly on my end.
It shaves some time off, but I don't think I have a large enough cache yet for it to be noticeable. |
NetBox version
v3.4.4
Python version
3.10
Steps to Reproduce
This issue will be used to collect what I find while investigating the issue, as such the scenario is based on my test database and test devicetype.
Expected Behavior
The device should be create within a reasonable time period.
Observed Behavior
The device takes many seconds to create.
Do note, that while the performance was never great, it was made worse by both the search indexing and the fact that we fixed a bug where changelog entries were not created for instantiated components.
The text was updated successfully, but these errors were encountered: