Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IPAM migration 0072 not performant with large number of existing prefixes #18438

Closed
jeremystretch opened this issue Jan 20, 2025 · 2 comments · Fixed by #18439
Closed

IPAM migration 0072 not performant with large number of existing prefixes #18438

jeremystretch opened this issue Jan 20, 2025 · 2 comments · Fixed by #18439
Assignees
@jeremystretch
Labels
severity: medium Results in substantial degraded or broken functionality for specfic workflows status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application

Comments

@jeremystretch
Copy link
Member

Deployment Type

NetBox Cloud

NetBox Version

v4.2.2

Python Version

3.10

Steps to Reproduce

  1. Create a large number of prefixes (e.g. 20,000) in a previous NetBox release.
  2. Upgrade to NetBox v4.2.0 or later.

Expected Behavior

The database migrations should complete within a reasonable amount of time.

Observed Behavior

The 0072_prefix_cached_relations migration within the ipam app can take an excessively long time to complete when there exist many thousands of prefixes in the database. This is due to a missing batch_size parameter in the bulk_update() operation.
@jeremystretch jeremystretch added severity: medium Results in substantial degraded or broken functionality for specfic workflows status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application labels Jan 20, 2025
@jeremystretch jeremystretch self-assigned this Jan 20, 2025
jeremystretch added a commit that referenced this issue Jan 20, 2025
@jeremystretch
Fixes #18438: Specify batch_size for migrations which run bulk_update()
0d08a6b
@jeremystretch
Copy link
Member Author

I've found a similar issue with two other v4.2 migrations affecting clusters and circuit terminations.

@jeremystretch jeremystretch mentioned this issue Jan 20, 2025
Merged
jeremystretch added a commit that referenced this issue Jan 21, 2025
@jeremystretch
Fixes #18438: Specify batch_size for migrations which run bulk_update()
b913661
@tyler-8
Copy link
Contributor

tyler-8 commented Jan 21, 2025 •
edited
Loading

Glad to see this as my environment has 50K+ prefixes and 12K+ circuits w/terminations.

bctiemann pushed a commit that referenced this issue Jan 28, 2025
@jeremystretch @bctiemann
Fixes #18438: Specify batch_size for migrations which run bulk_update()
da07055
bctiemann pushed a commit that referenced this issue Jan 29, 2025
@jeremystretch @bctiemann
Fixes #18438: Specify batch_size for migrations which run bulk_update()
b25ffc3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jeremystretch jeremystretch

Labels
severity: medium Results in substantial degraded or broken functionality for specfic workflows status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes #18438: Specify batch_size for migrations which run bulk_update() netbox-community/netbox
2 participants
@jeremystretch @tyler-8