Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model object serialization can cause memory limits to be exceeded #617

Closed
jaredhendrickson13 opened this issue Dec 17, 2024 · 0 comments · Fixed by #618
Closed

Model object serialization can cause memory limits to be exceeded #617

jaredhendrickson13 opened this issue Dec 17, 2024 · 0 comments · Fixed by #618
Labels
bug Something isn't working

Comments

@jaredhendrickson13
Copy link
Owner

Describe the bug
When updating or deleting objects in larger configuration datasets, it's possible for memory limits to be exceeded due to the way Model object's are currently being serialized. Model objects keep a separate copy of themselves in the initial_object property that is intended to represent the object as it currently exists in the config. When a Model object has changed written to config, it will overwrite this property with a new copy of itself using PHP serialization. Currently, serialization includes many properties that are not necessary and can cause exponential recursion for certain Models. Serialization of the Model class should be optimized to only include properties that are absolutely necessary.

To Reproduce
Steps to reproduce the behavior:

  1. Create a large number of DHCP Server Static Mappings.
  2. Try to delete any of these static mappings using DELETE /api/v2/services/dhcp_server/static_mapping

You will receive a 'memory limit Allowed memory size of ***** bytes exhausted' error.

Expected behavior
We should be able to update or delete this objects without exceeding memory limits.

pfSense Version & Package Version:

  • pfSense Version: CE 2.7.2 (likely affects all versions)
  • Package Version v2.2.2

Affected Endpoints:

  • /api/v2/services/dhcp_server/static_mapping

This is not exclusive to this endpoint, this issue theoretically could occur in multiple endpoints given a large enough scale. But it is easily reproducible at this endpoint.

Additional context

Discussed in #616

Originally posted by couraillon December 13, 2024
Hello,

I'm using pfSense v24.03 and the pfsense-api v2.2.2 to remove and add some static_mappings.
My script should read two lists of MAC addresses, those to be removed from static_mappings and those to be added.
The script first checks whether each MAC address is already registered, and that part works well. But then when it comes to delete all the MAC addresses I get the following error:

PHP ERROR: Type: 1, File: /usr/local/pkg/RESTAPI/Core/Model.inc, Line: 2289, Message: Allowed memory size of 536870912 bytes exhausted (tried to allocate 8192 bytes) @ 2024-12-13 11:49:24

I have read this discussion, but the error still seems to exist on my side, despite the patch.
Is this a known error when there are too many queries in a row?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

1 participant