Skip to content

DRF does not release memory after large JSON/XML request data is read or send through response #8472

@Dhananjay1992

Description

@Dhananjay1992

##Framework version details
Django==3.1.4
djangorestframework==3.12.2
python 3.8.10
gunicorn==20.0.4

I have observed that when some data in the form of JSON or XML is read through request or serializers the memory is never released.

I have one very basic API with APIView which has kind of below code.

`

class SomeClass(APIView):

serializer_class = JsonObjectSerializer
def post(self, request):
    serializer_object = JsonObjectSerializer(data=self.request.data)
    assert serializer_object.is_valid(), "Some error"
    obj1 = serializer_object.validated_data.get('obj1')
    obj2 = serializer_object.validated_data.get('obj2')
    result = perform_some_operation(obj1, obj2, serializer_object)
    response = construct_fe_response(result)
    return Response(status=status.HTTP_200_OK, data=response)

`
This does some JSON object reading through serializer and it process it and then send processed result as response. In this operation I see every time when JSON object comes this operation increases memory lets say by ~20MB (for large/medium sized JSON data) and if I have 100 calls for this API it goes around ~2GB of my RAM. even after few days the memory is never released.

I have used memory_profile on my processing and post method of above code and result is that when serializer_object = JsonObjectSerializer(data=self.request.data) this line is called memory increased to ~15 MB and then it never releases it.
It might be duplicate of memory issues in DRF but this still happens and needs attention.

I am using gunicorn with wsgi

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions