Memory leak on every request

[alexpts]

Simple example for reproduce:

<?php
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Message\ResponseInterface;

require_once __DIR__ . '/../../vendor/autoload.php';

$handler = function (ServerRequestInterface $request): ResponseInterface {
    $memory = round(memory_get_peak_usage() / 1024, 2) . ' kbyte';
    $response = new \React\Http\Response(200, [], 'message');
    $response = $response->withHeader('X-Memory', $memory);
    return $response;
};

$loop = \React\EventLoop\Factory::create();
$server = new \React\Http\Server($handler);
$socket = new \React\Socket\Server(8081, $loop);
$server->listen($socket);
$loop->run();

Memory grows on every request after 20-40 requests.

---

"react/http": "^0.8"
PHP 7.2.0 (cli) (built: Dec 3 2017 21:46:44) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.2.0, Copyright (c) 1999-2017, by Zend Technologies

[alexpts]

Screen with memory leak from chrome dev tools: https://www.dropbox.com/s/60tfvj2ixhv0kr9/%D0%A1%D0%BA%D1%80%D0%B8%D0%BD%D1%88%D0%BE%D1%82%202017-12-25%2021.46.47.png?dl=0

[WyriHaximus]

Hey @alexpts for how long are you running it and how busy is the process? PHP usually takes a while before garbage collection kicks in. If it rises for a few minutes onder low load I wouldn't be alarmed, I would when it doesn't collects garbage after 5 minutes.

The following graph is from one of my projects, as you can see from the yellow line is that memory usage climbs before getting cleaned up:

Also the peak usage function you use will only go up, try memory_get_peak_usage instead

[alexpts]

@WyriHaximus I test not long. Ok, I will monitoring it in most long.

[kelunik]

@WyriHaximus That means there are cyclic references that need a GC run to collect these. It'd be better to avoid these cyclic references if possible.

[clue]

@alexpts Thank you, keep us posted should you happen to run into any memory issues 👍

@kelunik No objections. PRs to avoid any cyclic references are much appreciated 👍

[mmoreram]

@WyriHaximus I think that is responsibility of the library to maintain a flat memory consumption over the time. Basicaly because in services like AWS ECS, where you can define hard top memory limits where the service should be restarted, this turns a mess. And the feature has to be disabled.

Each iteration, would be nice to set to null these variables that keeps in memory until the GC makes its job.

Would the Owners be interested in this investigation?
Already done in the PHP-PM project.

Thanks :)

[kelunik]

If you don't have concurrent requests, you can just call gc_collect_cycles() between the requests.

[WyriHaximus]

@mmoreram go for it, the flatter the better 👍

[andig]

Do we have any idea where the cycles are coming from, i.e. where they are created?

[mmoreram]

@kelunik I will never defend gc_collect_cycles. In fact, this is the lazy option for a problem that should be solved by good practices. Having 1M requests per minute means making 1M of gc_collect_cycles calls?

@WyriHaximus that's the attitude.

[mmoreram]

After reviewing some code, a BIG jump on memory comes from Stream - DuplexResourceStream::handleData
This method is called by using an anonymous function, and this is where PHP memory has strange behaviors.

[WyriHaximus]

This method is called by using an anonymous function

Yup those are everywhere

[clue]

Just to be clear: We're not currently are of any memory leaks in any of ReactPHP's components. Plenty of people run these projects in production and I'm sure we would know by now if there were any such leaks.

That being said, I'm sure there are still ways to improve memory usage in long-running applications. For instance, recently we've put significant effort into avoiding many of the internal cyclic references inside promise rejection handlers. This has significantly improved temporary memory usage for rejected promises.

Cyclic references do not cause memory leaks. But they do avoid PHP's deterministic garbage collector using reference counting to kick in. Instead, these references will be kept in memory until PHP's cyclic garbage collector (http://php.net/manual/de/features.gc.collecting-cycles.php) kicks in. This means that left-over cyclic references can accumulate some memory (commonly in the range of ~5 MB) before their memory is freed again (which would cause a zick-zack effect as depicted in @WyriHaximus screenshot). Additionally, you can manually invoke this using gc_collect_cycles() at any time. This should not cause any issues, but it does have a noticeable performance impact, so it's usually best to leave this up to PHP to decide unless you absolutely know your application does accumulate some cyclic references in some part that you want to clear instantly.

Again, we're not aware of any memory leaks in ReactPHP at the time of writing this. If you happen to find one, please make sure to let us know, file this as a new bug report and we're happy to sort this out! 💪