Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak when using persistent connections #514

Closed
mmoreram opened this issue Feb 5, 2024 · 3 comments · Fixed by #515
Closed

Memory leak when using persistent connections #514

mmoreram opened this issue Feb 5, 2024 · 3 comments · Fixed by #515
Labels
Milestone

Comments

@mmoreram
Copy link

mmoreram commented Feb 5, 2024

When using persistent connections, this code makes me think that could cause problems... This happens inside the handleRequest in StreamingServer file. If we use persistent connections, we are adding one close event per request, causing all responses being referenced, even when it is returned properly (not allowing GC make its work).

// cancel pending promise once connection closes
if ($response instanceof PromiseInterface && \method_exists($response, 'cancel')) {
    $conn->on('close', function () use ($response) {
        $response->cancel();
    });
}
@mmoreram mmoreram added the bug label Feb 5, 2024
@WyriHaximus
Copy link
Member

Hey @mmoreram, just tried to see if keep alive indeed leaks memory due to that. And I can't prove that it does. Modified example 51 into this:

<?php

use React\EventLoop\Loop;

$reportMemory = static function () {
    $rawInt = memory_get_usage(false);
    $rawExt = memory_get_usage(true);
    echo date('r'), ' Internal: ', ($rawInt / 1024), 'KB External: ', ($rawExt / 1024), 'KB (Raw Internal: ', $rawInt, ' Raw External: ', $rawExt, ')', PHP_EOL;
};

$reportMemory();
echo 'Loading autoloader', PHP_EOL;
require __DIR__ . '/../vendor/autoload.php';

$reportMemory();
echo 'Creating HTTP Server', PHP_EOL;
$http = new React\Http\HttpServer(function (Psr\Http\Message\ServerRequestInterface $request) {
    return React\Http\Message\Response::plaintext(
        "Hello World!\n"
    );
});

$reportMemory();
echo 'Creating Socket Server', PHP_EOL;
$socket = new React\Socket\SocketServer(isset($argv[1]) ? $argv[1] : '0.0.0.0:0');

$reportMemory();
echo 'Listening on socket', PHP_EOL;
$http->listen($socket);

Loop::addPeriodicTimer(60, $reportMemory);

echo 'Listening on ' . str_replace('tcp:', 'http:', $socket->getAddress()) . PHP_EOL;

$reportMemory();
echo 'Starting Event Loop', PHP_EOL;

And then ran ab against it with the following arguments to do 100.000.000 requests against it with 100 keep alive connections:

ab -k -c 100 -n 100000000 http://localhost:42353/

It came back after just over an hour with this:

image

This is the output from the server. As you can see it stays stable at 3.9MB of memory used by PHP and 6MB allocated with the system throughout the test. After completed it dropped back to just over 2MB. I would love to have more information on what makes this a memory leak for you.

Sat, 17 Feb 2024 18:29:57 +0100 Internal: 402.5546875KB External: 2048KB (Raw Internal: 412216 Raw External: 2097152)                                                                                      
Loading autoloader                                                                                                                                                                                         
Sat, 17 Feb 2024 18:29:57 +0100 Internal: 1262.78125KB External: 2048KB (Raw Internal: 1293088 Raw External: 2097152)                                                                                      
Creating HTTP Server                                                                                                                                                                                       
Sat, 17 Feb 2024 18:29:57 +0100 Internal: 1599.9921875KB External: 2048KB (Raw Internal: 1638392 Raw External: 2097152)                                                                                    
Creating Socket Server                                                                                                                                                                                     
Sat, 17 Feb 2024 18:29:57 +0100 Internal: 1658.4453125KB External: 2048KB (Raw Internal: 1698248 Raw External: 2097152)                                                                                    
Listening on socket                                                                                                                                                                                        
Listening on http://0.0.0.0:42353                                                                                                                                                                          
Sat, 17 Feb 2024 18:29:57 +0100 Internal: 1667.5859375KB External: 2048KB (Raw Internal: 1707608 Raw External: 2097152)                                                                                    
Starting Event Loop                                                                                                                                                                                        
Sat, 17 Feb 2024 18:30:57 +0100 Internal: 3951.8046875KB External: 6144KB (Raw Internal: 4046648 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:31:57 +0100 Internal: 3938.9140625KB External: 6144KB (Raw Internal: 4033448 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:32:57 +0100 Internal: 3935.984375KB External: 6144KB (Raw Internal: 4030448 Raw External: 6291456)                                                                                     
Sat, 17 Feb 2024 18:33:57 +0100 Internal: 3947.1171875KB External: 6144KB (Raw Internal: 4041848 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:34:57 +0100 Internal: 3955.90625KB External: 6144KB (Raw Internal: 4050848 Raw External: 6291456)                                                                                      
Sat, 17 Feb 2024 18:35:57 +0100 Internal: 3964.109375KB External: 6144KB (Raw Internal: 4059248 Raw External: 6291456)                                                                                     
Sat, 17 Feb 2024 18:36:57 +0100 Internal: 3961.1796875KB External: 6144KB (Raw Internal: 4056248 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:37:57 +0100 Internal: 3940.671875KB External: 6144KB (Raw Internal: 4035248 Raw External: 6291456)                                                                                     
Sat, 17 Feb 2024 18:38:57 +0100 Internal: 3958.8359375KB External: 6144KB (Raw Internal: 4053848 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:39:57 +0100 Internal: 3962.9375KB External: 6144KB (Raw Internal: 4058048 Raw External: 6291456)                                                                                       
Sat, 17 Feb 2024 18:40:57 +0100 Internal: 3951.8046875KB External: 6144KB (Raw Internal: 4046648 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:41:57 +0100 Internal: 3951.8046875KB External: 6144KB (Raw Internal: 4046648 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:42:57 +0100 Internal: 3938.9140625KB External: 6144KB (Raw Internal: 4033448 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:43:57 +0100 Internal: 3956.4921875KB External: 6144KB (Raw Internal: 4051448 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:44:57 +0100 Internal: 3967.0390625KB External: 6144KB (Raw Internal: 4062248 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:45:57 +0100 Internal: 3954.1484375KB External: 6144KB (Raw Internal: 4049048 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:46:57 +0100 Internal: 3942.4296875KB External: 6144KB (Raw Internal: 4037048 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:47:57 +0100 Internal: 3959.421875KB External: 6144KB (Raw Internal: 4054448 Raw External: 6291456)                                                                                     
Sat, 17 Feb 2024 18:48:57 +0100 Internal: 3941.84375KB External: 6144KB (Raw Internal: 4036448 Raw External: 6291456)                                                                                      
Sat, 17 Feb 2024 18:49:57 +0100 Internal: 3955.90625KB External: 6144KB (Raw Internal: 4050848 Raw External: 6291456)                                                                                      
Sat, 17 Feb 2024 18:50:57 +0100 Internal: 3945.359375KB External: 6144KB (Raw Internal: 4040048 Raw External: 6291456)                                                                                     
Sat, 17 Feb 2024 18:51:57 +0100 Internal: 3928.953125KB External: 6144KB (Raw Internal: 4023248 Raw External: 6291456)                                                                                     
Sat, 17 Feb 2024 18:52:57 +0100 Internal: 3923.6796875KB External: 6144KB (Raw Internal: 4017848 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:53:57 +0100 Internal: 3932.46875KB External: 6144KB (Raw Internal: 4026848 Raw External: 6291456)                                                                                      
Sat, 17 Feb 2024 18:54:57 +0100 Internal: 3937.7421875KB External: 6144KB (Raw Internal: 4032248 Raw External: 6291456)                                                                                    
Sat, 17 Feb 2024 18:55:57 +0100 Internal: 3966.453125KB External: 6144KB (Raw Internal: 4061648 Raw External: 6291456)                                                                                                      
Sat, 17 Feb 2024 18:56:57 +0100 Internal: 3955.90625KB External: 6144KB (Raw Internal: 4050848 Raw External: 6291456)
Sat, 17 Feb 2024 18:57:57 +0100 Internal: 3945.359375KB External: 6144KB (Raw Internal: 4040048 Raw External: 6291456)
Sat, 17 Feb 2024 18:58:57 +0100 Internal: 3942.4296875KB External: 6144KB (Raw Internal: 4037048 Raw External: 6291456)
Sat, 17 Feb 2024 18:59:57 +0100 Internal: 3961.765625KB External: 6144KB (Raw Internal: 4056848 Raw External: 6291456)
Sat, 17 Feb 2024 19:00:57 +0100 Internal: 3954.1484375KB External: 6144KB (Raw Internal: 4049048 Raw External: 6291456)
Sat, 17 Feb 2024 19:01:57 +0100 Internal: 3923.6796875KB External: 6144KB (Raw Internal: 4017848 Raw External: 6291456)
Sat, 17 Feb 2024 19:02:57 +0100 Internal: 3954.734375KB External: 6144KB (Raw Internal: 4049648 Raw External: 6291456)
Sat, 17 Feb 2024 19:03:57 +0100 Internal: 3934.8125KB External: 6144KB (Raw Internal: 4029248 Raw External: 6291456)
Sat, 17 Feb 2024 19:04:57 +0100 Internal: 3959.421875KB External: 6144KB (Raw Internal: 4054448 Raw External: 6291456)
Sat, 17 Feb 2024 19:05:57 +0100 Internal: 3964.109375KB External: 6144KB (Raw Internal: 4059248 Raw External: 6291456)
Sat, 17 Feb 2024 19:06:57 +0100 Internal: 3928.953125KB External: 6144KB (Raw Internal: 4023248 Raw External: 6291456)
Sat, 17 Feb 2024 19:07:57 +0100 Internal: 3956.4921875KB External: 6144KB (Raw Internal: 4051448 Raw External: 6291456)
Sat, 17 Feb 2024 19:08:57 +0100 Internal: 3964.6953125KB External: 6144KB (Raw Internal: 4059848 Raw External: 6291456)
Sat, 17 Feb 2024 19:09:57 +0100 Internal: 3929.5390625KB External: 6144KB (Raw Internal: 4023848 Raw External: 6291456)
Sat, 17 Feb 2024 19:10:57 +0100 Internal: 3950.6328125KB External: 6144KB (Raw Internal: 4045448 Raw External: 6291456)
Sat, 17 Feb 2024 19:11:57 +0100 Internal: 3962.9375KB External: 6144KB (Raw Internal: 4058048 Raw External: 6291456)
Sat, 17 Feb 2024 19:12:57 +0100 Internal: 3945.9453125KB External: 6144KB (Raw Internal: 4040648 Raw External: 6291456)
Sat, 17 Feb 2024 19:13:57 +0100 Internal: 3941.84375KB External: 6144KB (Raw Internal: 4036448 Raw External: 6291456)
Sat, 17 Feb 2024 19:14:57 +0100 Internal: 3937.7421875KB External: 6144KB (Raw Internal: 4032248 Raw External: 6291456)
Sat, 17 Feb 2024 19:15:57 +0100 Internal: 3947.703125KB External: 6144KB (Raw Internal: 4042448 Raw External: 6291456)
Sat, 17 Feb 2024 19:16:57 +0100 Internal: 3953.5625KB External: 6144KB (Raw Internal: 4048448 Raw External: 6291456)
Sat, 17 Feb 2024 19:17:57 +0100 Internal: 3948.875KB External: 6144KB (Raw Internal: 4043648 Raw External: 6291456)
Sat, 17 Feb 2024 19:18:57 +0100 Internal: 3936.5703125KB External: 6144KB (Raw Internal: 4031048 Raw External: 6291456)
Sat, 17 Feb 2024 19:19:57 +0100 Internal: 3947.703125KB External: 6144KB (Raw Internal: 4042448 Raw External: 6291456)
Sat, 17 Feb 2024 19:20:57 +0100 Internal: 3914.3046875KB External: 6144KB (Raw Internal: 4008248 Raw External: 6291456)
Sat, 17 Feb 2024 19:21:57 +0100 Internal: 3947.1171875KB External: 6144KB (Raw Internal: 4041848 Raw External: 6291456)
Sat, 17 Feb 2024 19:22:57 +0100 Internal: 3970.5546875KB External: 6144KB (Raw Internal: 4065848 Raw External: 6291456)
Sat, 17 Feb 2024 19:23:57 +0100 Internal: 3929.5390625KB External: 6144KB (Raw Internal: 4023848 Raw External: 6291456)
Sat, 17 Feb 2024 19:24:57 +0100 Internal: 3959.421875KB External: 6144KB (Raw Internal: 4054448 Raw External: 6291456)
Sat, 17 Feb 2024 19:25:57 +0100 Internal: 3955.3203125KB External: 6144KB (Raw Internal: 4050248 Raw External: 6291456)
Sat, 17 Feb 2024 19:26:57 +0100 Internal: 3924.265625KB External: 6144KB (Raw Internal: 4018448 Raw External: 6291456)
Sat, 17 Feb 2024 19:27:57 +0100 Internal: 3945.359375KB External: 6144KB (Raw Internal: 4040048 Raw External: 6291456)
Sat, 17 Feb 2024 19:28:57 +0100 Internal: 3936.5703125KB External: 6144KB (Raw Internal: 4031048 Raw External: 6291456)
Sat, 17 Feb 2024 19:29:57 +0100 Internal: 3939.5KB External: 6144KB (Raw Internal: 4034048 Raw External: 6291456)
Sat, 17 Feb 2024 19:30:57 +0100 Internal: 3956.4921875KB External: 6144KB (Raw Internal: 4051448 Raw External: 6291456)
Sat, 17 Feb 2024 19:31:57 +0100 Internal: 3951.21875KB External: 6144KB (Raw Internal: 4046048 Raw External: 6291456)
Sat, 17 Feb 2024 19:32:57 +0100 Internal: 3943.6015625KB External: 6144KB (Raw Internal: 4038248 Raw External: 6291456)
Sat, 17 Feb 2024 19:33:58 +0100 Internal: 3964.109375KB External: 6144KB (Raw Internal: 4059248 Raw External: 6291456)
Sat, 17 Feb 2024 19:34:58 +0100 Internal: 3926.0234375KB External: 6144KB (Raw Internal: 4020248 Raw External: 6291456)
Sat, 17 Feb 2024 19:35:58 +0100 Internal: 3935.3984375KB External: 6144KB (Raw Internal: 4029848 Raw External: 6291456)
Sat, 17 Feb 2024 19:36:57 +0100 Internal: 3939.5KB External: 6144KB (Raw Internal: 4034048 Raw External: 6291456)
Sat, 17 Feb 2024 19:37:58 +0100 Internal: 3958.25KB External: 6144KB (Raw Internal: 4053248 Raw External: 6291456)
Sat, 17 Feb 2024 19:38:58 +0100 Internal: 3944.7734375KB External: 6144KB (Raw Internal: 4039448 Raw External: 6291456)
Sat, 17 Feb 2024 19:39:58 +0100 Internal: 3944.7734375KB External: 6144KB (Raw Internal: 4039448 Raw External: 6291456)
Sat, 17 Feb 2024 19:40:58 +0100 Internal: 3954.1484375KB External: 6144KB (Raw Internal: 4049048 Raw External: 6291456)
Sat, 17 Feb 2024 19:41:58 +0100 Internal: 3950.6328125KB External: 6144KB (Raw Internal: 4045448 Raw External: 6291456)
Sat, 17 Feb 2024 19:42:58 +0100 Internal: 3936.5703125KB External: 6144KB (Raw Internal: 4031048 Raw External: 6291456)
Sat, 17 Feb 2024 19:43:58 +0100 Internal: 3911.1875KB External: 6144KB (Raw Internal: 4005056 Raw External: 6291456)
Sat, 17 Feb 2024 19:44:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:45:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:46:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:47:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:48:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:49:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:50:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:51:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:52:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:53:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:54:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:55:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:56:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:57:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:58:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 19:59:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 20:00:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)
Sat, 17 Feb 2024 20:01:58 +0100 Internal: 2327.59375KB External: 6144KB (Raw Internal: 2383456 Raw External: 6291456)

@WyriHaximus
Copy link
Member

While having a chat with @SimonFrings this morning, we realised I wasn't using a promise in the test. After adjusting the test we confirmed the reported issue very clearly:

Mon, 19 Feb 2024 09:12:14 +0100 Internal: 1667.7109375KB External: 2048KB (Raw Internal: 1707736 Raw External: 2097152)                                                                                    
Starting Event Loop
Mon, 19 Feb 2024 09:13:14 +0100 Internal: 1638539.09375KB External: 1648640KB (Raw Internal: 1677864032 Raw External: 1688207360)

Currently working on adding unit tests to confirm the clean up of those close calls.

WyriHaximus added a commit to WyriHaximus-labs/http that referenced this issue Feb 19, 2024
This changeset resolves a small memory leak that causes roughly 1KB per connection tops. Which isn't a big issue but will make memory fluctuate more. The changeset doesn't introduce any performance degradation.

Resolves: reactphp#514
Builds on top of: reactphp#405, reactphp#467, and many others
WyriHaximus added a commit to WyriHaximus-labs/http that referenced this issue Feb 19, 2024
This changeset resolves a small memory leak that causes roughly 1KB per connection tops. Which isn't a big issue but will make memory fluctuate more. The changeset doesn't introduce any performance degradation.

Resolves: reactphp#514
Builds on top of: reactphp#405, reactphp#467, and many others
WyriHaximus added a commit to WyriHaximus-labs/http that referenced this issue Feb 19, 2024
This changeset resolves a small memory leak that causes roughly 1KB per connection tops. Which isn't a big issue but will make memory fluctuate more. The changeset doesn't introduce any performance degradation.

Resolves: reactphp#514
Builds on top of: reactphp#405, reactphp#467, and many others
WyriHaximus added a commit to WyriHaximus-labs/http that referenced this issue Mar 8, 2024
This changeset resolves a small memory leak that causes roughly 1KB per connection tops. Which isn't a big issue but will make memory fluctuate more. The changeset doesn't introduce any performance degradation.

Resolves: reactphp#514
Builds on top of: reactphp#405, reactphp#467, and many others
@clue clue added this to the v1.10.0 milestone Mar 12, 2024
@SimonFrings SimonFrings changed the title Possible problematic code in persistent connections Memory leak when using persistent connections Mar 12, 2024
@SimonFrings
Copy link
Member

@mmoreram Thanks for bringing this up, #515 by @WyriHaximus is now merged and should fix the reported issue. I also changed the title of this ticket to better match the described topic 👍

If you'd like to support this development, consider sponsoring ReactPHP! ❤️ Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants