EventBase::loop(): kevent: Bad file descriptor

[slince]

hello,

I'm writing forking TCP server with master and forked worker processes. the ExtEventLoop instance in child process will raise some error like this:

Warning: EventBase::looPHP Warning:  EventBase::loop(): kevent: Bad file descriptor in /Users/xxx/vendor/react/event-loop/src/ExtEventLoop.php on line 205

Warning: EventBase::loop(): kevent: Bad file descriptor in /Users/xxxxx/vendor/react/event-loop/src/ExtEventLoop.php on line 205
tserver/vendor/react/event-loop/src/ExtEventLoop.php on line 205

Sample code:

include __DIR__ . '/vendor/autoload.php';

$loop = \React\EventLoop\Loop::get();
$server = new \React\Socket\SocketServer('tcp://127.0.0.1:1234', [], $loop);

const WORKER_NUM = 3;

for ($i = 0; $i <= WORKER_NUM; $i++) {
    $pid = pcntl_fork();
    if (-1 === $pid) {
        exit('fork error');
    } elseif ($pid) {
       
    } else {
        // run the server.
        $loop->run();
        exit;
    }
}
// close in master process.
$server->close();
pcntl_wait($status);

environment:

php: 7.4.25 NTS
os: macos 11.5.2
machine: Macbook pro(Apple M1)

Reference https://stackoverflow.com/questions/33735302/reactphp-libevent-and-socket-pair-throws-error

Is it possible to add a method getEventBase() in https://github.com/reactphp/event-loop/blob/master/src/ExtEventLoop.php to expose the raw eventBase object?

[slince]

@clue @WyriHaximus Can anyone pay attention to this issue, tks

[WyriHaximus]

@slince We can expose the raw event base, but why not start the server after forking. Or use react/child-process to spawn more server processes?

[slince]

@WyriHaximus

hello

why not start the server after forking

If so, then we need to enable the so_reuseport option of the socket. After my local test, using reuseport is not as efficient as using the child process fork directly

use react/child-process to spawn more server processes?

fork is more convenient than exec when ipc; in fact, I also implemented exec in my project, but this is the second option, and exec will only be used when fork is not available.

In fact, i have an idea. i can create an socket server importing an existing socket stream . before this, i have a discusssion #220

// create raw socket befor fork.
$rawSocket = stream_socket_server('xxxx');

const WORKER_NUM = 3;

for ($i = 0; $i <= WORKER_NUM; $i++) {
    $pid = pcntl_fork();
    if (-1 === $pid) {
        exit('fork error');
    } elseif ($pid) {
       
    } else {
        // create a server  using raw socket
        $loop = \React\EventLoop\Loop::get();
        $server = new \React\Socket\SocketServer($rawSocket,  $loop);
        $loop->run();
        exit;
    }
}
pcntl_wait($status);

[clue]

@slince Thanks for bringing this up, excellent question!

Whether SO_REUSEPORT scales better or not indeed depends on the protocol and application workload, but from my experiments I would say this is usually preferable. As an alternative, I would highly recommend using the ChildProcess component and/or duplicating the file descriptor as recently implemented in #269.

PHP's pcntl_fork() is quite low-level and unfortunately PHP only provides somewhat limited support for low-level OS operations, so this may open up a can of worms due to the PHP's limited access to low-level APIs. PHP provides higher-level APIs for process creation as used in our ChildProcess component. These APIs internally use the underlying fork() and exec() calls but avoid many of their shortcoming. Once PHP offers more low-level APIs in the future (close(), dup() etc. come to mind), I agree it might make sense to revisit this low-level API, but I don't see this anywhere in the foreseeable future at least. This refs reactphp/event-loop#184 and others.

The underlying problem is that libev and others rely on epoll/kevent which use a file descriptor which will be inherited to any child processes when forking the parent. As a consequence, any child process may affect events received by the parent process and vice versa. The underlying libev and others implement low-level APIs to essentially re-create this file descriptor in the child process, which is not currently exposed in our EventLoop API because it is somewhat specific to the platform and/or extension in use. For more background, see the above ticket and https://metacpan.org/dist/EV/view/libev/ev.pod#The-special-problem-of-fork

I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can always reopen this 👍

[slince]

PHP's pcntl_fork() is quite low-level and unfortunately PHP only provides somewhat limited support for low-level OS operations, so this may open up a can of worms due to the PHP's limited access to low-level APIs.

@clue hi I think fork is worthy using, you can refer to workerman, it has been working very well and has been trusted by everyone;

[clue]

@slince Forking works just fine, also in ReactPHP. Like I said, using the pcntl_fork() function is very low-level and easy to get wrong, because the defaults (while making sense from a historical perspective) don't really fit into long-running, event-driven applications, so you have to do a lot of legwork (which I'd rather avoid). If this sounds confusing, it's because it is, and I would recommend looking into using the ChildProcess component as an easier way to use forks.

If you really want to use the low-level forking functions in PHP to create a shared file descriptor referencing a socket server, here's how you can do this:

<?php

require __DIR__ . '/vendor/autoload.php';

// create raw socket befor fork.
$rawSocketFd = 3; // TODO: low-level stuff not exposed in PHP, but safe to assume in here if you don't have any other file descriptors except stdio
$rawSocket = stream_socket_server('xxxx');

const WORKER_NUM = 3;

for ($i = 0; $i <= WORKER_NUM; $i++) {
    $pid = pcntl_fork();
    if (-1 === $pid) {
        exit('fork error');
    } elseif ($pid) {
       
    } else {
        // create a server  using raw socket
        $loop = \React\EventLoop\Loop::get();
        $server = new \React\Socket\SocketServer('php://fd/' . $rawSocketFd,  $loop);
        $loop->run();
        exit;
    }
}
pcntl_wait($status);

The above example should work just fine with ReactPHP, but again I would not recommend using this unless you have a very good understanding of how lower-level file descriptors work. In particular, you may find that PHP does not expose file descriptor numbers at all, does not allow you to get a FD number for a resource or even close an FD (see #269, https://github.com/clue/fd and others).

I hope this helps 👍