Developing workerman-memcached, and the problems of multiworker and shared memory!

[cayolblake]

Hi @walkor

Do you have plans for creating a workerman based memcached server? or may be a Redis one.

[walkor]

Sorry, no plan.

[walkor]

If someone did, welcome to share 🤝

[cayolblake]

Nvm, I'm almost done implementing it.

Will upload on GitHub once complete and passes all test cases.

Meanwhile, I'm stuck finding a way to communicate my storage (which is basically an array) and make it accessible across all workers.

[walkor]

Don't try to communicate between processes. It is very performance consuming.
To avoid inter process communication, Memcache and redis are both single process.

[xpader]

I had implement a beanstalkd server via workerman, but it's not finish yet.

[cayolblake]

@xpader great to hear that. May be share your implementation repo when you have a chance.

@walkor my memcached 1.6.14 does support multiple threads.

$ memcached --help | grep threads

-t, --threads=<num>       number of threads to use (default: 4)
   - ext_threads:         number of IO threads to run. (default: 1)

I'm done partially implementing the set/get parts of the memcache protocol, and since I have no way using multiple workerman workers, I am stuck with a single worker thread.

Benchmarking both memcached and workerman-memcached server I wrote, I am getting the following results.

Server	memcached	workerman-4.x	workerman-5.x
Read Op/sec	210 MB/s	140 MB/s	150 MB/s
Write Op/sec	20 MB/s	14 MB/s	15 MB/s
Threads	8	1	1

I didn't add the benchmark with php-pecl-event since it was horrible, at least %30 hit or more in performance, and with php-pecl-ev workerman interestingly always chose \Workerman\Events\Select. I also tested with Workerman\Events\Revolt as event loop and it was similarly bad results.

@walkor do you think there is a way/workaround to running more workers while accessing same memory?

The problem is not reaching high benchmark rates, it related to being able to run another HTTP server that will be serving such data over my main application will definitely be hit if run using a single worker thread.

[walkor]

https://www.php.net/manual/zh/book.sem.php
https://www.php.net/manual/zh/function.shm-get-var.php
Yes, these are worth trying.

[cayolblake]

Just done implementing shared memory using shmop which relies on shared memory segments as in here https://www.php.net/manual/en/ref.shmop.php, it worked fine, yet... performance was horrible, almost dropped down to 70 M/s which is around %30 of normal performance.

Also tried apcu, it worked fine, yet, was around 95 M/s which is around %60 of normal performance.

[nop1984]

So, memcached is better than SysvSharedMemory ?

[cayolblake]

Well,

It appears workerman having a single worker or multiple workers does not make ANY difference at all. You were right @walkor ! 👍

I narrowed down the problem to the fact that the retrieval of data from the PHP array where I hold storage data is the main issue.

I reached to this after many hours of testing and stripping the code over and over. I wanted to try Judy arrays but it seems it totally disappeared out of the PHP world and isn't supported anymore, so may be I'll write an extension for it.

[cayolblake]

@walkor any suggestions regarding a faster array-like data structure? 🤔

[walkor]

I think the array is fast enough, it is not a bottleneck. There will be no obvious difference in optimizing it.

[joanhey]

https://github.com/josephscott/marmerine