About

NOTE! This report still not finalized and result numbers is topic for discussion

The main goal of this benchmark report is to investigate state of PHP native possibilities to provide non-blocking HTTP server implementation. This report is focused on already existing solution Amphp HTTP server (aka Aerys).

This benchmark report is inspired by: https://github.com/squeaky-pl/japronto

TL;DR

Overall Aerys server performance is good. It is close to NodeJS performance, but still slower. Aerys latency distribution not much higher standard deviation in comparision with NodeJS. This benchmark did not show big difference between reactors. For some reason native PHP reactor has almost same performance as reactors based on ev, libevent or php-uv extensions. It is hard to compare ReactPHP with other servers implementations, because it doesn't support keeping connections alive. Comparing old version of Aerys and latest version of ReactPHP with non-keep-alive we can see that Aerys performing better.

Latest update: 25.03.2018

Testing environment

Hardware

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro9,2
      Processor Name: Intel Core i7
      Processor Speed: 2,9 GHz
      Number of Processors: 1
      Total Number of Cores: 2
      L2 Cache (per Core): 256 KB
      L3 Cache: 4 MB
      Memory: 8 GB

PHP

Used following PHP stable supported versions ^7.1, ^7.2, 7.3-dev + jit.

During benchmark we use default PHP settings by providing option -n that guarantee only required modules are loaded.

Native assertion framework is disabled with option:

zend.assertions=-1

OpCache might provide additional byte code optimizations. It is enabled for CLI with maximal optimization level:

opcache.enable=1
opcache.enable_cli=1
opcache.optimization_level=0xffffffff
opcache.file_update_protection=0

Development version of PHP with JIT support based on branch from Zend Github repository: https://github.com/zendtech/php-src/tree/jit-dynasm/

For benchmarking used default JIT settings with following OPCache settings adjustments:

opcache.jit_buffer_size=32M

libuv

Library version: v1.19.0
Extension version: 0.2.2

libevent

Library version: 2.1.8
Extension version: 2.3.0

ev

Library version: ???
Extension version: 1.0.4

NodeJS

Latest stable version ^8.9

By default NodeJS keeping connections alive without limiting timeout and connections amount:

Connection: keep-alive

Server response with keep alive (185 byte)

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 12
X-Powered-By: Node Server
Keep-Alive: timeout=10000
Date: Sun, 25 Mar 2018 18:38:33 GMT
Connection: keep-alive

Amphp HTTP server (aka Ayres)

To avoid connections rejections HTTP server must be configured with higher amount of simultaneous connections per one IP address:

connectionsPerIP = 100

By default HTTP server keeping connections alive with following settings:

keep-alive: timeout=6, max=999

We have to simulate similar keep-alive behavior with NodeJS so we set:

connectionTimeout = 10000

Server response with keep alive (185 byte):

HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
x-powered-by: AerysServer
connection: keep-alive
content-length: 12
keep-alive: timeout=10000
date: Sun, 25 Mar 2018 18:35:14 GMT

w/o keep alive (154 byte):

HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
x-powered-by: AerysServer
content-length: 12
connection: close
date: Sat, 20 Jan 2018 10:44:44 GMT

UvReactor, EvReactor, LibeventReactor, NativeReactor

ReactPHP

At this moment ReactPHP does not support keeping alive connection:

• https://github.com/reactphp/http/blob/master/README.md#response
• reactphp/http#39

Connection: close

Server response w/o keep alive (154 byte)

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
X-Powered-By: React/alpha
Date: Sat, 20 Jan 2018 10:43:06 GMT
Content-Length: 13
Connection: close

HTTP benchmarking tool

wrk version 4.0.2

Testing settings:

• 1 thread
• 100 connections
• during 30 seconds

wrk -t1 -c100 -d30s --latency http://127.0.0.1:8080/

Build Docker images

To build Docker images just run:

sh ./build.sh

Run benchmark

To run benchmark you need installed Docker. Following command will fetch latest image from Docker hub with build-in all needed software and start benchmark:

docker pull dmitrybalabka/aerys-benchmark:${DOCKER_PHP_VERSION}
docker run --ulimit nofile=30000:30000 dmitrybalabka/aerys-benchmark:${DOCKER_PHP_VERSION}

Replace ${DOCKER_PHP_VERSION} with any available image tag (php71, php72, php72jit)

Run local copy of benchmark in docker

git clone git@github.com:torinaki/aerys-benchmark.git
cd aerys-benchmark
sh ./build/install-local.sh
docker pull dmitrybalabka/aerys-benchmark:php71
docker run --ulimit nofile=30000:30000 -v "`pwd`:/app" dmitrybalabka/aerys-benchmark:php71

Benchmark result

Server	Settings	50%	75%	90%	99%
ReactPHP	w/o keep-alive
ReactPHP	w/o keep-alive + OPCache
ReactPHP	w/o keep-alive + OPCache + JIT
Aerys + Amp1	w/o keep-alive
Aerys + Amp1	w/o keep-alive + OPCache
Aerys + Amp1	w/o keep-alive + OPCache + JIT
Aerys + Amp1	keep-alive + OPCache + JIT
Aerys + Amp2	keep-alive + OPCache + JIT
Aerys + Amp2 tiny	keep-alive + OPCache + JIT
Aerys + Amp2	keep-alive + OPCache
Aerys + Amp2	keep-alive + OPCache + JIT + ev
Aerys + Amp2	keep-alive + OPCache + JIT + event
Aerys + Amp2	keep-alive + OPCache + JIT + uv
NodeJS	keep-alive + w/o timeout