Rdebug is an open source tool chain that focusing on efficiency of daily developing, testing and debugging for developers. The code will be robust.
As micro-service architecture makes services isomerous, easy-to-deploy and superbly extensible, large systems are divided into several micro services.
But, micro-service architecture is not a silver bullet. It has several drawbacks. One service is divided into several small services, test environment becomes hard to maintain. Unit test needs to mock downstream services. This will downgrade the efficiency of developing, and increase production bugs.
We do believe this industry needs an evolution.
Efficiency improvement, release cycle reduction, code quality improvement, production bug reduction
Suitable for old api refactoring that traffic have already been recorded.
Not suitable for new api or api that has no traffic recorded.
Support for new api is under investigation.
# Modify php-fpm config, set `clear_env = no`
# Compile koala-libc.so & koala-recorder.so
# Start php-fpm with koala-libc.so & koala-recorder.so
# Set environment first
$ export KOALA_SO=/path/to/koala-recorder.so
$ export KOALA_RECORD_TO_DIR=/path/to/your-save-recorded-session-dir
$ export LC_CTYPE="C"
# Recording
# macOS
$ DYLD_INSERT_LIBRARIES="/path/to/koala-libc.so:/usr/lib/libcurl.dylib" DYLD_FORCE_FLAT_NAMESPACE="y" /path/to/sbin/php-fpm
# or, Linux
$ LD_PRELOAD="/path/to/koala-libc.so /usr/lib64/libcurl.so.4" /path/to/sbin/php-fpmThree ways to replay: replay from source, replay by midi.phar, replay by composer vendor/bin.
# Source
$ git clone https://github.com/didi/rdebug.git
$ cd rdebug/php/midi
$ sh install.sh
$ cd /path/to/your/project
$ /path/to/rdebug/php/midi/bin/midi run -f RECORD-SESSION-FILE
# Or, Phar
$ wget -O midi.phar -q https://github.com/didi/rdebug/raw/master/output/bin/midi.phar
$ midi.phar run -f RECORD-SESSION-FILE
# Or, Composer global
$ composer global require rdebug/midi
$ cd /path/to/your/project
$ ~/.composer/vendor/bin/midi run -f RECORD-SESSION-FILE
# Or, Composer
$ cd /path/to/your/project
$ composer require rdebug/midi --dev
$ ./vendor/bin/midi run -f RECORD-SESSION-FILEWe are using real production traffic to test on non-production environment. So, the core features of Rdebug are traffic recording and traffic replaying.
-
Traffic Recording
Recording production traffic, including RPC between micro-services. The most difficult part is to trace requests that will span multiple services.
-
Traffic Replaying
Replaying recorded production traffic against non-production environment, mock downstream traffic based on traffic matching. The most difficult part is interception and traffic matching.
Koala and Koala-libc is the engine of Rdebug.
Because the problems such as RPC tracing, libc interception that a traffic replaying system may concern is what a traffic recording system concerns. The traffic recording and replaying system shares the same engine.
Koala is written in Go, libc contains some c++ code for interception part which will complied to koala-recorder.so and koala-replayer.so
Koala support replaying and recording separately as well as simultaneously.
For more details: koala 。
koala-libc is written in C, and compiled to koala-libc.so
Go does not support fork. To avoid problems bring by PHP-FPM fork worker, you need to inject koala-libc.so into parent process of PHP. So that koala-libc.so will forward requests libc intercepted to *koala-recorder.so when accept in child process
For more details: koala-libc 。
Midi is a traffic replay client for PHP using command line written in PHP.
koala-replayer.so is embedded in Midi, and it can replay production traffic against new code, analyze test result, generate diff report, trace report, coverage report.
Midi support Xdebug.
For more details: midi 。
- GCC >= 4.8
- Go >= 1.8
- Glide
- macOS/Linux
- PHP >= 7.0
- Xdebug (Optional)
- Composer (Optional)
$ cd koala-libc
$ sh build.shwill generate ../output/libs/koala-libc.so。
$ cd koala
# install depends
$ sh build.sh vendor
# koala-recorder.so
$ sh build.sh recorder
# koala-replayer.so
$ sh build.shwill generate ../output/libs/koala-recorder.so and ../output/libs/koala-replayer.so。
Before compile midi.pahr, recommend compile koala-replayer.so at your system and store in php/midi/res/replayer directory.
php/midi/res/replayer/koala-replayer.so in repository is for macOS.
$ cd php/midi
$ sh build.shwill generate ../output/bin/midi.phar
Default phar will not include DiPlugin plugin, which is an internal plugin of didi.
If your want to compile with DiPlugin plugin:
$ cd php/midi
$ sh build.sh midi-diplugin
Record traffic in file or Elastic.
Set environment variable KOALA_RECORD_TO_DIR to save record to specified dir
Inject so. Use DYLD_INSERT_LIBRARIES in macOs or LD_PRELOAD in Linux
For more details: Recorder。
$ DYLD_INSERT_LIBRARIES="/path/to/koala-libc.so:/usr/lib/libcurl.dylib" DYLD_FORCE_FLAT_NAMESPACE="y" LC_CTYPE="C" KOALA_SO=/path/to/koala-recorder.so KOALA_RECORD_TO_DIR=/tmp /usr/local/sbin/php-fpmLD_PRELOAD="/path/to/koala-libc.so /usr/lib64/libcurl.so.4" LC_CTYPE="C" KOALA_SO="/path/to/koala-recorder.so" KOALA_RECORD_TO_DIR=/tmp /usr/local/sbin/php-fpmKoala support writing to Elastic by replacing KOALA_RECORD_TO_DIR with KOALA_RECORD_TO_ES which value is elastic write url
Traffic replaying is to replay recorded traffic in production against non-production environment. You need only to deploy your code w/o deploying downstream services.
Below is a simple example of replaying with '-f', which specified the file to replay.
# Source
$ /path/to/rdebug/php/midi/bin/midi run -f RECORD-SESSION-FILE
# Or, phar
$ midi.phar run -f RECORD-SESSION-FILE
# Or, composer vendor bin
$ ./vendor/bin/midi -f RECORD-SESSION-FILEFor more details: Replay
Use -v, -vv or -vvv for more verbose log.
Use -R，-T，-C to generate：Replay report, Trace report, Coverage report
For more details: Midi。
Any contributions are welcome. More Details
- QQ Group
Rdebug is licensed under the Apache 2.0 License. LICENSE
TaoWen Developer of Koala & Koala-libc
TanMingliang, WangBizhou, FangJunda, YangJing, YangBochen and LiXiaodong Developer of Midi
A special thanks to Symfony、Xdebug、PHPUnit & Code Coverage、guzzlehttp…… contributors, develop such a useful software.
Also thanks to DingWei Wujun, FanYitian, ZhaoLu, XuKaiwen, HuXu, FuYao and HuMin for suggestions.