The 502 error occasionally occurs in the php7.3 version, and the SkyAPM-php-sdk used is version 3.3.2

[xinfeingxia85]

Describe the bug
When the SkyAPM-php-sdk used is version 3.3.2, the 502 bad gateway occasionally ocuurs in the php7.3 version。
But the errors don't ocuurs in the php7.0 and 7.1 version。
The development environment uses php-fpm+nginx to run, and the traffic is not large。
Skywaling's log collection is normal, that is, 502 occasionally appears in version 7.3。

PHP (please complete the following information):

• OS: Ubuntu 18.04.1 LTS (bionic)
• Version:
  PHP 7.3.12-1+ubuntu18.04.1+deb.sury.org+1 (cli) (built: Nov 28 2019 07:37:16) ( NTS )
  Copyright (c) 1997-2018 The PHP Group
  Zend Engine v3.3.12, Copyright (c) 1998-2018 Zend Technologies
  with Zend OPcache v7.3.12-1+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies

Additional context
nginx version: nginx/1.14.0 (Ubuntu)
nginx error log
2020/09/11 16:19:43 [error] 16467#16467: *18964 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 127.0.0.1, server: xxxx, request: "GET /api/tool/coco/tag/list?course_id=824901229 HTTP/1.0", upstream: "fastcgi://127.0.0.1:9073", host: "xxx.com"

How should I fix this problem?

[heyanlong]

Open coredump to view the reason

[xinfeingxia85]

Open coredump to view the reason

GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/php-fpm7.3...(no debugging symbols found)...done.
[New LWP 24795]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `php-fpm: pool www '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000555c04c25570 in zend_hash_real_init_mixed ()
(gdb) bt
#0 0x0000555c04c25570 in zend_hash_real_init_mixed ()
#1 0x0000555c04c262b5 in zend_hash_add ()
#2 0x0000555c04baa2b1 in ?? ()
#3 0x00007fe97330da35 in sky_execute_internal (execute_data=0x7fe98381d340, return_value=0x7ffedd2d71a0)
at /tmp/source/skywalking/skywalking.c:800
#4 0x0000555c04ca7f8e in execute_ex ()
#5 0x00007fe97330b6ea in sky_execute_ex (execute_data=0x7fe98381d2e0) at /tmp/source/skywalking/skywalking.c:403
#6 0x0000555c04c9b69c in ?? ()
#7 0x0000555c04ca076f in execute_ex ()
#8 0x00007fe97330b6ea in sky_execute_ex (execute_data=0x7fe98381d250) at /tmp/source/skywalking/skywalking.c:403
#9 0x0000555c04ca8547 in execute_ex ()
#10 0x00007fe97330b6ea in sky_execute_ex (execute_data=0x7fe98381d140) at /tmp/source/skywalking/skywalking.c:403
#11 0x0000555c04ca6f07 in execute_ex ()
#12 0x00007fe97330b6ea in sky_execute_ex (execute_data=0x7fe98381d0e0) at /tmp/source/skywalking/skywalking.c:403
#13 0x0000555c04c9af4c in ?? ()
#14 0x0000555c04ca1e84 in execute_ex ()
#15 0x00007fe97330b6ea in sky_execute_ex (execute_data=0x7fe98381d030) at /tmp/source/skywalking/skywalking.c:403
#16 0x0000555c04ca8683 in zend_execute ()
#17 0x0000555c04c193e2 in zend_execute_scripts ()
#18 0x0000555c04bb9430 in php_execute_script ()
#19 0x0000555c04a7297b in ?? ()
#20 0x00007fe986d1ab97 in __libc_start_main (main=0x555c04a71af0, argc=4, argv=0x7ffedd2da418, init=,
fini=, rtld_fini=, stack_end=0x7ffedd2da408) at ../csu/libc-start.c:310
#21 0x0000555c04a736fa in _start ()

[heyanlong]

Can you provide the PHP program that went wrong?

[xinfeingxia85]

Can you provide the PHP program that went wrong?

There is no error in the code, operating the database and redis and http calls. That is, there is a 502 error, and the code is associated with more content.
I want to ask, does the master branch need to install go agent?

[heyanlong]

No, Agent loaded inside

[xinfeingxia85]

@heyanlong

This problem should be that the php-fpm process will be killed for a certain number of concurrent curls, but the specific problem has not been resolved.
Is there any way to solve this problem? When will the next version be released?
Thank you！！

[heyanlong]

Provide an example

[xinfeingxia85]

Provide an example

There is too much code relevance to provide.
My own simulation did not achieve similar errors.
PHP package errors are Out-of-memory errors
as follows:
[17-Sep-2020 18:45:35 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115166720 bytes) in Unknown on line 0
[17-Sep-2020 18:45:50 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115162624 bytes) in Unknown on line 0
[17-Sep-2020 18:45:51 Asia/Chongqing] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115146240 bytes) in Unknown on line 0
[17-Sep-2020 18:45:51 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115166720 bytes) in Unknown on line 0
[17-Sep-2020 18:47:11 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115162624 bytes) in Unknown on line 0
[17-Sep-2020 18:47:11 Asia/Chongqing] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115146240 bytes) in Unknown on line 0
[17-Sep-2020 18:47:11 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115166720 bytes) in Unknown on line 0
[17-Sep-2020 18:47:25 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115162624 bytes) in Unknown on line 0
[17-Sep-2020 18:47:25 Asia/Chongqing] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115146240 bytes) in Unknown on line 0
[17-Sep-2020 18:47:25 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115166720 bytes) in Unknown on line 0
[17-Sep-2020 19:04:19 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115162624 bytes) in Unknown on line 0
[17-Sep-2020 19:04:50 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115162624 bytes) in Unknown on line 0
[17-Sep-2020 19:04:51 Asia/Chongqing] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115146240 bytes) in Unknown on line 0
[17-Sep-2020 19:04:51 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115166720 bytes) in Unknown on line 0
[17-Sep-2020 19:05:30 PRC] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 140429115162624 bytes) in Unknown on line 0

What could be the cause?

[xinfeingxia85]

@heyanlong

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `php-fpm: pool ww'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 zend_hash_find_bucket (known_hash=0 '\000', key=0x7f6792103e50, ht=0x7f679dc66770)
at /usr/local/src/php-7.3.22/Zend/zend_hash.c:595
595 if (p->key == key) { /* check for the same interned string */
(gdb) bt
#0 zend_hash_find_bucket (known_hash=0 '\000', key=0x7f6792103e50, ht=0x7f679dc66770)
at /usr/local/src/php-7.3.22/Zend/zend_hash.c:595
#1 _zend_hash_add_or_update_i (flag=2, pData=0x7ffd981c5790, key=0x7f6792103e50, ht=0x7f679dc66770)
at /usr/local/src/php-7.3.22/Zend/zend_hash.c:670
#2 zend_hash_add (ht=0x7f679dc66770, key=0x7f6792103e50, pData=pData@entry=0x7ffd981c5790)
at /usr/local/src/php-7.3.22/Zend/zend_hash.c:812
#3 0x0000555d68ca6f61 in zend_hash_add_ptr (pData=0x7f679dc81020, key=, ht=)
at /usr/local/src/php-7.3.22/Zend/zend_hash.h:570
#4 zif_stream_filter_register (execute_data=, return_value=0x7ffd981c5930)
at /usr/local/src/php-7.3.22/ext/standard/user_filters.c:588
#5 0x00007f6799d27a35 in sky_execute_internal (execute_data=0x7f679dc1d340, return_value=0x7ffd981c5930)
at /tmp/source/skywalking/skywalking.c:803
#6 0x0000555d68ddba3b in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER ()
at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:986
#7 execute_ex (ex=0x7f679dc66770) at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:55527
#8 0x00007f6799d256ea in sky_execute_ex (execute_data=0x7f679dc1d2e0) at /tmp/source/skywalking/skywalking.c:404
#9 0x0000555d68dcefac in ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER ()
at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:37498
#10 0x0000555d68dd3fff in execute_ex (ex=0x7f679dc66770) at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:59367
#11 0x00007f6799d256ea in sky_execute_ex (execute_data=0x7f679dc1d250) at /tmp/source/skywalking/skywalking.c:404
#12 0x0000555d68dd92e5 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER ()
at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:965
#13 execute_ex (ex=0x7f679dc66770) at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:55527
#14 0x00007f6799d256ea in sky_execute_ex (execute_data=0x7f679dc1d140) at /tmp/source/skywalking/skywalking.c:404
#15 0x0000555d68dd9dde in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER ()
at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:1087
#16 execute_ex (ex=0x7f679dc66770) at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:55531
#17 0x00007f6799d256ea in sky_execute_ex (execute_data=0x7f679dc1d0e0) at /tmp/source/skywalking/skywalking.c:404
#18 0x0000555d68dce85c in ZEND_INCLUDE_OR_EVAL_SPEC_CONST_HANDLER ()
at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:3300
#19 0x0000555d68dd5733 in execute_ex (ex=0x7f679dc66770) at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:55813
#20 0x00007f6799d256ea in sky_execute_ex (execute_data=0x7f679dc1d030) at /tmp/source/skywalking/skywalking.c:404
#21 0x0000555d68ddbede in zend_execute (op_array=op_array@entry=0x7f679dc85000, return_value=return_value@entry=0x0)
at /usr/local/src/php-7.3.22/Zend/zend_vm_execute.h:60939
#22 0x0000555d68d4dbd2 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0,
file_count=file_count@entry=3) at /usr/local/src/php-7.3.22/Zend/zend.c:1568
#23 0x0000555d68cee7b0 in php_execute_script (primary_file=0x7ffd981c8650)
at /usr/local/src/php-7.3.22/main/main.c:2637
#24 0x0000555d688f234b in main (argc=, argv=)
at /usr/local/src/php-7.3.22/sapi/fpm/fpm/fpm_main.c:1951

Could you please help to see the specific problem

gdb) zbacktrace
[0x7f679dc1d340] stream_filter_register("chunk", "Http\Message\Encoding\Filter\Chunk") [internal function]
[0x7f679dc1d2e0] (main) /xxxxx/12fd404ae2113912fa54c4c3d1a4a549841f746e/vendor/php-http/message/src/filters.php:5

https://github.com/php-http/message/blob/v1.7.1/src/filters.php

[heyanlong]

Caused by stream_filter_register?

[xinfeingxia85]

Caused by stream_filter_register?

Zbacktrace debugging with php.gdbinit looks like this.
Is there a temporary solution?