PHP :: Bug #77955 :: Random segmentation fault in mysqlnd from php-fpm

Bug #77955

Random segmentation fault in mysqlnd from php-fpm

Submitted:

2019-05-01 12:04 UTC

Modified:

2019-06-03 07:16 UTC

Votes:	11
Avg. Score:	4.9 ± 0.3
Reproduced:	11 of 11 (100.0%)
Same Version:	11 (100.0%)
Same OS:	9 (81.8%)

From:

victoria at sharksmedia dot dk

Assigned:

dmitry (profile)

Status:

Closed

Package:

PDO MySQL

PHP Version:

7.3.4

OS:

CentOS 7.6.1810

Private report:

CVE-ID:

None

View Developer Edit

[2019-05-01 12:04 UTC] victoria at sharksmedia dot dk

Description:
------------
We are running a medium sized ecommerce site on a custom code base, and since php 7.3.1 was released in january, we have been trying to upgrade from 7.2.x to newest 7.3.x but have faced random segmentation faults on each release.

With a backend traffic in the early moning hours of 20-40 req/sec we see ~8-12 segmentation faults per hour (php-fpm workers). Despite having tried hard we are unable to trigger the fault on demand, the segmentation fault is almost exclusively triggered by visiting customers seemingly at random.

We are using the remi repository for all php modules.

Maybe relevant:
We are seeing a lot of "Allowed memory size of x bytes exhausted" in database intensive cronjobs when upgrading, hinting at a memory related issue with pdo/mysqlnd but no segmentation faults in cronjobs.
Everything runs smoothly on 7.2.18

Relevant Versions
PHP: 7.3.4
PDO MySQL / Mysqlnd: mysqlnd 5.0.12-dev

Installed extensions
php-pecl-scrypt
php-cli
php-gd
php-pecl-igbinary
php-pecl-imagick
php-pecl-vld
php-pecl-uuid
php-pdo
php-xml
php-pecl-hrtime
php-pecl-ssh2
php-json
php-process
php-sodium
php-mbstring
php-brotli
php-pecl-oauth
php-common
php-intl
php-opcache
php-bcmath
php-soap
php-pecl-redis4
php-pecl-varnish
php-mysqlnd
php-pecl-xdebug
php-snuffleupagus
php-fpm
php-gmp

We thought we were affected by following but apparently not, but it sounds similar (we are using a mix of buffered and unbuffered queries, with native prepared statements)
https://bugs.php.net/bug.php?id=77308
https://bugs.php.net/bug.php?id=77430

Thanks for your time, it is greatly appreciated.

Actual result:
--------------
Bactrace with xDebug disabled:

# gdb /usr/sbin/php-fpm core.31927
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/php-fpm...Reading symbols from /usr/lib/debug/usr/sbin/php-fpm.debug...done.
done.
[New LWP 31927]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `php-fpm: pool bm_live          '.
Program terminated with signal 11, Segmentation fault.
#0  php_mysqlnd_free_field_metadata (meta=0x7f4929401018) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:38
38			if (meta->sname) {
(gdb) bt
#0  php_mysqlnd_free_field_metadata (meta=0x7f4929401018) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:38
#1  mysqlnd_mysqlnd_res_meta_free_pub (meta=0x7f492977a718) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:106
#2  0x00007f49392c3daa in mysqlnd_mysqlnd_res_free_result_contents_internal_pub (result=0x7f492977a048) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:300
#3  0x00007f49392c46a0 in mysqlnd_mysqlnd_res_free_result_internal_pub (result=0x7f492977a048) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:316
#4  0x00007f49392c444b in mysqlnd_mysqlnd_res_free_result_pub (result=0x7f492977a048, implicit=<optimized out>) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:1498
#5  0x00007f49363fc9d9 in pdo_mysql_stmt_cursor_closer (stmt=<optimized out>) at /usr/src/debug/php-7.3.4/ext/pdo_mysql/mysql_statement.c:901
#6  0x00007f4939098381 in zim_PDOStatement_closeCursor (execute_data=<optimized out>, return_value=0x7ffc13242710) at /usr/src/debug/php-7.3.4/ext/pdo/pdo_stmt.c:2089
#7  0x000055ced84959c1 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:980
#8  execute_ex (ex=0x7f492977a718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55485
#9  0x000055ced8497e93 in zend_execute (op_array=op_array@entry=0x7f4948e7c000, return_value=0x0, return_value@entry=0x7f492a3d84d0) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:60881
#10 0x000055ced8408bc2 in zend_execute_scripts (type=type@entry=8, retval=0x7f492a3d84d0, retval@entry=0x0, file_count=1222766048, file_count@entry=3) at /usr/src/debug/php-7.3.4/Zend/zend.c:1568
#11 0x000055ced83a88c0 in php_execute_script (primary_file=primary_file@entry=0x7ffc13244d20) at /usr/src/debug/php-7.3.4/main/main.c:2630
#12 0x000055ced820cc0b in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/php-7.3.4/sapi/fpm/fpm/fpm_main.c:1950
(gdb)


Backtrace with xDebug enabled (our production setup)

gdb /usr/sbin/php-fpm core.24492
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/php-fpm...Reading symbols from /usr/lib/debug/usr/sbin/php-fpm.debug...done.
done.
[New LWP 24492]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `php-fpm: pool bm_live          '.
Program terminated with signal 11, Segmentation fault.
#0  php_mysqlnd_free_field_metadata (meta=0x7faee1e01018) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:38
38			if (meta->sname) {
(gdb) bt
#0  php_mysqlnd_free_field_metadata (meta=0x7faee1e01018) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:38
#1  mysqlnd_mysqlnd_res_meta_free_pub (meta=0x7faee21fc718) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:106
#2  0x00007faef327ddaa in mysqlnd_mysqlnd_res_free_result_contents_internal_pub (result=0x7faee21fc048) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:300
#3  0x00007faef327e6a0 in mysqlnd_mysqlnd_res_free_result_internal_pub (result=0x7faee21fc048) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:316
#4  0x00007faef327e44b in mysqlnd_mysqlnd_res_free_result_pub (result=0x7faee21fc048, implicit=<optimized out>) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:1498
#5  0x00007faef03b69d9 in pdo_mysql_stmt_cursor_closer (stmt=<optimized out>) at /usr/src/debug/php-7.3.4/ext/pdo_mysql/mysql_statement.c:901
#6  0x00007faef3052381 in zim_PDOStatement_closeCursor (execute_data=<optimized out>, return_value=0x7ffe05bfa1c8) at /usr/src/debug/php-7.3.4/ext/pdo/pdo_stmt.c:2089
#7  0x00007faefc61ce75 in xdebug_execute_internal (current_execute_data=0x7faf0301da00, return_value=0x7ffe05bfa1c8) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:2050
#8  0x0000557196c3891d in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:982
#9  execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55485
#10 0x00007faf030b16b8 in ?? ()
#11 0x0000557196e321d2 in zend_llist_apply_with_argument (l=<optimized out>, func=0x557196c3891d <execute_ex+4292297949>, arg=0x7faee43bd9d0) at /usr/src/debug/php-7.3.4/Zend/zend_llist.c:234
#12 0x00007faefc84fba0 in ?? () from /usr/lib64/php/modules/xdebug.so
#13 0x00005571991df130 in ?? ()
#14 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#15 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d930) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#16 0x0000557196c38be8 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:1083
#17 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#18 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d830) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#19 0x0000557196c38be8 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:1083
#20 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#21 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d780) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#22 0x0000557196c38be8 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:1083
#23 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#24 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d680) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#25 0x0000557196c388aa in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:961
#26 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#27 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d590) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#28 0x0000557196c388aa in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:961
#29 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#30 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d490) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#31 0x0000557196ebfeac in ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:37708
#32 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#33 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d1e0) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#34 0x0000557196c388aa in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:961
#35 0x0000557196ec6768 in execute_ex (ex=0x7faee21fc718) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:55557
#36 0x00007faefc61c5dd in xdebug_execute_ex (execute_data=0x7faf0301d030) at /usr/src/debug/php-pecl-xdebug-2.7.1/NTS/xdebug.c:1928
#37 0x0000557196ecce93 in zend_execute (op_array=op_array@entry=0x7faf03061380, return_value=return_value@entry=0x0) at /usr/src/debug/php-7.3.4/Zend/zend_vm_execute.h:60881
#38 0x0000557196e3dbc2 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at /usr/src/debug/php-7.3.4/Zend/zend.c:1568
#39 0x0000557196ddd8c0 in php_execute_script (primary_file=primary_file@entry=0x7ffe05bfd220) at /usr/src/debug/php-7.3.4/main/main.c:2630
#40 0x0000557196c41c0b in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/php-7.3.4/sapi/fpm/fpm/fpm_main.c:1950
(gdb)

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports

[2019-05-01 12:36 UTC] victoria at sharksmedia dot dk

php-fpm error log excerpt:

[01-May-2019 05:50:26] WARNING: [pool bm_live] child 24492 exited on signal 11 (SIGSEGV - core dumped) after 424.195966 seconds from start
[01-May-2019 05:57:36] WARNING: [pool bm_live] child 24494 exited on signal 11 (SIGSEGV - core dumped) after 854.271177 seconds from start
[01-May-2019 05:58:29] WARNING: [pool bm_live] child 28154 exited on signal 11 (SIGSEGV - core dumped) after 53.852203 seconds from start
[01-May-2019 06:03:36] WARNING: [pool bm_live] child 28484 exited on signal 11 (SIGSEGV - core dumped) after 294.918476 seconds from start
[01-May-2019 06:08:02] WARNING: [pool bm_live] child 31927 exited on signal 11 (SIGSEGV - core dumped) after 0.909109 seconds from start

Just to state what might be obvious; we have not been able to identify the peice of php code that is responsible for the segfault, the fault is too rare.

[2019-05-03 10:08 UTC] sjon at hortensius dot net

You could try reverting https://github.com/php/php-src/commit/5eb1f92f31cafc48384f9096012f421b37f6d425 locally (at least in ext/mysqlnd/mysqlnd_result_meta.c) as that seems the most relevant change from your backtrace and only appeared in 7.3

[2019-05-07 09:16 UTC] sjon@php.net

Related To: Bug #77734

[2019-05-07 09:30 UTC] sjon@php.net

-Status: Open +Status: Assigned -Assigned To: +Assigned To: dmitry

[2019-05-07 09:30 UTC] sjon@php.net

Dimitry, could you confirm/deny this might be caused by https://github.com/php/php-src/commit/5eb1f92f31cafc48384f9096012f421b37f6d425 ?

[2019-05-07 10:09 UTC] victoria at sharksmedia dot dk

Thanks for the suggestion,

For your information; i have been trying to get Remi's package setup working on my server in order recompile and test this, but i have not had much success building php-mysqlnd yet, i'm currently waiting for a reply from Remi.

If you happen to know Remi's setup, or can provide a setup that allows me to compile a identical or near identical mysqlnd.so binary i'm happy to try it out.

I could compile vanilla php from scratch, but i'm afraid to introduce new problems.

[2019-05-07 12:30 UTC] dmitry@php.net

https://github.com/php/php-src/commit/5eb1f92f31cafc48384f9096012f421b37f6d425 shouldn't be the reason of the crash.

Victoria, can you provide more info from gdb and coredump.
Please, execute the following commands and capture the output.

(gdb) p meta
(gdb) p *meta
(gdb) p *meta->sname
(gdb) disassemble

[2019-05-07 12:37 UTC] victoria at sharksmedia dot dk

Here goes:

(gdb) p meta
$1 = (MYSQLND_FIELD *) 0x7f4929401018
(gdb) p *meta
Cannot access memory at address 0x7f4929401018
(gdb) p *meta->sname
Cannot access memory at address 0x7f4929401018
(gdb) disassemble
Dump of assembler code for function mysqlnd_mysqlnd_res_meta_free_pub:
   0x00007f49392c73c0 <+0>:	push   %r12
   0x00007f49392c73c2 <+2>:	push   %rbp
   0x00007f49392c73c3 <+3>:	push   %rbx
   0x00007f49392c73c4 <+4>:	mov    (%rdi),%rbx
   0x00007f49392c73c7 <+7>:	test   %rbx,%rbx
   0x00007f49392c73ca <+10>:	je     0x7f49392c7424 <mysqlnd_mysqlnd_res_meta_free_pub+100>
   0x00007f49392c73cc <+12>:	mov    0x14(%rdi),%eax
   0x00007f49392c73cf <+15>:	mov    %rdi,%r12
   0x00007f49392c73d2 <+18>:	lea    (%rax,%rax,4),%rbp
   0x00007f49392c73d6 <+22>:	shl    $0x5,%rbp
   0x00007f49392c73da <+26>:	add    %rbx,%rbp
   0x00007f49392c73dd <+29>:	jmp    0x7f49392c7417 <mysqlnd_mysqlnd_res_meta_free_pub+87>
   0x00007f49392c73df <+31>:	nop
   0x00007f49392c73e0 <+32>:	test   %rbx,%rbx
   0x00007f49392c73e3 <+35>:	je     0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80>
=> 0x00007f49392c73e5 <+37>:	mov    (%rbx),%rdi
   0x00007f49392c73e8 <+40>:	movq   $0x0,0x90(%rbx)
   0x00007f49392c73f3 <+51>:	movq   $0x0,0x48(%rbx)
   0x00007f49392c73fb <+59>:	test   %rdi,%rdi
   0x00007f49392c73fe <+62>:	je     0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80>
   0x00007f49392c7400 <+64>:	testb  $0x40,0x4(%rdi)
   0x00007f49392c7404 <+68>:	jne    0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80>
   0x00007f49392c7406 <+70>:	subl   $0x1,(%rdi)
   0x00007f49392c7409 <+73>:	jne    0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80>
   0x00007f49392c740b <+75>:	callq  0x7f49392b3340 <_efree@plt>
   0x00007f49392c7410 <+80>:	add    $0xa0,%rbx
   0x00007f49392c7417 <+87>:	cmp    %rbp,%rbx
   0x00007f49392c741a <+90>:	jne    0x7f49392c73e0 <mysqlnd_mysqlnd_res_meta_free_pub+32>
   0x00007f49392c741c <+92>:	movq   $0x0,(%r12)
   0x00007f49392c7424 <+100>:	pop    %rbx
   0x00007f49392c7425 <+101>:	pop    %rbp
   0x00007f49392c7426 <+102>:	pop    %r12
   0x00007f49392c7428 <+104>:	retq
End of assembler dump.

[2019-05-07 12:52 UTC] Scott at exussum dot co dot uk

Not sure if it helps but we have a long running job which gets this same issue. It happens at a different point each time.  My guess was it was something to do with gc but not proved it. 

I have 7.3 from ondrej

[2019-05-07 13:05 UTC] nikic@php.net

My theory here would be that we need to swap the free_result_buffer() and free_metadata() calls inside https://github.com/php/php-src/blob/master/ext/mysqlnd/mysqlnd_result.c#L293, because free_result_internal() will destroy the mempool, which is shared with the metadata.

Does that sound plausible Dmitry?

[2019-05-07 13:14 UTC] dmitry@php.net

Few more commands.

(gdb) up
(gdb) p *meta
(gdb) p i
(gdb) p meta.fields[0]
(gdb) p meta.fields[i-1]

Probably, "meta.fields" was somehow corrupted or used after free, but I can't guess how this might happen just analysing sources. I need a way to reproduce this, to catch the problem.

[2019-05-07 13:19 UTC] victoria at sharksmedia dot dk

(gdb) up
#1  mysqlnd_mysqlnd_res_meta_free_pub (meta=0x7f492977a718) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:106
106				php_mysqlnd_free_field_metadata(fields++);
(gdb) p *meta
$1 = {fields = 0x7f4929401018, m = 0x7f49394e1080 <mysqlnd_mysqlnd_res_meta_methods>, current_field = 0, field_count = 159}
(gdb) p i
$2 = <optimized out>
(gdb) p meta.fields[0]
Cannot access memory at address 0x7f4929401018
(gdb) p meta.fields[i-1]
value has been optimized out
(gdb)

[2019-05-07 14:20 UTC] dmitry@php.net

Nikita, I can't create a script to confirm your idea.
Victoria, please check this by printing data.

(gdb) up
(gdb) up
(gdb) p result->meta
(gdb) p *result->memory_pool.arena

[2019-05-07 14:24 UTC] victoria at sharksmedia dot dk

(gdb) up
#1  mysqlnd_mysqlnd_res_meta_free_pub (meta=0x7f492977a718) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:106
106				php_mysqlnd_free_field_metadata(fields++);
(gdb) up
#2  0x00007f49392c3daa in mysqlnd_mysqlnd_res_free_result_contents_internal_pub (result=0x7f492977a048) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result.c:300
300			result->meta->m->free_metadata(result->meta);
(gdb) p result->meta
$1 = (MYSQLND_RES_METADATA *) 0x7f492977a718
(gdb) p *result->memory_pool.arena
$2 = {ptr = 0x7f492977a188 "pK,9I\177", end = 0x7f492977de80 "\001", prev = 0x0}
(gdb)

[2019-05-10 05:48 UTC] victoria at sharksmedia dot dk

Are there any way i can help?

[2019-05-19 17:56 UTC] peku33 at gmail dot com

Is there any update on this issue? I'm facing the same problems on debian buster, after upgrading from php7.2 to php7.3. I can observe this from ~january (after switching to 7.3.1) up to now with latest PHP 7.3.4-2.

Random segfaults are reported by the php-fpm:
[15808956.997420] php-fpm7.3[26936]: segfault at 7f1bc4c01018 ip 00007f1bd35fd7c5 sp 00007ffccc7f6f70 error 4 in mysqlnd.so[7f1bd35e9000+1e000]
[15808956.999217] Code: 00 00 00 00 00 41 54 55 53 48 8b 1f 48 85 db 74 58 8b 47 14 49 89 fc 48 8d 2c 80 48 c1 e5 05 48 01 dd eb 38 90 48 85 db 74 2b <48> 8b 3b 48 c7 43 48 00 00 00 00 48 c7 83 90 00 00 00 00 00 00 00
[15809230.166757] php-fpm7.3[26977]: segfault at 7f1bc5401018 ip 00007f1bd35fd7c5 sp 00007ffccc7f6f70 error 4 in mysqlnd.so[7f1bd35e9000+1e000]
[15809230.177642] Code: 00 00 00 00 00 41 54 55 53 48 8b 1f 48 85 db 74 58 8b 47 14 49 89 fc 48 8d 2c 80 48 c1 e5 05 48 01 dd eb 38 90 48 85 db 74 2b <48> 8b 3b 48 c7 43 48 00 00 00 00 48 c7 83 90 00 00 00 00 00 00 00

[2019-05-22 18:42 UTC] syslogbeta at gmail dot com

Any updates on that?

[2019-05-22 21:34 UTC] dmitry@php.net

unfortunately, no updates. I need a way to reproduce this crash.

[2019-05-23 05:12 UTC] victoria at sharksmedia dot dk

Dmitry, can you some how guide me to whatever is necessary to catch the needed information? In a week i'm updating that server anyway so i have a maintenance window of around 30 minutes where i would be able to test things.

[2019-05-23 11:41 UTC] nikic@php.net

Automatic comment on behalf of nikita.ppv@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=6f9dfd947302f9a0d2fa6a78bf385b1ca7dafdf3
Log: Fix bug #77955

[2019-05-23 11:41 UTC] nikic@php.net

-Status: Assigned +Status: Closed

[2019-05-23 15:12 UTC] dmitry@php.net

Most probably, the bug is fixed by https://github.com/php/php-src/commit/6f9dfd947302f9a0d2fa6a78bf385b1ca7dafdf3

Victoria, it would be great if you could test your app, using DEBUG build of PHP-7.3 snapshot.

[2019-05-24 05:24 UTC] victoria at sharksmedia dot dk

That looks good.

Will this fix be in 7.3.6?

Since we're using the remi repo, it is quite a task to compile it all to be a direct swapable replacement (Remi couldn't provide info on how to compile his SRPMS)

[2019-06-03 06:02 UTC] victoria at sharksmedia dot dk

Hi Dmitry

I see the fixes weren't inclded in 7.3.6.

Will they be in 7.3.7, or do you need to have confirmation of it working before it is included in the release?

As mentioned, i can't "just" run a debug build, as remi doesn't provide it and wont help make one.

Can i compile a debug version of the php binary, and mysqlnd.so, and use that with all the extensions not compiled in debug mode?
Or must ALL loaded binaries be compiled in the same debug mode?

Because, than i can just compile php, and replace the two binaries on the production server without messing more with the production setup.

[2019-06-03 07:16 UTC] dmitry@php.net

The fixes are going to stay in all versions. Confirmation is not necessary. If you'll see the problem - reopen the bug report.

It's not possible to run different PHP components, built in different "debug" mode.

[2019-06-03 07:56 UTC] scott at exussum dot co dot uk

Hi Dmitry / Nikita

That fix has changed something we now get

terminated by signal SIGSEGV (Address boundary error)

It terminates at a similar place though.

Anything I can do to help debug this further ?

[2019-07-08 11:14 UTC] victoria at sharksmedia dot dk

Hi,

I'm just writing to confirm that the segmentation fault is gone with php 7.3.7.

But there seem to be massive memory leaks still present, i'm still getting the "Allowed memory size of x bytes exhausted", very quickly in longer running scripts such as cronjobs.

I'm isolating the problem, and posting a new bug.

Thanks for fixing this one though :)

	php.net \| support \| documentation \| report a bug \| advanced search \| search howto \| statistics \| random bug \| login
go to bug id or search bugs for


Copyright © 2001-2024 The PHP Group All rights reserved.	Last updated: Thu Nov 21 08:01:29 2024 UTC