php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #66694 php-fpm is crashing when using eventport
Submitted: 2014-02-11 08:57 UTC Modified: 2021-12-07 23:04 UTC
Votes:3
Avg. Score:4.3 ± 0.9
Reproduced:3 of 3 (100.0%)
Same Version:0 (0.0%)
Same OS:1 (33.3%)
From: eugene at zhegan dot in Assigned: bukka (profile)
Status: Closed Package: FPM related
PHP Version: 5.4.25 OS: Solaris 11.1
Private report: No CVE-ID: None
View Developer Edit
 [2014-02-11 08:57 UTC] eugene at zhegan dot in 
Description:
------------
php-fpm is crashing when 'events.mechanism' is set to 'port'. It doesn't crash on first request, it can process some, however, the crash is imminent and my installation dumps cores like 1 per minute (this server is used by ~50 developers).

When set to 'events.mechanism = /dev/poll' php-fpm stop to crash.

Backtrace (they all do look similar):

# /usr/local/solarisstudio12.3/bin/dbx /usr/local/php-5.4.25/sbin/php-fpm core_hyperion_php-fpm_0_%c_5046 
For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 7.9' in your .dbxrc
Reading php-fpm
core file header read successfully
Reading ld.so.1
Reading preloadable_libiconv.so
Reading libcrypt.so.1
Reading libresolv.so.2
Reading librt.so.1
Reading libmysqlclient.so.18.0.0
Reading libsybdb.so.5.0.0
Reading libnsl.so.1
Reading libsocket.so.1
Reading libpthread.so.1
Reading libldap.so.5
Reading liblber-2.4.so.2.8.3
Reading libpng15.so.15.13.0
Reading libz.so.1
Reading libm.so.2
Reading libjpeg.so.62.0.0
Reading libcrypto.so.1.0.0
Reading libssl.so.1.0.0
Reading libxml2.so.2
Reading libfreetype.so.6
Reading libc.so.1
Reading libgcc_s.so.1
Reading libdl.so.1
Reading libnspr4.so
Reading libmd.so.1
Reading libmp.so.2
Reading libsasl.so.1
Reading libplc4.so
Reading libnss3.so
Reading libnssutil3.so
Reading libplds4.so
Reading libssl3.so
Reading libthread.so.1
Reading libbz2.so.1
Reading libcryptoutil.so.1
Reading ru_RU.UTF-8.so.3
Reading methods_unicode.so.3
Reading xdebug.so
Reading bz2.so
Reading curl.so
Reading libcurl.so.4.2.0
Reading imap.so
Reading libpam.so.1
Reading memcache.so
Reading mcrypt.so
Reading libmcrypt.so.4.4.8
Reading libltdl.so.7.3.0
Reading oci8.so
Reading libclntsh.so.11.1
dbx: internal warning: "(null)"::srcname(InfoOrigin_CMDLINE): value "ntcontab.c" being set to "nt.c"
Reading libnnz11.so
Reading libkstat.so.1
Reading libgen.so.1
Reading libsched.so.1
Reading libaio.so.1
Reading libpool.so.1
Reading openssl.so
Reading pinba.so
Reading redis.so
Reading xmlrpc.so
Reading xsl.so
Reading libexslt.so.0
Reading libxslt.so.1
Reading nss_files.so.1
Reading nss_ldap.so.1
Reading libsldap.so.1
Reading libscf.so.1
t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address)
0x00000000008f75d7: fpm_event_fire+0x0009:      movq     0x0000000000000028(%rdi),%rax
(dbx) where                                                                  
current thread: t@1
=>[1] fpm_event_fire(0x75723d63642c61, 0x26, 0x0, 0x1, 0x4, 0xffff80ffbfffd0b0), at 0x8f75d7 
  [2] fpm_event_port_wait(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x13b6, 0x7d141, 0xe18ed, 0x1, 0x0, 0xffff80ffbffffb3c, 0xffff80ffbffffc18, 0x1, 0xd55d0c, 0xffff80ffbfffd160, 0x8f2925, 0xe23c20, 0x0), at 0x902998 
  [3] fpm_event_loop(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f7927 
  [4] fpm_run(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f2925 
  [5] main(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8fa704 
(dbx)


Another one:

# /usr/local/solarisstudio12.3/bin/dbx /usr/local/php-5.4.25/sbin/php-fpm core_hyperion_php-fpm_0_%c_4964
For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 7.9' in your .dbxrc
Reading php-fpm
core file header read successfully
Reading ld.so.1
Reading preloadable_libiconv.so
Reading libcrypt.so.1
Reading libresolv.so.2
Reading librt.so.1
Reading libmysqlclient.so.18.0.0
Reading libsybdb.so.5.0.0
Reading libnsl.so.1
Reading libsocket.so.1
Reading libpthread.so.1
Reading libldap.so.5
Reading liblber-2.4.so.2.8.3
Reading libpng15.so.15.13.0
Reading libz.so.1
Reading libm.so.2
Reading libjpeg.so.62.0.0
Reading libcrypto.so.1.0.0
Reading libssl.so.1.0.0
Reading libxml2.so.2
Reading libfreetype.so.6
Reading libc.so.1
Reading libgcc_s.so.1
Reading libdl.so.1
Reading libnspr4.so
Reading libmd.so.1
Reading libmp.so.2
Reading libsasl.so.1
Reading libplc4.so
Reading libnss3.so
Reading libnssutil3.so
Reading libplds4.so
Reading libssl3.so
Reading libthread.so.1
Reading libbz2.so.1
Reading libcryptoutil.so.1
Reading ru_RU.UTF-8.so.3
Reading methods_unicode.so.3
Reading xdebug.so
Reading bz2.so
Reading curl.so
Reading libcurl.so.4.2.0
Reading imap.so
Reading libpam.so.1
Reading memcache.so
Reading mcrypt.so
Reading libmcrypt.so.4.4.8
Reading libltdl.so.7.3.0
Reading oci8.so
Reading libclntsh.so.11.1
dbx: internal warning: "(null)"::srcname(InfoOrigin_CMDLINE): value "ntcontab.c" being set to "nt.c"
Reading libnnz11.so
Reading libkstat.so.1
Reading libgen.so.1
Reading libsched.so.1
Reading libaio.so.1
Reading libpool.so.1
Reading openssl.so
Reading pinba.so
Reading redis.so
Reading xmlrpc.so
Reading xsl.so
Reading libexslt.so.0
Reading libxslt.so.1
Reading nss_files.so.1
Reading nss_ldap.so.1
Reading libsldap.so.1
Reading libscf.so.1
t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address)
0x00000000008f75d7: fpm_event_fire+0x0009:      movq     0x0000000000000028(%rdi),%rax
(dbx) where                                                                  
current thread: t@1
=>[1] fpm_event_fire(0x75723d63642c61, 0x26, 0x0, 0x1, 0x4, 0xffff80ffbfffd0b0), at 0x8f75d7 
  [2] fpm_event_port_wait(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x1364, 0x7d0da, 0xb7ce3, 0x1, 0x0, 0xffff80ffbffffb3c, 0xffff80ffbffffc18, 0x1, 0xd55d0c, 0xffff80ffbfffd160, 0x8f2925, 0xe23c20, 0x0), at 0x902998 
  [3] fpm_event_loop(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f7927 
  [4] fpm_run(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f2925 
  [5] main(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8fa704 
(dbx)

Patches

php-7.0.12.patch (last revision 2016-11-09 19:43 UTC by christian at kuehnke dot de)

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2014-05-02 13:56 UTC] d dot v dot taylor at leedsmet dot ac dot uk 
I've hit this too on 5.5.11, but with a Bus Error rather than a SEGV (which will be a SPARC vs x86 thing). It particularly happens if I kill -QUIT FPM before it's handled *any* requests (and so hasn't received any events). This is the backtrace (using mdb):

fpm_event_fire+0x1c(2e70687035002e, ffffffff, 1, ffffffff7fffd470, ffffffff7fffd460, ffffffffffffffff)
fpm_event_port_wait+0x1ec(1014a3c50, 3e8, 2, 100efe858, 1768, 0)
fpm_event_loop+0x3f0(0, 0, 0, 0, 0, 0)
fpm_run+0xa8(ffffffff7ffff928, 0, a, 100effc80, ffffffff7df02a40, ffffffff7df02a40)
main+0x12bc(1, ffffffffffffffff, ffffffff7ffffaa8, ffffffff7df001c0, 2880, 2800)
_start+0x12c(0, 0, 0, 0, 0, 0)



This is what I think is going on in my case (maybe yours too?) - long story short it's a memory reuse problem:

fpm_event_port_init() malloc()s a chunk of memory to store events retrieved from the port, and doesn't initialise that memory at all. In my case it happened to have previously stored a pointer to a string used in processing php-fpm.conf.

fpm_event_port_wait() uses port_getn() to look for events, with a parameter nget  - the intended number of events - set to 1. In normal use that nget parameter is updated to the number of events actually retrieved, but if port_getn is interrupted by a signal and no events have been sent to the port, nget is left at 1 (see http://marc.info/?l=opensolaris-networking-discuss&m=125441701003618)

The code below the port_getn() call sees nget > 0, assumes an event has been returned, and tries to process it. Because the memory assigned to store events hasn't been zeroed out, it gets through the brief sanity check and passes a junk pointer to fpm_event_fire - which then reads the random string (in my case ".php5\0.") as itself being a pointer (0x2e70687035002e), tries to dereference it and falls in a heap.

I'm not sure if it's a safe assumption that if port_getn() was interrupted by a signal and we're only asking for one event, then there *definitely* won't have been any events retrieved - I suspect not. I also don't know if just zeroing out the memory in fpm_event_port_init() is enough, or if that only works because I'm only hitting it immediately after startup.
 [2016-11-09 19:47 UTC] christian at kuehnke dot de 
I experienced this problem (php-fpm SIGSEGVs after sending it SIGQUIT immediately after startup) with php 7.0.12 and 5.6.27 on Solaris 11.3.

I attached a one-liner patch for this problem. It is still not well tested, will update here later...
 [2017-03-02 11:32 UTC] stadtkind2 at gmx dot de 
I have the same problem with PHP 7.1.2 on Solaris 11.3/SPARC. The applied patch did not fix it, I'm still getting coredumps:

# pstack /var/cores/core.php-fpm.0.15837
core '/var/cores/core.php-fpm.0.15837' of 15837:        /opt/php7/sbin/php-fpm
 0000000100caa288 fpm_event_fire (0, 0, 1, 20000, 0, 101076720) + 8
 0000000100caa0ac fpm_event_loop (0, 0, ffffffff7ffff550, 100de4880, 10059c5c8, fffffffffff0bdc0) + 2fc
 0000000100ca0024 fpm_run (ffffffff7ffffa24, bd350, 0, bd000, 6, ffffffff7f782a40) + 64
 0000000100cb22cc main (7b9400, ffffffffff7d8674, 827800, 0, 100dc3fb0, ffffffffff846a18) + eec
 000000010060dee8 _start (0, 0, 0, 0, 0, 100dc3fb0) + 108
 [2017-03-06 19:09 UTC] stadtkind2 at gmx dot de 
The 03-event_port.patch patch at https://github.com/OpenIndiana/oi-userland/tree/oi/hipster/components/web/php/php-7_0/patches works better, no core dumps anymore.
 [2021-12-07 23:04 UTC] bukka@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: bukka
 [2021-12-07 23:04 UTC] bukka@php.net 
This should be fixed by https://github.com/php/php-src/pull/6913 in PHP 7.4+
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Sun Dec 22 06:01:30 2024 UTC