php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #52501 Misconfigured Sendmail sends PHP-FPM into an infinite loop
Submitted: 2010-07-31 00:25 UTC Modified: 2010-12-20 12:56 UTC
Votes:3
Avg. Score:4.0 ± 0.8
Reproduced:3 of 3 (100.0%)
Same Version:3 (100.0%)
Same OS:1 (33.3%)
From: marekroman1 at gmail dot com Assigned: fat
Status: Closed Package: FPM related
PHP Version: 5.3.3 OS: DebianLenny2.6.26-2-openvz-amd64
Private report: No CVE-ID:
 [2010-07-31 00:25 UTC] marekroman1 at gmail dot com
Description:
------------
(Misconfigured) Sendmail sends PHP-FPM into an infinite loop of spawning/exiting worker processes when attempting to send an email through the mail() function. (The php script itself just times out without any output.)

Since this loop produces 100% cpu load and warnings AND logs these warnings, the log file has grown to 400MB+ before I noticed the load spike and stopped PHP-FPM master process with kill -9 PID (nothing else worked e.g. -QUIT).

After I purge-removed the Sendmail package and installed Postfix instead, everything is working as it should (I didn't made any changes to nginx/php-fpm/php config files whatsoever).


Server: nginx-0.8.47
PHP ini settings related to mail: "sendmail_path = /usr/sbin/sendmail -t -i"
Sendmail state: daemon was not running (sendmail process was called, it did it's work i.e. nothing, then exited)
PHP configure:
./configure --prefix=/usr --enable-fpm --disable-rpath --with-pear --disable-debug --with-openssl --with-pcre-regex --with-zlib --enable-bcmath --with-bz2 --enable-calendar --with-curl --enable-exif --enable-inline-optimization --with-gd --enable-gd-native-ttf --with-gettext --with-imap --with-imap-ssl --with-kerberos --with-ldap --enable-mbstring --enable-mbregex --with-mcrypt --with-mysql --with-mysqli --enable-pcntl --enable-pdo --with-pdo-firebird --with-pdo-mysql --with-pdo-pgsql --with-pgsql --enable-shmop --enable-soap --enable-sockets --enable-sqlite-utf8 --enable-sysvmsg --enable-sysvsem --enable-sysvshm --with-tidy --enable-wddx --with-xmlrpc --with-xsl --enable-zip --with-pic --enable-ftp --enable-dom --enable-xmlwriter --enable-xmlreader --enable-tokenizer --enable-simplexml --enable-session --enable-posix --enable-phar --enable-libxml --enable-json --with-iconv --enable-filter --enable-fileinfo --enable-dba --enable-ctype
Dynamic extensions:
pecl apc-beta (apc.so) and rar (rar.so).
Changed PHP-FPM settings: 
listen = /var/run/php-fpm/php-fpm.sock
listen.owner = www-data
listen.group = www-data
user = www-data
group = www-data
pm = dynamic
pm.max_children = 10
pm.start_servers = 4
pm.min_spare_servers = 3
pm.max_spare_servers = 6
request_terminate_timeout = 60
rlimit_files = 10240
chdir = /var/www
catch_workers_output = yes



Test script:
---------------
$to  = 'xyz@abc.com';

// subject
$subject = 'Birthday Reminders for August';

// message
$message = '
<html>
<head>
  <title>Birthday Reminders for August</title>
</head>
<body>
  <p>Here are the birthdays upcoming in August!</p>
  <table>
    <tr>
      <th>Person</th><th>Day</th><th>Month</th><th>Year</th>
    </tr>
    <tr>
      <td>Joe</td><td>3rd</td><td>August</td><td>1970</td>
    </tr>
    <tr>
      <td>Sally</td><td>17th</td><td>August</td><td>1973</td>
    </tr>
  </table>
</body>
</html>
';

// To send HTML mail, the Content-type header must be set
$headers  = 'MIME-Version: 1.0' . "\r\n";
$headers .= 'Content-type: text/html; charset=UTF-8' . "\r\n";

// Additional headers
$headers .= 'To: Dude Dudeson <xyz@abc.com>, Kelly <kelly@gals.com>' . "\r\n";
$headers .= 'From: Birthday Reminder <xyz@abc.com>' . "\r\n";
$headers .= 'Cc: topstalker@thehomeofallinternetpredators.com' . "\r\n";

// Mail it
if (mail($to, $subject, $message, $headers)) echo "OKAY!!!!!";

Expected result:
----------------
OKAY!!!!!

Actual result:
--------------
php-fpm error log:

...
Trying to run a mail script
...

Jul 28 11:42:20.169022 [WARNING] [pool www] child 6344, script '/var/www/xyz.com/public/mail.php' execution timed out (60.095817 sec), terminating
Jul 28 11:42:20.169719 [WARNING] [pool www] child 6344 exited on signal 15 SIGTERM after 1532.840955
 seconds from start
Jul 28 11:42:20.170087 [NOTICE] [pool www] child 6409 started
Jul 28 11:42:20.170752 [WARNING] [pool www] child 6409 exited on signal 11 SIGSEGV after 0.000681 seconds from start
Jul 28 11:42:20.171011 [NOTICE] [pool www] child 6410 started
Jul 28 11:42:20.171602 [WARNING] [pool www] child 6410 exited on signal 11 SIGSEGV after 0.000605 seconds from start

...
Gazillion lines
...
I've noticed it's killing my cpus and tried to kill it
...
...

Jul 28 12:01:45.644600 [NOTICE] [pool www] child 2017 started
Jul 28 12:01:45.644615 [WARNING] [pool www] child 2011 exited on signal 6 SIGABRT after 0.002469 seconds from start
Jul 28 12:01:45.644621 [WARNING] [pool www] child 2011 said into stderr:  "php-fpm: event.c:261: event_base_free: Assertion `(((&base->eventqueue)->tqh_first) == ((void *)0))' failed."
Jul 28 12:01:45.645047 [NOTICE] [pool www] child 2018 started
Jul 28 12:01:45.645062 [WARNING] [pool www] child 2012 exited on signal 6 SIGABRT after 0.002622 seconds from start
Jul 28 12:01:45.645071 [WARNING] [pool www] child 2012 said into stderr: "php-fpm: event.c:261: event_base_free: Assertion `(((&base->eventqueue)->tqh_first) == ((void *)0))' failed."
Jul 28 12:01:45.646422 [NOTICE] [pool www] child 2021 started
Jul 28 12:01:45.646438 [WARNING] [pool www] child 2013 exited on signal 6 SIGABRT after 0.003529 seconds from start
Jul 28 12:01:45.646444 [WARNING] [pool www] child 2013 said into stderr: "php-fpm: event.c:261: event_base_free: Assertion `(((&base->eventqueue)->tqh_first) == ((void *)0))' failed."

Patches

fpm-nomorelibevent.v9.patch (last revision 2010-11-15 00:10 UTC) by fat@php.net)
fpm-nomorelibevent.v8.patch (last revision 2010-10-09 12:43 UTC) by fat@php.net)
fpm-nomorelibevent.v7.patch (last revision 2010-10-06 09:36 UTC) by fat@php.net)
fpm-nomorelibevent.v6.patch (last revision 2010-10-06 00:37 UTC) by fat@php.net)
fpm-nomorelibevent.v5.patch (last revision 2010-09-17 07:49 UTC) by fat@php.net)
fpm-nomorelibevent.v2.patch (last revision 2010-09-14 15:36 UTC) by fat@php.net)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2010-07-31 03:23 UTC] fat@php.net
-Status: Open +Status: Analyzed -Assigned To: +Assigned To: fat
 [2010-07-31 03:23 UTC] fat@php.net
when calling the mail() function on unix, there is a fork to run the sendmail command. If the sendmail does not return 
quickly and FPM tries to kill the process because of request_terminate_timeout the bug happens.

Here is a sample C code to simulate a bug sendmail config:

// sendmail_bug.c
#include <stdio.h>

int main() {
  char buf[11];
  while (1) {
    read(0, buf, 10);
  }
  return 0;
}

gcc -o /tmp/sendmail_bug sendmail_bug.c

then configure your php.ini this way:
sendmail_path = /tmp/sendmail_bug

set request_terminate_timeout to a 5s and load a page with the mail function.
 [2010-07-31 17:25 UTC] fat@php.net
the problem does not exist when using kqueue,poll or select on openbsd (and 
certainely on freebsd also).

the problem exists on linux when using epoll,poll or select.

It's all the problem of the libevent when forking.
 [2010-07-31 21:46 UTC] marekroman1 at gmail dot com
Oh, thanks fat.
 [2010-09-14 17:36 UTC] fat@php.net
The following patch has been added/updated:

Patch Name: fpm-nomorelibevent.v2.patch
Revision:   1284478582
URL:        http://bugs.php.net/patch-display.php?bug=52501&patch=fpm-nomorelibevent.v2.patch&revision=1284478582
 [2010-09-14 17:37 UTC] fat@php.net
-Status: Analyzed +Status: Feedback
 [2010-09-14 17:37 UTC] fat@php.net
Can you please test the following patch ?

This patch removes libevent from FPM. A simple select base event mechanism has 
been made instead. It should prevent the forking problem when using libevent.
 [2010-09-17 09:49 UTC] fat@php.net
The following patch has been added/updated:

Patch Name: fpm-nomorelibevent.v5.patch
Revision:   1284709749
URL:        http://bugs.php.net/patch-display.php?bug=52501&patch=fpm-nomorelibevent.v5.patch&revision=1284709749
 [2010-09-17 09:49 UTC] fat@php.net
can you please test the new patch (v5) ?
 [2010-10-02 13:23 UTC] danger at freebsd dot org
I have been experiencing similar problems with a busy site on FreeBSD pretty regurarly as well. I am now testing your patch, will report later how it goes.
 [2010-10-03 12:30 UTC] danger at freebsd dot org
well so far it seems to be running pretty fine. Only errors I am seeing now are like this:

Oct 03 11:45:51.547273 [WARNING] php_poll2() returns 4

Don't know what it means...
 [2010-10-03 12:35 UTC] fat@php.net
It's a case I didn't care about as it's a first release debug patch. Thx you very 
much for your tests and comments.

++ Jerome
 [2010-10-03 12:35 UTC] fat@php.net
-Status: Feedback +Status: Analyzed
 [2010-10-06 02:37 UTC] fat@php.net
The following patch has been added/updated:

Patch Name: fpm-nomorelibevent.v6.patch
Revision:   1286325458
URL:        http://bugs.php.net/patch-display.php?bug=52501&patch=fpm-nomorelibevent.v6.patch&revision=1286325458
 [2010-10-06 02:39 UTC] fat@php.net
-Status: Analyzed +Status: Feedback
 [2010-10-06 02:39 UTC] fat@php.net
Can you please test the updated version of the patch ?

I'm waiting for your returns to commit.
 [2010-10-06 11:36 UTC] fat@php.net
The following patch has been added/updated:

Patch Name: fpm-nomorelibevent.v7.patch
Revision:   1286357775
URL:        http://bugs.php.net/patch-display.php?bug=52501&patch=fpm-nomorelibevent.v7.patch&revision=1286357775
 [2010-10-06 11:36 UTC] fat@php.net
small bug correction.
 [2010-10-08 23:18 UTC] mplomer at gmx dot de
I did some general tests with patch.v7 (apachebench with hitting the limits; increasing and decreasing server load; ...) and I could not detect any errors.
But I did not do any more detailed tests.
 [2010-10-08 23:41 UTC] fat@php.net
-Status: Feedback +Status: Analyzed
 [2010-10-09 14:43 UTC] fat@php.net
The following patch has been added/updated:

Patch Name: fpm-nomorelibevent.v8.patch
Revision:   1286628184
URL:        http://bugs.php.net/patch-display.php?bug=52501&patch=fpm-nomorelibevent.v8.patch&revision=1286628184
 [2010-10-09 14:43 UTC] fat@php.net
small bug correction.
 [2010-10-11 14:40 UTC] tony2001@php.net
I'd appreciate if you give me some time to test it in my environment. A week or so would be enough, I guess.
 [2010-10-11 14:53 UTC] fat@php.net
no problem tony :)
 [2010-10-22 08:23 UTC] info at hninsider dot com
Can we have the patch working with the current 5.3.3 production version. Dev snapshots are not for production use and v8 patch does not apply cleanly to latest production version. Please provide a working patch for latest 5.3.3 ( not dev ) production version.


Thank you.
 [2010-11-11 18:17 UTC] fat@php.net
I can't create a specific patch for 5.3.3.

If you really want to test on 5.3.3, you can do:

tar -xzvf php-5.3.3.tar.gz
cd php-5.3.3
rm -rf sapi/fpm
wget http://snaps.php.net/php5.3-201011111530.tar.gz
tar -xzvf php5.3-201011111530.tar.gz
mv php5.3-201011111530/sapi/fpm sapi/fpm
rm -rf php5.3-201011111530
patch -p0 < fpm-nomorelibevent.v8.patch
./buildconf --force
./configure --enable-fpm --other-configure-args
make && make install

it should work and you have php 5.3.3 core and up to date and patched FPM.
 [2010-11-15 01:10 UTC] fat@php.net
The following patch has been added/updated:

Patch Name: fpm-nomorelibevent.v9.patch
Revision:   1289779853
URL:        http://bugs.php.net/patch-display.php?bug=52501&patch=fpm-nomorelibevent.v9.patch&revision=1289779853
 [2010-11-15 01:12 UTC] fat@php.net
Here is the v9 version of the patch. Nothing changes despite the fact that it 
applies on current svn revision (305343).
 [2010-11-18 22:54 UTC] fat@php.net
The v9 revision of the patch can be applied to the php 5.3.4RC1.

To do so:
wget http://downloads.php.net/johannes/php-5.3.4RC1.tar.bz2
tar -xjvf php-5.3.4RC1.tar.bz2
cd php-5.3.4RC1
./buildconf --force
./configure ... --enable-fpm
make
make install
 [2010-11-26 14:46 UTC] fat@php.net
Automatic comment from SVN on behalf of fat
Revision: http://svn.php.net/viewvc/?view=revision&amp;revision=305775
Log: - Fixed bug #52501 (libevent made FPM crashed when forking -- libevent has been removed)
 [2010-11-26 14:50 UTC] fat@php.net
-Status: Analyzed +Status: Closed
 [2010-11-26 14:50 UTC] fat@php.net
This bug has been fixed in SVN.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.


 [2010-12-20 12:56 UTC] jani@php.net
-Package: Tidy +Package: FPM related
 [2011-01-26 23:59 UTC] philip@php.net
Automatic comment from SVN on behalf of philip
Revision: http://svn.php.net/viewvc/?view=revision&amp;revision=307772
Log: Removed libevent option/bug, which only affected PHP 5.3.3. Closes PHP Bug #53691 and deals with PHP Bug #52501
 [2012-03-27 18:27 UTC] kevin7911 at hotmail dot com
Could you test the new patch (v5) ?
Thanks
 
PHP Copyright © 2001-2014 The PHP Group
All rights reserved.
Last updated: Wed Apr 16 16:02:23 2014 UTC