php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #44994 exec() produces zombie processes on Windows, hangs Apache
Submitted: 2008-05-14 15:09 UTC Modified: 2011-10-10 17:08 UTC
Votes:89
Avg. Score:4.9 ± 0.4
Reproduced:85 of 86 (98.8%)
Same Version:52 (61.2%)
Same OS:60 (70.6%)
From: dbarrett at vistaprint dot com Assigned: pajoye
Status: Closed Package: Program Execution
PHP Version: 5.2.6 OS: win32 only - 2003 Server, 64-bit
Private report: No CVE-ID:
 [2008-05-14 15:09 UTC] dbarrett at vistaprint dot com
Description:
------------
This is an intermittent but fatal problem. I cannot reproduce it on demand, but it occurs regularly on our MediaWiki 1.12.0 site running on Apache 2.2.8.

When invoking a non-interactive Microsoft Windows command via exec() or shell_exec(), sometimes the resulting process will hang.  Once this occurs, most or all subsequent exec() calls (within the same Apache instance) will hang as well.  Eventually the wiki becomes unusable.  The only way to kill these processes is to restart Apache.  Other traditional "kill" techniques fail.

This is not the same as Bug #30794, which was about interactive commands like Notepad.  I'm speaking of diverse shell commands like perl.exe, gnuplot.exe, latex.exe, and others.  All of them process text input, if that matters.

If the process is "foo" and you run exec("foo"), you wind up with several hung processes:

foo.exe
cmd.exe*32

This may be a race condition. I ran the "handle" utility (from sysinternals) on the hung processes. It seemed to indicate that the processes were waiting on c:\Apache2\logs\error.log.  Then handle itself (handle64.exe) hung!

Because this is an intermittent problem, I am looking for advice on how to debug it and report more detailed information to you. Please don't append the generic "install the latest snapshot and let us know if the problem is still there" message to this ticket, because I can't install an experimental PHP release on a production server, and the problem happens ONLY on our production server (probably because of the load), not on any of our development or test servers.

Expected result:
----------------
exec() and shell_exec() should never hang.


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2008-05-29 13:21 UTC] dbarrett at vistaprint dot com
I replaced all occurrences of exec() and shell_exec() in MediaWiki with calls to a custom COM object that invokes programs.  The problem went away. This strongly suggests that PHP's implementation of exec(0 and shell_exec() is the culprit. I suspect a race condition.
 [2008-07-07 18:57 UTC] stephen dot cuppett at sas dot com
I am also experiencing this problem.  I have been able to recreate it with system() as well as proc_open().  With proc_open() it doesn't seem to matter if I set bypass_shell or not, just determines whether I have a dangling cmd.exe process or not...

cmd.exe seems to stop at Wait:UserReq, but that probably doesn't mean much.

I am able to reproduce it with XAMPP 1.6.6a (PHP 5.2.5) on Windows XP or Windows Server 2003 (32-bit).

I don't have much to add in terms of documentation as I don't get anything in the httpd logs, but using the following code snippet:

                protected function runSAS($filename) {
                                $this->log("Requested to run SAS file: " . $filename, LOG_DEBUG);

                                $toRun  = "\"" . SAS . "\"";
                                $toRun .= " -sysin '" . $filename . "'";
                                $toRun .= " -CONFIG '" . SASV9CFG . "'";
                                $toRun .= " -PRINT '" . TEMP_DIR . "/" . basename($filename) . ".lst'";
                                $toRun .= " -LOG '" . TEMP_DIR . "/" . basename($filename) . ".log'";
                                $toRun .= " -NOSPLASH";
                                
                                $this->log("SAS command to run: " . $toRun, LOG_DEBUG);
                                
                                $descriptorspec = array(
                                                0 => array("pipe", "r"),  // stdin is a pipe that the child will read from
                                                1 => array("pipe", "w"),  // stdout is a pipe that the child will write to
                                                2 => array("pipe", "w")   // stderr is a file to write to
                                );
                                
                                $res = proc_open($toRun, $descriptorspec, $pipes, null, null, array(/*"bypass_shell" => true*/));
                                
                                if (is_resource($res)) {
                                                //fwrite($pipes[0], "exit\n");
                                                //fflush($pipes[0]);
                                                fclose($pipes[0]);
                                                fclose($pipes[1]);
                                                fclose($pipes[2]);
                                                
                                                $return_value = proc_close($res);
                                                
                                                $this->log("Return value from SAS: " . $return_value, LOG_DEBUG);
                                }
                }


When it does seem to complete, I'll get this in my log:

2008-06-26 20:23:55 Debug: Return value from SAS: 1

Nothing should come out on the pipes at all and when I set it up to read those and it completes, there is no data read.  However, in a hang case, PHP blocks reading STDOUT instead of on proc_close never getting any data... even if I set the stream to be non-blocking on the pipe explicitely after proc_open.
 [2008-07-23 02:02 UTC] efesar at gmail dot com
I just upgraded from PHP 5.1.2 to 5.2.6, and this exact problem happens to me. The exec and shell_exec used to work, and now they don't. They hang the php process, and every subsequent call to exec produces a lost cmd.exe ... 

Incidentally passthru() causes the same behavior.

Maybe it has something to do with the rotatelogs.exe ... somebody earlier mentioned that it was waiting on c:\Apache2\logs\error.log ...  

So, basically, we can't run any system calls whatsoever via PHP. I don't want to use a COM object, because I like portable code ... but that might be the best step until this bug is fixed.
 [2008-07-29 15:03 UTC] nobody at dontsendmespam dot com
i can reproduce this problem on a debian server with apache2 and php5.2.6 running as fcgi
 [2008-08-12 23:50 UTC] jkflash at gmail dot com
I am experiencing this same problem with PHP 5.2.1 running on IIS as an ISAPI module on Windows Server 2003 R2 SP2. Intermitently (a handful of times in a hundred) a process started with exec() will appear to complete ok but remain running in a zombie state. The IIS worker process that created the process will not respond to any further requests. I need to either open a new client browser or kill the worker process.

I get the same result with system, passthru, p_open and proc_open.
 [2008-08-13 01:08 UTC] jkflash at gmail dot com
PS: shell_exec() DOES seem to work without hanging on my setup, but none of the others.
 [2008-08-19 02:58 UTC] jkflash at gmail dot com
Sorry, I take that last comment back, it still hangs annoyingly intermittently with shell_exec as well. It's a killer :(
 [2008-11-18 08:46 UTC] bernd dot ebert at gmx dot net
Same Problem here with an commandline-script calling other php-commandline-scripts. It seems to happen more frequently if you have a lot of exec-calls.
 [2008-12-02 14:34 UTC] brem at martinbreton dot com
If that's any help, I'm using a COM object as a workaround for this bug, and while it makes this bug appear less frequently, it still happens, randomly, from time to time.

Here's what i did as a workaround using the exec() function call:

$WshShell = new COM("WScript.Shell");
$OutputFile = "C:\\Temp\\output.txt"; 
                
$Command = "cmd /C MyApp.exe > $OutputFile";
$WshShell->Run( $Command, 0, true );

file_get_contents( $OutputFile ) );   
                
unlink( $OutputFile );
 [2009-01-13 18:44 UTC] irdev at rogers dot com
I have been experiencing this issue as well. The server is Windows 2003 with all patches up to date running IIS. PHP version is 5.2.5. I am running a commandline process using exec(). I have also tried passthru and shell_exec with the same results. Problem is intermittent and can not be replicated on demand. The processes themselves can not be killed until the IIS worker process is killed first although they do complete all the work they are supposed to. The commands do not die on script timeout.
 [2009-01-16 17:17 UTC] schreiber at cs-itc dot de
Same problems on w2k3 server x64 SP2 with apache 2.2.8 + 2.2.11 and   php 5.2.0 up to 5.2.8 -> using exec() the shellproc cmd.exe hangs and httpd is blocked for further ops, can just be cured by restart of apache services. Workaround with COM object helps, but thats no lucky solution...
 [2009-02-27 18:58 UTC] alex at alexi dot ch
Hi all,

I can fully confirm the above behaviour on the Windows platform. We are using Apache FOP to create PDF-Files out of PHP, so we are invoking exec() to start the command line FOP processor.
With several versions of PHP 5.2.x (5.2.4, 5.2.6, 5.2.8), this problem occurs soon as more than one user is invoking the exec().

Luckily we could not reproduce this behaviour using PHP 5.1.x, so downgrading to a 5.1.x version might also be an option for you.

Regards
alex
 [2009-04-22 02:03 UTC] aliks0905 at gmail dot com
I have the same issue as well. I use the exec() function to simultaneously upload files and sometimes cmd.exe freezes and all following uploads fail.

I run on a Windows 2003 Server using Apache 2.2.11 and PHP 5.2.9

Looking forward to hearing a solution.
 [2009-04-28 22:52 UTC] diackne at gmail dot com
I have the same issue in windows 2008 web server iis 7 !!!
 [2009-05-04 14:56 UTC] mk1992 at hotmail dot com
same problem here, php 5.2.9, w2k3, IIS: shell_exec hanging on production server but not on development server.  Can't do downgrade to 5.1.x.
 [2009-06-08 16:40 UTC] alex at bartl dot net
Reproducable with PHP 5.2.9-1 on Windows2003 Server with Apache2.2
Workaround with calling session_write_close() before calling exec() confirmed working

NOT reproducable with PHP 5.2.1 on Windows 2000 Server with IIS5

anyway, seems to be a duplicate of Bug#44942
 [2009-08-19 17:26 UTC] garretts@php.net
I've seen this happen in other languages too.

This happens when the pipe between the parent and child fills up, and cmd.exe ends up blocking and creates a race condition. (Windows script host languages trip up on this quickly)

in popen_ex() ( in tsrm_win32.c) the pipe is creates with a 2k buffer:

   if (!str_len || !CreatePipe(&in, &out, &security, 2048L)) {

this should probably be significantly larger. I'd certainly go with at least 16k or 32k. (hey, it's only memory :D)

as well, the elimination of the unrequired cmd.exe as the immediate child process would eliminate the possibility that *it's* buffer gets overwhelmed too. (which solves bug #43327, and I've passed a patch to Pierre for that.)



 [2009-08-19 19:06 UTC] garretts@php.net
Please try using this snapshot:

  http://snaps.php.net/php5.3-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/


 [2009-08-27 01:00 UTC] php-bugs at lists dot php dot net
No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
 [2009-09-01 18:42 UTC] garretts@php.net
I'm trying to come up with a reproducible test case for this bug.

If anyone has a complete test scenario for this please post it. I simply can't replicate the effects here.

G
 [2010-01-06 17:35 UTC] patrick dot bueker at triangle-solutions dot de
I once had the problem too. I had the feeling that it has something to do with threadsafe or non-threadsafe versions of PHP. I still have the problem on some installations, others work.
 [2011-01-18 09:50 UTC] pajoye@php.net
-Assigned To: garretts +Assigned To: pajoye
 [2011-01-18 09:50 UTC] pajoye@php.net
-Status: No Feedback +Status: Assigned
 [2011-05-05 17:52 UTC] php at mmtbb dot com
Testing on PHP v5.3.1 (installed with XAMPP), I'm able to run 50 concurrent popen requests without any fault. All processes terminate after completion.

I understand that the popen() and exec() were both experiencing this issue in the past, the 5.3.x line seems to resolve the issue.
 [2011-05-05 18:09 UTC] pajoye@php.net
-Status: Assigned +Status: Closed
 [2011-05-05 18:09 UTC] pajoye@php.net
Fixed in 5.3.x
 [2011-10-10 15:31 UTC] jameslively at insightbb dot com
I am still seeing this issue on PHP 5.3.8.  I am running the site on Windows 2k3 sp3 and Apache 2.2.
 [2011-10-10 17:08 UTC] pajoye@php.net
@jameslively at insightbb dot com

What do you see exactly? Can you provide a small script and the cmd you use to 
test?
 [2012-06-06 01:31 UTC] mark at clevertech dot biz
WinXP Home SP3
XAMPP with:
PHP 5.3.8 built Aug 23, 2011
Apache 2.2.21 built Sep 10, 2011

This problem occurs with proc_open(), exec(), and popen().

Once a child process hangs, PHP in new page requests continues to run fine 
unless I call proc_open(), exec(), popen(), or session_start().

The problem ceases to occur when I call session_write_close() before my call to 
proc_open(), exec(), or popen().

I'm on a team of 6 developers, and we are running this on Linux, Mac, and 
several versions of Windows, but we've only seen this problem on Windows XP.

This is happening on a complex application using a large framework. I intend to 
post a straightforward test script later.
 [2012-06-28 11:35 UTC] jaw at condidact dot dk
I have also experienced this problem on debian linux kernel 2.6.22.6-1-amd64 / php 5.3.8 / apache 2.2.11
It seems like the problem stems from php running multiple similar/same commands.
The session_write_close() fix did nothing for me, but a sleep(2) (2 might be relative to the execution time of the command, in my case a ping with a timeout of 1) after exec(), seems to alleviate the problem, but not completely solving it.
Hope that helps somebody
 
PHP Copyright © 2001-2014 The PHP Group
All rights reserved.
Last updated: Sat Apr 19 19:02:15 2014 UTC