php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #18833 exec() : After throusands calls, causes "Unable to fork" error
Submitted: 2002-08-09 12:08 UTC Modified: 2002-09-11 11:51 UTC
Votes:33
Avg. Score:4.5 ± 0.8
Reproduced:31 of 32 (96.9%)
Same Version:1 (3.2%)
Same OS:10 (32.3%)
From: antoine dot bajolet at tdf dot fr Assigned:
Status: No Feedback Package: *General Issues
PHP Version: 4.2.1 OS: GNU/Linux 2.4.9 RH 7.1
Private report: No CVE-ID: None
 [2002-08-09 12:08 UTC] antoine dot bajolet at tdf dot fr
Hello,

I use php 4.2.1 as stand-alone binary to proceed batch file conversions with exec() and other utilites (ie mapserver, shapelib and others)

The binary is compiled with postgresql and ming support.
The amount of files to convert is over 10,000.
Each file needs 8 (height) exec() to be converted.

After lots of loops (throusands but it varies), i obtain the error message :
<br />
<b>Warning</b>:  Unable to fork [/usr/local/tdf/www/src/mapserver_3.5/shp2img -m /usr/local/tdf/www/src/swfs/temp.map -o /usr/local/tdf/www/src/swfs/13441_66.png -t -d base 13441] in <b>/usr/local/tdf/www/src/shape_to_tvview.php</b> on line <b>336</b><br />
[.../...]
<br />
<b>Warning</b>:  Unable to fork [/usr/local/tdf/www/src/ming-0.2a/util/png2dbl /usr/local/tdf/www/src/swfs/13441_66.png] in <b>/usr/local/tdf/www/src/shape_to_tvview.php</b> on line <b>25</b><br />
<br />

When i ran a #top in the same time, i saw that pid numbers of executed programs increases during execution of the script, and the exec did not reuse pid numbers already used in the same script if the loop is too quick (when the script converts small shapefiles).
the highest number i saw was 

The total number of processes on the system was not over 71 in the same time.

Any idea ?

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2002-08-09 13:09 UTC] sander@php.net
Can't reproduce. Might be a problem with your system.
 [2002-08-09 13:28 UTC] kalowsky@php.net
Marking as feedback
 [2002-09-11 11:51 UTC] sniper@php.net
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Open". Thank you.


 [2007-12-05 05:22 UTC] jinglerobs at yahoo dot com
Im using PHP 5.2.4 (cli), Apache & Mysql combination on a Slackware box.

While running a script that processes large XML files and writes the log to a log file, I get
Warning: exec(): Unable to fork

Initially the script runs fine for some 3 or 4 hours after which it gives out the warnings. The script has a huge amount of data to process. I also tried to use the nohup command but was of no use.
A screenshot of the problem is given below:

DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13001','Microsoft','Windows XP Home','0.0.0SP1','0','0','0','SP1','Software')

Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13001','Microsoft','Windows XP Home','0.0.0SP1','0','0','0','SP1','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454
DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13002','Microsoft','Windows XP Professional','0.0.0SP1','0','0','0','SP1','Software')

Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13002','Microsoft','Windows XP Professional','0.0.0SP1','0','0','0','SP1','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454
DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13134','KaZaA','KaZaA Media Desktop','2.0.0','2','0','0','','Software')

Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13134','KaZaA','KaZaA Media Desktop','2.0.0','2','0','0','','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454
DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('11051','KaZaA','KaZaA Media Desktop','1.6.1','1','6','1','','Software')

Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('11051','KaZaA','KaZaA Media Desktop','1.6.1','1','6','1','','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454
DEBUG:INSERT into infection_vector (vector,deployed,effectiveness) values ('File Sharing / Peer to Peer / Kazaa','3','8')

Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into infection_vector (vector,deployed,effectiveness) values ('File Sharing / Peer to Peer / Kazaa','3','8') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454
DEBUG:INSERT into infection_vector (vector,deployed,effectiveness) values ('File Transfer / Email Attachment','10','7')

Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into infection_vector (vector,deployed,effectiveness) values ('File Transfer / Email Attachment','10','7') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454

Fatal error: Out of memory (allocated -1155268608) (tried to allocate 11080 bytes) in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1447

[1]+  Exit 1                  ./feed2vendorDB.sh


Is this related to the max number of processes allowed in the memory, or a PHP-Apache prob??. Having a tough time ....
 [2008-06-01 16:48 UTC] pahan at hubbitus dot spb dot su
Have same trouble in PHP snapshot 200805080630:

PHP Warning:  exec(): Unable to fork [echo -ne '2008-06-01 05:15:27: Login: Unable to login (User: helpme@simpla
y.ru)! reason:not connected! (CMD:LOGIN)
         sleep 50 seconds

' >> /home/pasha/temp/php-IMAP/logs/log_ERR 2>&1] in /var/www/_SHARED_/Debug/HuLOG.php on line 115
PHP Stack trace:
PHP   1. {main}() /home/pasha/temp/php-IMAP/IMAP_answer.php:0
PHP   2. IMAP_mailbox_change_answer() /home/pasha/temp/php-IMAP/IMAP_answer.php:201
PHP   3. HuLOG->toLog() /home/pasha/temp/php-IMAP/IMAP_answer.php:67
PHP   4. HuLOG->writeLogs() /var/www/_SHARED_/Debug/HuLOG.php:158
PHP   5. HuLOG->log_to_file() /var/www/_SHARED_/Debug/HuLOG.php:162
PHP   6. exec() /var/www/_SHARED_/Debug/HuLOG.php:115

Warning: exec(): Unable to fork [echo -ne '2008-06-01 05:15:27: Login: Unable to login (User: helpme@simplay.ru)
! reason:not connected! (CMD:LOGIN)
         sleep 50 seconds

' >> /home/pasha/temp/php-IMAP/logs/log_ERR 2>&1] in /var/www/_SHARED_/Debug/HuLOG.php on line 115

Call Stack:
    0.0019     375364   1. {main}() /home/pasha/temp/php-IMAP/IMAP_answer.php:0
184145.8861   16442044   2. IMAP_mailbox_change_answer() /home/pasha/temp/php-IMAP/IMAP_answer.php:201
184145.8903   16451552   3. HuLOG->toLog() /home/pasha/temp/php-IMAP/IMAP_answer.php:67
184145.8908   16451648   4. HuLOG->writeLogs() /var/www/_SHARED_/Debug/HuLOG.php:158
184145.8908   16451648   5. HuLOG->log_to_file() /var/www/_SHARED_/Debug/HuLOG.php:162
184145.8917   16451824   6. exec() /var/www/_SHARED_/Debug/HuLOG.php:115
 [2009-02-05 11:37 UTC] scherbakov_koksa at mail dot ru
I'm having the same issue with PHP 5.2.8, Apache 2.2.11, GNU/Linux (2.6.20-1.2320.fc5). A php script periodically (per user request) executes the following command:

xsltproc <filename> <filename>

In some time the script starts throwing the "unable to fork" warning. It is fixed by Apache restart.
 [2009-03-09 13:33 UTC] steveg at bscopes dot com
1. did you ever figure out what caused this problem?
2. do you know which apache/php parameters to adjust ?
 [2009-05-04 14:42 UTC] scherbakov_koksa at mail dot ru
It seems I've finally figured out the reason (after a huge amount of googling UNIX forums).

The PHP fork() error is accompanied by the following entries in Apache error_log:

[Mon May 04 20:55:59 2009] [alert] (11)Resource temporarily unavailable: setuid: unable to change to uid: 48
[Mon May 04 20:55:59 2009] [alert] Child 10053 returned a Fatal error... Apache is exiting!

The system configuration doesn't allow Apache to execute more than N child processes simultaneously. Apache runs out of this limit and can't start any more processes. Hence is the error.

The per-user process number limit is set in /etc/security/limits.conf. In my case, it was enough to raise it and reboot.
 [2009-12-19 12:26 UTC] klaus dot wendel at lednew dot de
[Ubuntu karmic 9.10; php-cli 5.2.10]

> The per-user process number limit is set in 
> /etc/security/limits.conf.
> In my case, it was enough to raise it and reboot.

In my case, this wasn't the answer, unfortunately. Or fortunately, because I was forced to find a workaround, which could help others, too.

In my code I replaced exec by following function:

function user_proc_exec ($command,& $result,& $status)
	{
	$process = proc_open(
			$command,
			array(
			0 => array("pipe", "r"), //STDIN
			1 => array("pipe", "w"), //STDOUT
			2 => array("pipe","w")), //STDERR
			$pipes
			);
	$result = stream_get_contents($pipes[1]);
	$error = stream_get_contents($pipes[2]);
	fclose($pipes[1]);
	fclose($pipes[2]);
	$status= proc_close($process);
    return($error);
    }

Everything is working fine now.
 [2009-12-19 12:39 UTC] klaus dot wendel at lednew dot de
I've forgotten: Please aslo add a

$result = explode("\n",trim($result));

function user_proc_exec ($command,& $result,& $status)
	{
	$process = proc_open(
			$command,
			array(
			0 => array("pipe", "r"), //STDIN
			1 => array("pipe", "w"), //STDOUT
			2 => array("pipe","w")), //STDERR
			$pipes
			);
	$result = stream_get_contents($pipes[1]);
	$error = stream_get_contents($pipes[2]);
	fclose($pipes[1]);
	fclose($pipes[2]);
	$status= proc_close($process);
	$result = explode("\n",trim($result));
    return($error);
	}
 [2010-02-18 02:09 UTC] simeonl at dbc dot co dot nz
I got the same problem with a CLI script running repeated exec commands.  

PHP 5.2.11-0.dotdeb.1 with Suhosin-Patch 0.9.7 (cli) (built: Sep 19 2009 23:21:37)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2009 Zend Technologies
    with Suhosin v0.9.29, Copyright (c) 2007, by SektionEins GmbH

(Debian - Lenny)

Adopting the suggested "user_proc_exec" workaround has fixed the problem, which does suggest the exec might not be closing it's pipes properly.
 [2010-02-19 00:26 UTC] simeonl at dbc dot co dot nz
OK, sorry, further testing revealed that klaus' fix hadn't worked for me after all, and the problem *doesn't* seem to be in exec.  I can't be certain that we even have the same problem.

I was able to track it down though by using a function like this:

// Before using this method, you need to run "apt-get install lsof" on your server.
public static function countOpenFiles()
{
  return count(explode("\n", trim(shell_exec("lsof -p '" . posix_getpid() . "'" ))));
}

Logging the output at key points helped me track down the handle "leak", which turned out to be PDO (using persistent connections fixed it).
 [2021-11-22 22:57 UTC] ciprian dot dimofte dot 1982 at gmail dot com
This saved my life. And it's sooo elegant ! :)

function user_proc_exec ($command,& $result,& $status)
	{
	$process = proc_open(
			$command,
			array(
			0 => array("pipe", "r"), //STDIN
			1 => array("pipe", "w"), //STDOUT
			2 => array("pipe","w")), //STDERR
			$pipes
			);
	$result = stream_get_contents($pipes[1]);
	$error = stream_get_contents($pipes[2]);
	fclose($pipes[1]);
	fclose($pipes[2]);
	$status= proc_close($process);
	$result = explode("\n",trim($result));
    return($error);
	}
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Dec 05 10:01:32 2024 UTC