|
|
php.net | support | documentation | report a bug | advanced search | search howto | statistics | random bug | login |
[2002-08-09 12:08 UTC] antoine dot bajolet at tdf dot fr
Hello, I use php 4.2.1 as stand-alone binary to proceed batch file conversions with exec() and other utilites (ie mapserver, shapelib and others) The binary is compiled with postgresql and ming support. The amount of files to convert is over 10,000. Each file needs 8 (height) exec() to be converted. After lots of loops (throusands but it varies), i obtain the error message : <br /> <b>Warning</b>: Unable to fork [/usr/local/tdf/www/src/mapserver_3.5/shp2img -m /usr/local/tdf/www/src/swfs/temp.map -o /usr/local/tdf/www/src/swfs/13441_66.png -t -d base 13441] in <b>/usr/local/tdf/www/src/shape_to_tvview.php</b> on line <b>336</b><br /> [.../...] <br /> <b>Warning</b>: Unable to fork [/usr/local/tdf/www/src/ming-0.2a/util/png2dbl /usr/local/tdf/www/src/swfs/13441_66.png] in <b>/usr/local/tdf/www/src/shape_to_tvview.php</b> on line <b>25</b><br /> <br /> When i ran a #top in the same time, i saw that pid numbers of executed programs increases during execution of the script, and the exec did not reuse pid numbers already used in the same script if the loop is too quick (when the script converts small shapefiles). the highest number i saw was The total number of processes on the system was not over 71 in the same time. Any idea ? PatchesPull RequestsHistoryAllCommentsChangesGit/SVN commits
|
|||||||||||||||||||||||||||||||||||||
Copyright © 2001-2024 The PHP GroupAll rights reserved. |
Last updated: Wed Apr 24 13:01:29 2024 UTC |
Im using PHP 5.2.4 (cli), Apache & Mysql combination on a Slackware box. While running a script that processes large XML files and writes the log to a log file, I get Warning: exec(): Unable to fork Initially the script runs fine for some 3 or 4 hours after which it gives out the warnings. The script has a huge amount of data to process. I also tried to use the nohup command but was of no use. A screenshot of the problem is given below: DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13001','Microsoft','Windows XP Home','0.0.0SP1','0','0','0','SP1','Software') Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13001','Microsoft','Windows XP Home','0.0.0SP1','0','0','0','SP1','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454 DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13002','Microsoft','Windows XP Professional','0.0.0SP1','0','0','0','SP1','Software') Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13002','Microsoft','Windows XP Professional','0.0.0SP1','0','0','0','SP1','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454 DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13134','KaZaA','KaZaA Media Desktop','2.0.0','2','0','0','','Software') Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('13134','KaZaA','KaZaA Media Desktop','2.0.0','2','0','0','','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454 DEBUG:INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('11051','KaZaA','KaZaA Media Desktop','1.6.1','1','6','1','','Software') Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into package (id,vendor,title,version,ver_major,ver_minor,ver_sub,ver_ext,type) values ('11051','KaZaA','KaZaA Media Desktop','1.6.1','1','6','1','','Software') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454 DEBUG:INSERT into infection_vector (vector,deployed,effectiveness) values ('File Sharing / Peer to Peer / Kazaa','3','8') Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into infection_vector (vector,deployed,effectiveness) values ('File Sharing / Peer to Peer / Kazaa','3','8') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454 DEBUG:INSERT into infection_vector (vector,deployed,effectiveness) values ('File Transfer / Email Attachment','10','7') Warning: exec(): Unable to fork [/usr/bin/bash -c "exec nohup setsid echo \"DEBUG: INSERT into infection_vector (vector,deployed,effectiveness) values ('File Transfer / Email Attachment','10','7') \">>feed2vendorDB.log 2>&1 &"] in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1454 Fatal error: Out of memory (allocated -1155268608) (tried to allocate 11080 bytes) in /usr/local/apache2/htdocs/xml_feed/sircc_agnostic/feed2vendorDB/feed2vendorDB.php on line 1447 [1]+ Exit 1 ./feed2vendorDB.sh Is this related to the max number of processes allowed in the memory, or a PHP-Apache prob??. Having a tough time ....Have same trouble in PHP snapshot 200805080630: PHP Warning: exec(): Unable to fork [echo -ne '2008-06-01 05:15:27: Login: Unable to login (User: helpme@simpla y.ru)! reason:not connected! (CMD:LOGIN) sleep 50 seconds ' >> /home/pasha/temp/php-IMAP/logs/log_ERR 2>&1] in /var/www/_SHARED_/Debug/HuLOG.php on line 115 PHP Stack trace: PHP 1. {main}() /home/pasha/temp/php-IMAP/IMAP_answer.php:0 PHP 2. IMAP_mailbox_change_answer() /home/pasha/temp/php-IMAP/IMAP_answer.php:201 PHP 3. HuLOG->toLog() /home/pasha/temp/php-IMAP/IMAP_answer.php:67 PHP 4. HuLOG->writeLogs() /var/www/_SHARED_/Debug/HuLOG.php:158 PHP 5. HuLOG->log_to_file() /var/www/_SHARED_/Debug/HuLOG.php:162 PHP 6. exec() /var/www/_SHARED_/Debug/HuLOG.php:115 Warning: exec(): Unable to fork [echo -ne '2008-06-01 05:15:27: Login: Unable to login (User: helpme@simplay.ru) ! reason:not connected! (CMD:LOGIN) sleep 50 seconds ' >> /home/pasha/temp/php-IMAP/logs/log_ERR 2>&1] in /var/www/_SHARED_/Debug/HuLOG.php on line 115 Call Stack: 0.0019 375364 1. {main}() /home/pasha/temp/php-IMAP/IMAP_answer.php:0 184145.8861 16442044 2. IMAP_mailbox_change_answer() /home/pasha/temp/php-IMAP/IMAP_answer.php:201 184145.8903 16451552 3. HuLOG->toLog() /home/pasha/temp/php-IMAP/IMAP_answer.php:67 184145.8908 16451648 4. HuLOG->writeLogs() /var/www/_SHARED_/Debug/HuLOG.php:158 184145.8908 16451648 5. HuLOG->log_to_file() /var/www/_SHARED_/Debug/HuLOG.php:162 184145.8917 16451824 6. exec() /var/www/_SHARED_/Debug/HuLOG.php:115[Ubuntu karmic 9.10; php-cli 5.2.10] > The per-user process number limit is set in > /etc/security/limits.conf. > In my case, it was enough to raise it and reboot. In my case, this wasn't the answer, unfortunately. Or fortunately, because I was forced to find a workaround, which could help others, too. In my code I replaced exec by following function: function user_proc_exec ($command,& $result,& $status) { $process = proc_open( $command, array( 0 => array("pipe", "r"), //STDIN 1 => array("pipe", "w"), //STDOUT 2 => array("pipe","w")), //STDERR $pipes ); $result = stream_get_contents($pipes[1]); $error = stream_get_contents($pipes[2]); fclose($pipes[1]); fclose($pipes[2]); $status= proc_close($process); return($error); } Everything is working fine now.I've forgotten: Please aslo add a $result = explode("\n",trim($result)); function user_proc_exec ($command,& $result,& $status) { $process = proc_open( $command, array( 0 => array("pipe", "r"), //STDIN 1 => array("pipe", "w"), //STDOUT 2 => array("pipe","w")), //STDERR $pipes ); $result = stream_get_contents($pipes[1]); $error = stream_get_contents($pipes[2]); fclose($pipes[1]); fclose($pipes[2]); $status= proc_close($process); $result = explode("\n",trim($result)); return($error); }I got the same problem with a CLI script running repeated exec commands. PHP 5.2.11-0.dotdeb.1 with Suhosin-Patch 0.9.7 (cli) (built: Sep 19 2009 23:21:37) Copyright (c) 1997-2009 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2009 Zend Technologies with Suhosin v0.9.29, Copyright (c) 2007, by SektionEins GmbH (Debian - Lenny) Adopting the suggested "user_proc_exec" workaround has fixed the problem, which does suggest the exec might not be closing it's pipes properly.OK, sorry, further testing revealed that klaus' fix hadn't worked for me after all, and the problem *doesn't* seem to be in exec. I can't be certain that we even have the same problem. I was able to track it down though by using a function like this: // Before using this method, you need to run "apt-get install lsof" on your server. public static function countOpenFiles() { return count(explode("\n", trim(shell_exec("lsof -p '" . posix_getpid() . "'" )))); } Logging the output at key points helped me track down the handle "leak", which turned out to be PDO (using persistent connections fixed it).This saved my life. And it's sooo elegant ! :) function user_proc_exec ($command,& $result,& $status) { $process = proc_open( $command, array( 0 => array("pipe", "r"), //STDIN 1 => array("pipe", "w"), //STDOUT 2 => array("pipe","w")), //STDERR $pipes ); $result = stream_get_contents($pipes[1]); $error = stream_get_contents($pipes[2]); fclose($pipes[1]); fclose($pipes[2]); $status= proc_close($process); $result = explode("\n",trim($result)); return($error); }