php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #11676 A few apache children consume all memory and CPU.
Submitted: 2001-06-25 17:40 UTC Modified: 2002-01-02 15:08 UTC
From: valerio at wnet dot it Assigned:
Status: Closed Package: Scripting Engine problem
PHP Version: 4.0 Latest CVS (2001-06-25) OS: linux 2.4.5 i386
Private report: No CVE-ID: None
 [2001-06-25 17:40 UTC] valerio at wnet dot it
I'm using php 4.0.7-dev (downloaded from the CVS tree), Apache 1.3.20 on a P3 600 1 Gig RAM.
After some hours that the server is running, in no particularly loaded confdition, CPU and memory utilization grows abnormally, making the server unusable.
"top" shows that 4 o 5 httpd processes are eating 20% CPU each, and are all in "running" state. I managed to attach a gdb to one of these processes, and here's the result:

Attaching to program `/usr/local/apache/bin/httpd', Pid 21784
Reading symbols from /lib/libpam.so.0...done.
Reading symbols from /lib/libdl.so.2...done.
Reading symbols from /lib/libcrypt.so.1...done.
Reading symbols from /lib/libresolv.so.2...done.
Reading symbols from /lib/libm.so.6...done.
Reading symbols from /lib/libnsl.so.1...done.
Reading symbols from /lib/libdb.so.3...done.
Reading symbols from /lib/libc.so.6...done.
Reading symbols from /lib/ld-linux.so.2...done.
Reading symbols from /lib/libnss_compat.so.2...done.
Reading symbols from /lib/libnss_files.so.2...done.
0x4012c23e in chunk_free ()
(gdb) bt
#0  0x4012c23e in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x80fbcbf in shutdown_memory_manager ()

This is all I get. I saw MANY MANY bug reports with similarities to this one in the db. The only solution is stopping and restarting apache. In apache's error_log i get many lines with "child xxx still did not exit...sending a SIGTERM" and "child xxx still did not exit...sending a SIGKILL" as soon as i stop it.
Let me know if I can make something to help you out more!

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2001-06-26 07:04 UTC] sniper@php.net
It would help a lot to track this down if you could find
out if this happens by running some script.

 [2001-06-26 07:32 UTC] valerio at wnet dot it
Well, the problem is that we have many virtual hosts on that server, each using it's own log file...
I can confirm that the gdb trace for ALL the many dead process, which are in running state, gives the three lines:
#0  0x4012c23e in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x80fbcbf in shutdown_memory_manager ()
(it happened this morning too)

Is there some way to know which script was executing just before the child died, please tell me! 
 [2001-06-26 14:31 UTC] zeev@php.net
Output buffering indeed cannot be used inside output handler functions.  However, the code that handled that error situation contained a bug that caused PHP to enter an infinite loop.  This bug is now fixed (although the limitation of not being able to use output buffering inside output handlers still applies)
 [2001-06-26 14:32 UTC] zeev@php.net
Ignore the last comment, wrong bug report :)
 [2001-06-27 11:57 UTC] sniper@php.net
what was the configure line used to configure PHP ?

 [2001-06-27 14:30 UTC] valerio at wnet dot it
Here it is:
CFLAGS="-O9 -funroll-loops -ffast-math -malign-double -mcpu=pentiumpro -march=pentiumpro -fomit-frame-pointer -fno-exceptions" \
./configure \
--with-apache=../apache_1.3.20 \
--with-mysql \
--disable-debug \
--with-gnu-ld \
--enable-memory-limit \
--enable-inline-optimization

I don't remember exactly, but i think i had tried removing ALL gcc optimization options, with the same results...
During another crash (pre-crash actually :) i managed to backtrace a httpd which had this stack:

#0  0x4012c238 in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x80fb8f2 in _efree ()

The other httpd i backtraced had all the stack like this:

#0  0x4012c23e in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x80fbcbf in shutdown_memory_manager ()

Hope this helps...
 [2001-06-28 07:09 UTC] derick@php.net
You don't need to set the CFLAGS yourself, the -O9 is known to cause problems.
Please try to compile without setting ANY CFLAGS yourself.

Derick
 [2001-06-28 15:21 UTC] valerio at wnet dot it
ok, i have recompiled php AND apache without any optimization settings, and things look worse...
The problem happened more frequently in these hours, and these are backtraces of stuck processes:

#0  0x4012c243 in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x810acbc in shutdown_memory_manager ()
#3  0x8079d58 in php_request_shutdown ()
#4  0x8077611 in php_apache_request_shutdown ()
#5  0x814a93e in run_cleanups ()
#6  0x814916d in ap_clear_pool ()
#7  0x81491e1 in ap_destroy_pool ()
#8  0x8158ded in child_main ()
#9  0x8158fdc in make_child ()
#10 0x8159089 in startup_children ()
#11 0x81596c6 in standalone_main ()
#12 0x8159e53 in main ()
#13 0x400ea9f3 in __libc_start_main ()

#0  0x4012c23e in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x810a5a5 in _efree ()
#3  0x811e8cc in zend_hash_destroy ()
#4  0x8112b7c in destroy_zend_class ()
#5  0x811ea67 in zend_hash_apply_deleter ()
#6  0x811ecfa in zend_hash_apply ()
#7  0x8110820 in shutdown_executor ()
#8  0x8119d53 in zend_deactivate ()
#9  0x8079d1a in php_request_shutdown ()
#10 0x8077611 in php_apache_request_shutdown ()
#11 0x814a93e in run_cleanups ()
#12 0x814916d in ap_clear_pool ()
#13 0x81491e1 in ap_destroy_pool ()
#14 0x8160eb2 in ap_destroy_sub_req ()
#15 0x8067cb8 in handle_include ()
#16 0x806acb5 in send_parsed_content ()
#17 0x806b28d in send_parsed_file ()
#18 0x814dc23 in ap_invoke_handler ()
#19 0x8161769 in process_request_internal ()
#20 0x8161b88 in ap_internal_redirect ()
#21 0x806b6dd in handle_dir ()
#22 0x814dc23 in ap_invoke_handler ()
#23 0x8161769 in process_request_internal ()
#24 0x81617cc in ap_process_request ()
#25 0x8158d9e in child_main ()
#26 0x8158fdc in make_child ()
#27 0x8159356 in perform_idle_server_maintenance ()
#28 0x8159895 in standalone_main ()
#29 0x8159e53 in main ()
#30 0x400ea9f3 in __libc_start_main ()

#0  0x4012c24b in chunk_free ()
#1  0x4012bfaa in __cfree ()
#2  0x810acbc in shutdown_memory_manager ()
#3  0x8079d58 in php_request_shutdown ()
#4  0x8077611 in php_apache_request_shutdown ()
#5  0x814a93e in run_cleanups ()
#6  0x814916d in ap_clear_pool ()
#7  0x81491e1 in ap_destroy_pool ()
#8  0x8160eb2 in ap_destroy_sub_req ()
#9  0x8067cb8 in handle_include ()
#10 0x806acb5 in send_parsed_content ()
#11 0x806b28d in send_parsed_file ()
#12 0x814dc23 in ap_invoke_handler ()
#13 0x8160e87 in ap_run_sub_req ()
#14 0x8067c35 in handle_include ()
#15 0x806acb5 in send_parsed_content ()
#16 0x806b28d in send_parsed_file ()
#17 0x814dc23 in ap_invoke_handler ()
#18 0x8161769 in process_request_internal ()
#19 0x8161b88 in ap_internal_redirect ()
#20 0x806b6dd in handle_dir ()
#21 0x814dc23 in ap_invoke_handler ()
#22 0x8161769 in process_request_internal ()
#23 0x81617cc in ap_process_request ()
#24 0x8158d9e in child_main ()
#25 0x8158fdc in make_child ()
#26 0x8159356 in perform_idle_server_maintenance ()
#27 0x8159895 in standalone_main ()
#28 0x8159e53 in main ()
#29 0x400ea9f3 in __libc_start_main ()

tell me what to do next and i'll gladly do it...
 [2001-06-29 05:39 UTC] valerio at wnet dot it
Maybe this can help...i talked to the other guy who has the same problem than us , Denis (bug #11723), and we agreed that the problems began to arise after php 4.0.4pl1 (including it or not, we don't remember exactly :( ).
Maybe this can help narrowing down the issue...
 [2001-07-13 04:43 UTC] valerio at wnet dot it
anyone alive???
 [2001-08-19 04:56 UTC] sniper@php.net
A short script that causes this would help a lot..

 [2001-08-19 05:31 UTC] valerio at wnet dot it
There's only one issue... how can I determine which script was running on the processes that i find stuck?

In the meanwhile i installed the Zend Optimizer on the server, and the problem seems to happen VERY less frequently... i know this is nearly no help at all, but I'm doing all I can.

Anyway, i see some other guy had my same problem...
 [2001-12-12 08:19 UTC] yohgaki@php.net
Could you try 4.1.0? see if it helps?
 [2001-12-12 08:20 UTC] yohgaki@php.net
Status = feedback
 [2002-01-02 13:56 UTC] lobbin@php.net
No feedback. Closing.
 [2002-01-02 15:08 UTC] valerio at wnet dot it
Ops...guess i missed the mail with the "feedback" status..
I will check as soon as i install php 4.1 (should be tomorrow if all goes well...). Don't close this one since many were experiencing the same problem than me...
i'll let you know!
 [2002-03-06 15:11 UTC] rodrigo at labhpardini dot com dot br
Valerio, did you happen to solve the bug? I'm running through practically the same bug.
Besides that I'm getting a lot of Segmentation faults in error_log. I've been able to GDB some of child httpd processes before they died and it happened with some PHP stuff too as detailed in:

http://marc.theaimsgroup.com/?l=apache-httpd-users&m=101482626416049&w=2

I'm anxious to know in what position this case is.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Fri Apr 19 23:01:28 2024 UTC