php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #33281 Possible deadlock in PHP-cleanup under heavy load
Submitted: 2005-06-09 09:49 UTC Modified: 2005-06-19 01:00 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:1 (100.0%)
From: jimmy dot makela at loopia dot se Assigned:
Status: No Feedback Package: Apache related
PHP Version: 4.3.11 OS: FreeBSD 4.9-RELEASE-p10
Private report: No CVE-ID: None
View Developer Edit
 [2005-06-09 09:49 UTC] jimmy dot makela at loopia dot se 
Description:
------------
Periodically on one of our webservers one apache-process starts using up all CPU until it is killed.

A ktrace of the process gives the output:
 63458 httpd    CALL  sigprocmask(0x1,0x280a1060,0xbfbff540)
 63458 httpd    RET   sigprocmask 0
 63458 httpd    CALL  sigprocmask(0x3,0xbfbff540,0)
 63458 httpd    RET   sigprocmask 0
 63458 httpd    CALL  sigprocmask(0x1,0x280a1060,0xbfbff540)
 63458 httpd    RET   sigprocmask 0
 63458 httpd    CALL  sigprocmask(0x3,0xbfbff540,0)
 63458 httpd    RET   sigprocmask 0

ad infinitum.

When attaching to the process with GDB and doing a backtrace, we get the following update:
#0  0x28092648 in symlook_obj () from /usr/libexec/ld-elf.so.1
#1  0x28091032 in dlclose () from /usr/libexec/ld-elf.so.1
#2  0x283b3251 in object.11 () from /usr/local/libexec/apache/libphp4.so
#3  0x283b4bfc in object.11 () from /usr/local/libexec/apache/libphp4.so
#4  0x283b4d3a in object.11 () from /usr/local/libexec/apache/libphp4.so
#5  0x283b0a00 in object.11 () from /usr/local/libexec/apache/libphp4.so
#6  0x2838cb7a in object.11 () from /usr/local/libexec/apache/libphp4.so
#7  0x2838cb57 in object.11 () from /usr/local/libexec/apache/libphp4.so
#8  0x283c911d in object.11 () from /usr/local/libexec/apache/libphp4.so
#9  0x80557e8 in ap_child_exit_modules ()
#10 0x805a7b1 in clean_child_exit ()
#11 0x805bba2 in just_die ()
#12 0xbfbfffac in ?? ()
#13 0x28091f83 in r_debug_state () from /usr/libexec/ld-elf.so.1
#14 0x28091e1e in r_debug_state () from /usr/libexec/ld-elf.so.1
#15 0x2809021b in find_symdef () from /usr/libexec/ld-elf.so.1
#16 0x2808fa78 in _rtld_bind () from /usr/libexec/ld-elf.so.1
#17 0x2808f461 in _rtld_bind_start () from /usr/libexec/ld-elf.so.1
#18 0x805d3dd in make_child ()
#19 0x805d64c in perform_idle_server_maintenance ()
#20 0x805db5d in standalone_main ()
#21 0x805e0a7 in main ()
#22 0x804fd0e in _start ()

which to me suggest that something in the PHP cleanup code is deadlocking.

Any help in getting to the bottom of this would be greatly appreciated. If you need any additional information, we will provide it.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2005-06-09 16:45 UTC] rasmus@php.net 
A couple of questions.

1. Why is gdb just reporting object.11 instead of a real symbol?  Have the symbols been stripped from your libphp4.so?  Could you recompile and put them back to get a better backtrace?

2. Do you have any custom extensions loaded?  And if you do, are they in C++?  The FreeBSD4 rtld has notoriously bad support for C++ shared libraries.
 [2005-06-09 22:17 UTC] jimmy dot makela at loopia dot se 
Regarding your questions:

1. I don't know, I was hoping for a suggestion to that part. The library have not been stripped manually, and doesn't seem to be stripped. A recompile could be done if that would help, just specify what to change.

# file /usr/local/libexec/apache/libphp4.so 
/usr/local/libexec/apache/libphp4.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (FreeBSD), not stripped

objdump -T does show symbols, but I have no idea of what object.11 maps to. I can provide a complete list of the dynamic symbol table if that helps.

2. A list of the loaded extensions follows:
php4-bcmath-4.3.11_1
php4-calendar-4.3.11_1
php4-ctype-4.3.11_1
php4-curl-4.3.11_1
php4-domxml-4.3.11_1
php4-exif-4.3.11_1
php4-extensions-1.0
php4-ftp-4.3.11_1
php4-gd-4.3.11_1
php4-gettext-4.3.11_1
php4-iconv-4.3.11_1
php4-imap-4.3.11_1
php4-mcrypt-4.3.11_1
php4-mysql-4.3.11_1
php4-overload-4.3.11_1
php4-pcre-4.3.11_1
php4-posix-4.3.11_1
php4-session-4.3.11_1
php4-tokenizer-4.3.11_1
php4-xml-4.3.11_1
php4-zlib-4.3.11_1

Let me know if you need additional information.
 [2005-06-10 07:58 UTC] jimmy dot makela at loopia dot se 
When it happened again yesterday the last request before the problem occured (and the process started looping) was a script using GD-functions for creating thumbnails, so GD is the prime suspect.

The version of the GD-port installed is gd-2.0.33_1,1

The GD-library is not stripped either, and it is compiled with standard options. We have verified that the dynamic symbols look ok, I'm guessing that it is GDB which is confused by the dlopen'ed library?

I'll try to create a small script for reproducing the problem and stress-testing that script.
 [2005-06-10 09:41 UTC] rasmus@php.net 
Which gdb version are you using?  You might get some better results with a more recent gdb than what came with FreeBSD 4.9.  Also, could you do an "info shared" in gdb and add the results here?  I am mainly interested in seeing whether those object.11 addresses actually do fall within the libphp4.so address space.  I suppose they must since gdb says so and the range is pretty tight so they look like they are all from the same shared library.  It will of course also tell me which other libraries you have loaded.  And yes, the simpler you can make the reproducing example, the better.
 [2005-06-11 15:20 UTC] sniper@php.net 
And I'd use the bundled GD anyway. (drop the path from --with-gd)
 [2005-06-19 01:00 UTC] php-bugs at lists dot php dot net 
No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Sat Dec 21 11:01:30 2024 UTC