Description:
------------
In production, we see an occasional deadlock in APC.
It's clearly some kind of MP hazard, as it occurs far
more often on the machines with more CPUs, and never
happens at all without a production load on the server.

It always starts like this:

Jan 19 17:47:14 web182 httpd[6439]: [error] [client 
10.0.0.18] PHP Fatal error:  include() [<a 
href='function.include'>function.include</a>]: unable to 
obtain pthread lock (EDEADLK) in /home/html/__autoload.php
Jan 19 17:47:14 web182 httpd[6439]: [error] [client 
10.0.0.15] PHP Fatal error:  Unknown: unable to obtain 
pthread lock (EDEADLK) in Unknown on line 0...

Then that last line is repeated with no other server 
activity until Apache restart.

Gdb some minutes later shows this:
(gdb) attach 6439
...
(gdb) where
#0  0x00002aea10d20920 in __accept_nocancel () from 
/lib64/libpthread.so.0
#1  0x00002aea10b061e4 in apr_socket_accept () from 
/usr/lib64/libapr-1.so.0
#2  0x00002aea0f249d01 in unixd_accept () from 
/usr/sbin/httpd
#3  0x00002aea0f2487a7 in ?? () from /usr/sbin/httpd
#4  0x00002aea0f248aea in ?? () from /usr/sbin/httpd
#5  0x00002aea0f24934d in ap_mpm_run () from /usr/sbin/httpd
#6  0x00002aea0f223e48 in main () from /usr/sbin/httpd
(gdb) p *apc_cache->header
$7 = {lock = {__data = {__lock = 0, __nr_readers = 0, 
__readers_wakeup = 998, __writer_wakeup = 39, 
__nr_readers_queued = 74, 
      __nr_writers_queued = 0, __writer = 6439, __shared = 
128, __pad1 = 0, __pad2 = 0, __flags = 1}, 
    __size = "\000\000\000\000\000\000\000\000?
\003\000\000'\000\000\000J\000\000\000\000\000\000\000'\031\
000\000\200", '\0' <repeats 19 times>, 
"\001\000\000\000\000\000\000", __align = 0}, wrlock = 
{__data = {__lock = 0, __nr_readers = 0, __readers_wakeup = 
0, __writer_wakeup = 0, 
      __nr_readers_queued = 0, __nr_writers_queued = 0, 
__writer = 0, __shared = 128, __pad1 = 0, __pad2 = 0, 
__flags = 1}, 
    __size = '\0' <repeats 28 times>, "\200", '\0' <repeats 
19 times>, "\001\000\000\000\000\000\000", __align = 0}, 
num_hits = 88732964, 
  num_misses = 1967, num_inserts = 1469, expunges = 0, 
deleted_list = 0x2aea1fb08918, start_time = 1295474742, busy 
= 0 '\0', 
  num_entries = 1376, mem_size = 62166880, lastkey = {h = 0, 
keylen = 0, mtime = 0, pid = 0}}

Since apc can die while holding the write lock,
maybe the first time a new apc invocation gets EDEADLK from
pthread_rwlock_{rd,wr}lock it should just be a warning and 
not an error?

---

APC configured with --enable-apc-pthreadrwlocks, although 
deadlock also happens with the default pthread locks.