php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #59249 Possible Stale Lock in apc_store()
Submitted: 2010-06-02 00:17 UTC Modified: 2010-06-02 17:39 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:0 (0.0%)
Same OS:0 (0.0%)
From: alex at malexmedia dot net Assigned:
Status: Suspended Package: APC (PECL)
PHP Version: 5.1.6 OS: Centos 5 (Linux)
Private report: No CVE-ID: None
View Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
If you reported this bug, you can edit this bug over here.
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: alex at malexmedia dot net
New email:
PHP Version: OS:

 

 [2010-06-02 00:17 UTC] alex at malexmedia dot net 
Description:
------------
Hi all. I am experiencing a very strange issue which I may need help troubleshooting...

I just recently deployed APC into my production Drupal site after several weeks of successful testing in my sandbox environment. However, twice now I have had my application go down because of what appears to be a (very) rare deadlock and/or stale lock scenario.

Each time the site has gone down, the following error has appeared in the logs:

[Mon May 24 18:49:23 2010] [error] [client 74.78.54.197] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /opt/mmndrupal/docroot/sites/all/modules/contrib/cacherouter/engines/apc.php on line 197, referer: http://malexmedia.net/search
[Tue May 18 13:24:21 2010] [error] [client 71.50.211.153] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /opt/mmndrupal/docroot/sites/all/modules/contrib/cacherouter/engines/apc.php on line 197, referer: http://malexmedia.net/search

Line 197 of apc.php is a simple call to apc_store:

apc_store($this->lookup, $lookup, 0);

I am not 100% confident what was happening immediately prior to the error, but it _seems_ as though some calls to apc_store() are occasionally locking for an obscene amount of time.

That would be bad enough, but once PHP has terminated the long-running thread, the whole site goes down.

In my investigation, I have found that the web server will still serve simple files, AND I have found that basic PHP scripts will run without a problem. However, calls to the APC api result in an eternally-locked page load. (Attempts to load the APC dashboard never complete. All calls to my Drupal application never complete. Etc..)

So far, I have seen this problem happen only twice, and I do not yet know how to reproduce it. But even when it does happen, I am at a loss as to how to troubleshoot the error.

I realize this is probably not enough information to constitute a real bug report; my real hope here is that someone can assist me in troubleshooting this issue until valuable debugging information comes out.

(By the way, I have considered running a newer version of APC, but it currently will not compile. See Bug #17553)

Thanks for your time,
--Alex

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2010-06-02 00:40 UTC] rasmus@php.net 
Are you writing a lot to the user cache?  There is a global 
lock and if you have multiple processes constantly writing 
to the user cache, I can see this happening.  The APC user 
cache is designed for very infrequent writes and very very 
frequent reads.  If you are writing more than a couple of 
times a day on a busy site, you are using the wrong 
mechanism.

Having said that, PHP 5.1.6?  That's a version from 2006.  4 
years is a generation in Internet time and we no longer 
support that version at all.  Please upgrade.
 [2010-06-02 07:46 UTC] alex at malexmedia dot net 
Well, my application is definitely designed to store a lot in the user cache. That said, it is certainly intended to be read-heavy, not write-heavy. (That is that once the user cache is populated, the application should run with only occasional writes.)

As for the PHP version, I agree that PHP 5.1.6 is old, but that is the latest version being shipped currently with RHEL and related distros. I can see taking a stance against supporting versions being shipped by small or irresponsible vendors (to force the vendor to behave) but excluding all users of RHEL seems like a decision one would not want to make in haste.

Let me know what you think, I want to do the right thing technically as well as the right thing for my business here.
 [2010-06-02 07:56 UTC] alex at malexmedia dot net 
As for the locking issue, are you suggesting that apc_store() could have been waiting tens of seconds for thousands of other threads to complete apc_store() transactions?

I don't think I buy that. Both times the crash occurred, the user cache had been fully populated, so any writes should have been very occasional. Besides that, by the time we call apc_*() with the intention of writing, we already have the content we want to write, so there should be no waiting on our end to complete the transaction with the user cache.

Let me know what you think,
--Alex
 [2010-06-02 17:34 UTC] rasmus@php.net 
RHEL/Centos is the only major distribution that is so far 
behind on their PHP version.  Every other distro out there 
has long since moved on, and we announced the end of life of 
PHP 5.1 in November 2006, but continued to provide support 
for it for 2 years past that date.  We are now an additional 
2 years beyond that and we refuse to be held hostage by slow 
moving distros.  None of the developers have a 5.1 tree 
anymore.  We don't test extensions against it and we don't 
fix bugs in it.  If you can reproduce it on a current 
version, I'll have a look.
 [2010-06-02 17:39 UTC] alex at malexmedia dot net 
I guess that makes sense. Thank you for clearly stating the situation. I'll work on an upgrade plan for my environments, and I'll report back if I continue to have any trouble.

Thanks, and have a great day!
--Alex
 [2014-08-13 23:03 UTC] frc dot gabriel at gmail dot com 
debian 7
php-apc version 3.1.13-1
php5    version 5.4.4-14+deb7u9

We are facing exactly the same problem with the function apc_store taking more than 120 seconds and causing php-fpm crashes.


Regards.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Sat Dec 21 11:01:30 2024 UTC