php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #59605 RED as workaround for thundering herd problem
Submitted: 2011-02-03 08:24 UTC Modified: 2016-11-18 21:47 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:0 (0.0%)
Same OS:0 (0.0%)
From: need4spam at bk dot ru Assigned:
Status: Wont fix Package: APC (PECL)
PHP Version: 5.3.5 OS:
Private report: No CVE-ID: None
 [2011-02-03 08:24 UTC] need4spam at bk dot ru
Description:
------------
In our environment we have a problem with APC. It's 
described on wiki as thundering herd problem -
http://en.wikipedia.org/wiki/Thundering_herd_problem

If we have some long queries that are cached in APC and 
5000+ users using website. When cache expires ~300 users 
produce same query to DB that stalls it for some time.

You can use RED ( 
http://en.wikipedia.org/wiki/Random_early_drop ) to avoid 
this problem.

For example if TTL for cache record is almost ended (e.g. 
((creation_time+ttl)-current_time)/ttl < some_tunable) you 
can fail apc_fectch() for this record based on probabilistic 
function inverse proportional to ((creation_time+ttl)-
current_time)/ttl.

For now we workaround it using cronjobs or emulating RED in 
our php code, but with time it's getting more and more 
complicated.


Patches

patch-apc_red.patch (last revision 2011-12-03 15:36 UTC by need4spam at bk dot ru)

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2011-02-03 14:35 UTC] need4spam at bk dot ru
I've kinda messed up math in previous post, so there is patch 
to latest APC-svn with RED implementation. 

PS. drop_cf should not be hardcoded. Better be tunable via 
php.ini

http://nnm-club.ru/apc.patch
 [2011-02-22 03:44 UTC] gopalv@php.net
This is very interesting, I'll move it to post the 3.1.8 stable release.
 [2011-12-03 15:38 UTC] need4spam at bk dot ru
Slightly modified patch - removed floating point.
It's of cause only PoC, but it has pretty good results:
http://img17.imageshack.us/img17/6543/apcthunderingherdpatch.png
 [2016-11-18 21:47 UTC] kalle@php.net
-Status: Open +Status: Wont fix
 [2016-11-18 21:47 UTC] kalle@php.net
APC is no longer supported in favor of opcache that comes bundled with PHP, if you wish to use the user cache, then look at PECL/APCu.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Sat Dec 21 13:01:31 2024 UTC