php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #59605 RED as workaround for thundering herd problem
Submitted: 2011-02-03 08:24 UTC Modified: 2016-11-18 21:47 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:0 (0.0%)
Same OS:0 (0.0%)
From: need4spam at bk dot ru Assigned:
Status: Wont fix Package: APC (PECL)
PHP Version: 5.3.5 OS:
Private report: No CVE-ID: None
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: need4spam at bk dot ru
New email:
PHP Version: OS:

 

 [2011-02-03 08:24 UTC] need4spam at bk dot ru
Description:
------------
In our environment we have a problem with APC. It's 
described on wiki as thundering herd problem -
http://en.wikipedia.org/wiki/Thundering_herd_problem

If we have some long queries that are cached in APC and 
5000+ users using website. When cache expires ~300 users 
produce same query to DB that stalls it for some time.

You can use RED ( 
http://en.wikipedia.org/wiki/Random_early_drop ) to avoid 
this problem.

For example if TTL for cache record is almost ended (e.g. 
((creation_time+ttl)-current_time)/ttl < some_tunable) you 
can fail apc_fectch() for this record based on probabilistic 
function inverse proportional to ((creation_time+ttl)-
current_time)/ttl.

For now we workaround it using cronjobs or emulating RED in 
our php code, but with time it's getting more and more 
complicated.


Patches

patch-apc_red.patch (last revision 2011-12-03 15:36 UTC by need4spam at bk dot ru)

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2011-02-03 14:35 UTC] need4spam at bk dot ru
I've kinda messed up math in previous post, so there is patch 
to latest APC-svn with RED implementation. 

PS. drop_cf should not be hardcoded. Better be tunable via 
php.ini

http://nnm-club.ru/apc.patch
 [2011-02-22 03:44 UTC] gopalv@php.net
This is very interesting, I'll move it to post the 3.1.8 stable release.
 [2011-12-03 15:38 UTC] need4spam at bk dot ru
Slightly modified patch - removed floating point.
It's of cause only PoC, but it has pretty good results:
http://img17.imageshack.us/img17/6543/apcthunderingherdpatch.png
 [2016-11-18 21:47 UTC] kalle@php.net
-Status: Open +Status: Wont fix
 [2016-11-18 21:47 UTC] kalle@php.net
APC is no longer supported in favor of opcache that comes bundled with PHP, if you wish to use the user cache, then look at PECL/APCu.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Sun Dec 22 01:01:30 2024 UTC