php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #58927 Fragmentation stops cache being used
Submitted: 2009-10-29 03:10 UTC Modified: 2013-02-18 00:35 UTC
Votes:1
Avg. Score:3.0 ± 0.0
Reproduced:0 of 0 (0.0%)
From: apc at tequilasolutions dot com Assigned:
Status: No Feedback Package: APC (PECL)
PHP Version: 5.3.0 OS: Centos 5.3
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2009-10-29 03:10 UTC] apc at tequilasolutions dot com
Description:
------------
In APC 3.0.19, it would build to 60000 user entries before filling, now over a number of hours it gets very fragmented and now it can't seem to store anything as I have usually 150Mb+ free but adding new items causes a cache full.

3.0.19 would also lock the server on being full, 3.1.3 no longer causes the server to lock up which is good but it seems its not really using the cache properly as now its struggling to store more than 1000 items before a cache full event happens.

Reproduce code:
---------------
Feast your eyes on this.

http://heart-web.tequilasolutions.com/apc_cache.php

Expected result:
----------------
Memory defrag a better on cache full.

Actual result:
--------------
Cache not being used properly because there's apparently not enough (contiguous ?) space to store new items.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2009-12-09 08:18 UTC] apc at tequilasolutions dot com
I've found its ok if the cache never hits cache full.  By storing less stuff it manages to store more items despite the fragmentation.
 [2009-12-16 03:34 UTC] daniz at rocketmail dot com
I was experiencing the same thing on 3.0.19! Php-spawner (php-fpm or php-cgi standalone) Just stalled when APC hit 100% cache. And it was impossible to guess the problem, luckely guessing it was APC and checking this bug report confirms it.

Regarding the fragmentation I can confirm this on 3.1.3p1  (where in 3.0.19 it was much much 'cleaner'). No crash yet which is great, and I have no input on it not using 100% of the memory when reaching the limit yet...

http://img194.imageshack.us/img194/1395/apc.png
 [2009-12-16 08:28 UTC] daniz at rocketmail dot com
My cache has now reached it's maximum and can say that APC cleared ALL cache. This isn't the behaviour I was expecting. I have loads of small cache that are suppsed to expire since they have a ttl. After their ttl they should be overwritten with new cache sets thus I should never use 100% of the memory (since I never did before with memcache). The cache sets that have 0 ttl should NEVER get deleted (as they do with the clear when 100% was reached).

Hoping this will be sorted out in some way or I'll have to go back to memcached where it worked, but speedwise isn't near APC cache...
 [2009-12-16 11:46 UTC] rasmus@php.net
The TTL is used for opportunistic cache expiration.  It is 
not used when the cache hits %100 full.  It is simply too 
slow to walk through the entire cache looking for entries to 
TTL out.  When we hit 100% (which you should try hard to 
avoid) we dump the entire cache because that can be done 
quickly.

The user cache was not designed to hold a lot of short 
expiry entries.  The single lock design simply doesn't 
support that.  It is for very static blocks of data that 
ideally doesn't change for weeks.
 [2009-12-16 14:24 UTC] daniz at rocketmail dot com
Thanks rasmus for the clarification. I understand having a garbage collector that checks all the ttl's would be a performance killer. I'll just rethink my cache strategy so that it doesn't consume that much memory.

But basically the ttl has no inpact on the memory usage, assigned to APC, other than if another cache->set is done in the 'future'? Then it will be replaced (cleared/re-used) if it was set weeks ago?
 [2009-12-16 14:43 UTC] rasmus@php.net
Like I said, it is used for opportunistic expiration.  That 
means that if in the case of hash collision either on a 
lookup or an insert we happen to come across an expired 
entry, we will remove it.  You can improve the odds of 
getting a lot of hash collisions by reducing the number of 
hash slots, with the trade-off of course being that the more 
hash collisions you have, the slower it will be since it 
will have to walk the linked list of entries tied to that 
hash.  But you will improve your opportunistic garbage 
collection.
 [2010-10-04 11:56 UTC] mxxcon at gmail dot com
I'm in the process of deploying a new Ubuntu 10.04 server for our web cluster and I noticed a rather extreme memory fragmentation.
Screenshot from APC's control panel: http://imgur.com/LU1Gh.png
Screenshot from eAccelerator's control panel: http://imgur.com/od1SN.png
Both servers are configured for 256MB cache size. Both servers are in the same web-cluster serving identical content.
APC's ttl and gc are set to 600.
 [2010-10-06 09:53 UTC] joe at lastpass dot com
Definitely still the same issue with APC 3.1.4 

As soon as "Cache full count reaches" '1' in APC's console 
APC's shared memory segment is entirely unreliable -- 
apc_store reports it's able to store data, that you can't 
immediately apc_fetch, apc_delete() reports that it worked 
when you can then immediately apc_fetch the data back out, 
so both are not even telling you they failed.  

The only workaround to which I've found is clearing the 
cache entirely / reloading apache.

Likely the reason more people don't notice this is that it's 
insidious -- none of my defensive code to check return codes 
from apc_store and apc_delete is ever hit, and we use it 
strictly for cache so it's data that can be pulled if it 
fails.
 [2011-08-10 16:39 UTC] php at casret dot com
Why wasn't this a problem in the 3.0 series?  We'd like to use 
the 3.1 for the inc feature, but this is blocking us, as we do 
have many small counters than get flushed every 15 minutes.
 [2011-08-11 03:03 UTC] gopalv@php.net
I checked out this bug - this only chokes if apc_bin_* functions are used.

That pools all allocations for an entire cache-set, so the memory for expiring entries are not cleaned up (because it's a single allocation). 

If you aren't using apc_bin_* functions, re-open the bug. I will be fixing the documentation about it, soon.
 [2013-02-18 00:35 UTC] pecl-dev at lists dot php dot net
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Open". Thank you.
 
PHP Copyright © 2001-2022 The PHP Group
All rights reserved.
Last updated: Sun May 29 02:05:45 2022 UTC