php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #57599 Lots of 40Bytes allocs that dont defrag
Submitted: 2007-03-29 04:56 UTC Modified: 2007-04-03 03:32 UTC
From: steve at tequilasolutions dot com Assigned:
Status: Closed Package: APC (PECL)
PHP Version: 5.2.1 OS: FC4
Private report: No CVE-ID: None
View Developer Edit
 [2007-03-29 04:56 UTC] steve at tequilasolutions dot com 
Description:
------------
One for Gopal this perhaps as he fixed an earlier bug.

The initial release of 3.0.13 led to quick fragmentation with loads of 16B entries.   A fix involving some byte aligning in sma_alloc reduced this however now over several hours you get left with lots of 40B entries instead which eventually cause the cache to fill up.

Reproduce code:
---------------
http://web01.tequilasolutions.com/apc_cache.php

Expected result:
----------------
Some sort of defrag going on.

Actual result:
--------------
Frags build and the cache fills.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2007-03-30 15:02 UTC] gopalv82 at yahoo dot com 
I debugged this one to bits, but couldn't find exactly any problem in the code. But a fair bit of testing showed up some interesting statistics on allocations.

http://t3.dotgnu.info/blog/php/apc-and-40bytes.html

So memory sizes which don't lend itself well to splitting (i.e 2*block_size + alignword(1)) turn up as fragments much more easily. 

Maybe a special case to handle those ? I'm sort of poking around for a good solution.

Still not sure how to fix the excessive fragmentation.
 [2007-03-30 15:24 UTC] rasmus@php.net 
I poked the allocation distribution a while ago as well.  And I went back and forth on padding all allocs up to a certain size to avoid small chunks.  Things have shuffled a bit now and based on the current alloc distribution profile, it looks to me like all allocs should be in multiples of 24-byte chunks.  And yes, this does mean higher memory usage since there will be more padding on average, but we will have more uniform and useful fragments to work with.
 [2007-03-31 19:13 UTC] gopalv82 at yahoo dot com 
Here's a patch which minimizes fragmentation (again)

http://t3.dotgnu.info/code/apc-fraglimit.patch

It only consumes extra memory if your memory is already fragmented, which is better than padding up everything to multiples of 24 and stuff.

Test and confirm, please.
 [2007-04-01 18:41 UTC] steve at tequilasolutions dot com 
Sorry not quite sure how to apply the patch, I unwrapped back to the 3.0.13 dist and then did....

[root@web01 APC-3.0.13]# patch < apc-fraglimit.patch
patching file apc_sma.c
Hunk #3 FAILED at 131.
Hunk #4 FAILED at 162.
Hunk #5 succeeded at 191 (offset -1 lines).
Hunk #7 succeeded at 241 (offset -1 lines).
Hunk #9 succeeded at 509 (offset -1 lines).
2 out of 9 hunks FAILED -- saving rejects to file apc_sma.c.rej
 [2007-04-01 18:44 UTC] rasmus@php.net 
It's against current CVS HEAD.

cvs -d:pserver:cvsread@cvs.php.net:/repository login
password: phpfi
cvs -d:pserver:cvsread@cvs.php.net:/repository co pecl/apc

Then apply the patch in the pecl/apc directory
 [2007-04-01 19:05 UTC] steve at tequilasolutions dot com 
OK - rebuilt and installed against CVS head, lets see what happens.....
 [2007-04-01 19:13 UTC] steve at tequilasolutions dot com 
Looks good so far get the champagne/cider ready...
 [2007-04-02 04:47 UTC] steve at tequilasolutions dot com 
Been running for 7 hours now with 28000 user entries, hardly any frags - looks really good.
 [2007-04-02 04:48 UTC] gopalv82 at yahoo dot com 
Any memory usage differences ? (% of used and stuff ?)
 [2007-04-02 05:04 UTC] steve at tequilasolutions dot com 
hard to tell as there's thousands of entries but I think it used to manage about 30000 + opcode cache in 128Mb so it looks about the same
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Thu Apr 24 23:01:28 2025 UTC