php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #57599 Lots of 40Bytes allocs that dont defrag
Submitted: 2007-03-29 04:56 UTC Modified: 2007-04-03 03:32 UTC
From: steve at tequilasolutions dot com Assigned:
Status: Closed Package: APC (PECL)
PHP Version: 5.2.1 OS: FC4
Private report: No CVE-ID: None
View Developer Edit
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: steve at tequilasolutions dot com
New email:
PHP Version: OS:

 

 [2007-03-29 04:56 UTC] steve at tequilasolutions dot com 
Description:
------------
One for Gopal this perhaps as he fixed an earlier bug.

The initial release of 3.0.13 led to quick fragmentation with loads of 16B entries.   A fix involving some byte aligning in sma_alloc reduced this however now over several hours you get left with lots of 40B entries instead which eventually cause the cache to fill up.

Reproduce code:
---------------
http://web01.tequilasolutions.com/apc_cache.php

Expected result:
----------------
Some sort of defrag going on.

Actual result:
--------------
Frags build and the cache fills.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2007-03-30 15:02 UTC] gopalv82 at yahoo dot com 
I debugged this one to bits, but couldn't find exactly any problem in the code. But a fair bit of testing showed up some interesting statistics on allocations.

http://t3.dotgnu.info/blog/php/apc-and-40bytes.html

So memory sizes which don't lend itself well to splitting (i.e 2*block_size + alignword(1)) turn up as fragments much more easily. 

Maybe a special case to handle those ? I'm sort of poking around for a good solution.

Still not sure how to fix the excessive fragmentation.
 [2007-03-30 15:24 UTC] rasmus@php.net 
I poked the allocation distribution a while ago as well.  And I went back and forth on padding all allocs up to a certain size to avoid small chunks.  Things have shuffled a bit now and based on the current alloc distribution profile, it looks to me like all allocs should be in multiples of 24-byte chunks.  And yes, this does mean higher memory usage since there will be more padding on average, but we will have more uniform and useful fragments to work with.
 [2007-03-31 19:13 UTC] gopalv82 at yahoo dot com 
Here's a patch which minimizes fragmentation (again)

http://t3.dotgnu.info/code/apc-fraglimit.patch

It only consumes extra memory if your memory is already fragmented, which is better than padding up everything to multiples of 24 and stuff.

Test and confirm, please.
 [2007-04-01 18:41 UTC] steve at tequilasolutions dot com 
Sorry not quite sure how to apply the patch, I unwrapped back to the 3.0.13 dist and then did....

[root@web01 APC-3.0.13]# patch < apc-fraglimit.patch
patching file apc_sma.c
Hunk #3 FAILED at 131.
Hunk #4 FAILED at 162.
Hunk #5 succeeded at 191 (offset -1 lines).
Hunk #7 succeeded at 241 (offset -1 lines).
Hunk #9 succeeded at 509 (offset -1 lines).
2 out of 9 hunks FAILED -- saving rejects to file apc_sma.c.rej
 [2007-04-01 18:44 UTC] rasmus@php.net 
It's against current CVS HEAD.

cvs -d:pserver:cvsread@cvs.php.net:/repository login
password: phpfi
cvs -d:pserver:cvsread@cvs.php.net:/repository co pecl/apc

Then apply the patch in the pecl/apc directory
 [2007-04-01 19:05 UTC] steve at tequilasolutions dot com 
OK - rebuilt and installed against CVS head, lets see what happens.....
 [2007-04-01 19:13 UTC] steve at tequilasolutions dot com 
Looks good so far get the champagne/cider ready...
 [2007-04-02 04:47 UTC] steve at tequilasolutions dot com 
Been running for 7 hours now with 28000 user entries, hardly any frags - looks really good.
 [2007-04-02 04:48 UTC] gopalv82 at yahoo dot com 
Any memory usage differences ? (% of used and stuff ?)
 [2007-04-02 05:04 UTC] steve at tequilasolutions dot com 
hard to tell as there's thousands of entries but I think it used to manage about 30000 + opcode cache in 128Mb so it looks about the same
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Fri May 09 21:01:27 2025 UTC