php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #51155 Unreasonable non-emalloc allocations of memory
Submitted: 2010-02-26 13:35 UTC Modified: 2010-10-04 18:09 UTC
Votes:2
Avg. Score:5.0 ± 0.0
Reproduced:2 of 2 (100.0%)
Same Version:1 (50.0%)
Same OS:1 (50.0%)
From: flavius dot as at gmail dot com Assigned: dmitry (profile)
Status: Closed Package: Scripting Engine problem
PHP Version: 5.3.2 OS: ArchLinux x86_64
Private report: No CVE-ID: None
 [2010-02-26 13:35 UTC] flavius dot as at gmail dot com
Description:
------------
When serializing a SplFixedArray with serialize(), the script dies with "Fatal error: Allowed memory size of 134217728 bytes exhausted"

The "expected result" works and allocates at most 20.96 mb for $cnt = 8565 on line 15.

The "actual result" crashes when serialize()'ing with $cnt only incremented by one, which is not understandable.

The actual values may vary, but if you play enough with it you'll find at which amount of items serialize() has that spark.

Then you can toggle to using plain arrays on line 17, and that problem will disappear, although arrays actually consume more memory (in my experiments around 1.2 mb more).

Reproduce code:
---------------
     1  <?php
     2  function rand_str($length = 32, $chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz1234567890') {
     3      $chars_length = (strlen($chars) - 1);
     4      $string = $chars{rand(0, $chars_length)};
     5      for ($i = 1; $i < $length; $i = strlen($string)) {
     6          $r = $chars{rand(0, $chars_length)};
     7          if ($r != $string{$i - 1}) $string .=  $r;
     8      }
     9      return $string;
    10  }
    11  function convert($size) {
    12      $units=array('b','kb','mb','gb','tb','pb');
    13      return @round($size/pow(1024,($unit=floor(log($size,1024)))),2).' '.$units[$unit];
    14  }
    15  $cnt = 8566;
    16  $r = new SplFixedArray($cnt);
    17  //$r = array();
    18  for($i=0;$i<$cnt;$i++) {
    19          $r[$i] = rand_str(1024);
    20  }
    21  echo "\nGENERATING DONE\n";
    22  echo 'peak before serialize() ',convert(memory_get_peak_usage()),PHP_EOL;
    23  echo "---\n";
    24  $r = serialize($r);
    25  echo 'peak after dumping ',convert(memory_get_peak_usage()),PHP_EOL;
    26  unset($r);


Expected result:
----------------
GENERATING DONE
peak before serialize() 10.19 mb
---
peak after dumping 20.96 mb


Actual result:
--------------
GENERATING DONE
peak before serialize() 10.19 mb
---

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8913127 bytes) in /home/flav/softpedia/bigarray-bugreport.php on line 24


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2010-02-26 13:38 UTC] flavius dot as at gmail dot com
Updated OS: ArchLinux x86_64
 [2010-02-26 13:41 UTC] flavius dot as at gmail dot com
Oh and I've forgot to mention, there's plenty of RAM and swap space before running php -f:

free -m
             total       used       free     shared    buffers     cached
Mem:          1975       1732        243          0        131       1027
-/+ buffers/cache:        573       1402
Swap:         5718          0       5718
 [2010-03-01 17:02 UTC] jani@php.net
Please try using this snapshot:

  http://snaps.php.net/php5.3-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/


 [2010-03-04 14:26 UTC] flavius dot as at gmail dot com
Using it gives the configure error:
configure: error: utf8_mime2text() has new signature, but U8T_CANONICAL is missing. This should not happen. Check config.log for additional information.
 [2010-03-04 14:30 UTC] pajoye@php.net
That's unrelated to this bug, disable imap to test a new build.

However, about this error, get a decent c-client and it will work (like less than 4-5 years old).
 [2010-03-04 15:10 UTC] flavius dot as at gmail dot com
Ok, I've managed to compile and test with php5.3-201003041130, the bug is still there, but it seems to crash for higher values.

for 20000 items, the peak is
peak 21.71 mb before serialize,
peak 45.59 mb after serialize. simple maths: it should need approximatively 5 mb for every 5000 items.

The following test run for 25000 confirms my theory (roughly, +/- due to temporarily saving the return value, runtime memory, etc): 
before serialization: peak 26.98 mb
after: peak 56.7 mb

Ok, it's slightly higher than 5mb, after serialization is a small increase over the expected (which I presumed) 2*n linear growth.

Now the final run. But first and foremost
$ sapi/cli/php -i |grep memory
memory_limit => 128M => 128M

And the test run, for 30000 items:
GENERATING DONE
peak 32.24 mb
---

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 28574231 bytes)


The expected value was somewhere around (and I'll be generous) 80mb. But I suppose it still has that spark of exponential growth somewhere between 25000 and 30000 items.
 [2010-03-27 14:44 UTC] flavius dot as at gmail dot com
-Status: Feedback +Status: Open -PHP Version: 5.3.1 +PHP Version: 5.3.2
 [2010-03-27 14:45 UTC] flavius dot as at gmail dot com
updated version
 [2010-04-25 06:55 UTC] colder@php.net
-Summary: serialize() crashes with unreasonable/unexplicable "out of memory" for objects +Summary: Unreasonable non-emalloc allocations of memory -Package: SPL related +Package: Scripting Engine problem
 [2010-04-25 06:55 UTC] colder@php.net
This is not related to SPL nor to serialize. 

In fact, it's simply that your functions make PHP allocate a lot of memory on the 
side (not using emalloc). Try with memory_get_usage(true); you'll see that you hit 
already ~128M before even serializing, 

This is a bit smaller using a normal array, so it's not really related to 
SplFixedArray.
 [2010-04-29 13:48 UTC] colder@php.net
-Status: Open +Status: Assigned -Assigned To: +Assigned To: dmitry
 [2010-10-04 17:50 UTC] dmitry@php.net
Automatic comment from SVN on behalf of dmitry
Revision: http://svn.php.net/viewvc/?view=revision&amp;revision=304031
Log: Bug #51155 (Unreasonable non-emalloc allocations of memory)
 [2010-10-04 18:09 UTC] dmitry@php.net
-Status: Assigned +Status: Closed
 [2010-10-04 18:09 UTC] dmitry@php.net
This bug has been fixed in SVN.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.


 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Fri Apr 19 17:01:30 2024 UTC