php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #48781 Cyclical garbage collector memory leak
Submitted: 2009-07-02 18:35 UTC Modified: 2011-11-02 07:55 UTC
Votes:9
Avg. Score:4.9 ± 0.3
Reproduced:8 of 8 (100.0%)
Same Version:1 (12.5%)
Same OS:1 (12.5%)
From: nate at frickenate dot com Assigned: dmitry
Status: Closed Package: Scripting Engine problem
PHP Version: 5.3.0 OS: Debian 5.0 kernel 2.6.24-23-xen
Private report: No CVE-ID:
 [2009-07-02 18:35 UTC] nate at frickenate dot com
Description:
------------
The new cyclical garbage collector isn't collecting everything it should be - somewhere there is a memory leak. The issue I am bringing up is not about delayed collection - this is a true memory leak with memory that is never reclaimed.

When a variable (in this example, $user) contains an object with a cyclical reference, there is a small amount of memory that is leaked/not reclaimed when you set $user to another object (of any class). If you set $user to an int/string/boolean/null before setting $user to another object, all memory is collected properly.

I came to the conclusion that this must be a problem with the garbage collector, since if you replace the "$this->user = $user;" with "$this->user = new StdClass;" (thus preventing the creation of a cyclical reference), the leak goes away.

So basically right now:

<?php

while ($user = get_next_user()) {
    // do stuff with $user

    // php bug: have to unset before next iteration
    // to prevent memory leak caused by buggy gc
    unset($user);
}
?>

Reproduce code:
---------------
<?php

/*
How to use:
    a) run test 1 by running code as-is
    b) run test 2 by commenting out test 1, uncomment test 2
    c) run test 3 by commenting out test 1 & 2, uncomment test 3
*/

class User {
    protected
        $profile;

    public function __construct () {
        $this->profile = new UserProfile($this);
    }
}

class UserProfile {
    private
        $user;

    public function __construct ($user) {
        $this->user = $user;
    }
}

for ($userid = 1; ; $userid++) {
    $user = new User;

    if ($userid % 100000 == 0)
        printf("memory usage after %s users: %s MB.\n", number_format($userid), number_format(memory_get_usage() / 1024 / 1024, 2));

    // TEST 1 - do nothing before we set the
    // new User on next iteration of the loop).
    // RESULT: leaks memory (see "Actual result" section)
    continue;

    // TEST 2 - set $user to an empty object, before
    // we set the new User on next iteration
    // RESULT: leaks memory (see "Actual result" section)
    //$user = new StdClass; continue;

    // TEST 3 - set $user to anything other than an object,
    // before we set the new User on next iteration.
    // RESULT: does NOT leak memory (see "Expected result" section)
    //$user = 'not an object'; continue;
}

?>


Expected result:
----------------
memory usage after 100,000 users: 1.54 MB.
memory usage after 200,000 users: 1.54 MB.
memory usage after 300,000 users: 1.54 MB.
memory usage after 400,000 users: 1.54 MB.
memory usage after 500,000 users: 1.54 MB.
memory usage after 600,000 users: 1.54 MB.
memory usage after 700,000 users: 1.54 MB.
memory usage after 800,000 users: 1.54 MB.
memory usage after 900,000 users: 1.54 MB.
memory usage after 1,000,000 users: 1.54 MB.

[snip]

memory usage after 99,000,000 users: 1.54 MB.
memory usage after 99,100,000 users: 1.54 MB.
memory usage after 99,200,000 users: 1.54 MB.
memory usage after 99,300,000 users: 1.54 MB.
memory usage after 99,400,000 users: 1.54 MB.
memory usage after 99,500,000 users: 1.54 MB.
memory usage after 99,600,000 users: 1.54 MB.
memory usage after 99,700,000 users: 1.54 MB.
memory usage after 99,800,000 users: 1.54 MB.
memory usage after 99,900,000 users: 1.54 MB.
memory usage after 100,000,000 users: 1.54 MB.

Actual result:
--------------
memory usage after 100,000 users: 1.55 MB.
memory usage after 200,000 users: 1.57 MB.
memory usage after 300,000 users: 1.58 MB.
memory usage after 400,000 users: 1.59 MB.
memory usage after 500,000 users: 1.61 MB.
memory usage after 600,000 users: 1.62 MB.
memory usage after 700,000 users: 1.64 MB.
memory usage after 800,000 users: 1.65 MB.
memory usage after 900,000 users: 1.66 MB.
memory usage after 1,000,000 users: 1.68 MB.

[snip]

memory usage after 99,000,000 users: 18.43 MB.
memory usage after 99,100,000 users: 18.44 MB.
memory usage after 99,200,000 users: 18.46 MB.
memory usage after 99,300,000 users: 18.47 MB.
memory usage after 99,400,000 users: 18.48 MB.
memory usage after 99,500,000 users: 18.50 MB.
memory usage after 99,600,000 users: 18.51 MB.
memory usage after 99,700,000 users: 18.53 MB.
memory usage after 99,800,000 users: 18.54 MB.
memory usage after 99,900,000 users: 18.55 MB.
memory usage after 100,000,000 users: 18.57 MB.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2009-12-07 09:12 UTC] jani@php.net
Please try using this snapshot:

  http://snaps.php.net/php5.3-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/


 [2009-12-08 15:23 UTC] nate at frickenate dot com
No change, same problem. Considering this bug hasn't even been looked at or assigned, no surprise there.
 [2009-12-15 01:00 UTC] php-bugs at lists dot php dot net
No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
 [2010-01-05 19:31 UTC] nightstorm at tlen dot pl
I confirm, there is a true memory leak. Consider the following script, where the references are unmounted manually with an extra dispose() method:

~~~~
<?php

/*
How to use:
    a) run test 1 by running code as-is
    b) run test 2 by commenting out test 1, uncomment test 2
    c) run test 3 by commenting out test 1 & 2, uncomment test 3
*/

class User {
    protected
        $profile;

    public function __construct () {
        $this->profile = new UserProfile($this);
    }
    
    public function dispose()
    {
		$this->profile->dispose();
		$this->profile = null;
    } // end dispose();
}

class UserProfile {
    private
        $user;

    public function __construct ($user) {
        $this->user = $user;
    }
    
    public function dispose()
    {
		$this->user = null;
    } // end dispose();
}

for ($userid = 1; ; $userid++) {
	if(isset($user))
	{
		$user->dispose();
	}
    $user = new User;

    if ($userid % 100000 == 0)
        printf("memory usage after %s users: %s MB.\n",
number_format($userid), number_format(memory_get_usage() / 1024 / 1024,
2));

    continue;
}
~~~~~

In this case the result is still the same (PHP 5.3.1):

~~~~
memory usage after 100,000 users: 0.61 MB.
memory usage after 200,000 users: 0.61 MB.
memory usage after 300,000 users: 0.61 MB.
memory usage after 400,000 users: 0.61 MB.
memory usage after 500,000 users: 0.61 MB.
memory usage after 600,000 users: 0.61 MB.
memory usage after 700,000 users: 0.61 MB.
memory usage after 800,000 users: 0.61 MB.
memory usage after 900,000 users: 0.61 MB.
memory usage after 1,000,000 users: 0.61 MB.
memory usage after 1,100,000 users: 0.61 MB.
memory usage after 1,200,000 users: 0.61 MB.
~~~~

If we enable the garbage collector and remove the dispose() method, the used memory level begins to increase, and if we call gc_collect_cycles() after creating a new object, the used memory increases even much faster. On my PC, it is not able to perform display even a single control message after 100000 iterations. It stopped after approx. 33000 users:

~~~~
memory usage after 30,000 users: 25.37 MB.
memory usage after 31,000 users: 26.07 MB.
memory usage after 32,000 users: 26.76 MB.
memory usage after 33,000 users: 31.46 MB.

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 89 bytes) in /home/me/test/memleak.php on line 17
~~~~
 [2010-03-07 18:17 UTC] felipe@php.net
-Status: No Feedback +Status: Assigned
 [2010-04-18 02:28 UTC] joey@php.net
I am unable to duplicate this behaviour using 5.3.2. The code given below consistently uses 0.65 MB even up to 10 million iterations - I pasted the output of up to 6 million, with the extended GC reporting turned on:

<?php

class User {
    protected $profile;

    public function __construct () {
        $this->profile = new UserProfile($this);
    }
    
    public function dispose() {
		$this->profile->dispose();
		$this->profile = null;
    }
}

class UserProfile {
    private $user;

    public function __construct ($user) {
        $this->user = $user;
    }
    
    public function dispose() {
		$this->user = null;
    }
}

for ($userid = 1; ; $userid++) {
	if(isset($user)) {
		$user->dispose();
	}
    $user = new User;

    if ($userid % 100000 == 0)
        printf("memory usage after %s users: %s MB.\n", number_format($userid), number_format(memory_get_usage() / 1024 / 1024, 2));

    continue;
}

?>


memory usage after 100,000 users: 0.65 MB.
memory usage after 200,000 users: 0.65 MB.
memory usage after 300,000 users: 0.65 MB.

[snip]

memory usage after 5,700,000 users: 0.65 MB.
memory usage after 5,800,000 users: 0.65 MB.
memory usage after 5,900,000 users: 0.65 MB.

GC Statistics
-------------
Runs:               0
Collected:          0
Root buffer length: 0
Root buffer peak:   9

      Possible            Remove from  Marked
        Root    Buffered     buffer     grey
      --------  --------  -----------  ------
ZVAL        16         7          7         0
ZOBJ  77999984  11999998   11999998         0
 [2010-04-19 09:54 UTC] dmitry@php.net
Yeah. I confirm the problem. It looks like on each 100,000 iterations PHP runs GC about 12 times. And on each GC run it looses 2 objects referenced by $user. I'll try to fix it.
 [2010-04-19 12:27 UTC] rquadling@php.net
The gc cycle seems to miss the last 2 items on this cycle, but picks up from the 
last one for the next cycle, so is only actually dropping 1 item per cycle.

My cycle goes something like this ...

Create and assign #1 to #4999
Create #5000
Destroy #1 to #4998
Assign #5000
Create and assign #5001 to #9999
Create #10000
Destroy #5000 to #9998 --- Skipped #4999
Assign #10000
Create and assign #10001 to #14999
Create #15000
Destroy #10000 to #14998 --- Skipped #9999
etc.

My code changes ...

class User {
    protected
        $profile, $usercount;

    public function __construct ($usercount) {
    echo str_pad($this->usercount = $usercount, 5, '0', STR_PAD_LEFT), ':#1', 
PHP_EOL;
        $this->profile = new UserProfile($this);
    }

   public function __destruct() {
    echo str_pad($this->usercount, 5, '0', STR_PAD_LEFT), ':#3', PHP_EOL;
   }
}

class UserProfile {
    private
        $user;

    public function __construct ($user) {
        $this->user = $user;
    }
}

for ($userid = 1; ; $userid++) {
    $user = new User($userid);

    echo str_pad($userid, 5, '0', STR_PAD_LEFT), ':#2', PHP_EOL;

// Same code as provided by Nate.
 [2010-04-19 12:31 UTC] rquadling@php.net
If the loop is limited to 11111 items, using my code to report the 
construct/use/destruct, the log ends with ...


11108:#3
11109:#3
11110:#3
11111:#3
04999:#3
09999:#3
10000:#3


So, it is completely losing the items, just not cleaning them during the cycle. 
At shutdown, they are cleaned.
 [2010-04-19 12:37 UTC] rquadling@php.net
That should have said ...

"So, it is _NOT_ completely losing the items, just not cleaning them during the 
cycle. At shutdown, they are cleaned."
 [2010-04-20 14:30 UTC] dmitry@php.net
Automatic comment from SVN on behalf of dmitry
Revision: http://svn.php.net/viewvc/?view=revision&amp;revision=298213
Log: Fixed bug #48781 (Cyclical garbage collector memory leak)
 [2010-04-20 14:50 UTC] dmitry@php.net
-Status: Assigned +Status: Closed
 [2010-04-20 14:50 UTC] dmitry@php.net
This bug has been fixed in SVN.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.


 [2010-04-20 14:55 UTC] dmitry@php.net
Note that your test and expectation are not completely correct because the actual garbage collection occurs at undefined moments of time. As result the memory usage on different iteration will be different. To avoid this ambiguity add explicit call to gc_collect_cycles() before printing memory usage.
 [2011-04-03 02:24 UTC] jas at rephunter dot net
I believe this bug has regressed. We only recently upgraded from 5.3.2 to 5.3.5. 
And now several scripts are broken that have been working for years. By adding the 
unset of the variable at the end of the loop as shown in the first snippet below, 
the problem is fixed.

So I believe that the symptom has regressed, although not necessarily the cause.
 [2011-09-13 11:28 UTC] nex2hex at ya dot ru
On php 5.38 (win32) this bug still exists
 [2011-10-31 20:16 UTC] nate at frickenate dot com
-Status: Closed +Status: Assigned
 [2011-10-31 20:16 UTC] nate at frickenate dot com
Confirmed, this bug has regressed. Since it was originally fixed, there are few 
commits to zend_execute.c where that fix was effectuated.

http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3/Zend/zend_execute.c?
r1=293985&r2=309342

This shows all changes to that file since the fix (inclusive). The added line at 
273 was the fix for this bug. But then we see the other changes made around this 
fix, one of which must have caused the cyclical fix to be no longer useful.
 [2011-11-02 07:55 UTC] dmitry@php.net
-Status: Assigned +Status: Closed
 [2011-11-02 07:55 UTC] dmitry@php.net
As I already told before, the test case is not excellent.
It measures amount of memory with one period, while the GC occurs with another one. Anyway the results donsn't grow forever.

$ sapi/cli/php bug48781.php

memory usage after 100,000 users: 0.82 MB.
memory usage after 200,000 users: 0.83 MB.
memory usage after 300,000 users: 0.85 MB.
memory usage after 400,000 users: 0.86 MB.
memory usage after 500,000 users: 0.88 MB.
memory usage after 600,000 users: 0.89 MB.
...
memory usage after 24,700,000 users: 4.40 MB.
memory usage after 24,800,000 users: 4.41 MB.
memory usage after 24,900,000 users: 4.43 MB.
memory usage after 25,000,000 users: 0.81 MB.
memory usage after 25,100,000 users: 0.82 MB.
memory usage after 25,200,000 users: 0.83 MB.
...
memory usage after 49,700,000 users: 4.40 MB.
memory usage after 49,800,000 users: 4.42 MB.
memory usage after 49,900,000 users: 4.43 MB.
memory usage after 50,000,000 users: 0.81 MB.
memory usage after 50,100,000 users: 0.82 MB.
memory usage after 50,200,000 users: 0.84 MB.

It's better to insert gc_collect_cycles() before measuring the memory usage. In this case the results would be same.

So the bug is fixed. May be you suffer from another one but I can do nothing without a test case.
 
PHP Copyright © 2001-2014 The PHP Group
All rights reserved.
Last updated: Wed Apr 16 16:02:23 2014 UTC