php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #41713 Persistent memory consumption since 5.2
Submitted: 2007-06-16 16:48 UTC Modified: 2007-07-25 11:15 UTC
Votes:12
Avg. Score:4.1 ± 0.8
Reproduced:12 of 12 (100.0%)
Same Version:7 (58.3%)
Same OS:11 (91.7%)
From: mplomer at gmx dot de Assigned: dmitry (profile)
Status: Closed Package: Performance problem
PHP Version: 5.2CVS-2007-07-12 OS: win32 only
Private report: No CVE-ID: None
 [2007-06-16 16:48 UTC] mplomer at gmx dot de
Description:
------------
When using php arrays with a lot of entries (about 200000), I figured out the following problem: PHP sometimes doesn't free all used memory after completing the request. Apache uses under some circumstances 20-30 MB more RAM after the request. The problem is that this happens per child. If an Apache runs with 64 threads, it is possible, that httpd.exe consumes persistently 300-600 MB of RAM, even without any active request.


Reproduce environment:
- I tested it under an Apache (1.3 and 2.2) environment unter Windows (XP) (I didn't test it under Linux yet)
- Used PHP-Version: 5.2.3 - the problem was introduced with PHP 5.2. With PHP 5.1.6 the problem does not appear (but I noticed, that PHP 5.1.6's memory management is much slower than the new one :-)
- Set ThreadsPerChild to 1 in httpd.conf to make sure, you hit always the same PHP instance and avoid any constraints

Reproduce procedure:
- Freshly start your 1-thread-Apache [It will consume about 10 MB]
- Execute the following script [Memory usage will grow to ~50-60 MB, and after execution memory usage shrinks back to ~10 MB again ... works fine so long]
- Execute the script again 2 or more times [... and surprisingly Apache consumes about 35MB after the request is complete!] (The number of executions you need to reproduce the problem depends on the elementCount of the test-array, and eventually some system dependent factors; see reproduce code)
- If you excecute the script some more times again, someimes the memory is freed and memory usage is about 10MB again, but after some further requests, the memory is NOT freed again.

Because of the last point, I think, it is not a memory leak. I also compiled PHP as debug version and there where no memory leaks reported (with report_memleaks = On).
But I still think, the consumed memory should be completely freed after _each_ request. If this is a feature, because something is cached, it requires a maximum of the cache size. 20-30MB per child is definitely too much.
If you put an "echo memory_get_peak_usage(true);" at the beginning of the script, you will see, that PHP claims to use only about 200-300 KB at every script start time. It doesn't report the 20-30MB that it consumes since the last execution.

I tested this with an array, but the problem can, of course, be deeper in the new memory management of PHP 5.2

Reproduce code:
---------------
<?php

  // elementCount: Count of elements which are created in the test array:
  // - a count of 200000 demonstrates the problem at best on my machine
  // - a count of 100000 or lesser reproduces the problem too, but you need more
  //   calls of the script (browser refreshes) until the problem occurs
  // - with a count of 400000 or more I couldn't reproduce the problem,
  //   there it works fine
  // I couldn't figure out exactly, on which factors this count depends on, but
  // it _seems_ to be not machine dependent, but play around with it.
  // Note also, that the count of array elements is important, not the
  // size of their keys or values.
  $elementCount = 200000;

  for ($i = 0; $i < $elementCount; $i++) {
    $variables[$i] = 'x';
  }


  // Even when you unset each array-element manually, the problem occurs ...
  //for ($i = 0; $i < $elementCount; $i++) {
  //  unset($variables[$i]);
  //}

  // ... and/or if you unset the array itself ...
  //unset($variables);

  // ... but it does not occur, when you put the unset directly in the first
  // for-loop, that the variable will be unset immediately. Only a small amount
  // of memory is required for the whole request - so PHP can't forgot to free
  // many memory.

?>

Expected result:
----------------
see "Description": All memory being freed after execution of a PHP script.

Actual result:
--------------
see "Description": Not all memory is freed under some circumstances after script execution.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2007-06-17 09:04 UTC] mplomer at gmx dot de
I tested this under Linux today, and I could not reproduce it here. Apache's memory consumption is ~10MB after each request, regardless of the elementCount of the test-array. It seems to be a Windows specific problem.

I compiled PHP under Linux without and with Thread-Safety (--enable-maintainer-zts) enabled (because under Windows this is activated), but it seems this has no influence.
 [2007-06-19 06:25 UTC] mplomer at gmx dot de
By the way ... I disabled all PHP extensions, and used the standard php.ini settings.
 [2007-06-26 06:52 UTC] mplomer at gmx dot de
I can still reproduce this. I tested this with the actual "5.2.4-dev 2007-06-26 00:09" snapshot without php.ini.
Today with 200000 array elements I needed 6 or 7 script executions,
with 400000 elements I need 2-3 script executions,
and with 800000 I could not reproduce the problem.

Could you reproduce the problem? Or do you need some additional infos to reproduce it?

PS: Now httpd.exe crashes on every shutdown, but this possibly another bug ;-)
 [2007-06-26 11:10 UTC] tony2001@php.net
Is this windows only issue?
 [2007-06-26 11:37 UTC] mplomer at gmx dot de
It seems so. I could reproduce it only under windows yet. (See my comment from [17 Jun 9:04am UTC]).
 [2007-06-27 11:42 UTC] tony2001@php.net
Too bad we don't have any developers maintaining Windows port on a daily basis.
So you're encouraged to help us and investigate the issue.
Any additional information you can find would be appreciated.
 [2007-06-27 11:54 UTC] sniper@php.net
It's most likely a ZTS issue, so testing on *nix with e.g. apache worker and apache2handler SAPI might be a good idea too..
 [2007-06-27 20:56 UTC] mplomer at gmx dot de
OK, no problem. ... Today I tested this with apache worker, but I still cannot reproduce it under linux.

I used a Debian 4.0 system and I compiled Apache 2.2.4 with:
./configure --prefix=/usr/local/apache2 --with-included-apr --with-mpm=worker --enable-so

... and PHP 5.2.3 with:
./configure --prefix=/usr/local/php --with-apxs2=/usr/local/apache2/bin/apxs --enable-memory-limit

So phpinfo() said, "Thread Safety" is enabled. And I played around with the elementCount ... but under linux the memory is always completely freed.

If you have some tips to track this down inside PHP, please let me know. I now have a working PHP build-environment under windows.
 [2007-06-28 00:41 UTC] sniper@php.net
Thank you, at least we know it only happens with Windows.
Have you tried running PHP as FastCGI under windows?
That might cure this too..

 [2007-06-30 10:19 UTC] mplomer at gmx dot de
OK, it took me some time to get FastCGI running, but now it works,
and yes ... I can reproduce it there, too.

To be more detailed, I set up the following environment:
- Apache 2.0.59 (minimalistic configuration) with
  mod_fastcgi-SNAP-0404142202-AP2.dll

- Configured FastCGI with 1 process:
  FastCgiServer ../php5/php-cgi.exe -processes 1

- PHP 5.2.3 (without php.ini; only php-cgi.exe and php5ts.dll)

- and the test-script with 400,000 array elements:

<?php
  $elementCount = 400000;

  for ($i = 0; $i < $elementCount; $i++) {
    $variables[$i] = 'x';
  }
?>


Reproducing by looking at the Task-Manager's memory usage
values after each execution of the script:
- Freshly starting Apache
- Child process "php-cgi.exe" is started
                         [php-cgi.exe consumes ~ 4.1 MB]
- Execute the test-script the first time:
                         [php-cgi.exe consumes ~ 4.8 MB]
- Execute one more time: [php-cgi.exe consumes ~ 4.8 MB]
- Execute one more time: [php-cgi.exe consumes ~ 5.3 MB]
  ... might be OK so long

- Execute one more time: [php-cgi.exe consumes ~ 36.4 MB !!!]
  ... from now on this is not OK anymore
- Execute one more time: [php-cgi.exe consumes ~ 36.7 MB]
- Execute one more time: [php-cgi.exe consumes ~ 6.1 MB]

- Execute one more time: [php-cgi.exe consumes ~ 36.4 MB]
- Execute one more time: [php-cgi.exe consumes ~ 36.5 MB]
- Execute one more time: [php-cgi.exe consumes ~ 7.7 MB]

- Execute one more time: [php-cgi.exe consumes ~ 36.7 MB]
- Execute one more time: [php-cgi.exe consumes ~ 36.8 MB]
- Execute one more time: [php-cgi.exe consumes ~ 5.8 MB]

- Execute one more time: [php-cgi.exe consumes ~ 36.4 MB]
- Execute one more time: [php-cgi.exe consumes ~ 36.5 MB]
- Execute one more time: [php-cgi.exe consumes ~ 10.6 MB]

- Execute one more time: [php-cgi.exe consumes ~ 36.7 MB]
- Execute one more time: [php-cgi.exe consumes ~ 36.8 MB]
- Execute one more time: [php-cgi.exe consumes ~ 5.5 MB]

- ...

This is always reproducable, and the memory usage values
are quite the same on each reproducion. (They differ only
sometimes in 4-12 KB.)

But you see, there is a systematic. There are 2 executions,
after which the memory is not freed, and after the third
execution, the memory is mostly freed.

Any ideas for further testing?
 [2007-06-30 11:32 UTC] mplomer at gmx dot de
Another developer tested this on his own machine now, with the same Apache/PHP environment, and could affirm this behavior. The memory usage values are respectively 0,1-0,3 MB different, but the principle is the same. So the behavior seems not to be system dependent.
 [2007-07-10 23:39 UTC] spamtrap at psychoticwolf dot net
I see this with PHP 5.2.1 - 5.2.3 (mod_php5 with Apache 2.0.59 and 2.2.4 on WinXP and Win2003). I did some regression testing and it seems to have started between 5.2.0 and 5.2.1 which points at the new memory management on win32. Memload was normal under 5.2.0.  After awhile, Apache consumes as previously reported, 300-600mb (usually around 330mb + 6-700mb virtual), and, curiously, PHP thows a Fatal Error that its exceeded its memory limit for that script, even though it hasn't, as the script doesn't use more than about 300k. (Only seen this last part once, so far, so that might be a fluke.)
 [2007-07-11 08:43 UTC] mplomer at gmx dot de
I tested with PHP 5.2.0 now, and I can reproduce the described behaviour from [30 Jun 10:19am UTC] too. Only when I am testing with PHP 5.1.6, I can't reproduce it.
But I agree with you, that this points at the new memory management on win32.
 [2007-07-14 09:15 UTC] jani@php.net
Dmitry, please check this out.
 [2007-07-14 21:15 UTC] jani@php.net
And I can reproduce this too on Windows, using latest snapshot available.
 [2007-07-25 11:15 UTC] dmitry@php.net
This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.


 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Tue Oct 15 10:01:27 2024 UTC