php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #26577 [str_replace] Variable tracing 'optimization' causes 1.8GB memory usage
Submitted: 2003-12-10 06:30 UTC Modified: 2010-12-31 21:04 UTC
Votes:4
Avg. Score:5.0 ± 0.0
Reproduced:4 of 4 (100.0%)
Same Version:4 (100.0%)
Same OS:4 (100.0%)
From: vijolicni dot oblak at gmx dot net Assigned: jani (profile)
Status: Closed Package: *General Issues
PHP Version: 4.3.4 OS: Windows 2003
Private report: No CVE-ID: None
 [2003-12-10 06:30 UTC] vijolicni dot oblak at gmx dot net
Description:
------------
There is obviously some kind of tracing of what happens to variables through their lifetime ? a kind of optimization, I guess. But there is a problem when doing str_replace on large html files (more than 600kb): memory usage jumps from 350MB to 1.8GB according to Windows Task Manager performance monitor. My computer has 512MB physical memory and having more that one gigabyte of quickly changing data in swap file doesn?t help performance at all. It renders my system to almost unresponsive.

I solved the problem by doing some ?disoptimizations?: I put the contents of variable $htmlfile, that str_replace was done on to a new variable $tempvar, then I unset($htmlfile), and then reestablish $htmlfile as a new variable with $tempvar as it contents and continue execution where ended [?case? is used as a ?goto? statement]. The set-unset code is commented.

The solution to the problem should be that Zend Engine or whatever module is responsible for string tracing and/or memory management should check how much !physical! memory is available and never waste more that hundred(s) of available !physical! MBs on a script, but it should never run into swap file memory gigabytes for ?optimizations?.
[I have my swap file on Windows2003 set to 4GB if this is of any relevance]


Reproduce code:
---------------
  $htmlfile=str_replace("\t",' ',$htmlfile);
  //if (sovertime(8)) savestate($destfilename,$htmlfile,10); return outr(10); case 10:
  while ($pos=strpos($htmlfile,'  ')) {
    $htmlfile=str_replace('  ',' ',$htmlfile);
    if ($pos===false) break;}
  //if (sovertime(8)) savestate($destfilename,$htmlfile,20); return outr(20); case 20:    
  while ($pos=strpos($htmlfile,"\r\n\r\n")) {
    $htmlfile=str_replace("\r\n\r\n","\r\n",$htmlfile);
    if ($pos===false) break;}
  //if (sovertime(8)) savestate($destfilename,$htmlfile,30); return outr(30); case 30:
  while ($pos=strpos($htmlfile,"\r\n ")) {
    $htmlfile=str_replace("\r\n ","\r\n",$htmlfile);
    if ($pos===false) break;}
  //if (sovertime(8)) savestate($destfilename,$htmlfile,40); return outr(40); case 40:
  while ($pos=strpos($htmlfile," \r\n")) {
    $htmlfile=str_replace(" \r\n","\r\n",$htmlfile);
    if ($pos===false) break;}
            
  $htmlfile=str_replace('<table ','<table width=100% ',$htmlfile);
  $htmlfile=str_replace('<TABLE ','<TABLE width=100% ',$htmlfile);
  $htmlfile=str_replace('<table>','<table width=100%>',$htmlfile);
  $htmlfile=str_replace('<TABLE>','<TABLE width=100%>',$htmlfile);
  //if (sovertime(8)) savestate($destfilename,$htmlfile,105);return outr(105);case 105:
  $htmlfile=str_replace('&Scaron;','?',$htmlfile);
  $htmlfile=str_replace('&scaron;','?',$htmlfile);
  $htmlfile=str_replace('&Ccaron;','?',$htmlfile);
  $htmlfile=str_replace('&ccaron;','?',$htmlfile);
  $htmlfile=str_replace('&Zcaron;','?',$htmlfile);
  $htmlfile=str_replace('&zcaron;','?',$htmlfile);    
  //if (sovertime(8)) savestate($destfilename,$htmlfile,110);return outr(110);case 110:
  $htmlfile=str_replace('<table>','<table width=100%>',$htmlfile);
  $htmlfile=str_replace('<TABLE>','<TABLE width=100%>',$htmlfile);
  $htmlfile=str_replace('background:','bx:',$htmlfile);
  $htmlfile=str_replace('mso-highlight:','bx:',$htmlfile);
  $htmlfile=str_replace('text-align:justify','text-align:left',$htmlfile);

Expected result:
----------------
Usage of only of available physical RAM as a tempspace for optimizations.

Actual result:
--------------
Usage of only of more than 1GB of swapfile as a tempspace for 'optimization' of a script.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2003-12-10 06:54 UTC] alex at pc4p dot net
As a personal sidenote, I find your code highly inefficient and thus it is no wonder, that you reach the limits on your system. You should definetly read up on PREGs and http://www.php.net/preg_replace which is much faster than str_replace. str_replace should only be used for small strings and small replacements, larger tasks are the domain of regular expressions. Btw. I doubt, that this behaviour is being seen as a bug...
 [2003-12-10 06:57 UTC] vijolicni dot oblak at gmx dot net
- On the server the script is going to run the script execution just ended with
FATAL: emalloc() : Unable to allocate 418kB bytes

- On my local computer the swapfile utilisation was so high that the script wan't able to save a 500kB file to a file and exit in 8 seconds (the saving function started at 22seconds since the start and ran into 30s limit) [sovertime= start_overtime, and parameter 8 means that it should save result and end execution at 22seconds, not to reach the 30s limit]
 [2003-12-10 07:03 UTC] vijolicni dot oblak at gmx dot net
To alex.
Regardless how inefficient the code is, the PHP engine should never be allowed to almost handicap a rented server that should serve requests for 30+ sites.
 [2003-12-10 09:16 UTC] sniper@php.net
Sorry, but your problem does not imply a bug in PHP itself.  For a
list of more appropriate places to ask for help using PHP, please
visit http://www.php.net/support.php as this bug system is not the
appropriate forum for asking support questions. 

Thank you for your interest in PHP.

Your argument that PHP "should not do this or that" is flawed. 
This works just as expected. There are dozens of other ways too how you can "shoot yourself in the leg"..

 [2003-12-10 09:17 UTC] sniper@php.net
See also: php.ini, memory_limit directive..

 [2003-12-10 09:52 UTC] vijolicni dot oblak at gmx dot net
Sniper!

My php.ini has in it the following lines:
;;;;;;;;;;;;;;;;;;;
; Resource Limits ;
;;;;;;;;;;;;;;;;;;;

max_execution_time = 30     ; Maximum execution time of each script, in seconds
max_input_time = 60	; Maximum amount of time each script may spend parsing request data
memory_limit = 8M      ; Maximum amount of memory a script may consume (8MB)



>Sorry, but your problem does not imply a bug in PHP itself. 
??? When I run the PHP script memory usage jumps to 1.8GB from around 400MB. Is it the Holy Ghost?
 
>Thank you for your interest in PHP.
??? What does this mean? Should I switch to ASP.NET?

>Your argument that PHP "should not do this or that" is flawed. This works just as expected. There are dozens of other ways too how you
can "shoot yourself in the leg"..
??? What works just as expected? Try it on larger file. Should PHP render the computer useless by consuming more that 1GB? You don't even have to be a sniper to shoot it.

BTW:The problem disappers with unsetting and setting back the variable...
 [2003-12-10 10:08 UTC] vijolicni dot oblak at gmx dot net
W3WP.EXE (World Wide Web Worker Process) on Win2003 IIS6.0, while executing the script near max shows the following usage:
Mem Usage:331.632K
Peak Mem Usage:378.864K
VM Size:794.532

Even if the bug is not in the PHP 'itself', there is a bug in improper interfacing between PHP and W3WP.EXE that causes W3WP.EXE to consume so much memory 
:)
 [2003-12-10 10:35 UTC] sniper@php.net
Apparently the win32 builds don't have 'MEMORY_LIMIT' defined.

 [2010-12-31 21:04 UTC] jani@php.net
-Status: Open +Status: Closed -Package: Feature/Change Request +Package: *General Issues -Assigned To: +Assigned To: jani
 [2010-12-31 21:04 UTC] jani@php.net
Should be fixed by now in PHP 5.x
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Wed May 01 07:01:30 2024 UTC