php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #40720 Huge arrays cause large memory usage in Ubuntu, but not Windows
Submitted: 2007-03-05 01:40 UTC Modified: 2007-03-05 20:13 UTC
From: soadlink at gmail dot com Assigned:
Status: Not a bug Package: Performance problem
PHP Version: 5.2.1 OS: Ubuntu 6.10
Private report: No CVE-ID: None
View Developer Edit
 [2007-03-05 01:40 UTC] soadlink at gmail dot com 
Description:
------------
I am working with a script that loads a list into an array and then echos each word in the array on a new line. See the 'reproduce code' for the code.

I run my scripts with the CLI (not on a website), and it works great in Windows XP when loading lists as large as 375,000 words, and I can run plenty of instances of this script without lag issues (8+ command lines going at once!). But when I try to run the same script on Ubuntu 6.10 I get the following error: Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 13491373 bytes). But that can easily be fixed by setting a larger memory limit such as: ini_set("memory_limit","75M");

However I only have to increase the memory limit on Ubuntu, but not Windows. And when I do up the memory limit on Ubuntu, only 1 or 2 instances can be run before the machine starts to bog down and get VERY slow. I'd assume this is because php is trying to load the entire list into memory in Ubuntu, but I am curious as to why Windows runs it fine. Is there something I can change on Ubuntu to make it run better? Or is this a bug with the Linux version oh PHP?

The PCs are the same that I'm working with (I just removed the Windows install recently and put Ubuntu on there), and it has 1GB of ram, and an AMD64 3200+ processor. I'm just wondering if there is something I need to change to get the script to run like it did on Windows, ...without being a memory hog or having to increase the memory limit (since I didnt have to on Windows).

Reproduce code:
---------------
<?php
$loadlist = array_map("rtrim", file("c:\\list.txt"));
foreach ($loadlist as $word) {
echo ($word . "\n");
}
?>

Expected result:
----------------
I expect it to echo the words in the list (1 word per line). It works fine on Windows, but on Ubuntu 6.10 it is very very laggy with multiple instances, whereas with Windows it runs just fine. I can have dozens of instances of the script running on Windows, but not Ubuntu.

Patches

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2007-03-05 10:08 UTC] tony2001@php.net 
>However I only have to increase the memory limit on Ubuntu, but not
Windows.
Well, must likely on Windows you're using default php.ini with memory_limit = 128M

>See the 'reproduce code' for the code.
The code is quite useless, since we apparently cannot run it without the "c:\\list.txt".
 [2007-03-05 10:38 UTC] soadlink at gmail dot com 
>Well, must likely on Windows you're using default php.ini with
memory_limit = 128M

Yes. And when I increase the memory limit on Ubuntu to match this the script runs, but more than 1-2 instances of it running causes the machines memory usage to go very high and slow the performance of everything (including the script). It doesn't do this in Windows XP Pro SP2 though. I can have plenty of instances running when that OS is installed with no problems.

>The code is quite useless, since we apparently cannot run it without the "c:\\list.txt".

395000 line test file took 20 seconds to make - http://zero-labs.net/list.zip
 [2007-03-05 10:46 UTC] tony2001@php.net 
>Yes. 
Ok, so that's not a real problem then.

>And when I increase the memory limit on Ubuntu to match this the
>script runs, but more than 1-2 instances of it running causes the
>machines memory usage to go very high and slow the performance of
>everything (including the script).

If each instance takes say 100Mb to run, then it's quite expected that 10 instances running in the same time would take ~1Gb and make the machine to swap.
 [2007-03-05 10:54 UTC] soadlink at gmail dot com 
>If each instance takes say 100Mb to run, then it's quite expected that
>10 instances running in the same time would take ~1Gb and make the
>machine to swap.

Then why does Windows handle plenty of instances just fine? I am trying to get the Linux machine to handle the script instances the same way Windows did. Linux lags after 1 or 2. Windows doesn't lag even with 8-12 running at once. Is there something inside Windows itself that is handling it better than in Linux, or something in PHP I can change?
 [2007-03-05 11:02 UTC] tony2001@php.net 
>Then why does Windows handle plenty of instances just fine?
Because your windows machine has more RAM?
Because the script doesn't actually work?
Because it works differently? 
I don't think I can tell you which of these is true.
 [2007-03-05 11:08 UTC] soadlink at gmail dot com 
>Because your windows machine has more RAM?

Just as I figured, you obviously didn't read my first post entirely, and have little clue what I was talking about... which is why I was repeating myself.

>Because the script doesn't actually work?

Actually it works just fine on Windows. Ubuntu is where I have the problems. What's that the 3rd time I've said that now?

>Because it works differently? 

Really? Gee I didn't notice that. /sarcasm


Anyways forget it. Don't even bother helping.
 [2007-03-05 11:20 UTC] tony2001@php.net 
.
 [2007-03-05 11:25 UTC] soadlink at gmail dot com 
Closed, not bogus, as I voluntarily decided to end the problem.
 [2007-03-05 11:30 UTC] tony2001@php.net 
"Closed" means there was a problem and it has been fixed.
 [2007-03-05 11:37 UTC] soadlink at gmail dot com 
Then it is neither bogus or closed, because a problem does exist, so it is definitely not bogus.
 [2007-03-05 19:15 UTC] stas@php.net 
I the Linux OS 64-bit? Then difference in memory requirements are easily explained - on 64-bit pointers and longs are twice as long as on 32-bit,  and PHP uses both a lot in variables, so memory requirement for huge array can be significantly bigger on 64-bit OS.
 [2007-03-05 20:13 UTC] soadlink at gmail dot com 
Yea it's a 64 bit OS. Thanks for the explanation on the differences, I didn't know that.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Mon Sep 09 09:01:27 2024 UTC