php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #19971 file() function extremally slow
Submitted: 2002-10-18 02:48 UTC Modified: 2002-10-18 15:40 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:1 (100.0%)
From: sazonenkov at yandex dot ru Assigned:
Status: Closed Package: *Directory/Filesystem functions
PHP Version: 4.3.0-dev OS: any
Private report: No CVE-ID:
 [2002-10-18 02:48 UTC] sazonenkov at yandex dot ru
This function is extrimally slow read a file.
On php4.0.4pl1 it much faster (about 100-1000x)!!!
:(

<?php
require("Benchmark/Timer.php");
$time = new Benchmark_Timer;
$time -> setMarker('Start');
$fp = fopen("Photoshop.exe", "r");
//$s  = explode("\n", fread($fp, filesize("Photoshop.exe") ) ) ;
fclose($fp);
//print_r($s);
$time -> setMarker('fread'); 
$s = file("Photoshop.exe");
$time -> setMarker('file'); 
$time -> setMarker('Stop');
$time -> display(); 
?>

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2002-10-18 03:04 UTC] mfischer@php.net
Thank you for taking the time to report a problem with PHP.
Unfortunately you are not using a current version of PHP -- 
the problem might already be fixed. Please download a new
PHP version from http://www.php.net/downloads.php

If you are able to reproduce the bug with one of the latest
versions of PHP, please change the PHP version on this bug report
to the version you tested and change the status back to "Open".
Again, thank you for your continued support of PHP.


 [2002-10-18 03:05 UTC] mfischer@php.net
Please try using this CVS snapshot:

  http://snaps.php.net/php4-latest.tar.gz
 
For Windows:
 
  http://snaps.php.net/win32/php4-win32-latest.zip

Ops, wrong quick fix.
 [2002-10-18 03:53 UTC] wez@php.net
file() is not binary safe; you should not use it on binary files.
When testing the snapshot, try the new file_get_contents() function too, which is 100% binary safe.
$s = file_get_contents();
 [2002-10-18 04:11 UTC] sazonenkov at yandex dot ru
It slow even with text files
Sorry for Photoshop.exe-example
I test it even with apache/logs/access.log
 [2002-10-18 04:31 UTC] sazonenkov at yandex dot ru
<?php
require("Benchmark/Timer.php");
$time = new Benchmark_Timer;
$time -> setMarker('Start');
$fp = fopen("access.log", "r");
$s  = explode("\n", fread($fp, filesize("access.log") ) ) ;
fclose($fp);
$time -> setMarker('fread'); 
$s = file("access.log");
$time -> setMarker('file'); 
$time -> setMarker('Stop');
$time -> display(); 
?>

access.log 16Mb
fread + explode faster when file(). Hmmm...

 time indexex time% Start1034933111.79707300-0.00% fread1034933112.398477000.60140417.87% file1034933115.162048002.76357182.12% Stop1034933115.162183000.0001350.00% total-3.365110100.00%
 [2002-10-18 06:13 UTC] yohgaki@php.net
Please paste readable output... 

================================
[yohgaki@dev DEV]$ cat t.php
<?php
$filename = '/var/log/httpd/error_log.1';

require("Benchmark/Timer.php");
$time = new Benchmark_Timer;

$time -> setMarker('Start');

$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
  $s = fread($fp, filesize($filename));
fclose($fp);

$time -> setMarker('fread'); 

for ($i = 0; $i<100; $i++)
  $s = file($filename);
$time -> setMarker('file'); 

$time -> setMarker('Stop');
$time -> display(); 
?>


[yohgaki@dev DEV]$ ./sapi/cli/php  t.php 
-------------------------------------------------------------
marker    time index            ex time               perct
-------------------------------------------------------------
Start     1034939438.53591700   -                       0.00%
-------------------------------------------------------------
fread     1034939438.55064400   0.014726996421814       2.74%
-------------------------------------------------------------
file      1034939439.07360900   0.52296495437622       97.21%
-------------------------------------------------------------
Stop      1034939439.07387800   0.00026905536651611     0.05%
-------------------------------------------------------------
total     -                     0.53796100616455      100.00%
-------------------------------------------------------------


[yohgaki@dev DEV]$ 

 [2002-10-18 06:16 UTC] wez@php.net
Please also compare with fgets().
 [2002-10-18 06:25 UTC] yohgaki@php.net
file_get_contents() is alow slow. It's better than file(), though.

[yohgaki@dev DEV]$ cat t.php 
<?php
$filename = '/var/log/httpd/error_log.1';

require("Benchmark/Timer.php");
$time = new Benchmark_Timer;

$time -> setMarker('Start');

$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
  $s = fread($fp, filesize($filename));
fclose($fp);

$time -> setMarker('fread'); 

for ($i = 0; $i<100; $i++)
  $s = file($filename);
$time -> setMarker('file'); 

for ($i = 0; $i<100; $i++)
  $s = file_get_contents($filename);
$time -> setMarker('file_get_contents'); 


$time -> setMarker('Stop');
$time -> display(); 
?>


[yohgaki@dev DEV]$ ./sapi/cli/php  t.php 
-------------------------------------------------------------------------
marker                time index            ex time               perct
-------------------------------------------------------------------------
Start                 1034940250.73024800   -                       0.00%
-------------------------------------------------------------------------
fread                 1034940250.74181700   0.011569023132324       1.86%
-------------------------------------------------------------------------
file                  1034940251.26557900   0.52376198768616       84.25%
-------------------------------------------------------------------------
file_get_contents     1034940251.35169600   0.086117029190063      13.85%
-------------------------------------------------------------------------
Stop                  1034940251.35192900   0.00023293495178223     0.04%
-------------------------------------------------------------------------
total                 -                     0.62168097496033      100.00%
-------------------------------------------------------------------------



 [2002-10-18 06:50 UTC] sazonenkov at yandex dot ru
Damn Mozilla wont work with buffer normally :(
Lets try again.
Problem: read from a text file into array
1. using fread()+explode()
2. using file()

Here my tests

<?php
require("Benchmark/Timer.php");
$time = new Benchmark_Timer;
$time -> setMarker('Start');
$fp = fopen("access.log", "r");
$s  = explode("\n", fread($fp, filesize("access.log") ) ) ;
fclose($fp);
$time -> setMarker('fread+explode'); 
$s = file("access.log");
$time -> setMarker('file'); 
$time -> setMarker('Stop');
$time -> display(); 
?>

And results

--------------------------------------------------- 
              time index          ex time  %
Start         1034941339.44905500 - 0.00%
fread+explode 1034941340.05736200 0.608307 34.72%
file          1034941341.20114900 1.143787 65.28%
Stop          1034941341.20128500 0.000136 0.01%
total         -                   1.752230 100.00%
---------------------------------------------------

Is it right?
I dont think so.
 [2002-10-18 07:04 UTC] yohgaki@php.net
fgets() is fast.

yohgaki@dev DEV]$ cat t.php
<?php
$filename = '/var/log/httpd/error_log.1';

require("Benchmark/Timer.php");
$time = new Benchmark_Timer;

$time -> setMarker('Start');

$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
  $s = explode("\n", fread($fp, filesize($filename)));
fclose($fp);

$time -> setMarker('fread'); 

for ($i = 0; $i<100; $i++)
  $s = file($filename);
$time -> setMarker('file'); 

for ($i = 0; $i<100; $i++)
  $s = file_get_contents($filename);
$time -> setMarker('file_get_contents'); 

$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
  $s = fgets($fp,filesize($filename));
$time -> setMarker('fgets'); 
fclose($fp);

[yohgaki@dev DEV]$ ./sapi/cli/php  t.php 
-------------------------------------------------------------------------
marker                time index            ex time               perct
-------------------------------------------------------------------------
Start                 1034942600.41311100   -                       0.00%
-------------------------------------------------------------------------
fread                 1034942600.43045000   0.017338991165161       2.68%
-------------------------------------------------------------------------
file                  1034942600.95939400   0.52894401550293       81.66%
-------------------------------------------------------------------------
file_get_contents     1034942601.04506900   0.085675001144409      13.23%
-------------------------------------------------------------------------
fgets                 1034942601.06049200   0.015423059463501       2.38%
-------------------------------------------------------------------------
Stop                  1034942601.06084500   0.00035297870635986     0.05%
-------------------------------------------------------------------------
total                 -                     0.64773404598236      100.00%
-------------------------------------------------------------------------


[yohgaki@dev DEV]$ 

 [2002-10-18 09:59 UTC] iliaa@php.net
I got the following times from a tests I've run on PHP.
The test involved openning a 2.5 meg binary file 1000 times.
The results are everages of several runs.
 
file_get_contents() - 30.4 seconds (4.3 HEAD)
file() - 99.2 seconds (4.3 HEAD)
file() - 40.3 seconds (4.2.3)

Looks like a VERY  serious performance loss to me, I'd even go as far as to say this is a critical issue that should be resolved before release.

For reference perpouses a simple C program that freads() entire file to memory took only 18.5 seconds to run (1000 runs on the same file).
 [2002-10-18 15:40 UTC] iliaa@php.net
This bug has been fixed in CVS.

In case this was a PHP problem, snapshots of the sources are packaged
every three hours; this change will be in the next snapshot. You can
grab the snapshot at http://snaps.php.net/.
 
In case this was a documentation problem, the fix will show up soon at
http://www.php.net/manual/.

In case this was a PHP.net website problem, the change will show
up on the PHP.net site and on the mirror sites in short time.
 
Thank you for the report, and for helping us make PHP better.


 [2003-01-24 19:10 UTC] jamie at jamiebecker dot com
Please reopen. 
 
PHP 4.2.2 and 4.2.1 appeared to cause delays as well but   
PHP 4.3.0 made Squirrelmail unusable. phpMyAdmin appeared   
to be affected as well. Downgrading to PHP 4.0.6 resolved   
problem. Not clear if this file() function is culprit, but   
system load remained low and Cyrus and MySQL do not appear   
to be culprit of the application-level slowdown. (Command   
line clients do not experience problems.) NOTE: This does   
NOT appear to be resolved by CVS snapshot as of 2002 01-23,   
which is contrary to above PHP bug!  
  
Additional information:  
Gentoo Linux 1.2, nightly update (gcc 2.95)  
Cyrus IMAPd 2.1.11  
MySQL 3.23.54a  
PHP 4.3.0, 4.2.2, 4.2.1  
SquirrelMail 1.2.10  
phpMyAdmin 2.3.2 (Gentoo rev 1)  
  
Please also see  
http://bugs.gentoo.org/show_bug.cgi?id=14513
 
PHP Copyright © 2001-2014 The PHP Group
All rights reserved.
Last updated: Mon Apr 21 12:02:07 2014 UTC