php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #10701 readfile usage on large files
Submitted: 2001-05-07 07:34 UTC Modified: 2001-05-18 15:03 UTC
From: stephen at national-net dot com Assigned:
Status: Closed Package: Filesystem function related
PHP Version: 4.0.5 OS: Linux 2.4.x
Private report: No CVE-ID: None
 [2001-05-07 07:34 UTC] stephen at national-net dot com
Ok, this is a pretty intersting one, and I'm not even sure if it's a bug or just a memory limit thing. I am using a php wrapper for content files (.jpg|.gif|.asf|.mov|.ram), as you can guess, the movie files can be pretty large, upwards of 10MB, all getting read into memory, then spit back out. Couple that with a site that gets 10K+ visits a day and the server is on its knees with a load average of about 10 (when it gets to 25/30 things start swapping and it will dies not long after that.)

here's the code used to wrap the content:

<?php
$mime_type = strtolower(strrchr($f,'.'));
$mime_type_array = array(
        '.asf' => 'application/vnd.ms-asf',
        '.jpg' => 'image/jpeg',
        '.gif' => 'image/gif'
        );

if(!in_array($mime_type,array_keys($mime_type_array)))
        {
        header("Location: /error.php");
        }

$offset = 86400 * 3;
header("Expires: ".gmdate("D, d M Y H:i:s \G\M\T", time() + $offset));
header("Cache-Control: max-age=".$offset.", must-revalidate");
header("Last-modified : ".gmdate("D, d M Y H:i:s \G\M\T", filemtime("/web/sites/contentsite/".$f)));
header("Content-type: ".$mime_type_array[$mime_type]);
header("Content-length: ".filesize("/web/sites/contentsite/".$f));
@readfile("/web/sites/contentsite/".$f);
?>

so, I would pass an image or movie to the content file with a url like so:

http://contentsite.com/content.php?f=movies/bigmovie.asf


This is really just a heads up at this point, I know it will take you guys a little while to sort through this one, I'm not even sure it's a bug considering readfile is SUPPOSED to read a file into memory and spit it back out. I dunno, for now I'm going to do some .htaccess tricks where I force php to parse .htaccess files. If anyone has come across this or has any insight on wrapping content in php files, please email me at stephen@national-net.com

Thanks!
Stephen VanDyke

PS - aside from that, great language, I love PHP :)

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2001-05-17 17:29 UTC] stephen at national-net dot com
Anyone plan on claiming this?
 [2001-05-18 00:28 UTC] rasmus@php.net
What's to claim.  This is a support question that belongs on one of the mailing lists and not in the bug database.

But a hint.  Don't use readfile(), fopen() the file and read it a bit at a time instead of sticking the entire thing in memory.
 [2001-05-18 00:51 UTC] stephen at national-net dot com
You are kidding right? Nice way to take down a server.

load average: 66.52, 33.25, 15.76

$fp = fopen("/web/sites/contentsite/".$f);
while(!feof($fp))
       {
       echo fgets($fd, 4096);
       }
fclose($fp);
 [2001-05-18 11:10 UTC] sniper@php.net
Reopened. This is not fixed.

--Jani

 [2001-05-18 15:03 UTC] sas@php.net
It is almost impossible to tell from the output of system commands like top why certain processes have such a huge memory footprint and what kind of memory they are associated with.

In this case, you get the impression that the processes use much memory, because PHP maps files into the web-server's address space and sends them out in one quick move. These maps do not cause swapping though, as they can be removed  from physical RAM without any swapping.

Your problem is probably caused by using Apache for serving large static files. Every download ties up one Apache instance and hence the available RAM limits the number of users you can serve. Switching the file-server to e.g. thttpd would free up significant resources (RAM and CPU time).
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Mar 28 15:01:29 2024 UTC