php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #20863 md5_file() and exec() freeze
Submitted: 2002-12-06 10:38 UTC Modified: 2002-12-06 18:29 UTC
From: mike dot duncan at sonopress dot com Assigned:
Status: Not a bug Package: Filesystem function related
PHP Version: 4CVS-2002-12-06 (stable) OS: SuSE 7.3 Pro
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: mike dot duncan at sonopress dot com
New email:
PHP Version: OS:

 

 [2002-12-06 10:38 UTC] mike dot duncan at sonopress dot com 
Here is my src:

--------------------
$files = array();
$dirs = array();

function getMD5OfFile($f)  {
  return md5_file($f);
  //return exec("md5sum ".$f." | awk '{print $1}'");
}

function getFiles($d)  {
  global $files, $dirs;
  clearstatcache();
        
  if($dir = @opendir($d))  {
    while(($file = @readdir($dir)) !== false)  {
      if(($file != ".") && ($file != ".."))  {
        if(is_dir($d."/".$file)) $dirs[] = $d."/".$file;
        elseif(is_file($d."/".$file) && (substr($file,0,1) != ".")) $files[] = $d."/".$file;
      }               
    }
  }
}

/** Get files for directories **/
getFiles("/home/webdev");
while(($dir = array_shift($dirs)) !== null)  {
  getFiles($dir);
}

/** Ouptut findings **/
while(($file = array_shift($files)) !== null)  {
  echo $file.": ". getMD5FromFile($file) ."\n";
}

------------------

I first tried this with v4.2.1. What I am trying to do is get the MD5 for every file within the filesystem. Unfortunately this code freezes after about 20 or so files (md5_file() or exec() returns ok until then). If I uncomment the exec() return and comment the md5_file() return, the script hangs at exactly the same spot. If comment both out and run the script without md5'ing any files, I get a listing of all files (no problem). 

I have also tried replacing the md5_file() code with one that was listed here http://www.php.net/manual/en/function.md5-file.php that will read the file into a variable and do a md5() on that var. Unfortunately I got a FATAL: emalloc() error using that method. 

Finally, I tried getting the upgrading to v4.3.0-dev (latest snapshot from snaps) and got the same results.

I cannot figure out why this hangs and does not bomb out. I think it has something to do with maybe a cache of some sort or the memory buffers are filling up. Maybe even something to with how I am traversing the array of directories? But then, why would it work without the md5 cals? I do not know, but I thought I would post it here and see what you guys think. TIA.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2002-12-06 10:44 UTC] mike dot duncan at sonopress dot com 
BTW, The files I am examining are all text.
 [2002-12-06 11:06 UTC] iliaa@php.net 
Does your PHP have a memory limit?
 [2002-12-06 11:38 UTC] mike dot duncan at sonopress dot com 
I do not have this enabled. My configure line looks like so:

./configure --enable-ftp --with-ldap=/usr/local/openldap --with-mysql=/usr --with-openssl=/usr/local/openssl

I thought for the memory_limit directive to take effect would mean I would to have had to add --enable-momory-limit to the configuration. Has this changed recently (i.e. >= 4.2.1)?
 [2002-12-06 11:41 UTC] mike dot duncan at sonopress dot com 
I also tried adding ini_set("max_execution_time", "180") to the script and it changed nothing. Just to make sure it was not hitting this limit too.
 [2002-12-06 11:45 UTC] iliaa@php.net 
Could you please remove the @ from your script, which block error message and set error repoting level to E_ALL.
Please include any warning messages in your reply.
 [2002-12-06 12:08 UTC] mike dot duncan at sonopress dot com 
I added error_reporting(E_ALL) and got no errors or warnings. I actually do not get anything but the expected output...

/home/webdev/$dir/$file : $md5

...unless I change the md5 calculation to that of the binary one here: http://www.php.net/manual/en/function.md5-file.php.  Then I get a FATAL: emalloc() Could not allocate xxxxx bytes. 

But this is not how I run this script. I always use the md5_file() or exec() functions to calc. the MD5 of the file. 

Other than the expected output, with or without E_ALL set, I get nothing, hanging after about 20 or so md5 calculations.

Can you reproduce this when you copy+paste this and run it?
 [2002-12-06 12:14 UTC] mike dot duncan at sonopress dot com 
This is running as a script, from a shell (i.e. #!/usr/local/bin/php); not as a web app. We are trying to create a script that (scheduled from cron) will create MD5 for all files for a type of tracking system.
 [2002-12-06 12:15 UTC] iliaa@php.net 
Does your directory structure you are doing md5s on, have any recursive symlinks or any symlinks for that matter?
 [2002-12-06 12:32 UTC] mike dot duncan at sonopress dot com 
I am unsure as to what is going on. I do not see any symlinks at all actually. 

When I ran the same script with /usr/bin (no sub-direcories in /usr/bin) it ran fine (slowly, but fine). So this cannot be something referring to a memory buffer or cache. 

It is probably my coding. I appreciate your help very much.
 [2002-12-06 18:29 UTC] sniper@php.net 
Most likely user error since this is the only report about it..
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Tue Apr 16 20:01:31 2024 UTC