php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #36365 scandir duplicates file name at every 65535th file
Submitted: 2006-02-11 03:17 UTC Modified: 2015-07-23 22:22 UTC
Votes:4
Avg. Score:4.2 ± 0.8
Reproduced:2 of 2 (100.0%)
Same Version:0 (0.0%)
Same OS:1 (50.0%)
From: pilhoon at gmail dot com Assigned: pajoye
Status: Closed Package: Directory function related
PHP Version: * OS: Windows only
Private report: No CVE-ID:
 [2006-02-11 03:17 UTC] pilhoon at gmail dot com
Description:
------------
scandir duplicates file name at every 65535th file

Reproduce code:
---------------
I made over 260000 files in a folder.
Their names are 'f100001' ... 'f264001'.

$file_names = scandir('files/');

$base_names= array();
for($i=100001; $i<=263064; $i++)
{
	$base_names["f".$i] = 0;
}

foreach($file_names as $a_name)
{
	if(1 == $base_names[$a_name])
		echo $a_name."\n";
	else
		$base_names[$a_name] = 1;
}

Expected result:
----------------
They must be showed only once.

Actual result:
--------------
scandir(THAT_FOLDER) returns large array but
f165534 and f231069 are duplicated.



Patches

Add a Patch

Pull Requests

Pull requests:

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2006-02-11 13:21 UTC] sniper@php.net
Please try using this CVS snapshot:

  http://snaps.php.net/php5.1-latest.tar.gz
 
For Windows:
 
  http://snaps.php.net/win32/php5.1-win32-latest.zip


 [2006-02-19 01:00 UTC] php-bugs at lists dot php dot net
No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
 [2010-01-07 18:02 UTC] enzo at smshome dot net
Same problem here using PHP Version 5.2.5 (x64) with Windows XP 64bit Professional Edition.

Duplicated files every 65535 files read using scandir(). I think you should put an alert on manual page:
http://php.net/manual/en/function.scandir.php
 [2010-01-07 18:26 UTC] pajoye@php.net
Please try using this snapshot:

  http://snaps.php.net/php5.3-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/


 [2010-01-15 01:00 UTC] php-bugs at lists dot php dot net
No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
 [2012-09-08 03:24 UTC] gobie at centrum dot cz
Same problem reproduced
PHP Version 5.4.4 Windows 7 x64
PHP Version 5.3.13 Windows 7 x32

But all PHP methods listing files are affected scandir, glob, DirectoryIterator.

Reproduce code:
---------------
// Settings
$dir = './test/';
$totalFiles = 1e5;

// Create empty files
!is_dir($dir) && mkdir($dir);
for ($i = 0; $i < $totalFiles; ++$i) {
    $filename = $dir . str_pad($i, 6, '0', STR_PAD_LEFT);
    touch($filename);
}

// Glob
$files = glob($dir . '*');
echo 'glob: ' . count($files) . '/' . $totalFiles . PHP_EOL;

// Scandir
$files = scandir($dir);
echo 'scandir: ' . (count($files) - 2) . '/' . $totalFiles . PHP_EOL; // . and 
..

// DirectoryIterator
$it = new DirectoryIterator($dir);
echo 'DirectoryIterator: ' . (iterator_count($it) - 2) . '/' . $totalFiles . 
PHP_EOL; // . and ..
unset($it);

Expected result:
----------------
glob: 100000/100000
scandir: 100000/100000
DirectoryIterator: 100000/100000

Actual result:
--------------
glob: 100001/100000
scandir: 100001/100000
DirectoryIterator: 100001/100000
 [2012-09-08 09:24 UTC] pajoye@php.net
-Status: No Feedback +Status: Assigned -PHP Version: 5.1.2 +PHP Version: * -Assigned To: +Assigned To: pajoye
 [2015-06-07 23:51 UTC] cmb@php.net
The problem is in readdir_r()[1]. If dp->offset != 0
FindNextFile() is called, then dp->offset is increased. Finally,
the directory entry is composed and "returned". However,
dp->offset is a short int[2], so once for every 65535 entries
FindNextFile() is not called, but a directory entry is added
nonetheless. Furthermore, dp->dent.d_off is not correct for high
values.

[1] <https://github.com/php/php-src/blob/php-5.6.9/win32/readdir.c#L92-L118>
[2] <https://github.com/php/php-src/blob/php-5.6.9/win32/readdir.h#L35>
 [2015-07-23 05:59 UTC] cbader92 at gmail dot com
Can reproduce bug with PHP 5.5.12 (cli) (built: Apr 30 2014 11:20:58) in Windows 7 Home Premium x64, build 7601.

I thought I was going crazy. My 100,000 files generated an array with 100,001 indexes. Took forever to find out why my script was trying to copy more files than there were in the directory.
 [2015-07-23 12:59 UTC] cmb@php.net
-Status: Assigned +Status: Analyzed
 [2015-07-23 12:59 UTC] cmb@php.net
I've made a respective PR.
 [2015-07-23 22:22 UTC] cmb@php.net
-Operating System: windows +Operating System: Windows only
 [2015-07-27 23:07 UTC] cmb@php.net
Automatic comment on behalf of cmb
Revision: http://git.php.net/?p=php-src.git;a=commit;h=4e8f01cb6e75e09ad7ffa5201c42d1bea2a61264
Log: Fix #36365: scandir duplicates file name at every 65535th file
 [2015-07-27 23:07 UTC] cmb@php.net
-Status: Analyzed +Status: Closed
 [2015-08-04 20:54 UTC] ab@php.net
Automatic comment on behalf of cmb
Revision: http://git.php.net/?p=php-src.git;a=commit;h=4e8f01cb6e75e09ad7ffa5201c42d1bea2a61264
Log: Fix #36365: scandir duplicates file name at every 65535th file
 [2016-07-20 11:37 UTC] davey@php.net
Automatic comment on behalf of cmb
Revision: http://git.php.net/?p=php-src.git;a=commit;h=4e8f01cb6e75e09ad7ffa5201c42d1bea2a61264
Log: Fix #36365: scandir duplicates file name at every 65535th file
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Tue Aug 29 15:01:52 2017 UTC