php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #35326 Concurrency issue with recursive mkdir
Submitted: 2005-11-22 11:32 UTC Modified: 2013-10-30 08:49 UTC
Votes:13
Avg. Score:4.1 ± 0.8
Reproduced:12 of 12 (100.0%)
Same Version:2 (16.7%)
Same OS:9 (75.0%)
From: bugs dot php dot net at chsc dot dk Assigned:
Status: Not a bug Package: *General Issues
PHP Version: 5.1.6 OS: Linux
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: bugs dot php dot net at chsc dot dk
New email:
PHP Version: OS:

 

 [2005-11-22 11:32 UTC] bugs dot php dot net at chsc dot dk
Description:
------------
There appears to be a concurrency issue when mkdir is called on two separate threads with the following parameters:

Thread 1:
mkdir('/tmp/foo/bar/1', 0777, true);

Thread 2:
mkdir('/tmp/foo/bar/2', 0777, true);

Sometimes one of the threads (e.g. thread 2) two fails with "Warning: mkdir(): File exists" and the directory /tmp/foo/bar/2 does not exist afterwards.

What I think is happening in thread 2 is is this:
PHP looks for /tmp/foo/bar/2 and sees that it does not exist. Neither does /tmp/foo/bar and /tmp/foo. So it tries to create /tmp/foo, but that directory was just created a millisecond second ago by another thread, so mkdir gives instead of just moving on to create /tmp/foo/bar and /tmp/foo/bar/2.

Reproduce code:
---------------
This code reproduces the problem in about every second attempt. It consists of two files, index.php and iframe.php.


index.php:

<?php
$fire = time() + 2;
?>
<h1>Nr. 1</h1>
<iframe src="iframe.php?fire=<?= $fire ?>&amp;no=1"></iframe>
<h1>Nr. 2</h1>
<iframe src="iframe.php?fire=<?= $fire ?>&amp;no=2"></iframe>


iframe.php:

<?php
$fire = $_GET['fire'];
$no = $_GET['no'];
// synchronize the two threads
while ($fire > time());
$dir = '/tmp/' . $fire . '/foo/bar/foo/bar/' . $no;
//sleep($no);
var_dump(is_dir($dir));
echo "Making $dir ";
mkdir($dir, 0777, true);
var_dump(is_dir($dir));
?>


Expected result:
----------------
Thread 2 should output this:
bool(false) Making /tmp/1132653651/foo/bar/foo/bar/2 bool(true)


Actual result:
--------------
Thread 2 sometimes outputs this:
bool(false) Making /tmp/1132655181/foo/bar/foo/bar/2 Warning: mkdir(): File exists in /home/chsc/public_html/mkdir/iframe.php on line 10 bool(false) 

Notice that the directory does not exist afterwards.

If I uncomment the sleep() call above, there are no problems.


A workaround:
$ok = mkdir($dir, 0777, true);
if (!$ok && !is_dir($dir)) {
    sleep(1);
    mkdir($dir, 0777, true);
}


In other words, it works if I try again. I hope that mkdir() can be changed so that it only returns false if the directory cannot be created at all (e.g. because of lack of permissions or because it already exists).


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2005-11-22 12:13 UTC] tony2001@php.net
Please try using this CVS snapshot:

  http://snaps.php.net/php5-latest.tar.gz
 
For Windows:
 
  http://snaps.php.net/win32/php5-win32-latest.zip


 [2005-11-22 13:52 UTC] bugs dot php dot net at chsc dot dk
Yes, it also occurs with the CVS snapshot (this time I tried with the CGI version).

AFAICS the problem is in php_plain_files_mkdir in main/streams/plain_wrapper.c. If php_mkdir() or VCWD_MKDIR() returns false, the method gives up instead of checking whether directory exists anyway, i.e. if it was created by another thread.
 [2006-10-10 07:25 UTC] bugs dot php dot net at chsc dot dk
Problem still exists in PHP 5.1.6.
 [2013-07-23 22:55 UTC] gustavo dot straube at gmail dot com
Problem still exists in PHP 5.3.3.
 [2013-09-30 10:39 UTC] me at vlastv dot ru
Problem still exists in PHP 5.4.1.
 [2013-10-01 04:15 UTC] pajoye@php.net
-Package: Feature/Change Request +Package: *General Issues
 [2013-10-01 04:15 UTC] pajoye@php.net
I am not sure it is actually a PHP problem. To create the right locking mechanism 
to avoid such issue to happen between (almost) simultaneous requests is the 
application developer job, not the core.

Adding such tests should be done in almost all file creation operations (and some 
other) and will drastically performance without actually solving the problem.
 [2013-10-01 04:21 UTC] pajoye@php.net
Should have been:

... and will drastically affect performance without actually solving the 
problem....
 [2013-10-01 04:52 UTC] pajoye@php.net
-Status: Open +Status: Not a bug
 [2013-10-01 04:52 UTC] pajoye@php.net
Update the status.

And adding a small note about "why is it not a php bug", what is described here 
can happen anywhere outside php as well, be a shell script, ftp/ssh, etc.
 [2013-10-30 08:49 UTC] bugs dot php dot net at chsc dot dk
> To create the right locking mechanism to avoid such issue to happen

We don't need a locking mechanism. If the creation of a directory fails, we just need to check if it exists anyway (i.e. it has been created by a concurrent thread). If it does, we can proceed to create the subdirectories.
 [2013-10-30 10:13 UTC] gustavo dot straube at gmail dot com
> We don't need a locking mechanism. If the creation of a directory fails, we just need to check if it exists anyway [...]

In the application I experienced this bug, I do have a condition before the directory creation.

1. if (!is_dir($path))
2.   mkdir($path, 0755, true);

If I have two concurrent process ("A" and "B") running the code in this order (process - line):

A - 1
B - 1
B - 2
A - 2

When process "A" verified the directory existence, it really didn't exist.
 [2018-08-21 19:23 UTC] nasretdinov at gmail dot com
You do not need any locking at all. mkdir() returns EEXIST when directory exists and it should not be treated as error in this function. "mkdir -p" does this, for example.
 [2021-12-06 18:54 UTC] toonitw at gmail dot com
Fixed in 8.1
https://github.com/php/php-src/pull/7383
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Tue Apr 16 21:01:28 2024 UTC