php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #63709 flock() doesn't trigger mandatory locks on linux
Submitted: 2012-12-06 13:12 UTC Modified: 2013-04-04 08:32 UTC
Votes:2
Avg. Score:5.0 ± 0.0
Reproduced:2 of 2 (100.0%)
Same Version:1 (50.0%)
Same OS:2 (100.0%)
From: eric dot saintetienne at gmail dot com Assigned:
Status: Analyzed Package: Filesystem function related
PHP Version: 5.3.19 OS: Linux
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: eric dot saintetienne at gmail dot com
New email:
PHP Version: OS:

 

 [2012-12-06 13:12 UTC] eric dot saintetienne at gmail dot com
Description:
------------
Locking exclusively via flock a file opened for writing doesn't trigger a 
mandatory lock.
The python script below could trigger the mandatory lock.

Maybe it's because PHP flock() is built on the C function call flock():
"When a program attempts to lock a file with lockf or fcntl that has mandatory 
locking set, the kernel will prevent all other programs from accessing the file. 
Processes which use flock will not trigger a mandatory lock."
source: http://www.hackinglinuxexposed.com/articles/20030623.html

Python script:
#!/usr/bin/python

import os, fcntl

fd = os.open('mandlock-file', os.O_RDWR, 0755)
print 'fd=', fd
fcntl.lockf(fd, fcntl.LOCK_EX)

a = raw_input() # during that time, any attempt to open the file will hang
os.write(fd, a+'\n')

fcntl.lockf(fd, fcntl.LOCK_UN)
# now any attempt to open the file will succeed

os.close(fd)

# eof

Test script:
---------------
<?php
define('FILENAME', 'mandlock-file');

/* first create a file with a mandatory lock:
 *   $ touch mandlock-file
 *   $ chmod g+s,g-x mandlock-file
 *   $ sudo mount -o remount,mand / # my file is inside the '/' mountpoint
*/

function openlockedfile($filename) {
  $fp = fopen($filename, 'w+');

  while (($fp === false) || (flock($fp, LOCK_EX) !== true)) {
    sleep(1);
    echo "spin lock\n";

    if ($fp === false)
      $fp = fopen($filename, 'w+');
  }
  return $fp;
}

$fd = openlockedfile(FILENAME); // open + lock EX

sleep(10);

/* During this sleep time, accessing the file should hang until the
 * php script releases the exclusive lock:
 *   $ cat mandlock-file # should hang until php script termination
 * Practically, cat doesn't hang (and works) proving the file isn't
 * exclusively locked by the PHP script. */

closefile($fd); // unlock + close

/* eof */
?>

Expected result:
----------------
once the file is created and the mandatory lock setup for it:
during the 10 sec timer, opening the file (with or without explicit locking) 
should hand until the php script terminates.

Actual result:
--------------
it's possible to opening the locked file during the 10s timer for reading and 
writing.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2012-12-06 14:19 UTC] eric dot saintetienne at gmail dot com
Note that dio_fcntl() of the "Direct IO" extension can successfully exclusively 
lock the file, but this shouldn't be considered as a workaround as it's not always 
possible to install extensions.

If flock() couldn't be modified for backward compatibility reasons, options could 
be added to alter its behaviour, or a new call lockf() would be welcome too.
 [2012-12-07 01:48 UTC] aharvey@php.net
The key difference between Python and PHP here is that Python always uses fcntl() internally, whereas PHP will use flock() if it's available (which it obviously is on Linux) and will only fall back to fcntl() if it's not. flock() will never create a mandatory lock, so the manual page is wrong, which I'm pretty sure is my fault. Mea culpa.

We can probably fix this by switching to preferring fcntl() within our flock() function as Python does, since that's actually the more useful behaviour, but that would be a (mild) BC break in how flock() behaves in practice — although it would actually bring it into line with what's documented.

The options I see are:

1. Change the behaviour of flock() as described above to prefer fcntl().
2. Add a new lockf() function, as suggested.
3. Just bite the bullet and expose fcntl() as a PHP function on POSIX platforms.
4. Do nothing and update the manual. :)

Does anyone have any thoughts? I'm happy to do the donkey work, but am not really sure on the best way to proceed.
 [2012-12-07 01:48 UTC] aharvey@php.net
-Status: Open +Status: Analyzed
 [2012-12-07 03:24 UTC] laruence@php.net
I like 3 :)

change the behavior of flock will intruduce a  visible bc break
 [2012-12-07 08:42 UTC] eric dot saintetienne at gmail dot com
A fifth option is to pull the "Direct IO" extension into the mainline.
http://pecl.php.net/package/dio

This extension already expose fcntl() as well as a few other low-level
POSIX routines, and have some amount of testing as it's in its fourth
version already (though it's said to be in beta state)
 [2012-12-07 08:48 UTC] aharvey@php.net
My worry there is that dio resources are (as I recall, anyway) completely distinct from normal file resources, so you couldn't fopen() a file and then dio_fcntl() it: it's all or nothing.
 [2012-12-07 08:55 UTC] eric dot saintetienne at gmail dot com
You're right, dio is a plain inteface to the underlying C function hence exposing 
real file descriptors (integers).

That's also what Python does: it exposes two different types of file objects: 
standard file objects via the builtin open() and file descriptors via os.open()

Is that is feasible with php?
 [2012-12-07 09:08 UTC] aharvey@php.net
That's true, but they're still somewhat interchangeable in Python — higher level file objects returned by open() work with fcntl methods. That wouldn't be the case if we bundled dio without further work.
 [2012-12-07 09:43 UTC] eric dot saintetienne at gmail dot com
You're right, Python is smart and the trick is simple: fnctl module functions 
are coded such that they detect the type of the object they're given as 
argument. If it's an integer they assume it is a file descriptor otherwise they 
call its fileno() method to retrieve the file descriptor integer.

So it's a matter of adding your own fileno() method to the PHP standard file 
object and making the dio_* routines calling it, when not provided with an 
integer.

Does that makes sense to you?

It's a suggestion, at the end of the day it's your decision of how to handle 
this issue, even though it seems to, I'm actually not pushing to get direct io 
integrated at any cost (I don't have any stake) but I just feel it's the way to 
go.
 [2013-04-01 23:01 UTC] mi+php at aldan dot algebra dot com
I am puzzled, what can the current behavior be possibly used for?

If the lock is not really locking (and it does not -- neither on Linux nor on FreeBSD), then why bother with it at all? And if nobody bothers, then why not fix it properly?

BTW, at least, on BSD the different locking mechanisms create compatible locks:

     The flock(), fcntl(2), and lockf(3) locks are compatible.
     Processes using different locking interfaces can cooperate
     over the same file safely. However, only one of such interfaces
     should be used within the same process.  If a file is locked by
     a process through flock(), any record within the file will be
     seen as locked from the viewpoint of another process using
     fcntl(2) or lockf(3), and vice versa.
 [2013-04-02 07:58 UTC] eric dot saintetienne at gmail dot com
I hope we're still speaking of MANDATORY locks (the ones provided by "mount -o 
mand") and not standard file locks? Other locks (advisory) behave as expected.

So what's the solution you chose to allow locking a file with a MANDATORY lock 
using PHP?
 [2013-04-04 01:49 UTC] mi+php at aldan dot algebra dot com
This is my test case:

<?php
function warn($message) {
	global $argv;
	fputs(STDERR, "$argv[0]: $message\n");
}

function add_dir($dir) {
	$lname = $dir . '/distd.pid';
	$lock = fopen($lname, 'c+');
	if (!$lock) {
		warn("$lname: can not open");
		return FALSE;
	}
	if (!flock($lock, LOCK_EX|LOCK_NB)) {
		warn("$lname: can not lock");
		return FALSE;
	}
	warn(getmypid() . " ". date('H:i:s') . " $lname successfully locked");
	return (fputs($lock, getmypid() . "\n". gethostname() . "\n") != FALSE);
}

date_default_timezone_set('America/New_York');
add_dir('/tmp');
sleep(3);
warn(getmypid() . " ". date('H:i:s') . " exiting in peace");
?>

Running TWO of the above in parallel:
% ( php t.php < /dev/null & php t.php < /dev/null )  >& l
I see BOTH of them claim to have "successfully" gotten the lock:
% cat l
[1] 26815
t.php: 26815 21:48:34 /tmp/distd.pid successfully locked
t.php: 26814 21:48:34 /tmp/distd.pid successfully locked
t.php: 26815 21:48:37 exiting in peace
t.php: 26814 21:48:37 exiting in peace
 [2013-04-04 08:32 UTC] eric dot saintetienne at gmail dot com
I appreciate your code and efforts but please stay focus on the topic: MANDATORY 
locks only. The issue is that PHP doesn't provide a standard way of triggering a 
mandatory lock.

MANDATORY locks are specific to Linux and occurs on a volume that is mounted 
with the "-o mand" mount flag.
ADVISORY locks exist on any system via flock() or lockf() calls and are not the 
subject of this thread.

Now I'm speaking to PHP developpers: how do you plan to fill in the gap?

Thanks
 [2013-04-04 14:19 UTC] mi+php at aldan dot algebra dot com
Eric, I'm confused. Are you saying, the problem I am illustrating with my test case is different? Should I file a separate bug-report?
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Sun Nov 19 01:31:42 2017 UTC