php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #78092 Opcache shared lockfile and fcntl()
Submitted: 2019-05-31 12:11 UTC Modified: -
From: stefan dot djordjevic at humanity dot com Assigned:
Status: Open Package: opcache
PHP Version: Irrelevant OS: Centos 7.6
Private report: No CVE-ID: None
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If this is not your bug, you can add a comment by following this link.
If this is your bug, but you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: stefan dot djordjevic at humanity dot com
New email:
PHP Version: OS:

 

 [2019-05-31 12:11 UTC] stefan dot djordjevic at humanity dot com
Description:
------------
Hi,

We got some strange behaviour which seems related to shared lockfile stored on a default path (/tmp/.ZendSem.04tkPY)
Randomly our Apache server with forked workers enter in some strange "loop", which slows them down and brings our load and number of apache workers to max.
I mange to strace them and what I see is:

cntl(10, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
fcntl(10, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
.
.

At one point in time it just passed this "condition" and execute, finish request.

File Descriptor is actually pointing to opcode lock files:
ls -l  /proc/30174/fd/10 
lrwx------ 1 root root 64 May 30 13:35 /proc/30174/fd/10 -> /tmp/.ZendSem.04tkPY (deleted)

As this condition happens pretty randomly, it's hard to identify real cause of this, but from what I see is that it entering some kind of loop, or race condition with locking - unlocking pattern.

I am trying to figure out how we get to this condition, as there are many "variables" included in our new setup (docker containers etc.) I am wondering if you guys have any clue if this can be triggered by some kind of bug in Zend opcode cache-ing mechanism or similar ?

Thanks.





Patches

Add a Patch

Pull Requests

Add a Pull Request

 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Sun Dec 08 21:01:25 2019 UTC