Having looked into a few server, it seems the same file is used for all FPM processes. So if there is any issue with the lock file and kill_all_lockers comes into action, there's a good chance on most multi-pool servers of the process to start spinning.

Ideally:
- Lock file should be opened after fork in FPM, or maybe even defer extension activation until after fork, in any case, sharing things amongst the same pool rather than the entire PHP-FPM pool set.
- A backoff on the kill so it performs once every X milliseconds instead of spinning.

I guess I have some concern on the latter though, in that it will fix the CPU but it will then have the side effect of hiding the fact that a process is effectively deadlocked and the pool-size reduced. So I wonder if anyone knows if the former, or similar solution, is viable?

I have not noticed this happen since moving to 5.6.31 on my servers, has been 6 weeks now. With 5.6.30 was happening every few days.

Hopefully that is the case for the others experiencing this.

I spoke too soon. Running 5.6.31 just had the problem again. strace shows:

kill(16099, SIGKILL)                    = -1 EPERM (Operation not permitted)
fcntl(90, F_GETLK, {type=F_RDLCK, whence=SEEK_SET, start=1, len=1, pid=16099}) = 0

I am experiencing the same issue on Ubuntu 16.04.3 with php 7.0.22 on 8 core system with 16G of RAM.

I have two pools (one with pm = dynamic, and the other is pm = static with single child). I get this bug constantly, every few hours my server goes into that spinning mode with all dynamic pool processes getting killed and only master process left with the process from static pool. Master process takes around 80% of all 8 cores. In the fpm-log I get the following lines:

[16-Aug-2017 20:49:03] NOTICE: [pool www] child 8169 exited with code 0 after 0.034016 seconds from start
[16-Aug-2017 20:49:03] NOTICE: [pool www] child 8179 started

Repeated about 600 times per second. You can understand that the log grows several GB in size quite fast and only this log issue can lead to self-produced denial of service once all server space gets used. I had to reduce log level to warning in order to prevent this.

Reloading FPM service is not helping, I have to restart it in order to fix this behavior.

Hi tarasov dot igor at gmail dot com

Possibly your issue is unrelated to this one. This issue pertains to kill_all_lockers failing and triggering a loop within a child process as it attempts to kill opcache lockers for processes running as other users.

Your issue seems to be related to child death and the parent looping to respawn (or along those lines) - I suggest raising your issue in a new bug ticket so it can be looked at independently.

devel at jasonwoods dot me dot uk, what process was spinning in your case - master or child?

tarasov dot igor at gmail dot com

child process

So, then this seems to be unrelated. My case should be one of these:

https://bugs.php.net/bug.php?id=61558
https://bugs.php.net/bug.php?id=70185
https://bugs.php.net/bug.php?id=73056

There is no infinite loop since https://github.com/php/php-src/commit/bcee2fdbec4f4bba59d4134003cfaf5b1f9b67ab
However, if some pool keeps the lock all other pools fail to process queries because of bailing out at https://github.com/php/php-src/blob/bcee2fdbec4f4bba59d4134003cfaf5b1f9b67ab/ext/opcache/ZendAccelerator.c#L657.


STR
1. set opcache.force_restart_timeout to X seconds (eg 10sec)
2. create 3 domains with php-fpm handler (sleep.cc, reset.cc, info.cc)
3. create do.php with call of `sleep(Y)` when Y > X (eg 120sec) in the scope of sleep.cc
4. create do.php with call of `opcache_reset()` in the scope of reset.cc
5. create do.php with call of `phpinfo()` in the scope of info.cc
6. open info.cc/do.php, it should work as expected
7. open sleep.cc/do.php to acquire a long lock of cache
8. open reset.cc/do.php to schedule reset of cache
9. wait X+ seconds and try to open info.cc/do.php

Actual result
pool which serves info.cc tries to kill the process which executes sleep.cc/do.php, failed with EPERM, info.cc/do.php isn't executed but downloaded

Expected result
info.cc/do.php is executed


I solve it with replacing of `ACCEL_LOG_ERROR` with `ACCEL_LOG_WARNING` so all queries which are received after scheduling of reset and before performing of reset are just processed without opcache.