php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #76482 opcache + MPM ITK might cause apache2 to halt
Submitted: 2018-06-15 14:00 UTC Modified: -
Votes:6
Avg. Score:4.2 ± 0.9
Reproduced:3 of 3 (100.0%)
Same Version:2 (66.7%)
Same OS:2 (66.7%)
From: helge at monsternett dot no Assigned:
Status: Open Package: opcache
PHP Version: 5.6.36 OS: Debian Stretch
Private report: No CVE-ID: None
View Add Comment Developer Edit
Anyone can comment on a bug. Have a simpler test case? Does it work for you on a different platform? Let us know!
Just going to say 'Me too!'? Don't clutter the database with that please — but make sure to vote on the bug!
Your email address:
MUST BE VALID
Solve the problem:
13 + 16 = ?
Subscribe to this entry?

 
 [2018-06-15 14:00 UTC] helge at monsternett dot no
Description:
------------
The exact version is 5.6.33+dfsg-0+deb8u1, packaged by Debian.

Reading the source code in ZendAccelerator.c and your documentation for `opcache.log_verbosity_level`, it would seem like opcache will in some cases kill other apache2 processes (in `kill_all_lockers`) if the accelerator hasn't been restarted in a timely fashion.

This does not work well with the MTM ITK apache2 module enabled, since the sibling process to be killed might be owned by a different unix user, resulting in an infinite stream of:

kill(16329, SIGKILL)                    = -1 EPERM (Operation not permitted)
fcntl(523, F_GETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1, l_len=1, l_pid=16329}) = 0 
.. repeating ..

Here's the backtrace when this happens:

#0  0x00007f8f5a2c5317 in kill () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f8efe7238c6 in ?? () from target:/usr/lib/php5/20131226/opcache.so
#2  0x00007f8f554702ce in zend_llist_apply () from target:/usr/lib/apache2/modules/libphp5.so
#3  0x00007f8f5546c18a in init_executor () from target:/usr/lib/apache2/modules/libphp5.so
#4  0x00007f8f5547cd43 in zend_activate () from target:/usr/lib/apache2/modules/libphp5.so
#5  0x00007f8f55417da5 in php_request_startup () from target:/usr/lib/apache2/modules/libphp5.so
#6  0x00007f8f5552e46a in ?? () from target:/usr/lib/apache2/modules/libphp5.so
#7  0x0000564b7b544c40 in ap_run_handler ()
#8  0x0000564b7b5451d6 in ap_invoke_handler ()
#9  0x0000564b7b55cdd3 in ap_process_async_request ()
#10 0x0000564b7b55cee0 in ap_process_request ()
#11 0x0000564b7b558f9d in ?? ()
#12 0x0000564b7b54ea70 in ap_run_process_connection ()
#13 0x00007f8f568929dd in itk_fork_process () from target:/usr/lib/apache2/modules/mpm_itk.so
#14 0x0000564b7b54ea70 in ap_run_process_connection ()
#15 0x00007f8f564856bf in ?? () from target:/usr/lib/apache2/modules/mod_mpm_prefork.so
#16 0x00007f8f56485944 in ?? () from target:/usr/lib/apache2/modules/mod_mpm_prefork.so
#17 0x00007f8f5648680d in ?? () from target:/usr/lib/apache2/modules/mod_mpm_prefork.so
#18 0x0000564b7b52800e in ap_run_mpm ()
#19 0x0000564b7b520c4d in main ()

It's hard for me to find out if this is fixed in later versions or not, since I'm not sure how I can even force opcache to execute `kill_all_lockers`. This happened sporadically (2 times per day on average) in a production server before I understood the problem and disabled the opcache extension. But I thought I would send a report in case this was an unknown problem. I understand it's not strictly a problem with opcache, but at the same time, it should handle the case where it's not able to kill the process better.


Patches

dont-while-loop-for-ever-php-7.3-only-for-noperm (last revision 2019-02-01 13:52 UTC by arekm at maven dot pl)
dont-while-loop-for-ever-php-7.3 (last revision 2019-02-01 13:43 UTC by arekm at maven dot pl)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2019-02-01 13:41 UTC] arekm at maven dot pl
It was reported in 5.6.36 but the bug is in 7.1 (reproduced here) and by looking into source code also in current 7.3 branch.

Not only mpm itk is affected but any MPM which switches child processes to different UIDs (via setuid()). Other example is mod_ruid2 - https://github.com/mind04/mod-ruid2/issues/17

kill_all_lockers() has a loop which never ends if killing fails
as kill(mem_usage_check->l_pid, SIGKILL) returns EPERM in errno.


There is

                if (!success) {
                        /* errno is not ESRCH or we ran out of tries to kill the locker */
                        ZCSG(force_restart_time) = time(NULL); /* restore forced restart request */
                        /* cannot kill the locker, bail out with error */
                        zend_accel_error(ACCEL_LOG_ERROR, "Cannot kill process %d: %s!", mem_usage_check->l_pid, strerror(errno));
                }

but it doesn't break the loop. Is brake missing here?
 [2019-02-01 13:51 UTC] arekm at maven dot pl
If the break is not a good thing (because the rest of code needs to run then perhaps the other variant).
 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Tue Oct 22 11:01:29 2019 UTC