php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #77443 FPM hangs on reload due to children handling SIGTERM since PHP 7.1
Submitted: 2019-01-11 07:47 UTC Modified: 2019-01-11 10:26 UTC
Votes:1
Avg. Score:3.0 ± 0.0
Reproduced:0 of 0 (0.0%)
From: emayoral at arsys dot es Assigned:
Status: Open Package: FPM related
PHP Version: 7.2.14 OS: CentOS 6
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: emayoral at arsys dot es
New email:
PHP Version: OS:

 

 [2019-01-11 07:47 UTC] emayoral at arsys dot es
Description:
------------
I run a system with a relatively large number of pfp-fpm pools. Whenever a pool is added or removed a php-fpm reload is issued.

The behaviour of php-fpm on reload is to terminate children with SIGQUIT, wait process_control_timeout (10 seconds in my config), then issue SIGTERM to the remaining children.

Now, some of my pool children will not die on SIGQUIT when opcache is used, but that is a different bug. Point is until PHP 7.0, SIGTERM would finish the remaining children, the reload would complete in 10 seconds and everything would be fine.

But starting with PHP 7.1 the php processes also handle SIGTERM (I suspect it uses the same signal handler as for SIGQUIT, although I am not 100% sure of that).

Now the children which did not die on SIGQUIT do not die on SIGTERM either and the reload will not complete until those children are manually issued a SIGKILL.

I think php-fpm 7.1 and higher should do its first round of SIGQUIT to its children on reload, but after process_control_timeout, issue a SIGKILL instead of a SIGTERM, since SIGTERM is not an effective way of ending rogue children any more.


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2019-01-11 10:26 UTC] emayoral at arsys dot es
Digging further into the code, I think this case is covered in:

sapi/fpm/fpm/fpm_process_ctl.c

static void fpm_pctl_action_next() /* {{{ */
{
        int sig, timeout;

        if (!fpm_globals.running_children) {
                fpm_pctl_action_last();
        }

        if (fpm_signal_sent == 0) {
                if (fpm_state == FPM_PCTL_STATE_TERMINATING) {
                        sig = SIGTERM;
                } else {
                        sig = SIGQUIT;
                }
                timeout = fpm_global_config.process_control_timeout;
        } else {
                if (fpm_signal_sent == SIGQUIT) {
                        sig = SIGTERM;
                } else {
                        sig = SIGKILL;
                }
                timeout = 1;
        }

        fpm_pctl_kill_all(sig);
        fpm_signal_sent = sig;
        fpm_pctl_timeout_set(timeout);
}

(Send SIGQUIT, wait process_control_timeout , send SIGTERM, wait 1 second, send SIGKILL)

But my experience is that after the SIGQUIT and SIGTERM, the SIGKILL never happens (Which didn't matter until php 7.1 as SIGTERM would do the job). Not sure why, though.
 [2019-01-22 09:55 UTC] mnikulin at plesk dot com
Please, have a look at
Bug #76601 "Partially working php-fpm after incomplete reload"
that happens for both 5.6 and 7.x versions and at
Bug #76895 "PHP-FPM7.x can't finish reloading after encountering a fatal error"
that is specific to 7.x versions and easily reproducible

The following one-line patch fix SIGKILL so the problem becomes invisible,
however the proper way would be to avoid ignoring of SIGQUIT and SIGTERM
by child processes
https://bugs.php.net/patch-display.php?bug_id=76601&patch=php-76601_kill-not-rescheduled_7.2.11_2018-09-28.patch&revision=latest
 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Thu Nov 14 09:01:30 2019 UTC