php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #74083 master PHP-fpm is stopped on multiple reloads
Submitted: 2017-02-11 06:03 UTC Modified: -
Votes:11
Avg. Score:4.3 ± 0.7
Reproduced:10 of 10 (100.0%)
Same Version:6 (60.0%)
Same OS:6 (60.0%)
From: afalaleev at plesk dot com Assigned:
Status: Closed Package: FPM related
PHP Version: 7.1.1 OS: CentOS 7.x
Private report: No CVE-ID: None
View Developer Edit
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: afalaleev at plesk dot com
New email:
PHP Version: OS:

 

 [2017-02-11 06:03 UTC] afalaleev at plesk dot com 
Description:
------------
PHP-FPM is stopped, when it receive more than one SIGUSR2 in a short time.
 
The reason:
- On receiving SIGUSR2 php-fpm calls execvp(path, ...)
- Process starts. 
--  It inherit all opened handles.
--  But all signal handlers is reseted to SIG_DFL

So, if SIGUSR2 income before process change the signal handler, it will exit, because it's a default action on receiving SIGUSR2.

The bug is actual for all versions of PHP: 5.6, 7.0, 7.1.

The possible decision (in attached patch) is to block the signal before execvp(...), and unblock it after changing the signal handlers.

The attached patch solve the problem for all versions of PHP: 5.6, 7.0, 7.1.


Test script:
---------------
With a high probability, it may happened on send 2 reloads without any timeout:

# systemctl reload php71-fpm.service; systemctl reload php71-fpm.service

For greater reliability, the following script can be executed:

# for i in $(seq 1 100); do systemctl reload php71-fpm.service; done
 

Expected result:
----------------
# systemctl status php71-fpm.service 
● php71-fpm.service - The PHP 7.1.1 FastCGI Process Manager
   Loaded: loaded (/usr/lib/systemd/system/php71-fpm.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2017-02-11 12:41:10 +07; 6min ago
  Process: 17828 ExecReload=/bin/kill -USR2 $MAINPID (code=exited, status=0/SUCCESS)
 Main PID: 17002 (php-fpm)
   Status: "Processes active: 0, idle: 0, Requests: 0, slow: 0, Traffic: 0req/sec"
   CGroup: /system.slice/php71-fpm.service
           └─17002 php-fpm: master process (/opt/php/7.1/etc/php-fpm.conf)

Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.
Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.
Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.
Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.
Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.
Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.
Feb 11 12:41:30 a10-52-35-250.qa.ru systemd[1]: Reloaded The PHP 7.1.1 FastCGI Process Manager.


Actual result:
--------------
# systemctl status php71-fpm.service; 
● php71-fpm.service - The PHP 7.1.1 FastCGI Process Manager
   Loaded: loaded (/usr/lib/systemd/system/php71-fpm.service; enabled; vendor preset: disabled)
   Active: failed (Result: signal) since Sat 2017-02-11 12:27:11 +07; 13min ago
  Process: 16348 ExecReload=/bin/kill -USR2 $MAINPID (code=exited, status=0/SUCCESS)
 Main PID: 15808 (code=killed, signal=USR2)
   Status: "Processes active: 0, idle: 0, Requests: 0, slow: 0, Traffic: 0req/sec"

Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.
Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.
Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.
Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.
Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.
Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.
Feb 11 12:27:12 a10-52-35-250.qa.ru systemd[1]: Unit php71-fpm.service cannot be reloaded because it is inactive.

Patches

php-fpm-74083-sig-block-reexec_7.2.11_2018-09-28.patch (last revision 2018-10-16 04:49 UTC by mnikulin at plesk dot com)
php-74083_draft-block-signal-reload_2018-07-10.patch (last revision 2018-07-10 10:38 UTC by mnikulin at plesk dot com)

Pull Requests

Pull requests:

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2018-07-10 10:44 UTC] mnikulin at plesk dot com 
Let's start discussion in which way this bug can be fixed with a patch that uses sigprocmask() to block other reload signals during execvp() and early initialization.

Do not use it in production. Instead of simply dead processes you may get more peculiar state described in https://bugs.php.net/bug.php?id=76601
 [2018-07-11 08:28 UTC] mnikulin at plesk dot com 
It seems that combining the attached patch with the one from https://bugs.php.net/bug.php?id=76601 helps in the case of concurrent reload SIGUSR2 signals. At least it may be considered as quick plumbing.
 [2018-10-16 04:53 UTC] mnikulin at plesk dot com 
I have refreshed the patch. I recommend to apply patches for #76601
on the top of it. It is made for 7.2.11 but can be applied to
7.1.23, 7.0.32, and 5.6.38 as well.
 [2018-10-16 11:03 UTC] mnikulin at plesk dot com 
Related To: Bug #76895
 [2019-02-20 03:59 UTC] mnikulin at plesk dot com 
Related To: Bug #76601
 [2019-07-24 10:07 UTC] mnikulin at plesk dot com 
The following pull request has been associated:

Patch Name: Block signals during fpm master initialization
On GitHub:  https://github.com/php/php-src/pull/4471
Patch:      https://github.com/php/php-src/pull/4471.patch
 [2019-10-20 15:58 UTC] bukka@php.net 
Automatic comment on behalf of bukka
Revision: http://git.php.net/?p=php-src.git;a=commit;h=666fb9a1946a6eee3019583fc705894a300b0f6d
Log: Add NEWS for the fixed bug #74083
 [2019-10-20 15:58 UTC] bukka@php.net
-Status: Open +Status: Closed
 [2019-10-20 15:59 UTC] bukka@php.net 
Automatic comment on behalf of bukka
Revision: http://git.php.net/?p=php-src.git;a=commit;h=666fb9a1946a6eee3019583fc705894a300b0f6d
Log: Add NEWS for the fixed bug #74083
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Thu Nov 21 11:01:29 2024 UTC