php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #76280 Performance degrades permantly when overwriting PHP files, until apache restart
Submitted: 2018-04-27 19:09 UTC Modified: 2018-04-28 15:27 UTC
Votes:2
Avg. Score:4.0 ± 1.0
Reproduced:2 of 2 (100.0%)
Same Version:2 (100.0%)
Same OS:1 (50.0%)
From: teo8976 at gmail dot com Assigned:
Status: Open Package: opcache
PHP Version: 5.6.36 OS: debian
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2018-04-27 19:09 UTC] teo8976 at gmail dot com
Description:
------------
I have a website (well, it's not mine actually) that gets about 1k requests per minute.

Sometimes I upload updated PHP files. Obviously this means I'm overwriting files at the very same time they are being executed. I expect that to cause errors (experienced by the users) during the short period of time while files are being overwritten (usually a few seconds).

However, what I wouldn't expect and happens VERY often is that: after that, general performance degrades, meaning that the execution of EVERY SINGLE script in every single request is significantly slower. Sometimes it's just a few dozens % increase in execution time, sometimes it explodes to the point that the whole websites goes down.

Restarting apache returns performance to normal every time, whether there had been a total collapse or just a degradation in performance.

Expected result:
----------------
When you overwrite PHP files in a website that is being visited by thousands of people, you'd expect a small percentage of them to experience random errors, if a file is being overwritten just when it's being executing.

However, as soon as all the requests that were being handled are done, everything should go back to working normally, and there shoul be no need to restart Apache

Actual result:
--------------
See description.

Seriously, this is fucking pathetic.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2018-04-27 19:18 UTC] teo8976 at gmail dot com
Here's some example performance data (oh come on, I can't attach a screenshot??) https://ibb.co/h3J6bx

This is avarage execution time (measured on the server side) of all requests, but graphs for any given single url would show exactly the same pattern.
 [2018-04-27 19:44 UTC] nikic@php.net
-Status: Open +Status: Feedback
 [2018-04-27 19:44 UTC] nikic@php.net
Please check whether the deployment of new files triggered an opcache restart (because the opcache SHM memory limit was reached). Generally any slowdown should only be temporary though.

In any case, PHP 5 is no longer supported and PHP 7 has quite a lot of changes to improve performance, including in opcache. Additionally, the second-level file cache can be used to reduce the performance impact of opcache restarts.
 [2018-04-28 14:54 UTC] teo8976 at gmail dot com
> Please check whether the deployment 
> of new files triggered an opcache restart

How can I check that?

> Generally any slowdown should only be temporary though

It isn't. When this happens, the slowdown is permanent until I restart apache.
In the screenshot I attached (where I unintentionally cut off the part that showed the times), it went on for several hours. It was also surprisingly "stable", everything being consistently X% slower (of course it is possible that the slowdown was affecting a specific piece of code being called by all scripts)

> In any case, PHP 5 is no longer supported

http://php.net/supported-versions.php says "security support until 31 december" this year. Something that can cause a website to go down permanently because of a performance degradation with no good reason, I would call a security issue.
 [2018-04-28 15:00 UTC] spam2 at rhsoft dot net
>> Please check whether the deployment 
>> of new files triggered an opcache restart
>>
> How can I check that?

look at the opcache vars and in doubt feed "opcache status panel" into Google and ask yourself why you don't have such tool installed - not knowing what a server does would i personally call a security issue (and no such a panel out-of-the-box is no good idea because *that* would be a security issue)

> Something that can cause a website to go down 
> permanently because of a performance degradation 
> with no good reason, I would call a security issue

nosense - following your logic running a server without opcache would be insecure at all
 [2018-04-28 15:13 UTC] teo8976 at gmail dot com
-Status: Feedback +Status: Open
 [2018-04-28 15:13 UTC] teo8976 at gmail dot com
> nosense - following your logic running a server without 
> opcache would be insecure at all

No, that is not following my logic. I meant something that can cause a SUDDEN crash of a website just because you do something as trivial as upload some files. I thought "security support" implicitly covered fixes for bugs so serious as to cause a system to go completely down triggered by trivial events.

I know *strictly speaking* this is not security related, but I thought in the context of support version lifecycle "security" was short for "is secure and at least barely usable".
 [2018-04-28 15:20 UTC] spam2 at rhsoft dot net
the problem is pretty sure that your "opcache.memory_consumption" setting is too low and you are trahsing opcache with a lot of changes - so either increase you config or reload the webserver after chaning lot of files

the most lkiley problem is that your configuration don't suite your workload and since you even don't know how to look pretty clear says you never gave a damn about your configs
 [2018-04-28 15:27 UTC] teo8976 at gmail dot com
> the most lkiley problem is that your configuration don't suite your workload

Let's assume this is the case.

That would explain a TEMPORARY degradation in performance.

Even if that is justified by a bad configuration which is my fault, if that doesn't recover on its own after a while that no file is being changed, then there's a bug.

> pretty clear says you never gave a damn about your configs

Indeed, I never did. But again, if I need to worry about fine-tuning my configs just for PHP to be stable, then it's ill-designed.
 
PHP Copyright © 2001-2018 The PHP Group
All rights reserved.
Last updated: Tue Oct 16 20:01:27 2018 UTC