php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #73183 Segfaults / zend_mm_heap corrupted
Submitted: 2016-09-27 12:13 UTC Modified: 2021-06-09 15:00 UTC
Votes:10
Avg. Score:4.5 ± 0.8
Reproduced:8 of 8 (100.0%)
Same Version:4 (50.0%)
Same OS:2 (25.0%)
From: schnederle at futureweb dot at Assigned: cmb (profile)
Status: Closed Package: Reproducible crash
PHP Version: 7.0.11 OS: Centos 7.2.1511
Private report: No CVE-ID: None
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: schnederle at futureweb dot at
New email:
PHP Version: OS:

 

 [2016-09-27 12:13 UTC] schnederle at futureweb dot at
Description:
------------
Hello,

since we switched from PHP 5.3 to PHP 7.0.x (tried multiple Versions of PHP 7 - now we are on 7.0.11) we get random segfaults / zend_mm_heap corrupted with our CMS.

Unfortunately not predictable when they will occur or on which Script. (unfortunately never the same)
Sometimes a few Days no problems - sometimes 5 times a Day ...

Not 100% sure - but I believe that opcache could be the cause of the Problem - as a simple "opcache_reset();" is enough to get everything up & running again - no complete restart of Apache needed.

Error Log / Backtrace: http://temp.in.futureweb.at/php/error.log 

Hope someone can help me on this ...

Thank you
Andreas Schnederle-Wagner



Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2016-09-28 03:39 UTC] laruence@php.net
you may try to set opcache.protect_memory=1 (please note this will significantly slowdown the server).  and see if there is any unexpected write to shared memory
 [2016-09-28 08:01 UTC] schnederle at futureweb dot at
alright - I activated protect_memory and will report back.

Will I see Errors in the Logs when there are any unexpected writes to shared memory? (unfortunately there is not much documentation available what protect_memory excactly does)
 [2016-09-28 11:25 UTC] schnederle at futureweb dot at
again Segfaults with opcache.protect_memory=1:

[Wed Sep 28 13:22:25.387677 2016] [core:notice] [pid 25800] AH00052: child pid 4680 exit signal Segmentation fault (11)
...
[Wed Sep 28 13:22:46.415573 2016] [core:notice] [pid 25800] AH00052: child pid 4885 exit signal Segmentation fault (11)
 [2016-09-28 15:31 UTC] laruence@php.net
paste the backtrace out (coredump)
 [2016-09-29 08:46 UTC] schnederle at futureweb dot at
Please find BT here: http://temp.in.futureweb.at/php/core.30883.bt.txt
 [2016-09-29 12:52 UTC] schnederle at futureweb dot at
another segfault - BT:

http://temp.in.futureweb.at/php/core.23374.bt.txt
http://temp.in.futureweb.at/php/core.25625.bt.txt
 [2016-09-30 01:42 UTC] laruence@php.net
is this the coredump after you setting opcache.protect_memory?
 [2016-09-30 11:26 UTC] schnederle at futureweb dot at
They were done with opcache.protect_memory=0

This one is with opcache.protect_memory=1 
http://temp.in.futureweb.at/php/core.17683.bt.txt
 [2016-09-30 11:40 UTC] schnederle at futureweb dot at
In case it helps - here a "bt full" with opcache.protect_memory=1: http://temp.in.futureweb.at/php/core.17683.bt_full.txt
 [2016-09-30 19:40 UTC] schnederle at futureweb dot at
again segfault with opcache.protect_memory=1: 
- http://www.futureweb.at/userdata/115/temp/core.bt.29305
- http://www.futureweb.at/userdata/115/temp/core.bt.29468

bt full with opcache.protect_memory=1:
- http://www.futureweb.at/userdata/115/temp/core.bt.full.29305
 [2016-10-04 10:54 UTC] schnederle at futureweb dot at
Don't know if it helps ... some more Segfault BTs:

02.10.2016: http://temp.in.futureweb.at/php/core.25917.bt.txt
03.10.2016: http://temp.in.futureweb.at/php/core.20048.bt.txt
04.10.2016: http://temp.in.futureweb.at/php/core.27118.bt.txt
 [2016-10-20 12:43 UTC] cadonline at gmx dot net
We are facing similar Problems on our Servers - Did you already find a Solution Andreas?

thx
Markus
 [2016-10-28 08:30 UTC] schnederle at futureweb dot at
Unfortunately no News yet ... same Problems occur with 7.0.12
Still hoping for some News on this Topic from PHP Guys! ;-)
 [2016-11-23 08:58 UTC] schnederle at futureweb dot at
still no luck with 7.0.13!

Maybe this additional Information can help?

Followed by the Segfaults I often see those Kind of Errors within Logs:

----------------------------------
error_log-20161116:[Tue Nov 15 14:35:52.478805 2016] [:error] [pid 30926] [client 80.110.0.0:54073] PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 140380662153216 bytes) in /var/www/path/html/templates/path/php/news_nuggets.php on line 218, referer: http://domain

----------------------------------

216 Alle
217 </div>
218 <div class="referenzen_cst_filter_right_entry<?php if(isset($_GET['filter']) && $_GET['filter'] == 2) { ?> active<?php } ?>" id="referenzen_entry_2" rel="2">
219 Tourismus
220 </div>
----------------------------------

As one can see - there is nothing which could take 127TB of Memory on Line 218 ... could it be that OpCache is trying to Output complete nonsense?
 [2016-12-01 16:23 UTC] schnederle at futureweb dot at
Some new Information - Today I set OPCache Log Level to Debug ... no Segfault so far - but some other weird Errors - maybe somehow related to the Segfaults Problem?

At first some of those:
< Thu Dec 1 11:44:34 2016 (12618): Warning Interned string buffer overflow
< Thu Dec 1 11:44:34 2016 (12618): Warning Interned string buffer overflow
< ...
< ...
< ...
< Thu Dec 1 11:44:34 2016 (12618): Warning Interned string buffer overflow
< Thu Dec 1 11:44:34 2016 (12618): Warning Interned string buffer overflow

Right after those Errors - some OOM Errors occured:
< [Thu Dec 01 11:45:52.556057 2016] [:error] [pid 11958] [client 80.110.0.0:50912] PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 6935160699037703352 bytes) in /var/www/path/html/includes/path1/file.php on line 38, referer: http://www.domain.at/admin/menue/fil...111&id_usr=111

Line 38 of the File where OOM occured:
< if (!function_exists("get_galleries")) {

Did increase opcache.interned_strings_buffer from 8 to 16 now ... but there shouldn't be such OOM Troubles when reaching Limit I guess ... (not to speak about 'maybe' Segfaults ...)
 [2016-12-12 14:28 UTC] schnederle at futureweb dot at
Since I increased opcache.interned_strings_buffer from 8 to 16 there was no single SEGFAULT anymore ... guess the Problem must lie somewhere within Interned Strings Buffer Logic then?
 [2016-12-19 12:01 UTC] schnederle at futureweb dot at
I can confirm that raising opcache.interned_strings_buffer from 8 to 16 MB did solve that Segfault issues! ~3 Weeks without a single Segfault ...

Actual OPCache Stats - interned_strings_usage - more than the initial 8MB are used ...

----------------------------------------------------
Array
(
    [opcache_enabled] => 1
    [file_cache] => /var/www/ortsinfo/opcache
    [cache_full] => 
    [restart_pending] => 
    [restart_in_progress] => 
    [memory_usage] => Array
        (
            [used_memory] => 86297760
            [free_memory] => 47915104
            [wasted_memory] => 4864
            [current_wasted_percentage] => 0.0036239624023438
        )

    [interned_strings_usage] => Array
        (
            [buffer_size] => 16777216
            [used_memory] => 8840536
            [free_memory] => 7936680
            [number_of_strings] => 74538
        )

    [opcache_statistics] => Array
        (
            [num_cached_scripts] => 4397
            [num_cached_keys] => 12811
            [max_cached_keys] => 16229
            [hits] => 15118014
            [start_time] => 1482122407
            [last_restart_time] => 0
            [oom_restarts] => 0
            [hash_restarts] => 0
            [manual_restarts] => 0
            [misses] => 7
            [blacklist_misses] => 0
            [blacklist_miss_ratio] => 0
            [opcache_hit_rate] => 99.999953697643
        )

)
----------------------------------------------------

Andreas Schnederle-Wagner
 [2018-06-28 15:06 UTC] t dot motylewski at gmail dot com
similar issues:
https://bugs.php.net/bug.php?id=73598
https://bugs.php.net/bug.php?id=65590
https://bugs.php.net/bug.php?id=66569
 [2018-06-28 15:23 UTC] t dot motylewski at gmail dot com
and a few more:
https://bugs.php.net/bug.php?id=69251
https://bugs.php.net/bug.php?id=59191
 [2021-06-09 14:47 UTC] cmb@php.net
-Status: Open +Status: Feedback -Assigned To: +Assigned To: cmb
 [2021-06-09 14:47 UTC] cmb@php.net
Is this still an issue with any of the actively supported PHP
versions[1]?

[1] <https://www.php.net/supported-versions.php>
 [2021-06-09 14:57 UTC] schnederle at futureweb dot at
-Status: Feedback +Status: Assigned
 [2021-06-09 14:57 UTC] schnederle at futureweb dot at
@cmb - unfortunately I cannot test it on the affected system. The system is live & productive, I can't do a deliberately induced downtime here. Since the segfaults occurred sporadically and unfortunately I could not find a testcase with which I can trigger the segfault, unfortunately I currently have no way to test it.
The affected CMS is currently running on PHP 7.2.34, but with the option "opcache.interned_strings_buffer = 16" which also fixed the problem with V7.0.x ...
best regards
A.
 [2021-06-09 15:00 UTC] cmb@php.net
-Status: Assigned +Status: Closed
 [2021-06-09 15:00 UTC] cmb@php.net
The point is that prior to PHP 7.2.0, there have been known issues
regarding interned strings.  I assume that this issue has been
fixed, so I'm closing.  If you happen to reproduce it with any
actively supported PHP version, feel free to reopen or file a new
ticket.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Nov 21 18:01:29 2024 UTC