php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #74833 Random crashes, seemingly related to DateTimeImmutable and mod_php
Submitted: 2017-06-29 22:12 UTC Modified: 2017-08-14 10:22 UTC
Votes:2
Avg. Score:4.5 ± 0.5
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:1 (100.0%)
From: mike dot fellows at shaw dot ca Assigned: ab
Status: Closed Package: Apache2 related
PHP Version: 7.0.1 to 7.1.6 OS: Windows 7 64 bit, MacOs, Linux
Private report: No CVE-ID:
 [2017-06-29 22:12 UTC] mike dot fellows at shaw dot ca
Description:
------------
The CakePHP project includes a class called Chronos that extends DateTimeImmutable.  It seems to cause occasional random crashes when run through Apache and mod_php.  In all cases described below the same PHP software would run flawlessly most of the time, with occasional random crashes (no changes to the PHP software between everything being OK and crashing).

The crashes have inconsistent symptoms and I have not been able to capture a backtrace.  They are possibly caused by instantiation of the Chronos class, although this is hard to prove.  Symptoms include:

- Complaints about the Chronos class containing abstract methods (when it doesn't).  Once this symptom appears it would continue in this state on page reload.  Eventually Apache usually crashes given enough page reloads, sometimes witht the zend_mm_heap corrupted mentioned below, somtimes not.
- Sometimes a "zend_mm_heap corrupted" message is placed in the log file followed by a crash and Apache being restared.
- Sometimes Apache crashes with no log messages.

In all cases restarted Apache would "fix" the problem page for awhile.

I've run the same software using PHP's built in web server and not observed any crashes.

I connected Visual Studio 2015's debugger to the Apache process handling the request - but the symptoms stopped once the debugger was connected.

Any suggestions for capturing backtraces would be welcome BTW.  I'm currently running Apache from Apache Lounge and PHP from php.net.

There is copious amounts of information about this bug found here:

https://github.com/cakephp/chronos/issues/147

It seems to have been reported with PHP version 7.0 and up.  My experience has been on Windows 7, but others have reported similar symptoms with Linux intalls (again with mod_php).

Test script:
---------------
Code can be inspected here for the Class:

https://github.com/cakephp/chronos/blob/master/src/Chronos.php

and here for its interface:

https://github.com/cakephp/chronos/blob/master/src/ChronosInterface.php


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2017-06-29 22:25 UTC] mike dot fellows at shaw dot ca
I just re-reviewed the linked Chronos issue and realized that one user has seen this with Apache and suphp.  So the problem is not exclusive to mod_php.
 [2017-06-30 18:32 UTC] mike dot fellows at shaw dot ca
-Operating System: Windows 7 64 bit +Operating System: Windows 7 64 bit, MacOs, Linux -PHP Version: 7.1.6 +PHP Version: 7.0.1 to 7.1.6
 [2017-06-30 18:32 UTC] mike dot fellows at shaw dot ca
Re-examining the Chronos issue linked and just improved the description of the PHP versions and OSes impacted.

Please note that despite the Chronos issue being opened a couple of days ago, the original CakePHP issue was opened almost a year ago.  This has been a problem for some time.
 [2017-06-30 22:58 UTC] mike dot fellows at shaw dot ca
I have managed to gather a Call Stack when Apache crashes.  Here are the messages from the Apache error log immediately before the crash:

[Fri Jun 30 15:44:18.309323 2017] [php7:error] [pid 3444:tid 2756] [client ::1:10998] PHP Fatal error:  Class Cake\\Chronos\\MutableDateTime contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (Cake\\Chronos\\ChronosInterface::__wakeup) in C:\\genesis\\vendor\\cakephp\\chronos\\src\\MutableDateTime.php on line 51, referer: <snip>
[Fri Jun 30 15:44:18.310323 2017] [php7:error] [pid 3444:tid 2764] [client ::1:10995] PHP Fatal error:  Class Cake\\Chronos\\MutableDateTime contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (Cake\\Chronos\\ChronosInterface::__wakeup) in C:\\genesis\\vendor\\cakephp\\chronos\\src\\MutableDateTime.php on line 51, referer: <snip>
[Fri Jun 30 15:44:18.328325 2017] [php7:error] [pid 3444:tid 2772] [client ::1:10997] PHP Fatal error:  Class Cake\\Chronos\\MutableDate contains 2 abstract methods and must therefore be declared abstract or implement the remaining methods (Cake\\Chronos\\ChronosInterface::getTimezone, Cake\\Chronos\\ChronosInterface::__wakeup) in C:\\genesis\\vendor\\cakephp\\chronos\\src\\MutableDate.php on line 50, referer: <snip>



Here is the Call Stack from Visual Studio:

ntdll.dll!RtlSizeHeap()	Unknown
AcXtrnal.dll!NS_FaultTolerantHeap::APIHook_RtlFreeHeap(void *,unsigned long,void *)	Unknown
ucrtbase.dll!free()	Unknown
php7ts.dll!zend_hash_reverse_apply(_zend_array * ht, int(*)(_zval_struct *) apply_func) Line 1603	C
php7ts.dll!shutdown_executor() Line 376	C
php7ts.dll!zend_deactivate() Line 1001	C
php7ts.dll!php_request_shutdown(void * dummy) Line 1880	C
php7apache2_4.dll!php_handler(request_rec * r) Line 725	C
libhttpd.dll!000000007475e9e5()	Unknown
libhttpd.dll!000000007475d7e0()	Unknown
libhttpd.dll!000000007475a1e6()	Unknown
mod_rewrite.so!00000000743753c8()	Unknown
libhttpd.dll!000000007475e9e5()	Unknown
libhttpd.dll!000000007475d7e0()	Unknown
libhttpd.dll!000000007475a49a()	Unknown
libhttpd.dll!000000007475a527()	Unknown
libhttpd.dll!0000000074753b73()	Unknown
libhttpd.dll!0000000074761885()	Unknown
libhttpd.dll!0000000074761675()	Unknown
libhttpd.dll!0000000074771734()	Unknown
[External Code]
 [2017-06-30 23:09 UTC] mike dot fellows at shaw dot ca
FWIW here is the Visual Studio:

The thread 0x346c has exited with code 0 (0x0).
The thread 0x359c has exited with code 0 (0x0).
The thread 0x804 has exited with code 0 (0x0).
'httpd.exe' (Win32): Unloaded 'C:\Windows\System32\security.dll'
Unhandled exception at 0x0000000077346789 (ntdll.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x00000AEE2AF79F18.

The thread 0x37e0 has exited with code 0 (0x0).
'httpd.exe' (Win32): Unloaded 'C:\Windows\System32\1033\msodbcsqlr13.rll'
Exception thrown at 0x0000000077346789 (ntdll.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x00000AEE2AF79F18.

Unhandled exception at 0x0000000077346789 (ntdll.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x00000AEE2AF79F18.

Exception thrown at 0x0000000077346789 (ntdll.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x00000AEE2AF79F18.

Unhandled exception at 0x0000000077346789 (ntdll.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x00000AEE2AF79F18.

Exception thrown at 0x0000000077346789 (ntdll.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x00000AEE2AF79F18.

... (repeating many times)
 [2017-06-30 23:34 UTC] nikic@php.net
Are you using opcache? Are you using ZTS builds?

If the answers are no and yes, this sounds like it might be a persistent string corruption issue, in which case you should check whether using a PHP 7.2 alpha resolves the issue.
 [2017-07-01 00:56 UTC] mike dot fellows at shaw dot ca
Thanks for taking time to look and comment on this problem.

The PHP_ZTS constant is set to 1 (so yes to ZTS build?).  It is the thread safe windows build from php.net.

I am not setting any opcache configuration items in PHP.ini, so if I understand the docs correctly that means I am getting opcache.enable = 1 by default?  Please correct me if I'm wrong on this.

So yes to ZTS and yes to opcache enabled (I think).

My database driver (SQL server) does not support PHP 7.2 yet so I'm not sure if I can test 7.2 or not.  I will try.

Will disabling opcache with 7.1 fix the problem as well?
 [2017-07-01 08:54 UTC] nikic@php.net
-Assigned To: +Assigned To: ab
 [2017-07-01 08:54 UTC] nikic@php.net
Opcache is not loaded by default, so if you did not explicitly enable the extension, you're not using opcache (though you can double check using e.g. phpinfo()).

@ab: Can you please check whether this looks like a persistent string issue to you that will be resolved by 7.2?
 [2017-07-01 16:44 UTC] ab@php.net
-Status: Assigned +Status: Feedback
 [2017-07-01 16:44 UTC] ab@php.net
@nikic, looks pretty much like the topic. TS SAPI, sporadic fails, very much like in many other bugs.

@mike dot fellows at shaw dot ca which exact extension do you need to be present to test your code under 7.2? If it builds with 7.2, i can provide it so you can have the exact constellation. It'd be great also if you could provide a backtrace, even a minidump from WER, etc.

Thanks.
 [2017-07-01 16:47 UTC] ab@php.net
Ah, i see i've overseen the backtraces already there. Yeah, the part like this

AcXtrnal.dll!NS_FaultTolerantHeap::APIHook_RtlFreeHeap(void *,unsigned long,void *)	Unknown
ucrtbase.dll!free()	Unknown
php7ts.dll!zend_hash_reverse_apply(_zend_array * ht, int(*)(_zval_struct *) apply_func) Line 1603	C
php7ts.dll!shutdown_executor() Line 376	C
php7ts.dll!zend_deactivate() Line 1001	C
php7ts.dll!php_request_shutdown(void * dummy) Line 1880	C
php7apache2_4.dll!php_handler(request_rec * r) Line 725	C

looks very much like the issue already fixed in 7.2. No further backtrace needed.

Thanks.
 [2017-07-01 18:06 UTC] mike dot fellows at shaw dot ca
@nikic definitely not using opcache then as nothing was showing in php_info, and I have not enabled it.

I can capture more backtraces if you change your minds, it is not a big deal to do.

Are they any recommendations for avoiding this problem in 7.1?  Or is the only option to use 7.2?

For 7.2 The only extension I would need beyond the standard PHP ones is the Microsoft SQL Server driver for PHP:

https://github.com/Microsoft/msphpsql

No 7.2 support yet (https://github.com/Microsoft/msphpsql/issues/444).  I can't imagine that much has changed between 7.1 to 7.2 though.  I'll see about building it next week for 7.2 and see what happens.

I'm not really in a jam with this yet, but I will need to have some kind of workaround ready in about 8-10 weeks - I think a 7.2 PHP beta would be fine.  Right now this problem is just a (very minor) development headache.

Thx again for your help with this.
 [2017-07-01 19:39 UTC] ab@php.net
The repo linked contains at least two exts, pdo_sqlsrv and sqlsrv. Which one do you use? Also, which flavour, x64, x86, etc. These exts are on PECL, so some latest release could be built. Right now it looks like it's indeed the known issue already fixed in 7.2, so it would be great to have confirmed.

For 7.1 - as long as it's mod_winnt + PHP, there's no sublime workaround :( Moving to FCGI might show some more stability.

Thanks.
 [2017-07-02 03:10 UTC] mike dot fellows at shaw dot ca
-Status: Feedback +Status: Assigned
 [2017-07-02 03:10 UTC] mike dot fellows at shaw dot ca
I'm using the 64 bit thread safe pdo_sqlsrv extension.

I'm happy to test as much as I can, be great if you think you can build the extension.  But I can do it as well.

I won't be able to start testing until Tuesday at the earliest though.

Cheers.
 [2017-07-04 17:04 UTC] mike dot fellows at shaw dot ca
@ab thanks for build the MS SQL server extension.  Unfortunately I get a 401 error when I try to download it (Access is denied due to invalid credentials).  Not sure how to get around that one.
 [2017-07-04 17:11 UTC] ab@php.net
I've just unpacked it here http://windows.php.net/downloads/snaps/ostc/74833/pdo_sqlsrv/ts/vc15/x64/ , could you please check? Seems that's the server config that doesn't like it for some reason.

Thanks.
 [2017-07-04 17:13 UTC] ab@php.net
Oh, or the zip. I think I was just able to fix the permissions.

Thanks.
 [2017-07-04 17:17 UTC] mike dot fellows at shaw dot ca
Got it, thanks.  I will let you know how the testing goes in the next few days.
 [2017-07-06 17:15 UTC] mike dot fellows at shaw dot ca
@ab I've been testing php7.2.0 alpha2 for the last two days.  I was not able to test it as much as I'd like because I've been fixing application bugs due to 7.2 changes.  There were no crashes for the last two days.  But this morning I did have a crash.

Here is the error output from Visual Studio:

The thread 0x28d8 has exited with code 0 (0x0).
The thread 0xa50 has exited with code 0 (0x0).
The thread 0x1668 has exited with code 0 (0x0).
Unhandled exception at 0x000007FED85F2A44 (php7ts.dll) in httpd.exe: 0xC0000005: Access violation reading location 0x0000000000000000.


And here is the Call Stack:

php7ts.dll!zif_get_defined_constants(_zend_execute_data * execute_data, _zval_struct * return_value) Line 2103	C
php7ts.dll!ZEND_DO_FCALL_BY_NAME_SPEC_RETVAL_USED_HANDLER(_zend_execute_data * execute_data) Line 827	C
php7ts.dll!execute_ex(_zend_execute_data * ex) Line 59218	C
php7ts.dll!zend_execute(_zend_op_array * op_array, _zval_struct * return_value) Line 63244	C
php7ts.dll!zend_execute_scripts(int type, _zval_struct * retval, int file_count, ...) Line 1538	C
php7ts.dll!php_execute_script(_zend_file_handle * primary_file) Line 2548	C
php7apache2_4.dll!php_handler(request_rec * r) Line 702	C
libhttpd.dll!000000007422e9e5()	Unknown
libhttpd.dll!000000007422d7e0()	Unknown
libhttpd.dll!000000007422a1e6()	Unknown
mod_rewrite.so!0000000073ed53c8()	Unknown
libhttpd.dll!000000007422e9e5()	Unknown
libhttpd.dll!000000007422d7e0()	Unknown
libhttpd.dll!000000007422a49a()	Unknown
libhttpd.dll!000000007422a527()	Unknown
libhttpd.dll!0000000074223b73()	Unknown
libhttpd.dll!0000000074231885()	Unknown
libhttpd.dll!0000000074231675()	Unknown
libhttpd.dll!0000000074241734()	Unknown
[External Code]


I notice that alpha 3 was released today.  I will upgrade to that release and continue with my testing.
 [2017-07-07 13:28 UTC] ab@php.net
Thanks for that. So there might be still some leftover. Did your system generate a minidump? Or, maybe you've found some better way to repro?

On your side, i would try to make some stress test while using get_defined_constants(). Like use ab with high concurrency. With or without your app, with core only and with full set of the exts you use. While the last backtrace is different, it could be still some leftover not fixed, or something completely different. Lets see for further info.

Thanks.
 [2017-07-12 18:04 UTC] mike dot fellows at shaw dot ca
I've been running the 7.2 alpha3 release for almost a week now without any crashes.  I only saw the one crash in alpha2 I noted in my comment above.

I have not been doing any specific testing to try to trigger the bug.  Just the normal development work with CakePHP, including it's Chronos package, which was triggering crashes regularly in my windows environment with 7.1.  Before I changed to 7.2 alpha2/3 I was seeing about 10-15 crashes a day.  So a very big improvement with the 7.2 alpha releases.

I will continue to work with 7.2 and report any new crashes when/if they happen.  I can also create a full dump from SQL studio if any occur.
 [2017-07-17 17:44 UTC] mike dot fellows at shaw dot ca
I've seen another crash, this one with 7.2 alpha 3.  Here is the Call Stack from Visual Studio:

php7ts.dll!zif_get_defined_constants(_zend_execute_data * execute_data, _zval_struct * return_value) Line 2102	C
php7ts.dll!ZEND_DO_FCALL_BY_NAME_SPEC_RETVAL_USED_HANDLER(_zend_execute_data * execute_data) Line 827	C
php7ts.dll!execute_ex(_zend_execute_data * ex) Line 59085	C
php7ts.dll!zend_execute(_zend_op_array * op_array, _zval_struct * return_value) Line 63117	C
php7ts.dll!zend_execute_scripts(int type, _zval_struct * retval, int file_count, ...) Line 1491	C
php7ts.dll!php_execute_script(_zend_file_handle * primary_file) Line 2550	C
php7apache2_4.dll!php_handler(request_rec * r) Line 702	C
libhttpd.dll!00000000549ae9e5()	Unknown
libhttpd.dll!00000000549ad7e0()	Unknown
libhttpd.dll!00000000549aa1e6()	Unknown
mod_rewrite.so!000000006f3353c8()	Unknown
libhttpd.dll!00000000549ae9e5()	Unknown
libhttpd.dll!00000000549ad7e0()	Unknown
libhttpd.dll!00000000549aa49a()	Unknown
libhttpd.dll!00000000549aa527()	Unknown
libhttpd.dll!00000000549a3b73()	Unknown
libhttpd.dll!00000000549b1885()	Unknown
libhttpd.dll!00000000549b1675()	Unknown
libhttpd.dll!00000000549c1734()	Unknown
[External Code]

Same as the previous one it seems.

I have saved a mini dump with the heap. The file is about 160 MBytes.  @ab if you would like the dump file does bugs.php.net have a way to upload the file somehow?
 [2017-07-17 17:49 UTC] mike dot fellows at shaw dot ca
I also have just the mini-dump.  It is about 1 MByte in size.
 [2017-07-17 17:54 UTC] ab@php.net
Thanks for staying on this. Nope, bugs.php.net doesn't have this facilities. But a couple of days ago I've seen people sharing files through https://www.transfernow.net/en/ , any other service like that will suffice. The full dump were of course great. Please ensure before, that it doesn't contain any sensitive data.

So basically it's get_defined_constants() that has some issues. It would still make sense to check a reproduce possibility. As it might be something simple, or something that depends on the app code or the extensions constellation, etc. Anyway, a dump is a good start already.

Thanks.
 [2017-07-17 19:12 UTC] mike dot fellows at shaw dot ca
@ab, I have emailed you a link to the minidump with heap.
 [2017-07-19 20:29 UTC] ab@php.net
@mike thanks for the crash dump. I'll get back to you with some findings after investigating on it.

Thanks.
 [2017-07-26 10:58 UTC] ab@php.net
Finally i got to the point on this. After the investigation, I don't see any issues with the string interning. I'm going to push a patch soon. Otherwise, you might have luck with reproducing on alpha3 with the code like this

session_start();
session_regenerate_id();
session_regenerate_id();
get_defined_constants(true);


I'm pretty sure the issue lays on the SID constant handling, but i don't reproduce it on my environment. I guess it depends very much on your application logic and ini, maybe some system specific factor, too. So you might luck reproducing it either by stress testing the snippet above or even better by analyzing your application and paying attention to the parts where session_regenerate_id() is used.

Thanks.
 [2017-07-26 11:22 UTC] ab@php.net
Automatic comment on behalf of ab
Revision: http://git.php.net/?p=php-src.git;a=commit;h=bd00fe81cc4525b7d8e5c36a3844ee2254e5f69c
Log: Fixed bug #74833, SID constant created with wrong module number
 [2017-07-26 11:22 UTC] ab@php.net
-Status: Assigned +Status: Closed
 [2017-07-26 17:27 UTC] mike dot fellows at shaw dot ca
I've inspected my code and am only finding calls to session_start(), no session_regenerate_id().  I'll continue to test with the newer releases of PHP7.2 and report back if the issue re-occurs.

Glad to hear the second set of crashes are not due to the string interning problem.  Thanks for looking into all of this @ab.
 [2017-08-11 16:04 UTC] mike dot fellows at shaw dot ca
@ab - just a final follow up.  I have been using 7.2 beta2 for the last week.

I have not seen the session related crash once.  With alpha3, I was seeing it 3 or 4 times a week.  So from my point of view the issue is fixed.

Also, I've seen no occurences of the original persistent string corruption issue.  As was the case with alpha2/3 of course.

Thanks to all for sorting these issues out.
 [2017-08-14 10:22 UTC] ab@php.net
Thanks for the continuous checks, @mike :)
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Tue Aug 29 15:01:52 2017 UTC