php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #70002 apache2 / php5ts crashes, sometimes reporting zend_mm_heap corrupted
Submitted: 2015-07-06 16:34 UTC Modified: 2016-07-04 15:24 UTC
Votes:2
Avg. Score:4.5 ± 0.5
Reproduced:2 of 2 (100.0%)
Same Version:2 (100.0%)
Same OS:0 (0.0%)
From: jeremy dot j dot dunn at gmail dot com Assigned: kaplan
Status: Closed Package: Apache2 related
PHP Version: 5.5.26 OS: Windows 2008 R2
Private report: No CVE-ID: 2015-8878
 [2015-07-06 16:34 UTC] jeremy dot j dot dunn at gmail dot com
Description:
------------
Windows 2008 R2; apache 2.4.12 (32-bit) from apachelounge.
was running 5.5.24/32-bit just fine.
upgraded to 5.5.26/32-bit, and immediately started getting errors like this:

Faulting application name: httpd.exe, version: 2.4.12.0, time stamp: 0x54c90386
Faulting module name: php5ts.dll, version: 5.5.26.0, time stamp: 0x5578acb1
Exception code: 0xc0000005
Fault offset: 0x003ce564
Faulting process id: 0x16f0
Faulting application start time: 0x01d0b773c098bfd0
Faulting application path: C:\Program Files (x86)\Apache2.4\bin\httpd.exe
Faulting module path: C:\Program Files (x86)\PHP5.5\php5ts.dll

The crash occurred about 40 times in one hour.  

Sometimes, but not always, the apache error log contained:
  zend_mm_heap corrupted

Reverting to 5.5.24 completely removed the problem. 

Unfortunately, I cannot recreate the program in a development or test environment; and I cannot afford to cause the problem in production again.  Therefor I cannot provide a backtrace.

The problem seemed mainly to occur when generating large PDF files, which in our app involves merging PDFs.

We are using:
 * SQL Server 2008 R2
 * ImageMagick 6.7.9-Q8
 * WkHtmlToPDF 0.12.2.1
 * ghostscript 9.14 (64-bit)
 * PDFtk 2.02

PHP modules:
 * PHP/PDO driver for SQL Server (v3.0.2, 32-bit)
 * igbinary 
 * xcache (v3.2.0)

If there is any way I can provide more information, short of causing the problem to occur in production, I would be very happy to do so.



Patches

temp_dir_ts_4 (last revision 2015-07-28 14:40 UTC) by ab@php.net)
temp_dir_ts_3 (last revision 2015-07-28 08:51 UTC) by ab@php.net)
temp_dir_ts_2 (last revision 2015-07-28 08:40 UTC) by ab@php.net)
temporary_dir_ts (last revision 2015-07-27 16:28 UTC) by ab@php.net)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2015-07-07 13:52 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-07-07 13:52 UTC] ab@php.net
Thanks for the report. Yes, something obviously goes wrong, but we still need something concrete about the crash itself.

Clear, you don't want to let this into production. Maybe you could check whether some crash dumps stayed on the production machine? Or, maybe you can use some HTTP traffic recorder and then replay it on a dev machine? Maybe that can reproduce it. Also, in that case you probably could play with your PHP code to figure out reproduce snippet.

Thanks.
 [2015-07-07 17:38 UTC] jeremy dot j dot dunn at gmail dot com
-Status: Feedback +Status: Open
 [2015-07-07 17:38 UTC] jeremy dot j dot dunn at gmail dot com
there are 50 crash dump files in the /WER/ReportQueue/ (windows error reporting) directory that list all the executables and modules loaded in memory. 

Here is the full contents of one such file:
===========================================
Version=1
EventType=APPCRASH
EventTime=130806084712205860
ReportType=2
Consent=1
ReportIdentifier=2358b32d-2364-11e5-bb27-0050569dd849
IntegratorReportIdentifier=2358b32c-2364-11e5-bb27-0050569dd849
WOW64=1
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=httpd.exe
Sig[1].Name=Application Version
Sig[1].Value=2.4.12.0
Sig[2].Name=Application Timestamp
Sig[2].Value=54c90386
Sig[3].Name=Fault Module Name
Sig[3].Value=php5ts.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=5.5.26.0
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=5578acb1
Sig[6].Name=Exception Code
Sig[6].Value=c0000005
Sig[7].Name=Exception Offset
Sig[7].Value=003ce564
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=6.1.7600.2.0.0.274.10
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=0a9e
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=0a9e372d3b4ad19135b953a78882e789
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=0a9e
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=0a9e372d3b4ad19135b953a78882e789
UI[2]=C:\Program Files (x86)\Apache2.4\bin\httpd.exe
UI[5]=Check online for a solution (recommended)
UI[6]=Check for a solution later (recommended)
UI[7]=Close
UI[8]=Apache HTTP Server stopped working and was closed
UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available.
UI[10]=&Close
LoadedModule[0]=C:\Program Files (x86)\Apache2.4\bin\httpd.exe
LoadedModule[1]=C:\Windows\SysWOW64\ntdll.dll
LoadedModule[2]=C:\Windows\syswow64\kernel32.dll
LoadedModule[3]=C:\Windows\syswow64\KERNELBASE.dll
LoadedModule[4]=C:\Program Files (x86)\Apache2.4\bin\libhttpd.dll
LoadedModule[5]=C:\Program Files (x86)\Apache2.4\bin\pcre.dll
LoadedModule[6]=C:\Windows\system32\MSVCR110.dll
LoadedModule[7]=C:\Windows\syswow64\WS2_32.dll
LoadedModule[8]=C:\Windows\syswow64\msvcrt.dll
LoadedModule[9]=C:\Windows\syswow64\RPCRT4.dll
LoadedModule[10]=C:\Windows\syswow64\SspiCli.dll
LoadedModule[11]=C:\Windows\syswow64\CRYPTBASE.dll
LoadedModule[12]=C:\Windows\SysWOW64\sechost.dll
LoadedModule[13]=C:\Windows\syswow64\NSI.dll
LoadedModule[14]=C:\Windows\syswow64\ADVAPI32.dll
LoadedModule[15]=C:\Program Files (x86)\Apache2.4\bin\libaprutil-1.dll
LoadedModule[16]=C:\Program Files (x86)\Apache2.4\bin\libapriconv-1.dll
LoadedModule[17]=C:\Program Files (x86)\Apache2.4\bin\libapr-1.dll
LoadedModule[18]=C:\Windows\system32\MSWSOCK.dll
LoadedModule[19]=C:\Windows\syswow64\user32.dll
LoadedModule[20]=C:\Windows\syswow64\GDI32.dll
LoadedModule[21]=C:\Windows\syswow64\LPK.dll
LoadedModule[22]=C:\Windows\syswow64\USP10.dll
LoadedModule[23]=C:\Windows\syswow64\SHELL32.dll
LoadedModule[24]=C:\Windows\syswow64\SHLWAPI.dll
LoadedModule[25]=C:\Windows\system32\IMM32.DLL
LoadedModule[26]=C:\Windows\syswow64\MSCTF.dll
LoadedModule[27]=C:\Windows\system32\CRYPTSP.dll
LoadedModule[28]=C:\Windows\system32\rsaenh.dll
LoadedModule[29]=C:\Windows\System32\wship6.dll
LoadedModule[30]=C:\Windows\System32\wshtcpip.dll
LoadedModule[31]=C:\Windows\system32\DNSAPI.dll
LoadedModule[32]=C:\Windows\system32\NLAapi.dll
LoadedModule[33]=C:\Windows\System32\winrnr.dll
LoadedModule[34]=C:\Windows\system32\napinsp.dll
LoadedModule[35]=C:\Windows\system32\IPHLPAPI.DLL
LoadedModule[36]=C:\Windows\system32\WINNSI.DLL
LoadedModule[37]=C:\Windows\system32\rasadhlp.dll
LoadedModule[38]=C:\Windows\System32\fwpuclnt.dll
LoadedModule[39]=C:\Program Files (x86)\Apache2.4\modules\mod_actions.so
LoadedModule[40]=C:\Program Files (x86)\Apache2.4\modules\mod_alias.so
LoadedModule[41]=C:\Program Files (x86)\Apache2.4\modules\mod_asis.so
LoadedModule[42]=C:\Program Files (x86)\Apache2.4\modules\mod_auth_basic.so
LoadedModule[43]=C:\Program Files (x86)\Apache2.4\modules\mod_authn_file.so
LoadedModule[44]=C:\Program Files (x86)\Apache2.4\modules\mod_authz_core.so
LoadedModule[45]=C:\Program Files (x86)\Apache2.4\modules\mod_authz_groupfile.so
LoadedModule[46]=C:\Program Files (x86)\Apache2.4\modules\mod_authz_host.so
LoadedModule[47]=C:\Program Files (x86)\Apache2.4\modules\mod_authz_user.so
LoadedModule[48]=C:\Program Files (x86)\Apache2.4\modules\mod_autoindex.so
LoadedModule[49]=C:\Program Files (x86)\Apache2.4\modules\mod_cgi.so
LoadedModule[50]=C:\Program Files (x86)\Apache2.4\modules\mod_deflate.so
LoadedModule[51]=C:\Program Files (x86)\Apache2.4\bin\zlib1.dll
LoadedModule[52]=C:\Program Files (x86)\Apache2.4\modules\mod_dir.so
LoadedModule[53]=C:\Program Files (x86)\Apache2.4\modules\mod_env.so
LoadedModule[54]=C:\Program Files (x86)\Apache2.4\modules\mod_expires.so
LoadedModule[55]=C:\Program Files (x86)\Apache2.4\modules\mod_filter.so
LoadedModule[56]=C:\Program Files (x86)\Apache2.4\modules\mod_include.so
LoadedModule[57]=C:\Program Files (x86)\Apache2.4\modules\mod_isapi.so
LoadedModule[58]=C:\Program Files (x86)\Apache2.4\modules\mod_log_config.so
LoadedModule[59]=C:\Program Files (x86)\Apache2.4\modules\mod_log_rotate.so
LoadedModule[60]=C:\Program Files (x86)\Apache2.4\modules\mod_mime.so
LoadedModule[61]=C:\Program Files (x86)\Apache2.4\modules\mod_negotiation.so
LoadedModule[62]=C:\Program Files (x86)\Apache2.4\modules\mod_rewrite.so
LoadedModule[63]=C:\Program Files (x86)\Apache2.4\modules\mod_setenvif.so
LoadedModule[64]=C:\Program Files (x86)\Apache2.4\modules\mod_socache_shmcb.so
LoadedModule[65]=C:\Program Files (x86)\Apache2.4\modules\mod_socache_dbm.so
LoadedModule[66]=C:\Program Files (x86)\Apache2.4\modules\mod_ssl.so
LoadedModule[67]=C:\Program Files (x86)\Apache2.4\bin\LIBEAY32.dll
LoadedModule[68]=C:\Program Files (x86)\Apache2.4\bin\SSLEAY32.dll
LoadedModule[69]=C:\Program Files (x86)\Apache2.4\modules\mod_userdir.so
LoadedModule[70]=C:\Program Files (x86)\PHP5.5\php5apache2_4.dll
LoadedModule[71]=C:\Program Files (x86)\PHP5.5\php5ts.dll
LoadedModule[72]=C:\Windows\system32\ODBC32.dll
LoadedModule[73]=C:\Windows\syswow64\ole32.dll
LoadedModule[74]=C:\Windows\system32\odbcint.dll
LoadedModule[75]=C:\Program Files (x86)\PHP5.5\ext\php_pdo_sqlsrv_55_ts.dll
LoadedModule[76]=C:\Windows\system32\MSVCP110.dll
LoadedModule[77]=C:\Program Files (x86)\PHP5.5\ext\php_igbinary.dll
LoadedModule[78]=C:\Program Files (x86)\PHP5.5\ext\php_xcache_3.2.0_5.5.16_ts_VC11.dll
LoadedModule[79]=C:\Program Files (x86)\PHP5.5\ext\php_bz2.dll
LoadedModule[80]=C:\Program Files (x86)\PHP5.5\ext\php_curl.dll
LoadedModule[81]=C:\Program Files (x86)\PHP5.5\libssh2.dll
LoadedModule[82]=C:\Windows\syswow64\WLDAP32.dll
LoadedModule[83]=C:\Windows\syswow64\Normaliz.dll
LoadedModule[84]=C:\Program Files (x86)\PHP5.5\ext\php_fileinfo.dll
LoadedModule[85]=C:\Program Files (x86)\PHP5.5\ext\php_gd2.dll
LoadedModule[86]=C:\Program Files (x86)\PHP5.5\ext\php_imap.dll
LoadedModule[87]=C:\Windows\system32\WINMM.dll
LoadedModule[88]=C:\Windows\system32\Secur32.dll
LoadedModule[89]=C:\Windows\syswow64\CRYPT32.dll
LoadedModule[90]=C:\Windows\syswow64\MSASN1.dll
LoadedModule[91]=C:\Program Files (x86)\PHP5.5\ext\php_mbstring.dll
LoadedModule[92]=C:\Program Files (x86)\PHP5.5\ext\php_pdo_odbc.dll
LoadedModule[93]=C:\Program Files (x86)\PHP5.5\ext\php_soap.dll
LoadedModule[94]=C:\Program Files (x86)\PHP5.5\ext\php_xsl.dll
LoadedModule[95]=C:\Windows\system32\credssp.dll
LoadedModule[96]=C:\Windows\system32\bcrypt.dll
LoadedModule[97]=C:\Windows\SysWOW64\bcryptprimitives.dll
LoadedModule[98]=C:\Windows\SysWOW64\sqlncli11.dll
LoadedModule[99]=C:\Windows\SysWOW64\MSVCR100.dll
LoadedModule[100]=C:\Windows\syswow64\OLEAUT32.dll
LoadedModule[101]=C:\Windows\SysWOW64\VERSION.dll
LoadedModule[102]=C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.7600.16661_none_ebfb56996c72aefc\COMCTL32.dll
LoadedModule[103]=C:\Windows\syswow64\COMDLG32.dll
LoadedModule[104]=C:\Windows\SysWOW64\NETAPI32.dll
LoadedModule[105]=C:\Windows\SysWOW64\netutils.dll
LoadedModule[106]=C:\Windows\SysWOW64\srvcli.dll
LoadedModule[107]=C:\Windows\SysWOW64\wkscli.dll
LoadedModule[108]=C:\Windows\system32\MTXDM.DLL
LoadedModule[109]=C:\Windows\syswow64\CLBCatQ.DLL
LoadedModule[110]=C:\Windows\system32\comsvcs.dll
LoadedModule[111]=C:\Windows\system32\ATL.DLL
LoadedModule[112]=C:\Windows\system32\RpcRtRemote.dll
LoadedModule[113]=C:\Windows\SysWOW64\1033\SQLNCLIR11.RLL
LoadedModule[114]=C:\Windows\SysWOW64\Kerberos.DLL
LoadedModule[115]=C:\Windows\system32\cryptdll.dll
LoadedModule[116]=C:\Windows\SysWOW64\msv1_0.DLL
LoadedModule[117]=C:\Windows\system32\ntdsapi.dll
LoadedModule[118]=C:\Windows\system32\LOGONCLI.DLL
LoadedModule[119]=C:\Windows\system32\CLUSAPI.DLL
LoadedModule[120]=C:\Windows\system32\RESUTILS.DLL
LoadedModule[121]=C:\Windows\system32\security.dll
LoadedModule[122]=C:\Windows\SysWOW64\schannel.dll
LoadedModule[123]=C:\Windows\system32\ncrypt.dll
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=Apache HTTP Server
AppPath=C:\Program Files (x86)\Apache2.4\bin\httpd.exe
 [2015-07-08 12:59 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-07-08 12:59 UTC] ab@php.net
Hi, thanks for the follow up. Unfortunately it is not a backtrace. Please check https://bugs.php.net/bugs-generating-backtrace-win32.php for how a backtrace looks like. Also, the tool mentioned there http://www.microsoft.com/en-us/download/details.aspx?id=40336 can be used to extract backtraces from the crash dump files.

Please don't embed such long outputs into the posts, pastebin would be fine for it.

Thanks.
 [2015-07-08 13:50 UTC] jeremy dot j dot dunn at gmail dot com
-Status: Feedback +Status: Open
 [2015-07-08 13:50 UTC] jeremy dot j dot dunn at gmail dot com
> Please don't embed such long outputs into the posts, pastebin would be fine for it.
sorry, I knew this was not good; but didn't know where to attach the files.  

Suggestion to add a note on the bug-tracker for inexperienced persons such as me, to use pastebin for attachments.  When entering the ticket I looked for information on how to attach files, but did not see it.

> Unfortunately it is not a backtrace.
yes, I see that. 

> ... can be used to extract backtraces from the crash dump files
unfortunately the server was not configured to collect crash dump files at the time of the crashes.  This feature is not enabled by default:

"Collecting User-Mode dumps"
https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181%28v=vs.85%29.aspx

All I have is the /WER/ files, as previously posted.  It does not appear that backtraces can be gotten from .wer files.

I understand that what has been provided is insufficient to debug, especially since I cannot recreate the problem.  Perhaps others will have a similar problem and can provide more information.

Thanks for your time, help, and attention; and for developing PHP ! :-)
 [2015-07-09 09:32 UTC] ab@php.net
Hi,

with pastebin - it's just for huge inputs, so the posts are more readable.

Yeah, you was mentioning WER, so i just hoped some dump files are there as well. so lets keep it open, maybe you have more info later. But regarding others - without a backtrace or repro code it'll be hard to identify the issue is same.

Thanks.
 [2015-07-24 20:27 UTC] php_150725 at ayd dot jp
I also experienced the same problem.

Description:
------------
OS:CentOS(64bit)
Web:apache 2.4.10
PHP:5.5.27/5.5.26
(PHP 5.5.25 does not problem occurs.)

Test script:
---------------
I can not reproduce the problem in a test script.
However, problems occur when applying a load to my server.

For example)

/usr/local/apache2/bin/ab -n 100 -c 50 http://mysite/xxxx.php

I have investigated this problem.
Following commit seem to be the cause.

https://github.com/php/php-src/commit/c117548ea9365adac00960fe5f43425b2955310d

When I return the "main/php_open_temporary_file.c" file, problem no longer occurs.
(efree -> free , estrndup -> zend_strndup etc...)

and,I think running program this flow.

https://github.com/php/php-src/blob/PHP-5.5.27/Zend/zend_alloc.c#L2105 -> 
https://github.com/php/php-src/blob/PHP-5.5.27/Zend/zend_alloc.c#L838
 [2015-07-25 07:04 UTC] php_150725 at ayd dot jp
Additional information.

I've compiled the php-5.5.27 with "--enable-debug" option.
The following messages has been displayed in the apache error log.


[Sat Jul 25 15:51:32 2015]  Script:  '/var/www/xxxx.php'
---------------------------------------
/PHP_SRC_PATH/main/php_open_temporary_file.c(184) : Block 0x2aaaaad0c2d0 status:
Invalid pointer: ((thread_id=0x42478940) != (expected=0x4BA87940))

---------------------------------------
[Sat Jul 25 15:51:37 2015]  Script:  '/var/www/xxxx.php'
/PHP_STC_PATH/main/php_open_temporary_file.c(250) :  Freeing 0x2AAAC04B7EA0 (5 bytes), script=/var/www/xxxx.php
=== Total 1 memory leaks detected ===
 [2015-07-27 14:15 UTC] ab@php.net
@php_150725 thanks for the follow up. Unfortunately it can't be said for sure whether you've the same bug because we had no backtrace.

But the code in main/php_open_temporary_file.c is definitely not thread safe. Before the patch you've linked it was only freeing the tmp dir in MSHUTDOWN, now it frees in RSHUTDOWN. That means - race conditions.

I've just looked through and seems this is present in 5.6 as well.

As a quick solution for you - if your app doesn't suffer under #66048 (you don't change the tmp dir), you can just revert this particular part. Also yep, you can downgrade PHP. Or you can upgrade to 5.6 after there's a fix flowed in there.

Does this crash happen with any script, or there's one you use which easy reproduces the crash?

Thanks

Anatol
 [2015-07-27 14:16 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-07-27 15:05 UTC] jpauli@php.net
Definitely a race condition.
I forgot about it, yes, my bad ;-)
Fixing it...
 [2015-07-27 16:28 UTC] ab@php.net
The following patch has been added/updated:

Patch Name: temporary_dir_ts
Revision:   1438014482
URL:        https://bugs.php.net/patch-display.php?bug=70002&patch=temporary_dir_ts&revision=1438014482
 [2015-07-27 16:28 UTC] ab@php.net
@php_150725 please check the attached patch.

Thanks.
 [2015-07-28 08:40 UTC] ab@php.net
The following patch has been added/updated:

Patch Name: temp_dir_ts_2
Revision:   1438072803
URL:        https://bugs.php.net/patch-display.php?bug=70002&patch=temp_dir_ts_2&revision=1438072803
 [2015-07-28 08:51 UTC] ab@php.net
The following patch has been added/updated:

Patch Name: temp_dir_ts_3
Revision:   1438073468
URL:        https://bugs.php.net/patch-display.php?bug=70002&patch=temp_dir_ts_3&revision=1438073468
 [2015-07-28 14:40 UTC] ab@php.net
The following patch has been added/updated:

Patch Name: temp_dir_ts_4
Revision:   1438094443
URL:        https://bugs.php.net/patch-display.php?bug=70002&patch=temp_dir_ts_4&revision=1438094443
 [2015-07-28 15:45 UTC] ab@php.net
Automatic comment on behalf of ab
Revision: http://git.php.net/?p=php-src.git;a=commit;h=ce5c4500cd942ab85efa1e916ef00de860be9e6e
Log: Fixed bug #70002 TS issues with temporary dir handling
 [2015-07-28 15:45 UTC] ab@php.net
-Status: Feedback +Status: Closed
 [2015-07-30 18:01 UTC] php_150725 at ayd dot jp
It seems working correctly after applying the patch.
Error log also does not output.
Thank you fix the program.
 [2015-08-04 20:54 UTC] ab@php.net
Automatic comment on behalf of ab
Revision: http://git.php.net/?p=php-src.git;a=commit;h=ce5c4500cd942ab85efa1e916ef00de860be9e6e
Log: Fixed bug #70002 TS issues with temporary dir handling
 [2015-08-15 12:37 UTC] truniger at bluewin dot ch
This patch produces broken binaries on Solaris, see #70212 or #70254.
 [2015-12-08 21:52 UTC] jake dot dawley62 at gmail dot com
I am new and could use some help. My server is Windows Server 2012 R2 running AMPPS for my Apache, Mysql, and php. I have having the above and need to apply this patch, but I am completely lost as to how to do so. Can someone point me in the right direction? 

Thank you
 [2016-07-04 15:14 UTC] kaplan@php.net
-Assigned To: +Assigned To: kaplan -CVE-ID: +CVE-ID: 2015-8878
 [2016-07-04 15:24 UTC] jeremy dot j dot dunn at gmail dot com
@Jake

> I am new and could use some help. My server is Windows Server 2012 R2 running AMPPS for my Apache, Mysql, and php. I have having the above and need to apply this patch, but I am completely lost as to how to do so. Can someone point me in the right direction? 

just use the most release version 5.5.37 or 5.6.23 and it should be stable.

http://windows.php.net/download/
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Sun Feb 19 14:01:37 2017 UTC