php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #75551 SIGABRT when multi-threading PHP in Litespeed
Submitted: 2017-11-21 20:26 UTC Modified: 2017-11-21 21:06 UTC
From: rperper at litespeedtech dot com Assigned:
Status: Feedback Package: Reproducible crash
PHP Version: 5.6.32 OS: OpenSuSE
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2017-11-21 20:26 UTC] rperper at litespeedtech dot com
Description:
------------
I am a developer at LiteSpeed Technologies and am working on a thread-capable version of the PHP module to be included in the Open-LiteSpeed web server.  During load testing, we got a SIGABRT crash in php_pcre.c (see backtrace below) in line 282 in a call to setlocale.  setlocale is not a thread-safe function and this is expected behavior.  

Test script:
---------------
This can not be demonstrated in a script at this time.

Actual result:
--------------
=================================================================
[1m[31m==65270==ERROR: AddressSanitizer: heap-use-after-free on address 0x6020000f2a90 at pc 0x00000051578f bp 0x7fffe4cc6510 sp 0x7fffe4cc5cd0
[1m[0m[1m[34mWRITE of size 2 at 0x6020000f2a90 thread T6[1m[0m
    #0 0x51578e in __interceptor_setlocale /home/abuild/rpmbuild/BUILD/llvm-3.8.0.src/stage2/../projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2527:12
    #1 0x7fffeadbb312 in pcre_get_compiled_regex_cache /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/ext/pcre/php_pcre.c:282:11
    #2 0x7fffeadcd971 in zif_preg_split /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/ext/pcre/php_pcre.c:1553:13
    #3 0x7fffec899d4c in zend_do_fcall_common_helper_SPEC /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/Zend/zend_vm_execute.h:558:5
    #4 0x7fffec770341 in ZEND_DO_FCALL_SPEC_CONST_HANDLER /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/Zend/zend_vm_execute.h:2602:9
    #5 0x7fffec70a7e3 in execute_ex /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/Zend/zend_vm_execute.h:363:14
    #6 0x7fffec70ac0c in zend_execute /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/Zend/zend_vm_execute.h:388:2
    #7 0x7fffec5de371 in zend_execute_scripts /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/Zend/zend.c:1341:4
    #8 0x7fffec2d0616 in php_execute_script /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/main/main.c:2613:14
    #9 0x7fffec95f06d in lsiapi_execute_script /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/sapi/mod_lsphp/mod_lsphp.c:1397:19
    #10 0x7fffec9581b4 in lsiapi_module_main /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/sapi/mod_lsphp/mod_lsphp.c:1506:9
    #11 0x7fffec955401 in process_req /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/sapi/mod_lsphp/mod_lsphp.c:1533:19
    #12 0x7fffec94d222 in mod_lsphp_begin_process /home/user/proj/openlitespeed/src/modules/mod_lsphp/php-5.6/sapi/mod_lsphp/mod_lsphp.c:1664:10
    #13 0x98eb11 in MtHandlerProcess(ls_lfnodei_s*) /home/user/proj/openlitespeed/src/lsiapi/modulehandler.cpp:46:9
    #14 0xab0380 in WorkCrew::workerRoutine(CrewWorker*) /home/user/proj/openlitespeed/src/thread/workcrew.cpp:448:25
    #15 0xab0c88 in CrewWorker::thr_main(void*) /home/user/proj/openlitespeed/src/thread/crewworker.cpp:36:12
    #16 0xaac840 in Thread::start_routine(void*) /home/user/proj/openlitespeed/src/thread/thread.cpp:43:11
    #17 0x7ffff7bc7743 in start_thread (/lib64/libpthread.so.0+0x8743)
    #18 0x7ffff67f7aac in __clone (/lib64/libc.so.6+0xe9aac)
[1m[32m0x6020000f2a90 is located 0 bytes inside of 12-byte region [0x6020000f2a90,0x6020000f2a9c)
[1m[0m[1m[35mfreed by thread T7 here:[1m[0m
    #0 0x57798b in free /home/abuild/rpmbuild/BUILD/llvm-3.8.0.src/stage2/../projects/compiler-rt/lib/asan/asan_malloc_linux.cc:38:3
    #1 0x7ffff6739281 in __GI_setlocale (/lib64/libc.so.6+0x2b281)
[1m[35mpreviously allocated by thread T6 here:[1m[0m
    #0 0x577cab in __interceptor_malloc /home/abuild/rpmbuild/BUILD/llvm-3.8.0.src/stage2/../projects/compiler-rt/lib/asan/asan_malloc_linux.cc:52:3
    #1 0x7ffff678ec89 in __GI___strdup (/lib64/libc.so.6+0x80c89)
Thread T6 created by T0 here:
    #0 0x560069 in pthread_create /home/abuild/rpmbuild/BUILD/llvm-3.8.0.src/stage2/../projects/compiler-rt/lib/asan/asan_interceptors.cc:238:3
    #1 0x7b30d6 in Thread::start(void*) /home/user/proj/openlitespeed/src/thread/thread.h:74:19
    #2 0x7b1a97 in Worker::start(void*) /home/user/proj/openlitespeed/src/thread/worker.h:67:20
    #3 0xab0996 in CrewWorker::start() /home/user/proj/openlitespeed/src/thread/crewworker.h:44:16
    #4 0xaae309 in WorkCrew::addWorker() /home/user/proj/openlitespeed/src/thread/workcrew.cpp:168:21
    #5 0xaaf9ad in WorkCrew::addJob(ls_lfnodei_s*) /home/user/proj/openlitespeed/src/thread/workcrew.cpp:338:9
    #6 0x99127d in ModuleHandler::mt_process(HttpSession*, lsi_reqhdlr_s const*) /home/user/proj/openlitespeed/src/lsiapi/modulehandler.cpp:344:9
    #7 0x99020d in ModuleHandler::process(HttpSession*, HttpHandler const*) /home/user/proj/openlitespeed/src/lsiapi/modulehandler.cpp:201:16
    #8 0x91695b in HttpSession::handlerProcess(HttpHandler const*) /home/user/proj/openlitespeed/src/http/httpsession.cpp:1814:11
    #9 0x90cd5a in HttpSession::smProcessReq() /home/user/proj/openlitespeed/src/http/httpsession.cpp:4561:19
    #10 0x916c3e in HttpSession::onReadEx() /home/user/proj/openlitespeed/src/http/httpsession.cpp:2086:15
    #11 0x8c9010 in NtwkIOLink::onRead(NtwkIOLink*) /home/user/proj/openlitespeed/src/http/ntwkiolink.cpp:864:16
    #12 0x8cf9c6 in NtwkIOLink::handleEvents(short) /home/user/proj/openlitespeed/src/http/ntwkiolink.cpp:400:9
    #13 0x8cf732 in NtwkIOLink::tryRead() /home/user/proj/openlitespeed/src/http/ntwkiolink.cpp:373:5
    #14 0x8e1a61 in HttpListener::addConnection(conn_data*, int*) /home/user/proj/openlitespeed/src/http/httplistener.cpp:516:5
    #15 0x8e06a4 in HttpListener::handleEvents(short) /home/user/proj/openlitespeed/src/http/httplistener.cpp:333:13
    #16 0xa47654 in epoll::waitAndProcessEvents(int) /home/user/proj/openlitespeed/src/edio/epoll.cpp:216:13
    #17 0x8a9232 in EventDispatcher::run() /home/user/proj/openlitespeed/src/http/eventdispatcher.cpp:231:15
    #18 0x810341 in HttpServerImpl::start() /home/user/proj/openlitespeed/src/main/httpserver.cpp:484:5
    #19 0x827ad0 in HttpServer::start() /home/user/proj/openlitespeed/src/main/httpserver.cpp:4153:12
    #20 0x80a3b5 in LshttpdMain::main(int, char**) /home/user/proj/openlitespeed/src/main/lshttpdmain.cpp:980:9
    #21 0x5a499e in main /home/user/proj/openlitespeed/src/main.cpp:109:15
    #22 0x7ffff672e6e4 in __libc_start_main (/lib64/libc.so.6+0x206e4)
Thread T7 created by T0 here:
    #0 0x560069 in pthread_create /home/abuild/rpmbuild/BUILD/llvm-3.8.0.src/stage2/../projects/compiler-rt/lib/asan/asan_interceptors.cc:238:3
    #1 0x7b30d6 in Thread::start(void*) /home/user/proj/openlitespeed/src/thread/thread.h:74:19
    #2 0x7b1a97 in Worker::start(void*) /home/user/proj/openlitespeed/src/thread/worker.h:67:20
    #3 0xab0996 in CrewWorker::start() /home/user/proj/openlitespeed/src/thread/crewworker.h:44:16
    #4 0xaae309 in WorkCrew::addWorker() /home/user/proj/openlitespeed/src/thread/workcrew.cpp:168:21
    #5 0xaaf9ad in WorkCrew::addJob(ls_lfnodei_s*) /home/user/proj/openlitespeed/src/thread/workcrew.cpp:338:9
    #6 0x99127d in ModuleHandler::mt_process(HttpSession*, lsi_reqhdlr_s const*) /home/user/proj/openlitespeed/src/lsiapi/modulehandler.cpp:344:9
    #7 0x99020d in ModuleHandler::process(HttpSession*, HttpHandler const*) /home/user/proj/openlitespeed/src/lsiapi/modulehandler.cpp:201:16
    #8 0x91695b in HttpSession::handlerProcess(HttpHandler const*) /home/user/proj/openlitespeed/src/http/httpsession.cpp:1814:11
    #9 0x90cd5a in HttpSession::smProcessReq() /home/user/proj/openlitespeed/src/http/httpsession.cpp:4561:19
    #10 0x916c3e in HttpSession::onReadEx() /home/user/proj/openlitespeed/src/http/httpsession.cpp:2086:15
    #11 0x8c9010 in NtwkIOLink::onRead(NtwkIOLink*) /home/user/proj/openlitespeed/src/http/ntwkiolink.cpp:864:16
    #12 0x8cf9c6 in NtwkIOLink::handleEvents(short) /home/user/proj/openlitespeed/src/http/ntwkiolink.cpp:400:9
    #13 0x8cf732 in NtwkIOLink::tryRead() /home/user/proj/openlitespeed/src/http/ntwkiolink.cpp:373:5
    #14 0x8e1a61 in HttpListener::addConnection(conn_data*, int*) /home/user/proj/openlitespeed/src/http/httplistener.cpp:516:5
    #15 0x8e06a4 in HttpListener::handleEvents(short) /home/user/proj/openlitespeed/src/http/httplistener.cpp:333:13
    #16 0xa47654 in epoll::waitAndProcessEvents(int) /home/user/proj/openlitespeed/src/edio/epoll.cpp:216:13
    #17 0x8a9232 in EventDispatcher::run() /home/user/proj/openlitespeed/src/http/eventdispatcher.cpp:231:15
    #18 0x810341 in HttpServerImpl::start() /home/user/proj/openlitespeed/src/main/httpserver.cpp:484:5
    #19 0x827ad0 in HttpServer::start() /home/user/proj/openlitespeed/src/main/httpserver.cpp:4153:12
    #20 0x80a3b5 in LshttpdMain::main(int, char**) /home/user/proj/openlitespeed/src/main/lshttpdmain.cpp:980:9
    #21 0x5a499e in main /home/user/proj/openlitespeed/src/main.cpp:109:15
    #22 0x7ffff672e6e4 in __libc_start_main (/lib64/libc.so.6+0x206e4)
SUMMARY: AddressSanitizer: heap-use-after-free /home/abuild/rpmbuild/BUILD/llvm-3.8.0.src/stage2/../projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2527:12 in __interceptor_setlocale
Shadow bytes around the buggy address:
  0x0c0480016500: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m02[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[35mfd[1m[0m
  0x0c0480016510: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m01[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m05[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m07[1m[0m
  0x0c0480016520: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m02[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m00[1m[0m
  0x0c0480016530: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m01[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m07[1m[0m [1m[31mfa[1m[0m
  0x0c0480016540: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m07[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m06[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m04[1m[0m
=>0x0c0480016550: [1m[31mfa[1m[0m [1m[31mfa[1m[0m[[1m[35mfd[1m[0m][1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[35mfd[1m[0m
  0x0c0480016560: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m00[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m00[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m07[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m00[1m[0m
  0x0c0480016570: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m02[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m04[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m05[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m01[1m[0m
  0x0c0480016580: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m05[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m07[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m00[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m04[1m[0m
  0x0c0480016590: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m04[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m04[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m04[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m06[1m[0m [1m[31mfa[1m[0m
  0x0c04800165a0: [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[35mfd[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m01[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m03[1m[0m [1m[31mfa[1m[0m [1m[31mfa[1m[0m [1m[0m00[1m[0m [1m[0m07[1m[0m
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           [1m[0m00[1m[0m
  Partially addressable: [1m[0m01[1m[0m [1m[0m02[1m[0m [1m[0m03[1m[0m [1m[0m04[1m[0m [1m[0m05[1m[0m [1m[0m06[1m[0m [1m[0m07[1m[0m 
  Heap left redzone:       [1m[31mfa[1m[0m
  Heap right redzone:      [1m[31mfb[1m[0m
  Freed heap region:       [1m[35mfd[1m[0m
  Stack left redzone:      [1m[31mf1[1m[0m
  Stack mid redzone:       [1m[31mf2[1m[0m
  Stack right redzone:     [1m[31mf3[1m[0m
  Stack partial redzone:   [1m[31mf4[1m[0m
  Stack after return:      [1m[35mf5[1m[0m
  Stack use after scope:   [1m[35mf8[1m[0m
  Global redzone:          [1m[31mf9[1m[0m
  Global init order:       [1m[36mf6[1m[0m
  Poisoned by user:        [1m[34mf7[1m[0m
  Container overflow:      [1m[34mfc[1m[0m
  Array cookie:            [1m[31mac[1m[0m
  Intra object redzone:    [1m[33mbb[1m[0m
  ASan internal:           [1m[33mfe[1m[0m
  Left alloca redzone:     [1m[34mca[1m[0m
  Right alloca redzone:    [1m[34mcb[1m[0m
==65270==ABORTING
*** Program received signal SIGABRT (Aborted) ***

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2017-11-21 20:32 UTC] spam2 at rhsoft dot net
besides that PHP5 is EOL did you compile PHP with ZTS enabled?
 [2017-11-21 20:39 UTC] rperper at litespeedtech dot com
Sorry, should have been clear about that.  Yes indeed, ZTS was enabled fully.  We couldn't test at all without it.  

We're having problems with 7.1 as well, but we figured that those problems being different and with most customers still on 5.6, we'd need to get that working first.

Our tests have been quite generally successful.  We need to truly pound PHP with quite a number of simultaneous PHP functions occurring before the crashes happen.  With PHP 7.1, it can't handle nearly as many simultaneous functions before crashing.  I'll open that up as a separate problem, but since 5.6 is so important, we figured we'd start here.
 [2017-11-21 21:02 UTC] nikic@php.net
The setlocale() call is no longer present in 7.x, instead the locale from the last call to the PHP setlocale() function is used. As mentioned by rhsoft2, PHP 5.6 is no longer maintained, so this issue will not be fixed there. I'm not familiar with your distribution model, but if you're patching PHP, you may want to apply a patch similar to https://github.com/php/php-src/commit/4514ba016ff158cd113deef1a215fcdcb6913b48.

As for PHP 7.x, unfortunately both PHP 7.0 and 7.1 suffer from data-races in refcounting of permanent strings. These issues were resolved in PHP 7.2. As such, PHP 7.2 is the minimum viable version for running 7.x in a multi-threaded context :(
 [2017-11-21 21:06 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2017-11-21 21:06 UTC] ab@php.net
@rperper 5.6 won't get any non security related fixes anymore. 7.0 and 7.1 have known thread safety issues that was fixed in 7.2. I'd ask you to please check your new SAPI against 7.2 and master, anything else makes a little sense nowadays.

thanks.
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Sun Nov 19 01:31:42 2017 UTC