php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #73901 Transient Segfault in zend_startup()
Submitted: 2017-01-10 00:47 UTC Modified: 2017-01-16 20:26 UTC
Votes:2
Avg. Score:5.0 ± 0.0
Reproduced:2 of 2 (100.0%)
Same Version:2 (100.0%)
Same OS:2 (100.0%)
From: tmwilder at gmail dot com Assigned:
Status: Open Package: Apache2 related
PHP Version: 7.1.0 OS: Ubuntu 14.04.3
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2017-01-10 00:47 UTC] tmwilder at gmail dot com
Description:
------------
When running a node with php7:

php -v

PHP Warning:  Module 'newrelic' already loaded in Unknown on line 0
PHP 7.1.0-5+deb.sury.org~trusty+1 (cli) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.1.0-dev, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.1.0-5+deb.sury.org~trusty+1, Copyright (c) 1999-2016, by Zend Technologies

And apache modules:

apache2 -M
[Tue Jan 10 00:41:15.548712 2017] [core:warn] [pid 12766] AH00111: Config variable ${APACHE_PID_FILE} is not defined
Loaded Modules:
 core_module (static)
 so_module (static)
 watchdog_module (static)
 http_module (static)
 log_config_module (static)
 logio_module (static)
 version_module (static)
 unixd_module (static)
 authz_core_module (shared)
 authz_host_module (shared)
 deflate_module (shared)
 dir_module (shared)
 filter_module (shared)
 headers_module (shared)
 mime_module (shared)
 php7_module (shared)
 mpm_prefork_module (shared)
 proxy_module (shared)
 proxy_http_module (shared)
 rewrite_module (shared)

With apache:

Server version: Apache/2.4.7 (Ubuntu)
Server built:   Jul 15 2016 15:34:04

I am running into a transient segfault that I can only reproduce under prod traffic conditions. I'm working on a more narrow repro but having trouble with the narrowing aspect of that.

I get the following bt that I have used apport-unpack to isolate from the crash file:

gdb /usr/sbin/apache2 ~/crashdumps/CoreDump

GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done.
[New LWP 3602]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/apache2 -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f89d2c2b8c6 in zend_startup () from /usr/lib/apache2/modules/libphp7.1.so
(gdb) bt
#0  0x00007f89d2c2b8c6 in zend_startup () from /usr/lib/apache2/modules/libphp7.1.so
#1  0x00007f89d2bcb6a3 in php_module_startup () from /usr/lib/apache2/modules/libphp7.1.so
#2  0x00007f89d2cc4fe5 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#3  0x00007f89d2cc5bd8 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#4  0x00007f89d8bea809 in ap_run_post_config ()
#5  0x00007f89d8bcae68 in main ()

Since there is no visible execute call and Google isn't turning anything up - I'm a bit stuck here. Is this a PHP bug - is there an environment miss in my setup? I'm more than happy to do anything I can to provide more information.



Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2017-01-10 01:07 UTC] pollita@php.net
Unfortunately, a LOT happens in zend_startup(), so it is a bit hard to pin down from what you have so far.

Is there any way you can try a debug build and tee some prod traffic at it?  A backtrace from a debug build would provide way more information.
 [2017-01-10 01:32 UTC] tmwilder at gmail dot com
I'll try to give that a shot.

I was just running for a short window again and found the following different bt:

ubuntu@remitlyweb-onebox-i-03e9af1d3d4e1be2b:~/crashlogs/2$ gdb /usr/sbin/apache2 CoreDump
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done.
[New LWP 14034]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/apache2 -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fa568e25371 in _emalloc_16 () from /usr/lib/apache2/modules/libphp7.1.so
(gdb) bt
#0  0x00007fa568e25371 in _emalloc_16 () from /usr/lib/apache2/modules/libphp7.1.so
#1  0x00007fa55c84d6f5 in nr_php_zval_alloc () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_zval.h:60
#2  nr_php_call_user_func () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_call.c:42
#3  0x00007fa55c84dd8e in nr_php_curl_set_default_response_header_callback () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_curl.c:126
#4  nr_php_curl_init () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_curl.c:171
#5  0x00007fa568edf948 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#6  0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#7  0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#8  0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#9  0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#10 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#11 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#12 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#13 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#14 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#15 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#16 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#17 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#18 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#19 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#20 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#21 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#22 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#23 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#24 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#25 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#26 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#27 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#28 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#29 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#30 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#31 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#32 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#33 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#34 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#35 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#36 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#37 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#38 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#39 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#40 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#41 0x00007fa568edf735 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#42 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#43 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#44 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#45 0x00007fa568edfbb5 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#46 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#47 0x00007fa55c850f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
#48 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#49 0x00007fa568edfbb5 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#50 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#51 0x00007fa55c85105a in nr_php_execute_file () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:837
#52 nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1059
#53 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#54 0x00007fa568edc99f in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#55 0x00007fa568e8ee7b in execute_ex () from /usr/lib/apache2/modules/libphp7.1.so
#56 0x00007fa55c85105a in nr_php_execute_file () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:837
#57 nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1059
#58 0x00007fa55c851562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
#59 0x00007fa568ee1b94 in zend_execute () from /usr/lib/apache2/modules/libphp7.1.so
#60 0x00007fa568e4a284 in zend_execute_scripts () from /usr/lib/apache2/modules/libphp7.1.so
#61 0x00007fa568dea6c8 in php_execute_script () from /usr/lib/apache2/modules/libphp7.1.so
#62 0x00007fa568ee3ae2 in ?? () from /usr/lib/apache2/modules/libphp7.1.so
#63 0x00007fa56b9d0be0 in ap_run_handler ()
#64 0x00007fa56b9d1129 in ap_invoke_handler ()
#65 0x00007fa56b9e66fa in ap_process_async_request ()
#66 0x00007fa56b9e69d4 in ap_process_request ()
#67 0x00007fa56b9e3472 in ?? ()
#68 0x00007fa56b9da250 in ap_run_process_connection ()
#69 0x00007fa567898767 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
#70 0x00007fa5678989a6 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
#71 0x00007fa567898a06 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
#72 0x00007fa5678996e0 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
#73 0x00007fa56b9b78ae in ap_run_mpm ()
#74 0x00007fa56b9b1046 in main ()
 [2017-01-10 01:34 UTC] tmwilder at gmail dot com
This strikes me as odd, none of those hudson paths exist on my system.
 [2017-01-11 04:47 UTC] krakjoe@php.net
-Status: Open +Status: Feedback
 [2017-01-11 04:47 UTC] krakjoe@php.net
I have a feeling this is caused by that (seen traces like that before): Can I suggest that you remove the newrelic extension and retest ?
 [2017-01-16 20:26 UTC] tmwilder at gmail dot com
-Status: Feedback +Status: Open
 [2017-01-16 20:26 UTC] tmwilder at gmail dot com
Ok, I removed that extension and added debug symbols for some related libraries. Seeing a different backtrace but with the same message as the original:
build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_alloc.c: No such file or directory.

BT:

gdb /usr/sbin/apache2 ./CoreDump
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done.
[New LWP 4003]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/apache2 -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  zend_mm_alloc_small (size=<optimized out>, bin_num=<optimized out>, heap=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_alloc.c:1261
1261	/build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_alloc.c: No such file or directory.
(gdb) bt full
#0  zend_mm_alloc_small (size=<optimized out>, bin_num=<optimized out>, heap=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_alloc.c:1261
No locals.
#1  zend_mm_alloc_heap (size=<optimized out>, heap=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_alloc.c:1332
        ptr = 0x800000001
#2  _emalloc (size=3, size@entry=32) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_alloc.c:2417
No locals.
#3  0x00007f350f036db1 in zend_string_alloc (persistent=<optimized out>, len=4) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_string.h:122
        ret = <optimized out>
#4  zend_string_init (persistent=<optimized out>, len=4, str=0x7f350154365a "LLEN") at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_string.h:158
No locals.
#5  _zend_hash_str_update (ht=ht@entry=0x7f350cb272a0, str=str@entry=0x7f350154365a "LLEN", len=len@entry=4, pData=pData@entry=0x7ffda5397f80) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_hash.c:650
        key = 0x7f350154365a
#6  0x00007f350f02cb9f in zend_symtable_str_update (pData=0x7ffda5397f80, len=4, str=0x7f350154365a "LLEN", ht=0x7f350cb272a0) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_hash.h:444
        idx = 139865609994241
#7  add_assoc_bool_ex (arg=arg@entry=0x7f350cb6b128, key=key@entry=0x7f350154365a "LLEN", key_len=key_len@entry=4, b=b@entry=1) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_API.c:1345
        tmp = {value = {lval = 140727375462480, dval = 6.9528561645413773e-310, counted = 0x7ffda5398050, str = 0x7ffda5398050, arr = 0x7ffda5398050, obj = 0x7ffda5398050, res = 0x7ffda5398050, ref = 0x7ffda5398050, ast = 0x7ffda5398050, zv = 0x7ffda5398050, ptr = 0x7ffda5398050, ce = 0x7ffda5398050, func = 0x7ffda5398050, ww = {w1 = 2772009040, w2 = 32765}},
          u1 = {v = {type = 3 '\003', type_flags = 0 '\000', const_flags = 0 '\000', reserved = 0 '\000'}, type_info = 3}, u2 = {next = 32565, cache_slot = 32565, lineno = 32565, num_args = 32565, fe_pos = 32565, fe_iter_idx = 32565, access_flags = 32565, property_guard = 32565}}
#8  0x00007f350152c77d in ra_init_function_table (ra=ra@entry=0x7f350cb6b0e0) at /build/php-redis-3FeCB6/php-redis-3.1.1~rc1/build-7.1/redis_array_impl.c:124
No locals.
#9  0x00007f350152ca66 in ra_make_array (hosts=0x7f34f8087e30, z_fun=z_fun@entry=0x7ffda5398060, z_dist=z_dist@entry=0x7ffda5398070, hosts_prev=hosts_prev@entry=0x0, b_index=b_index@entry=0 '\000', b_pconnect=b_pconnect@entry=0 '\000', retry_interval=retry_interval@entry=0, b_lazy_connect=0 '\000', connect_timeout=connect_timeout@entry=0.10000000000000001)
    at /build/php-redis-3FeCB6/php-redis-3.1.1~rc1/build-7.1/redis_array_impl.c:366
        i = <optimized out>
        count = <optimized out>
        ra = 0x7f350cb6b0e0
#10 0x00007f350152888c in zim_RedisArray___construct (execute_data=0x7f350ca11bd0, return_value=<optimized out>) at /build/php-redis-3FeCB6/php-redis-3.1.1~rc1/build-7.1/redis_array.c:300
        z0 = 0x7f350ca11c20
        z_fun = {value = {lval = 139865928057552, dval = 6.9102950076939755e-310, counted = 0x7f3512f542d0, str = 0x7f3512f542d0, arr = 0x7f3512f542d0, obj = 0x7f3512f542d0, res = 0x7f3512f542d0, ref = 0x7f3512f542d0, ast = 0x7f3512f542d0, zv = 0x7f3512f542d0, ptr = 0x7f3512f542d0, ce = 0x7f3512f542d0, func = 0x7f3512f542d0, ww = {w1 = 318063312, w2 = 32565}},
          u1 = {v = {type = 1 '\001', type_flags = 0 '\000', const_flags = 0 '\000', reserved = 0 '\000'}, type_info = 1}, u2 = {next = 32565, cache_slot = 32565, lineno = 32565, num_args = 32565, fe_pos = 32565, fe_iter_idx = 32565, access_flags = 32565, property_guard = 32565}}
        z_dist = {value = {lval = 139865466915616, dval = 6.910272224255132e-310, counted = 0x7f34f778cb20, str = 0x7f34f778cb20, arr = 0x7f34f778cb20, obj = 0x7f34f778cb20, res = 0x7f34f778cb20, ref = 0x7f34f778cb20, ast = 0x7f34f778cb20, zv = 0x7f34f778cb20, ptr = 0x7f34f778cb20, ce = 0x7f34f778cb20, func = 0x7f34f778cb20, ww = {w1 = 4151888672,
              w2 = 32564}}, u1 = {v = {type = 1 '\001', type_flags = 0 '\000', const_flags = 0 '\000', reserved = 0 '\000'}, type_info = 1}, u2 = {next = 32565, cache_slot = 32565, lineno = 32565, num_args = 32565, fe_pos = 32565, fe_iter_idx = 32565, access_flags = 32565, property_guard = 32565}}
        zpData = <optimized out>
        z_opts = 0x7f350ca11c30
        ra = 0x0
        b_index = 0 '\000'
        b_autorehash = 0 '\000'
        b_pconnect = 0 '\000'
        hPrev = 0x0
        hOpts = <optimized out>
        l_retry_interval = 0
        b_lazy_connect = <optimized out>
        d_connect_timeout = 0.10000000000000001
#11 0x00007f350f0bfe76 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:970
        retval = {value = {lval = 140727375462752, dval = 6.9528561645548159e-310, counted = 0x7ffda5398160, str = 0x7ffda5398160, arr = 0x7ffda5398160, obj = 0x7ffda5398160, res = 0x7ffda5398160, ref = 0x7ffda5398160, ast = 0x7ffda5398160, zv = 0x7ffda5398160, ptr = 0x7ffda5398160, ce = 0x7ffda5398160, func = 0x7ffda5398160, ww = {w1 = 2772009312,
              w2 = 32765}}, u1 = {v = {type = 1 '\001', type_flags = 0 '\000', const_flags = 0 '\000', reserved = 0 '\000'}, type_info = 1}, u2 = {next = 32565, cache_slot = 32565, lineno = 32565, num_args = 32565, fe_pos = 32565, fe_iter_idx = 32565, access_flags = 32565, property_guard = 32565}}
        call = 0x7f350ca11bd0
        fbc = 0x7f3512f54530
        object = <optimized out>
        ret = 0x7ffda53980d0
#12 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca11a00
#13 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#14 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#15 0x00007f350f0bf735 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:1076
        call = 0x7f350ca11ae0
        fbc = 0x7f350cb0f210
        object = 0x7f350cb0f210
        ret = <optimized out>
#16 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca11990
#17 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#18 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#19 0x00007f350f0bf735 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:1076
        call = 0x7f350ca11a00
        fbc = 0x7f350cab4978
        object = 0x7f350cab4978
        ret = <optimized out>
#20 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca118e0
#21 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#22 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#23 0x00007f350f0bf735 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:1076
        call = 0x7f350ca11990
        fbc = 0x7f350caf6ff8
---Type <return> to continue, or q <return> to quit---
        object = 0x7f350caf6ff8
        ret = <optimized out>
#24 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca11880
#25 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#26 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#27 0x00007f350f0bf735 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:1076
        call = 0x7f350ca118e0
        fbc = 0x7f350cad5518
        object = 0x7f350cad5518
        ret = <optimized out>
#28 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca11650
#29 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#30 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#31 0x00007f350f0bf735 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:1076
        call = 0x7f350ca11880
        fbc = 0x7f350cad56b8
        object = 0x7f350cad56b8
        ret = <optimized out>
#32 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca11440
#33 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#34 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#35 0x00007f350f0bf735 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:1076
        call = 0x7f350ca11650
        fbc = 0x7f350cad2d48
        object = 0x7f350cad2d48
        ret = <optimized out>
#36 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca11250
#37 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#38 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#39 0x00007f350f0bfbb5 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:949
        call = 0x7f350ca11440
        fbc = 0x7f350cad26c8
        object = <optimized out>
        ret = 0x0
#40 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f351307f5f0
        orig_execute_data = 0x7f350ca110d0
#41 0x00007f3502a50f1b in nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1177
No locals.
#42 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#43 0x00007f350f0bfbb5 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:949
        call = 0x7f350ca11250
        fbc = 0x7f350cad25f8
        object = <optimized out>
        ret = 0x0
#44 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f3502ce2640 <all_frameworks>
        orig_execute_data = 0x7f3502ce2640 <all_frameworks>
#45 0x00007f3502a5105a in nr_php_execute_file () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:837
No locals.
#46 nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1059
No locals.
#47 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#48 0x00007f350f0bc99f in ZEND_INCLUDE_OR_EVAL_SPEC_TMPVAR_HANDLER () at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:51605
        return_value = <optimized out>
        new_op_array = 0x7f350ca870e0
        free_op1 = <optimized out>
        inc_filename = 0x7f350ca110d0
#49 0x00007f350f06ee7b in execute_ex (ex=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:429
        orig_opline = 0x7f3502ce2640 <all_frameworks>
        orig_execute_data = 0x7f3502ce2640 <all_frameworks>
#50 0x00007f3502a5105a in nr_php_execute_file () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:837
No locals.
#51 nr_php_execute_enabled () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1059
---Type <return> to continue, or q <return> to quit---
No locals.
#52 0x00007f3502a51562 in nr_php_execute () at /home/hudson/slave-workspace/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1287
No locals.
#53 0x00007f350f0c1b94 in zend_execute (op_array=op_array@entry=0x7f350ca87000, return_value=return_value@entry=0x0) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend_vm_execute.h:474
No locals.
#54 0x00007f350f02a284 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend.c:1474
        files = {{gp_offset = 40, fp_offset = 32565, overflow_arg_area = 0x7ffda5398e80, reg_save_area = 0x7ffda5398e10}}
        i = 1
        file_handle = 0x7ffda539b0d0
        op_array = 0x7f350ca87000
#55 0x00007f350efca6c8 in php_execute_script (primary_file=primary_file@entry=0x7ffda539b0d0) at /build/php7.1-hzF8RG/php7.1-7.1.0/main/main.c:2533
        realfile = '\000' <repeats 856 times>...
        __orig_bailout = 0x7ffda539b140
        __bailout = {{__jmpbuf = {139865908461728, -7540880771196452665, 139865908461728, 139865909841824, 0, 139865908630160, -7540880771798335289, -7581740228942167865}, __mask_was_saved = 0, __saved_mask = {__val = {0 <repeats 16 times>}}}}
        prepend_file_p = 0x0
        append_file_p = 0x0
        prepend_file = {handle = {fd = 0, fp = 0x0, stream = {handle = 0x0, isatty = 0, mmap = {len = 0, pos = 0, map = 0x0, buf = 0x0, old_handle = 0x0, old_closer = 0x0}, reader = 0x0, fsizer = 0x0, closer = 0x0}}, filename = 0x0, opened_path = 0x0, type = ZEND_HANDLE_FILENAME, free_filename = 0 '\000'}
        append_file = {handle = {fd = 0, fp = 0x0, stream = {handle = 0x0, isatty = 0, mmap = {len = 0, pos = 0, map = 0x0, buf = 0x0, old_handle = 0x0, old_closer = 0x0}, reader = 0x0, fsizer = 0x0, closer = 0x0}}, filename = 0x0, opened_path = 0x0, type = ZEND_HANDLE_FILENAME, free_filename = 0 '\000'}
        old_cwd = 0x7ffda5398e80 "/"
        retval = 0
#56 0x00007f350f0c3ae2 in php_handler (r=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/sapi/apache2handler/sapi_apache2.c:712
        zfd = {handle = {fd = 0, fp = 0x0, stream = {handle = 0x0, isatty = 299306184, mmap = {len = 139865880843079, pos = 139865898034542, map = 0x100000001, buf = 0x0, old_handle = 0x7f3511d65ce0, old_closer = 0x7f3511d70cc8}, reader = 0x7f3511d70cc8, fsizer = 0x7f3511ca40a0, closer = 0x1}},
          filename = 0x7f3511cad598 "/var/deploy/apps/remitlyweb/latest/site/index.php", opened_path = 0x0, type = ZEND_HANDLE_FILENAME, free_filename = 0 '\000'}
        __orig_bailout = 0x0
        __bailout = {{__jmpbuf = {139865908461728, 7539626758952255687, 139865908461728, 139865909841824, 0, 139865908630160, -7540880771198549817, -7581743386317619001}, __mask_was_saved = 0, __saved_mask = {__val = {139865908461728, 0, 139865908462152, 139865909508104, 139865909325472, 139865908501464, 0, 0, 0, 139865908630160, 139865870163107,
                139865909841824, 139865909259368, 0, 139865908461728, 139865909841824}}}}
        ctx = 0x7f3511cadc40
        conf = <optimized out>
        brigade = 0x7f3511caea80
        bucket = <optimized out>
        rv = <optimized out>
        parent_req = 0x0
#57 0x00007f3511bb0be0 in ap_run_handler ()
No symbol table info available.
#58 0x00007f3511bb1129 in ap_invoke_handler ()
No symbol table info available.
#59 0x00007f3511bc66fa in ap_process_async_request ()
No symbol table info available.
#60 0x00007f3511bc69d4 in ap_process_request ()
No symbol table info available.
#61 0x00007f3511bc3472 in ?? ()
No symbol table info available.
#62 0x00007f3511bba250 in ap_run_process_connection ()
No symbol table info available.
#63 0x00007f350da78767 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
No symbol table info available.
#64 0x00007f350da789a6 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
No symbol table info available.
#65 0x00007f350da78a06 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
No symbol table info available.
#66 0x00007f350da796e0 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so
No symbol table info available.
#67 0x00007f3511b978ae in ap_run_mpm ()
No symbol table info available.
#68 0x00007f3511b91046 in main ()
No symbol table info available.
 [2017-01-16 20:30 UTC] tmwilder at gmail dot com
My mistake - the original error message is actually this, doing more digging but Google is thin here. This occurred while the newrelic extension was still installed while the bt above did not:

gdb /usr/sbin/apache2 ./CoreDump
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done.
[New LWP 3603]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/apache2 -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f34ee3498c6 in zend_startup (utility_functions=0x7ffe01e18700, extensions=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend.c:718
718	/build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend.c: No such file or directory.
(gdb) bt full
#0  0x00007f34ee3498c6 in zend_startup (utility_functions=0x7ffe01e18700, extensions=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/Zend/zend.c:718
No locals.
#1  0x00007f34ee2e96a3 in php_module_startup (sf=0x7f34ee713dc0 <apache2_sapi_module>, additional_modules=0x7f34f091fde0 <php_apache_module>, num_additional_modules=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/main/main.c:2123
        zuf = {error_function = 0x7f34ee1cf93f <php_error_cb>, printf_function = 0x7f34ee2e7f90 <php_printf>, write_function = 0x7f34ee2e7f80 <php_output_wrapper>, fopen_function = 0x7f34ee2e86d0 <php_fopen_wrapper_for_zend>, message_handler = 0x7f34ee1d0712 <php_message_handler_for_zend>,
          get_configuration_directive = 0x7f34ee2e86c0 <php_get_configuration_directive_for_zend>, ticks_function = 0x7f34ee2f8be0 <php_run_ticks>, on_timeout = 0x7f34ee2e7f70 <php_on_timeout>, stream_open_function = 0x7f34ee2e8b20 <php_stream_open_for_zend>, vspprintf_function = 0x7f34ee2ee530 <vspprintf>, vstrpprintf_function = 0x7f34ee2ee660 <vstrpprintf>,
          getenv_function = 0x7f34ee2f3240 <sapi_getenv>, resolve_path_function = 0x7f34ee2e86b0 <php_resolve_path_for_zend>}
        zuv = {import_use_extension = 0x40 <error: Cannot access memory at address 0x40>, import_use_extension_length = 31558560, html_errors = 254 '\376'}
        retval = 0
        php_os = 0x7f34ee411203 "Linux"
#2  0x00007f34ee3e2fe5 in php_apache2_startup (sapi_module=<optimized out>) at /build/php7.1-hzF8RG/php7.1-7.1.0/sapi/apache2handler/sapi_apache2.c:385
No locals.
#3  0x00007f34ee3e3bd8 in php_apache_server_startup (pconf=0x7f34f453c028, plog=<optimized out>, ptemp=<optimized out>, s=0x7f34f450df28) at /build/php7.1-hzF8RG/php7.1-7.1.0/sapi/apache2handler/sapi_apache2.c:498
        data = 0x1
        userdata_key = 0x7f34ee4393d3 "apache2hook_post_config"
#4  0x00007f34f4308809 in ap_run_post_config ()
No symbol table info available.
#5  0x00007f34f42e8e68 in main ()
No symbol table info available.
(gdb)
 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Fri Aug 23 15:01:27 2019 UTC