php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #73265 Loading browscap.ini at startup causes high memory usage
Submitted: 2016-10-07 11:58 UTC Modified: 2017-01-02 22:53 UTC
From: spam2 at rhsoft dot net Assigned: nikic
Status: Closed Package: Performance problem
PHP Version: 7.0.14 OS: Linux
Private report: No CVE-ID:
 [2016-10-07 11:58 UTC] spam2 at rhsoft dot net
Description:
------------
we have two different inhouse cms-systems, while one is 46% faster with PHP7 the other one sucks terrible - are there things known by developers which are internally slower now while most other become afster and should be avoided?

my first guess was mbstring but it's not, i have setup a cli environment with both php-versions + modules and a shell wrapper "php5" and "php7" and until now i am not able to find any isolted peice of code with a PHP7 slowdown

it looks like others see similar results
https://laracasts.com/discuss/channels/general-discussion/is-it-just-me-or-is-php-7-slow

unluckily the cms developed by me is the faster one and now both have a difference of factor 30-36

Requests per second:    107.24 [#/sec] (mean)
Time per request:       279.741 [ms] (mean)
Time per request:       9.325 [ms] (mean, across all concurrent requests)
Transfer rate:          1531.03 [Kbytes/sec] received

Requests per second:    56.30 [#/sec] (mean)
Time per request:       532.864 [ms] (mean)
Time per request:       17.762 [ms] (mean, across all concurrent requests)
Transfer rate:          803.75 [Kbytes/sec] received

Test script:
---------------
impossible to provide :-(

Expected result:
----------------
have at least the same performance but not half

Actual result:
--------------
massive slowdown

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2016-10-07 13:40 UTC] nikic@php.net
Assuming you've already considered all the obvious causes (configuration and especially opcache), I'd suggest running your CMS through "perf record" + "perf report" and see what the top elements look like (on both versions). Additionally it wouldn't hurt to use strace to check whether there are any significant differences in IO/syscalls.
 [2016-10-07 14:05 UTC] spam2 at rhsoft dot net
configuration is 100% identical, even the CFLAGS and ./configure as well as RPM sub-pckaging, two days before the upgrade i "backported" the RPM-SPEC and deployed a new 5.5.26 build

something eats twice CPU it seems

a second VM on the same host has exactly twice requests per second with PHP7 which is caused by the fact the VM has 12 instead of only 6 cores

well, we will downgrade a developer machine on monday and try to compare xdebug outputs with PHP5 and PHP7, IMHO that should show changes in how expensive parts of the application are and lead to find the critical code paths

hopefully we can nail that down to specific native php-functions which would lead in isolted loop cases and reprocuder-sample code, i guess the php developers are also curious in which bordercases the results go in the exactly wrong direction
 [2016-12-14 23:36 UTC] spam2 at rhsoft dot net
-PHP Version: 7.0.11 +PHP Version: 7.0.14
 [2016-12-14 23:36 UTC] spam2 at rhsoft dot net
what i can assure you is that with 7.0.14 things got *much* worser and the system dropped down below 4 per second with "ab -c 20" and is raising OOM killers because each apache preforker in htop is showing 120 MB to 160 MB RES memory usage in htop while with PHP5.6 it was constantly around 50 MB

this can not be opcache (or at least should not be) because shared memory typically is not counted as "resident" for each process 

Requests per second:    3.85 [#/sec] (mean)
Time per request:       5198.089 [ms] (mean)
Time per request:       259.904 [ms] (mean, across all concurrent requests)
Transfer rate:          55.88 [Kbytes/sec] received
 [2016-12-14 23:39 UTC] spam2 at rhsoft dot net
and that memory usage is independet of load, here it is 00:38 AM and a few seconds after hard restart httpd all processes are afr above 100 MB, the fattest one cosumes 173 MB
 [2016-12-15 00:07 UTC] nikic@php.net
-Status: Open +Status: Feedback
 [2016-12-15 00:07 UTC] nikic@php.net
Please provide either a means to reproduce, or some profiles showing a difference. Profiles may be xdebug, perf, strace, or anything else that shows some kind of actionable cause of this problem. There is nothing we can do based on the information currently provided.
 [2016-12-15 00:21 UTC] spam2 at rhsoft dot net
-Status: Feedback +Status: Open
 [2016-12-15 00:21 UTC] spam2 at rhsoft dot net
xdebug looks normal

*how* do i find out what allocates that much memory inside of PHP and especially what made it *that* worse with 7.0.14

given that httpd-prefork regulary kills processes and forks news ones allocating so much memory explains the performance drop-down - i guess that are not memory leaks or at least not only and they are obviously not easy to debug, as shown even not with debug builds crying out loud at their own - see 
https://bugs.php.net/bug.php?id=72734

that below is not normal on a machine with no traffic and nearly anything loaded as shared extension

[root@prometheus:~]$ ps aux | grep httpd | grep -v grep
root       789  0.0  8.3 315396 127268 ?       Ss   Dez14   0:03 /usr/sbin/httpd -D FOREGROUND
apache     893  0.0  6.4 315140 98548 ?        S    Dez14   0:00 /usr/sbin/httpd -D FOREGROUND
apache   23994  0.0  6.3 315428 96600 ?        S    00:42   0:00 /usr/sbin/httpd -D FOREGROUND
apache   23995  0.0  6.3 315428 96600 ?        S    00:42   0:00 /usr/sbin/httpd -D FOREGROUND

[root@prometheus:~]$ php -m
[PHP Modules]
apcu
bz2
calendar
Core
ctype
curl
date
dom
exif
fileinfo
filter
gd
hash
iconv
json
libxml
mbstring
mysqli
mysqlnd
openssl
pcre
readline
Reflection
session
SimpleXML
soap
SPL
standard
tokenizer
xml
zlib

[root@prometheus:~]$ ls /lib64/php/modules/
insgesamt 6,4M
-rwxr-xr-x 1 root root  80K 2016-12-08 12:59 apcu.so
-rwxr-xr-x 1 root root  31K 2016-12-08 12:48 bcmath.so
-rwxr-xr-x 1 root root  28K 2016-12-08 12:48 calendar.so
-rwxr-xr-x 1 root root  15K 2016-12-08 12:48 ctype.so
-rwxr-xr-x 1 root root  91K 2016-12-08 12:48 curl.so
-rwxr-xr-x 1 root root 167K 2016-12-08 12:48 dom.so
-rwxr-xr-x 1 root root  59K 2016-12-08 12:48 exif.so
-rwxr-xr-x 1 root root 3,1M 2016-12-08 12:48 fileinfo.so
-rwxr-xr-x 1 root root  95K 2016-12-08 12:48 gd.so
-rwxr-xr-x 1 root root 151K 2016-12-08 12:48 hash.so
-rwxr-xr-x 1 root root  39K 2016-12-08 12:48 iconv.so
-rwxr-xr-x 1 root root  43K 2016-12-08 12:48 json.so
-rwxr-xr-x 1 root root 1,4M 2016-12-08 12:48 mbstring.so
-rwxr-xr-x 1 root root 131K 2016-12-08 12:48 mysqli.so
-rwxr-xr-x 1 root root 236K 2016-12-08 12:48 opcache.so
-rwxr-xr-x 1 root root 135K 2016-12-08 12:48 openssl.so
-rwxr-xr-x 1 root root  96K 2016-12-08 12:48 session.so
-rwxr-xr-x 1 root root  47K 2016-12-08 12:48 simplexml.so
-rwxr-xr-x 1 root root 431K 2016-12-08 12:48 soap.so
-rwxr-xr-x 1 root root  51K 2016-12-08 12:48 tidy.so
-rwxr-xr-x 1 root root  19K 2016-12-08 12:48 tokenizer.so
-rwxr-xr-x 1 root root  59K 2016-12-08 12:48 zip.so
 [2016-12-15 00:24 UTC] spam2 at rhsoft dot net
and that this simple shellscript running as systemd service with 7.1.0 and 7.0.14 consumes 102 MB is also not normal

[root@testserver:~]$ cat /usr/local/bin/check-dbmail-service.php
#!/usr/bin/php
<?php
 /** make sure we are running as shell-script */
 if(PHP_SAPI != 'cli')
 {
  exit('FORBIDDEN');
 }

 /** verify that port and binary-name are given */
 if(empty($_SERVER['argv'][1]) || empty($_SERVER['argv'][2]))
 {
  exit('USAGE: check-dbmail-service <port> <process-name>' . "\n");
 }

 /** delay monitoring for 30 seconds */
 sleep(30);

 /** service loop */
 while(1 == 1)
 {
  if(!check_service())
  {
   sleep(5);
   if(!check_service())
   {
    passthru('/usr/bin/killall -s SIGTERM ' . escapeshellarg($_SERVER['argv'][2]));
    usleep(750000);
    passthru('/usr/bin/killall -s SIGKILL ' . escapeshellarg($_SERVER['argv'][2]));
   }
  }
  sleep(30);
 }

 /**
  * check if service is available and responds
  *
  * @access public
  * @return boolean
 */
 function check_service()
 {
  $errno  = 0;
  $errstr = '';
  $fp = @fsockopen('tcp://127.0.0.1', $_SERVER['argv'][1], $errno, $errstr, /**$timeout*/5);
  if($fp)
  {
   $response = @fgets($fp, 128);
   @fclose($fp);
   if(!empty($response))
   {
    return true;
   }
   else
   {
    return false;
   }
  }
  else
  {
   return false;
  }
 }
?>
 [2016-12-15 00:54 UTC] nikic@php.net
You can use

    USE_ZEND_ALLOC=0 valgrind --tool=massif php script.php

to profile the memory usage of PHP, and then run

    ms_print massif.out.NNNNNN

to convert it into a more readable output.

Maybe this will provide some insight into the problem.
 [2016-12-15 01:09 UTC] spam2 at rhsoft dot net
sadly no - there is not much output at all, see at bottom, before CTRL+C it cosumes around 220 MB memory, likely the valgrind overhead
____________________________

by the quoted comment in the bugreport below we hoped that the 50% dropdown could be solved with 7.0.14 - the opposite is true - is it possible there some bug and nobody noticed the large memory allocation?

i have currently only one machine running a while(true) loop to look if there are call files for asterisk which is around 30 MB and fine as well as our adminpanel in the same range - anything else running PHP 7.0.14/7.1.0 is around 100 MB and above

"check-dbmail-service.php" is far above 100 MB (CLI script) while the idle webserver on that machine "only" consumes 90 MB per forker


https://bugs.php.net/bug.php?id=72736
"It's not really necessary to bring mysql into this. Just allocating a bunch of strings without deallocating them in between is enough"
____________________________

[root@testserver:~]$ USE_ZEND_ALLOC=0 valgrind --tool=massif /usr/bin/php /usr/local/bin/check-dbmail-service.php 20143 dbmail-imapd
==8780== Massif, a heap profiler
==8780== Copyright (C) 2003-2015, and GNU GPL'd, by Nicholas Nethercote
==8780== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==8780== Command: /usr/bin/php /usr/local/bin/check-dbmail-service.php 20143 dbmail-imapd
==8780==
^C==8780==
==8780== Process terminating with default action of signal 2 (SIGINT)
==8780==    at 0x685BA97: kill (in /usr/lib64/libc-2.23.so)
==8780==    by 0x2F7BD8: ??? (in /usr/bin/php)
==8780==    by 0x40BB89: ??? (in /usr/bin/php)
==8780==    by 0x661BC2F: ??? (in /usr/lib64/libpthread-2.23.so)
==8780==    by 0x68EF9DF: __nanosleep_nocancel (in /usr/lib64/libc-2.23.so)
==8780==    by 0x68EF949: sleep (in /usr/lib64/libc-2.23.so)
==8780==    by 0x24251B: ??? (in /usr/bin/php)
==8780==    by 0x3C5FEF: ??? (in /usr/bin/php)
==8780==    by 0x3C3CF2: execute_ex (in /usr/bin/php)
==8780==    by 0x40CA1C: zend_execute (in /usr/bin/php)
==8780==    by 0x3A70F1: zend_execute_scripts (in /usr/bin/php)
==8780==    by 0x379500: php_execute_script (in /usr/bin/php)
==8780==
 [2016-12-15 01:14 UTC] nikic@php.net
Massif writes to a file, not to stdout...
 [2016-12-15 01:24 UTC] spam2 at rhsoft dot net
"Massif writes to a file, not to stdout..." - well, how should a serveradmin and php-developer (with PHP the language) know that....

http://access.thelounge.net/harry/massif.txt
 [2016-12-15 01:37 UTC] yohgaki@php.net
Much of time is spent on malloc.
This is wild guess

  --disable-huge-code-pages

configure option might help.
 [2016-12-15 01:48 UTC] spam2 at rhsoft dot net
no it won't - the binary is already compiled with --disable-huge-code-pages and keep in mind that this script is CLI, that setting affects opcode and as you can see in my previous comments for CLI opcache is even not loaded at all since the webservers have their own "php.ini" with PHPIniDir in the httpd.conf

you have the whole script in my comment from [2016-12-15 00:24 UTC] and there is no sane reason to allocate 100 MB memory for that

with some luck the php.spec makes it to that comment
_________________

%global dist         .fc%fedora.%(echo $(/usr/bin/date +%Y%m%d.%H%M)).rh
%global phpver        7
%global runselftest   1
%global pgo_build     1
%global lto_build     0
%global break_build   0
%global debug_build   0
%global extension_dir %{_libdir}/%{name}/modules

# macro 'php_debug_build' is global and re-used for pecl-builds linked against
# system updated with a php debug-build while 'debug_build' is local and
# allows to be combined with 'break_build' to generate temporary debug builds
# without installing them - both implicit disable 'pgo_build' and 'lto_build'
%if %php_debug_build
 %global _include_minidebuginfo 1
 %global RH_GCC_DEBUG_OPTION "-g3"
 %global RH_CONFIGURE_DEBUG_OPTION "enable-debug"
 %global RH_STRIP_BINARY 0
 %global pgo_build 0
 %global lto_build 0
%else
 %if %debug_build
  %global _include_minidebuginfo 1
  %global RH_GCC_DEBUG_OPTION "-g3"
  %global RH_CONFIGURE_DEBUG_OPTION "enable-debug"
  %global RH_STRIP_BINARY 0
  %global pgo_build 0
  %global lto_build 0
 %else
  %global _include_minidebuginfo 0
  %global RH_GCC_DEBUG_OPTION "-g0"
  %global RH_CONFIGURE_DEBUG_OPTION "disable-debug"
  %global RH_STRIP_BINARY 1
 %endif
%endif

# link-time-optimization needs '--disable-gcc-global-regs' for configure
%if %lto_build
 %global RH_LTO_CONFIGURE_OPTION "disable-gcc-global-regs"
%else
 %global RH_LTO_CONFIGURE_OPTION "enable-gcc-global-regs"
%endif

Summary:        PHP Scripting Language
Name:           php
Version:        7.0.14
Release:        1%{?dist}
License:        PHP/Zend/BSD
Group:          Development/Languages
URL:            http://www.php.net/
Source0:        http://www.php.net/distributions/php-%{version}.tar.xz
Source1:        opcache-zendoptimizer.php
Source2:        php-httpd-dummy.conf
Source3:        php-disabled-autotests.txt
Source4:        php-test-dirs.txt
Source5:        php-debug.ini
Patch1:         %{name}-dlopen.patch
Patch2:         %{name}-realpath-cache-openbasedir.patch
BuildRequires:  autoconf
BuildRequires:  automake
BuildRequires:  binutils
BuildRequires:  bison
BuildRequires:  bzip2-devel
BuildRequires:  curl-devel
BuildRequires:  cyrus-sasl-devel
BuildRequires:  freetype-devel
BuildRequires:  gcc-c++
BuildRequires:  gd-devel
BuildRequires:  gettext-devel
BuildRequires:  httpd-devel
BuildRequires:  krb5-devel
BuildRequires:  libc-client-devel
BuildRequires:  libedit-devel
BuildRequires:  libjpeg-turbo-devel
BuildRequires:  libpng-devel
BuildRequires:  libstdc++-devel
BuildRequires:  libtidy-devel
BuildRequires:  libtool-ltdl-devel
BuildRequires:  libxml2-devel
BuildRequires:  libzip-devel
BuildRequires:  openssl-devel
BuildRequires:  pam-devel
BuildRequires:  pcre-devel
BuildRequires:  perl
BuildRequires:  postfix
BuildRequires:  re2c
BuildRequires:  zlib-devel
Provides:       mod_%{name}
Requires:       %{name}-common

%description
PHP Scripting Language

%package        common
Summary:        common files

%package        cli
Summary:        command-line interface for php
Requires:       %{name}-common
Provides:       %{name}-cgi, %{name}-readline

%package        devel
Summary:        files needed for building extensions
Requires:       autoconf, automake, pcre-devel, re2c, %{name}-cli

%package        bcmath
Summary:        bcmath

%package        calendar
Summary:        calendar

%package        ctype
Summary:        ctype

%package        curl
Summary:        curl

%package        dom
Summary:        dom

%package        exif
Summary:        exif

%package        fileinfo
Summary:        fileinfo

%package        gd
Summary:        gd

%package        gettext
Summary:        gettext

%package        hash
Summary:        hash

%package        iconv
Summary:        iconv

%package        imap
Summary:        imap

%package        json
Summary:        json

%package        mbstring
Summary:        mbstring

%package        mysqli
Summary:        mysqli

%package        opcache
Summary:        opcache

%package        openssl
Summary:        openssl

%package        pcntl
Summary:        pcntl

%package        pdo
Summary:        pdo

%package        phar
Summary:        phar

%package        posix
Summary:        posix

%package        session
Summary:        session

%package        simplexml
Summary:        simplexml

%package        soap
Summary:        soap

%package        sockets
Summary:        socket

%package        tidy
Summary:        tidy

%package        tokenizer
Summary:        tokenizer

%package        xmlreader
Summary:        xmlreader

%package        xmlwriter
Summary:        xmlwriter

%package        zip
Summary:        zip

# https://bugzilla.redhat.com/show_bug.cgi?id=1371549
%description    bcmath
%description    calendar
%description    cli
%description    common
%description    ctype
%description    curl
%description    devel
%description    dom
%description    exif
%description    fileinfo
%description    gd
%description    gettext
%description    hash
%description    iconv
%description    imap
%description    json
%description    mbstring
%description    mysqli
%description    opcache
%description    openssl
%description    pcntl
%description    pdo
%description    phar
%description    posix
%description    session
%description    simplexml
%description    soap
%description    sockets
%description    tidy
%description    tokenizer
%description    xmlreader
%description    xmlwriter
%description    zip

%prep
%setup -q -n php-%{version}
%patch1 -p1
%patch2 -p1

# drop windows specific headers and fix bogus permissions
rm -f TSRM/tsrm_win32.h TSRM/tsrm_config.w32.h Zend/zend_config.w32.h ext/mysqlnd/config-win.h ext/standard/winver.h main/win32_internal_function_disabled.h main/win95nt.h
find . -name \*.[ch] -exec chmod 644 {} \;

# delete known failing or skipped tests
%if %runselftest
 xargs rm -f < %{SOURCE3}
%else
 xargs rm -rf < %{SOURCE4}
%endif

%build
# force use of system libtool and regenerate configure scripts
libtoolize --force --copy --quiet
cat `aclocal --print-ac-dir`/{libtool,ltoptions,ltsugar,ltversion,lt~obsolete}.m4 > build/libtool.m4
touch configure.in
./buildconf --force

# compiler and linker flags
%if %lto_build
 RH_LTO_FLAGS="-flto -fno-fat-lto-objects -fuse-ld=gold -fuse-linker-plugin"
%else
 RH_LTO_FLAGS="-fno-lto -fuse-ld=gold -fuse-linker-plugin"
%endif
export LOCAL_CFLAGS="-O3 %{RH_GCC_DEBUG_OPTION} -fstack-protector-strong --param=ssp-buffer-size=8 -fipa-pta -fira-loop-pressure -fivopts -fmerge-all-constants -fsemantic-interposition -ftree-loop-distribution -ftree-loop-if-convert -ftree-loop-if-convert-stores -ftree-loop-ivcanon -fvect-cost-model=unlimited -fwrapv -minline-all-stringops -fno-align-labels -fno-exceptions -fno-gcse -fno-math-errno -fno-strict-aliasing -Wno-pointer-sign -Wno-stack-protector $RH_LTO_FLAGS -Wa,--noexecstack"
export CFLAGS="%{optflags} $LOCAL_CFLAGS"
export CC="gcc $CFLAGS"
export CXXFLAGS="$CFLAGS"
export SH_LDFLAGS="-Wl,--as-needed -Wl,-z,now -Wl,-z,relro -Wl,-z,noexecstack -Wl,-z,nodump $CFLAGS"
export LDFLAGS="$SH_LDFLAGS -pie -fPIE"

# define extension directory
export EXTENSION_DIR=%{extension_dir}

# configure build process
./configure --quiet \
 --host=x86_64-redhat-linux \
 --build=x86_64-redhat-linux \
 --target=x86_64-redhat-linux \
 --prefix=%{_prefix} \
 --program-prefix= \
 --libdir=%{_libdir}/%{name} \
 --disable-all \
 --enable-bcmath=shared \
 --enable-calendar=shared \
 --enable-cli \
 --enable-ctype=shared \
 --enable-dom=shared \
 --enable-exif=shared \
 --enable-fileinfo=shared \
 --enable-filter \
 --enable-gd-native-ttf \
 --enable-hash=shared \
 --enable-inline-optimization \
 --enable-json=shared \
 --enable-libxml \
 --enable-mbregex \
 --enable-mbstring=shared \
 --enable-mysqlnd \
 --enable-opcache=shared \
 --enable-pcntl=shared \
 --enable-pdo=shared \
 --enable-phar=shared \
 --enable-posix=shared \
 --enable-re2c-cgoto \
 --enable-session=shared \
 --enable-shared \
 --enable-simplexml=shared \
 --enable-soap=shared \
 --enable-sockets=shared \
 --enable-tokenizer=shared \
 --enable-xml \
 --enable-xmlreader=shared \
 --enable-xmlwriter=shared \
 --enable-zip=shared \
 --with-apxs2=%{_bindir}/apxs \
 --with-bz2=%{_prefix} \
 --with-config-file-path=%{_sysconfdir} \
 --with-config-file-scan-dir=%{_sysconfdir}/%{name}.lounge.d \
 --with-curl=shared,%{_prefix} \
 --with-freetype-dir=%{_prefix} \
 --with-gd=shared,%{_prefix} \
 --with-gettext=shared,%{_prefix} \
 --with-iconv=shared \
 --with-imap-ssl=%{_prefix} \
 --with-imap=shared,%{_prefix} \
 --with-kerberos=%{_prefix} \
 --with-layout=GNU \
 --with-libdir=%{_lib} \
 --with-libedit=%{_prefix} \
 --with-libxml-dir=%{_prefix} \
 --with-libzip=%{_prefix} \
 --with-mysql-sock=%{_sharedstatedir}/mysql/mysql.sock \
 --with-mysqli=shared,mysqlnd \
 --with-openssl=shared,%{_prefix} \
 --with-pcre-regex=%{_prefix} \
 --with-pcre-jit \
 --with-pdo-mysql=shared,mysqlnd \
 --with-pic \
 --with-system-ciphers \
 --with-tidy=shared,%{_prefix} \
 --with-zlib \
 --with-zlib-dir=%{_prefix} \
 --disable-cgi \
 --disable-dmalloc \
 --disable-dtrace \
 --disable-gcov \
 --disable-gd-jis-conv \
 --disable-huge-code-pages \
 --disable-ipv6 \
 --disable-opcache-file \
 --disable-phpdbg \
 --disable-rpath \
 --disable-short-tags \
 --disable-static \
 --%{RH_LTO_CONFIGURE_OPTION} \
 --%{RH_CONFIGURE_DEBUG_OPTION}
if test $? != 0; then
 tail -500 config.log
 : configure failed
 exit 1
fi

# build php with 'profile-guided-optimization' when enabled
%if %pgo_build
 make %{?_smp_mflags} prof-gen
 /usr/bin/bash /rpmbuild/PHP-PGO/profile.sh --php_build $PWD
 make prof-clean
 make %{?_smp_mflags} prof-use
%else
 make %{?_smp_mflags}
%endif

# stop rpmbuild and output infos for usage of the temporary binaries
%if %break_build
 cp %{SOURCE5} "$PWD/php-debug.ini" > /dev/null
 EXTENSION_DIR="$PWD/modules"
 sed -i "s@__EXTENSION_DIR__@$EXTENSION_DIR@" "$PWD/php-debug.ini"
 echo -e "\n\n\e[1;31mBREAK-BUILD ENABLED\e[0m\n\n\e[1;32mPLEASE RUN:\e[0m    export PHP_INI_SCAN_DIR=$PWD\n\e[1;32mCLI-BINARY:\e[0m    $PWD/sapi/cli/php\n\e[1;32mAPACHE-MODULE:\e[0m $PWD/libs/libphp7.so\n\e[1;32mWEBSERVER:    \e[0m http://localhost:9000\n"
 /usr/bin/bash /rpmbuild/PHP-PGO/profile.sh  --php_build $PWD --webserveronly 1
 exit 1
%endif

# run test-suite when enabled
%if %runselftest
 ulimit -s 32712
 unset TZ LANG LC_ALL
 export LANG="C" TEST_PHP_EXECUTABLE="$PWD/sapi/cli/php" EXTENSION_DIR="$PWD/modules" PHP_INI_SCAN_DIR="$PWD/modules" NO_INTERACTION=1 MALLOC_CHECK_=2 MYSQL_TEST_HOST="localhost" MYSQL_TEST_SOCKET="/var/lib/mysql/mysql.sock" MYSQL_TEST_PORT="3306" MYSQL_TEST_USER="php_autotest" MYSQL_TEST_PASSWD="php_autotest" MYSQL_TEST_DB="php_autotest" PDO_MYSQL_TEST_DSN="mysql:host=localhost;dbname=php_autotest" PDO_MYSQL_TEST_SOCKET="/var/lib/mysql/mysql.sock" PDO_MYSQL_TEST_USER="php_autotest" PDO_MYSQL_TEST_PASS="php_autotest" PDO_MYSQL_TEST_ENGINE="MyISAM"
 cp %{SOURCE5} "$PWD/tmp-php.ini" > /dev/null
 sed -i "s@__EXTENSION_DIR__@$EXTENSION_DIR@" "$PWD/tmp-php.ini"
 $TEST_PHP_EXECUTABLE -n -c $PWD/tmp-php.ini $PWD/run-tests.php -n -c $PWD/tmp-php.ini
 find /var/www/sessiondata -user builduser -delete
%endif

%install
install -m 0755 -d %{buildroot}%{_sysconfdir} %{buildroot}%{_sysconfdir}/httpd/conf %{buildroot}%{_libdir}/httpd/modules %{buildroot}%{_datadir}/%{name}
install -m 0644 %{SOURCE1} %{buildroot}%{_datarootdir}/%{name}/zendoptimizer.php
install -m 0644 %{SOURCE2} %{buildroot}%{_sysconfdir}/httpd/conf/httpd.conf
INSTALL_ROOT=%{buildroot} make install

# remove unpackaged files
rm -rf %{buildroot}%{extension_dir}/*.a %{buildroot}%{_bindir}/{phptar} %{buildroot}%{_datadir}/pear %{buildroot}%{_libdir}/libphp%{phpver}.la %{buildroot}%{_mandir} %{buildroot}%{_sysconfdir}/httpd %{buildroot}%{_bindir}/phar %{buildroot}%{_bindir}/phar.phar

# strip binaries for non-debug builds
%if %RH_STRIP_BINARY
 strip -s --strip-unneeded %{buildroot}%{_libdir}/httpd/modules/libphp%{phpver}.so %{buildroot}%{extension_dir}/*.so %{buildroot}%{_bindir}/php
%endif

%files
%attr(755,root,root) %{_libdir}/httpd/modules/libphp%{phpver}.so

%files common
%dir %{_datadir}/%{name}
%dir %{_libdir}/%{name}
%dir %{extension_dir}

%files bcmath
%{extension_dir}/bcmath.so

%files calendar
%{extension_dir}/calendar.so

%files cli
%{_bindir}/php

%files ctype
%{extension_dir}/ctype.so

%files curl
%{extension_dir}/curl.so

%files devel
%{_bindir}/php-config
%{_bindir}/phpize
%{_includedir}/php
%{_libdir}/%{name}/build

%files dom
%{extension_dir}/dom.so

%files exif
%{extension_dir}/exif.so

%files fileinfo
%{extension_dir}/fileinfo.so

%files gd
%{extension_dir}/gd.so

%files gettext
%{extension_dir}/gettext.so

%files hash
%{extension_dir}/hash.so

%files iconv
%{extension_dir}/iconv.so

%files imap
%{extension_dir}/imap.so

%files json
%{extension_dir}/json.so

%files mbstring
%{extension_dir}/mbstring.so

%files mysqli
%{extension_dir}/mysqli.so

%files opcache
%{extension_dir}/opcache.so
%{_datarootdir}/%{name}/zendoptimizer.php

%files openssl
%{extension_dir}/openssl.so

%files pcntl
%{extension_dir}/pcntl.so

%files pdo
%{extension_dir}/pdo.so
%{extension_dir}/pdo_mysql.so

%files phar
%{extension_dir}/phar.so

%files posix
%{extension_dir}/posix.so

%files session
%{extension_dir}/session.so

%files simplexml
%{extension_dir}/simplexml.so

%files soap
%{extension_dir}/soap.so

%files sockets
%{extension_dir}/sockets.so

%files tidy
%{extension_dir}/tidy.so

%files tokenizer
%{extension_dir}/tokenizer.so

%files xmlreader
%{extension_dir}/xmlreader.so

%files xmlwriter
%{extension_dir}/xmlwriter.so

%files zip
%{extension_dir}/zip.so

%changelog
* Thu Dec 8 2016 Reindl Harald <h.reindl@thelounge.net>
- update to PHP 7.0.14
- add 'with-pcre-jit' for upcoming 7.1 to configure

* Sat Nov 26 2016 Reindl Harald <h.reindl@thelounge.net>
- make sure that 'selftest' is using the intermediate-binaries in every
  case which became obvious on a machine running 7.1.0RC6 and building
  7.0.14RC1 resulting in warnings about non-loadable extensions

* Thu Nov 10 2016 Reindl Harald <h.reindl@thelounge.net>
- update to PHP 7.0.13

* Sat Nov 5 2016 Reindl Harald <h.reindl@thelounge.net>
- added patch to enable 'realpath_cache' with enabled 'open_basedir'
  we disable 'symlink' and 'link' for years and so CVE-2006-5178 has no point
  http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-5178

* Sat Oct 15 2016 Reindl Harald <h.reindl@thelounge.net>
- macro 'lto_build' enables link-time-optimization which increases
  build memory-usage combined with profile-guided-optimization dramatically
  but may improve performance for production builds
- adds '--disable-gcc-global-regs' to configure-flags
- adds '-flto -fno-fat-lto-objects -fuse-ld=gold -fuse-linker-plugin' to compiler and linker flags

* Tue Oct 4 2016 Reindl Harald <h.reindl@thelounge.net>
- add '--disable-huge-code-pages' to configure
- performance seems to be unpredictable and in doubt faster without

* Sat Oct 1 2016 Reindl Harald <h.reindl@thelounge.net>
- macro 'break_build' starts a webserver on port 9000 with the temporary mod_php
- script '/rpmbuild/PHP-PGO/profile.sh' supports as second param 'onlywebserver'
  instead profiling and kill it afterwards so we can use it's generate-logic for
  'httpd.conf' and 'php.ini' with current locations in the working directory
- remove '-funroll-loops' from compiler-flags - enabled for pgo-builds by '-fprofile-use'
- add '-fno-exceptions -fvect-cost-model=unlimited -fipa-pta -fsemantic-interposition' to compiler flags
- add '-fwrapv' to compiler flags to avoid https://bugs.php.net/bug.php?id=71201
- compiler-flags for documentation purpose: '-m64 -O3 -g0 -fopenmp -mfpmath=sse -pipe -fomit-frame-pointer -finline-functions -fstack-protector-strong --param=ssp-buffer-size=8 -D_FORTIFY_SOURCE=2 -fipa-pta -fira-loop-pressure -fivopts -fmerge-all-constants -fsemantic-interposition -ftree-loop-distribution -ftree-loop-if-convert -ftree-loop-if-convert-stores -ftree-loop-ivcanon -fvect-cost-model=unlimited -fwrapv -minline-all-stringops -fno-align-labels -fno-exceptions -fno-gcse -fno-math-errno -fno-strict-aliasing -Wformat -Werror=format-security -Wno-pointer-sign -Wno-stack-protector'

* Thu Sep 29 2016 Reindl Harald <h.reindl@thelounge.net>
- change PGO sequence to use "make prof-gen; profile.sh; make prof-clean; make prof-use"
- remove "-Wno-coverage-mismatch" from compiler flags
- https://bugs.php.net/bug.php?id=73111

* Sun Sep 25 2016 Reindl Harald <h.reindl@thelounge.net>
- add '-fmerge-all-constants' to compiler flags

* Sat Sep 24 2016 Reindl Harald <h.reindl@thelounge.net>
- add '-fno-strict-aliasing -fno-align-labels -fno-gcse' to compiler flags
  https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
  Note: When compiling a program using computed gotos, a GCC extension, you may get better
  run-time performance if you disable the global common subexpression elimination pass by
  adding -fno-gcse to the command line

* Fri Sep 23 2016 Reindl Harald <h.reindl@thelounge.net>
- remove 'ftp', 'xmlrpc' and 'xslt' from build
- rework call of test-suite to use default-settings instead cloned /etc/php.ini
- removed useless options from configure now that we build with system-gd instead bundeled

* Wed Sep 21 2016 Reindl Harald <h.reindl@thelounge.net>
- define 'break_build' and local 'debug_build' macros to stop before tests and packaging
- ship a 'debug-ini.php' with extensions and 'extension_dir' in the working directory
- now we can use the built binaries without installing them on the system
- usecase: temporary debug-builds or tests without running the whole build-process
- output tells the needed env-export and how to call the binary
  export PHP_INI_SCAN_DIR=/home/builduser/rpmbuild/BUILD/php-7.0.11
  CLI-BINARY: /home/builduser/rpmbuild/BUILD/php-7.0.11/sapi/cli/php

* Sun Sep 18 2016 Reindl Harald <h.reindl@thelounge.net>
- use 'profile-guided-optimization' instead of 'link-time-optimization' for memory/optimize-balance
- define 'pgo_build' macro to disable 'profile-guided-optimization' temporary
- we run scripted requests from a temporary webserver between make-stages to profile our application
- benchmark our cms-system shows 7 percent higher performance for php7
- see: https://software.intel.com/en-us/blogs/2015/10/09/pgo-let-it-go-php
- link with 'nodump' for enhanced security
- remove 'mcrypt' from build (if-you-are-typing-the-word-mcrypt-into-your-code-you-are-doing-it-wrong)

* Sat Sep 17 2016 Reindl Harald <h.reindl@thelounge.net>
- move all extensions to sub-packages for minimzed dependencies

* Tue Sep 6 2016 Reindl Harald <h.reindl@thelounge.net>
- add '-fivopts -ftree-loop-if-convert -ftree-loop-if-convert-stores -ftree-loop-ivcanon' to compiler flags
- explicit export of 'CC' environment variable with CFLAGS for optimized builds
- cleanup ./configure params
- use system-gd and disable some autotests (now supported without patches)
- use system-libzip (now supported without patches)
- remove skipped and failing autotests to keep focus on new regressions

* Mon Aug 29 2016 Reindl Harald <h.reindl@thelounge.net>
- cleanup spec-file to build more or less 'vanilla' php with only own optimizations
- add 'php-httpd-dummy.conf' as temporary 'httpd.conf' at build-time for 'apxs' without hacks

* Sun Aug 28 2016 Reindl Harald <h.reindl@thelounge.net>
- add 're2c' to build-requires and '--enable-re2c-cgoto' to configure

* Sat Aug 27 2016 Reindl Harald <h.reindl@thelounge.net>
- build all extensions loadable to share them between httpd and cli

* Fri Aug 19 2016 Reindl Harald <h.reindl@thelounge.net>
- cleanup configure options
- start with '--disable-all' and enable used features
- build without unused modules - improves time for build and selftest
- add '--disable-opcache-file' for smaller builds since we use only the shm-cache
- add '-fno-math-errno -fira-loop-pressure -ftree-loop-distribution' to compiler flags
 [2016-12-15 10:20 UTC] spam2 at rhsoft dot net
extensions are also out of question - the tiny leeop-script from comment [2016-12-15 00:24 UTC] consumes 90 MB resident memory sceonds after start with all loadablke modules disabled

[root@testserver:~]$ php -m
[PHP Modules]
bz2
Core
date
filter
libxml
mysqlnd
pcre
readline
Reflection
SPL
standard
xml
zlib

[root@testserver:~]$ ps aux | grep php
dbmail   13097  0.5  1.4 128576 91064 ?        SNs  11:17   0:00 /usr/bin/php /usr/local/bin/check-dbmail-service.php 20143 dbmail-imapd
 [2016-12-15 11:25 UTC] nikic@php.net
Unfortunately, the massif profile is missing lots of symbols. If possible, can you rerun this with complete debug symbols?

Assuming this is not just an artifact of broken symbols, the profile does show that most of the memory usage seems to come from zend_parse_ini_file(), which would be very unusual. (This could also explain why you're seeing a difference in 7.0.14, because there were some changes to the ini parser.)

Can you please check whether running PHP without ini (-n) changes the situation?
 [2016-12-15 11:34 UTC] spam2 at rhsoft dot net
yeah i saw the repeatet ini stuff too and though "WTF when the script is already running" - the difference is 100 MB versus 10 MB memory usage

root     26262  0.0  0.1  44020 10256 pts/1    S<+  12:30   0:00 /usr/bin/php -n /usr/local/bin/check-dbmail-service.php 20143 dbmail-imapd
root     26267  5.2  1.7 277092 105008 pts/1   S<+  12:30   0:00 /usr/bin/php /usr/local/bin/check-dbmail-service.php 20143 dbmail-imapd

P.S: that's 7.1.0 since my testserver which is currently running 7.1 to test our software against but the behavior auf 7.0.14 is pretty identical
 [2016-12-15 11:41 UTC] spam2 at rhsoft dot net
since i have one instance with a normal memory usage maybe it depends on some specific part of my php.ini and you could even reprodcue the behavior with it easily on your machine?

 [root@testserver:~]$ cat /etc/php.ini 
[PHP]
default_charset                           = "ISO-8859-1"
zend.enable_gc                            = 0
zend.detect_unicode                       = 0
register_argc_argv                        = 0
register_globals                          = 0
always_populate_raw_post_data             = -1
upload_tmp_dir                            = "/var/www/uploadtemp"
open_basedir                              = ""
include_path                              = ".:/www/phpincludes:/usr/share/pear:/usr/share/php:/usr/share/php/php-reader"
error_log                                 = "/Volumes/dune/www-servers/_logs/php_error.log"
zlib.output_compression                   = 0
zlib.output_compression_level             = 4
max_execution_time                        = 120
max_input_time                            = 120
max_input_nesting_level                   = 32
memory_limit                              = "-1"
post_max_size                             = "100M"
upload_max_filesize                       = "100M"
file_uploads                              = 1
max_file_uploads                          = 30
allow_url_fopen                           = 1
allow_url_include                         = 0
realpath_cache_size                       = 64K
realpath_cache_ttl                        = 300
error_reporting                           = E_ALL
docref_root                               = "http://at.php.net/manual/de/"
docref_ext                                = ".php"
disable_functions                         = ""
disable_classes                           = ""
engine                                    = 1
short_open_tag                            = 1
asp_tags                                  = 0
y2k_compliance                            = 1
output_buffering                          = 0
output_handler                            = ""
implicit_flush                            = 0
expose_php                                = 0
display_errors                            = 1
display_startup_errors                    = 1
log_errors                                = 1
log_errors_max_len                        = 2048
html_errors                               = 0
track_errors                              = 0
warn_plus_overloading                     = 1
enable_dl                                 = 0
cgi.force_redirect                        = 0
cgi.rfc2616_headers                       = 0
fastcgi.impersonate                       = 1
ignore_repeated_errors                    = 0
ignore_repeated_source                    = 0
arg_separator.output                      = "&amp;"
arg_separator.input                       = "&"
variables_order                           = "EGPCS"
default_mimetype                          = "text/html"
default_socket_timeout                    = 10
auto_detect_line_endings                  = 0
unserialize_callback_func                 = ""
precision                                 = 14
serialize_precision                       = 17
user_agent                                = "PHP"
gd.jpeg_ignore_warning                    = 1

highlight.string                          = "#dd0000"
highlight.comment                         = "#ff8000"
highlight.keyword                         = "#007700"
highlight.bg                              = "#ffffff"
highlight.default                         = "#0000bb"
highlight.html                            = "#000000"

error_prepend_string                      = ""
error_append_string                       = ""
auto_prepend_file                         = ""
auto_append_file                          = ""

[Session]
session.save_path                         = "/var/www/sessiondata"
session.save_handler                      = "files"
session.use_cookies                       = 1
session.use_only_cookies                  = 1
session.use_strict_mode                   = 1
session.name                              = "LOUNGE_ID"
session.referer_check                     = ""
session.auto_start                        = 0
session.cookie_lifetime                   = 0
session.cookie_path                       = "/"
session.cookie_domain                     = ""
session.cookie_secure                     = 0
session.cookie_httponly                   = 1
session.serialize_handler                 = "php"
session.gc_probability                    = 0
session.entropy_file                      = "/dev/urandom"
session.entropy_length                    = 16
session.cache_limiter                     = "nocache"
session.cache_expire                      = 180
session.use_trans_sid                     = 0
session.bug_compat_42                     = 0
session.bug_compat_warn                   = 0
session.hash_function                     = 1
session.hash_bits_per_character           = 6
session.upload_progress.enabled           = 1
session.lazy_write                        = 1
session.sid_bits_per_character            = 5
session.sid_length                        = 40
url_rewriter.tags                         = "disabled"

[MySQLI]
mysqli.default_host                       = "localhost"
mysqli.default_port                       = 3306
mysqli.default_socket                     = "/var/lib/mysql/mysql.sock"
mysqli.default_user                       = ""
mysqli.default_password                   = ""
mysqli.reconnect                          = 1
mysqli.max_links                          = 300

[mysqlnd]
mysqlnd.collect_statistics                = 0
mysqlnd.collect_memory_statistics         = 0
mysqlnd.debug                             = 0
mysqlnd.net_read_timeout                  = 60
pdo_mysql.default_socket                  = /var/lib/mysql/mysql.sock

[Assertion]
assert.active                             = 0
assert.quiet_eval                         = 0

[Sockets]
sockets.use_system_read                   = 1

[bcmath]
bcmath.scale                              = 20

[soap]
soap.wsdl_cache_enabled                   = 1
soap.wsdl_cache_dir                       = "/var/www/uploadtemp"
soap.wsdl_cache_ttl                       = 5

[mbstring]
mbstring.language                         = "German"
mbstring.internal_encoding                = "ISO-8859-1"
mbstring.http_input                       = "auto"
mbstring.encoding_translation             = 0
mbstring.detect_order                     = "auto"
mbstring.func_overload                    = 0

[Date]
date.timezone                             = "Europe/Vienna"

[browscap]
browscap                                  = "/etc/php/browscap.ini"
 [2016-12-15 12:04 UTC] spam2 at rhsoft dot net
; [browscap]
; browscap = "/etc/php/browscap.ini"

and the memory usage goes down

that is the one from the servers and i think with that and my whole config in the last comment you should have a fine reproducer - will remove that from all machines now

http://access.thelounge.net/harry/browscap.ini.txt
 [2016-12-15 12:55 UTC] spam2 at rhsoft dot net
in fact browscap is the reason for the whole bugreport, obviously with or without the memory overhead bug get_browser() which is used by the CMS of my co-developer which dropped from 100 to 50 requests per second with PHP7

luckily he is using it within function_exists() and add it to disabled_functions boosted his system from 100 requests per second with PHP5.6 to 320 with PHP 7.0.14

Requests per second: 319.89 [#/sec] (mean)
Time per request: 62.522 [ms] (mean)
Time per request: 3.126 [ms] (mean, across all concurrent requests)
Transfer rate: 4644.90 [Kbytes/sec] received
 [2016-12-15 12:57 UTC] nikic@php.net
-Summary: PHP7 dramatically slower in some cases +Summary: Loading browscap.ini at startup causes high memory usage
 [2016-12-15 12:57 UTC] nikic@php.net
Okay, not terribly surprised that loading an 8MB ini file is going to use lots of memory.

From a quick test, performance of loading browscap.ini did not significantly change between 5.6 and 7.0. When you compared to PHP 5.6, were you also loading this browscap.ini?

There are some optimization opportunities here:

a) We shouldn't be loading the browscap.ini unless it is actually necessary. In fact, browscap.ini is already loaded lazily by get_browser() if the browscap ini option is only available during activation (but not startup). However, this mode has the disadvantage that the data is loaded per-thread and per-request. If the get_browser() function is actually used on every request, this is going to be (significantly) more expensive than loading it once. Ideally, we'd have lazy loading that still shares across requests and threads. The latter would require manual synchronization, but I think we could implement the former easily and then drop the startup loading. Threads are not relevant in most PHP deployments.

b) We can optimize the loading and in-memory representation of browscap.ini. As we use a generic ini parser that is certainly not optimized for this use-case the former may be hard, but there are probably some cheap wins for the memory representation (e.g. we could intern ini keys, which for browscap are repeating).
 [2016-12-15 13:27 UTC] spam2 at rhsoft dot net
i have updated it on 2016-11-19

* Do Dez 08 2016 Reindl Harald <h.reindl@thelounge.net>
- update to PHP 7.0.14

so i can't say it for 100% sure but i am very certain that 7.0.14 made this part much worser - what is *not* normal is zend_parse_ini_file() calling that often within a while(true) loop -> http://access.thelounge.net/harry/massif.txt

since that was a cli script with a loop i would expect that only happen *once* at startup and your comment (This could also explain why you're seeing a difference in 7.0.14, because there were some changes to the ini parser) could explain some parts of it
 [2016-12-15 16:02 UTC] spam2 at rhsoft dot net
> From a quick test, performance of loading browscap.ini 
> did not significantly change between 5.6 and 7.0. 
> When you compared to PHP 5.6, were you also loading 
> this browscap.ini?

*yes* - i synced the "php.spec" for 5.6 and 7.0 days before the upgrade so that it's drop in and only the loadmodule line in httpd needs to be changed

what we did was 

* restart webserver
* benchmark cms 1
* restart webserver
* benchmark cms 2
* note numbers

"dnf -y upgrade"

* restart webserver
* benchmark cms 1
* restart webserver
* benchmark cms 2
* note numbers

drop from 100 to 50 requests per second, well, that application is *using* get_browser() within a if(function_exists('get_browser')) - hence i was able to add it to 'disabled_functions' - so it's not only about loading browscap and a likely regression in 7.0.14 but also about get_browser() with the same "browscap.ini" became noticeable slower

PHP 5.6.26:
Requests per second:    107.24 [#/sec] (mean)
Time per request:       279.741 [ms] (mean)
Time per request:       9.325 [ms] (mean, across all concurrent requests)
Transfer rate:          1531.03 [Kbytes/sec] received

PHP 7.0.11 PGO:
Requests per second:    56.30 [#/sec] (mean)
Time per request:       532.864 [ms] (mean)
Time per request:       17.762 [ms] (mean, across all concurrent requests)
Transfer rate:          803.75 [Kbytes/sec] received

PHP 7.0.14 NON-BROWSCAP:
Requests per second:    327.45 [#/sec] (mean)
Time per request:       61.077 [ms] (mean)
Time per request:       3.054 [ms] (mean, across all concurrent requests)
Transfer rate:          4754.80 [Kbytes/sec] received
 [2016-12-16 16:38 UTC] spam2 at rhsoft dot net
so - now it is proven that get_browser() with PHP7 is magnitudes slower than with PHP5, i have stored the binaries and extensions of the two versions running before and after the upgrade and gave them identical configurations

PHP 5.6.26 (cli) (built: Oct  3 2016 12:45:59)
PHP 7.0.11 (cli) (built: Oct  5 2016 22:28:49)

______________________________

10 x get_browser('Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0');

[harry@srv-rhsoft:/data/scripts/php5-versus-7]$ ./test.sh
PHP5

real    0m5.876s
user    0m5.799s
sys     0m0.034s


PHP7

real    0m14.396s
user    0m14.216s
sys     0m0.085s
______________________________

100 x get_browser('Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0');

[harry@srv-rhsoft:/data/scripts/php5-versus-7]$ ./test.sh
PHP5

real    0m57.583s
user    0m57.180s
sys     0m0.052s


PHP7

real    2m23.615s
user    2m22.030s
sys     0m0.691s
______________________________

that's factor 2.4 and so in fact the asnwer to my initial question below is "get_browser()" which was used until yesterday on every inital request with no active session which is always true for "ab"-benchmarks

so one thing is the dramatical memory usage with 7.0.14 but in general the function got extremely slow with the same "browscap.ini"
_________________________

we have two different inhouse cms-systems, while one is 46% faster with PHP7 the other one sucks terrible - are there things known by developers which are internally slower now while most other become faster and should be avoided?
 [2016-12-16 16:45 UTC] nikic@php.net
get_browser() is probably slower in PHP 7 because of the PCRE JIT. Most of the time in get_browser() is spent is compiling regular expressions, and the JIT requires more complication time.

I have spent some time yesterday implementing many memory usage and performance optimizations for browscap, I'll probably put up a PR today. However, the changes are very extensive, so not sure if we target PHP 7.0 for that.
 [2016-12-16 16:57 UTC] spam2 at rhsoft dot net
wait - the JIT makes things slower?
i had assumed the opposite

so you tell me the change below should be the exactly opposite and with 7.1 configure can disable it while --without-pcre-jit don't exist for 7.0 and it's always enabled there?

* Thu Dec 8 2016 Reindl Harald <h.reindl@thelounge.net>
- update to PHP 7.0.14
- add 'with-pcre-jit' for upcoming 7.1 to configure
 [2016-12-16 17:29 UTC] spam2 at rhsoft dot net
indeed PHP 7.1 build with "--without-pcre-jit" is faster, while fast is relative given that the test did only 10 calls - what is the purpose of the JIT then and can this not be a runtime option for the cases where it brings a benefit?


<?php
 $loops = 10;
 for($count=1; $count<=$loops; $count++)
 {
  $x = get_browser('Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0');
 }
?>

[harry@srv-rhsoft:/scripts/php5-versus-7]$ ./test.sh
PHP 5.6
real    0m5.922s
user    0m5.861s
sys     0m0.027s

PHP 7.0
real    0m14.894s
user    0m14.716s
sys     0m0.079s


PHP 7.1
real    0m3.335s
user    0m3.285s
sys     0m0.031s
 [2016-12-16 18:31 UTC] nikic@php.net
It is a runtime option, see pcre.jit.

The PCRE JIT optimizes the case where a pattern is compiled once and used multiple times. Browscap instead uses a huge amount of patterns only once. In this case the overhead of JIT compilation is larger than the benefit during the matching of the pattern.
 [2016-12-16 19:01 UTC] nikic@php.net
PR up at https://github.com/php/php-src/pull/2242. Quoting the performance numbers:

> * According to massif, the peak memory usage of PHP drops from 85MB to 23MB.
> * The startup time of PHP drops from 0.19s to 0.10s.
> * The time of running the get_browser_basic.phpt test with this ini drops from 19s to 0.23s. (!!!)
 [2016-12-17 14:45 UTC] spam2 at rhsoft dot net
is the result of the prce-jit somewhere stored between requests?

i still try to understand what goes up behind the scenes

without the pcre-jit 3 preg_replace() and 2 preg_match() goes up from 1.22 / 0.54% runtime costs to 15% (higly optimized core-cms) and so the JIT makes a really perofrmance boost - but looking at the source i have 5 different patterns and so within the request there is hardly a reuse and only reuse within requests could explain a benefit

that would also explain the dramatical memory usage of browscap but on the other hand not the drop of get_browser() within a loop instead make if faster after the first call
 [2016-12-17 15:32 UTC] nikic@php.net
-Assigned To: +Assigned To: nikic
 [2016-12-17 15:32 UTC] nikic@php.net
Compiled patterns are cached across requests. The cache size is limited (IIRC 4k patterns). get_browser() with the current implementation and the large browscap.ini file matches significantly more than 4k patterns on each call, so you don't benefit from the caching (as patterns are displaced before they can be used again).
 [2016-12-17 15:34 UTC] spam2 at rhsoft dot net
thanks foor the feedback and especially for the patch - the numbers are damned impressive!
 [2016-12-17 17:43 UTC] spam2 at rhsoft dot net
BTW: shouldn't the whole 'browsecap.ini' parsing not only happen when 'get_browser' is the first time called - that would save the whole overhead on a lot of systems and especially for CLI scripts which don't make use of it at all
 [2016-12-17 20:12 UTC] nikic@php.net
For the server SAPIs it makes sense to load the browscap.ini at startup, so it's loaded only once. E.g. if you're using fpm this means it's loaded once and then will be shared (via cow) across all forked worker processes.

However, I agree that for the cli SAPI it doesn't make sense to load the browscap.ini at startup -- we don't gain anything by that, but increase memory usage and startup time even if get_browser() is never used.
 [2016-12-18 00:02 UTC] spam2 at rhsoft dot net
> For the server SAPIs it makes sense to load the browscap.ini at startup

honestly i doubt - in case a server has configured it but no application is using get_browser() you have the full overhead for no good reason

doing the same as happnes at startup but only on-demand when it#s first used would also free the servers where it is unused from the overhead and for the first get_browser() call it don't make a difference if that overhead happened at startup or on-demand
 [2017-01-02 22:53 UTC] nikic@php.net
-Status: Assigned +Status: Closed
 [2017-01-02 22:53 UTC] nikic@php.net
PR merged, this should resolve the performance issue and reduce memory usage.
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Fri Apr 28 14:01:35 2017 UTC