php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #71375 Memory not released from token_get_all()
Submitted: 2016-01-14 22:11 UTC Modified: 2016-01-15 06:53 UTC
From: pete at peteward dot net Assigned:
Status: Not a bug Package: Performance problem
PHP Version: 7.0.2 OS: Centos Linux 7.0
Private report: No CVE-ID: None
 [2016-01-14 22:11 UTC] pete at peteward dot net
Description:
------------
Memory leaked from token_get_all() is much worse in php 7.x than in 5.x.

I have a large Symfony app with a container file string which when passed through token_get_all() hangs on to a large amount of it's consumed memory, despite the peak consumption being less.

I haven't managed to identify which tokens are causing this as a basic very large string didn't have this issue, however the test script below shows how it works with the 'real' content (a symfony container).



Test script:
---------------
<?php

printMemory();
$string = file_get_contents('http://dev.radiancedaily.com/appDevDebugProjectContainer.txt');
printMemory();
getAllTokensInScope($string);
printMemory();

function printMemory() {
    $mem = str_pad(round((memory_get_usage(true)/1024/1024)), 3, " ", STR_PAD_LEFT) . "MB " .
           str_pad(round((memory_get_peak_usage(true)/1024/1024)), 3, " ", STR_PAD_LEFT) . "MB ";
    print $mem . PHP_EOL;
}

function getAllTokensInScope($string) {
    $something = token_get_all($string);
}

Expected result:
----------------
Memory should be released outside of the scope of the function from which it's called.

PHP 5.4 and 5.6:

$ php -v
PHP 5.6.14-0+deb8u1 (cli) (built: Oct  4 2015 16:13:10) 
Copyright (c) 1997-2015 The PHP Group

$ php test.php 
  0MB   0MB 
  3MB   3MB 
 11MB 210MB 


Actual result:
--------------
Most of the memory is not released.

$ php -v
PHP 7.0.2 (cli) (built: Jan  9 2016 14:00:11) ( NTS )
Copyright (c) 1997-2015 The PHP Group

$ php test.php 
  2MB   2MB 
  7MB   7MB 
111MB 130MB 



Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2016-01-15 04:40 UTC] bwoebi@php.net
-Status: Open +Status: Not a bug
 [2016-01-15 04:40 UTC] bwoebi@php.net
This isn't a bug, but a side effect of the new memory manager which caches more memory and doesn't release memory back to the OS so quickly.
Try setting the first parameter of memory_get_usage to false. You'll then see the memory PHP really needs.

Also, try calling gc_mem_caches() first before measuring. This releases unused memory segments.

This is intentional behavior, though it is possible that this gets quite extreme in cases like token_get_all().

See also bug #70098
 [2016-01-15 06:53 UTC] pete at peteward dot net
Thank you!
 
PHP Copyright © 2001-2020 The PHP Group
All rights reserved.
Last updated: Tue Mar 31 10:01:25 2020 UTC