|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #71375 Memory not released from token_get_all()
Submitted: 2016-01-14 22:11 UTC Modified: 2016-01-15 06:53 UTC
From: pete at peteward dot net Assigned:
Status: Not a bug Package: Performance problem
PHP Version: 7.0.2 OS: Centos Linux 7.0
Private report: No CVE-ID: None
 [2016-01-14 22:11 UTC] pete at peteward dot net
Memory leaked from token_get_all() is much worse in php 7.x than in 5.x.

I have a large Symfony app with a container file string which when passed through token_get_all() hangs on to a large amount of it's consumed memory, despite the peak consumption being less.

I haven't managed to identify which tokens are causing this as a basic very large string didn't have this issue, however the test script below shows how it works with the 'real' content (a symfony container).

Test script:

$string = file_get_contents('');

function printMemory() {
    $mem = str_pad(round((memory_get_usage(true)/1024/1024)), 3, " ", STR_PAD_LEFT) . "MB " .
           str_pad(round((memory_get_peak_usage(true)/1024/1024)), 3, " ", STR_PAD_LEFT) . "MB ";
    print $mem . PHP_EOL;

function getAllTokensInScope($string) {
    $something = token_get_all($string);

Expected result:
Memory should be released outside of the scope of the function from which it's called.

PHP 5.4 and 5.6:

$ php -v
PHP 5.6.14-0+deb8u1 (cli) (built: Oct  4 2015 16:13:10) 
Copyright (c) 1997-2015 The PHP Group

$ php test.php 
  0MB   0MB 
  3MB   3MB 
 11MB 210MB 

Actual result:
Most of the memory is not released.

$ php -v
PHP 7.0.2 (cli) (built: Jan  9 2016 14:00:11) ( NTS )
Copyright (c) 1997-2015 The PHP Group

$ php test.php 
  2MB   2MB 
  7MB   7MB 
111MB 130MB 


Add a Patch

Pull Requests

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2016-01-15 04:40 UTC]
-Status: Open +Status: Not a bug
 [2016-01-15 04:40 UTC]
This isn't a bug, but a side effect of the new memory manager which caches more memory and doesn't release memory back to the OS so quickly.
Try setting the first parameter of memory_get_usage to false. You'll then see the memory PHP really needs.

Also, try calling gc_mem_caches() first before measuring. This releases unused memory segments.

This is intentional behavior, though it is possible that this gets quite extreme in cases like token_get_all().

See also bug #70098
 [2016-01-15 06:53 UTC] pete at peteward dot net
Thank you!
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Fri Apr 19 18:01:28 2024 UTC