php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #79519 Memory leak in PHP 7.4
Submitted: 2020-04-24 17:57 UTC Modified: 2021-12-09 19:54 UTC
Votes:23
Avg. Score:4.5 ± 0.8
Reproduced:22 of 22 (100.0%)
Same Version:18 (81.8%)
Same OS:13 (59.1%)
From: kieran at miami-nice dot co dot uk Assigned: nikic (profile)
Status: Closed Package: Performance problem
PHP Version: 7.4.5 OS: Linux
Private report: No CVE-ID: None
 [2020-04-24 17:57 UTC] kieran at miami-nice dot co dot uk
Description:
------------
I'm running the same code through PHP Unit 8.5.2 on PHP 7.3 and PHP 7.4 and there's a huge memory leak in 7.4. Several others have confirmed a similar issue in https://github.com/sebastianbergmann/phpunit/issues/3915

PHPUnit 8.5.2 - PHP 7.3.17 - Memory: 769.00 MB
PHPUnit 8.5.2 - PHP 7.4.5  - Memory: 3.89 GB

Both instances are running in slightly modified images of docker php. The same changes but based off php:7.3-cli-stretch and php:7.4-cli respectively. I can provide full php.ini of both which shows nothing special going on...

I have generated a blackfire profile to show the issue:
* PHP 7.3 - https://blackfire.io/profiles/6b973d98-014a-4dd2-8b13-f0cf5842221c/graph
* PHP 7.4 - https://blackfire.io/profiles/c80f648b-a778-4b3c-8acf-00a628e1630f/graph

Comparison of PHP 7.3 against 7.4 - https://blackfire.io/profiles/compare/a2dfa36c-3203-4599-959b-95d87143776a/graph

--

I do note that since first running the tests (poss. PHP 7.4.4 or earlier) it was returning 4.16 GB and on 7.4.5 it's now 3.89 GB so the memory leak fixes in recent releases has helped some what...

Test script:
---------------
Unfortunately I cannot provide access to the code. It's a large Laravel application.

I would be grateful if based on the memory profiles someone could point in the right direction to help debug the issue.

Expected result:
----------------
n/a

Actual result:
--------------
n/a

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2020-04-26 19:37 UTC] nikic@php.net
Not seeing anything obvious based on these profiles. Without a reproducer, the only suggestion I can make is to a) gather garbage collection statistics using gc_status() (maybe Blackfire already collects them and I can't find it) and b) create a memory profile using "USE_ZEND_ALLOC=0 valgrind --tool=massif php vendor/bin/phpunit" followed by "ms_print massif.*" (this will take a long time to run). This may or may not pinpoint the source of the leak.
 [2020-04-27 15:35 UTC] cmb@php.net
The PHP 7.3 memory profile states that the peak memory would be
761 MB, but for the Text::getConsecutiveWords the allocated memory
is reported as 4060 MB.  That appears to be contradictory.
 [2020-04-28 14:55 UTC] kieran at miami-nice dot co dot uk
> maybe Blackfire already collects them and I can't find it

Sent a message to Blackfire to see if they can provide any insight.

> The PHP 7.3 memory profile states that the peak memory would be
> 761 MB, but for the Text::getConsecutiveWords the allocated memory
> is reported as 4060 MB.  That appears to be contradictory.

I think this may be something else.

https://github.com/bytestream/php74-memory-leak

$ php7.3 vendor/bin/phpunit --repeat 100
Time: 3.78 seconds, Memory: 396.00 MB

$ php7.4 vendor/bin/phpunit --repeat 100
Time: 11.56 seconds, Memory: 734.00 MB

with gc_collect_cycles in tearDown - ref; https://github.com/fzaninotto/Faker/pull/1730

$ php7.3 vendor/bin/phpunit --repeat 100
Time: 4.71 seconds, Memory: 32.00 MB

$ php7.4 vendor/bin/phpunit --repeat 100
Time: 13.54 seconds, Memory: 34.00 MB

If I add gc_collect_cycles to the full app, PHP 7.4 memory usage remains unchanged at 3.89GB
 [2020-04-28 16:27 UTC] kieran at miami-nice dot co dot uk
Adding more routes to the test case seems to show some interesting results.

https://github.com/bytestream/php74-memory-leak/tree/routes

$ php7.2 vendor/bin/phpunit --repeat 1000
Time: 1.09 minutes, Memory: 144.00 MB

$ php7.3 vendor/bin/phpunit --repeat 1000
Time: 17.47 seconds, Memory: 138.00 MB

$ php7.4 vendor/bin/phpunit --repeat 1000
Time: 58.73 seconds, Memory: 510.00 MB
 [2020-04-28 17:33 UTC] kieran at miami-nice dot co dot uk
It's something to do with the use of require

Another example, different code, but same principle. 

https://github.com/bytestream/php74-memory-leak/tree/factories

$ php7.2 vendor/bin/phpunit --repeat 100
Time: 1.75 minutes, Memory: 322.00 MB

$ php7.3 vendor/bin/phpunit --repeat 100
Time: 1.02 minutes, Memory: 310.50 MB

$ php7.4 vendor/bin/phpunit --repeat 100
Time: 2.24 minutes, Memory: 1.32 GB
 [2020-04-30 11:37 UTC] maggus dot staab+php at gmail dot com
Could it be related to the recently fixed leak 

https://github.com/php/php-src/commit/3151676f520555bfadb39ea76779e93552d13fc1
 [2020-04-30 11:56 UTC] cmb@php.net
Please tests with PHP 7.4.6RC1 (has just been released).
 [2020-04-30 12:11 UTC] kieran at miami-nice dot co dot uk
https://github.com/bytestream/php74-memory-leak/tree/routes

$ ~/php7.4.6-rc.1/bin/php -d memory_limit=-1 vendor/bin/phpunit --repeat 1000
Time: 42.64 seconds, Memory: 520.00 MB

https://github.com/bytestream/php74-memory-leak/tree/factories

$ ~/php7.4.6-rc.1/bin/php -d memory_limit=-1 vendor/bin/phpunit --repeat 100
Time: 3.37 minutes, Memory: 1.45 GB

Another leak? Memory usage up in both cases compared to 7.4.5
 [2020-05-05 12:07 UTC] jerome dot vieilledent at blackfire dot io
It seems that my profile comparisons are not easy to share, so I did some screenshots which you can view here: https://github.com/bytestream/php74-memory-leak/issues/1
 [2020-05-08 18:23 UTC] kieran at supportpal dot com
Here's the result from valgrind on the CI server (nothing to do with my reproducers). Anything jump out Nikita? Not sure why it's blanked a lot of the lines - it's running with --enable-debug

$ USE_ZEND_ALLOC=0 valgrind --tool=massif php vendor/phpunit/phpunit/phpunit
 ==25== Massif, a heap profiler
 ==25== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
 ==25== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
 ==25== Command: php vendor/phpunit/phpunit/phpunit
 ==25== 
 PHPUnit 8.5.4 by Sebastian Bergmann and contributors.

https://gist.githubusercontent.com/bytestream/ad3270993599c720eb7db8ada597eb24/raw/fe94995d1aee1eb64a9e61da07810717fb32e0da/massif.out.25
 [2020-05-15 08:54 UTC] nikic@php.net
@kieran: The "factories" case looks like https://bugs.php.net/bug.php?id=76982, where a file declaring an anonymous function is included many times. The anonymous function declaration will leak in that case. This is a bug in PHP, but not one that will be fixed in 7.4. I'm not familiar with Laravel's factory system, but it can probably be addressed there by making that the includes are memoized.
 [2020-05-15 09:16 UTC] kieran at supportpal dot com
I thought it might be that too, but it seems odd that 7.4 is so much higher when that bug report suggests the issue exists in most of 7.x

Nothing in the ms_print in my previous message?

I'll check if includes can be memorised.
 [2020-05-15 09:22 UTC] nikic@php.net
@kieran: PHP 7.4 fixed some issues related to anonymous functions "overwriting" previous anonymous functions in the same file under some circumstances. Unfortunately this also means that this leak is now more pronounced, because this accidental destruction vector no longer exists.
 [2020-05-15 10:45 UTC] nikic@php.net
It looks like the cause for the "routes" case is the same. However, the "master" branch is a different issue. Here are the massif outputs for 7.3 and 7.4: https://gist.github.com/nikic/2a5d1a6eb4a4bb82be5e18225ae16d87
 [2020-05-15 10:53 UTC] kieran at supportpal dot com
Can ignore the master branch. That was an issue in Laravel that I pushed a fix for. I've just not updated that repository yet.
 [2020-05-15 10:58 UTC] nikic@php.net
I see. The master branch case is still potentially actionable from our side. The problem there is that GC runs too rarely. PHP 7.4 made this worse, because collecting objects with destructors requires two GC runs, and is seems Laravel makes use of destructors somewhere in there.

There's probably two things we can do to improve this at least:
1. Count any GC run with destructors as a "successful" run, so threshold is reduced.
2. Allow threshold reductions below the default of 10000, which seems too high for this case.

Alternatively we could always automatically rerun GC if we encountered destructors, instead of waiting for the next GC run.
 [2020-05-21 19:32 UTC] kieran at supportpal dot com
@nikita I think you're probably right:
> PHP 7.4 fixed some issues related to anonymous functions "overwriting" previous anonymous functions in the same file

I "memorised" the faker requires as suggested and it cut usage from 3.8G to 2.6G. So I assume similar changes elsewhere would further reduce memory usage. The only problem is anonymous function usage is rife in Laravel!

Not sure if you want to close this as duplicate of https://bugs.php.net/bug.php?id=76982 or repurpose following below comment:
> The master branch case is still potentially actionable from our side.
 [2020-08-10 20:19 UTC] kieran at supportpal dot com
Will there be a fix for this issue in PHP 8? Just tested against the PHP 8.0-beta.1 docker image and memory usage matches PHP 7.4
 [2020-08-10 20:30 UTC] nikic@php.net
I did some work on this in https://github.com/php/php-src/pull/5593 and https://github.com/php/php-src/pull/5595, but unfortunately it's not easy to fix.
 [2021-06-09 13:01 UTC] nikic@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: nikic
 [2021-06-09 13:01 UTC] nikic@php.net
In PHP 8.1, the memory leak for repeated included anonymous functions has been addressed by https://github.com/php/php-src/pull/5595 (or at least, mostly addressed).

The GC with destructors issue has been addressed by https://github.com/php/php-src/pull/5581.

I believe that covers the issues raised here.
 [2021-12-09 12:20 UTC] brendt at stitcher dot io
I want to follow up on this one, since the issue doesn't seem to be resolved.

Tested on PHP 8.1.0 and PHPUnit 9.5.10; running a Laravel test suite of 1400 tests results in a rather significant memory leak. Here's a screenshot of the memory consumption plotted in a chart: https://user-images.githubusercontent.com/6905297/145394833-dc488206-238d-4164-86e4-e2ad8c824590.png

This is the issue related to Laravel discussing the problem: https://github.com/laravel/framework/issues/30736
And here's the phpunit issue: https://github.com/sebastianbergmann/phpunit/issues/3915
 [2021-12-09 19:54 UTC] nikic@php.net
@brendt: Is this still the same issue? The fact that a leak occurs when running tests doesn't say a lot about what the root cause is.

PHP 8.1 fixes specifically the case where a file defining an anon function gets included many times, but there are variations on this that still leak (e.g. if it's an anon class instead).
 [2021-12-15 12:11 UTC] brendt at stitcher dot io
@nikic It's probably the other cases that haven't been fixed

> but there are variations on this that still leak (e.g. if it's an anon class instead).

Is this something I should open a bug report for if you're already aware of the issue?
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Oct 03 14:01:27 2024 UTC