php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #81659 stream_get_contents() may unnecessarily overallocate
Submitted: 2021-11-26 04:17 UTC Modified: 2021-11-26 12:46 UTC
From: terence-marks at zencontrol dot com Assigned: cmb (profile)
Status: Closed Package: Streams related
PHP Version: 7.4 OS: Debian Buster
Private report: No CVE-ID: None
 [2021-11-26 04:17 UTC] terence-marks at zencontrol dot com
Description:
------------
Passing length of -1 to stream_get_contents() with the file position indicator of resource near the end of file still loads the entire file into memory. Passing a fixed length to stream_get_contents() does not produce the same behaviour.

Note: when running test script set memory_limit=64M

Test script:
---------------
<?php

$resource = fopen('/tmp/file1', 'a+');

for ($i = 0; $i < 64; ++$i)
{
    $data = random_bytes(1024 * 1024);
    $position = ftell($resource);
    fwrite($resource, $data);
    fseek($resource, $position);
    stream_get_contents($resource, strlen($data));
}

$resource = fopen('/tmp/file2', 'a+');

for ($i = 0; $i < 64; ++$i)
{
    $data = random_bytes(1024 * 1024);
    $position = ftell($resource);
    fwrite($resource, $data);
    fseek($resource, $position);
    stream_get_contents($resource, -1);
}


Actual result:
--------------
PHP Fatal error:  Allowed memory size of 67108864 bytes exhausted (tried to allocate 62922784 bytes) in script.php on line 22


Patches

Pull Requests

Pull requests:

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2021-11-26 05:41 UTC] requinix@php.net
-Status: Open +Status: Analyzed
 [2021-11-26 05:41 UTC] requinix@php.net
PHP doesn't actually read the whole file, but will attempt to allocate enough memory to hold the whole thing: there's a small optimization[1] that recognizes it knows the length of the stream and so can make an educated guess as to how much it might need to read to get everything, however that isn't taking into account that the current position in the stream can reduce that number by quite a bit.

[1] https://github.com/php/php-src/blob/PHP-8.0.13/main/streams/streams.c#L1493
 [2021-11-26 12:14 UTC] cmb@php.net
-Assigned To: +Assigned To: cmb
 [2021-11-26 12:46 UTC] cmb@php.net
-Summary: stream_get_contents() loading entire file into memory +Summary: stream_get_contents() may unnecessarily overallocate -PHP Version: 8.0.13 +PHP Version: 7.4
 [2021-11-26 12:46 UTC] cmb@php.net
The following pull request has been associated:

Patch Name: Fix #81659: stream_get_contents() may unnecessarily overallocate
On GitHub:  https://github.com/php/php-src/pull/7693
Patch:      https://github.com/php/php-src/pull/7693.patch
 [2021-11-29 13:51 UTC] git@php.net
Automatic comment on behalf of cmb69
Revision: https://github.com/php/php-src/commit/31749aac62a59fb215018ccb8099e1547182cdfc
Log: Fix #81659: stream_get_contents() may unnecessarily overallocate
 [2021-11-29 13:51 UTC] git@php.net
-Status: Analyzed +Status: Closed
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Nov 21 15:01:30 2024 UTC