|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #81092 Calling fflush before stream_filter_remove results in corrupted stream
Submitted: 2021-05-31 18:34 UTC Modified: 2021-06-07 14:08 UTC
From: mark at klb dot jp Assigned: cmb (profile)
Status: Closed Package: Bzip2 Related
PHP Version: 7.4 OS: Linux
Private report: No CVE-ID: None
 [2021-05-31 18:34 UTC] mark at klb dot jp
When using filters to output bzip2 compressed data, one may be tempted to call fflush() before stream_filter_remove() to ensure all data has been written.

This used to work fine, however since php8, this causes the generated stream to be corrupt.

Test script:
if ($_SERVER['argv'][1]??'' == 'output') {
        $stream = fopen('php://output', 'wb+');
        $filter = stream_filter_append($stream, 'bzip2.compress', STREAM_FILTER_WRITE, ['blocks' => 9, 'work' => 0]);
        fwrite($stream, random_bytes(8192));
        stream_filter_remove($filter); // should call dtor

$script = $_SERVER['argv'][0];

$res = shell_exec('php '.escapeshellarg($script).' output'); // should output bz encoded data

$data = bzdecompress($res);

Expected result:

Actual result:


Add a Patch

Pull Requests

Pull requests:

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2021-05-31 23:17 UTC] mark at klb dot jp
A couple notes:

I am not sure this is a bz2 issue, this could come from filters too. Outputting to php://output using a filter and fflush()ing before removing the filter are conditions for this bug to happen.

The bz2 stream generated is cut partially, using bzcat will for example output:

php bug.php output | bzcat -tvv
    [1: huff+mtf file ends unexpectedly
 [2021-06-03 08:08 UTC]
-Assigned To: +Assigned To: cmb
 [2021-06-03 12:42 UTC]
-Status: Assigned +Status: Verified -Package: Bzip2 Related +Package: Streams related -PHP Version: 8.0.6 +PHP Version: 7.4
 [2021-06-03 12:42 UTC]
I can confirm that the script outputs int(0), but I get the same
result with PHP-7.4 either (regardless of explicitly calling
fflush()).  Interestingly, using a str_repeat("*", 8192) instead
of random_bytes(8192) gives the expected result.  If I use the
zlib.deflate and gzinflate(), I get an explicit:

    Warning: gzinflate(): data error

So apparently a general stream (compression) filter issue.
 [2021-06-03 16:50 UTC] mark at klb dot jp
Somehow this issue does not happen for me with php 7.4, and we started to have that issue as we upgraded from 7.4 to 8.0.6, so I assumed it was a php8 issue.


$ php -v
PHP 7.4.14 (cli) (built: Mar  6 2021 04:12:14) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.14, Copyright (c), by Zend Technologies

(php 7.4.14 from Linux Gentoo, so might contain some patches)

$ php bug.php 

This seems to be related to large output of compressed data, so outputting something that compresses well (str_repeat, etc) will not cause this issue. random_bytes() generate a string with a lot of entropy, which will be difficult to compress and will result in larger compressed block(s).
 [2021-06-04 17:33 UTC]
-Package: Streams related +Package: Bzip2 Related
 [2021-06-04 17:33 UTC]
My first assessment of this bug report was done on Windows, where
I got more strange results, so I switched to Linux, where it
quickly turned out that this regression has been introduced by the
fix for bug #75776 (available as of PHP 7.4.17), and that there
are no issues with zlib.deflate.

> random_bytes() generate a string with a lot of entropy, which
> will be difficult to compress and will result in larger compressed
> block(s).

Indeed, that's the issue.

I'll investigate the Windows issues, and file bug reports if
appropriate, but lets keep these out of this ticket.
 [2021-06-07 14:05 UTC]
The following pull request has been associated:

Patch Name: Fix #81092: fflush before stream_filter_remove corrupts stream
On GitHub:
 [2021-06-07 14:08 UTC]
Argh, the erratic behavior on Windows was caused by shell_exec()
which uses text mode[1].  I've used a file for the regression
test; that fails without the patch, and works on Windows as well.

[1] <>
 [2021-06-08 13:41 UTC]
Automatic comment on behalf of cmb69
Log: Fix #81092: fflush before stream_filter_remove corrupts stream
 [2021-06-08 13:41 UTC]
-Status: Verified +Status: Closed
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Jun 20 16:01:30 2024 UTC