php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #49816 output corruption using flush
Submitted: 2009-10-08 20:06 UTC Modified: 2010-08-07 13:53 UTC
Votes:14
Avg. Score:4.3 ± 0.9
Reproduced:14 of 14 (100.0%)
Same Version:12 (85.7%)
Same OS:7 (50.0%)
From: paul at wcclan dot net Assigned:
Status: Not a bug Package: Apache2 related
PHP Version: 5.* OS: *
Private report: No CVE-ID: None
 [2009-10-08 20:06 UTC] paul at wcclan dot net
Description:
------------
Looping through the output of a process opened with popen while using flush return "garbage" in the browser. Using telnet the intended output can be seen in (likely corrupted) chuncked encoding. Commenting out the flush() in below code fixes the problem.

Reproduce code:
---------------
<?
    $handle = popen("whois php.net", "r");

    if($handle)
    {
        while(!feof($handle))
        {
            $buffer = fgets($handle);
            echo $buffer . "<br />\r\n";
            flush();
        }
        pclose($handle);
    }
?>


Expected result:
----------------
the output of the command whois php.net

Actual result:
--------------
seemingly random garbage in browser. Above script available through: http://ipv6.wcclan.net/test.php

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2009-10-08 20:44 UTC] Sjoerd@php.net
Thank you for bug report.

The garbage data you see in the browser is the correct page, but compressed. The data is probably automatically compressed by Apache or PHP, without setting the correct headers. Because the headers are not set, the browser does not uncompress the page.

Please check your server settings and confirm this is the problem. There may still be a bug here, in that flush() breaks gzip compression. In that case we need some more information on how the compression is configured.
 [2009-10-08 21:05 UTC] paul at wcclan dot net
Thanks for the response. As far as I know I didn't make any changes to the compression settings. Anything specific I should look at? The request and headers are as follows:

GET /ipinfo HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, application/x-ms-application, application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap, */*
Accept-Language: nl
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC1; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
UA-CPU: AMD64
Accept-Encoding: gzip, deflate
Host: ipv6.wcclan.net
Connection: Keep-Alive

With flush in code (broken output):
HTTP/1.1 200 OK
Date: Thu, 08 Oct 2009 20:58:50 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8k PHP/5.2.11
X-Powered-By: PHP/5.2.11
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 664
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

Without flush in code (correct output):
HTTP/1.1 200 OK
Date: Thu, 08 Oct 2009 20:59:19 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8k PHP/5.2.11
X-Powered-By: PHP/5.2.11
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 750
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

Your hint about compression is correct. Both the requests are handled with compression. The code using flush returns "garbage" the code not using flush returns correct output.
 [2009-10-08 21:12 UTC] paul at wcclan dot net
possibly interresting php.ini setting:
zlib.output_compression = On
 [2009-10-09 17:07 UTC] paul at wcclan dot net
setting zlib.output_compression to Off fixes the problem. I have no output handlers defined. Is there anything more that might of interrest?
 [2009-10-10 16:57 UTC] alec at alec dot pl
Same here on Gentoo. Nothing was changed in configuration only PHP has been updated to 5.2.11 and my scripts stops working. zlib.output_compression=Off fixes issue, but also disables compression. Maybe important, I've got updated zlib library to version 1.2.3 in the same time.
 [2009-10-15 20:28 UTC] radek at pinkbike dot com
I have the same problem on Centos 5.2, php 5.2.11

Encoding bug using flush with zlib output handler.  
To reproduce...
Turn on zlib.output_compression = On
and load a page containing the following with a browser.

<?
echo "test";
flush();
?>

the result is garbled up 
?&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;*I-.&#65533;&#65533;&#65533;??&#65533;~?&#65533;&#65533;&#65533;

If you dont use flush() works correctly.
If you dont turn on zlib.output_compression works correctly.

Note that it's not simply user config error as it works correctly without flush().
 [2009-10-19 15:08 UTC] jani@php.net
What was the last PHP version it worked with..? (try with same libs!)
 [2009-10-19 15:15 UTC] jani@php.net
See also bug #48725
 [2009-10-20 06:57 UTC] alec at alec dot pl
The last PHP version it works with is 5.2.10. Compiled in the same system (libs).
 [2009-10-22 19:27 UTC] paul at wcclan dot net
To shed some more light on this I did the following:

- In the source directory I have of my old 5.2.10 php installation I executed make install clean
- Apache restart
- The test url now displays correctly in the browser. A request through telnet shows compression is also being applied.

- In the source directory of my 5.2.11 installation I executed make install clean
- Apache restart
- The test URL is displaying corrupt data again. A request through telnet shows the datastream is compressed or at least altered.

For this test I did notice something weird. With PHP 5.2.10 the response header is as follows:
HTTP/1.1 200 OK
Date: Thu, 22 Oct 2009 19:12:50 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8k PHP/5.2.10
X-Powered-By: PHP/5.2.10
Content-Encoding: gzip
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html

With PHP 5.2.11 the response header is as follows:
HTTP/1.1 200 OK
Date: Thu, 22 Oct 2009 19:23:13 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8k PHP/5.2.11
X-Powered-By: PHP/5.2.11
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html

So in this example there is a problem with the headers. Note that this is not only a header problem as a request for /phpinfo still returns the correct headers as in the example given before. The output is however still corrupted when using flush.
 [2009-10-22 19:49 UTC] paul at wcclan dot net
I installed php 5.3.0. Here the bug doesn't appear either. Unfortunately I can't move to 5.3.0 yet due to some compatibility issues.
 [2009-10-23 12:35 UTC] jani@php.net
Please try using this snapshot:

  http://snaps.php.net/php5.3-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/

If it fails also with this snapshot, I think I broke it when fixing bug #49248 :(
 [2009-10-23 21:35 UTC] srinatar@php.net
 this is reproduced with latest php 5.2 svn snapshot as well. rolling back to revision 287422 within main/SAPI.c seems to resolve this issue. 
 [2009-10-23 21:46 UTC] jani@php.net
Rolling back that is not an option. Need to figure out proper way to 
handle this.
 [2009-10-23 23:31 UTC] jani@php.net
Can not reproduce using lighttpd + PHP (FastCGI), need to test more.
Assuming this is Apache only issue for now.
 [2009-10-24 00:12 UTC] jani@php.net
Reproduced with Apache.
 [2009-10-24 00:21 UTC] jani@php.net
Apparently Apache2 is special. See bug #27424 (thanks Scott..)
 [2009-10-25 02:28 UTC] magnusf at users dot sourceforge dot net
Confirmed problem on Solaris 10 X86_64, PHP 5.2.11, Apache 2.2.13.
Disabling zlib output compression or reverting to 5.2.9 solves the problem. See application specific issue http://www.sugarcrm.com/forums/showthread.php?t=52694 when using 5.2.11 and flush. Problem with Apache 2.2.11 and PHP 5.2.11 so seems to be PHP issue, quote forum: "and the problem seems to rely on a change/bugfix? of the authentication/encryption mechanism in the latest 5.2.x and 5.3.x versions."
 [2009-10-26 13:05 UTC] jani@php.net
At least you can disable zlib compression in script now.. :/
See bug #49248, fix for that borked the compression on Apache if you use flush(). 
 [2009-11-14 22:28 UTC] jani@php.net
Caused by fixing bug #49248 and does not happen with HEAD.
 [2009-11-15 00:13 UTC] svn@php.net
Automatic comment from SVN on behalf of jani
Revision: http://svn.php.net/viewvc/?view=revision&revision=290765
Log: - Temporary hack to fix bug #49816 (works fine in HEAD which has working output buffering..)
 [2009-11-18 21:19 UTC] paul at wcclan dot net
I might be reading this wrong, but aren't you just opting not to compress with this code? If so, how do you explain that compression worked fine before 5.2.11? If I am reading it wrong, great job on the fix :)
 [2009-11-19 15:59 UTC] jani@php.net
Unfortunately flush() with apache2handler SAPI will also disable compression. Like I've said in the commit it's temporary "fix" to at least keep things working. The real fix requires doing MFH of new output buffering code where this works fine.

Johannes, please reply, I've asked you several times now whether I can merge the new and working output buffering code from HEAD (like I already suggested before 5.3.0 was released!).
 [2010-01-30 15:39 UTC] ben at xnode dot org
Still experiencing this issue with PHP 5.3.1 and Apache 2.2.14. Quite annoying as it means either not being able to use Flush (which isn't an option for some apps) or not being able to use compression.
 [2010-08-07 13:26 UTC] johannes@php.net
-Status: Assigned +Status: Bogus -Assigned To: johannes +Assigned To:
 [2010-08-07 13:26 UTC] johannes@php.net
The new output buffering is in trunk, it won't be ported to 5.3.
 [2010-08-07 13:53 UTC] paul at wcclan dot net
with php 5.3.3 the "just don't compress" solution seems to be in place so at least it doesn't cause any problems. However, shouldn't flush() also work when applying compression?
 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Thu Dec 12 21:01:26 2019 UTC