php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #60826 raw POST data missing with chunked encoding, FastCGI
Submitted: 2012-01-20 21:15 UTC Modified: 2012-08-17 08:12 UTC
Votes:18
Avg. Score:4.9 ± 0.2
Reproduced:13 of 13 (100.0%)
Same Version:10 (76.9%)
Same OS:6 (46.2%)
From: clarkwise at gmail dot com Assigned: ab
Status: Not a bug Package: CGI/CLI related
PHP Version: 5.3.9 OS: Windows XP
Private report: No CVE-ID:
 [2012-01-20 21:15 UTC] clarkwise at gmail dot com
Description:
------------
When a POST is sent with the header "Transfer-Encoding: chunked" and PHP 5.3 is 
running via FastCGI, $HTTP_RAW_POST_DATA is not set. In IIS, the receiving PHP 
process simply hangs and does not execute at all. If chunked encoding is not set, 
it executes successfully and $HTTP_RAW_POST_DATA is populated.

Comparing ISAPI to FastCGI (using PHP 5.2 which has both implementations), PHP 
ISAPI works fine with "Transfer-Encoding: chunked" but PHP FastCGI does not.

This issue also occurred running Linux/Apache with PHP 5.3 FastCGI. In that 
scenario, the PHP process did not completely hang, but $HTTP_RAW_POST_DATA and 
php://input were empty when the script executed.

Test script:
---------------
Two files, postsend.php and postreceive.php, can be found within the question here:

http://stackoverflow.com/questions/8899239/http-raw-post-data-not-being-populated-after-upgrade-to-php-5-3

Expected result:
----------------
$HTTP_RAW_POST_DATA and the php://input stream should contain the raw binary data 
that was sent in the POST.

Actual result:
--------------
On Windows/IIS, the PHP process hangs and does not execute.
On Linux/Apache, the PHP process executes but $HTTP_RAW_POST_DATA and php://input 
are empty.

Patches

IIS-FastCGI-transfer-encoding-chuncked (last revision 2012-04-11 13:52 UTC) by andres at fenestrae dot com)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2012-02-02 00:35 UTC] timo dot witte at googlemail dot com
I have the same problem on my machine and can confirm this bug!
i think this bug affects mod_spdy aswell, because it sends the POST data chunked! 
( http://code.google.com/p/mod-spdy/issues/detail?id=22 ).
 [2012-02-09 19:17 UTC] ab@php.net
I've additionally tested the two files on nginx with PHP as FCGI. The request simply doesn't go throught to PHP and nginx gives 401 back. I've attached gdb to the PHP and set a break in the accept loop - there was no reaction when using the two test files, but requests without data got down to PHP. So my conclusion were - apache simply ignores the request body where nginx gives a correct http status. Anyways, this has nothing to do with PHP. IIS, Apache and Nginx seem to have no implementation for chunked incoming body stuff, but they do handle that a different way.

Btw. Nginx has the "chunkin" module for such actions.
 [2012-02-29 09:42 UTC] pajoye@php.net
-Status: Open +Status: Assigned -Assigned To: +Assigned To: ab
 [2012-03-01 08:31 UTC] ab@php.net
@clarkwise at gmail dot com

So could you confirm it works with nginx+chunkin?

As well, it's was not hanging with IIS (v7.5) for me. Which IIS version do you 
have?
 [2012-03-01 20:24 UTC] clarkwise at gmail dot com
Since I am using Windows XP, I'm stuck using IIS 5.1. When I have the opportunity 
to try nginx w/ chunkin or IIS 7.5, I'll update with further details. Thanks.
 [2012-03-05 10:08 UTC] florian dot fernandez at navaho dot fr
Same problem here. Apache 2.2.15 and PHP 5.3.10 running as FastCGI.

POST datas sent with "Transfer-encoding: chunked" are not received as they should be. The raw request looks like this :

POST receiveChunk.php HTTP/1.1
User-Agent: PECL::HTTP/1.7.2 (PHP/5.3.8)
Host: www.test.fr
Accept: */*
Pragma: no-cache
Connection: keep-alive
Cache-Control: no-cache
Content-Length: 682668
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

3ff4
(here, the datas)
0

and the request as the receiving script sees it :

User-Agent: PECL::HTTP/1.7.2 (PHP/5.3.8)
Host: www.test.fr
Accept: */*
Pragma: no-cache
Connection: close
Cache-Control: no-cache


As you can see, the request body is empty.

However, this problem does not occur when PHP runs as mod_php under Apache. Problem confirmed under PHP 5.3.8 and 5.3.9.

I don't know if that is a PHP bug and maybe the developers could confirm or not that the request body are wrong before it reaches PHP. I think that it appears to be a mod_fcgid bug and should be reported as one to Apache bug list.


Here are the scripts to reproduce the bug :

sendChunk.php :
<?php
$fileContentToUp = 'ICAgICAgICAgIgICAgIA==';

$header_array = array(
	'Pragma' => 'no-cache',
	'Connection' => 'keep-alive',
	'Cache-Control' => 'no-cache',
	'Content-Length' => strlen($fileContentToUp) ,
	'Transfer-Encoding' => 'chunked'
	);

$options = array(headers => $header_array);

$r = new HttpRequest('receiveChunk.php', HTTP_METH_POST, $options);
$r->setContentType('text/html; charset=UTF-8');
$r->setBody($fileContentToUp);
$result = $r->send();

echo '$r->getRawRequestMessage() => ';
print_r($r->getRawRequestMessage());
echo "\n\n";
echo '$r->send()->getBody() => ';
print_r($r->send()->getBody());
echo "\n\n";
?>

receiveChunk.php :
<?php
$HttpResponse = new HttpResponse();
echo 'http_get_request_header => '."\n";
print_r($HttpResponse->getRequestHeaders());
echo "\n\n";
echo 'http_get_request_body => '."\n";
print_r($HttpResponse->getRequestBody());
echo "\n\n";
?>

Hope that helps.
 [2012-03-05 11:46 UTC] florian dot fernandez at navaho dot fr
I forgot to mention that this is on a Linux system
 [2012-03-09 09:33 UTC] florian dot fernandez at navaho dot fr
The problem seems to have been resolved with php 5.4.0 and apache 2.4.1. I can't reproduce it with this configuration and chunked http requests are now coming through to php now.

I am going to try with php 5.3.10 and apache 2.4.1 and give feedback on where it goes.
 [2012-03-09 09:48 UTC] pajoye@php.net
Try a snapshot please, 5.3.10 has nothing else but security fixes unrelated to 
this issue.
 [2012-03-13 05:58 UTC] florian dot fernandez at navaho dot fr
First of all, the problem is still here with PHP 5.4 and Apache 2.4.1, I previously made the mistake to precise the content-length in my headers.

So here is what I found. If you send chunked http request but do not send the content-length (as we should do using HTTP 1.1 protocol) the body part of the request is empty (php://input returns nothing). This result remains valid for the following configuration where PHP runs via FastCGI : 

PHP 5.3.10 - APACHE 2.4.1
PHP 5.3-201203101230 - APACHE 2.4.1
PHP 5.4 - APACHE 2.4.1
PHP 5.4-201203101230 - APACHE 2.4.1

The same configuration with PHP running as mod_php is fine, the body request is available through php://input.
 [2012-03-23 15:27 UTC] clarkwise at gmail dot com
Making the chunked request to a stock nginx 1.0.13 installation
on Ubuntu yields a "411 Length Required" message, but enabling
the ngx_chunkin module does indeed correctly serve the request
and populate php://input and $HTTP_RAW_POST_DATA.

This was done with both Ubuntu's distribution of
PHP 5.3.6 and the php5.4-201203221530 snapshot.
 [2012-03-27 08:52 UTC] florian dot fernandez at navaho dot fr
So, it is not a PHP problem ?
 [2012-03-27 13:34 UTC] ab@php.net
@florian dot fernandez at navaho dot fr

Looks like that. Could you try nginx+chunkin to confirm please?
 [2012-03-27 20:26 UTC] clarkwise at gmail dot com
I finally got my hands on Windows Server 2008 running IIS 7.0.
Same problem; when POST request is chunked, the response times out.

Perhaps I'm missing a setting that would enable the receipt of 
chunked incoming requests, but I have only read about settings
that chunk the outgoing response.
 [2012-04-03 13:02 UTC] florian dot fernandez at navaho dot fr
@ab at php dot net

Unfortunately, I won't be able to test with nginx because the system administrators won't let me install it (Only Apache is allowed). Sorry for the inconvenience. I'll try to open a new bug on Apache bug list or mod_fcigd bug list if I find it : maybe they'll look into it.
 [2012-04-11 13:24 UTC] andres at fenestrae dot com
The problem lies in php-cgi.exe.
I have been able to reproduce the problem on IIS 7.5.
In Fast-cgi the data is delivered using method similar to chunking.

In case of "Transfer-Encoding: chunked" IIS supplies Content-length:-1 in the request data, which is casted to uint in sapi_cgi_read_post. Causing the routine to read data until it has read the full 4G (2^32-1).
Also, both the methods sapi_cgi_read_post(cgi_main.c) and fcgi_read(fastcgi.c) try to read the complete buffer time after time, causing the end-marker (0-sized chunk) only to be seen when it would be at buffer-boundary. However the sapi_cgi_read_post funtion is called several times after the actual content has been read causing the fcgi_read method to block.
I expect similar cause to apply for apache.

Solution appears
1) do not loop in fcgi_read; this way when reading the final marker, it is returned to the calling code.
2) Set SG(request_info).content_length to SG(read_post_bytes) + read_bytes when tmp_read_bytes ==0 && SG(request_info).content_length == -1
 [2012-04-23 07:52 UTC] florian dot fernandez at navaho dot fr
@ andres at fenestrae dot com

Have you been able to try what you suggested in your last post ?
 [2012-04-23 08:00 UTC] andres at fenestrae dot com
florian dot fernandez at navaho dot fr 

I have been able to try my solution in combination with 5.2.17.

See the patches in this topic for the exact code-changes I made.
 [2012-04-23 08:08 UTC] florian dot fernandez at navaho dot fr
Sorry, I didn't see it before. I'll give it a try with 5.4.0
 [2012-04-23 10:59 UTC] florian dot fernandez at navaho dot fr
Maybe I did something wrong but the patch doesn't seem to work with PHP 5.4.0 on Linux.
 [2012-04-23 14:34 UTC] andres at fenestrae dot com
@florian dot fernandez at navaho dot fr

Florian,

I'm afraid i cannot help you any further on Linux.
The solution is not specific for Windows.
What I did to tackle the problem at first is to write a small program that ran instead of php-cgi which copied whatever it received on stdin to an other a mailslot.
This output I fed into php-cgi to reproduce an pinpoint the cause. 

Regards Andre
 [2012-05-30 21:10 UTC] dominic dot benson at thirdlight dot com
For Apache mod_fcgid, I have a patch available in the bug I filed at 
https://issues.apache.org/bugzilla/show_bug.cgi?id=53332

Essentially, PHP is following the spec, and not reading beyond the FastCGI 
CONTENT_LENGTH header value - so correct fixes are necessarily above PHP, in the 
FastCGI or web server implementation.
 [2012-08-17 08:12 UTC] ab@php.net
-Status: Assigned +Status: Not a bug
 [2012-08-17 08:12 UTC] ab@php.net
at the end of the day, this is not a php bug
 
PHP Copyright © 2001-2014 The PHP Group
All rights reserved.
Last updated: Fri Apr 18 18:01:58 2014 UTC