php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #45945 Apache byterange output filter nullified if mod_php5 output > 8000 bytes
Submitted: 2008-08-29 04:15 UTC Modified: 2021-10-28 09:26 UTC
Votes:32
Avg. Score:4.0 ± 1.1
Reproduced:22 of 22 (100.0%)
Same Version:17 (77.3%)
Same OS:8 (36.4%)
From: djimenez at conduit-it dot com Assigned: cmb (profile)
Status: Not a bug Package: Apache2 related
PHP Version: 5.*, 6CVS (2009-07-15) OS: Ubuntu
Private report: No CVE-ID: None
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If this is not your bug, you can add a comment by following this link.
If this is your bug, but you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: djimenez at conduit-it dot com
New email:
PHP Version: OS:

 

 [2008-08-29 04:15 UTC] djimenez at conduit-it dot com
Description:
------------
Apache2 supports byte range response with an output filter. Static files 
over 8000 bytes work fine, but mod_php5 served content only works 
correctly if it is <= 8000 bytes

To test I used telnet to submit an HTTP Range request for the included 
PHP file from an apache2 server configured with mod_php5. Example 
requests are in the actual results.






Reproduce code:
---------------
<?php

// upto and including 8000 bytes will allow the byterange filter to work
//
// NOTE: 8000 byte limit was found by manual trial and error, may vary per system, but
// my guess would be it has to do with a byte buffer either in mod_php5 or apache2
echo str_repeat(".", 8000);

// anything over 8000 bytes will "fail" (full response)
//echo "\n...one toke over the line";

?>

Expected result:
----------------
Apache's byterange output filter should handle range requests for PHP 
responses > 8000 bytes.





Actual result:
--------------
While the test scripts second echo is commented, I get the expected 
results:
 
$ telnet dev.conduit-it.com 80
Trying 10.42.84.2...
Connected to dev.conduit-it.com.
Escape character is '^]'.
GET /test.php HTTP/1.1
Host:dev.conduit-it.com
Range:bytes=0-24
Connection:close

HTTP/1.1 206 Partial Content
Date: Fri, 29 Aug 2008 03:43:20 GMT
Content-Range: bytes 0-24/8000
Content-Length: 25
Connection: close
Content-Type: text/html

.........................Connection closed by foreign host.

We can also see it working for multiple ranges:
  
$ telnet dev.conduit-it.com 80
Trying 10.42.84.2...
Connected to dev.conduit-it.com.
Escape character is '^]'.
GET /test.php HTTP/1.1
Host:dev.conduit-it.com
Range:bytes=0-24,50-74
Connection:close

HTTP/1.1 206 Partial Content
Date: Fri, 29 Aug 2008 03:45:44 GMT
Content-Length: 240
Connection: close
Content-Type: multipart/byteranges; boundary=455911696d6f354a2


--455911696d6f354a2
Content-type: text/html
Content-range: bytes 0-24/8000

.........................
--455911696d6f354a2
Content-type: text/html
Content-range: bytes 50-74/8000

.........................
--455911696d6f354a2--
Connection closed by foreign host.


So apache is doing all the work for us, until we uncomment the second 
echo statement in the repro. script. This puts us over the 8000 byte 
limit.  At that point, both example requests will return full 200 OK 
responses (so content is 8000 periods + our message)





Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2008-08-31 00:32 UTC] jani@php.net
What have you set your output_buffering to? (check from phpinfo() output)
 [2008-08-31 01:48 UTC] djimenez at conduit-it dot com
output buffering was set to Off, though setting it to something else 
still had the same result with the test on my box.
 [2008-08-31 02:22 UTC] djimenez at conduit-it dot com
I used amazon ec2 to test and reconfirm this bug on a base install of 
apache2 and php5.2.6 in ubuntu intrepid ibex. No changes to the 
php.ini (which I've included without comments and empty sections):

[PHP]
engine = On
zend.ze1_compatibility_mode = Off

short_open_tag = On
asp_tags = Off
precision    =  12
y2k_compliance = On

output_buffering = Off

zlib.output_compression = Off
implicit_flush = Off

unserialize_callback_func=

serialize_precision = 100

allow_call_time_pass_reference = On

safe_mode = Off
safe_mode_gid = Off
safe_mode_include_dir =
safe_mode_exec_dir =
safe_mode_allowed_env_vars = PHP_
safe_mode_protected_env_vars = LD_LIBRARY_PATH
disable_functions =
disable_classes =

expose_php = On

max_execution_time = 30     ; Maximum execution time of each script, 
in seconds
max_input_time = 60 ; Maximum amount of time each script may spend 
parsing request data
memory_limit = 16M      ; Maximum amount of memory a script may 
consume (16MB)

error_reporting  =  E_ALL & ~E_NOTICE

display_errors = On
display_startup_errors = Off
log_errors = Off
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
track_errors = Off

variables_order = "EGPCS"
register_globals = Off
register_long_arrays = On
register_argc_argv = On
auto_globals_jit = On

post_max_size = 8M

magic_quotes_gpc = On
magic_quotes_runtime = Off
magic_quotes_sybase = Off

auto_prepend_file =
auto_append_file =

default_mimetype = "text/html"

doc_root =
user_dir =
enable_dl = Off

file_uploads = On
upload_max_filesize = 2M
allow_url_fopen = On
allow_url_include = Off
default_socket_timeout = 60

[Syslog]
define_syslog_variables  = Off

[mail function]
SMTP = localhost
smtp_port = 25

[SQL]
sql.safe_mode = Off

[ODBC]
odbc.allow_persistent = On
odbc.check_persistent = On
odbc.max_persistent = -1
odbc.max_links = -1
odbc.defaultlrl = 4096
odbc.defaultbinmode = 1

[MySQL]
mysql.allow_persistent = On
mysql.max_persistent = -1
mysql.max_links = -1
mysql.default_port =
mysql.default_socket =
mysql.default_host =
mysql.default_user =
mysql.default_password =
mysql.connect_timeout = 60
mysql.trace_mode = Off

[MySQLi]
mysqli.max_links = -1
mysqli.default_port = 3306
mysqli.default_socket =
mysqli.default_host =
mysqli.default_user =
mysqli.default_pw =
mysqli.reconnect = Off

[mSQL]
msql.allow_persistent = On
msql.max_persistent = -1
msql.max_links = -1

[PostgresSQL]
pgsql.allow_persistent = On
pgsql.auto_reset_persistent = Off
pgsql.max_persistent = -1
pgsql.max_links = -1
pgsql.ignore_notice = 0
pgsql.log_notice = 0

[Sybase]
sybase.allow_persistent = On
sybase.max_persistent = -1
sybase.max_links = -1
sybase.min_error_severity = 10
sybase.min_message_severity = 10
sybase.compatability_mode = Off

[Sybase-CT]
sybct.allow_persistent = On
sybct.max_persistent = -1
sybct.max_links = -1
sybct.min_server_severity = 10
sybct.min_client_severity = 10

[bcmath]
bcmath.scale = 0

[Informix]
ifx.default_host =
ifx.default_user =
ifx.default_password =
ifx.allow_persistent = On
ifx.max_persistent = -1
ifx.max_links = -1
ifx.textasvarchar = 0

ifx.byteasvarchar = 0
ifx.charasvarchar = 0
ifx.blobinfile = 0
ifx.nullformat = 0

[Session]
session.save_handler = files
session.use_cookies = 1
session.name = PHPSESSID
session.auto_start = 0
session.cookie_lifetime = 0
session.cookie_path = /
session.cookie_domain =
session.cookie_httponly = 
session.serialize_handler = php
session.gc_divisor     = 100
session.gc_maxlifetime = 1440
session.bug_compat_42 = 1
session.bug_compat_warn = 1
session.referer_check =
session.entropy_length = 0
session.entropy_file =
session.cache_limiter = nocache
session.cache_expire = 180
session.use_trans_sid = 0
session.hash_function = 0
session.hash_bits_per_character = 4

url_rewriter.tags = 
"a=href,area=href,frame=src,input=src,form=,fieldset="

[MSSQL]
mssql.allow_persistent = On
mssql.max_persistent = -1
mssql.max_links = -1
mssql.min_error_severity = 10
mssql.min_message_severity = 10
mssql.compatability_mode = Off
mssql.secure_connection = Off


[Tidy]
tidy.clean_output = Off

[soap]
soap.wsdl_cache_enabled=1
soap.wsdl_cache_dir="/tmp"
soap.wsdl_cache_ttl=86400
 [2009-07-15 19:01 UTC] djimenez at conduit-it dot com
I tested against nightly snapshot php5.3-200907151630 and still obtain 
the same results. The bug is probably in the php5 apache2 module.
 [2009-09-16 12:35 UTC] jani@php.net
Are you using the filter or handler module? What exactly do you put in 
your httpd.conf to enable PHP?
 [2009-09-16 14:15 UTC] djimenez at conduit-it dot com
Using the handler module:

LoadModule php5_module /usr/lib/apache2/modules/libphp5.so
<IfModule mod_php5.c>
  AddType application/x-httpd-php .php .phtml .php3 .html .inc .func 
.clss
  AddType application/x-httpd-php-source .phps
</IfModule>
 [2009-10-13 21:30 UTC] dylan at io dot com
I have found that using sing X-Sendfile will solve this issue:
http://tn123.ath.cx/mod_xsendfile/
(at least for Apache/mod_php).

Also even if this "bug" was fixed, I'm not sure the results would be 
very desirable.  Since there's no way to efficiently "fseek" to a 
particular byte in the script output, I assume mod_php would have to 
transfer the entire document internally for each chunk that was 
requested.
 [2010-03-24 10:02 UTC] thwien at gmx dot net
I can confirm this problem. Exactly over 8000 Bytes it does not work anymore. Also using the Apache module x-sendfile does not solve this error in my case. The one and only work-around in my case is a header call with location of a static url.

header('Location: http://any.url/toanyfile.html');
exit;

After this just Apache controls the range operator and it works as expected.
 [2012-11-15 11:49 UTC] richard_s_yeo at hotmail dot com
Has there been any update to this?
I am experiencing the same issue on Windows and Linux with 5.3.5 of php.
This is a very important issue for me and the workaround of using redirect won't 
work for me.
 [2020-09-13 11:51 UTC] masterofsql at emailfromgoogle dot com
I hope someone will find this useful
EXPLICITLY adding

header("Transfer-Encoding: chunked");

solved the problem (not sure how reliably though).
That Transfer-Encoding header was being added automatically even if i didn't set it - but the problem was there. Adding the header explicitly in PHP code solved it.
Tested using websniffer.cc, PHP 7.4.9 CentOS 7.8
 [2021-10-28 09:26 UTC] cmb@php.net
-Status: Open +Status: Not a bug -Assigned To: +Assigned To: cmb
 [2021-10-28 09:26 UTC] cmb@php.net
I can confirm the reported behavior.  However, doing the same
request without Range header sheds some light on what's happening,
namely that Apache buffers up to 8000 bytes, and if that buffer is
not full at the end of the request, a plain payload and a
Content-Length header are sent.  But if the buffer is full, Apache
sends a chunked payload and the respective Transfer-Encoding
header, so the Range request is ignored, what is permissible
according to RFC 7233[2].

You can increase the default buffer size by using mod_buffer[1],
if you desire so.

However, the proper solution is to handle Range request yourself,
where sensible.  It doesn't make sense to actually send a large
payload to the Web server, and to let the Web server only send a
small part of that to the client.  Instead, just send the
requested part to Apache in the first place.  Make sure that
Apache's buffer is large enough, and respond with a proper 416
otherwise.

[1] <https://httpd.apache.org/docs/trunk/mod/mod_buffer.html>
[2] <https://httpwg.org/specs/rfc7233.html#header.range>
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Wed Apr 24 00:01:32 2024 UTC