php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #77930 stream_copy_to_stream should use mmap more often
Submitted: 2019-04-23 07:31 UTC Modified: 2019-10-30 11:58 UTC
Votes:4
Avg. Score:4.8 ± 0.4
Reproduced:4 of 4 (100.0%)
Same Version:4 (100.0%)
Same OS:4 (100.0%)
From: maggus dot staab at googlemail dot com Assigned:
Status: Closed Package: Streams related
PHP Version: 7.2.17 OS:
Private report: No CVE-ID: None
 [2019-04-23 07:31 UTC] maggus dot staab at googlemail dot com
Description:
------------
we got a bug report in our http client library https://github.com/sabre-io/http/pull/119 which suggests to change our call to stream_copy_to_stream() to stream in chunks with a size of 4MiB because its nearly 2x faster .

the reason is that when you only copy in 4MiB chunks php internally uses mmap.

wouldn't it make sense that stream_copy_to_stream() internally would do the chunking to speedup this use-case instead (on php-src level)?


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2019-04-23 07:35 UTC] maggus dot staab at googlemail dot com
please find the very detailed bug report on https://github.com/sabre-io/http/pull/119
 [2019-10-30 11:23 UTC] nikic@php.net
Is there any script that can be used to benchmark this?
 [2019-10-30 11:58 UTC] nikic@php.net
After staring at this code for a bit, I think we should just drop the size limitation entirely, for a couple of reasons:

First, the limitation already doesn't trigger if you copy the whole file (i.e. use copy() or stream_copy_to_stream() and don't specify a length). This happens because length will be 0 at the time of the check and only later calculated based on the file size. This means that we're already completely blowing the length limit for what is is likely the most common case, and it doesn't seem like anyone complained about that.

Second, the premise of the code comment ("to avoid runaway swapping") seems incorrect to me. Because this performs a file-backed non-private mmap, no swap backing is needed for the mapping. Concerns over "memory usage" are also misplaced, as this is a virtual mapping.
 [2019-10-31 08:37 UTC] nikic@php.net
Automatic comment on behalf of nikita.ppv@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=333d607d47ca6eec96637c1c8686591dec6f5a0b
Log: Fix bug #77930: Remove mmap limit
 [2019-10-31 08:37 UTC] nikic@php.net
-Status: Open +Status: Closed
 [2019-11-23 14:06 UTC] divinity76 at gmail dot com
is it using sendfile() under the hood?
 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Mon Dec 09 05:01:26 2019 UTC