php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #77930 stream_copy_to_stream should use mmap more often
Submitted: 2019-04-23 07:31 UTC Modified: 2019-10-30 11:58 UTC
Votes:4
Avg. Score:4.8 ± 0.4
Reproduced:4 of 4 (100.0%)
Same Version:4 (100.0%)
Same OS:4 (100.0%)
From: maggus dot staab at googlemail dot com Assigned:
Status: Closed Package: Streams related
PHP Version: 7.2.17 OS:
Private report: No CVE-ID: None
View Developer Edit
 [2019-04-23 07:31 UTC] maggus dot staab at googlemail dot com 
Description:
------------
we got a bug report in our http client library https://github.com/sabre-io/http/pull/119 which suggests to change our call to stream_copy_to_stream() to stream in chunks with a size of 4MiB because its nearly 2x faster .

the reason is that when you only copy in 4MiB chunks php internally uses mmap.

wouldn't it make sense that stream_copy_to_stream() internally would do the chunking to speedup this use-case instead (on php-src level)?

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2019-04-23 07:35 UTC] maggus dot staab at googlemail dot com 
please find the very detailed bug report on https://github.com/sabre-io/http/pull/119
 [2019-10-30 11:23 UTC] nikic@php.net 
Is there any script that can be used to benchmark this?
 [2019-10-30 11:58 UTC] nikic@php.net 
After staring at this code for a bit, I think we should just drop the size limitation entirely, for a couple of reasons:

First, the limitation already doesn't trigger if you copy the whole file (i.e. use copy() or stream_copy_to_stream() and don't specify a length). This happens because length will be 0 at the time of the check and only later calculated based on the file size. This means that we're already completely blowing the length limit for what is is likely the most common case, and it doesn't seem like anyone complained about that.

Second, the premise of the code comment ("to avoid runaway swapping") seems incorrect to me. Because this performs a file-backed non-private mmap, no swap backing is needed for the mapping. Concerns over "memory usage" are also misplaced, as this is a virtual mapping.
 [2019-10-31 08:37 UTC] nikic@php.net 
Automatic comment on behalf of nikita.ppv@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=333d607d47ca6eec96637c1c8686591dec6f5a0b
Log: Fix bug #77930: Remove mmap limit
 [2019-10-31 08:37 UTC] nikic@php.net
-Status: Open +Status: Closed
 [2019-11-23 14:06 UTC] divinity76 at gmail dot com 
is it using sendfile() under the hood?
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Thu Nov 21 08:01:29 2024 UTC