php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #69049 Unable to get output of slow process
Submitted: 2015-02-13 05:44 UTC Modified: 2015-02-22 04:22 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:1 (100.0%)
From: aserbulov at parallels dot com Assigned:
Status: No Feedback Package: Streams related
PHP Version: 5.5.21 OS: Windows
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2015-02-13 05:44 UTC] aserbulov at parallels dot com
Description:
------------
If you use proc_open to open slow process (it should write to stdout not immediately, but after some time), you never get process output.

Slow process emulation:
---
<?php
sleep(60); // Do some work
fwrite(STDOUT, 'done');
exit(0);
---

It is side effect of bug https://bugs.php.net/bug.php?id=51800 fix.

Test script:
---------------
$cmd = "\"C:/Program Files/php/php.exe\" process.php";
$status;
$stdout = "";
$pipes = [];

$descriptors = [
	0 => ["pipe", "r"],	// stdin
	1 => ["pipe", "w"],	// stdout
	2 => ["pipe", "w"],	// stderr
];
$process = proc_open($cmd, $descriptors, $pipes);
fclose($pipes[0]);
fclose($pipes[2]);

while (!feof($pipes[1])) $stdout .= fread($pipes[1], 1024);

fclose($pipes[1]);
$status = proc_close($process);

print_r(["status" => $status, "stdout" => $stdout]);

Expected result:
----------------
1. fread() function should read the pipe until the desired number of bytes has been read or reaches the end of the pipe.
2. feof() function should return true if fread() function reaches the end of the pipe.


Actual result:
--------------
1. fread() return empty string
2. feof($pipes[1]) return true after ~36 seconds.

Patches

stream_oef.diff (last revision 2015-02-13 12:39 UTC) by aserbulov at parallels dot com)
plain_wrapper.c.diff (last revision 2015-02-13 05:45 UTC) by aserbulov at parallels dot com)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2015-02-13 09:06 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-02-13 09:06 UTC] ab@php.net
Hi,

thanks for the report. Just removing that part doesn't seem to be a correct solution. Please reread the tickets you've linked for why it was done so.

With the particular issue, you can still implement it with a slow process. I'd suggest to not to close strerr but read from it simultaneously. You expect a keyword 'done' from the stdout, but that doesn't prevent you to read some kind of keep-alive stuff from stderr. You might also look up some example in the tests appended with the bug #51800.

Of course it's bad when not having control over process.php . I was thinking about introducing a userspace function to peek on the pipe content. Another possibility were to introduce one more option to proc_open which would set the expected timeout. But none of that is yet done, lets see whether it's needed at all.

Thanks.
 [2015-02-13 10:34 UTC] aserbulov at parallels dot com
-Status: Feedback +Status: Open
 [2015-02-13 10:34 UTC] aserbulov at parallels dot com
I understand, why it was done and think what you should implement another solution for bug https://bugs.php.net/bug.php?id=51800.

With the particular issue, I can't implement it with a slow process.
---
$process = proc_open($cmd, $descriptors, $pipes);
fclose($pipes[0]);

while (!feof($pipes[1]) || !feof($pipes[2])) {
    $stdout .= fread($pipes[1], 1024); // wait ~32 seconds, set eof and return empty string
    $stderr .= fread($pipes[2], 1024); // set eof and return empty string
}
fclose($pipes[1]);
fclose($pipes[2]);

status = proc_close($process);
---
 [2015-02-13 10:39 UTC] aserbulov at parallels dot com
while (!feof($pipes[1]) || !feof($pipes[2])) {
    $stdout .= fread($pipes[1], 1024); // wait ~32 seconds, set eof and return empty string
    $stderr .= fread($pipes[2], 1024); // wait ~28 seconds, set eof and return empty string
}
 [2015-02-13 10:53 UTC] aserbulov at parallels dot com
In any case, you should not break working code during bug https://bugs.php.net/bug.php?id=51800 fixing.
 [2015-02-13 11:27 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-02-13 11:27 UTC] ab@php.net
I mean doing it like this

process.php
<?php
sleep(30); // Do some work
fwrite(STDERR, "alive");
sleep(30); // Do some work
fwrite(STDOUT, 'done');
exit(0);
?>

test.php
<?php

$cmd = "\"" . PHP_BINARY . "\" " . dirname(__FILE__) . "/process.php";
$status;
$stdout = "";
$stderr = "";
$pipes = [];

$descriptors = [
        0 => ["pipe", "r"],     // stdin
        1 => ["pipe", "w"],     // stdout
        2 => ["pipe", "w"],     // stderr
];
$process = proc_open($cmd, $descriptors, $pipes);
fclose($pipes[0]);

while (!feof($pipes[1]) || !feof($pipes[2])) {
        $stdout .= fread($pipes[1], 1024);
        $stderr .= fread($pipes[2], 1024); /* throw away */
}

fclose($pipes[1]);
fclose($pipes[2]);
$status = proc_close($process);

print_r(["status" => $status, "stdout" => $stdout]);
 [2015-02-13 11:40 UTC] ab@php.net
@aserbulov, there are multiple issues with proc_open

- blocking read as file descriptors on windows don't support async, it's not possible to select()
- internal streams buffering, say you write 1024 but process.php outs only 4 bytes
- pipe buffer size which is way to small on windows, that leads to deadlocks

At the end line - it is much more likely to get a proc_open() call hanging before that patch. Not mentioning the cases where process.php (or any counterpart) dies leaking the descriptors - then we've no chance to check it at all. Now we discovered that there is this side effect, however please keep in mind that this topic is way too complicated. For now, maybe it'd make sense to increase the internal retry time to something like 60 seconds.

Cheers.
 [2015-02-13 12:37 UTC] aserbulov at parallels dot com
-Status: Feedback +Status: Open
 [2015-02-13 12:37 UTC] aserbulov at parallels dot com
Look at https://bugs.php.net/patch-display.php?bug_id=69049&patch=stream_oef.diff&revision=latest

It should fix particular issue...
 [2015-02-13 13:37 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-02-13 13:37 UTC] ab@php.net
With your patch, the snippet doesn't terminate.

Thanks.
 [2015-02-13 13:42 UTC] aserbulov at parallels dot com
-Status: Feedback +Status: Open
 [2015-02-13 13:42 UTC] aserbulov at parallels dot com
The patch https://bugs.php.net/patch-display.php?bug_id=69049&patch=stream_oef.diff&revision=latest does not fix particular issue - process hangs forever.

But the idea is correct - set eof to true only if end of the pipe is reached.
 [2015-02-13 13:44 UTC] ab@php.net
-Status: Open +Status: Feedback
 [2015-02-13 13:44 UTC] ab@php.net
But what is actually missing here is the blocking read. After the first circle it's 

errno 2, ret 0, eof 1

Because nothing was available in the pipe ... as we emulate the blocking read now, it might make sense to just increase the timeout.

Cheers.
 [2015-02-13 13:49 UTC] ab@php.net
@aserbulov, yeah, however in practice, if one tries to do the blocking read from the deadlocked pipe - here we are. Increasing timeout will probably add the idle circles in the simple cases though. Heh.
 [2015-02-13 13:58 UTC] ab@php.net
The pipe needs to be peeked, otherwise in much more cases this will just hang. Especially if there's a lot of data passed. And we cannot control the pipe buffer size.

Another solution I had in mind for this was using threads. However it's much more complicated and brings overheads. For every proc_open call a thread or two would have to be started, not sure it really justifies it.

But otherwise, is the solution i've described at the start with stderr and co. applicable to your case?

Thanks.
 [2015-02-22 04:22 UTC] php-bugs at lists dot php dot net
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Re-Opened". Thank you.
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Sun Nov 19 01:31:42 2017 UTC