php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Doc Bug #61141 curl_multi_select returns -1
Submitted: 2012-02-20 10:24 UTC Modified: 2012-09-22 14:13 UTC
Votes:16
Avg. Score:4.6 ± 0.8
Reproduced:15 of 15 (100.0%)
Same Version:10 (66.7%)
Same OS:12 (80.0%)
From: amoo_miki at yahoo dot com Assigned: pierrick (profile)
Status: Not a bug Package: cURL related
PHP Version: 5.3.10 OS: Windows 7 x64
Private report: No CVE-ID: None
 [2012-02-20 10:24 UTC] amoo_miki at yahoo dot com
Description:
------------
Trying to run multiple curl calls, fails with a timeout while checking for activity.

PHP 5.3.9 and before work just fine.

Test script:
---------------
<?php //Taken from 
$ch1 = curl_init();
$ch2 = curl_init();
curl_setopt($ch1, CURLOPT_URL, "http://windows.php.net/downloads/releases/md5sum.txt");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch2, CURLOPT_URL, "http://windows.php.net/downloads/releases/sha1sum.txt");
curl_setopt($ch2, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, 1);
$mh = curl_multi_init();
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$active = null;
do { $mrc = curl_multi_exec($mh, $active); } while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) if (curl_multi_select($mh) != -1) do { $mrc = curl_multi_exec($mh, $active); } while ($mrc == CURLM_CALL_MULTI_PERFORM);
echo strlen(curl_multi_getcontent($ch1))."-".strlen(curl_multi_getcontent($ch2));
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);
?>

Expected result:
----------------
Seen on PHP 5.3.9:
531-1375

Actual result:
--------------
Seen on PHP 5.3.10:
Fatal error: Maximum execution time of 30 seconds exceeded in <filepath> on line 15

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2012-02-20 10:46 UTC] rasmus@php.net
-Status: Open +Status: Feedback
 [2012-02-20 10:46 UTC] rasmus@php.net
There were no code changes between 5.3.9 and 5.3.10 that could possibly have 
caused this. I notice you are using php.net sites in your test script. We had 
some dns problems around the time you reported this. Are you sure this wasn't 
related to that?
 [2012-02-20 11:42 UTC] amoo_miki at yahoo dot com
I used these URLs just to make sure the content is small while my original code looks at various news feeds. I did make sure the URLs return stuff, via simple usage of cURL too.

The timeout happens within the loop that performs the curl_multi_select.
 [2012-02-20 20:28 UTC] amoo_miki at yahoo dot com
I put some debug code and got to "curl_multi_select" returning -1 no matter what, which seems to mean failure from the underlying select system call.

So I guess, the conversation between Win7 x64 and cURL in 5.3.10 is not going smooth.
 [2012-02-20 20:28 UTC] amoo_miki at yahoo dot com
-Summary: curl_multi_exec functionality broken +Summary: curl_multi_select returns -1 -Status: Feedback +Status: Open
 [2012-02-28 16:52 UTC] bompus at gmail dot com
Same exact problem and has been driving me nuts. I keep having to revert back to 
PHP 5.3.6 -- I have not tried 5.3.9 yet, but did try 5.3.10 and had the same 
issue. I am running loops with curl_multi_select and they always time out because 
of an infinite loop since it's always returning -1
 [2012-02-28 22:09 UTC] bompus at gmail dot com
I hope this helps. curl_multi_select works fine on my PHP 5.3.6 under Windows 7 x64 with the following CURL 
information:

cURL support => enabled
cURL Information => 7.21.2
Age => 3
Features
AsynchDNS => Yes
Debug => No
GSS-Negotiate => No
IDN => No
IPv6 => Yes
Largefile => Yes
NTLM => Yes
SPNEGO => No
SSL => Yes
SSPI => Yes
krb4 => No
libz => Yes
CharConv => No
Protocols => dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, pop3, pop3s, rtsp, scp, sftp, 
smtp, smtps, telnet, tftp
Host => i386-pc-win32
SSL Version => OpenSSL/1.0.0d
ZLib Version => 1.2.3
libSSH Version => libssh2/1.2.7

-----

Under PHP 5.3.10 on Windows 7 x64, curl_multi_select is always returning -1 with the following CURL 
information:

cURL support => enabled
cURL Information => 7.24.0
Age => 3
Features
AsynchDNS => Yes
Debug => No
GSS-Negotiate => No
IDN => No
IPv6 => Yes
Largefile => Yes
NTLM => Yes
SPNEGO => No
SSL => Yes
SSPI => No
krb4 => No
libz => Yes
CharConv => No
Protocols => dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, pop3, pop3s, rtsp, scp, sftp, 
smtp, smtps, telnet, tftp
Host => i386-pc-win32
SSL Version => OpenSSL/0.9.8t
ZLib Version => 1.2.5
libSSH Version => libssh2/1.3.0

I am trying to make GET requests to https:// is that helps. I noticed that there are different versions of 
libcurl and OpenSSL used, so it could end up being an issue with either one of those, most likely libcurl or 
PHP itself. Hope this helps!
 [2012-02-29 08:17 UTC] amoo_miki at yahoo dot com
The curl details on 5.3.9 are:
cURL support => enabled
cURL Information => 7.21.7
Age => 3
Features
AsynchDNS => Yes
Debug => No
GSS-Negotiate => Yes
IDN => No
IPv6 => Yes
Largefile => Yes
NTLM => Yes
SPNEGO => No
SSL => Yes
SSPI => Yes
krb4 => No
libz => Yes
CharConv => No
Protocols => dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, pop3, pop3s, rtsp, scp, sftp, smtp, smtps, telnet, tftp
Host => i386-pc-win32
SSL Version => OpenSSL/0.9.8r
ZLib Version => 1.2.5
libSSH Version => libssh2/1.2.7


meaning the changes are related to one of the following:
                   5.3.9 -> 5.3.10
libcURL:          7.21.7 -> 7.24.0
GSS/Negotiate:       Yes -> NO
SSPI:                Yes -> No
OpenSSL:          0.9.8r -> 0.9.8t
libSSH:            1.2.7 -> 1.3.0

I don't see a reason for the last 2 effecting anything.
 [2012-04-03 16:26 UTC] bompus at gmail dot com
Related to 60790 and 61240
 [2012-05-25 12:08 UTC] kulminaator at gmail dot com
Running PHP 5.4.0 on Windows 7 has the same issue,  curl_multi_select always  
returns -1 


PHP Version => 5.4.0
System => Windows NT MSA-3644048 6.1 build 7601 (Windows 7 Enterprise Edition 
Service Pack 1) i586

cURL support => enabled
cURL Information => 7.24.0
Age => 3
Features
AsynchDNS => Yes
Debug => No
GSS-Negotiate => Yes
IDN => No
IPv6 => Yes
Largefile => Yes
NTLM => Yes
SPNEGO => No
SSL => Yes
SSPI => Yes
krb4 => No
libz => Yes
CharConv => No
Protocols => dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, 
pop3, pop3s, rtsp, scp, sftp, smtp, smtps, telnet, tftp
Host => i386-pc-win32
SSL Version => OpenSSL/0.9.8t
ZLib Version => 1.2.5
libSSH Version => libssh2/1.3.0
 [2012-05-29 11:52 UTC] amoo_miki at yahoo dot com
I can confirm that PHP 5.4.3 (cli) (built: May  8 2012 00:23:27) suffers from the same bug.
 [2012-05-30 02:48 UTC] laruence@php.net
-Assigned To: +Assigned To: pierrick
 [2012-05-30 02:48 UTC] laruence@php.net
Pierrick, could you please look at this? :)
 [2012-06-06 20:57 UTC] ihate at spam dot com
Always returns -1 on Windows 7 64bit. Tried on 5.4 and 5.4.3. It worked a couple 
of months ago, perhaps some Win7 update changed something.
 [2012-09-22 09:35 UTC] amoo_miki at yahoo dot com
PHP 5.3.17 and 5.4.7 too suffer from this bug.
 [2012-09-22 13:22 UTC] pierrick@php.net
-Status: Assigned +Status: Not a bug -Type: Bug +Type: Documentation Problem
 [2012-09-22 13:22 UTC] pierrick@php.net
I tried this on my windows setup with the same version of PHP but two different 
versions of libcurl (7.19 and 7.24) and this difference is due to the libcurl 
version used to compile ext/curl but this is in fact not a bug.

Internally php curl_multi_select uses libcurl curl_multi_fdset function to set 
all the fd_set and the maxfd value. The libcurl curl_multi_fdset documentation 
says :

When libcurl returns -1 in max_fd, it is because libcurl currently does 
something that isn't possible for your application to monitor with a socket and 
unfortunately you can then not know exactly when the current action is completed 
using select(). When max_fd returns with -1, you need to wait a while and then 
proceed and call curl_multi_perform anyway. How long to wait? I would suggest 
100 milliseconds at least, but you may want to test it out in your own 
particular conditions to find a suitable value.

If you modify the script to wait some time if the curl_multi_select return -1 
and then do the curl_multi_exec it should work (it worked on my setup with both 
curl versions).
 [2012-09-22 13:38 UTC] amoo_miki at yahoo dot com
I modified the script and put a sleep(10) and it still doesn't work. Here is the script i used; was this what you meant?

<?php 
$ch1 = curl_init();
$ch2 = curl_init();
curl_setopt($ch1, CURLOPT_URL, "http://windows.php.net/downloads/releases/md5sum.txt");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch2, CURLOPT_URL, "http://windows.php.net/downloads/releases/sha1sum.txt");
curl_setopt($ch2, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, 1);
$mh = curl_multi_init();
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$active = null;
do { $mrc = curl_multi_exec($mh, $active); } while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1)
        do { $mrc = curl_multi_exec($mh, $active); }
        while ($mrc == CURLM_CALL_MULTI_PERFORM);
    else {
        echo "* Sleeping for 10 secs\n";
        sleep(10);
    }
}
echo strlen(curl_multi_getcontent($ch1))."-".strlen(curl_multi_getcontent($ch2));
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);
?>
 [2012-09-22 13:44 UTC] pierrick@php.net
No, sorry if I was unclear. The libcurl documentations says "When max_fd returns 
with -1, you need to wait a while and then proceed and call curl_multi_perform 
anyway".

If you translate this in PHP (That was where I was unclear) : "When 
curl_multi_select returns with -1, you need to wait a while and then proceed and 
call curl_multi_exec anyway"

Then, you should have something like this :

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) == -1) usleep(100);
    do { $mrc = curl_multi_exec($mh, $active); }
    while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
 [2012-09-22 13:49 UTC] amoo_miki at yahoo dot com
I should also add that, the sample code, without the sleep works perfectly fine on non-windows setups. I verified it using 5.3.13 which uses cURL 7.24.0 on OSX.
 [2012-09-22 13:56 UTC] amoo_miki at yahoo dot com
Gotcha. The code change you propose works, but not sure why it doesn't make sense to me.

while ($active && $mrc == CURLM_OK) {
    do { $mrc = curl_multi_exec($mh, $active); }
    while ($mrc == CURLM_CALL_MULTI_PERFORM);
}

The above piece works too, without needing the timeout, but that mean skipping the curl_multi_select altogether, practically what you found is that we can do without checking curl_multi_select which doesn't solve the problem with curl_multi_select.

See where i am going?
Also, thanks for looking into this. curl_multi_select is a very useful function and it would be crazy not to use it.
 [2012-09-22 14:03 UTC] amoo_miki at yahoo dot com
i reread your comment about
The libcurl documentations says "When max_fd returns 
with -1, you need to wait a while and then proceed and call curl_multi_perform 
anyway"...

... the keyword being "anyway".
 [2012-09-22 14:13 UTC] pierrick@php.net
The change that introduced your problem is somewhere in libcurl. You could submit 
a bug to libcurl, but I'm pretty sure they will come with the same answer as I 
did since it's documented.

I know that the code will work even if you never call curl_multi_select but I 
would recommend you to use it anyway to make sure you'll not consume all your cpu 
resources and I would follow the libcurl recommandation to sleep some time before 
executing the curl_multi_exec
 [2012-09-22 14:16 UTC] amoo_miki at yahoo dot com
Got it. Thanks for explaining it.
Can't say i am happy with what libcurl would say :)

Also, i believe this would call for a change in "Example 1" on http://php.net/manual/en/function.curl-multi-init.php

Thanks again.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Tue Mar 19 11:01:28 2024 UTC