php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #78485 Random "couldn't resolve host" failures in curl_multi
Submitted: 2019-09-02 17:02 UTC Modified: 2021-10-03 04:22 UTC
Votes:3
Avg. Score:3.7 ± 0.9
Reproduced:3 of 3 (100.0%)
Same Version:2 (66.7%)
Same OS:2 (66.7%)
From: sinus at sinpi dot net Assigned: cmb (profile)
Status: No Feedback Package: opcache
PHP Version: 7.3.9 OS: CloudLinux
Private report: No CVE-ID: None
View Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
If you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: sinus at sinpi dot net
New email:
PHP Version: OS:

 

 [2019-09-02 17:02 UTC] sinus at sinpi dot net 
Description:
------------
I'm experiencing a pretty random failure of curl_multi to grab some of the handles added to it. I've already reported it to devs of libcurl, but they told me to toss this hot potato back here, as apparently curl_multi_exec is php7-curl's wrapper around libcurl's curl_multi_perform and any faults must lie in there.

So, the matter at hand. I create 12 handles for http:// URLs, all using the same target machine, just different files. I add them to curl_multi. And as soon as I curl_multi_exec, randomly sometimes they all work, and sometimes only exactly first 8 get fetched, others immediately get a 6=="couldn't resolve host name" message queued in curl_multi_info_read.

It's always the first handles that succeed, and the final handles that fail - n first handles work, n+1 till the end fail to resolve, with n sometimes being all 12, sometimes 8, a few times 4, but I also caught it on 11.

This is in cURL 7.62 under PHP 7.2.21.


Test script:
---------------
$cm = curl_multi_init();
for ($i=1;i<=12;i++) {
    $handles[$i] = curl_init ("http://the.same.host.com/file" . $i);
    curl_multi_add_handle ( $cm, $handles[$i]);
}
$status = curl_multi_exec ($cm, $still_running);   // $status gets 0, $still_running gets 8.
$info = curl_multi_info_read ($cm, $queue);  // $info['result']==6 "couldn't resolve host" for handle 9, $queue==3 as there are also failures queued for handles 10, 11 and 12


Expected result:
----------------
All 12 URLs retrieved, naturally.

Actual result:
--------------
Usually only 8 retrieved, or 12 as intended, or 4, or other numbers. It's always the last handles that fail.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2019-09-02 17:55 UTC] sinus at sinpi dot net 
To try and work around the issue, I wrote a retry mechanism - if a CURLE_COULDNT_RESOLVE_HOST message is detected, I curl_multi_remove_handle the handle, curl_close it, make a new curl_init and curl_multi_add_handle again, and return to calling curl_multi_exec. The docs don't state whether adding or removing handles during a multi session is permitted, so why not give it a try.

The result, however, was exactly the same. All of the newly re-added child handles immediately got CURLE_COULDNT_RESOLVE_HOST results.
 [2019-09-02 19:39 UTC] sinus at sinpi dot net 
Trying another workaround, I built another retry mechanism, storing the failed downloads for later and only re-queuing them once the "good" downloads complete (getting a result==0 in curl_multi_info_read). However, it would seem that curl_multi does not, in fact, like having its list modified when it's running: my newly created curl_init handles, added anew, simply got ignored: curl_multi_exec's $still_running return silently dropped by one on the next round, as if it merely removed the new handle from its list.

So another question arises - CAN curl_multi actually handle additions or removals of its child handles while in a curl_multi_exec session..?
 [2019-09-02 20:33 UTC] sinus at sinpi dot net 
I finally managed to make a workaround, the hard way: by stashing the failed handles and starting a whole new curl_multi_init session with just the failures. The original problem remains, though: why would n+1 handles immediately fail to resolve.
 [2019-09-02 20:35 UTC] nikic@php.net 
Is there any publicly accessible host for which this might be reproduced?
 [2019-09-02 20:51 UTC] sinus at sinpi dot net 
The issue can be seen at http://djab.eu/isuvu/bug.php with source viewable at http://djab.eu/isuvu/bug.php.txt .
 [2019-09-02 20:54 UTC] sinus at sinpi dot net 
You might have to reload it a few times, as I said, it's pretty random. Reproductions of the bug are seen as "Msg: 6 for handle <x>".
 [2019-09-02 21:17 UTC] sinus at sinpi dot net 
Seeing as I've never seen this happen on my previous host, I suspect it's a conflict with some local setup, not a bug in curl itself - but it'd be nice if there was a way to find out what could possibly cause it to behave like that.
 [2019-09-03 09:26 UTC] sinus at sinpi dot net 
I now added CURLOPT_VERBOSE output to the bug.php script, and it demonstrated the failing handle throwing the following at me:

* getaddrinfo() thread failed to start

The plot thickens.
 [2019-09-03 15:43 UTC] sinus at sinpi dot net 
The situation develops. Disabling the opcache module made the issue go away entirely!
 [2019-09-06 14:27 UTC] sinus at sinpi dot net 
Oddly enough, enabling or disabling of any PHP modules was causing the errors to either visibly decrease, or just as significantly increase in frequency.

Updating to PHP 7.3.9, still cURL 7.62.0, stabilized the issue enough to say: enabling opcache causes failures to create getaddrinfo threads.
 [2019-09-06 14:31 UTC] requinix@php.net
-Package: HTTP related +Package: opcache
 [2021-09-22 12:34 UTC] cmb@php.net
-Status: Open +Status: Feedback -PHP Version: 7.2.22 +PHP Version: 7.3.9 -Assigned To: +Assigned To: cmb
 [2021-09-22 12:34 UTC] cmb@php.net 
Is this still an issue with any of the actively supported PHP
versions[1]?

[1] <https://www.php.net/supported-versions.php>
 [2021-10-03 04:22 UTC] php-bugs at lists dot php dot net 
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Re-Opened". Thank you.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Sat Dec 21 14:01:32 2024 UTC