php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #80956 Forward PHP streams as extra_fds to curl_multi_wait in curl_multi_select
Submitted: 2021-04-15 14:07 UTC Modified: 2021-05-23 13:07 UTC
From: asmqb7 at gmail dot com Assigned:
Status: Closed Package: cURL related
PHP Version: 8.0.3 OS: Linux
Private report: No CVE-ID: None
View Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
If you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: asmqb7 at gmail dot com
New email:
PHP Version: OS:

 

 [2021-04-15 14:07 UTC] asmqb7 at gmail dot com
Description:
------------
It's currently not possible to efficiently run a curl_multi request while simultaneously selecting on other file descriptors. At first glance, this appears to be trivially easy to completely fix. (I'd implement it myself but I'm still getting familiar with PHP's internals.)

According to the cURL documentation at https://curl.se/libcurl/c/curl_multi_wait.html,

----

   CURLMcode curl_multi_wait(CURLM *multi_handle,
                             struct curl_waitfd extra_fds[],
                             unsigned int extra_nfds,
                             int timeout_ms,
                             int *numfds);

   (...Synopsis...)

   The calling application may pass additional curl_waitfd structures which are similar to poll(2)'s
   pollfd structure to be waited on in the same call.

----

It would seem fairly straightforward to simply introduce a 3rd parameter to curl_multi_select():

  curl_multi_select (CurlMultiHandle $multi_handle, float $timeout = 1.0, array $extra_fds = []) : int

The additional streams would then be inserted into extra_fds.

** Main question: are there scenarios in PHP (core) where a Stream-like-object doesn't boil down to a fd? **

FWIW, curl_multi_select() is currently not using extra_fds for anything else, so implementing this shouldn't step on anything's toes. From https://github.com/php/php-src/blob/5b01c4863fe9e4bc2702b2bbf66d292d23001a18/ext/curl/multi.c#L186:

  186  error = curl_multi_wait(mh->multi, /*extra_fds*/NULL, /*extra_nfds*/0,
       (unsigned long) (timeout * 1000.0), &numfds);

My general use case for this is to be able to achieve proper blocking polling on a bunch of streams, when one of those streams is a [set of] cURL requests, for example in the context of async or event loop frameworks.

IMO such libraries can easily bear the load of intelligently handling the "stream_select() vs curl_multi_select()?" question - this problem is endemic to cURL design idiosyncrasies, and cURL's efficiency (and HTTP/3 support, and general batteries-included behavior, etc etc) make it definitively worth it to plumb through the support described here so the PHP userspace can be enabled to fully adapt to cURL's way of doing things.

In my small event loop framework, I've basically had to implement recurring ticks that ultimately make stream_select() timeout every 100000us so I can then curl_multi_select($mh, /*timeout*/0). This is not at all particularly efficient, hence this little feature request.


Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2021-04-16 06:03 UTC] twosee@php.net
This can really improve the convenience of development and I am also interested to implement this. Perhaps the reason why it has not been implemented before is that the implementation of a data structure to represent pollfd/curl_waitfd (or use arrays to represent them) is a bit troublesome.
But, this may also add some workload to Swoole/Swow (extensions that replaced the implementation of cURL).
In addition, it seems that the implementation of select in PHP made some patches on the Windows platform to support non-socket handles.
And I took a quick look at the select() implementation in cURL[1], not only may it be different from PHP's select implementation, but it may also use poll. I don’t know what effect this will have if we use cURL's select to replace PHP's.

[1] <https://github.com/curl/curl/blob/master/lib/select.c>
 [2021-05-23 13:07 UTC] asmqb7 at gmail dot com
-Status: Open +Status: Closed
 [2021-05-23 13:07 UTC] asmqb7 at gmail dot com
CLOSED INVALID - the idea in this bug remains interesting, unresolved and relevant, but I've discovered the implementation approach suggested here is unnecessary and so boneheaded I'm nuking this ticket from orbit and starting again.

Please see new bug #81075 which contains a rehash of the context and a *slightly* saner approach to a solution :)

I also make some pragmatic suggestions regarding the fd_set issue.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Tue Nov 12 05:01:29 2024 UTC