php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #80830 Running out of local ports when using curl
Submitted: 2021-03-04 14:27 UTC Modified: 2021-03-12 08:39 UTC
Votes:2
Avg. Score:5.0 ± 0.0
Reproduced:2 of 2 (100.0%)
Same Version:1 (50.0%)
Same OS:1 (50.0%)
From: external dot maik dot dieterich at bosch dot com Assigned:
Status: Open Package: Network related
PHP Version: 7.4.15 OS: Windows Server 2019
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2021-03-04 14:27 UTC] external dot maik dot dieterich at bosch dot com
Description:
------------
We are having an application which runs in Apache and gets files via an API. These files are stored on a storage over a rest API. For transferring the files to the storage we are using PHP with curl.

Php opens a local port to local port connection when a curl handle is initialized with a connection to an external server. 
This local connection stays in TIME_WAIT after the external connection and the curl handle is closed.

When open connections with curl from command line without PHP these local port local port connections are not opened.

Test script:
---------------
for($i = 0; $i < 5000; $i++)
{
    $ch = curl_init("your url");
    $response = curl_exec($ch);
    curl_close($ch);
}

Expected result:
----------------
No connections in TIME_WAIT.

Actual result:
--------------
Many connections in TIME_WAIT.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2021-03-04 15:51 UTC] danack@php.net
This is a feature of Windows. It keeps network connections open for a long time by default.

You could either work around this by "Adjust the MaxUserPort and TcpTimedWaitDelay settings" - 
https://docs.microsoft.com/en-us/biztalk/technical-guides/settings-that-can-be-modified-to-improve-network-performance

Or you can re-use the same Curl object, which will almost certainly re-use the underlying network connection if possible, 

```
$ch = curl_init($url);

for($i = 0; $i < 5000; $i++)
{
    curl_setopt($ch, CURLOPT_URL, $url);
    $response = curl_exec($ch);
}

curl_close($ch);
```

I'll leave it open, so that someone who is more sure that there is nothing to be done in PHP, can close it.
 [2021-03-04 17:19 UTC] cmb@php.net
-Status: Open +Status: Not a bug -Assigned To: +Assigned To: cmb
 [2021-03-04 17:19 UTC] cmb@php.net
I think @danack is spot on.  But instead of fiddling with these
Windows settings, I suggest to re-use the same curl resource
(object as of PHP 8.0.0), like described by Dan, to avoid calling
curl_easy_cleanup() under the hood, because[1]

| re-using handles is a key to good performance with libcurl

Anyway, this is not a PHP issue.

[1] <https://curl.se/libcurl/c/curl_easy_cleanup.html>
 [2021-03-04 18:32 UTC] external dot maik dot dieterich at bosch dot com
First thanks for the quick response.

What is not clear for me, is why the connection I open to the external server can be closed immediately and why this local local port connections, which i do not have if i use curl from command line, are still open even if the external connection I open is closed.

Because each connection is opened in a different thread i think I can not reuse them.
 [2021-03-04 20:07 UTC] rtrtrtrtrt at dfdfdfdf dot dfd
it's just how TCP works

https://superuser.com/questions/173535/what-are-close-wait-and-time-wait-states
 [2021-03-09 08:23 UTC] michael dot mckeever at bosch dot com
Thank you @danack and @cmb for the quick response.

Your suggestions with reusing the the curl handle is correct if you're using a cli application and you're in control of the environment.
The problem described by Maik is that if you're using apache you can't reuse curl handles.

@rtrtrtrtrt, your comment is unfortunately not helping in this topic ;-)
The key question is, why is the mass usage of curl.exe not causing TIME_WAITs and the php libcurl functions causing TIME_WAITs. Furthermore the php functinos are not causing TIME_WAITs to the called server as one would expect, but it's causing TIME_WAITs for some kind of local redirects. It's causing localhost to localhost TIME_WAITs.
 [2021-03-09 11:54 UTC] cmb@php.net
-Status: Not a bug +Status: Open
 [2021-03-09 11:54 UTC] cmb@php.net
To clarify: you are not initializing multiple curl handles in a
single request, but rather for different requests, and you are
using the apache2handler SAPI?

In this case there may indeed be a problem, since the SAPI
intializes libcurl (curl_global_init()) at its startup and shuts
it down (curl_global_cleanup()) at its shutdown, so libcurl might
keep connections open, but may not reuse them.

> It's causing localhost to localhost TIME_WAITs.

Frankly, I have no idea why that happens.  The curl extension is
relatively thin wrapper around libcurl, so I wonder whether we set
up or use libcurl inappropriately, or whether Apache or the SAPI
interferes.
 [2021-03-09 16:54 UTC] cmb@php.net
-Assigned To: cmb +Assigned To:
 [2021-03-12 08:39 UTC] external dot maik dot dieterich at bosch dot com
Hello,

yes we are using apache2handler SAPI and each request initializes a curl handle.

But also if I'm initialize a curl handle in a cli application and close it the connection I openend is closed but the corresponding local to local port connection, which i have not explicit opened, stays in TIME_WAIT.

When I'm opening and close the same connection with curl from command line without PHP i can not see such a local to local port connection.

Thanks for the support.
 
PHP Copyright © 2001-2021 The PHP Group
All rights reserved.
Last updated: Wed Jul 28 11:01:24 2021 UTC