php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #45161 Reusing a curl handle leaks memory
Submitted: 2008-06-03 15:04 UTC Modified: 2008-12-17 14:20 UTC
Votes:2
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:1 (100.0%)
From: humbads at alum dot mit dot edu Assigned:
Status: Closed Package: cURL related
PHP Version: 5CVS, 6CVS (2008-12-08) OS: *
Private report: No CVE-ID: None
 [2008-06-03 15:04 UTC] humbads at alum dot mit dot edu
Description:
------------
Reusing a curl handle for multiple requests leaks memory.  It leaks about 100 bytes per request.  This is a problem when making a large number of requests using the same handle.  Libcurl documentation says to always reuse the handle when possible in order to reuse connections.  The bug occurs on Windows XP/IIS and Apache/FreeBSD 6.2.

Reproduce code:
---------------
<?php
// Reuse curl handle memory leak test
$ch = curl_init();
$startmem = memory_get_usage();
for($i = 0; $i < 100; $i++) {
  $fp = fopen('/dev/null', "w");
  curl_setopt($ch, CURLOPT_URL,
    'http://debug.atensoftware.com/r.php?echo='.rand());
  curl_setopt($ch, CURLOPT_FILE, $fp);
  curl_exec($ch);

  fclose($fp);
  unset($fp);
  print "$i-mem: ".(memory_get_usage() - $startmem)." bytes\n";
}

?>

Expected result:
----------------
The memory usage should not increase after each request.


Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2008-06-03 16:07 UTC] humbads at alum dot mit dot edu
Below is the r.php script.  When testing this bug report, please use this script (or some other URL) so my poor server does not get hammered.

<?php
// This script runs a variable amount of time
// and generates a variable amount of data

// Output a random number of blank space
$s = microtime(true);
$m = rand(100,200);
$bytes = 0;
for($i = 0; $i < $m; $i++) {
	$message = "         \n";
	print $message;
	$bytes += strlen($message);
	usleep(10);
}

// Print time taken and the value of the "echo" parameter
print isset($_REQUEST['echo']) ? 'echo: '.$_REQUEST['echo'].' ' : "";
print $bytes.' bytes ';
print " in ";
print round(microtime(true) - $s, 4)." seconds";
exit();
?>
 [2008-06-03 19:24 UTC] humbads at alum dot mit dot edu
I did some more investigation, and found that this is not really a leak as much as it is runaway memory usage.  PHP does indeed free the memory eventually.

The problem is in interface.c curl_setopt under CURLOPT_URL.   The function is making a copy of the string parameter, in this case, the URL, and saving that to the "to_free" list.  That list is not freed until sometime later, so repeatedly setting CURLOPT_URL on the same handle will keep using more memory.  In the sample code I posted, the memory only increases during the curl_setopt CURLOPT_URL function call.

This is just an inefficient design of the PHP curl handle.  One way around it is for the code to maintain only one copied string for each CURLOPT string option.  If the same CURLOPT string option is set again, it should free the previously created string, and create a new one.

At the PHP level, the only workaround for now is to periodically close the handle and then create a new one with curl_init.  This seems to cap the memory usage.
 [2008-06-22 22:06 UTC] daniel at haxx dot se
I could also add that libcurl does duplicate the input strings since a few versions back (since libcurl 7.17.0), so if such a recent version of libcurl is detected there's no reason at all for PHP to keep copies.

This goes for everything except CURLOPT_POSTFIELDS which isn't duplicated for backwards compatiblity reasons, but that one can then be replaced with CURLOPT_COPYPOSTFIELDS which does copy.
 [2008-08-15 15:37 UTC] jani@php.net
FYI: This is by design in any resource in PHP. Freeing happens in the request shutdown. Just do unset($ch); in the end of the for loop and $ch = curl_init(); in the start of the loop and you get "static" memory usage. 

I'm going to only look at what Daniel pointed out and make PHP not duplicate unnecessarily if curl version is >= 7.17.0.
 [2008-12-11 06:03 UTC] magicaltux@php.net
Hi,

Managed to get rid of to_free.str, with one exception bugging me: when we call curl_slist_append(), we duplicate the string before, however curl doc says: "curl_slist_append() copies the string."

Is this true with all versions of curl? I see nothing written about this in the curl man page.
 [2008-12-11 13:51 UTC] magicaltux@php.net
This patch will get rid of to_free.str if libcurl version is >= 7.17.0

Also:

[09:25:46] <MT`AwAy> btw curl doc says: "curl_slist_append() copies the string.", is this true for all versions of libcurl ?
[09:26:33] <B4gder> yes

So, code duplicating strings before passing them to curl_slist_append() has been completly removed.

http://ookoo.org/svn/snip/php_bug45161_curl_textcopy.patch
 [2008-12-17 14:20 UTC] jani@php.net
This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.


 [2020-02-03 23:56 UTC] cmb@php.net
Automatic comment on behalf of cmbecker69@gmx.de
Revision: http://git.php.net/?p=php-src.git;a=commit;h=079905acd536e772241def9ad098d16f45ef37d8
Log: Fix #78090: bug45161.phpt takes forever to finish
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Nov 21 12:01:29 2024 UTC