php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #81059 Memory leaks PDO MySQL reading blob field
Submitted: 2021-05-20 14:07 UTC Modified: 2021-05-27 14:02 UTC
From: andresr dot alz at gmail dot com Assigned:
Status: Duplicate Package: PDO MySQL
PHP Version: 8.0.6 OS: Docker FROM php:8.0.6-cli
Private report: No CVE-ID: None
 [2021-05-20 14:07 UTC] andresr dot alz at gmail dot com
Description:
------------
#### Dockerfile
FROM php:8.0.6-cli
RUN docker-php-ext-install pdo_mysql
RUN docker-php-ext-enable pdo_mysql
COPY . /usr/src/myapp
WORKDIR /usr/src/myapp
ENTRYPOINT ["php", "testing_pdo.php"]
####

This is not happening with php < 7.3


Test script:
---------------
// testing_pdo.php
<?php

ini_set('memory_limit', '-1');

function _get_connection()
{
  print "\nget_connection";
  $opt = array(
    PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
  );

  try {
    $dbconn = new PDO(
      "mysql:host=" . "localhost",
      "user",
      "user_test",
      $opt
    );
    $dbconn->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
    return $dbconn;
  } catch (PDOException $e) {
    print "Error: $e";
    return null;
  }
}

function _big_query($dbconn)
{
  print "\nbig query running ...";

  $CSV_results = "";
  $sql = "SELECT sent_blob from test WHERE id = 1";
  $stm = $dbconn->prepare($sql);
  $stm->execute();

  $stm->bindColumn(1, $CSV_results, PDO::PARAM_LOB);
  $stm->fetch(PDO::FETCH_BOUND);
  $stm->closeCursor();

  return $CSV_results;
}


print "\nStarting ....";
$conn = _get_connection();
if ($conn != null) {
  $result = _big_query($conn);
  print $result;
}

Expected result:
----------------
Memory on docker container keeps almost the same than network IO

Actual result:
--------------
Memory is increasing in a weird way

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2021-05-27 08:23 UTC] nikic@php.net
-Status: Open +Status: Feedback
 [2021-05-27 08:23 UTC] nikic@php.net
Could you please provide the schema/contents of the test table you're using here? I tried with some dummy values, but wasn't able to reproduce a leak.

Also, as you don't have a loop inside the code, I'm assuming that the leak you're seeing occurs across multiple requests? Each request increasing memory usage?
 [2021-05-27 12:54 UTC] andresr dot alz at gmail dot com
You can reproduce the problem with a big blob content, maybe > 400mb
The file need to be big to notice the memory leak.


and i am not using a loop, the code waits on : $stm->fetch(PDO::FETCH_BOUND);
It take a lot of time there obviously if the blob is big and you can see how the memory increase a lot

php: 8.0.6
mysql: 5.1.40

screenshot docker: https://ibb.co/q11mnBG


ps: in my case i tested with a internal company file: 740 mb
 [2021-05-27 13:31 UTC] nikic@php.net
Okay, I can reproduce the issue now. This is a duplicate of bug #80761, which is fixed in PHP 8.1 by https://github.com/php/php-src/commit/1fc4c89214c82fabbf997da58051a385d8fe50ab. We may be able to backport the change.
 [2021-05-27 13:54 UTC] cmb@php.net
-Status: Feedback +Status: Analyzed
 [2021-05-27 14:02 UTC] nikic@php.net
-Status: Analyzed +Status: Duplicate
 [2021-05-27 14:02 UTC] nikic@php.net
I've backported the relevant change to PHP-8.0 now. Marking this as a duplicate of bug #80761.

Note that PHP will still use 2x the size of the BLOB peak memory, but this is hard to avoid. At least it won't use 10x now (depending on the size of BLOB...)
 [2021-05-27 14:11 UTC] andresr dot alz at gmail dot com
Thanks you so much for the information
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Apr 18 18:01:28 2024 UTC