|
|
php.net | support | documentation | report a bug | advanced search | search howto | statistics | random bug | login |
[2010-01-14 19:09 UTC] ssufficool at gmail dot com
Description:
------------
When querying large tables (> 800,000 rows) with PDO DBLIB I get out of memory.
The same select query works fine using:
linux# tsql -H host -U user -P pass
SELECT * from aVeryLargeTable
go
quit
on the command line using the freetds (dblib) library without consuming client-side memory.
Reproduce code:
---------------
$pdo = new PDO('dblib:host=host','user','pass');
echo "Creating table...\n";
$pdo->query("CREATE TABLE large_table (field_1 nvarchar(4000))");
$pdo->query("DECLARE @n int;
set @n = 0;
WHILE (@n < 50000) BEGIN
insert into large_table values( replicate(4000,'-') );
set @n = @n + 1;
END");
echo "Prepare\n";
$rs = $pdo->prepare("SELECT * FROM large_table");
echo "Execute\n";
/*********OOM HERE**************/
$rs->execute( );
Expected result:
----------------
A valid handle to a resultset in $rs
Actual result:
--------------
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 48 bytes) in /home/ssufficool/tds_test.php on line 12
It looks like the guts of ext/pdo_dblib/dblib_stmt.c:pdo_dblib_stmt_execute()
at "/* let's fetch all the data */"
Should be moved to:
pdo_dblib_stmt_fetch()
and only when a scrollable cursor is requested should the data be buffered at the client (not required for ct-lib)
Patchespdo_dblib_unbuffer-5.3.patch (last revision 2010-04-05 17:35 UTC by ssufficool at gmail dot com)pdo_dblib_unbuffer-6.patch (last revision 2010-03-09 22:24 UTC by ssufficool at gmail dot com) pdo_dblib_unbuffer-5.2.patch (last revision 2010-03-09 22:23 UTC by ssufficool at gmail dot com) Pull RequestsHistoryAllCommentsChangesGit/SVN commits
|
|||||||||||||||||||||||||||||||||||||
Copyright © 2001-2024 The PHP GroupAll rights reserved. |
Last updated: Thu Nov 21 09:01:32 2024 UTC |
Patch revised: ------------------------------------------------ 1. Reverted driver always registering as dblib. Question: Should the user really have to know the library the extension was compiled against? Seems like we should settle on a constant registration since you really can't mix and match. 2. Reverted whitespace modifications. Removed spurrious comments. Reverted DBSETOPT --> dbsetopt. 3. Reverted SYB* --> SQL* define deletions. These are required for compile against the depreciated MS DBLIB. 4. Removed automagic compute column naming (which was clobbering library memory). Just return what the server returns including empty strings. The user will need to alias in their sql query as "select 1+1 as oneplusone" instead of just "select 1+1" magically returning array('compute1'=>'2'). Question: Who if anyone relies on this behavior? I don't see other drivers doing this. Some unrelated/unmentioned "fixes" ------------------------------------------------ Allow multiple rowsets with varying column definitions. This was implemented incorrectly. Include the recent update to SQLMONEY formatting. Tested against SQL Server 2008 Express, PHP-5.3 svn-296442, FreeTDS 0.64, Linux 2.6.30 - i686.