php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #16939 tokenizer: emalloc causes segfault
Submitted: 2002-05-01 02:40 UTC Modified: 2002-05-21 17:24 UTC
From: alan at akbkhome dot com Assigned:
Status: Closed Package: Reproducible crash
PHP Version: 4.2.0 OS: linux
Private report: No CVE-ID: None
 [2002-05-01 02:40 UTC] alan at akbkhome dot com
A short script to reproduce this
(sorry it not to easy to reproduce without the exact code, I did at one point get to the situation where adding a extra space in the white space area solved this!), but I suspect it is that the machine is running low on memory or something....

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/phpmole-ide/phpmole/tools/uml_generator.class?rev=1.2&content-type=text/vnd.viewcvs-markup

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/phpmole-ide/phpmole/tools/uml_dia.xml.php?rev=1.1&content-type=text/vnd.viewcvs-markup

-- just use it do do a large number of files. eg.
php uml_generator.class /usr/local/php/pear/

Possible reasons that I could think of:
-the tokenizer extension (which may be leaking memory)
-somewhere not exiting if no memory can be allocated and then emalloc / malloc being called again.

Configuration on this machine (4.2.0RC2), also reproduced with 4.2 release version
'./configure' '--prefix=/usr' '--without-mysql' '--with-config-file-path=/etc/php4/cgi/' '--enable-dba' '--with-db2'
and tokenizer as a dl'd module

Backtrace

Program received signal SIGSEGV, Segmentation fault.
0x4014db2b in malloc () from /lib/libc.so.6
(gdb) bt
#0  0x4014db2b in malloc () from /lib/libc.so.6
#1  0x4014d1e4 in malloc () from /lib/libc.so.6
#2  0x80c4c5c in _emalloc (size=35) at zend_alloc.c:165
#3  0x80d4b1a in zend_hash_index_update_or_next_insert (ht=0x817ae9c, h=207, 
    pData=0xbfffda28, nDataSize=4, pDest=0x0, flag=4) at zend_hash.c:404
#4  0x80d303a in add_next_index_stringl (arg=0x8326d64, str=0x8268c08 ")", length=1, 
    duplicate=1) at zend_API.c:847
#5  0x401ffc61 in tokenize () from /usr/lib/php4/cgi/tokenizer.so
#6  0x402005ac in zif_token_get_all () from /usr/lib/php4/cgi/tokenizer.so
#7  0x80eb4b0 in execute (op_array=0x816631c) at ./zend_execute.c:1598
#8  0x80eb679 in execute (op_array=0x8166a84) at ./zend_execute.c:1638
#9  0x80eb679 in execute (op_array=0x816004c) at ./zend_execute.c:1638
#10 0x80d19e8 in zend_execute_scripts (type=8, retval=0x0, file_count=3) at zend.c:810
#11 0x805e1dd in php_execute_script (primary_file=0xbffffb98) at main.c:1381
#12 0x805c0ac in main (argc=3, argv=0xbffffc14) at cgi_main.c:785
#13 0x400f86cf in __libc_start_main () from /lib/libc.so.6


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2002-05-01 03:09 UTC] alan at akbkhome dot com
Workaround:

Original code did this
$this->tokens = token_get_all($filename)

by not reassigning the same variable on each file, but rather really storing the token array in a huge array it works around this issue... (albiet with a huge memory overhead)
  
$this->all_tokens[$filename] = token_get_all($contents);
$this->tokens =& $this->all_tokens[$filename];
 [2002-05-04 01:06 UTC] alan at akbkhome dot com
OK, Got a nice simple test case - I'm pretty certain its the tokenizer code, Andrei :)

-- theres not category for tokenizer yet! :)

test script as follows (modify the directory to point at pear)..

<?
dl('tokenizer.so');
class test {
    function testing($f) {
       if (!preg_match('/\.(php|class)$/',$f)) return;
       echo "$f\n";
       $this->data =& token_get_all($f);
   }
   function recurse($dir) {
       $dh = opendir($dir);
       while (($f = readdir($dh)) !== FALSE) {
          if ($f{0} == ".") continue;
           $fn = "{$dir}/{$f}";
           if (is_dir($fn)) {
                $this->recurse($fn);
                continue;
           }
           // assume its a file
           $this->testing($fn);
       }
   }
}
$t = new test;
// change this to your pear directory.
$t->recurse('/usr/lib/php');
?>
 [2002-05-21 17:24 UTC] andrei@php.net
This bug has been fixed in CVS. You can grab a snapshot of the
CVS version at http://snaps.php.net/. In case this was a documentation 
problem, the fix will show up soon at http://www.php.net/manual/.
In case this was a PHP.net website problem, the change will show
up on the PHP.net site and on the mirror sites.
Thank you for the report, and for helping us make PHP better.


 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Fri Jun 28 18:01:31 2024 UTC