php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #35655 Missing end-of-line token
Submitted: 2005-12-13 10:52 UTC Modified: 2005-12-13 22:45 UTC
From: tomas_matousek at hotmail dot com Assigned: iliaa (profile)
Status: Closed Package: Scripting Engine problem
PHP Version: 5CVS-2005-12-13 (snap) OS: *
Private report: No CVE-ID: None
View Developer Edit
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: tomas_matousek at hotmail dot com
New email:
PHP Version: OS:

 

 [2005-12-13 10:52 UTC] tomas_matousek at hotmail dot com 
Description:
------------
PHP lexer omits end-of-line characters following T_END_HEREDOC token. This also causes incorrect output of highlight_string() function.

Reproduce code:
---------------
$code = '
<? 
  $x = <<<EOT
some string    
EOT
  $y = 2;
?>';

display_tokens(token_get_all($code));
highlight_string($code);



function display_tokens($tokens)
{
  foreach ($tokens as $token)
  { 
    if (is_array($token))
    {
      echo token_name($token[0]),"\n";
      echo $token[0]," '",htmlentities(addcslashes($token[1],"\n\r\t")),"'\n";
    }
    else
    {
      echo "    '",htmlentities($token),"'\n";
    }  
  }
}

Expected result:
----------------
T_INLINE_HTML
311 '\r\n'
T_OPEN_TAG
367 '<?'
T_WHITESPACE
370 ' \r\n  '
T_VARIABLE
309 '$x'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_START_HEREDOC
371 '<<<EOT\r\n'
T_STRING
307 'some'
T_ENCAPSED_AND_WHITESPACE
314 ' '
T_STRING
307 'string'
T_ENCAPSED_AND_WHITESPACE
314 '    \r\n'
T_END_HEREDOC
372 'EOT'
T_WHITESPACE
370 '\r\n  '      // '\r\n' is missing in actual result
T_VARIABLE
309 '$y'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_LNUMBER
305 '2'
    ';'
T_WHITESPACE
370 '\r\n'
T_CLOSE_TAG
369 '?>'


<? 
  $x = <<<EOT
some string    
EOT  $y = 2;
?>

Actual result:
--------------
T_INLINE_HTML
311 '\r\n'
T_OPEN_TAG
367 '<?'
T_WHITESPACE
370 ' \r\n  '
T_VARIABLE
309 '$x'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_START_HEREDOC
371 '<<<EOT\r\n'
T_STRING
307 'some'
T_ENCAPSED_AND_WHITESPACE
314 ' '
T_STRING
307 'string'
T_ENCAPSED_AND_WHITESPACE
314 '    \r\n'
T_END_HEREDOC
372 'EOT'         // missing '\r\n' 
T_WHITESPACE
370 '  '
T_VARIABLE
309 '$y'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_LNUMBER
305 '2'
    ';'
T_WHITESPACE
370 '\r\n'
T_CLOSE_TAG
369 '?>'


<? 
  $x = <<<EOT
some string    
EOT  $y = 2;
?>

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2005-12-13 10:54 UTC] tomas_matousek at hotmail dot com 
Correction:
Expected result of highlighting:
$code = '
<? 
  $x = <<<EOT
some string    
EOT
  $y = 2;
?>'

Actual result:
$code = '
<? 
  $x = <<<EOT
some string    
EOT  $y
 = 2;
?>'
 [2005-12-13 11:31 UTC] tomas_matousek at hotmail dot com 
The same result with latest snapshot.
The problem is probably here (file zend_language_scanner.l, around line 1750):

		if (unput_semicolon) {
			unput(';');
		}

I would suggest to replace unputting with 
  yyless(label_len);
which would preserve both ';' and {NEWLINE}.
 [2005-12-13 13:34 UTC] tony2001@php.net 
Ilia, please check it out.
The proposed patch looks good for me.
 [2005-12-13 22:45 UTC] iliaa@php.net 
This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Sat Oct 25 08:00:01 2025 UTC