php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #76437 token_get_all with TOKEN_PARSE flag fails to recognise close tag
Submitted: 2018-06-09 11:26 UTC Modified: 2018-06-17 21:00 UTC
Votes:10
Avg. Score:4.7 ± 0.6
Reproduced:6 of 8 (75.0%)
Same Version:6 (100.0%)
Same OS:2 (33.3%)
From: nathanielzimmermann at gmail dot com Assigned: dmitry (profile)
Status: Closed Package: Scripting Engine problem
PHP Version: 7.3.0alpha1 OS:
Private report: No CVE-ID: None
View Developer Edit
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: nathanielzimmermann at gmail dot com
New email:
PHP Version: OS:

 

 [2018-06-09 11:26 UTC] nathanielzimmermann at gmail dot com 
Description:
------------
token_get_all with the TOKEN_PARSE flag fails to recognise the T_CLOSE_TAG token

Test script:
---------------
<?php

var_dump(token_get_all('<?php echo 1; ?>', TOKEN_PARSE)[6]);

Expected result:
----------------
array(3) {
  [0]=>
  int(381)
  [1]=>
  string(2) "?>"
  [2]=>
  int(1)
}

Actual result:
--------------
string(2) "?>"

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2018-06-09 13:38 UTC] cmb@php.net
-Status: Open +Status: Verified -Assigned To: +Assigned To: dmitry
 [2018-06-09 13:38 UTC] cmb@php.net 
It seems that this behavioral change is an unintended side effect
of commit 8afb91c[1].  Dmitry, could you have a look at this
issue, please?

[1] <http://git.php.net/?p=php-src.git;a=commit;h=8afb91cdadaa5a40ee972dec278d89c76bf454e3>
 [2018-06-17 14:32 UTC] laruence@php.net 
I am afraid that the special cases should be handled in tokenizer side now :<

diff --git a/ext/tokenizer/tokenizer.c b/ext/tokenizer/tokenizer.c
index ef9d136..9524c69 100644
--- a/ext/tokenizer/tokenizer.c
+++ b/ext/tokenizer/tokenizer.c
@@ -191,8 +191,16 @@ void on_event(zend_php_scanner_event event, int token, int line, void *context)
 
        switch (event) {
                case ON_TOKEN:
-                   if (token == END) break;
-                   add_token(token_stream, token, LANG_SCNG(yy_text), LANG_SCNG(yy_leng), line);
+                 {
+                         if (token == END) break;
+                         /* Specical cases */
+                         if (token == ';' && LANG_SCNG(yy_leng) == 2) {
+                                 token = T_CLOSE_TAG;
+                         } else if (token == T_ECHO && LANG_SCNG(yy_leng) == 3) {
+                                 token = T_OPEN_TAG_WITH_ECHO;
+                         }
+                         add_token(token_stream, token, LANG_SCNG(yy_text), LANG_SCNG(yy_leng), line);
+                 }
                        break;
                case ON_FEEDBACK:
                        tokens_ht = Z_ARRVAL_P(token_stream);
 [2018-06-17 21:00 UTC] cmb@php.net 
Thanks!  A small improvement might be to replace the integer
literals with sizeof("?>")-1 and sizeof("<?=")-1, respectively.
 [2018-06-18 03:35 UTC] laruence@php.net 
Automatic comment on behalf of laruence@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=4d69bbeee79c6a94348935ad071b6c7c05dd8eae
Log: Fixed bug #76437 (token_get_all with TOKEN_PARSE flag fails to recognise close tag)
 [2018-06-18 03:35 UTC] laruence@php.net
-Status: Verified +Status: Closed
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Thu Nov 21 12:01:29 2024 UTC