php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #80462 Nullsafe operator tokenize with TOKEN_PARSE flag fails
Submitted: 2020-12-02 08:51 UTC Modified: -
From: possumfromspace at gmail dot com Assigned:
Status: Closed Package: *General Issues
PHP Version: 8.0.0 OS: ubuntu
Private report: No CVE-ID: None
 [2020-12-02 08:51 UTC] possumfromspace at gmail dot com
Description:
------------
On:
PHP 8.0.0 (cli) (built: Dec  1 2020 04:01:52) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.0-dev, Copyright (c) Zend Technologies


When tokenizing as following:

```
<?php token_get_all('<?php $foo = $a?->b();', TOKEN_PARSE);
```

the T_NULLSAFE_OBJECT_OPERATOR is missing

https://3v4l.org/t5aAA

Test script:
---------------
```
<?php token_get_all('<?php $foo = $a?->b();', TOKEN_PARSE);
```

results in something like:
```

array(10) {
  [0]=>
  array(3) {
    [0]=>
    int(390)
    [1]=>
    string(6) "<?php "
    [2]=>
    int(1)
  }
  [1]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(4) "$foo"
    [2]=>
    int(1)
  }
  [2]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [3]=>
  string(1) "="
  [4]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [5]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(2) "$a"
    [2]=>
    int(1)
  }
  [6]=>
  array(3) {
    [0]=>
    int(311)
    [1]=>
    string(1) "b"
    [2]=>
    int(1)
  }
  [7]=>
  string(1) "("
  [8]=>
  string(1) ")"
  [9]=>
  string(1) ";"
}
```

expected same result as 


```
<?php token_get_all('<?php $foo = $a?->b();');
```


```
array(11) {
  [0]=>
  array(3) {
    [0]=>
    int(390)
    [1]=>
    string(6) "<?php "
    [2]=>
    int(1)
  }
  [1]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(4) "$foo"
    [2]=>
    int(1)
  }
  [2]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [3]=>
  string(1) "="
  [4]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [5]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(2) "$a"
    [2]=>
    int(1)
  }
  [6]=>
  array(3) {
    [0]=>
    int(387)
    [1]=>
    string(3) "?->"
    [2]=>
    int(1)
  }
  [7]=>
  array(3) {
    [0]=>
    int(311)
    [1]=>
    string(1) "b"
    [2]=>
    int(1)
  }
  [8]=>
  string(1) "("
  [9]=>
  string(1) ")"
  [10]=>
  string(1) ";"
}
```

note the T_NULLSAFE_OBJECT_OPERATOR token @ index 6 

https://3v4l.org/t5aAA



Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2020-12-02 09:19 UTC] nikic@php.net
Automatic comment on behalf of nikita.ppv@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=7a61984a2bf42f2632a8c62245a8496a3e2009cd
Log: Fixed bug #80462
 [2020-12-02 09:19 UTC] nikic@php.net
-Status: Open +Status: Closed
 
PHP Copyright © 2001-2021 The PHP Group
All rights reserved.
Last updated: Wed Jan 20 14:01:23 2021 UTC