php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #46957 The tokenizer returns deprecated values
Submitted: 2008-12-27 23:00 UTC Modified: 2008-12-27 23:17 UTC
From: fjas at free dot fr Assigned:
Status: Closed Package: Unknown/Other Function
PHP Version: 5.3.0alpha3 OS: Win XP
Private report: No CVE-ID: None
View Developer Edit
 [2008-12-27 23:00 UTC] fjas at free dot fr 
Description:
------------
token_get_all() displays an old set of constants, not the current 5.3 ones.
For instance, T_NS_SEPARATOR is not defined.
Thus, the symbol \ is not recognized by the tokenizer (UNKNOWN), and namespaced PHP code is not correctly parsed by token_get_all()

---
PHP Version 5.3.0alpha4-dev
Build date:Dec 27 2008 22:05:04 

Reproduce code:
---------------
foreach(token_get_all('<? namespace A\B\C;?>') as $c)
{
    if(is_array($c))
    {
        print(token_name($c[0]))."\n";
    }
}

Expected result:
----------------
T_OPEN_TAG
T_WHITESPACE
T_NAMESPACE
T_WHITESPACE
T_STRING
T_NS_SEPARATOR
T_STRING
T_NS_SEPARATOR
T_STRING
T_CLOSE_TAG


Actual result:
--------------
T_OPEN_TAG
T_WHITESPACE
T_NAMESPACE
T_WHITESPACE
T_STRING
UNKNOWN
T_STRING
UNKNOWN
T_STRING
T_CLOSE_TAG

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2008-12-27 23:17 UTC] felipe@php.net 
This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Thu Oct 30 22:00:01 2025 UTC