php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #33857 token_get_all() inconsistent results?
Submitted: 2005-07-26 00:04 UTC Modified: 2005-08-03 01:00 UTC
Votes:1
Avg. Score:4.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:0 (0.0%)
From: sr at brightlight dot ch Assigned:
Status: No Feedback Package: Scripting Engine problem
PHP Version: 5.0.4 OS: Mac OS X 10.4.1
Private report: No CVE-ID: None
View Developer Edit
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: sr at brightlight dot ch
New email:
PHP Version: OS:

 

 [2005-07-26 00:04 UTC] sr at brightlight dot ch 
Description:
------------
Obviously the same problem as in Bug #33093, which is set to 
bogus (no comment).

When I tokenize the very same code over and over again I 
happen to get different tokens. About 9 times out of 10 I 
get:
Token'<',
Token '?',
Token T_STRING 'php',
Token T_WHITESPACE '\n',

About 1 time out of 10 Times I get
Token T_OPEN_TAG '<?php\n',

So the first case happens far more often.

By saying the same code btw. I mean that I put the 
tokenizing function into a for-loop. So the code doesn't 
even get touched.
PHP 5.0.4 (entropy.ch's binary), Apache 2.0.54, Mac OS X 
10.4.1 PPC.
The tokenized files encoding (I tried UTF-8 no BOM and ISO-
Latin-1) doesn't seem to play a role. Line endings are 
always Unix-style.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2005-07-26 10:27 UTC] sniper@php.net 
Please try using this CVS snapshot:

  http://snaps.php.net/php5-latest.tar.gz
 
For Windows:
 
  http://snaps.php.net/win32/php5-win32-latest.zip
 [2005-08-01 21:45 UTC] phpbugs at majiclab dot com 
I am experiencing similar problems on my Mac OSX 10.4.2 system 
(although I don't think it's just Mac related, just a 
coincidence probably).  Anyway, I am downloading the CVS now 
and will try it out in the next couple days.  The problem 
seems to go away for a bit after restarting Apache, and comes 
up randomly, so I might "think" it's gone for a couple days, 
and then it might show up later.  So I'll try it out for a 
couple days and see what happens.

Thanks!
 [2005-08-01 22:40 UTC] phpbugs at majiclab dot com 
I ran this modified code with the CVS and it seems to work 
so far.  However, I had problems compiling some required 
modules and I don't have enough time to play with those now, 
so I had to revert back to 5.0.4.  However, since I was 
unable to test it extensively, I am not sure if it truly 
fixed the bug.  It's working on 5.0.4 again now but that's 
because I had to restart  Apache after reverting to the old 
version... :)

So I can't confirm whether it's been fixed yet or not.  I 
might try to test it out on one of my non-dev boxes that are 
just sitting around collecting dust.  But just to confirm, 
the best debugging I was able to extrapolate (without even 
looking at any C code) was this:  It seemed that whatever 
internals/functions were called by token_get_all() would 
have had some kind of boolean to determine whether or not 
the current location of the code is within <?php ?> brackets 
or not.  My best guess is that somehow that variable was 
already set to TRUE even though it hasn't encountered the <?
php yet.  That would seem to be why the <?php was being 
interpreted as a < (less than), a ? (?: operator) and then a 
"string" (php being a function or constant name) and then 
whitespace rather than just the open tag.

Anyway, I'm sorry I can't provide better testing right now, 
but I've got work to do and I can live around this for now.
 [2005-08-03 01:00 UTC] php-bugs at lists dot php dot net 
No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Sat Jul 12 16:01:33 2025 UTC