PHP :: Bug #33857 :: token_get_all() inconsistent results?

[2005-07-26 00:04 UTC] sr at brightlight dot ch

Description:
------------
Obviously the same problem as in Bug #33093, which is set to 
bogus (no comment).

When I tokenize the very same code over and over again I 
happen to get different tokens. About 9 times out of 10 I 
get:
Token'<',
Token '?',
Token T_STRING 'php',
Token T_WHITESPACE '\n',

About 1 time out of 10 Times I get
Token T_OPEN_TAG '<?php\n',

So the first case happens far more often.

By saying the same code btw. I mean that I put the 
tokenizing function into a for-loop. So the code doesn't 
even get touched.
PHP 5.0.4 (entropy.ch's binary), Apache 2.0.54, Mac OS X 
10.4.1 PPC.
The tokenized files encoding (I tried UTF-8 no BOM and ISO-
Latin-1) doesn't seem to play a role. Line endings are 
always Unix-style.

[2005-07-26 10:27 UTC] sniper@php.net

Please try using this CVS snapshot:

  http://snaps.php.net/php5-latest.tar.gz
 
For Windows:
 
  http://snaps.php.net/win32/php5-win32-latest.zip

[2005-08-01 21:45 UTC] phpbugs at majiclab dot com

I am experiencing similar problems on my Mac OSX 10.4.2 system 
(although I don't think it's just Mac related, just a 
coincidence probably).  Anyway, I am downloading the CVS now 
and will try it out in the next couple days.  The problem 
seems to go away for a bit after restarting Apache, and comes 
up randomly, so I might "think" it's gone for a couple days, 
and then it might show up later.  So I'll try it out for a 
couple days and see what happens.

Thanks!

[2005-08-01 22:40 UTC] phpbugs at majiclab dot com

I ran this modified code with the CVS and it seems to work 
so far.  However, I had problems compiling some required 
modules and I don't have enough time to play with those now, 
so I had to revert back to 5.0.4.  However, since I was 
unable to test it extensively, I am not sure if it truly 
fixed the bug.  It's working on 5.0.4 again now but that's 
because I had to restart  Apache after reverting to the old 
version... :)

So I can't confirm whether it's been fixed yet or not.  I 
might try to test it out on one of my non-dev boxes that are 
just sitting around collecting dust.  But just to confirm, 
the best debugging I was able to extrapolate (without even 
looking at any C code) was this:  It seemed that whatever 
internals/functions were called by token_get_all() would 
have had some kind of boolean to determine whether or not 
the current location of the code is within <?php ?> brackets 
or not.  My best guess is that somehow that variable was 
already set to TRUE even though it hasn't encountered the <?
php yet.  That would seem to be why the <?php was being 
interpreted as a < (less than), a ? (?: operator) and then a 
"string" (php being a function or constant name) and then 
whitespace rather than just the open tag.

Anyway, I'm sorry I can't provide better testing right now, 
but I've got work to do and I can live around this for now.

[2005-08-03 01:00 UTC] php-bugs at lists dot php dot net

No feedback was provided for this bug for over a week, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Patches

Pull Requests

History

	php.net \| support \| documentation \| report a bug \| advanced search \| search howto \| statistics \| random bug \| login
go to bug id or search bugs for


Copyright © 2001-2024 The PHP Group All rights reserved.	Last updated: Fri May 17 18:01:33 2024 UTC