With token_get_all(), the return on win32 is slightly different to on *nix. The difference is with all those characters that do not have a T_FOO name, which results in them not being in an array, my problem is that the character in there is not correct.

Win32 var_dump output:
[29]=>
string(3) "; U"

*nix var_dump output:
[29]=>
string(1) ";"

require_once 'PEAR/Command/Common.php'U is the printed output on win32

require_once 'PEAR/Command/Common.php'; is the printed output for the same line on *nix (FBSD 4.4).

Every single one of these characters is
string(3) "CHAR U" which makes it impossible to even work around using str_replace or the like.

I believe the U is a \r or some such... not *too* sure. I think that might be mozillas way of showing unknown characters within <pre> output.

Sample script:

<?php
    $tokens = token_get_all(file_get_contents($_SERVER['SCRIPT_FILENAME']));
    echo "<pre>";
    var_dump($tokens);
    echo "</pre>";
?>

- Davey