|
php.net | support | documentation | report a bug | advanced search | search howto | statistics | random bug | login |
[2012-12-09 07:38 UTC] jmichae3 at yahoo dot com
Description:
------------
I am getting russian characters in my meail forms. I want to compare the characters to see if they are > '~' which is the last visible character in the ascii character set.
this comparison does not work. in UNICODE, these characters are about 1024, and ~ is 126 according to ord().
ord() thinks EVERY character is ascii. this is far from true. there are mb characters from utf8.
this is russian random characters from charmap: ЋϊЁγϋГИБЫЫЏАДрмдп
in fact, I don't have any working way to detect whether a character is KOI8-R or ASCII, or cyrillic, or whether the character ordinal number is actually beyond 127 or not. because according to ord(), it's all within 0-255.
Test script:
---------------
<?php
$s="п"; //russian character
echo substr_compare($s,"~",0,1);
echo "\n";
$i=0;
for ($i=0; $i < strlen($s); $i++) {
if (substr_compare($s[$i],"~",0,1) > 0) {
echo "OK";
} else {
echo "fail";
}
if (ord($s[$i]) > 126) {
echo "OK";
} else {
echo "fail";
}
if ($s[$i] > '~') {
echo "OK";
} else {
echo "fail";
}
echo ord($s[$i]);
}
echo "\n";
$i=0;
/*
strangely enough, I get 2 outputs with only 1 character.
Sat 12/08/2012 23:12:46.76||E:\www\jimm|>php t.php
1
OKOKOK208OKOKOK191
Sat 12/08/2012 23:14:27.34||E:\www\jimm|>
*/
?>
Expected result:
----------------
whole characters as a single unit. 1 result.
Actual result:
--------------
got 2 results from 1 UNICODE russian character in a string. should only get 1.
this file was encoded with utf8 without bom.
php is splitting the utf8 characters into a byte stream when it gets to strlen(). or it just treats unicode and utf8 characters like ascii.
this does not work well when trying to use mb_detect_encoding() - that breaks ability to detect encodings when it breaks up characters like that. nearly everything with strings actually.
this also breaks ability to detect foreign spam.
PatchesPull RequestsHistoryAllCommentsChangesGit/SVN commits
|
|||||||||||||||||||||||||||
Copyright © 2001-2025 The PHP GroupAll rights reserved. |
Last updated: Wed Nov 05 07:00:01 2025 UTC |
Personally I'd just convert from utf8 to iso-8959-1 or whichever encoding you are looking for here instead of checking each character. But if you really do want to do it, it isn't very hard. You just need to understand what UTF-8 looks like and it becomes a simple 5-line function in userspace: function utf8_ord($c) { $b0 = ord($c[0]); if($b0 < 0x10) return $b0; $b1 = ord($c[1]); if($b0 < 0xE0 )return (($b0 & 0x1F) << 6) + ($b1 & 0x3F); return (($b0 & 0x0F) << 12) + (($b1 & 0x3F) << 6) + (ord($c[2]) & 0x3F); } But you have to understand that there is absolutely no way to accurately detect the encoding of a short sequence of bytes. The above will work if you know the input is UTF-8. There is no way to write a magic function which will tell you the encoding from a couple of bytes of data which you seem to imply we should provide you.this code might be moreuseful, I am going to give it to you. I know there is unicode-16 and unicode-32 and such. if the string can hanbdle stuff like that, there really should be an internal function for that which also handles this internally. because although this is useful and I can use it, it is a workaround rather than a real and complete solution for multiple encodings such as you would find listed with mb_list_encodings(). //returns ordinal value of character in string $str at $index //and increments $index past current utf-8 character. function utf8_ord_next_char($str, &$index) { $b0 = ord($str[$index + 0]); if ($b0 < 0x10) { $index++; return $b0; } $b1 = ord($str[$index + 1]); if ($b0 < 0xE0) { $index += 2; return (($b0 & 0x1F) << 6) + ($b1 & 0x3F); } $index += 3; return (($b0 & 0x0F) << 12) + (($b1 & 0x3F) << 6) + (ord($str[$index + 2]) & 0x3F); } so for detecting non-ascii languages, //detect foreign languages for ($i=0;$i < strlen($comment);) { if (utf8_ord_next_char($comment,$i) > 126) { echo "<div style='color:red;'>ERRORb</div>"; return true; //error } }