php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #33997 Returned: Bug #16069 - ICONV transliteration failure
Submitted: 2005-08-04 20:23 UTC Modified: 2013-02-18 00:16 UTC
Votes:7
Avg. Score:3.7 ± 0.9
Reproduced:6 of 6 (100.0%)
Same Version:4 (66.7%)
Same OS:3 (50.0%)
From: RVaughn at pheedo dot com Assigned:
Status: No Feedback Package: ICONV related
PHP Version: 5.*, 6CVS (2009-04-25) OS: *
Private report: No CVE-ID: None
View Developer Edit
 [2005-08-04 20:23 UTC] RVaughn at pheedo dot com 
Description:
------------
In the PHP 4.4 test suite, the test for closed bug #16069 fails:

FAIL Bug #16069 [ext/iconv/tests/bug16069.phpt]

Checked the output and it does diverge from the expected output in the same way the old bug from 2002 did.  The 'iconv' libraries are up-to-date on the system:

# up2date glibc-common
The following packages you requested are already updated:
glibc-common

# rpm -qi glibc-common
Name: glibc-common           Relocations: (not relocatable)
Version: 2.3.2               Vendor: Red Hat, Inc.
Release: 95.33               Build Date: Wed 23 Feb 2005

The bug16069.phpt script will reproduce this every time.  Configured with the following settings:

./configure  '--host=i386-redhat-linux' '--build=i386-redhat-linux' '--target=i386-redhat-linux-gnu' '--cache-file=../config.cache' '--with-config-file-path=/etc' '--with-config-file-scan-dir=/etc/php.d' '--enable-force-cgi-redirect' '--disable-debug' '--enable-pic' '--disable-rpath' '--enable-inline-optimization' '--with-bz2' '--with-db4=/usr' '--with-curl' '--with-freetype-dir=/usr' '--with-png-dir=/usr' '--with-gd' '--enable-gd-native-ttf' '--without-gdbm' '--with-gettext' '--with-ncurses' '--with-gmp' '--with-iconv' '--with-jpeg-dir=/usr' '--with-openssl' '--with-png' '--with-aspell' '--with-regex=system' '--with-xml' '--with-expat-dir=/usr' '--with-dom=shared,/usr' '--with-dom-xslt=/usr' '--with-dom-exslt=/usr' '--with-xmlrpc=shared' '--with-pcre-regex=/usr' '--with-zlib' '--with-layout=GNU' '--enable-bcmath' '--enable-exif' '--enable-ftp' '--enable-magic-quotes' '--enable-safe-mode' '--enable-sockets' '--enable-sysvsem' '--enable-sysvshm' '--enable-discard-path' '--enable-track-vars' '--enable-trans-sid' '--disable-yp' '--enable-wddx' '--without-oci8' '--with-pear=/usr/share/pear' '--with-imap=shared' '--with-imap-ssl' '--with-kerberos' '--with-ldap=shared' '--with-mysql=shared,/usr' '--with-snmp=shared,/usr' '--with-snmp=shared' '--enable-ucd-snmp-hack' '--with-unixODBC=shared' '--enable-memory-limit' '--enable-bcmath' '--enable-shmop' '--enable-calendar' '--enable-dbx' '--enable-dio' '--enable-mcal' '--enable-mbstring' '--enable-mbstr-enc-trans' '--enable-mbregex' '--with-apxs2=/usr/sbin/apxs'

Using a standard /etc/php.ini file that works fine under PHP 4.3.6 which we upgraded to several weeks ago and this bug does not appear to be a problem with.

Got the following warning (the only one) when doing a 'make':

/usr/local/src/Tars/php-4.4.0/ext/xmlrpc/libxmlrpc/encodings.c:
In function `convert': /usr/local/src/Tars/php-4.4.0/ext/xmlrpc/libxmlrpc/encodings.c:74:
warning: passing arg 2 of `iconv' from incompatible pointer type

Guessing that the call to updated versions of 'libxml' or 'libxml2' is the problem, here's the versions for both:

# rpm -qi libxml
Name: libxml                 Relocations: (not relocatable)
Version: 1.8.17              Vendor: Red Hat, Inc.
Release: 9.2                 Build Date: Wed 17 Nov 2004

# rpm -qi libxml2
Name: libxml2                Relocations: /usr 
Version: 2.5.10              Vendor: Red Hat, Inc.
Release: 7                   Build Date: Wed 27 Oct 2004


Reproduce code:
---------------
Please use .../ext/iconv/tests/bug16069.phpt in the test suite.

Expected result:
----------------
The output returned does not match expected results.

Actual result:
--------------
This bug *cannot* be safely ignored as per the "Won't Fix" status in Bug #32367 because it is breaking our ability to use Japanese characters in our application with the 'mbstring' libraries.  As long as we use English it's fine, but it blows up our Japanese translations (via Smarty templates.)  Please do not claim it can be ignored or is a "Won't Fix" as multi-language support is becoming more and more necessary with PHP and this bug is causing major problems with it.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2005-08-04 22:25 UTC] RVaughn at pheedo dot com 
Was getting one warning in the 'make' output as reported, fixed the code in:

.../php-4.4.0/ext/xmlrpc/libxmlrpc/encodings.c

and now do not get a warning in the 'make' but it did not fix the problem, and multi-byte string output for Japanese text is being messed up by this reopened? never-fixed? bug.

Here's the diff to fix the warning for encodings.c, very simple:

64c64
<       char const *src_ptr = src;
---
>       char* src_ptr = (char*) src;

It clearly states the iconv() parameters and you can't pass a 'const' into the call, period.
 [2005-08-04 22:29 UTC] RVaughn at pheedo dot com 
Q: How exactly does it fail.

A: Cannot reproduce here, as it's a string of Japanese multi-byte characters which would just be garbage if cut and pasted here.  All I can say is that any char buffer string of a size greater than 50? 60? bytes long gets scrambled after that point, everything is fine up to it but not after.

There's no way for me to post the Japanese characters and since I don't speak Japanese, I am going on our developer in Japan's testing, and he says that short strings are translating fine but longer ones get garbled around that point.

Sorry that I can't provide more detailed info and cannot give you an example, but if you run the test script in the PHP test suite as I mentioned on this platform, you will get the same failure and the fact that a PHP-provided test is failing should be enough to (1) reopen the original bug #16069 as unresolved on this platform and (2) provide a way to go about seeing the failure and trying to figure it out.
Thanks!
 [2005-08-04 22:42 UTC] tony2001@php.net 
If it fails, you should get *.diff file which clearly shows what did you get instead of the result expected.
And you don't have to post something, you can just put the code and it's output somewhere in the Net and post a link here.
 [2005-08-04 23:32 UTC] RVaughn at pheedo dot com 
OK, here's the diff output from the test results, please note that the output is the same until part way through, and the ultimate length is different:

===============================================================================
/usr/local/src/Tars/php-4.4.0/ext/iconv/tests/bug16069.phpt
===============================================================================

---- EXPECTED OUTPUT
?ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С???(?맥??)
---- ACTUAL OUTPUT
?ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С????ߥ??С???(?맥??)
---- FAILED
 [2005-08-04 23:34 UTC] RVaughn at pheedo dot com 
Do let me know if you want me to put the output somewhere on our site where it can be downloaded, the code itself is just the PHP-provided test: bug16069.phpt.  Thanks!  Cheers!
 [2005-08-07 14:42 UTC] derick@php.net 
Please provide a location where we can download your failed test's .out and .exp file.
 [2005-08-08 00:10 UTC] sniper@php.net 
Assigning to Derick who's comment about my patch was "It's funny" and other comment about the iconv code "this is a mess"
:)
 [2009-04-27 16:13 UTC] jani@php.net 
Still fails.
 [2010-06-13 15:00 UTC] felipe@php.net
-Status: Assigned +Status: Feedback -Assigned To: derick +Assigned To:
 [2010-06-13 15:00 UTC] felipe@php.net 
Please try using this snapshot:

  http://snaps.php.net/php5.2-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/
 [2013-02-18 00:16 UTC] php-bugs at lists dot php dot net 
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Open". Thank you.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Thu Dec 26 11:01:30 2024 UTC