php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #61484 iconv //IGNORE option doesn't work anymore in PHP5.4/5.5
Submitted: 2012-03-22 23:59 UTC Modified: 2016-07-29 08:31 UTC
Votes:36
Avg. Score:4.7 ± 0.5
Reproduced:35 of 35 (100.0%)
Same Version:19 (54.3%)
Same OS:17 (48.6%)
From: juzna dot cz at gmail dot com Assigned: cmb
Status: Duplicate Package: ICONV related
PHP Version: 5.5.0 OS: Linux
Private report: No CVE-ID:
Password:
Status:
Package:
Bug Type:
Summary:
From: juzna dot cz at gmail dot com
New email:
PHP Version: OS:

 

 [2012-03-22 23:59 UTC] juzna dot cz at gmail dot com
Description:
------------
When adding "//IGNORE" to target encoding in iconv function, wrong characters 
should be skipped. This worked in PHP 5.3, but doesn't work anymore in 5.4.

I just compiled 5.3.10, where it works fine. Doesn't work on 5.4.0 nor on 5.4.1-
RC1.

Test script:
---------------
var_dump(iconv('UTF-8', 'UTF-16//IGNORE', "\xc5\xbea\x01b\xed\xa0\x80c\xef\xbb\xbfd\xf4\x90\x80\x80e"));

Expected result:
----------------
string(18) "��~abc��de"

Actual result:
--------------
bool(false)

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2012-03-23 07:01 UTC] laruence@php.net
this should be a issue of glibc, but I am not sure since when it has been 
changed. :(
 [2012-03-23 07:14 UTC] rasmus@php.net
Not sure this is a libc thing. 5.3 and 5.4 on the same libc behave differently.
They both report:

iconv implementation => glibc
iconv library version => 2.13
 [2012-03-25 08:15 UTC] laruence@php.net
@rasmus yes, 5.3 and 5.4 behavior inconsistently, that is because icovn doesn't 
accept //IGNORE anymore, and now return -1,  then comes a bug(see #52211), felipe 
fixed this in 5.4,  but didn't fix 5.3.  so now, 5.3 and 5.4 behave differently, 
but the fundamental reason seems still is the glibc change their iconv behavior. 
thanks, :)
 [2012-05-14 06:56 UTC] juzna dot cz at gmail dot com
The //ignore option is still broken as of the latest stable release (5.4.3).  

From reading some of the other bug reports on this, apparently new versions of iconv no longer support the //ignore option, and instead -c must be passed.  So PHP needs to detect the //ignore option and in that case pass the "-c" option to iconv.
 [2012-08-28 16:55 UTC] hrach dot cz at gmail dot com
PHP 5.4.6
iconv implementation      libiconv
iconv library version     1.11

works!
 [2012-09-13 08:09 UTC] alec at alec dot pl
The bug exists in PHP 5.3.10 and Ubuntu EGLIBC 2.15-0ubuntu10
 [2012-11-02 13:45 UTC] valeriy-v at yandex dot ru
PHP Version 5.4.8 Centos
Linux 2.6.32-279.11.1.el6.x86_64 #1 SMP Tue Oct 16 15:57:10 UTC 2012 x86_64
 [2013-08-02 01:56 UTC] yohgaki@php.net
-Summary: iconv //IGNORE option doesn't work anymore in PHP5.4 +Summary: iconv //IGNORE option doesn't work anymore in PHP5.4/5.5 -Operating System: Ubuntu +Operating System: Linux -PHP Version: 5.4.0 +PHP Version: 5.5.0
 [2013-08-02 01:56 UTC] yohgaki@php.net
Fedora19 x86_64's php

php > var_dump(iconv('UTF-8', 'UTF-16//IGNORE', 
"\xc5\xbea\x01b\xed\xa0\x80c\xef\xbb\xbfd\xf4\x90\x80\x80e")); 

Notice: iconv(): Detected an illegal character in input string in php shell code 
on line 1
bool(false)
php > echo phpversion();
5.5.0
 [2013-08-02 02:01 UTC] yohgaki@php.net
-Type: Bug +Type: Feature/Change Request
 [2013-08-02 02:01 UTC] yohgaki@php.net
iconv() returns FALSE when there is error. I haven't checked if out_buffer is 
usable, though.

ext/iconv/iconv.c
    err = php_iconv_string(in_buffer, (size_t)in_buffer_len,
        &out_buffer, &out_len, out_charset, in_charset);
    _php_iconv_show_error(err, out_charset, in_charset TSRMLS_CC);
    if (err == PHP_ICONV_ERR_SUCCESS && out_buffer != NULL) {
        RETVAL_STRINGL(out_buffer, out_len, 0);
    } else {
        if (out_buffer != NULL) {
            efree(out_buffer);
        }
        RETURN_FALSE;
    }
 [2013-08-02 02:11 UTC] yohgaki@php.net
It seems usable.

$ php -r 'echo "\xc5\xbea\x01b\xed\xa0\x80c\xef\xbb\xbfd\xf4\x90\x80\x80e";' | 
iconv -f 'UTF-8' -t 'UTF-16//IGNORE' | od -t x1
iconv: 位置 18 に不正な入力シーケンスがあります
0000000 ff fe 7e 01 61 00 01 00 62 00 63 00 ff fe 64 00
0000020 65 00
0000022
 [2015-06-05 10:36 UTC] php at s-hamel dot de
This issue seems to get fixed in PHP 5.5.26.

See http://git.php.net/?p=php-src.git;a=blob_plain;f=NEWS;hb=473ec539a1c3d242c8b171dd6a5a98fa17e05c13

- Iconv:
  . Fixed bug #48147 (iconv with //IGNORE cuts the string). (Stas)

It would be great to have it fixed for PHP 5.4 as well.

Best regards
S.
 [2016-07-29 08:31 UTC] cmb@php.net
-Status: Open +Status: Duplicate -Type: Feature/Change Request +Type: Bug -Assigned To: +Assigned To: cmb
 [2016-07-29 08:31 UTC] cmb@php.net
Actually, this is a duplicate of bug #48147.
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Tue Aug 29 15:01:52 2017 UTC