php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #40749 pack and unpack erroneous behavior on 64bits hosts
Submitted: 2007-03-07 17:12 UTC Modified: 2007-04-05 19:59 UTC
Votes:19
Avg. Score:4.8 ± 0.4
Reproduced:18 of 18 (100.0%)
Same Version:18 (100.0%)
Same OS:6 (33.3%)
From: ben at ateor dot com Assigned: iliaa (profile)
Status: Closed Package: Unknown/Other Function
PHP Version: 5.2.1 OS: OpenBSD amd64 and sparc64
Private report: No CVE-ID: None
Welcome back! If you're the original bug submitter, here's where you can edit the bug or add additional notes.
If you forgot your password, you can retrieve your password here.
Password:
Status:
Package:
Bug Type:
Summary:
From: ben at ateor dot com
New email:
PHP Version: OS:

 

 [2007-03-07 17:12 UTC] ben at ateor dot com
Description:
------------
This is a follow-up on #40543 (http://bugs.php.net/bug.php?id=40543, since
that bug is closed, I can't add comments). 
Please note : it's not identical to #4053 (other weird behaviors 
are demonstrated).

Iliaa,
Not sure why you suggest to use little endian or host conversions routines,
but in my standpoint if you reverse twice a number's byte ordering
then you should get the original number back (assuming the number don't
overflows php's internals).

At least, that's the standard behavior for perl and python.

Beside, I can't see why php should handles those endianness conversions
differently on an i386 (32 bits) and on an x86_64 (64 bits), both
having the same byte order.

The following on a 64bit, little endian host :
x86_64$ uname -mprsv
OpenBSD 4.0 GENERIC#0 amd64 AMD Sempron(tm) Processor 3400+

x86_64$ perl -e 'print unpack("N", pack("N", 41445)) ."\n"'
41445

x86_64$ python
Python 2.4.3 (#1, Sep  6 2006, 20:33:08)
[GCC 3.3.5 (propolice)] on openbsd4
Type "help", "copyright", "credits" or "license" for more information.
>>> from struct import *
>>> unpack('>L', pack('>L', 41445))
(41445L,)

And, indeed :
#include <stdio.h>
#include <sys/types.h>
int main(void)
{
        u_int32_t x, y, z; /* 32 bits unsigned longs */
        x = 41445;
        /* conv host (little) to network (big endian) long : pack("N", 41445) */
        y = htonl(x);
        /* conv network (big endian) to host (little) long : unpack("N", ...) */
        z = ntohl(y);
        printf("Host : %li\nBig : %li\nHost : %li\n", x, y, z);
        return 0;
}

x86_64$ gcc conv.c -o conv ; ./conv
Host : 41445
Big : -442433536
Host : 41445

But still (PHP 5.2.2-dev (cli) (built: Feb 27 2007 22:10:11)) :
x86_64$ php -r 'print_r(unpack("N", pack("N", 41445)));'
Array
(
    [1] => -2147442203
)

While on a plain old x86 little endian host (PHP 4.4.0), we get
a different result :
i386_32$ uname -mprsv
OpenBSD 3.9 GENERIC#0 i386 Intel(R) Pentium(R) 4 CPU 1.70GHz ("GenuineIntel" 686-class)
i386_32$ php -r 'print_r(unpack("N", pack("N", 41445)));'
Array
(
    [1] => 41445
)

Still on the 64 bits little endian host :
x86_64$ php -r '$a = unpack("N", pack("N", 65536)); printf("$a[1]\n");'
65536           # Ok
x86_64$ php -r '$a = unpack("N", pack("N", 65535)); printf("$a[1]\n");'
-2147418113     # Weird

x86_64$ php -r '$a = unpack("N", pack("N", 0x1234)); printf("0x%x\n", $a[1]);'
0x1234     # Ok
x86_64$ php -r '$a = unpack("L", pack("N", 0x1234)); printf("0x%x\n", $a[1]);'
0x34120000 # Ok
x86_64$ php -r '$a = unpack("L", pack("L", 0xffff)); printf("0x%x\n", $a[1]);'
0xffff     # Ok
x86_64$ php -r '$a = unpack("N", pack("N", 0xffff)); printf("0x%x\n", $a[1]);'
0xffffffff8000ffff # The doc says "N" gives you "always 32 bit", and we get
                   # 8 bytes. No wonder why we overflow.
x86_64$ php -r '$a = unpack("N", pack("N", 0xff )); printf("0x%x\n", $a[1]);'
0xffffffff800000ff # Same. Don't tell me 0xff is too large.

And now, all the following tests are on a 64 bits _big endian_ host
(sparc64, running php-5.2.1) :
sparc64$ uname -mprsv
OpenBSD 3.8 GENERIC#607 sparc64 SUNW,UltraSPARC-IIi @ 440 MHz, version 0 FPU
sparc64$ php -r '$a = unpack("N", pack("N", 0xffff)); printf("0x%x\n", $a[1]);'
0xffff # Ok
# The same, but prefixing 0000 to the argument :
sparc64$ php -r '$a = unpack("N", pack("N", 0x0000ffff)); printf("0x%x\n", $a[1]);'
0xffffffff8000ffff
# Weird (and with "N", we stayed on the host byte order this time).
# Shouldn't 0xffff == 0x0000ffff, even on big endian ? Apparently, yes :
sparc64$ php -r 'printf("0x%x\n", 0x0000ffff);'
0xffff

# Also, look at this :
sparc64$ php -r '$a = unpack("N", pack("N", 41445)); printf("$a[1]\n");'
41445
# And now let's just remove the line feed (\n) from the above printf :
sparc64$ php -r '$a = unpack("N", pack("N", 41445)); printf("$a[1]");'
-2147442203

# Same for 2^16 -1 / 65535 / 0xfff :
sparc64$ php -r '$a = unpack("N", pack("N", 65535)); printf("$a[1]\n");'
65535
sparc64$ php -r '$a = unpack("N", pack("N", 65535)); printf("$a[1]");'
-2147418113

# We get the opposite (bogus with \n, correct without) when converting
# to little endian and back to host :
sparc64$ php -r '$a = unpack("L", pack("L", 0xffff)); printf( $a[1]);'
65535
sparc64$ php -r '$a = unpack("L", pack("L", 0xffff)); printf( $a[1]."\n");'
-2147418113


This doesn't help :
SKIP Generic pack()/unpack() tests [ext/standard/tests/strings/pack.phpt] reason: 32bit test only




Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2007-03-09 09:06 UTC] windeler at mediafinanz dot de
Here is another example of a problem with unpack on 64bit systems. It worked in 5.1.6, but with 5.2.1 the results are bogus.

The expected value from the file content is 200, but PHP says -2147483448 when I echo $a['i'].

<?php
$f = fopen('test.pdf','rb');
//Read a 4-byte integer from file
$a = unpack('Ni',fread($f,4));
echo $a['i'];
fclose($f);
?>
 [2007-03-14 20:57 UTC] pz at mysqlperformanceblog dot com
In any case if you call it bug or a feature this is serious behavior change for something which a lot of people could be depending on. 

It breaks in MySQL 5.2.0 -> 5.2.1  which is minor version upgrade.
 [2007-03-16 14:03 UTC] martin at netimage dot dk
It appears that the sign bit is taken from LSB instead of MSB

> php -r 'print_r( unpack('N',pack('N',127)));'
Array
(
    [1] => 127
)

> php -r 'print_r( unpack('N',pack('N',128)));'
Array
(
    [1] => -2147483520
)

The last number is 2's complement of -128 for 32 bit integers

Cheers
 [2007-04-03 22:00 UTC] iliaa@php.net
Please try using this CVS snapshot:

  http://snaps.php.net/php5.2-latest.tar.gz
 
For Windows:
 
  http://snaps.php.net/win32/php5.2-win32-latest.zip


 [2007-04-05 13:49 UTC] ben at ateor dot com
Fix the problem for me, thanks. 
(ps: meanwhile, I can't verify there's no regressions, given how the testsuite has changed).
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Mon Dec 23 10:01:28 2024 UTC