php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #55477 crypt() returns inconsistent hashes for non-ASCII characters
Submitted: 2011-08-22 12:47 UTC Modified: 2011-08-22 13:29 UTC
From: christian at pingdom dot com Assigned:
Status: Not a bug Package: *Encryption and hash functions
PHP Version: 5.3.7 OS: Linux
Private report: No CVE-ID: None
 [2011-08-22 12:47 UTC] christian at pingdom dot com
Description:
------------
Hashes generated with crypt() (using Blowfish) on PHP 5.3.5 or 5.3.3 cannot be validated on 5.3.7, if the hashed strings contain non-ASCII characters. The reverse is also true, if the hashes were generated on 5.3.7, they cannot be validated on 5.3.3 or 5.3.5.

Test script:
---------------
$passwords = array(
    // these hashes were generated on PHP 5.3.5-1ubuntu7.2 with Suhosin-Patch (cli) (built: May  2 2011 23:00:17)
    'brownfox' => '$2a$07$usesomesillystringforeD/hyr5e1bWX2PzwckMuCRNQMTrQNr72',
    'Boxkämpfer' => '$2a$07$usesomesillystringfore36pVDWFz65CbxoLgSgVURqHWU4yEqye',
    'щастлива' => '$2a$07$usesomesillystringforeoM7K1pyDjeAG1F42k34MP.tbiMnNcy.',
    'Põdur' => '$2a$07$usesomesillystringfore1iPxMN9wh4Cr2oVR6nmdILWylX9D0iO',
);

foreach ($passwords as $password => $hash)
{
    $computedHash = crypt($password, $hash);
    if ($computedHash == $hash)
    {
        echo "hash OK\n";
    }
    else
    {
        echo "hash FAIL ($hash != $computedHash)\n";
    }
}


Expected result:
----------------
hash OK
hash OK
hash OK
hash OK


Actual result:
--------------
hash OK
hash FAIL ($2a$07$usesomesillystringfore36pVDWFz65CbxoLgSgVURqHWU4yEqye != $2a$07$usesomesillystringforeelZZJE6VQ2/DIcx1J.D.htZuAQIV43S)
hash FAIL ($2a$07$usesomesillystringforeoM7K1pyDjeAG1F42k34MP.tbiMnNcy. != $2a$07$usesomesillystringforevg24bYcXKv2WUiCZvAH627ba6aubiNC)
hash FAIL ($2a$07$usesomesillystringfore1iPxMN9wh4Cr2oVR6nmdILWylX9D0iO != $2a$07$usesomesillystringforeuqJNc6ZnvGzLGss/.ZcwQdygkbYamRq)


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2011-08-22 13:04 UTC] pajoye@php.net
-Type: Security +Type: Bug
 [2011-08-22 13:29 UTC] bjori@php.net
-Status: Open +Status: Bogus
 [2011-08-22 13:29 UTC] bjori@php.net
This is expected, see http://www.openwall.com/lists/announce/2011/06/21/1

You need to use $2x$ for non-ascii, sorry :(
 [2011-08-22 15:21 UTC] solar at openwall dot com
This is definitely the expected behavior, but saying that $2x$ should be used for non-ASCII is not entirely correct (and is a dangerous thing to say).  If it were this simple, we'd do it in PHP itself, but we had good reasons not to.  More specifically:

The change as implemented in PHP 5.3.7+ favors security and correctness over backwards compatibility, but it also lets users (admins of PHP app installs) use the new $2x$ prefix on existing hashes to preserve backwards compatibility for those and incur the associated security risk until all such passwords are changed (using $2a$ or $2y$ for newly changed passwords).

In versions of PHP older than 5.3.7, $2a$ inadvertently resulted in system-specific behavior for passwords with non-ASCII characters in them.  On some installs (mostly on PowerPC and ARM, as well as sometimes on *BSD's and Solaris regardless of CPU architecture), they were processed correctly.  On most installs (most Linux, many others), they were processed incorrectly most of the time (but not always), and moreover in a way where security was weakened.

In PHP 5.3.7, $2a$ results in almost the correct behavior, but with an additional countermeasure against security-weakened old hashes mentioned above.  $2x$ results in the buggy behavior, so if old hashes are known to be of the buggy type, this may be used on them to keep them working, accepting the associated security risk.

$2y$ results in perfectly correct behavior (without the countermeasure), so it may be used (if desired) when hashing newly set passwords.  For practical purposes, it does not really matter if you use $2a$ or $2y$ for newly set passwords, as the countermeasure is only triggered on some obscure passwords (not even valid UTF-8) that are unlikely to be seen outside of a deliberate attack (trying to match hashes produced by buggy pre-5.3.7 code).

BTW, PHP 5.3.7+ has been updated to crypt_blowfish 1.2, not the intermediate 1.1 release referenced in the previous comment.  The differences between 1.1 and 1.2 include introduction of the countermeasure for $2a$ mentioned above and the $2y$ prefix.

Summary: for passwords without characters with the 8th bit set, there's no issue, all three prefixes work exactly the same.  For occasional passwords with characters with the 8th bit set, if the app prefers security and correctness over backwards compatibility, no action is needed - just upgrade to new PHP and use its new behavior (with $2a$).  However, if an app install admin truly prefers backwards compatibility over security, and the problem is seen on the specific install (which is not always the case because not all platforms/builds were affected), then $2a$ in existing hashes in the database may be changed to $2x$.  Alternatively, a similar thing may be achieved by changing $2a$ to $2x$ in PHP app code after database queries, and using $2y$ on newly set passwords (such that the app's automatic change to $2x$ on queries is not triggered for them).
 [2011-08-22 15:25 UTC] solar at openwall dot com
Oops, I did not realize this interface would use <pre>.  Somehow the preview does not.  I think it's my first time posting a comment to a PHP bug.  Let me try again, with line wrapping at 79 for hopefully easier reading:

This is definitely the expected behavior, but saying that $2x$ should be used
for non-ASCII is not entirely correct (and is a dangerous thing to say).  If it
were this simple, we'd do it in PHP itself, but we had good reasons not to.
More specifically:

The change as implemented in PHP 5.3.7+ favors security and correctness over
backwards compatibility, but it also lets users (admins of PHP app installs)
use the new $2x$ prefix on existing hashes to preserve backwards compatibility
for those and incur the associated security risk until all such passwords are
changed (using $2a$ or $2y$ for newly changed passwords).

In versions of PHP older than 5.3.7, $2a$ inadvertently resulted in
system-specific behavior for passwords with non-ASCII characters in them.  On
some installs (mostly on PowerPC and ARM, as well as sometimes on *BSD's and
Solaris regardless of CPU architecture), they were processed correctly.  On
most installs (most Linux, many others), they were processed incorrectly most
of the time (but not always), and moreover in a way where security was
weakened.

In PHP 5.3.7, $2a$ results in almost the correct behavior, but with an
additional countermeasure against security-weakened old hashes mentioned above.
$2x$ results in the buggy behavior, so if old hashes are known to be of the
buggy type, this may be used on them to keep them working, accepting the
associated security risk.

$2y$ results in perfectly correct behavior (without the countermeasure), so it
may be used (if desired) when hashing newly set passwords.  For practical
purposes, it does not really matter if you use $2a$ or $2y$ for newly set
passwords, as the countermeasure is only triggered on some obscure passwords
(not even valid UTF-8) that are unlikely to be seen outside of a deliberate
attack (trying to match hashes produced by buggy pre-5.3.7 code).

BTW, PHP 5.3.7+ has been updated to crypt_blowfish 1.2, not the intermediate
1.1 release referenced in the previous comment.  The differences between 1.1
and 1.2 include introduction of the countermeasure for $2a$ mentioned above and
the $2y$ prefix.

Summary: for passwords without characters with the 8th bit set, there's no
issue, all three prefixes work exactly the same.  For occasional passwords with
characters with the 8th bit set, if the app prefers security and correctness
over backwards compatibility, no action is needed - just upgrade to new PHP and
use its new behavior (with $2a$).  However, if an app install admin truly
prefers backwards compatibility over security, and the problem is seen on the
specific install (which is not always the case because not all platforms/builds
were affected), then $2a$ in existing hashes in the database may be changed to
$2x$.  Alternatively, a similar thing may be achieved by changing $2a$ to $2x$
in PHP app code after database queries, and using $2y$ on newly set passwords
(such that the app's automatic change to $2x$ on queries is not triggered for
them).
 
PHP Copyright © 2001-2018 The PHP Group
All rights reserved.
Last updated: Fri Sep 21 19:01:25 2018 UTC