go to bug id or search bugs for
In https://github.com/ramsey/uuid/issues/80 it was determined that under specific circumstances openssl_random_pseudo_bytes may generate duplicate random values when using a forked process model (like Apache2 prefork MPM or FastCGI) because OpenSSL seeds its random number generator the first time with a blob of data from /dev/u?random (depends on config) and then only mixes in the current pid ( pre 20 sept 2013 ) and the current time ( post 20 sept 2013 ) for each call to RAND_bytes().
This has happened with other projects, like Android, see http://emboss.github.io/blog/2013/08/21/openssl-prng-is-not-really-fork-safe/ . OpenSSL documents it not being "fork-safe" here: https://wiki.openssl.org/index.php/Random_fork-safety.
One of the mitigations suggested is calling RAND_poll() after fork(). I've tried to implement such an approach in https://github.com/php/php-src/commit/afbe15c7e7d2fe650e08f48d62fc4f7928bab695 . It's probably incomplete (maybe reinit_rng() should be called on other functions too) but I wanted to get feedback first before spending more time on it.
Add a Patch
Add a Pull Request
I think that calling RAND_poll on the first call openssl_random_pseudo_bytes in each request is quite overkill and it might cause some perf regressions. RAND_poll is an expensive call. Wouldn't be enough to use RAND_seed / RAND_add (it's the same in the default rand method) with PID for example.
P.S. I'm not a rand expert but hope that it shouldn't have any impact on randomness. But might be wrong. :) In any case RAND_poll seems just too much...
It's not sensible to use module globals, these are duplicated on fork.
So if the parent process has already called a function that results in reseed, or poll, the child process will not make the call ...
Well, COW duplicated ... the same data whatever ...
Yeah there seems to be missing global init which should reset the value for each request (PHP_GINIT_FUNCTION). Then it should be fine and the entropy would be added for the first call of the openssl_random_pseudo_bytes in the request. It means it can be added multiple time if the same process is used for more requests (fpm) which I'm not sure is ideal but might not be an issue. Again I might be wrong.
I've moved it to https://github.com/php/php-src/pull/1843
@bukka Seeding with PID and time already happens after each call to RAND_bytes() and is what caused this behaviour in the first place. Apache's mod_ssl calls RAND_seed() with a few bytes from /dev/urandom (or egd) for each request - we could do that instead, but it seems like that's pretty much what RAND_poll() does. What are you seeing as the heavy part of RAND_poll?
@krakjoe Right. This would work fine in mod_php, but maybe not FPM. I could force the bool to false in PHP_RINIT_FUNCTION(openssl). Think that'd work better?
Ah I see what's the issue - finally read the links... :). Reading and thinking about it a bit more, the RAND_poll is probably reasonable. It would be good to know if there is any visible slow down though. Not that it would be a blocker but rather to see if we should have also consider some other solutions like addressing that in the SAPI's possibly (which might not be a great idea though). Anyway I will have to think about it a bit more so will leave this for a couple of days.
@bukka Any idea yet on what direction you'd like to go?
To follow up on some things:
People are asking whether just reseeding on fork() isn't the way to go. @krakjoe even implemented [a POC PR](https://github.com/php/php-src/pull/1844) - although it's with PID and time (which doesn't help). I have some doubts about whether we are able to register that pthread_atfork callback soon enough for all SAPIs for it to be usable. Even if we are, that means that all requests are now getting a perf-hit regardless of if you even use openssl. That's something I tried to prevent in my PR.
I'm still thinking (read didn't have time to think about it much yet :) ) what would be the best solution and will try to play with both patches later when I have time. Here are just few thoughts.
- The perf hit that you say is actually limited for fpm quite considerably because you have got a pool of processes so there is not so much forking and one process is used for many requests. In case of prefork, I'm not sure if people that use it actually care about perf :)
- I'm wondering where you see that OpenSSL adds time to the rand pool. I checked code and the only version that does so is current master (1.1 that is not supported by PHP yet). All others add only PID. I might have missed it thought. Could you point me to the code where it is added and could you point me to some article that explains why adding time doesn't help either. I'm just curious...
- The Rand is not used just in openssl_random_pseudo_bytes but also in private key generation (if you add partial parameters) and sealing as well so there are few more places but that could be addressed of course in your patch too.
- There is actually another thing on pthread_atfork that wasn't mentioned. OpenSSL would now require linking of libpthread which should be probably checked with packagers and RM's as it's a new lib dependency for openssl ext (especially if that should go to bug fixing release)
I will try to send an update when I get a bit more time to look into it.
> The perf hit that you say is actually limited for fpm quite considerably because you have got a pool of processes so there is not so much forking and one process is used for many requests. In case of prefork, I'm not sure if people that use it actually care about perf :)
That'd mean that for FPM you'd still get somewhat predictable bytes that way, especially if pid doesn't change much. This really should be done on a per request basis. Which brings us back to the perf hit :).
> I'm wondering where you see that OpenSSL adds time to the rand pool. I checked code and the only version that does so is current master (1.1 that is not supported by PHP yet). All others add only PID. I might have missed it thought. Could you point me to the code where it is added
From OpenSSL 1.0.1:
OpenSSL also adds the bytes in the uninitialised( but alloc'ed ) buffer to RAND_bytes() to its mix: https://github.com/openssl/openssl/blob/OpenSSL_1_0_1-stable/crypto/rand/md_rand.c#L490. However, in the case of Debian (Wheezy) that is commented out. Even if it weren't, memory contents may not be unguessable.
> and could you point me to some article that explains why adding time doesn't help either. I'm just curious...
Time and pid are guessable or simply iterated through, leading to the possibility of duplicates or reproduced values, as shown by my sample scripts. It probably wouldn't be trivial to abuse in the real world, but there's a lot of potential use-cases.
> The Rand is not used just in openssl_random_pseudo_bytes but also in private key generation (if you add partial parameters) and sealing as well so there are few more places but that could be addressed of course in your patch too.
I imagined there would be other places, I'll take a look.
> There is actually another thing on pthread_atfork that wasn't mentioned. OpenSSL would now require linking of libpthread which should be probably checked with packagers and RM's as it's a new lib dependency for openssl ext (especially if that should go to bug fixing release)
I would recon that a bugfixing release would be preferable for a change like this.
> RAND_poll(): https://github.com/openssl/openssl/blob/OpenSSL_1_0_1-stable/crypto/rand/rand_unix.c#L411-L417
Of course I meant just RAND_bytes as RAND_poll is irrelevant here (it's not called after fork).
> RAND_bytes(): https://github.com/openssl/openssl/blob/bfd53c32cd840ed381ba557c4de8f21e3615655c/crypto/rand/md_rand.c#L538-L552
Yeah I saw this but it's just in master (meaning it won't be available before 1.1) so it's not used with PHP for sure.
I just don't get why you say that mixing time and PID is not safe. Of course PID alone is an issue, but actually mixing it with time seems fine and it is also mentioned so in the last paragraph on OpenSSL wiki that you linked: https://wiki.openssl.org/index.php/Random_fork-safety .
So I will ask once again. What issue do you see with adding pid + time to entropy pool?
Sorry bout that RAND_bytes() source link, I thought I linked to the 1.0.1 branch version of md_rand.c but that link seems to go to master indeed.
For me it boils down to this:
* Using non-easily guessable seeds (/dev/urandom realistically being the best we have) seems better than ones that are (pid/time), even if this information doesn't seem easily (ab)usable at this time:
** PIDs are easily guessable
** Time is guessable, a machine's time may be reset, a website may even purposefully tell you the time it created a blob of bytes etc.
* OpenSSL itself suggests using /dev/urandom over RAND_bytes() and only offers pid/time as mitigations if that's not an option.
* RAND_poll() seems like an easy (cross-OS) way to mix in bytes from /dev/urandom or similar.
So, sure, time would help but given that we have an easily accessible way to create better randomness than pid/time, it seems like the right way to go to help the people who use openssl_random_pseudo_bytes() today.
That's about all I can give you on the matter :)
I have been thinking about it and just came up with a bit different PR ( https://github.com/php/php-src/pull/1857 ). It just adds time before each RAND_bytes which is sort of what OpenSSL 1.1 does. It means it adds a time to the entropy for OpenSSL 0.9.x and 1.0.x. Both of them adds already PID so there shouldn't be a need to add it again. I think that this is a more in line of what OpenSSL does. It also doesn't require any extra lib dependency or adding globals.
Could you elaborate on why you don't want to use RAND_poll() and/or why you think my approach isn't sufficient? Seems to me using RAND_poll() would give the best result in terms of providing random values. The value gets reset for each RINIT, which means it should work as expected for SAPI's like FPM.
I think that your approach is sufficient but I also think the same about my approach. I don't really like adding global just for something that won't be needed when linked with OpenSSL 1.1+. As I said, just adding the time will be more compatible with what OpenSSL 1.1 does.
Automatic comment on behalf of bukka
Log: Fix bug #71915 (openssl_random_pseudo_bytes is not fork-safe)