php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #63858 Segfault and Memory Leak in pthreads
Submitted: 2012-12-27 00:35 UTC Modified: 2013-01-04 09:35 UTC
From: bobwei9 at hotmail dot com Assigned: krakjoe (profile)
Status: Closed Package: pthreads (PECL)
PHP Version: master-Git-2012-12-26 (Git) OS: Mac OS X Mountain Lion
Private report: No CVE-ID: None
 [2012-12-27 00:35 UTC] bobwei9 at hotmail dot com
Description:
------------
I want to share an array with Classes between Thread and main-code (not in Thread). As you may notice, when you see the backtrace below, it didn't work.


Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
[Switching to process 9819 thread 0x1303]
0x0000000100963df2 in zend_std_object_get_class (object=0x10207cb18, tsrm_ls=0x102800000) at zend_object_handlers.c:1456
1456		return zobj->ce;
(gdb) bt
#0  0x0000000100963df2 in zend_std_object_get_class (object=0x10207cb18, tsrm_ls=0x102800000) at zend_object_handlers.c:1456
#1  0x0000000100910144 in zend_get_class_entry (zobject=0x10207cb18, tsrm_ls=0x102800000) at zend_API.c:238
#2  0x00000001006b593d in php_var_serialize_intern (buf=0x102902ef0, struc=0x10207cb18, var_hash=0x102a0aa00, tsrm_ls=0x102800000) at var.c:769
#3  0x00000001006b90de in php_var_serialize_intern (buf=0x102902ef0, struc=0x102080a88, var_hash=0x102a0aa00, tsrm_ls=0x102800000) at var.c:880
#4  0x00000001006ba3b4 in php_var_serialize (buf=0x102902ef0, struc=0x102780788, var_hash=0x102780758, tsrm_ls=0x102800000) at var.c:899
#5  0x00000001020dbe4f in pthreads_store_tostring (pzval=0x102080a88, pstring=0x1028fc040, slength=0x1028fc048, tsrm_ls=0x102800000) at /var/root/pthreads/src/store.c:244
#6  0x00000001020db3a4 in pthreads_store_create (unstore=0x102080a88, tsrm_ls=0x102800000) at /var/root/pthreads/src/store.c:333
#7  0x00000001020db6ab in pthreads_store_separate (pzval=0x102080a88, separated=0x102a08fe8, allocate=1 '\001', tsrm_ls=0x102800000) at /var/root/pthreads/src/store.c:202
#8  0x00000001020d922f in pthreads_prepared_entry (thread=0x1026c0910, candidate=0x102080508, tsrm_ls=0x102800000) at /var/root/pthreads/src/prepare.c:213
#9  0x00000001020d9d30 in pthreads_prepare (thread=0x1026c0910, tsrm_ls=0x102800000) at /var/root/pthreads/src/prepare.c:343
#10 0x00000001020de9ad in pthreads_routine (arg=0x1026c0910) at /var/root/pthreads/src/object.c:637
#11 0x00007fff8c4dc742 in _pthread_start ()
#12 0x00007fff8c4c9181 in thread_start ()



Test script:
---------------
<?php
class ThreadTest extends Thread {
public static $test = [0];
public static function staticTest() {
return self::$test;
}
public function run() {
$this->result = self::staticTest();
var_dump($this->result);
}
}
//ThreadTest::$test[1] = new stdClass;
//ThreadTest::$test[1]->lala = ":D";
$thread = new ThreadTest();
if($thread->start())
        if ($thread->join())
                var_dump($thread->result);


Expected result:
----------------
array(1) {
  [0]=>
  int(0)
}

and with the two lines uncommented:

array(1) {
  [0]=>
  int(0)
  [1]=>
  object(stdClass)#1 (0) {
    ["lala"]=>
    string(2) ":D"
  }
}

Actual result:
--------------
array(1) {
  [0]=>
  int(0)
}
[Thu Dec 27 01:21:47 2012]  Script:  '-'
/var/root/php-src/Zend/zend_execute.c(1655) :  Freeing 0x10A16CBF0 (16 bytes), script=-
/var/root/php-src/Zend/zend_alloc.c(2529) : Actual location (location was relayed)
=== Total 1 memory leaks detected ===
array(1) {
  [0]=>
  int(0)
}
[Thu Dec 27 01:21:47 2012]  Script:  '/private/var/root/pttest.php'
---------------------------------------
/var/root/php-src/Zend/zend_opcode.c(379) : Block 0x10a16cbf0 status:
Invalid pointer: ((thread_id=0x0A0D3000) != (expected=0x74AAE180))
---------------------------------------

with the two lines of the reproduce script uncommented: segmentation fault

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2012-12-27 00:45 UTC] bobwei9 at hotmail dot com
hmm, maybe the Zend engine doesn't support this, but couldn't you try at least to overload (or whatever) the access and modification of static variables.

What also crashes is trying to store a reference to an object in a normal class variable. Aren't you able to make a copy of the variables and synchronize on modifciation? I think only with an extended support of sharing variables, there will be a visible gain with pthreads.
 [2012-12-27 12:07 UTC] krakjoe@php.net
Sorry, but your problem does not imply a bug in PHP itself.  For a
list of more appropriate places to ask for help using PHP, please
visit http://www.php.net/support.php as this bug system is not the
appropriate forum for asking support questions.  Due to the volume
of reports we can not explain in detail here why your report is not
a bug.  The support channels will be able to provide an explanation
for you.

Thank you for your interest in PHP.

Access to static variables cannot be reasonably made thread safe. The solution 
is; the array you wish to share should be abstracted as an object that extends 
Stackable, this object can be passed around between threads and will work as 
expected.

Statics are destined for complete removal I think, I wanted them to be readable, 
and they should be, but people expect them to work in impossible ways. It would 
of course be possible, but not reasonably, it will be no good for pthreads to 
make it work, so avoid them completely.

Stackable/Thread/Worker are all objects prepared to work in a multi-threaded 
environment, if you wish to share an array, or object, or data that isn't a 
simple type, then abstract the data in one of the threaded objects and pass a 
reference to that around.

Passing variables by reference is unsupported due to the storage methods that 
pthreads uses to share data among contexts - which is serialization, the well 
tested and reliable way of sharing data among contexts. Any data that is not a 
simple type will be serialized, thereby making references pointless.

<?php
class MySharedData extends Stackable {
  public function __construct($initial) {
    $this->data = $initial;
  }
  /** merging will work [] may not **/
  public function append($data){
    $this->data = array_merge($this->data, array($data));
  }
  public function fetch(){
    return $this->data;
  }
  public function run(){
    /* you must declare a runnable method for threaded objects */
    /* this may not make sense to you right now, but it will be the case that
       these objects are actually runnable and do need to be executed, for 
instance
       in this example you might want to dump that shared data to a database */
  }
}

class MyThread extends Thread {
  public function __construct($data) {
    $this->data = $data;
  }
  public function run(){
    $this->data->append(array($this->getThreadId() => __CLASS__));
  }
}

$data = new MySharedData($_SERVER["argv"]);
$threads = array();

for(;$i<4;$i++) {
        $threads[$i]=new MyThread($data);
        $threads[$i]->start();
}

foreach($threads as $thread)
        $thread->join();

print_r($data->fetch());
?>
 [2012-12-27 12:07 UTC] krakjoe@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: krakjoe
 [2012-12-27 12:26 UTC] bobwei9 at hotmail dot com
Thank you, but then at least disable the static access in threads. Segmentation faults are not beautiful ;)

Why does this need so complicated... but I think I can work with this. Better than nothing. A question: why do you need to serialize and cannot simply copy the zvals, zend_classes etc? (and update the pointers)
 [2012-12-27 12:35 UTC] bobwei9 at hotmail dot com
Could you please provide in documentation a few more examples for certain situations? Otherwise users won't have a real idea how to use these classes and variables.
 [2012-12-27 13:24 UTC] krakjoe@php.net
-Status: Closed +Status: Re-Opened
 [2012-12-27 13:24 UTC] krakjoe@php.net
I will of course give the documentation the once over, it was changing very 
quickly while developing pthreads and so feels a bit incomplete. I will leave 
this bug open until I have found the time to update the documentation with some 
examples. 

There are examples provided with pthreads and available on git that cover in 
great detail many of the features of pthreads and how to do things and how not 
to do things. Additionally on the index of http://pthreads.org is a rundown of 
"how this works", which should give you a good basis for understanding all the 
examples. The text on the index was destined for the introduction in the manual, 
and still is.

Some walls are not worth jumping over, because the builder has put broken glass 
in cement on the top, just out of site.

What we are doing here is jumping over the necessary walls and avoiding the ones 
with the glass. We are not executing in C, we are executing in Zend, which is an 
architecture designed not to share anything, it forms the basis of the thread 
safety we manipulate in order to execute our threads. The only way to exist is 
co-exist, the defined, reliable and real world tested way of sharing data among 
contexts is serialization, and so that is what we do because almost all of PHP 
supports serialization it's the safest, most compatible, and importantly most 
forward compatible way of sharing data. Objects included with pthreads avoid it 
where they can, basic types are stored in their true form. At the object level, 
most things work as expected, I think the only exception being [] on account of 
the zend engine expecting a pointer that we cannot reasonably provide. Static 
variables are not at the object level, the engine does not permit you the same 
control over these variables.

I'll say it again; the only way to exist is co-exist. If PHP has a solution to a 
problem then absolutely we should use it - as it's been tested for millions of 
hours by hundreds of thousands of people. Where PHP doesn't provide, pthreads 
will attempt too, when it can be considered safe, and compatible.

There will always be limitations, but I should hope they are limitations that 
you can reasonably work around, safely.
 [2012-12-27 13:34 UTC] krakjoe@php.net
On serialization: you asked why we cannot copy the zend_class's and zvals. 
Infact, this is what we do actually do.

Serialization achieves a safe copy of the data, on normal objects this can incur 
overhead, but it's acceptable overhead because it provides safety, and 
compatibility. 
Threaded objects do not require serialization, they manipulate the serialization 
interface to pass around the address of the original object because it's 
handlers can be executed from anywhere by any context, these objects are 
prepared and should always be used over any type that you wish to have as a 
member. 
To force this, to redeclare every boolean/string/array/long as a threaded object 
for the user would incur unacceptable overhead and additionally all but an array 
and object are stored in their true form and wouldn't incur any overhead from 
serialization anyway.
Once all of this is documented you will make the right choices, pthreads does 
actually provide almost all you need, but it may not be where or how you expect 
it, for very good reason.
 [2012-12-27 13:49 UTC] bobwei9 at hotmail dot com
I can understand which overhead this would be. But maybe you could introduce a new keyword like "synchronized" before (static) variables that are then accessible via the global keyword or a normal static call in a thread (=> redeclaration of the few variables)?

The examples at github are helpful, but I think they remain unnoticed for most users...
 [2012-12-27 14:38 UTC] krakjoe@php.net
It would be to no end, I cannot actually synchronize access even if I could 
introduce the keyword ( which I cannot, from an extension ). When PHP code reads 
a static variable no handlers are invoked, handlers are for the object level and 
static variables are not at the object level.

The only way to achieve useable static members would be to re-declare for the 
user anything declared as static as a threaded object and this is not a 
reasonable solution to the problem because it would incur, for every entry in 
every thread, weather they use statics or not, unacceptable overhead and 
instructions that can be avoided.
 [2012-12-27 14:55 UTC] bobwei9 at hotmail dot com
Your test with the constants does also not work (tests/global-constants.diff):

001+ Notice: Use of undefined constant SCONST - assumed 'SCONST' in /private/var/root/pthreads/tests/global-constants.php on line 11
002+ 
003+ Notice: Use of undefined constant LCONST - assumed 'LCONST' in /private/var/root/pthreads/tests/global-constants.php on line 12
004+ 
005+ Notice: Use of undefined constant DCONST - assumed 'DCONST' in /private/var/root/pthreads/tests/global-constants.php on line 13
001- string:string(8) "mystring"
002- long:int(10)
003- double:float(1.19)
004- null:NULL
005- boolean:bool(true)
006+ 
007+ Notice: Use of undefined constant NCONST - assumed 'NCONST' in /private/var/root/pthreads/tests/global-constants.php on line 14
008+ 
009+ Notice: Use of undefined constant BCONST - assumed 'BCONST' in /private/var/root/pthreads/tests/global-constants.php on line 16
010+ string:string(6) "SCONST"
011+ long:string(6) "LCONST"
012+ double:string(6) "DCONST"
013+ null:string(6) "NCONST"
014+ boolean:string(6) "BCONST"
 [2012-12-27 15:16 UTC] bobwei9 at hotmail dot com
I know, you cannot serialize ressources. But is there no way to share a socket-ressource (over the Stackable object)? I want to listen in one thread in an continuing loop and pushing sometimes from the mainthread some messages over the socket. Also I would prefer not to have to duplicate a mysql-ressource (which is less problematic). But only one socket can listen and send on the same port, so I cannot open in mainthread and in the listening thread a socket. Yes, surely, I could use shmop (shared memory), but this is slow, I feel. Is there an alternative?
 [2012-12-27 16:03 UTC] bobwei9 at hotmail dot com
Other problem is when instantiating extended classes. I cannot call the __construct-function of the parent class, because the static access with parent::__construct fails.
 [2012-12-27 16:21 UTC] krakjoe@php.net
the constants bug I have fixed in git.

Both parent:: and self:: appear to suffer from the same bugs, noted...

It would be better if you opened new bugs rather than adding to this one.

Resources are explained in the introduction text on pthreads.org.
 [2012-12-27 16:42 UTC] bobwei9 at hotmail dot com
Ok, next time I will split up into different bug reports.

I read that your page, but doesn't exist here at least some indirect way? Like having a special mark to the variable which "holds" the ressource and operations on the ressource are executed in the mainthread and the result returned in the executing thread?

And thanks for your fixes :)
 [2012-12-27 16:51 UTC] krakjoe@php.net
A worker can have resources, stackables can manipulate those resources, any 
context with a reference to the stackable can read the results of it's work.

But there is no way, and there's not likely to be a way, to share resources among 
threads.
 [2012-12-27 19:11 UTC] krakjoe@php.net
parent:: and self:: bug resolved, tested inherited constructors they are now 
working in git.

I will still leave this open until I have updated the documentation, the thing to 
remember is that static variables now have the scope of a thread, not the scope 
of a process like they do without pthreads. So you can still read/write them 
safely and without segfaulting or errors, better :)
 [2012-12-30 14:51 UTC] krakjoe@php.net
-Status: Re-Opened +Status: Feedback
 [2012-12-30 14:51 UTC] krakjoe@php.net
Despite my instincts, I have made resource sharing a possibility, examples for a 
multi-threaded socket server are now in git, and working.

I will still leave this open until the documentation is updated, I am unsure of 
how much will work, and we are still testing, so not much point in updating the 
documentation just yet until I know what to document exactly, hopefully anyone 
having the same issues will come across this bug report for now.
 [2013-01-04 09:35 UTC] krakjoe@php.net
I have now updated the documentation with inline examples, it should all be 
available on php.net main docs today, I will also push another release of what is 
in git today to pecl so everything is up to date.
 [2013-01-04 09:35 UTC] krakjoe@php.net
-Status: Feedback +Status: Closed
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Tue Mar 19 09:01:30 2024 UTC