a little more info:  afaict the file is being created with zero size via some stream functions in dba.c.  when db->open() is called with DB_CREATE|DB_TRUNCATE libdb4.8 tries to read some metadata from the existing file and fails horribly.  i tried playing around with the flags (passing only DB_TRUNCATE does not work, passing only DB_CREATE seems to work but still produces the error message).

-Status: Open
+Status: Feedback

Please try using this snapshot:

  http://snaps.php.net/php5.3-latest.tar.gz
 
For Windows:

  http://windows.php.net/snapshots/

-Status: Feedback
+Status: Open

and i've now asked in the bdb forums what they think:

http://forums.oracle.com/forums/thread.jspa?threadID=1031805&stqc=true

oops, looks like there was a little race there.  yes i can see the behavior in the snapshot too:

rangda[/home/sean/Download/php5.3-201002160730] ./run-tests.php -p ./sapi/cli/php --show-diff ext/dba/tests/bug36436.phpt

=====================================================================
PHP         : ./sapi/cli/php 
PHP_SAPI    : cli
PHP_VERSION : 5.3.3-dev
ZEND_VERSION: 2.3.0
PHP_OS      : Linux - Linux rangda 2.6.32-trunk-amd64 #1 SMP Sun Jan 10 22:40:40 UTC 2010 x86_64
INI actual  : /home/sean/Download/php5.3-201002160730
More .INIs  :  
CWD         : /home/sean/Download/php5.3-201002160730
Extra dirs  : 
VALGRIND    : Not used
=====================================================================
Running selected tests.
TEST 1/1 [ext/dba/tests/bug36436.phpt]
========DIFF========
001+ Notice: dba_popen(): fop_read_meta: /home/sean/Download/php5.3-201002160730/ext/dba/tests/test0.dbm: unexpected file type or format in /home/sean/Download/php5.3-201002160730/ext/dba/tests/bug36436.php on line 6
========DONE========
FAIL Bug #36436 (DBA problem with Berkeley DB4) [ext/dba/tests/bug36436.phpt] 
=====================================================================
Number of tests :    1                 1
Tests skipped   :    0 (  0.0%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    1 (100.0%) (100.0%)
Expected fail   :    0 (  0.0%) (  0.0%)
Tests passed    :    0 (  0.0%) (  0.0%)
---------------------------------------------------------------------
Time taken      :    0 seconds
=====================================================================

=====================================================================
FAILED TEST SUMMARY
---------------------------------------------------------------------
Bug #36436 (DBA problem with Berkeley DB4) [ext/dba/tests/bug36436.phpt]
=====================================================================

-Summary: will not build/work with libdb4.8
+Summary: will not work with libdb4.8
-Operating System: Debian (and others)
+Operating System: *
-PHP Version: 5.3.1
+PHP Version: 5.3, 6 (2010-02-19)

At least it builds now. :)

-Summary: will not work with libdb4.8
+Summary: will not build/work with libdb4.8
-Operating System: *
+Operating System: Debian (and others)
-PHP Version: 5.3, 6 (2010-02-19)
+PHP Version: 5.3.1

sorry for all the noise, when trying the latest snapshot it apparently unpacked on top of a previous snapshot in which i had been investigating the issue.  the tests fail in the new snapshot with more than just the NOTICE error (i had been playing with the open flags as mentioned previously).  the actual failure  diff looks like:

TEST 1/25 [ext/dba/tests/bug36436.phpt]
========DIFF========
001+ Notice: dba_popen(): fop_read_meta: /home/sean/Download/php5.3-201002190730/ext/dba/tests/test0.dbm: unexpected file type or format in /home/sean/Download/php5.3-201002190730/ext/dba/tests/bug36436.php on line 6
002+ 
003+ Notice: dba_popen(): /home/sean/Download/php5.3-201002190730/ext/dba/tests/test0.dbm: Invalid type 5 specified in /home/sean/Download/php5.3-201002190730/ext/dba/tests/bug36436.php on line 6
004+ 
005+ Warning: dba_popen(/home/sean/Download/php5.3-201002190730/ext/dba/tests/test0.dbm,c): Driver initialization failed for handler: db4: Invalid argument in /home/sean/Download/php5.3-201002190730/ext/dba/tests/bug36436.php on line 6
001- resource(%d) of type (dba persistent)
002- string(3) "XYZ"
003- string(1) "X"
004- string(1) "Y"
005- ===DONE===

-Summary: will not build/work with libdb4.8
+Summary: will not work with libdb4.8
-Operating System: Debian (and others)
+Operating System: *
-PHP Version: 5.3.1
+PHP Version: 5.3, 6 (2010-02-19

heh, seems we're stepping on each other's toes now.  i'll set the stuff back that i just clobbered, and promise to be quiet for a few hours :)

actually it won't build correctly against db4.8.  i had to modify the snapshot to link against db4.8, as otherwise you see http://bugs.php.net/bug.php?id=51062 , though apparently that's a bogus issue, hrm... :)

The Berkeley DB developers are reviewing this.

The next patchset of Berkeley DB 4.8 will possibly have the root cause
fixed and the undefined behavior that DBA was depending on reverted.
In the meantime I've merged a fix and a workaround to PHP 5.2.14-dev,
PHP 5.2.3-dev and PHP 6.0.

Note: now when using Berkely DB 4.8 prior or equal to 4.8.26, the
workaround causes a message regarding meta data to be suppressed when
opening the database.  This causes a diff in a few cases where that
message was previously displayed in DB 4.7, but prevents the message
incorrectly displaying in all other tests.

Just a thought: what about leaving this open until oracle releases a new libdb, and then committing a second patch that refuses to accept db4.8 < the fixed version via config.m4?  either way, thanks for looking at this.

That seems restrictive, given that normal functionality should be fine with BDB 
<= 4.8.26.