If an OCI8 connection is opened with a null connection string, the oracle client spawns a child process and attemps to connect to a default database (localhost:1521) using the user credential of the apache process, and using the oracle BEQ protocol.

In our case, there was no database running on the localhost, and the connection failed.  The default behavior of oracle at this point is to make the child process defunct. It assumes that the parent will notice that the process is defunct and clean it it up. However PHP/apache continue waiting for the process and never clean it up.  The PHP timout is never reached either, because the time spent in child processes is not included in calculating the amount of time PHP has been running.

The defunct processes continue to consume as much cpu time as is availible, resulting in continuous 100% CPU utilization.  Other Apache/php processes continue to run and serve requests, but with slower than normal processing times. (Because they are sharing CPU with the defunct process) Over time several of these processes can cause a huge performance impact on the server.

We have changed the default oracle behavior (through the sqlnet.ora file) so that oracle releases these defunct processes to the OS, and the OS cleans them up. The setting is BEQUEATH_DETACH = yes

We have also put better error checking in place to try and prevent sending a null connection string. However, I belive that this same error could occur with other child process that become defunct, like when executing a shell command.

Server Config:

'./configure' '--prefix=/www' '--with-config-file-path=/www/etc' '--with-apache=/www/src/apache_1.3.24' '--enable-static' '--enable-track-vars' '--enable-magic-quotes' '--enable-trans-sid' '--enable-safe-mode' '--enable-sigchild' '--enable-session' '--enable-bcmath' '--disable-debug' '--without-mysql' '--without-pear' '--with-oci8=/u01/app/oracle/product/8.1.7' '--with-gd=/opt/gd' '--with-png=/opt/libpng' '--with-xpm-dir=/opt/xpm' '--with-jpeg-dir=/opt/jpeg-6' '--with-ttf=/opt/freetype' '--with-zlib=/opt/zlib' '--with-pdflib=/opt/pdflib' '--with-openssl=/opt/openssl' '--with-ldap=/opt/openldap' '--with-tiff-dir=/opt/tiff' '--with-curl=/opt/curl'