Setup:
RedHat 8.0
Apache 1.3.27
Courier-imap 1.7.3
C-Client - imap-2002d
PHP - 4.3.2 

Both php and apache are compiled from source. No RPMS for those.

IMAP fails to connect no matter which webmail package I use. In some cases it eats up all memory.

e.g. with phpgroupware:

Allowed memory size of 4194304 bytes exhausted at (null):0 (tried to allocate 184320 bytes)

In other cases it fails to connect at all (with imp)

server="godspeed.lusis.org" user="username" mailbox={godspeed.lusis.org:993/imap/notls/novalidate-cert}INBOX"

array(1) {
  [0]=>
  string(53) "Connection failed to godspeed,993: Connection refused"
}

Which is just silly since I've tested right after that with evolution and email comes up fine.

If I use the IP Address instead of the hostname, the connection just hangs there.

Having tried it with two different packages I'm guessing that it's a php problem as I also saw the Connection failed message with both horde/imp AND phpgroupware.

You can get a copy of phpinfo() at:
http://www.lusis.org/info.php

Configure Line:
/configure \
--prefix=/usr/local \
--with-apxs=/usr/bin/apxs \
--enable-exif \
--enable-track-vars \
--with-calendar=shared \
--enable-magic-quotes \
--enable-trans-sid \
--enable-wddx \
--enable-ftp \
--enable-inline-optimization \
--enable-memory-limit \
--with-openssl \
--with-curl \
--enable-sysvsem \
--enable-sysvshm \
--with-gettext="/usr" \
--with-mysql=/usr \
--with-imap=/usr/local/imap2002 \
--with-imap-ssl \
--with-kerberos \
--with-mcrypt \
--with-gd \
--with-zlib-dir=/usr \
--with-jpeg-dir=/usr \
--with-ldap

Output of ldd:
[root@godspeed logs]# ldd /usr/libexec/libphp4.so
        libcrypt.so.1 => /lib/libcrypt.so.1 (0x40275000)
        libssl.so.2 => /lib/libssl.so.2 (0x402a3000)
        libcrypto.so.2 => /lib/libcrypto.so.2 (0x402d3000)
        libmysqlclient.so.10 => /usr/lib/mysql/libmysqlclient.so.10 (0x403a7000)
        libmcrypt.so.4 => /usr/lib/libmcrypt.so.4 (0x403de000)
        libltdl.so.3 => /usr/lib/libltdl.so.3 (0x40410000)
        libldap.so.2 => /usr/lib/libldap.so.2 (0x40417000)
        liblber.so.2 => /usr/lib/liblber.so.2 (0x40442000)
        libpam.so.0 => /lib/libpam.so.0 (0x4044d000)
        libpng12.so.0 => /usr/lib/libpng12.so.0 (0x40455000)
        libz.so.1 => /usr/lib/libz.so.1 (0x40478000)
        libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0x40486000)
        libcurl.so.2 => /usr/lib/libcurl.so.2 (0x404a4000)
        libresolv.so.2 => /lib/libresolv.so.2 (0x404c6000)
        libm.so.6 => /lib/i686/libm.so.6 (0x404d8000)
        libdl.so.2 => /lib/libdl.so.2 (0x404fa000)
        libnsl.so.1 => /lib/libnsl.so.1 (0x404fd000)
        libgssapi_krb5.so.2 => /usr/kerberos/lib/libgssapi_krb5.so.2 (0x40513000)
        libkrb5.so.3 => /usr/kerberos/lib/libkrb5.so.3 (0x40526000)
        libk5crypto.so.3 => /usr/kerberos/lib/libk5crypto.so.3 (0x40585000)
        libcom_err.so.3 => /usr/kerberos/lib/libcom_err.so.3 (0x40595000)
        libc.so.6 => /lib/i686/libc.so.6 (0x42000000)
        libsasl.so.7 => /usr/lib/libsasl.so.7 (0x40597000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
        libgdbm.so.2 => /usr/lib/libgdbm.so.2 (0x405a2000)