BTW: when I make the same connection as in the first example, but use the mysql CLI client instead of php, I get
mysql> SHOW VARIABLES LIKE 'character_set%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+

So PHP's mysql_connect() behaves differently from mysql cli!

I connected to the same remote database from a Windows/WAMP system and got exactly the same results.

Are you using mysqlnd or libmysql. Libmysql uses its default charset, or set in my.cnf or by explicit call to mysqli_options before calling mysqli_real_connect, but after mysqli_init().
If you use mysqlnd, in this case mysqlnd always assumes the server default charset. This charset is sent during connection hand-shake/authentication and mysqlnd assumes it.

A ha! That's exactly the documentation I was looking for online. I've changed the type to "documentation problem" and think the proper way to resolve this bug is to add this documentation.

I was using mysqlnd by the way (standard setting on ubuntu I guess).

You can overwrite the charset, when using mysqlnd, by calling mysqli_options on a connection created with mysqli_init() but still not connected with mysqli_real_connect(). In this case mysqlnd won't assume the server default charset.

the doc team should take over this one

This bug has been fixed in the documentation's XML sources. Since the
online and downloadable versions of the documentation need some time
to get updated, we would like to ask you to be a bit patient.

Thank you for the report, and for helping us make our documentation better.