So it took me a little while to figure it out so I was happy but this is my solution:

There is a bug in using multiple connections in system/database/DB_Driver.php about line 400 should have $this->db_select(); just before the return.

Then you can do things like this in a model contructor:

$this->legacy_db = $this->load->database('legacy', true);

Which sets Model::legacy_db to the connection reference (the second argument as boolean true causes the function to return the reference – and of course you don’t have to call it legacy_db). You can then use it with:

$q = $this->legacy_db->query("SELECT * FROM table");