The palette is used to construct the blank screen buffer using
vmem cells, so the background would be in screwy colours when
connecting for modes that don't use the CGA 16-colour palette.