This should be made palette-aware as well, but since the current
consumer doesn't modify the palette, it's good enough for now.