Fix major performance regression in RIP rendering

A high number of RIP commands use setpixel(), and it was updated
to scan the whole vmem for changes on each call.  Now we just extract
the part that is relevant and only update the one cell if needed.

This also adds parameter validation to bitmap_setpixel() which can
prevent some memory corruption and crashes.