Coverity seems to hate it mostly in the mutex, so change it to
mostly outside of the mutex and see if that's fine.

It doesn't really matter where it's changed as it's an atomic bool.