The governor shall have the power, and it is his duty, to suspend alleged defaulting tax collectors pending the investigation of their respective accounts, whether made under the foregoing sections or otherwise, and to make temporary appointments of proper persons to fill the offices while such investigations are being made.