§ 33-a. Removal of heads of departments. Any officer who is, or any or either of the officers who are, the head of a department, if appointed by the governor by and with the advice and consent of the senate, may be removed from office by the governor whenever in his judgment the public interest shall so require. In case of such a removal the governor shall file with the department of state a statement of the cause of such removal and shall report such removal and the cause thereof to the legislature at its next session.