The state penitentiary shall be under the immediate executive control and management of a warden, and each of the other penal, reformatory or charitable institutions of the state shall be under the executive control and management of a superintendent, all subject to the rules and regulations of the department vested with control of such institution and this chapter, the superintendents and wardens to be appointed by the commissioner vested with the administration of such institutions, with the approval of the governor.