Upon petition or complaint being filed, praying for a writ of ouster against any of the officers herein named, and whether such action is brought by the attorney general and reporter, district attorney general, county attorney, city attorney, or by relator citizens and freeholders, the court, judge, or chancellor may, on application of the attorney general and reporter, the district attorney general for the state, the county attorney, city attorney, or relator citizens and freeholders bringing such action, suspend such officer or officers so accused from performing any of the duties of their office, pending a final hearing and determination of the matter; and, thereupon, the vacancy shall be filled as the law provides for the filling of vacancies in such office, and such person or persons so filling such vacancy shall carry on the duties of the office until such hearing shall be finally determined or until the successor of the officer so suspended shall be elected or appointed as provided by law, and shall have qualified.