The judge or chancellor may in his discretion at any time remove the court reporter from office for incompetency or neglect of duty, and may appoint a court reporter to fill the vacancy as often as such removals occur. The appointment of a successor to the court reporter shall be made, and he shall qualify in the same manner, and he shall comply with all the other requirements, as is provided in the foregoing sections of this chapter.