A surety on an undertaking, bond, or stipulation, executed or given in a judicial proceeding or filed in a court in the state or given on behalf of or by a public officer, employee, or agent in connection with the duties or performance of the duties of office shall be released and discharged from liability on the undertaking, bond, or stipulation and new surety or sureties substituted when the principal, surety, or obligee files in the court or presents to a person authorized by law to approve or accept on behalf of the obligee the undertaking, stipulation, or bond, a written application for discharge from the undertaking, stipulation, or bond, together with a new undertaking, stipulation, or bond, executed by a competent surety, qualified to act as a surety under the laws of the state, and conditioned as in the former undertaking, stipulation, or bond and providing that the surety accepts all liability theretofore or thereafter accruing under the former as well as the new undertaking, stipulation, or bond and stand in the place of and substitute for the surety on the former undertaking, stipulation, or bond.