Whoever, having been appointed an inspector or judge of election in any election district, refuses to serve shall be fined $50.