Section 19. A fence viewer who, when requested, unreasonably neglects to perform any duty required of him in this chapter shall forfeit five dollars to be recovered in tort to the use of the town, or on complaint to the use of the commonwealth, and shall also be liable for all damages to the party injured.