Whoever directly or indirectly uses or threatens to use force, coercion, violence or restraint, or inflicts or threatens to inflict damage, harm, or loss, upon or against any person to induce or compel such person to vote or refrain from voting for any candidate in any election, shall be fined not more than $500 or imprisoned not more than one year, or both.