When such lumber is not removed by the owner on or before November 1, annually, the owner of the lands or dams may take and convert it to his or her own use. The owner of the lumber, by paying the damages and costs, may remove the same from the lands or dams between November 1 and May 1 following.