The only time that villagers can harvest is when there are crops actually visible on the farm. When you drag villagers to the farm, they will choose which of the farming activities they want to perform. If you interrupt them and try again, they may choose the same activity, or a different one. All of those activities will improve a farmer's skill, and the farm will replenish in time (regardless of the apparent imbalance of those choices).
_________________________
Barbara
Unicorn
Last Day of Work