I've had that a few times. It depends on the difficulty I think, and the likes and dislikes of a villager. Some villagers are just stubborn, keep 'arguing' with them till they give in and then smile when they actually keep themselves working.

Also, make sure they are not worried about anything.