Part of the challenge is to keep them on task. They always want to move the boulder, do the laundry, curious about the ruins, rock, cactus, or strange patch. Some of the villagers are lazy or stubborn...some are "workaholics" Especially if they are a Master Parent! Whatever task you want them on, just keep dragging them back there till they get it right.