The game doesn't store the position of each villager in the savegame file, so each time the tribe is loaded (whether or not it's paused), the villagers are put onto the map in new locations.

Was your master doctor eating stew, by any chance? Some stews have the side-effect of lowering a villager's skill.