The same thing happens when you dump a villager on top of a hut. Maybe the villager can't occupy the same space as a hard stucture.