Have they died? Or have you had an event recently before that happend. If you had an event, it might of been on that takes away villagers...