It sounds like your villagers need more food. Starvation will cause your villagers to weaken from lack of food. The only thing that will help them is to build their food supply back up. You can look for mushrooms and have the children pick them to help get food until you have other food sources. In the early stages of the game, it's important not to increase your population until you have a more reliable food supply.