They got old, they got sick & remained weakened even after healing and worst of all, they only had boy babies in the last batch. The last villager, Mr. Lonely, finally got sick & had nobody to heal him. (I found out that the cactus only teaches healing; it can't help with healing someone.)