I'm in the same boat with 54 children. Last night I had 45 villagers and that was manageable. I have only level 2 fertility, and I have NO one selected as breeders. I had level three in everything else, but was selectively breeding my favorites.
Overnight, the game threw me 54 children. 54!! Mostly from females who had NO breeding level prior, and they still don't! I thought I had some decision making with the game...lol....