You have to be careful about how you train your villagers. If you "train" a male villager in parenting, that's what he will want to do, whether or not you have that skill checked. It's always safer to let the females gain all the parenting skill if you wnat to keep the population manageable.