You know that tip you get in the game that says that one male parent can father many children? It's there for a reason. There's no way to completely prevent it once a male villager has parenting skill. You can just try to keep him busy working elsewhere.