I have noticed that too! It's not a glitch, I think! There is some male villagers who are more attractive than others for the females and when they have catched one, they don't let him go! When I check the details of the kids, I often notice that this guy is the father of many of them (with different mothers!)!