I've noticed that when my villagers want to do an action that involves another villager, they always seem to go to the same one, a fellow named Bobo in my particular game. If a woman is set to breed, she'll always go to Bobo. If someone wants to complain or chat, it's Bobo they head for. Bobo was an original villager and also happens to be the first one on the "details list" when I cycle around past the children. So I suspect that he's the first one the code looks at when it's looking for a likely candidate for interaction, and it never seems to get past him. I've seen a couple of other people mention that they have a guy that all of the women insist on mating with.
It's not a major issue, but it does appear to be a bug.
Keith