LOL. They can get a bit lazy, but you just have to keep cracking that whip!
You can tell whom has been "fooling" around by viewing how has breeding skills. Only the villager who initiates the "breeding" will gain skill. So who ever have experience is the one who has been fooling around.
_________________________
The VI