I kiss the two villagers untill I see the number of villagers go up a number then I know what's done is done. Outside of that I don't care if they go indoors or not. I caught on to that and saved a lot of waisted time.