I have a bunch of women caring for a baby at the same time. While I was away from the game 2 of them wandered into the temple. I can't actually put them in the temple but they can take themselves there.