My male builder was almost ready with the first hut when he all of a sudden starts repairing the love hut. So I tried to bribe him:

"Go finish that hut you were building, and I'll see to that you and that gorgeous blonde will get it on in the love hut in a minute or so."

I don't think it would have worked, but you never know. I was too impatient though, about two seconds later I moved him back myself.