You cannot build more huts once you've completed the third one. There is a way in VV2, besides having multiple births, to increase the population. It's a spoiler, but you can see the answer in the guide section here.