15-20? When I normally start on the project I doubt that I have that many working villagers. It's only when I've got L3 farming do I feel safe expanding the population past about 15, and of course it takes several years for the kids to grow up. I realise that I still have to get two other L3 techs before I can start on clearing the floor, but the effects of any 'baby boom' won't start to be felt on the working population for a while longer. In addition, several of the female villagers will be out of action at least part of the time while they are nursing the new additions.
I usually have to split my workers between scientists and builders as gaining tech points is still important. Obviously, I'll also need farmers and a doctor or two. In all, 6-8 is usually the maximum I can spare.
_________________________
To err is human; to arr is pirate.