You don't have to start again. Select building for your current builder. He will keep fixing your current hut, and by doing so increase his skill. I would also keep breeding one of your females to slowly bring up you population. You can "finish" the game with the peeps you got, but it will take awhile. Now might be a good to start an additional game!
_________________________
The VI