If you don't have any building jobs for them, you can put them to work on farming or research. Then when you have something for them to build, you can them put them back to building. There is more for builders to do then build huts.