Usually they'll succeed or fail in fixing a hut then go off to do other things. They need to be dragged back to work all the time. If they are actually building them, they'll carry on doing so until it's finished. So yes, it does take longer to gain skills. However, when they are building, they have to walk long distances fetching wood and rocks, so it takes quite a while for them to complete each individual task.