Salem, the villagers improve always the task they are doing.
It's this manner, when I quit the game, I select at least 2 doctors to be sure they'll heal the sick villagers when I'm not here
. Then, I put them on the research table or on a farming task and they increase their scientist or farmer's skills! Less than if they were select as scientist or farmer, but they increase and bring food or tech points. And they also increase their healing skills because they go studying medicine time to time or they heal someone.
cammycam, to increase the buiding task, be sure that they are
"fixing hut" and not "examining". You may also make a potion for exercising, that gives some little building skills if they succeed. If they say "no" it doesn't work!
Good luck!