The best way is to keep dropping them on the huts to repair them. The problem is that, unlike VV1 and VV2, villagers won't do that themselves so you have to keep intercepting them when they wander off and re-drop them. The other way is even more convoluted. You can feed them an exercise potion. If they succeed, they'll gain a little building skill.

As mentioned above, though, the very best way is to avoid the problem by keeping back a building project on which to train the builders. Possibly too late now, though ...