I think the only other way to make your builder's skill increase is giving him a stew that makes him exersize. I had a villager in VV3 who's action line read "Exersising" then he stopped,and when I looked he had some building skills.

It's RRO (two reds and an orange) that will boost building skills. Not everyone, but many will go exercise by lifting boulders up by the blue rose, and they end up with building skill. This will work on children, too, which can be helpful when you've got some children with no initial skills.

The other exercising, RRR, doesn't add to skills that I can see. Nor the 'burst of energy - dancing' one, RRG (green (white?) plant by the dead grass).

Thanks for the tip!