Building skill increases too easily after mastery

Once a villager has completely filled their building skill bar, they tend to build nearly as often as they do whatever other preference you set that they are also adept or master in (farming, healing, research). As a result, most of my villagers have become more proficient in building than any other skill. Is there any specific reason for this?