IIRC, the villagers who are "worried about housing" when at the maximum population are builders. You may want to switch their jobs to farmers.

Or if I'm wrong, and they are farmers, switch them to builders so that they will "fix the huts" that exist.

...or just switch them to healers or don't worry about them worrying.