I have an Esteemed Elder who is Master in 3 areas but when I put that villager by the robe, he tries it on but then it says it doesn't fit!!!

I've tried repeatedly and it won't work....he is fully healthy. I have even tried making him eat a bunch and trying again, but nope, still doesn't fit.

What am I doing wrong?