Crosstitcher,

You can have up to 50 Elders so that's a lot of totems to collect! After that, any villagers gaining mastery in three professions don't get a totem or get given the Elder title. I know that several people have reached that limit.

It's also fun to bang the gong once a day.