It does get harder finding collectibles, the bigger the population is in the village. You can also get clumps of people by the waterfalls hiding turtle shells.
I've got 2 tribes going at the moment - one science and one magic tribe, and I completed the turtle shell collection in one tribe yesterday

which is my first ever complete collection! My most frustrating moment was getting the chief in my science tribe to point out a rare collectible - and she obligingly pointed out a coral that I didn't have - only to discover that the one child in the village had just turned 14 so wasn't able to pick it up.
I'm always careful to check that I have at least one eligible child now...