When they drink the potion does the text say 'feels springy!" or something like that? I had one villager try several times before they got it. Make sure they're right under the gem.