Have you solved all the puzzles, or at least the ones you are able to do, that require building? If so, I always had the villagers exercise to gain skill. I don't recall the specific recipe that makes the villagers want to exercise, but you can find it in the the VV2 guides.
Good luck!
