If any villager gets ready to talk to him (e.g. arguing, complaining, telling a joke) he will run away before they can even get near. That's normal and expected, given his dislikes. He's never going to be the life of the party in your village.