I have a villager who dislikes fish. So obviously he runs away from the ocean - however, sometimes I see him running away for no apparent reason.
If he's a farmer, when hr finishes a task he'll stop and think about what to do next. He probably decides to go fishing, then remembers he dislikes fish.