You are right! I have seen weakend villagers get much better when moved to farming- makes them tougher stock I guess!
It also helps if there is lots of food in the bin.
Older villagers will weaken and die... there's not much you can do for them!