I've played many games on hard, and in my games the builder usually kicks off first (probably because construction really is hazardous - I've watched them get hurt). The scientists usually decline in health and die, too, but not as quickly. The farmers are always healthy. I think you're onto something - those sneaky farmers are raiding the baskets on the way to the food bin!

If you're playing at the easier levels, though, I think it holds true that research is less hazardous to a villager's health!