As long as you're not at zero, I wouldn't worry too much. If you have at least 3 of them harvesting and you keep at them to make sure that's what they are doing, you should be fine. Make sure you have people researching as well so you can get that level 3 harvesting. Then you won't have anymore food worries. Good luck!!