Could be your farmer fell ill, and with no doctor to heal him... and the researchers and builder just kept to their tasks until all the food was gone

It's hard when you're just starting to leave 2 doctors on duty, but there's always a chance that if you leave one, he falls ill... and then the rest of the scenario plays out.