I did a lot of testing on the farm during the development of v1.1, because the farm production had been quirky and was critical (of course) to the game. In my tests, it was only the crop graphic that failed to appear, and the villagers were, indeed, gathering food from the farm. The amount of food that was produced from the farm over the 36-hour test period was almost 10,000 units, but the only time that the crop graphics reliably appeared on the farm was when I checked on the game at least once or twice an hour. If you were to check the stats page of your game during play, I think you would see the Food Harvested statistic going up, despite the fact that you never see crops.

I think the more likely cause of starvation is that you grew your tribe too quickly. I never increase my population to the number of villagers that you had until my villagers know how to fish. The farm produces only a finite amount of food during each crop cycle, while fishing supplies unlimited food. That way, I have excess food stored up in my food bin and can quickly recover from any Island Event that disrupts my food supply.