Villagers seem to consume about 20 units of food per village year. At 2x speed you still have about 51 minutes before the food supply reaches zero. Much longer at other speeds. And, as Arnie pointed out, it then will take a LONG time for there to be a really significant effect on the villers' health.
If you get to a point where you need to rely on mushrooms for food, you might want to consider switching to 1/2 speed, since the food value for mushrooms (and crabs, but you can't get those with algae), unlike other food sources, remain constant regardless of game speed and the consumption rate seems to be based on a villager year... taking longer at 1/2 speed. Of course you will need to monitor your village constantly (or nearly so) to take advantage of the mushroom crop.
Jer
-Well, to be Frank, I'd have to change my name.-