You don't say how long your crops are taking to grow. They should appear every 4 hours on normal speed. One thing that will get you into trouble with the food supply is growing your population too much before your villagers are able to fish. If you have more than a dozen or so villagers with only the berry bush and the farm, food will be a problem.