The farm was never intended to support 65 villagers - that's why there's another level of Harvesting technology! Your population is pushing the limits of how many villagers can be fed from the farm's produce. I normally don't go beyond 12 villagers until my villagers know how to fish, unless I've decided to flirt with disaster. You can always choose to play in such a way that your villagers are frequently on the brink of starvation, in order to have more villagers working on things. There is a certain sense of satisfaction when they are saved from starvation in the nick of time. Your villagers won't die right away when they run out of food, since it takes some time to starve to death!

Once your villagers know how to fish, food will never be an issue, as long as you have a sufficient number of them working on the food supply. Most experienced players seem to choose to research all of the levels of Harvesting as quickly as possible, so that food is not an issue and research goes much faster with the larger population that fishing allows.