That's a lot of villagers to support from just the farm. It's usually better to wait to grow your population that much until you have Harvesting Level 3. It's going to be tight until your villagers learn how to fish. It sounds like you should be OK, though, as long as you don't make any more baby villagers until you have that next food source.