The villagers will complain about hunger whenever the food supply is less than approximately 400. You can still make baby villagers when the villagers are hungry, but you'll have to be a lot more persistent.