As long as you have enough housing, you should be able to produce baby villagers. Sometimes it can take dozens of tries at the lowest level of fertility tech, but you will eventually be successful (even when they're hungry). The key is persistence.