Your game is progressing so slowly because you only have 5 villagers left. Of course, more villagers mean more food resources are used up so you should just have the amount that your current food will support. And like Cadillac said, the all males is a problem but you might get a miraculous event.