The fact that players are forced to keep the population at 100, with no other choice available, is in my opinion, the biggest FLAW in the game.
Where does the figure of 100 come from? I've effectively finished the game by converting all the heathens several times now with a population of around 90. Of course, when the heathens convert, the population will rise above the 90 limit for a while, but unless the older villagers are rejuvenated regularly the population will drop down to 90 or below again as they die. I say "effectively finished" because I've never yet completed a collection that would allow me a higher population cap.