Do I need to let all the men die out and start over. no women on my island

There are several different Island Events that can bring new villagers to your game, so it's never hopeless until all of your villagers have died. Good luck!