Oh definitely more workers make the task get done faster. But, please remember, if you are in the beginning of your game DON'T take all your villagers and put them on the same task. You will need someone to gain tech points, someone to keep the food level up and someone to build.