We have addressed this issue many many times. One additional problem in having more diversity would be in the babies. If you breed two villagers that have different skin color, how should the babies look? The game use separate grids for the heads and for the bodies. So we should always be careful not to mix bodies and heads that have different colors. It gets extremely complicated and hard to implement.