that is just the way it is
you can only use one faction
BUT you can have 5 villages, so make the next one magic