Definitely prefer a more cohesive looks for all the elements involved. Will the huts, stone idol, temples etc be cartoony like the villagers? If yes, then probably better to make the background more cartoony.