So, I found a neat cheat to get the orange masks out of your way. Get a villager you don't mind losing the use of for a while. Drop the villager on the heathens and pick him/her up to get that heathen fixated on your villager. Keep doing this until you have all the orange heathens (at least all the adults) fixated on your villager and following wherever you go. Have your villager in your grasp and off the ground. Push the "M" button for the map. You can let go of your mouse now. Push "Esc" to go back to the game. The heathens will stay in that one place, fixated on your villager. You can still select things and other villagers and it won't mess that up. I found this after all the heathens were in the old broken-down village. Probably be very hard to do with them all over the map.