I suspect that time warping is interfering with more than just the game timers. The real-time engine in Virtual Villagers is very complex and time travel is definitely not advised. Please see this thread for sad tales of time warping.