Huh, it sure did that in mine. After noticing it a couple of times I quit gathering it. Rather than time travel, I set my peeps to doing jobs that a long time at bedtime and put them on half speed.

When I think about it when I first started playing the game I did set the time clock a couple of times. That might have been what caused my tree to revert back...Though I quit doing it after that, because I didn't like the way it aged my little peeps.