The effect on the game depends upon whether you adjust the time forward or backward. Adjusting the clock backward usually affects only the "Realtime hours played" statistic in Village Sim, and can make it display an incredibly large number of hours played. That situation will usually be resolved once time "catches up with itself" (so to speak)!

Adjusting the clock forward by many hours can be more problematic for your villagers, because it's the same as leaving your little tribe unattended for that period of time. The good news is that ill effects are easily prevented by launching Village Sim (or any of LDW's realtime games) and pausing them before adjusting the system clock forward. That way your villagers (or plants or fish) will be placed in "stasis" and will be fine when you resume play again.