If you like change your time and speed up the game, when you change the time back, do the villagers go back too?

No, they don't go back, but there's a possibility that your game will be messed up by playing with the computer's clock. If you search this forum, you'll find plenty of posts about the effects of adjusting the clock.

I accidently killed all my villagers....*smacks hand*

I did that once (putting back my clock) and lost all my villagers, in all my 5 villages!!!!.
If you are not patient enough, you can put back the time BEFORE you start a village and then slowly adjust your time. I did it once and it was quite funny.
Now I make a backup from my village-files every night and if I find out in the morning that a disaster has happened, I copy my backup files and adjust the time hour by hour.
I start with an hour after the backup was made, look in every village if erverything is allright, help my villagers wherever needed, make another backup and adjust the time again. I repeat that untill my computerclock is back to "normal" time.
With doing so, I have saved a lot of villagers, whom otherwise would have died from hunger or disease.
Maybe it's cheating??