I think the mask says something along the lines of "Pause the game if you're going to be away for some time." I believe by that it means some days or weeks, not a matter of hours. Although in the early game it's wise to pause it during the period when the villagers are struggling to get established. Later on, especially if the food supply is assured, you can afford to leave the game running in real time.

If you're still feeling cautious, try dropping to half speed, or normal speed if you were running in 2 X speed, when the game is off.