I'm guilty of that too, but it's hard in the beginning to get everything going, you need to have villagers you can depend on to get the job done!
_________________________