Your villagers need a certain level of technology before they will be able to restore the temple. As to the boulder, only one special villager can move that. You might like to read the guides in http://www.ldwforums.com/ubbthreads/postlist.php?Cat=0&Board=vvguide