With my very first village, I did not get the last 2 tablets until well after finishing the rubble. You have to keep a constant eye on the area or you will miss it and have to wait a long time again. I think that is what makes it seem like it takes forever and may never happen.