Unusual that they pick one tree clean and not the other, but it is highly normal for farmers to strip a tree so quickly that you never ever see it with anything but 8 or 9 fruit, which is about where villagers stop harvesting it. (Less than that, and you get the message that there isn't enough fruit to harvest.)

Trees should replenish 111 food each, every three hours.