Elaine, your villagers will continue to gain farming skill as they perform farming activities, even when the farm is between crops. That extra skill will be very valuable later on in the game when you'll need a larger food supply. You could always divert your farmers to other tasks during that time and see what happens!
