It's really your choice whether or not to temporarily divert your farmers to researching or to let them continue to work the farm (or pick berries if there are still some on the bush). You will not disrupt the growth of the next crop, but you will reduce the benefit of fishing (when you get level 3 of harvesting) to the degree that your farmers have had a reduced opportunity to build farming skills. Either way works, and sometimes I go one way and sometimes another.
