The faster your farmers clear away the crops the longer the wait will be until the next crop. You might consider switching one or more of your farmers to other work, especially during the fallow period, unless farming is the only skill they need to gain Elderhood.