Again, that's a little glitch in the game, and it's more prevalent when you have 90 villagers or a slower or slowed computer. I've noticed it when my farmers were taking invisible food to the food bin too.