I think you actually have to have one person doing all 4 plants. I had a similar experience where I had one villager do 2 plants and another do the other 2 plants, and the puzzle remained unsolved. I didn't keep that village around much longer, but for as long as it was around I could never get either villager to gain knowledge about the two plants they'd missed. -_-