You have to put the person on the flower just right to get them to pick it, can be tricky. And wait until one person walks away before putting the next person on it. I have collected as many as 8-10 blossoms from one flower.

Also, the hummingbird appears about every 4-5 minutes, at least in my current game.