Once you purchase anything from the supply shop, it stays in your inventory (except for the things that you use up, like chemicals, medicines and eggs). It's great to have the extra babies when you need them, but sometimes you wish you didn't have that plant anymore!