if you lose the crate piece and restart the game the piece resets itself on top of the broken crate.

if you lose the mosaic piece and restart the game it re-buries itself in the mosaic again and you have to be aggressive with the totem to loosen it.

if you lose the piece in the pool and restart the game it hops back into the pool.

you can't lose the last piece.

my guess is that you are missing the mosaic piece and that you should just be very aggressive with the totem until it loosens up.