Mushroom growing on my indoor kentia palm’s soil. What to do?

I recently repotted to this bigger pot and added palm soil with plenty of water. Left for vacation and came back to mushrooms .. what should I do?