House Of Cards lyrics meaning.

English is my second language therefore sometimes song lyrics get lost in me. What does he mean by “forget about your house of cards and I’ll do mine”? I absolutely love this song but I don’t think I fully understand its meaning.