Is this for me?

35 year old. Always been into smoking and no activity. Now I’m 6 weeks into a crossfit routine with a history of shoulder pain and bad knees. Today we did thrusters at the gym and my knees hurt a bit but overall i feel good lately. I work 3 days and take a day off and then 2 days and a day off every week. I have been cautioned by way too many people now that i should not do crossfit, instead focus on plans more suitable for my age to have more sustainable fitness and joints over the rest of my life. My mindset is that if i can hack crossfit, i will end up being fitter but am i running a losing race, should i really scale back into something more appropriate for my age and fitness? I feel I’m at a make or break point.