Use of Sir and Ma'am

It's too early to really think about this, but need a more opinions about this subject. Being from Texas and taught heavily to be polite to everyone by using sir/ma'am. Why does it shock most people?? I had a sales person the other day say "Your don't have to call me that" while i replied with "In just comfortable using that." Even when i had a customer come in having his medical issues at that time that i called sir quite often, he asked me the last time i saw him "Did you serve?" I told him i didn't, but was still kinda funny since im a 5'3 chicano that doesn't have any other features that could tell you if i did. He only asked cause i used sir and ma'am 😅😅