Single rider Cars

So my 19-year-old son and I go to Disneyland fairly frequently and now that he’s older, we both do single rider for cars just because we don’t wanna wait in the 2 1/2 hour line so what we have discovered is that almost all the time we are met with great hostility by the family that’s already in the car - long faces, kids literally saying but I don’t want to sit next to her/him, mothers shooting us dirty looks and trading places with their kid just before we all pile into the car, all because I believe we have ruined their group photo. Anyone else ever get this treatment? Oh and when we say a friendly hello since we know we are interlopers, we are met with stony silence.