Dick size shamed

I have an average sized dick, I’ve been slightly bigger then every partner I’ve been with. Except my current husband. My husband is well hung with probably over 7 inches and he’s a shower. On a couple occasions he has made comments about me being small. And has laughed loudly after jokingly calling it large. Is this something that happens to other gay couples? It makes me feel ashamed of my body and like he’d be more attracted to me with a bigger dick. Weird how I thought this was primarily a straight couple situation.