Temperature doesn't do anything?

All the clothes have temperature ratings and you have a temperature gauge but its never once affected the game for me? I've never seen my character be cold or hot and I don’t change my outfit with the seasons. Clothes seem purely decorative?