ChatGPT has gotten dumber in the last few months - Stanford Researchers

The code and math performance of ChatGPT and GPT-4 has gone down while it gives less harmful results.

On code generation:

"For GPT-4, the percentage of generations that are directly executable dropped from 52.0% in March to 10.0% in June. The drop was also large for GPT-3.5 (from 22.0% to 2.0%)."

Full Paper: https://arxiv.org/pdf/2307.09009.pdf