Brubeck et al’s unicorn experiment. An unintended consequence of putting safety limitations on GPT-4 is that it became less visually creative.

Eventually, it started to degrade. Once they started to train for more safety, the unicorn started to degrade. So tonight if you go home and you ask GPT-4 and ChatGPT to draw a unicorn in TiKZ, you’re going to get something that doesn’t look great.

Sébastien Bubeck (26:24)