Things We Learned About LLMs in 2024

Simon Willison · Independent · 2024-12-31

Read on simonwillison.net

Willison's year-end round-up is the most-cited single page on LLM developments precisely because he refuses to abstract. Twenty distinct sections, each anchored in a concrete release or benchmark, walk you through the year a working practitioner actually had — GPT-4 commoditized, prices in free-fall, agents stubbornly not arriving, evals quietly becoming the bottleneck.

If you only read one mainstream-press AI summary of 2024, read this instead. It is the version skeptics, optimists, and engineers all quote against each other, which is a useful sign that the reporting is honest.

> The GPT-4 barrier was comprehensively broken.