Former OpenAI employee Miles Brundage: "o1 is just an LLM though, no reasoning infrastructure. The reasoning is in the chain of thought." Current OpenAI employee roon: "Miles literally knows what o1 does."
In another 6 months we will possibly have o1 (full), Orion/GPT-5, Claude 3.5 Opus and Gemini 2 (maybe with Alphaproof and Alphacode integrated), no one is ready for this
Nvidia CEO Jensen Huang Keynote at CES 2025: From One to Three Scaling Laws
For those who care about how o1 works technically, OpenAI has stated that o1 was built using reinforcement fine-tuning, which was announced by OpenAI on December 6 as day 2 of Shipmas
For those who care about how o1 works technically but are sleeping on reinforcement fine-tuning - announced by OpenAI on December 6 as day 2 of Shipmas - OpenAI has stated that o1 itself was built using reinforcement fine-tuning. For example here is a quote from an OpenAI job posting.
From Dylan Patel of SemiAnalysis: 1) "4o, o1, o1 preview, o1 pro are all the same size model". 2) The reason o1 is more expensive than gpt-4o is "related to seqlen kvcache overhead". 3) "o1 pro is same model [as o1] with adjustments at inference time".
Language models still can't pass complex Theory of Mind tests, Meta shows [about paper "Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoning"]
Microsoft expects to spend $80 billion on AI-enabled data centers in fiscal 2025
Anthropic to Enforce Copyright Guardrails on AI Tools
Results for the Putnam-AXIOM Variation benchmark, which compares language model accuracy for 52 math problems based upon Putnam Competition problems and variations of those 52 problems created by "altering the variable names, constant values, or the phrasing of the question"
Is o3 based on gpt4 or the new model they were training last year(gpt5) ?
Chinese researchers reveal how to reproduce Open-AI's o1 model from scratch
UK Government publishes consultation on Copyright and AI
AI Developments at the U.S. Copyright Office in 2024
OpenAI, Andrew Ng Introduce New Course on Reasoning with o1
IDF increases use of AI for Hamas target acquisition as war continues - report
Quote from a machine learning researcher about how o3 likely works from blog post "o3: The grand finale of AI in 2024"
Five breakthroughs that make OpenAI’s o3 a turning point for AI — and one big challenge
Now that we have a blueprint for reproducing OpenAI's o1 model, the only barrier is hardware performance improvements. Thoughts on what this means in the comments.
New SemiAnalysis article "Nvidia’s Christmas Present: GB300 & B300 – Reasoning Inference, Amazon, Memory, Supply Chain" has good hardware-related news for the performance of reasoning models, and also potentially clues about the architecture of o1, o1 pro, and o3
Verily, It Beginneth: the jobs are going
