one year on
OpenAI DevDay 2024 brings Realtime API, prompt caching, and vision fine-tuning
OpenAI unveils developer-focused tools at a subdued DevDay, emphasizing speed and cost savings over flashy model launches.
OpenAI opened its 2024 DevDay focusing on incremental developer tools rather than new flagship models. The centerpiece is a public beta of the Realtime API, which lets developers build low-latency, speech-to-speech experiences into their apps using six preset voices. In a demo, OpenAI’s head of developer experience, Romain Huet, showed a trip-planning app that could annotate a map with restaurant locations during a conversation. The API can also integrate with calling services like Twilio, though OpenAI is not adding automatic AI disclosures to calls, leaving that responsibility to developers.
The company also introduced vision fine-tuning for GPT-4o, allowing developers to use images alongside text to improve visual understanding tasks. Prompt caching, similar to Anthropic’s offering, gives a 50% discount on cached context—less than Anthropic’s 90% but significant. A model distillation feature lets developers use larger models like o1-preview to fine-tune smaller, cheaper ones like GPT-4o mini, along with a new evaluation tool in beta.
Notably absent: any mention of the GPT Store from last year’s DevDay, which had been piloting a revenue share program with top creators. OpenAI confirmed no new models are being released today—the full o1 model and Sora remain in the wings. With C-suite departures still fresh, OpenAI’s message is clear: the platform, not the hype, is what matters.
The record
OpenAI's chief product officer told reporters the departures of Mira Murati and Bob McGrew would not slow down the company's progress.
One year later — open only if you can handle spoilers
The Realtime API became a popular building block for voice agents and phone assistants. Prompt caching and vision fine-tuning were widely adopted but the GPT Store quietly faded from OpenAI's roadmap.