Markdown source

Transcript: The Golden Age of AI Engineering — Alexander Embiricos & Romain Huet & Peter Steinberger, OpenAI

Source Video

Local Cache

Transcript

Good morning everyone. I'm Raman. Hey everyone. I'm Alexander. Wow. This room is incredible. There's over over 7,000 AI engineers here today with us. And you know, it's not just about who's talking about this technology. It's also about who's actually using it and pushing the frontier every day. So, we couldn't be more proud to be here with all of you today. And when we were thinking about this event with Alex, we kept coming back to the World's Fair. And the World's Fair actually the future visible to everyone by building it in public. You know, ideas that previously sounded impossible were actually suddenly there. People could see them. They could walk into them and they could even start to believe in them. And honestly, this event has the exact same energy. The future of engineering is not arriving from somewhere else. It's really being built here by the people in this room and much faster than most expected. And that's why it's a little surprising that people keep saying that engineers are going away. The argument is that coding is, you know, abstracted away and therefore eventually we won't need engineers. Well, in fact, we think it's quite the opposite. You know, software ate the world and then AI ate software. But now what we're here to say is that the AI engineers are eating the world. AI engineers are the people here pushing the frontier. Yes. And you all are figuring out how this new capability can reach everyone. And there has never been a better time to be an engineer. In fact, because engineering was never about writing code. Engineering has always been about solving problems for yourself and for other people as well. It's about taking the latest science and combining it with design, with taste, with judgment, and most of all, imagination to make something that people can actually use. And in that sense, it's not the end of engineering. We think it's a return to the roots of engineering. And the technology we're building on is accelerating, getting faster and faster. For example, we used to ship a new model every 15 months or so, and now it's about roughly every six weeks. And in case you missed it, last week we launched a preview of the 5.6 series, and we're super excited to get into all of your hands. Now, building on top of all these models, the rate of product progress is relentless. And as a result, I don't have to tell you, engine the engineering feels completely different. So just to go over a couple years of what for me were successive like mind-blowing experiences. You know obviously for a long time we've had you know completion and then we went to inline prediction and then finally we had then we had command K where you can ask a model to make a change but they wouldn't test the work. Then models started testing the work and now we have models taking on long hard goals until they're done. And for me each of these phases I remember the first time was just mind-blowing and then obviously afterwards you just get used to it and you're trying to get your work done. Yeah. In fact, like I can't believe that build and test loop was not even part of the models just two years ago. This was a picture of me at dev day 2024 and I used 01 in preview at the time to build a mini drone drone interface from scratch. And the slightly insane part is the model could not actually run the code or verify its own work. And I knew the demo would work most of the time, but surely not all of the time. So I had to cross my fingers. You can kind of see here that I was pretty nervous. But hey, that's me. I only do live demos, so I never know what's actually going to happen each time. Luckily, it did work. And by dev day of last year in 2025, I was confident enough now the models could test their own work to kind of control an entire camera system and lighting system live. But yeah, we've come a long way. Yeah. So, we refer to as the demo god. And you know, before the demo, like I'll ask him, so how often does demo work? And be like, you know, three times out of four. And all right, good luck. Um, obviously we've come miles since then. And this year alone has been crazy. So what we're putting up here are all the things that we've shipped so far. No, actually not even all, a selection of the things that we've shipped so far this year. Uh my favorite things that we've shipped are like Codex app, goal mode, remote. These are things that really changes how it feels to do work. You know, obviously we couldn't do these things if we didn't use codecs to build codecs. But I think to me what is most interesting is that now Codex can do and agents can do any task that you can do on your own computer. And so that means they're not just helping you with the coding, but they're helping you with what happens before the coding and they're helping you with what happens after the coding. And this is really key, right? I think there's been a lot of talk, there will be a lot of talk today about loops. And if you can connect the agent to not only the work that you have to do, but why it has to be done, that's how you can get the agent to start to begin much more work. And then if you can connect it to what you do afterwards, review and deploy, that's how you help it land much more work. So with all of this, of course, we can move much faster. But to me as a product person the most exciting thing is actually that we make better decisions around what to build. For instance, we try we prototype many more ideas and we spend much more time with users. So yes, that's all of you. So wanted to pause and just give you all a big thank you both for the love and the constructive feedback. I would say it's safe to say that we Codeex and actually the entire industry wouldn't be here without you. Yeah, thank you so much for all the feedback. We're constantly listening to all of you. Thank you. [applause] Okay, so the models are getting really good. I would say if you pick like a medium length computer task and you give me and the model the same amount of time to get that task done, probably at least in my case, the model will do a better job than me for the average task. And so, okay, we're we're getting these models, you know, in some ways they're smarter than us. They can do almost anything. How should we shape that? What should the product that we use feel like? And so to answer that, we look to our mission. The part of it here that I've got up is, you know, AGI that benefits all of humanity. And I think in order to do this, there are two main questions that I think about now. One is how do we set up the agents to actually do things in the world? So what can they do? You know, gradually agents are getting connected to more and more things. Then where do they run? More on that later. And then the other question is how do we use these agents? For us, you know, what what should the product feel like around them? And for us, the goal is squarely not to automate engineers. Instead, the the product shape that we want is one that maximally empowers engineers. So, you know, if we think about what that product shape is, we actually think it's pretty simple. I I read a lot of sci-fi and, you know, watch superhero movies and I actually think that the the simple ideas in there are approximately right. So, there are two modalities roughly. Chat, I actually think I know some people think chat is dead. I think chat is underrated. uh and some kind of hands-on experience. So, what you want is a single entity that you can ask for help with anything anywhere. And then you want a a powerful collaborative UI that you can use when you want to inspect, steer, or shape things yourself. And so, I had codeex image gen me an illustration of this to help understand when you might want to use these things. And so, my analogy for you, yeah, I hope you like the image gen. Uh my analogy for you would be it's just like working with a team. Most of the time you're just talking about stuff and your team is just doing stuff. You don't actually want to watch over the shoulder or like have to walk over to the workbench of your teammate for every single unit of work. Mostly you just want to talk and let them cook. And then every now and then you want to dig in and really dig in all the way to the weeds of things and dig into that problem together. And for us as we build product, we have this idea that we want to make it so that you can retain this feeling of mastery of the work that you're doing because that's really powerful. We don't want to make it feel like actually it's really hard to like get to the details and like you know disassemble the hardware in this case. So the way that we're bringing to this to life is just the beginning, but this is why we built the Codex app. You get a very simple chat interface that you can use for coding and for anything else and you know you can have a conversation and then go as deep as you want. though in the case here, we have Roma's predicted score of this upcoming World Cup match. I hope I'm right. I hope I'm right. We'll see. [laughter] Okay. Um, and what you can do here is you could go in and you can point at a very specific thing and say, "Hey, I want you to make this change or you can make this change yourself." And a fun story here is that actually I remember pitching some of you who I know are in the audience uh this idea before we started and I was told squarely like we I will never use such a tool. I will never leave my terminal uh or VMware or Emacs. Um but actually those people are now using it and even internally like within our team there were a lot of questions like why should we build this? People love the CLI, they love the IDE and it's a little subtle but our take is that you can't really build that collaborative interface for any kind of work in a CLI. It's mostly chat and then in IDE the order is wrong. So you're starting with the code, but now it's time to transition to like working with teammates where you chat first and you dig in when you need it. Totally. And we're moving really fast on this on the product surface and the model layer of course, but we're also trying to keep pace with all of you. Right. Honestly, half the time I open X, I see someone in this room doing something that I had not realized Codex could actually do. And honestly, this is what pioneers do. You guys experiment. You set up tools for yourself, for your team. And in turn, we get inspired. we learn from you and eventually everyone benefits and so we are helping uh uh you are helping us figure out like what to uh what to build next and also what the future of engineering should look like but for that to work one thing that we we really care about is that codeex cannot be a closed product that only openai can improve so we've intentionally designed codeex as a set of layers that anyone can build on and we want to show you a little bit of that stack today and how it it manifests first it starts with the model and Alexander showed how quickly we're progressing on models and you guys use these models through the responses API and guess what this is how we build the CEX app right we use the same models through the same API and we actually are building on the same thing that we give to developers and when CEX needs something new we always try to bake it into the API first so you can benefit as well one example recently was compaction codex needed a way to compact long context for longunning tasks and so we'll build that into the API So that means your agents can use the same primitives that we built for ourselves. Moving on to the next layer, the CDEX harness is also open source. So you can inspect it, you can fork it, you can adapt it. And we also took the same approach with agents MD. Instead of reinventing a new file format uh for codeex to follow instructions, we thought let's pick a name that other agents can actually use as well. The models are the default in the in the harness, the models from OpenAI, but they're not hardcoded in there. So if you want to use an open model and keep the same agent loop, you can. And we also bring this codeex harness into the post-training process of our model. So that means the models can learn to call tools and navigate an environment that's actually something that's open source. Now take the open code team for instance. They actually were able to inspect how we have this like reference implementation and they could reuse the parts that make sense to them or change entirely all the rest and make different choices. I know for instance they were trying to see how it did like signing with chat GPT and so they could look at the code and and learn from it and we think it's better than having developers reverse engineering how it builds uh and and how we launch. But now let's say speaking of subscriptions that we want to go a level higher and how you bring this harness into an app and how do you let people sign in with their existing codec subscription for instance? Well, it turns out we had the same problem ourselves because we wanted to build the VS code extension and the codeex app and we wanted to have a unified way to like actually control this harness. So we built apps server and we also made that open source and the app server is not kind of a a community adapter. It's really the path that we use for our own products and you can use it too. Uh Tuma for instance here aka Dimilian on X uh he built his own native app for Codex called Codex Monitor before we even launched the Codex app because he could build that uh using the app server and now he works on our team and he actually built uh Codex for iOS. And moving up the stack at the app layer, we also want to make sure that innovation is not blocked on our own ideas. And so we build extensible primitives here uh like the inapp browser that we showed on the screen and plugins. So if you take for instance browser use and computer use these were built as plugins using the same extension points that we have available for for all of you. And lastly we also recently built RO specific plugins for Codex say to make it easier to to customize for people who work in data science or design for instance. And these plugins are also open source. So you can see under the hood and get inspired from them if that's useful. Our goal is really to keep making this as open and flexible as we can. And the best part is people can use their existing subscription in more and more places from open code, pi, droid, openclaw to even xcode and jet brains as IDEs. And you can see how they're becoming quite a meaningful part of how people use these tools and that's really why we want to care about building this open ecosystem with all of you. So really, if there's one thing I want you to take away from this section and this stack, it's this. We're not building one system for OpenAI and a second system that's simplified for developers. At every layer, we actually use the thing that we give to you. And we want to thank all of you because every time you fork the harness, every time you find the edge of capabilities of the models, it means we get to learn and improve. And honestly, with 7,000 of the finest AI engineers in this room today, I'm confident that all of you will define a lot of how we uh will experience AI and how the world will experience AI in the future. So, thank you. [applause] I want to give a shout out to whoever over there is injecting in that's you. Okay. Thank you so much. Um so with all of your help we are making agents explosively useful. Um and so now the question is how do we get value out of them? And you know that's not token maxing. We have a term for this that maybe you use as well. I don't know is it on screen? Value maxing. So you know when we talk to engineering leaders most of the conversation is about some themes relating to the idea of value maxing. So we're going to walk you through a few common topics that come up. some things where we've already made a lot of progress and some things where actually there's a lot more progress to still be made. So the first one of these is cost efficiency. Everyone wants frontier intelligence. Pick your favorite eval. You want the best model. So with terminal bench here for instance, that's GPT 5.6 SUL. And like I said, we can't wait for you to have it. But okay, you also want as much intelligence as you can get. And that's where efficiency comes in. Cost efficiency has been a focus for us for quite some time and the results are continuing to pay off. So for example, GPD 5.6 terra I think it's in like dark blue in there brings GP 5.5 level intelligence but at half the cost and Luna there beats some pretty notable models in this eval but at only $1 per million input tokens and $6 per million output tokens. I'll leave it up to you to compare those costs but that is insane value. Yeah, we we really can't wait to uh to see all of you build uh with GPT 5.6 and this new family of models. Now, the next thing I want to touch on is speed, right? GPT 5.3 Codex Spark showed you what speed can unlock, but we also know that you all want frontier intelligence. You don't want to have a model that's like not as great as what you can operate at the very best. Well, this is GPT 5.6 cell running on Cerebras. The frontier intelligence at now 750 tokens a second. We can't wait to see what you can build with this next month. And honestly, to put that in perspective, this is kind of like having a pretty substantial PR written in like 10 seconds. And it's not just about getting one answer faster, right? It's about what can you do with that speed. You can think about an agent taking different approaches, maybe like five or six in parallel, maybe like, you know, coming back and picking the best one in the time it would have taken to not even generate just one. So, we really can't wait to see what that can unlock when you have frontier intelligence, the very best at that speed. It really starts to feel less like waiting for an AI to respond and much more like a co-orker that's like already showing you the results as it goes. Speaking of working with co-workers, um, can I get a show of hands? Who who is familiar with this kind of site in offices? Okay. Okay. Wow, a lot of you are very well behaved. I see some people up front. Um, so yeah, a lot of people are keeping their laptops open so that agents can keep working. And this is funny, but you know what we really want is to be able to shut our computers. Um, and we want to be able to run many tasks in parallel isolated on their own box. Now, we've been actually aiming at this from the start. Our first major launch was Codex Cloud, and it is due for some major upgrades coming soon. But better yet, as we think about this, the future shouldn't have this awkward distinction between like a local task and a cloud task and you have to decide where to run everything. Really, what you should have is kind of going back to what I was saying earlier. You should just have an agent. You talk to it wherever, whenever, about anything, and it should figure out, okay, what do I need to do? Which environment is right for my work? And use whatever is available. In fact, Theo made this prediction over the weekend on this very topic and it's a pretty acute tweet. Like, Alex, what do you think? Sooner or later than six months? I think the not maybe not the exact details, but the vibe of this tweet much sooner than six months. Yeah. I mean, at the pace at which everything is go is going, I would not be surprised if it's sooner indeed. Um well, so now you might be wondering where's the live demo today? Uh well for this AI engineer we wanted to do something a little different this time around and we think it's a very unique moment for all of us to kind of reimagine how we work and how we build and so we wanted to bring a special guest who has bent what's possible with agents and really has pushed us to be more AGI at OpenAI. So with that, please welcome to the stage the claw father, Peter Steinberger. [applause] [applause] Peter, take it away. Thank you all. Thanks all. Good morning everyone. You know, I love this picture because it reminds me just how much has changed in a few months. I was juggling 10 or more terminal windows, always waiting for one of them to finish so I could steer the agent and cue new work. In in January, that felt like peak productivity. Today, it feels a little bit silly. I thought I was orchestrating really. I was Pauling. I was the scheduler, the router, and the memory. You know, at first I paired with one agent with 10 terminals. I was no longer pairing. I was managing 10 direct reports. Now I mostly talk to a longunning manager which delegates work to a team. For tricky work, I can still drop down and pair directly with a worker. But my default changed. I managed the manager of a small company of agents. Three changes made that possible. Number one, serverside compaction made longunning tasks reliable enough that I stopped optimizing around fresh sessions. Coordination lets one thread create and steer the right projects. And third, automation can wake the same manager when something happens. So we have persistent context, delegation and triggers. There's your loop. And once the loop starts working, you discover the next problem. The bottleneck keeps moving. You know, last year I was primarily constrained by tokens. Now I fixed that by joining OpenAI. I know. I know the strategy does not scale. Then my constraint shifted to token uh compute. All these threads run at the same time and my MacBook starts sounding like a chat engine. That's mostly fixed by using test boxes. So agents can run tests on a separate machine. Now I'm primarily constrained by attention. And unlike tokens or compute, I can't simply add more of it. So the most important skill is today is deciding where to spend it. Are you still staring at the agent while the code flies by? I know. I know it's it it feels cool, but with the earlier models, this was necessary. You know, you you you you see the agent go in a direction you don't like. You hit escape, you steer it, you steer it back. But the latest generation of models is so good at understanding intent that it's a little bit of a waste of time to watch the agent generate code. Imagine someone files an issue on one of my open source projects. The manager wakes up, reads it against the project's goals, notes, and vision, and decides whether it might be a fit. If it does, it creates a worker. That worker investigates, implements the change, runs the tests, and another agent can review the result. I don't need to watch those agents work or consume every intermediary message. When the manager needs me, it returns a PR, the original issue, a proposed diff, maybe a video or even a running build I can VNC into. I review once, I re leave a note, I maybe approve, the loop continues and it can land after the checks pass. The agent runs the inner execution loop. I set the direction and I make decisions in the outer loop. You know, Paul Paul Salt is already running a version of this. He pinned his chief of staff. It wakes up every 10 minutes and it coordinates his GitHub work. The agent creates threads in the sidebar so Paul can jump in whenever the work needs additional steering. You know, once the manager is long lived, tying it to a laptop just feels wrong. Codex can already move work between hosts. Open Claw has a gateway and nodes, but neither feels like the final form. I don't even want to think where I work. My agent should be able to connect to any of my machines. They should know which work can be done in the cloud or which work requires my local machine. The manager shouldn't be a session trapped inside your app. It should be an agent that I can text, steer from Slack, or hear from wherever I am. Really, why can't I talk to my agent and have it design the whole loop for me? We haven't solved that yet. Models are advancing faster than the harnesses and organizations around them. Designing those things is the next engineering problem. That's where all of you come in. The future is not 20 terminals. It's better loops. Let's build them. Thank you.