Smarter Models Killed the Harness. Here's What's Left — Jediah Katz, SpaceXAI

My name is Jedidiah, My name is Jedidiah, I joined Cursor I joined Cursor I joined Cursor in early 2025 as in early 2025 as in early 2025 as the longest- the longest- the longest- serving engineer on serving engineer on serving engineer on our our our agent shell, and recently agent shell, and recently agent shell, and recently transitioned to SpaceX as part of the transitioned to SpaceX as part of the transitioned to SpaceX as part of the Cursor acquisition. Cursor acquisition. So today I want to So today I want to talk to you about talk to you about talk to you about how the agent shell how the agent shell evolved to where it is evolved to where it is today today today and where I think and where I think and where I think things are headed. I things are headed. I things are headed. I have a major question have a major question have a major question we need to we need to we need to answer today: is the answer today: is the agent shell dead? So, let's begin. Here is a more natural and Here is a more natural and fluid version of this fluid version of this fluid version of this script, which script, which script, which is delivered as a is delivered as a is delivered as a coherent coherent coherent conference presentation rather than a conference presentation rather than a conference presentation rather than a collection of short abstracts. collection of short abstracts. collection of short abstracts. Oh. Just checking to see Oh. Just checking to see Oh. Just checking to see if everyone is still if everyone is still if everyone is still alert before alert before alert before dinner. dinner. dinner. Seriously. The story Seriously. The story Seriously. The story begins in begins in begins in early 2023, when early 2023, when early 2023, when "prompt engineer" was the "prompt engineer" was the "prompt engineer" was the most popular most popular most popular profession. By profession. By profession. By mid-2025, this mid-2025, this mid-2025, this had evolved into a “ had evolved into a “ context engineer”—a context engineer”—a person who programmatically person who programmatically person who programmatically generates the best generates the best generates the best context for a model “ context for a model “ on the fly.” And it wasn't until on the fly.” And it wasn't until on the fly.” And it wasn't until earlier this year that earlier this year that earlier this year that the industry settled on the industry settled on the industry settled on the terms " the terms " the terms " agent shell" and " agent shell" and " shell engineering" shell engineering" shell engineering" to describe the work that to describe the work that to describe the work that my team and I have been my team and I have been my team and I have been doing for a long time.

doing for a long time. So what is an So what is an agent shell? The shell is agent shell? The shell is agent shell? The shell is like the eyes and hands for the like the eyes and hands for the like the eyes and hands for the model. It encompasses model. It encompasses model. It encompasses the model, guides it, the model, guides it, the model, guides it, and connects it to the and connects it to the and connects it to the world around it. world around it. world around it. Users Users Users connect to connect to connect to our shell through our shell through our shell through applications, and the shell applications, and the shell applications, and the shell connects the model to connects the model to connects the model to computers, as computers, as computers, as well as to third-party well as to third-party well as to third-party services like MCP. I services like MCP. I services like MCP. I like to divide like to divide like to divide the shell into two levels the shell into two levels : the execution level and the : the execution level and the : the execution level and the context and context and context and orchestration level. The orchestration level. The orchestration level. The execution level is like the execution level is like the execution level is like the foundation of the shell. foundation of the shell. foundation of the shell. It controls where the It controls where the It controls where the main agent loop runs main agent loop runs main agent loop runs and how and how and how state is stored state is stored state is stored so that work can so that work can so that work can continue. continue. continue. Building this Building this Building this foundation is very foundation is very foundation is very similar to traditional similar to traditional similar to traditional software development software development software development . And the . And the . And the level of context and level of context and level of context and orchestration is like the orchestration is like the orchestration is like the Wild West. This is what Wild West. This is what Wild West. This is what many people mean many people mean many people mean when they talk when they talk when they talk about the shell. These about the shell. These about the shell. These parts of the shell parts of the shell parts of the shell control which control which control which models are called, models are called, models are called, when they happen, when they happen, when they happen, and what context they and what context they and what context they see. This includes see. This includes see. This includes prompts and prompts and prompts and tools, of course, but tools, of course, but tools, of course, but also also also memory compression and a bunch of memory compression and a bunch of memory compression and a bunch of tricks that you tricks that you tricks that you may not know about but may not know about but may not know about but that we that we that we use to make the use to make the use to make the agent behave the way agent behave the way agent behave the way we want. So this is the we want. So this is the we want. So this is the area I area I area I work in the most. At work in the most. At work in the most. At the beginning of the year, we the beginning of the year, we the beginning of the year, we reached the peak of the reached the peak of the reached the peak of the shell's capabilities shell's capabilities . Cursor announced that we . Cursor announced that we . Cursor announced that we have built a browser have built a browser have built a browser completely from scratch completely from scratch completely from scratch using a new

using a new using a new long-term long-term long-term shell called shell called shell called Grind Swarm. It was a Grind Swarm. It was a Grind Swarm. It was a multimodal multimodal multimodal shell that shell that shell that orchestrated a multitude of orchestrated a multitude of orchestrated a multitude of agents and machines, and agents and machines, and agents and machines, and it was very it was very it was very expensive to expensive to expensive to operate. operate. operate. Honestly, to do Honestly, to do Honestly, to do this demonstration, Cursor this demonstration, Cursor this demonstration, Cursor had to be "put down" had to be "put down" had to be "put down" once or twice. And now, as once or twice. And now, as once or twice. And now, as in the entire world of in the entire world of in the entire world of agents, just a agents, just a agents, just a few months into the peak of the few months into the peak of the strap's popularity, the strap's popularity, the internet started internet started internet started saying it was saying it was saying it was dead. Tariq from Anthropic dead. Tariq from Anthropic dead. Tariq from Anthropic talked about how they talked about how they talked about how they removed most of the removed most of the removed most of the system system system prompt from Claude code. Gary prompt from Claude code. Gary prompt from Claude code. Gary Tan from YC believed that Tan from YC believed that Tan from YC believed that the harness would disappear and the harness would disappear and only skills would matter. Even only skills would matter. Even my colleague Stefan and I my colleague Stefan and I my colleague Stefan and I wrote about wrote about wrote about how we removed all these how we removed all these how we removed all these fuses in Cursor that were fuses in Cursor that were fuses in Cursor that were supposed to patch supposed to patch supposed to patch gaps in the gaps in the gaps in the model's intelligence. model's intelligence. We used to have We used to have things like a things like a things like a small model that small model that small model that just made edits to the just made edits to the just made edits to the code of the file because the code of the file because the code of the file because the leading models could leading models could leading models could n't handle that. We n't handle that. We n't handle that. We also had all sorts of also had all sorts of also had all sorts of instructions for instructions for instructions for programming, programming, programming, such as how to study the such as how to study the such as how to study the codebase and how to codebase and how to codebase and how to check our check our check our work. So when all work. So when all work. So when all these people say that these people say that these people say that strapping is dead, they strapping is dead, they strapping is dead, they mean that it was mean that it was mean that it was killed by smarter killed by smarter killed by smarter models. And there is models. And there is models. And there is plenty of evidence for this. I plenty of evidence for this. I plenty of evidence for this. I want you all to want you all to want you all to think about a very think about a very think about a very simple binding where simple binding where simple binding where the model only has access the model only has access the model only has access to the shell and to the shell and to the shell and all my secrets.

all my secrets. all my secrets. By the way, please By the way, please By the way, please do not take screenshots do not take screenshots do not take screenshots of this slide. This one is of this slide. This one is banned. So, banned. So, banned. So, who here thinks that who here thinks that who here thinks that such a simple binding such a simple binding such a simple binding will handle will handle will handle code evaluation as well as a more code evaluation as well as a more complex one? Maybe complex one? Maybe someone will raise their hand someone will raise their hand someone will raise their hand who thinks they can who thinks they can who thinks they can handle it? Who handle it? Who handle it? Who thinks so? Who thinks they can thinks so? Who thinks they can thinks so? Who thinks they can handle it? handle it? handle it? Raise your hand if Raise your hand if Raise your hand if you think so. you think so. you think so. A few hands, but A few hands, but A few hands, but most don't. So, most don't. So, most don't. So, several recent several recent several recent studies studies studies have attempted to examine have attempted to examine have attempted to examine this very question. this very question. this very question. The first, conducted at the The first, conducted at the University of California, University of California, Berkeley, tested Berkeley, tested Berkeley, tested seven different leading seven different leading seven different leading models on a range of models on a range of models on a range of rigs, including Pi, rigs, including Pi, rigs, including Pi, Codex, and Cloud Code. And what Codex, and Cloud Code. And what Codex, and Cloud Code. And what they found, they found, they found, especially with the newer, especially with the newer, especially with the newer, smarter models, smarter models, smarter models, was that the choice of was that the choice of was that the choice of strap strap strap didn't really have a significant impact on the didn't really have a significant impact on the didn't really have a significant impact on the SweetBench light score. And SweetBench light score. And SweetBench light score. And then another then another then another study from Apple study from Apple study from Apple and EPFL similarly and EPFL similarly and EPFL similarly found that found that found that custom custom custom MLE binding didn't make much MLE binding didn't make much MLE binding didn't make much difference in MLEBench. So, difference in MLEBench. So, difference in MLEBench. So, what's going on here? what's going on here?

what's going on here? Is it really Is it really Is it really true that strapping is true that strapping is true that strapping is dead? Are they dead? Are they dead? Are they right? Here's what I right? Here's what I right? Here's what I think the think the think the evidence shows. Fancy evidence shows. Fancy evidence shows. Fancy binding binding binding no longer adds no longer adds no longer adds intelligence to intelligence to intelligence to programming these days. I programming these days. I programming these days. I think that's true. I think that's true. I think that's true. I know, it's strange. But these know, it's strange. But these know, it's strange. But these things that we used to things that we used to things that we used to do in a wrapper do in a wrapper do in a wrapper to fill in to fill in to fill in the gaps in the basic " the gaps in the basic " muscle memory" muscle memory" muscle memory" model of model of model of coding, like coding, like coding, like learning learning learning to explore the to explore the to explore the codebase, uh, how codebase, uh, how codebase, uh, how to work to work to work persistently for persistently for persistently for very long periods of time very long periods of time very long periods of time with commands with commands with commands like /goal, uh, how to like /goal, uh, how to like /goal, uh, how to run tests to run tests to run tests to verify your verify your verify your work. The model work. The model work. The model naturally naturally naturally learned these skills. learned these skills. So, why is this So, why is this happening? happening? It turns out that It turns out that the wrapper actually the wrapper actually the wrapper actually makes itself makes itself makes itself obsolete through what obsolete through what obsolete through what I call I call I call data flywheel. So data flywheel. So , in this process, the , in this process, the , in this process, the wrapper makes wrapper makes wrapper makes increasingly complex increasingly complex programming tasks programming tasks accessible. At this accessible. At this accessible. At this point, more point, more point, more users can users can users can productively productively productively perform these perform these perform these tasks with the tasks with the tasks with the help of agents. This help of agents. This help of agents. This means that more means that more means that more coding data coding data coding data is transmitted in is transmitted in is transmitted in the lab. And the lab. And the lab. And ultimately, models also ultimately, models also ultimately, models also naturally naturally naturally learn learn learn to perform these to perform these to perform these skills. Over time, the skills. Over time, the skills. Over time, the models become models become models become so good so good so good that the wrapper can that the wrapper can that the wrapper can actually remove what actually remove what actually remove what it previously it previously it previously had to do had to do had to do to accomplish these to accomplish these to accomplish these tasks. This is a very tasks. This is a very tasks. This is a very interesting process. That's interesting process. That's interesting process. That's why people say the why people say the why people say the wrapper is dead. And wrapper is dead. And wrapper is dead. And if the models are if the models are if the models are good enough to good enough to good enough to code well in code well in code well in any wrapper, any wrapper, any wrapper, why should you why should you why should you care? Does it

care? Does it care? Does it really matter what really matter what really matter what wrapper you wrapper you wrapper you use? Is use? Is use? Is the wrapper really the wrapper really the wrapper really dead? I say no, and there dead? I say no, and there dead? I say no, and there are two reasons for that. are two reasons for that. First, and I'll focus on First, and I'll focus on First, and I'll focus on both of these points, the both of these points, the both of these points, the wrapper actually wrapper actually wrapper actually opens up new opens up new opens up new horizons of tasks horizons of tasks horizons of tasks that the model can that the model can that the model can solve. And solve. And secondly, there are some secondly, there are some secondly, there are some tasks or parts of tasks or parts of tasks or parts of the wrapper, especially the wrapper, especially the wrapper, especially outside the outside the outside the context level, that the model context level, that the model context level, that the model cannot learn. cannot learn. So, to first So, to first talk about how the talk about how the talk about how the wrapper opens up the wrapper opens up the wrapper opens up the next horizon, I next horizon, I next horizon, I want to talk about want to talk about want to talk about Grokbot. Who here has tried Grokbot. Who here has tried Grokbot. Who here has tried Grock Bot? Can you see the Grock Bot? Can you see the Grock Bot? Can you see the hands raised? Perfectly. hands raised? Perfectly. hands raised? Perfectly. Let's go. This is great. Um Let's go. This is great. Um , so for those of you , so for those of you , so for those of you who haven't, who haven't, who haven't, Grock Bot is our Grock Bot is our Grock Bot is our autonomous colleague autonomous colleague autonomous colleague who does the work who does the work who does the work for you. Not just for you. Not just for you. Not just programming, but programming, but programming, but all types of all types of all types of intellectual intellectual intellectual work. And I highly work. And I highly work. And I highly recommend you recommend you recommend you try it. I won't try it. I won't try it. I won't oversell, oversell, oversell, but this is a groundbreaking but this is a groundbreaking but this is a groundbreaking product that product that product that has already created a new has already created a new has already created a new category. However, category. However, category. However, let's just let's just let's just talk about the talk about the talk about the wrapper. So, Grock Bot wrapper. So, Grock Bot wrapper. So, Grock Bot has a new type of has a new type of has a new type of wrapper that works wrapper that works wrapper that works not on the basis of not on the basis of not on the basis of one-time chats, but one-time chats, but one-time chats, but on the basis of on the basis of on the basis of long-term long-term coordinator bots that coordinator bots that coordinator bots that manage an entire fleet of manage an entire fleet of manage an entire fleet of performing agents.

performing agents. You can think of it You can think of it as hiring an influential as hiring an influential as hiring an influential lawyer who works with lawyer who works with lawyer who works with you, you know, you, you know, you, you know, directly and is directly and is directly and is responsible for responsible for responsible for strategy, but strategy, but strategy, but delegates his delegates his delegates his tasks to a team of tasks to a team of tasks to a team of assistants. It looks like assistants. It looks like assistants. It looks like that. An interesting thing about the that. An interesting thing about the that. An interesting thing about the GrokBot system: I GrokBot system: I GrokBot system: I mentioned that in Cursor we mentioned that in Cursor we mentioned that in Cursor we gradually removed gradually removed gradually removed controls, controls, controls, prompts, and prompts, and prompts, and constraints as the constraints as the constraints as the models got models got models got smarter. But, strangely enough smarter. But, strangely enough smarter. But, strangely enough , in GrokBot we , in GrokBot we , in GrokBot we started started started bringing a lot of bringing a lot of bringing a lot of these things back again, adding these things back again, adding these things back again, adding new prompts and new new prompts and new new prompts and new fuses. This is because fuses. This is because fuses. This is because in this new in this new in this new way of working, in the way of working, in the way of working, in the new areas where new areas where new areas where GrokBot has to work, GrokBot has to work, GrokBot has to work, there are many there are many there are many tasks that tasks that tasks that models cannot yet models cannot yet models cannot yet handle. Here are handle. Here are handle. Here are a few examples of what a few examples of what a few examples of what we are currently we are currently we are currently focused on at GrokBot: focused on at GrokBot: focused on at GrokBot: key key key workflows like workflows like workflows like creating creating creating presentations, presentations, presentations, booking trips, booking trips, booking trips, and communicating; An and communicating; An and communicating; An autonomous colleague autonomous colleague autonomous colleague should keep you should keep you should keep you informed of their work without informed of their work without informed of their work without overwhelming you.

overwhelming you. overwhelming you. Action against the request is Action against the request is Action against the request is another important aspect. another important aspect. another important aspect. GrokBot can do GrokBot can do GrokBot can do many things that are many things that are many things that are somewhat somewhat somewhat risky, risky, risky, such as sending such as sending such as sending emails to emails to emails to your colleagues. You your colleagues. You your colleagues. You want us to be able to want us to be able to want us to be able to do it, but only the way do it, but only the way do it, but only the way you intended it to be done you intended it to be done . And, of course, learning . And, of course, learning . And, of course, learning your work style over time your work style over time your work style over time through memory through memory is another is another is another big focus of ours. So those big focus of ours. So those big focus of ours. So those of you who of you who of you who have been experimenting with the have been experimenting with the programming capabilities of agents programming capabilities of agents may have noticed that may have noticed that may have noticed that none of these tasks none of these tasks none of these tasks are new. But for are new. But for are new. But for GrokBot, they are much more GrokBot, they are much more GrokBot, they are much more important, as important, as important, as everyone needs them to some everyone needs them to some everyone needs them to some extent. And extent. And extent. And now it is the now it is the now it is the control system in GrokBot that control system in GrokBot that control system in GrokBot that makes these tasks makes these tasks makes these tasks real. Of course, real. Of course, real. Of course, the model plays a the model plays a the model plays a big role, but it is the big role, but it is the big role, but it is the control system that control system that control system that helps make them helps make them helps make them achievable. So, achievable. So, achievable. So, going back to going back to going back to our our our data flywheel, GrokBot allows data flywheel, GrokBot allows data flywheel, GrokBot allows us to fill in the us to fill in the us to fill in the last arrow in last arrow in last arrow in our cycle, which is our cycle, which is when an opportunity when an opportunity like programming like programming like programming becomes becomes becomes publicly available publicly available publicly available thanks to the model. It is the thanks to the model. It is the thanks to the model. It is the control system that control system that control system that opens up new opens up new opens up new horizons for the model horizons for the model horizons for the model that it can that it can that it can solve and starts solve and starts solve and starts this flywheel again.

this flywheel again. So, this could So, this could mean, for example, mean, for example, mean, for example, taking on new taking on new taking on new tasks outside of tasks outside of tasks outside of programming. As we programming. As we programming. As we just saw with the just saw with the just saw with the GrokBot example, or it GrokBot example, or it GrokBot example, or it could simply mean could simply mean could simply mean performing tasks in performing tasks in performing tasks in software development software development software development that that that go beyond go beyond go beyond pure code. I'm pure code. I'm pure code. I'm sure some of sure some of sure some of you here have significant you here have significant you here have significant experience in development, and experience in development, and experience in development, and you know: the higher you know: the higher you know: the higher your level, the less your level, the less your level, the less code you actually code you actually code you actually write. So, there write. So, there write. So, there are many other are many other are many other important things in this work. This is the important things in this work. This is the important things in this work. This is the key idea I key idea I key idea I want you to want you to want you to understand: this understand: this understand: this flywheel and how the flywheel and how the flywheel and how the model and the model and the model and the control system together control system together control system together make it make it make it work. Another work. Another work. Another example of how example of how example of how an environment can an environment can an environment can open up new open up new open up new horizons: some of horizons: some of horizons: some of you may have heard of the you may have heard of the you may have heard of the idea of ​​a “ idea of ​​a “ software factory,” where software factory,” where many agents many agents many agents are coordinated together are coordinated together are coordinated together for the full for the full software development lifecycle. This is software development lifecycle. This is still a far from still a far from still a far from realized vision, realized vision, realized vision, but there is but there is but there is a lot of innovation happening in the industry in a lot of innovation happening in the industry in this direction. To this direction. To give an example of give an example of give an example of how this might work: how this might work: how this might work: internally we have internally we have internally we have many channels where many channels where many channels where someone can someone can someone can report an report an report an error, and error, and error, and automation automation automation instantly instantly instantly pre- pre- pre- classifies it. A cloud agent classifies it. A cloud agent classifies it. A cloud agent can then be launched can then be launched can then be launched to to to fix the error, fix the error, fix the error, and this agent and this agent and this agent will use our will use our will use our tools to tools to tools to show “before” and show “before” and show “before” and “after” videos as proof of “after” videos as proof of “after” videos as proof of the fix. And when the fix. And when the fix. And when a fix a fix a fix is ​​deployed or is ​​deployed or is ​​deployed or merged and merged and merged and begins begins begins to roll out, our to roll out, our deployment product runs deployment product runs another agent that another agent that another agent that monitors for any monitors for any monitors for any issues in the issues in the issues in the production production production environment. So you

environment. So you environment. So you can think of can think of can think of this as a kind of " this as a kind of " meta-system" that meta-system" that meta-system" that coordinates the work of coordinates the work of coordinates the work of many agents many agents many agents together. That's how I'd like you to together. That's how I'd like you to together. That's how I'd like you to take take take it. This is the first reason it. This is the first reason it. This is the first reason why it is worth taking care of why it is worth taking care of why it is worth taking care of the environment, because an the environment, because an the environment, because an advanced environment is advanced environment is advanced environment is necessary for necessary for necessary for performing advanced performing advanced performing advanced agent tasks. The agent tasks. The agent tasks. The next reason next reason next reason is that there are is that there are is that there are parts of this parts of this parts of this environment that the environment that the environment that the model cannot model cannot model cannot capture. So let's capture. So let's capture. So let's talk about them. talk about them. First, there is First, there is the infrastructure and the infrastructure and the infrastructure and runtime environment runtime environment required to required to required to make the make the make the autonomous counterpart autonomous counterpart autonomous counterpart reliable. Over the past reliable. Over the past reliable. Over the past year, we have gone from year, we have gone from year, we have gone from agents working agents working agents working on a typical on a typical on a typical task for minutes to task for minutes to task for minutes to hours, days, and even hours, days, and even hours, days, and even longer. And there's no longer. And there's no longer. And there's no way way way this would work on this would work on this would work on one one one person's laptop, which can person's laptop, which can person's laptop, which can go into sleep go into sleep go into sleep mode. This has to mode. This has to mode. This has to work in the cloud work in the cloud work in the cloud if you really want to if you really want to if you really want to have a standalone have a standalone have a standalone counterpart. Next, we have counterpart. Next, we have counterpart. Next, we have trust and security. trust and security. trust and security. Yes, as we saw Yes, as we saw Yes, as we saw earlier, a simple earlier, a simple earlier, a simple environment with bash and environment with bash and environment with bash and all your API all your API keys in a file can keys in a file can keys in a file can handle the handle the handle the coding tasks, but you coding tasks, but you coding tasks, but you probably don't want to probably don't want to probably don't want to give all give all give all your API keys to the model. Or your API keys to the model. Or your API keys to the model. Or if you don't mind, I'm if you don't mind, I'm if you don't mind, I'm sure your sure your sure your security manager would have security manager would have security manager would have something to say about something to say about something to say about this. This is this. This is this. This is obviously very obviously very obviously very important for companies important for companies important for companies where we want to have where we want to have where we want to have permissions limited to permissions limited to permissions limited to specific agents specific agents specific agents and people, but it's also and people, but it's also and people, but it's also important to have important to have important to have safeguards for those safeguards for those safeguards for those who don't want to who don't want to who don't want to wake up one morning with wake up one morning with wake up one morning with catastrophic catastrophic catastrophic consequences or a consequences or a $10,000 API bill, right? And

$10,000 API bill, right? And finally, finally, finally, efficiency. If you efficiency. If you efficiency. If you recall the two recall the two recall the two studies I studies I studies I cited where the choice of cited where the choice of cited where the choice of medium did not have a medium did not have a medium did not have a significant impact on significant impact on assessment outcomes, both of assessment outcomes, both of these studies these studies these studies found a significant found a significant found a significant difference in the cost of difference in the cost of difference in the cost of completing the task. For completing the task. For the past month, I have the past month, I have personally focused personally focused personally focused on optimizing the on optimizing the on optimizing the performance of the performance of the performance of the Grok Bot environment. Grok Bot environment. Grok Bot environment. By tuning By tuning By tuning parameters such as parameters such as parameters such as cache hit rate cache hit rate , cache , cache , cache efficiency, efficiency, efficiency, token savings, and token savings, and model combination selection, we were able to model combination selection, we were able to significantly reduce significantly reduce significantly reduce costs and costs and costs and speed up Grok speed up Grok speed up Grok Bot for users Bot for users Bot for users without sacrificing quality. without sacrificing quality. without sacrificing quality. Users Users Users really appreciated it. That really appreciated it. That really appreciated it. That was great to see. was great to see. was great to see. While it may be a While it may be a While it may be a separate topic for separate topic for separate topic for conversation, exactly how we conversation, exactly how we conversation, exactly how we did it, the main did it, the main did it, the main takeaway is takeaway is takeaway is that the more you that the more you that the more you use use use agents in your agents in your agents in your work, the more work, the more work, the more important and important and important and valuable it becomes for you to valuable it becomes for you to valuable it becomes for you to have the have the have the most most most efficient efficient efficient environment possible. So now environment possible. So now environment possible. So now you may be you may be you may be convinced that convinced that convinced that the environment still the environment still the environment still matters. Then you matters. Then you matters. Then you might ask, "What might ask, "What might ask, "What next?" Here are a few next?" Here are a few next?" Here are a few trends I trends I trends I foresee. First, foresee. First, foresee. First, this flywheel this flywheel this flywheel is accelerating, and is accelerating, and is accelerating, and now we have entered a now we have entered a now we have entered a new era where the product is new era where the product is new era where the product is once again evolving once again evolving once again evolving faster than the model. This faster than the model. This faster than the model. This means that means that means that the environment will become even more the environment will become even more the environment will become even more chaotic before chaotic before chaotic before it can be brought it can be brought it can be brought into order.

into order. More and more new features will be released. More and more new features will be released. Therefore, we expect Therefore, we expect Therefore, we expect people to build people to build people to build and design their and design their and design their environments in a way that makes it environments in a way that makes it environments in a way that makes it easier to easier to easier to implement and implement and implement and roll back new roll back new roll back new features. One features. One features. One way this is already way this is already way this is already evident in both Grok evident in both Grok evident in both Grok Bot and other Bot and other Bot and other similar products similar products similar products is by hiding the is by hiding the is by hiding the internal processes of the internal processes of the internal processes of the environment from the environment from the environment from the user. Some of user. Some of user. Some of you may have heard of you may have heard of you may have heard of Hyrum's Law, which Hyrum's Law, which Hyrum's Law, which states that whatever you states that whatever you states that whatever you reveal to reveal to reveal to users will become users will become users will become part of what part of what part of what they will they will they will depend on. Of course, there are depend on. Of course, there are depend on. Of course, there are some trade-offs in some trade-offs in some trade-offs in giving giving giving users less users less users less visibility into what's visibility into what's visibility into what's happening, but happening, but happening, but the advantage is that there's the advantage is that there's the advantage is that there's less less less change management required and change management required and change management required and you can iterate much faster you can iterate much faster you can iterate much faster . Another . Another . Another way we way we way we see this see this see this trend manifesting itself is in the trend manifesting itself is in the trend manifesting itself is in the increased focus on increased focus on increased focus on developing primitives developing primitives developing primitives for agents. for agents. for agents. Primitives are like the Primitives are like the Primitives are like the basic building basic building basic building blocks of interfaces blocks of interfaces blocks of interfaces that can be that can be that can be combined and combined and combined and extended to extended to extended to create powerful create powerful create powerful features. With features. With features. With primitives, you can primitives, you can primitives, you can add new add new add new features without features without features without complicating the complicating the complicating the agent's operation, and you can agent's operation, and you can agent's operation, and you can remove them without the remove them without the remove them without the need for need for need for complex complex complex refactoring of the refactoring of the refactoring of the codebase. This is simply codebase. This is simply codebase. This is simply quality engineering.

quality engineering. quality engineering. One example of a One example of a One example of a primitive we've primitive we've primitive we've added to our added to our added to our agent environments agent environments is subscriptions. Agents is subscriptions. Agents is subscriptions. Agents need a way to " need a way to " wake up" in wake up" in wake up" in response to events. response to events. response to events. This could be a This could be a This could be a message in Slack, a message in Slack, a message in Slack, a CI error in PR CI error in PR CI error in PR regarding an agent, regarding an agent, regarding an agent, or just a certain time of or just a certain time of or just a certain time of day. Now, instead of day. Now, instead of day. Now, instead of having separate having separate having separate integrations for integrations for integrations for each of these things, we each of these things, we each of these things, we have a single system have a single system have a single system and tool and tool and tool where a where a where a model or agent model or agent model or agent can sign up can sign up can sign up for an event. Another for an event. Another for an event. Another cool thing we cool thing we cool thing we do—I mentioned do—I mentioned do—I mentioned earlier about earlier about earlier about deterministic deterministic deterministic fuses to fuses to fuses to correct correct correct unwanted unwanted unwanted model behavior. For example, model behavior. For example, model behavior. For example, if GrokBot isn't responding to if GrokBot isn't responding to if GrokBot isn't responding to a user a user a user quickly enough, quickly enough, quickly enough, we can give it a little we can give it a little we can give it a little nudge nudge nudge by saying, "Hey, by saying, "Hey, by saying, "Hey, the user is waiting for the user is waiting for the user is waiting for you." Of course, we'd you." Of course, we'd you." Of course, we'd better not do that better not do that . This is kind of a "hack", . This is kind of a "hack", . This is kind of a "hack", but we'd rather but we'd rather but we'd rather fix it now fix it now fix it now than wait for the than wait for the than wait for the next next model checkpoint to be released. So to keep our model checkpoint to be released. So to keep our codebase less codebase less codebase less cluttered and easier for us to cluttered and easier for us to work with in work with in the future, we the future, we the future, we centrally centrally centrally track all these track all these track all these things, which we call " things, which we call " shims," ​​and every time a shims," ​​and every time a shims," ​​and every time a model model model uses one of uses one of uses one of them, it's them, it's them, it's logged. Now we have a bot logged. Now we have a bot logged. Now we have a bot that monitors this that monitors this that monitors this control panel, and control panel, and control panel, and every time the "shim" every time the "shim" every time the "shim" stops working, stops working, stops working, this bot creates a PR with this bot creates a PR with this bot creates a PR with a proposal a proposal a proposal to remove it. Another thing to remove it. Another thing to remove it. Another thing that I think is that I think is that I think is waiting for harness is a waiting for harness is a waiting for harness is a move to a higher move to a higher move to a higher level of abstraction. The level of abstraction. The level of abstraction. The previous generation of previous generation of harness was built harness was built harness was built by combining a by combining a by combining a large number of large number of large number of calls to LLM.

calls to LLM. calls to LLM. Today, the new harnesses Today, the new harnesses Today, the new harnesses we see, like we see, like we see, like GrokBot and the Software Factory, GrokBot and the Software Factory, GrokBot and the Software Factory, use use use agents as a basic agents as a basic agents as a basic building block. And building block. And building block. And finally, I predict finally, I predict finally, I predict that within 6 months, that within 6 months, that within 6 months, 90% of agents' work 90% of agents' work 90% of agents' work will be done off- will be done off- will be done off- laptop. Everything is already laptop. Everything is already laptop. Everything is already moving towards this, and I moving towards this, and I moving towards this, and I believe it is believe it is believe it is necessary in order to necessary in order to necessary in order to have truly have truly have truly autonomous agents autonomous agents autonomous agents who can who can who can work with us exactly work with us exactly work with us exactly as we want. as we want. So, is "harness" dead ? The parts that made ? The parts that made ? The parts that made it smarter in it smarter in it smarter in coding—yes, that's the coding—yes, that's the coding—yes, that's the flywheel in action. This flywheel in action. This flywheel in action. This is a working flywheel is a working flywheel . But "harness", as we . But "harness", as we . But "harness", as we have seen, is what have seen, is what have seen, is what allows agents allows agents allows agents to open up new to open up new to open up new horizons of work. horizons of work. horizons of work. Furthermore, the trust, Furthermore, the trust, Furthermore, the trust, efficiency, and efficiency, and efficiency, and infrastructure that infrastructure that infrastructure that harness provides will harness provides will harness provides will not go away. not go away. So, the harness of the past So, the harness of the past may be dead, may be dead, may be dead, but long live the agentic but long live the agentic but long live the agentic harness of harness of harness of tomorrow. Thank you.