---
title: "Transcript: I Run a Fleet of AI Agents Across Three Machines. Here's What Broke. - Kyle Jaejun Lee, KRAFTON"
category: "transcripts"
videoId: "4kYl2_mqmnQ"
sourceLabels: ["YouTube transcript", "Cached transcript markdown"]
wordCount: "4930"
---

# Transcript: I Run a Fleet of AI Agents Across Three Machines. Here's What Broke. - Kyle Jaejun Lee, KRAFTON

## Source Video
- [YouTube](https://www.youtube.com/watch?v=4kYl2_mqmnQ)

## Local Cache
- `raw/sources/youtube-transcripts/4kYl2_mqmnQ.txt`
- 4,930 words

## Transcript

Hello, I'm Kyle. I run a fleet of AI Hello, I'm Kyle. I run a fleet of AI coding agents across three machines coding agents across three machines coding agents across three machines every day, and this is the story of what every day, and this is the story of what every day, and this is the story of what broke, starting with me. broke, starting with me. broke, starting with me. This is my fleet, three machines I run This is my fleet, three machines I run This is my fleet, three machines I run every single day. The MacBook is where I every single day. The MacBook is where I every single day. The MacBook is where I do my heavy coding and my personal do my heavy coding and my personal do my heavy coding and my personal projects, and it sleeps because, well, projects, and it sleeps because, well, projects, and it sleeps because, well, it's a laptop. Then two Linux boxes, it's a laptop. Then two Linux boxes, it's a laptop. Then two Linux boxes, both headless and always on. Linux A both headless and always on. Linux A both headless and always on. Linux A takes the long-running coding tasks. takes the long-running coding tasks. takes the long-running coding tasks. Linux B runs my short-lived personal Linux B runs my short-lived personal Linux B runs my short-lived personal side projects. Two operating systems, side projects. Two operating systems, side projects. Two operating systems, one control plane tying it all together. one control plane tying it all together. one control plane tying it all together. And to be clear, this isn't a demo I And to be clear, this isn't a demo I And to be clear, this isn't a demo I spun up for the talk. This is what I spun up for the talk. This is what I spun up for the talk. This is what I actually use all day, every day. actually use all day, every day. actually use all day, every day. And here it is live, the whole thing And here it is live, the whole thing And here it is live, the whole thing running on one screen. Real agents, real running on one screen. Real agents, real running on one screen. Real agents, real work. This is a normal Tuesday for me. work. This is a normal Tuesday for me. work. This is a normal Tuesday for me. So, let's go back to the beginning So, let's go back to the beginning So, let's go back to the beginning because before any machine broke, I because before any machine broke, I because before any machine broke, I broke. When I started, it was just me broke. When I started, it was just me broke. When I started, it was just me and a terminal. and a terminal. and a terminal. A few tmux panes, a few agents, then a A few tmux panes, a few agents, then a A few tmux panes, a few agents, then a few more. And really fast, I'm sitting few more. And really fast, I'm sitting few more. And really fast, I'm sitting in front of four, five, six live in front of four, five, six live in front of four, five, six live contexts at the same time. And here's contexts at the same time. And here's contexts at the same time. And here's what nobody warns you about. At that what nobody warns you about. At that what nobody warns you about. At that point, I'm not running agents anymore. point, I'm not running agents anymore. point, I'm not running agents anymore. I've become the scheduler, deciding who I've become the scheduler, deciding who I've become the scheduler, deciding who does what. I'm the memory, holding what does what. I'm the memory, holding what does what. I'm the memory, holding what every one of them is doing, and I'm the every one of them is doing, and I'm the every one of them is doing, and I'm the reviewer, checking all of it. One human, reviewer, checking all of it. One human, reviewer, checking all of it. One human, three roles, six contexts. It does not three roles, six contexts. It does not three roles, six contexts. It does not scale. This scale. This scale. This This is the mess that broke me. This is the mess that broke me. This is the mess that broke me. I just couldn't hold what six agents I just couldn't hold what six agents I just couldn't hold what six agents were doing at once. My own attention was were doing at once. My own attention was were doing at once. My own attention was the bottleneck. the bottleneck. the bottleneck. So, I asked myself kind of a strange So, I asked myself kind of a strange So, I asked myself kind of a strange question. How does a handful of question. How does a handful of question. How does a handful of executives run a company of thousands of executives run a company of thousands of executives run a company of thousands of people? They don't hold all of it in people? They don't hold all of it in people? They don't hold all of it in their heads. They separate context. Each their heads. They separate context. Each their heads. They separate context. Each person only ever sees their own slice. person only ever sees their own slice. person only ever sees their own slice. And that was the unlock. What if my And that was the unlock. What if my And that was the unlock. What if my agents weren't a flat pile? What if they agents weren't a flat pile? What if they agents weren't a flat pile? What if they were an organization? were an organization? were an organization? So, that's what I built, a hierarchy, So, that's what I built, a hierarchy, So, that's what I built, a hierarchy, CEO, VP, manager, worker. And please CEO, VP, manager, worker. And please CEO, VP, manager, worker. And please hear me on this. These are real entity hear me on this. These are real entity hear me on this. These are real entity types in the system. It is not a cute types in the system. It is not a cute types in the system. It is not a cute metaphor. Each one is its own agent with metaphor. Each one is its own agent with metaphor. Each one is its own agent with its own scoped context and its own its own scoped context and its own its own scoped context and its own approval boundary. Context flows down. approval boundary. Context flows down. approval boundary. Context flows down. Each layer only gets the slice it needs. Each layer only gets the slice it needs. Each layer only gets the slice it needs. Results flow back up and I only review Results flow back up and I only review Results flow back up and I only review what reaches the very top. So, instead what reaches the very top. So, instead what reaches the very top. So, instead of holding six contexts in my head, I of holding six contexts in my head, I of holding six contexts in my head, I hold exactly one. hold exactly one. hold exactly one. Next problem, where does an agent's Next problem, where does an agent's Next problem, where does an agent's state actually live? Normally, it lives state actually live? Normally, it lives state actually live? Normally, it lives inside the model's context window and inside the model's context window and inside the model's context window and that window fills up. So, I moved it that window fills up. So, I moved it that window fills up. So, I moved it out. Every entity gets its own workspace out. Every entity gets its own workspace out. Every entity gets its own workspace on disk. Shared context that everyone on disk. Shared context that everyone on disk. Shared context that everyone has to honor lives in shared. has to honor lives in shared. has to honor lives in shared. Machine-bound state lives under Machine-bound state lives under Machine-bound state lives under machines. And inside each workspace, the machines. And inside each workspace, the machines. And inside each workspace, the mission, the current status, and a mission, the current status, and a mission, the current status, and a handoff folder, the actual work product handoff folder, the actual work product handoff folder, the actual work product that gets passed along. The state lives that gets passed along. The state lives that gets passed along. The state lives in files. It is not trapped inside one in files. It is not trapped inside one in files. It is not trapped inside one model. model. model. And this is the single most practical And this is the single most practical And this is the single most practical thing I learned all year. When that thing I learned all year. When that thing I learned all year. When that window fills up, the built-in move is to window fills up, the built-in move is to window fills up, the built-in move is to compact, summarize the history, make compact, summarize the history, make compact, summarize the history, make room. I stopped doing it. It's slow. I room. I stopped doing it. It's slow. I room. I stopped doing it. It's slow. I can't choose what survives. And whatever can't choose what survives. And whatever can't choose what survives. And whatever it throws away is just gone. So, it throws away is just gone. So, it throws away is just gone. So, instead, I don't compact, I reset. And instead, I don't compact, I reset. And instead, I don't compact, I reset. And by reset, I mean right inside Claude, I by reset, I mean right inside Claude, I by reset, I mean right inside Claude, I clear the context completely. And then clear the context completely. And then clear the context completely. And then it just reads back the handoff and the it just reads back the handoff and the it just reads back the handoff and the history files it wrote for itself and history files it wrote for itself and history files it wrote for itself and picks up exactly where it left off. The picks up exactly where it left off. The picks up exactly where it left off. The context can get wiped, the machine can context can get wiped, the machine can context can get wiped, the machine can even crash, and the work still survives even crash, and the work still survives even crash, and the work still survives because it was never only in the model. because it was never only in the model. because it was never only in the model. But there was still one problem, But there was still one problem, But there was still one problem, alignment. Plans flow down the alignment. Plans flow down the alignment. Plans flow down the hierarchy, but they drift. And I'd only hierarchy, but they drift. And I'd only hierarchy, but they drift. And I'd only catch it by walking into each pane by catch it by walking into each pane by catch it by walking into each pane by hand. So, I built a review gateway. Any hand. So, I built a review gateway. Any hand. So, I built a review gateway. Any layer that wants to act submits its plan layer that wants to act submits its plan layer that wants to act submits its plan and then it blocks. It waits. Nothing and then it blocks. It waits. Nothing and then it blocks. It waits. Nothing runs until I approve. And the second I runs until I approve. And the second I runs until I approve. And the second I approve, a hook fires the work off approve, a hook fires the work off approve, a hook fires the work off automatically. One web inbox, one automatically. One web inbox, one automatically. One web inbox, one control point, I never walk into the control point, I never walk into the control point, I never walk into the work windows anymore. work windows anymore. work windows anymore. Here's one full cycle. A plan comes in, Here's one full cycle. A plan comes in, Here's one full cycle. A plan comes in, I approve, the work resumes. That's the I approve, the work resumes. That's the I approve, the work resumes. That's the whole loop. And honestly, I didn't build whole loop. And honestly, I didn't build whole loop. And honestly, I didn't build this gateway by hand. An infra team this gateway by hand. An infra team this gateway by hand. An infra team inside the fleet built it. Agents inside the fleet built it. Agents inside the fleet built it. Agents building the tools that run the agents. building the tools that run the agents. building the tools that run the agents. I think that's kind of the whole point. I think that's kind of the whole point. I think that's kind of the whole point. So on one machine, all of this works So on one machine, all of this works So on one machine, all of this works beautifully. And then it didn't. Then beautifully. And then it didn't. Then beautifully. And then it didn't. Then one machine wasn't enough. one machine wasn't enough. one machine wasn't enough. Five things broke. Failure. One, my Five things broke. Failure. One, my Five things broke. Failure. One, my agents kept doing the work themselves agents kept doing the work themselves agents kept doing the work themselves instead of dispatching it down to a instead of dispatching it down to a instead of dispatching it down to a worker. The orchestrator is supposed to worker. The orchestrator is supposed to worker. The orchestrator is supposed to delegate. Instead, it just rolls up its delegate. Instead, it just rolls up its delegate. Instead, it just rolls up its sleeves and does the task itself. Wrong. sleeves and does the task itself. Wrong. sleeves and does the task itself. Wrong. So I forced its hand. A CLI harness with So I forced its hand. A CLI harness with So I forced its hand. A CLI harness with skills that call those CLIs. So skills that call those CLIs. So skills that call those CLIs. So dispatching becomes the only path it can dispatching becomes the only path it can dispatching becomes the only path it can take. take. take. Failure two, and this one's almost Failure two, and this one's almost Failure two, and this one's almost funny. As I kept throwing more tasks at funny. As I kept throwing more tasks at funny. As I kept throwing more tasks at my managers, they kept spinning up more my managers, they kept spinning up more my managers, they kept spinning up more and more worker panes inside a single and more worker panes inside a single and more worker panes inside a single window until it was so crowded the panes window until it was so crowded the panes window until it was so crowded the panes were too small to read. Even tmux were too small to read. Even tmux were too small to read. Even tmux capture pane, the thing I used to read a capture pane, the thing I used to read a capture pane, the thing I used to read a pane programmatically, couldn't pull pane programmatically, couldn't pull pane programmatically, couldn't pull anything meaningful out of them anymore. anything meaningful out of them anymore. anything meaningful out of them anymore. I'd literally run out of room to see I'd literally run out of room to see I'd literally run out of room to see what my own fleet was doing. what my own fleet was doing. what my own fleet was doing. Failure three, out of memory. Look at Failure three, out of memory. Look at Failure three, out of memory. Look at this. That's my activity monitor, this. That's my activity monitor, this. That's my activity monitor, completely buried under cloud code and completely buried under cloud code and completely buried under cloud code and MCP processes. Swap almost full. MCP processes. Swap almost full. MCP processes. Swap almost full. Basically no free memory left. The Basically no free memory left. The Basically no free memory left. The sessions just stacked up until the sessions just stacked up until the sessions just stacked up until the machine couldn't breathe. machine couldn't breathe. machine couldn't breathe. Failure four, get credentials. The Failure four, get credentials. The Failure four, get credentials. The expectation is clean. Credential A to expectation is clean. Credential A to expectation is clean. Credential A to workspace A, credential B to workspace workspace A, credential B to workspace workspace A, credential B to workspace B, one to one. The reality, they B, one to one. The reality, they B, one to one. The reality, they collided, crossed over, bound to the collided, crossed over, bound to the collided, crossed over, bound to the wrong workspaces. The fix was a clean, wrong workspaces. The fix was a clean, wrong workspaces. The fix was a clean, fully separated environment for each fully separated environment for each fully separated environment for each workspace. workspace. workspace. And failure five, the one that really And failure five, the one that really And failure five, the one that really hurt. The MacBook is a laptop, so the hurt. The MacBook is a laptop, so the hurt. The MacBook is a laptop, so the moment it loses power or drops off the moment it loses power or drops off the moment it loses power or drops off the network, every in progress job dies with network, every in progress job dies with network, every in progress job dies with it. And one time I'd pushed so many it. And one time I'd pushed so many it. And one time I'd pushed so many workers onto it that the whole machine workers onto it that the whole machine workers onto it that the whole machine just gave out. I came back later, opened just gave out. I came back later, opened just gave out. I came back later, opened the lid, and it had already restarted the lid, and it had already restarted the lid, and it had already restarted itself. Everything in flight just gone. itself. Everything in flight just gone. itself. Everything in flight just gone. So, the very first thing I did was build So, the very first thing I did was build So, the very first thing I did was build a boot command, one overlord boot, and a boot command, one overlord boot, and a boot command, one overlord boot, and the whole fleet comes straight back up the whole fleet comes straight back up the whole fleet comes straight back up because all the state was sitting in because all the state was sitting in because all the state was sitting in files. But a machine that can just files. But a machine that can just files. But a machine that can just vanish on you like that, that's when I vanish on you like that, that's when I vanish on you like that, that's when I knew one machine wasn't going to cut it. knew one machine wasn't going to cut it. knew one machine wasn't going to cut it. Every single one of these pointed at the Every single one of these pointed at the Every single one of these pointed at the same answer. So, I added machines. same answer. So, I added machines. same answer. So, I added machines. Here's the split. The MacBook was Here's the split. The MacBook was Here's the split. The MacBook was carrying everything, heavy coding and my carrying everything, heavy coding and my carrying everything, heavy coding and my personal projects. So, I offloaded it. personal projects. So, I offloaded it. personal projects. So, I offloaded it. Long-running coding tasks went to Linux Long-running coding tasks went to Linux Long-running coding tasks went to Linux A, short-lived personal projects went to A, short-lived personal projects went to A, short-lived personal projects went to Linux B, both always on. More resources, Linux B, both always on. More resources, Linux B, both always on. More resources, the long-running work finally off the the long-running work finally off the the long-running work finally off the laptop, and still one control plane over laptop, and still one control plane over laptop, and still one control plane over all of it. all of it. all of it. But now I've got context on one machine But now I've got context on one machine But now I've got context on one machine that I need on another. How do I move that I need on another. How do I move that I need on another. How do I move it? Git. I commit the context files and it? Git. I commit the context files and it? Git. I commit the context files and push. Then I poke the other machine with push. Then I poke the other machine with push. Then I poke the other machine with tmux send-keys over SSH and tell it to tmux send-keys over SSH and tell it to tmux send-keys over SSH and tell it to pull. It pulls, the agent there reads pull. It pulls, the agent there reads pull. It pulls, the agent there reads the files, and picks up exactly where the files, and picks up exactly where the files, and picks up exactly where the first one left off. the first one left off. the first one left off. Of course, it wasn't that clean. When Of course, it wasn't that clean. When Of course, it wasn't that clean. When two machines point at the same directory two machines point at the same directory two machines point at the same directory and both change it, you get conflicts. and both change it, you get conflicts. and both change it, you get conflicts. The same context quietly diverged, so I The same context quietly diverged, so I The same context quietly diverged, so I separated it. Per-machine directories separated it. Per-machine directories separated it. Per-machine directories for machine-specific state, and the for machine-specific state, and the for machine-specific state, and the shared stuff only changes through a pull shared stuff only changes through a pull shared stuff only changes through a pull request. It's boring, but boring is what request. It's boring, but boring is what request. It's boring, but boring is what stops the two machines from silently stops the two machines from silently stops the two machines from silently disagreeing. disagreeing. disagreeing. Then the gateway came back to bite me. I Then the gateway came back to bite me. I Then the gateway came back to bite me. I now had one on every machine, so I was now had one on every machine, so I was now had one on every machine, so I was checking multiple inboxes again. So, I checking multiple inboxes again. So, I checking multiple inboxes again. So, I collapsed them. Every machine sends its collapsed them. Every machine sends its collapsed them. Every machine sends its review requests over SSH into one main review requests over SSH into one main review requests over SSH into one main gateway, and that main one lives on an gateway, and that main one lives on an gateway, and that main one lives on an always-on Linux box because, remember, always-on Linux box because, remember, always-on Linux box because, remember, the Mac sleeps. Your one point of the Mac sleeps. Your one point of the Mac sleeps. Your one point of control can't be a thing that falls control can't be a thing that falls control can't be a thing that falls asleep. asleep. asleep. And then, my absolute favorite failure And then, my absolute favorite failure And then, my absolute favorite failure of the whole project. I was developing of the whole project. I was developing of the whole project. I was developing across all of these, bouncing back and across all of these, bouncing back and across all of these, bouncing back and forth, and at one point I went looking forth, and at one point I went looking forth, and at one point I went looking for a feature I'd built, and I genuinely for a feature I'd built, and I genuinely for a feature I'd built, and I genuinely could not remember which machine I'd could not remember which machine I'd could not remember which machine I'd built it on. built it on. built it on. And that's when it clicked. I need one And that's when it clicked. I need one And that's when it clicked. I need one place to root from. So, Discord became place to root from. So, Discord became place to root from. So, Discord became my single router. One bot per machine. my single router. One bot per machine. my single router. One bot per machine. Mac, Linux A, Linux B. My phone is now Mac, Linux A, Linux B. My phone is now Mac, Linux A, Linux B. My phone is now the remote control for the entire fleet. the remote control for the entire fleet. the remote control for the entire fleet. So, where am I now? Honestly, with a So, where am I now? Honestly, with a So, where am I now? Honestly, with a pile of things I have not solved. Four pile of things I have not solved. Four pile of things I have not solved. Four of them. Consistency across machines, of them. Consistency across machines, of them. Consistency across machines, abstracting away the local only tools abstracting away the local only tools abstracting away the local only tools that are still stuck on my Mac, the MCP that are still stuck on my Mac, the MCP that are still stuck on my Mac, the MCP servers, the browser, secure credential servers, the browser, secure credential servers, the browser, secure credential handoff between instances, and resource handoff between instances, and resource handoff between instances, and resource management. I really want to stop being management. I really want to stop being management. I really want to stop being the one who decides what runs where. the one who decides what runs where. the one who decides what runs where. And when I lined those four up, I And when I lined those four up, I And when I lined those four up, I realized these aren't new questions at realized these aren't new questions at realized these aren't new questions at all. An agent should just declare what all. An agent should just declare what all. An agent should just declare what it needs, not where it runs. Above it it needs, not where it runs. Above it it needs, not where it runs. Above it sits the orchestrator, the review gate, sits the orchestrator, the review gate, sits the orchestrator, the review gate, the logical hierarchy. Below it, the logical hierarchy. Below it, the logical hierarchy. Below it, compute, secrets, tools. A scheduler compute, secrets, tools. A scheduler compute, secrets, tools. A scheduler places it, and the machines just places it, and the machines just places it, and the machines just disappear underneath. These are the disappear underneath. These are the disappear underneath. These are the exact questions Kubernetes already exact questions Kubernetes already exact questions Kubernetes already answers. answers. answers. So, that's where I'm headed. I'm not So, that's where I'm headed. I'm not So, that's where I'm headed. I'm not going to reinvent compute, secrets, and going to reinvent compute, secrets, and going to reinvent compute, secrets, and tools. Kubernetes already nailed those. tools. Kubernetes already nailed those. tools. Kubernetes already nailed those. I'm stacking them underneath, and I'm stacking them underneath, and I'm stacking them underneath, and building my orchestration manager on building my orchestration manager on building my orchestration manager on top. Task orchestration, review flow, top. Task orchestration, review flow, top. Task orchestration, review flow, context management. Reuse what exists, context management. Reuse what exists, context management. Reuse what exists, build the new part on tops. build the new part on tops. build the new part on tops. And that's the honest state of it. One And that's the honest state of it. One And that's the honest state of it. One machine, I solved. Across machines, machine, I solved. Across machines, machine, I solved. Across machines, still rough, still building. If you're still rough, still building. If you're still rough, still building. If you're running agents at any kind of scale, I'd running agents at any kind of scale, I'd running agents at any kind of scale, I'd genuinely love to compare notes, because genuinely love to compare notes, because genuinely love to compare notes, because this is the part nobody has figured out this is the part nobody has figured out this is the part nobody has figured out yet. I'm on LinkedIn and on X. Links are yet. I'm on LinkedIn and on X. Links are yet. I'm on LinkedIn and on X. Links are in the description. Thank you so much.