--- title: "Transcript: AI Engineer World's Fair: Building Reelful - Agentic Video Editor" category: "transcripts" videoId: "AheG9p_JXVw" sourceLabels: ["External YouTube secondary-source transcript", "Cached transcript markdown"] wordCount: "1340" --- # Transcript: AI Engineer World's Fair: Building Reelful - Agentic Video Editor ## Source Video - [YouTube](https://www.youtube.com/watch?v=AheG9p_JXVw) ## Local Cache - `raw/sources/external-youtube-transcripts/AheG9p_JXVw.txt` - 1,340 words ## Transcript Hi, we're building real and we're trying to tackle agenda video editing problem from video editing problem from the agent standpoint. I'm K. I'm founder and co. But let's first talk about what's agentic video editing is. So as a user you just drop in your media photos and videos and provide some context. It can be uh the context what happened in this media part or it can be some directions for example like add captions, add music, add voice over and something like that and then the agent will go understand your media uh find the right moments assemble everything together generate captions, music, voice over B-rolls uh and give you a ready to share clip uh And um yeah so basically um that's uh video agentic video editing. So agent does everything by itself or another example do recorded uh a speak to camera video and have you have a lot of pauses unsuccessful shots and you expect an agent to figure it out to remove unsuccessful shots. Remove pauses and give you a ready to share with you. Um and uh the the interesting thing is that a lot of actually a lot of the things inside this pipeline can be automated and this is exactly what we're doing at this is um the example of uh bigger engine and um since we are at AI engineering conference I wanted to talk a little bit about infrastructure and from the infrastructure standpoint. Aentic uh video editor is very similar to aentic app builder. Uh sorry there is a title on the slide. So the second column is aentic video editor. So both of them have a prompt uh a component uh and in the video editor case is a media plus prompt and usually on back end what's happening there is a remote machine which is called sandbox uh which spinning up and inside this machine there is an agent with tools and skills which is working on uh what you're you're asking it to do. Uh in the case of the agentic uh app builder is a code base. Uh in the case of the agentic video editor, it's a video video composition and as a result in the agent builder user get an app preview and for the video editor users user get rendered a rendered video and but yes the the infra from infrastructural standpoint it's pretty similar but there are a couple of differences. Uh and uh this is actually most interesting to me generating versus editing. At real pool, we are focusing on editing real footage. So, we do not generate a lot of content. We are expecting you to provide your real life, your um personal content and we will edit it for you. And actually, this is uh a more complex problem because if the agent has a blank blank canvas, it can do whatever they have. But in the editing case, the agent has to figure out uh which moments are best uh what to what to use, how to organize everything together. And also um sometimes footage can be um messy or incomplete and agent still has to deliver a very polished result professional way so that ideally the viewers of this content don't get into this like AI or uh human edited. So let's actually have a look how we do it uh at real. So we start as I already mentioned with your media plus account some directions like how you want it to be edited and we need to get a polished clip. So let's go through it step by step. So we are doing first media understanding. We need to understand what's in what's actually happening uh on those clips and photos and we also need to transcribe speech. For example, in the case if you have speed to camera videos, then we are providing a creative plan for the users so that they can approve if they like it or not, what they want to change or maybe regenerate uh and we create this plan before actually starting editing. Once the user approved this plan uh we spin up a sandbox the remote uh remote machine that we already discussed and this is an environment for the agent to uh execute everything. So the agent comes with the skills and in our case in in the case of uh video editing our skills are for example cultures how for example how to select the best moments uh also funes which fonts are uh more suitable for this use case which are not for example how to generate beles and this is where taste and craft uh actually um and then also Asian uh can um can initiate some other subprocesses. For example, generating music that will fit this exact composition. Generating voice over, adding sounds, animating images. Yes, this is actually what we do. If you provide photos, we can animate your photos to make them more uh dynamic and engaging. And then comes remote composition. So here a little bit of background. What's remote motion? Remotion is a framework open source open source framework uh to create videos as code as react code. Uh so basically it's just like a a file with the order with all your assets and tracks and how they're following each other. And um why it is important because uh agents are really good at writing code and therefore we can use them to create videos with this promotion framework. And then uh the last thing is the verification layer. Of course, agents can make mistakes and that's why we developed this verification layer to make sure that all the um the composition is clean is well defined everything will be rendered and if there there are some problems then the agent will reiterate on the composition and this is how we got to a polished clip. So it's a lot right? Uh it's like a very complex workflow. Uh and ideally we don't want our users to even know anything about it. And this is even maybe bigger problem how to deliver this complex agent workflow to mass consumer. And this is how we're tackling that at real. So we decided to go mobile first so that users can edit videos while driving, walking or maybe reading ways. Uh also I know that prompting videos can sometimes be also challenging. That's why we create directional templates. For example, stick to camera videos or maybe you want to add B- rolls or voice over so that users can just select these directional templates, drop their media and that's it. Even without any prompt, it will it will work. And the third thing is a building editor. Uh why? Because we want to make this experience convenient and familiar for users. So a lot of people are already sort of uh using uh regular video editors and that's why we want to provide this experience as well. So how it works? User first generates a video pageentically but if they want to tweak it for example remove a second or maybe uh correct some word in the captions they can go with an editor and edit it a little bit. Um yeah andh actually I have a couple of examples here that I recently created with Google. Uh I will play them just once. Sorry. Do you hear a little bit? Okay. You just enjoy the So uh yeah basically all these videos they were assembled only using agent no uh regular video editor and I'm already posting them on social media And yeah, it I I have a lot of fun with that. And oh, sorry. Uh and exclusively for this conference, we are uh giving our beta uh which is our new new second version. Uh please give it a try. Uh and uh let me know if you have any feedback. Uh here is my uh email. Please uh feel free to reach out.