Markdown source

Dense Slides: Arrakis: How To Build An AI Sandbox From Scratch - Abhishek Bhardwaj, OpenAI

Source Video

Arrakis: How To Build An AI Sandbox From Scratch - Abhishek Bhardwaj, OpenAI

Method

This deck is slide-only. The existing captured video frame set supplies candidate frames, then local OpenCV rejects sponsor/title/speaker-only frames, crops visible slide surfaces, deduplicates, and saves the cropped slide images.

Cropped Visible Slides

slide-001.jpg

Slide text:

Background

Carnegie Mellon

Operating Systems x Distributed Systems.-.

8 Microsoft

Windows Subsystem for Linux

Google - G From scratch Os for pre-Apple Watch Smartwatch:

:.: MlcroVMs - Rust-based ightweight VMs for Linux dev

environments on Chrome Os.

ARQ- Android apps on Chrome Os In containers: Replit

Infra - Proxied through my servkce for your entire session

Arrakis.. RAG based codo chat: whenever you code on Replit:

Sandboxes for Al Agents - code execution and computer use

8Built during my time at South Park Commons

slide-002.jpg

Slide text:

Why Do We Need AI Sandboxes?

Smarter Models

Ability to call tools during inference

Used in RL training

Smarter Agents

Can debug actions by running Linux commands "ps", "lsof"

Backtrack, replan and work towards the goal with a Sandbox at their disposal

Security

Agent code is no different that Github or StackOverflow code running on your host or production server

slide-003.jpg

Slide text:

. How to Create a ChatGPT Clone. Manus's Computer '

Nuple Guts 1 truty

Executng comrind kdlr' -p thutgot ctare t cd chatgot_clone t& create_tlsik.+p-

Create a ChatGPT clone sheni.

: interface that mimics the functionaity of ChatGPT. Let me get started right: &manus: 1'll hclp you croate a ChatGpT clono. Ill work on doveloping a wob-based chat: _clone &s create_rlask_app. USAGE: ubuntugsandbox: S'cd /hone/ubuntu &s akdir -p chatgpt clone &s cd chatgpt. Creates a nev Flask application' create_flask app fapp_nase]

&manus AeMe Oi Select the most suitable web spplicauon template for a ChatGpT clone Connocted to datasource(1): ubuntuesondbox:~/chatgpt_clone. $. ARGUHENTS: EXAMPLEi. butu'dde create_flosk app ay_app Napc of. the Ftask application to 'create

1'1l be creating a ChatGPT clone using Flask, vhich is the appropriate choice

since we noed both s frontend Intcrface and backend processing for the chat: functionality. Let me set up the project structure and lmpiement the necessary.

. components.

) Creite Ue ChatGPT clone project using tha Flask terplatoA

: Creating the ChaiGPT clone prol+ct using tha Fusk templto.

Creating thie ChatGPT clone Flask project.

0 Using termnat:

slide-004.jpg

Slide text:

Arrakis

Arrakis provides a secure, fully customizable, and self-hosted solution to spawn and manage Sandboxes for code execution and computer use.

It has out-of-the-box support for backtracking via snapshot-and-restore.

Open Source

slide-005.jpg

Slide text:

D

Backtracking

. Pre-reasoning models - Intuition that a higher order task needs to be broken down by a Planner:

Agent should be able to snapshot progress and restart from last good snapshot versus starting from scratch

.Simple AP!

Python Api Golng Clt Micp Sarver opearbpi compatbio -+ gonerate cionts in sny Languogo

Configurable With Docker tooling'

Dockerfile to configure what runs nside tha Sanabox:

slide-006.jpg

Slide text:

arrakis-py-sdk _. REST API Linux Server

Arrakis Restserver

arrakis-client REST API:

vWc client Port Forrar. Auto Arrakis MicroVM DA: Sandbox Server Coda Server ws +poy.. Arrakis MicroVM Sandbox YAC' trver ort Foru Auto

Userspace

Kerne! Idxv/kva" /dev/kva

High level architecture of Arrakis

slide-007.jpg

Slide text:

Arrakis Restserver exposes the following REST APl. This is also available via a Python SDK, MCP Server and an openapi compliant YAML file

API-Sandboxes,Snapshots,Command Execution,Files

r.HandleFunc("/"+API VERSION+"/vms",s.startVM).Methods("POST")

r.HandleFunc("/"+API_VERSION+"/vms/{name}",s.updateVMState).Methods("PATCH")

r.HandleFunc("/"+API VERSION+"/vms/{name}",s.destroyVM).Methods("DELETE")

r.HandleFunc("/"+APIVERSION+"/vms",S.destroyALlVMs).Methods("DELETE")

r.HandleFunc("/"+API VERSION+"/vms",s.listAlLVMs).Methods("GET")

r.HandleFunc("/"+API_VERSION+"/vms/(name}",s.listVM).Methods("GET")

HandleFunc("/"+API_VERSION+"/vms/(name}/snapshots",s.snapshotVM).Methods("POST")

ndleFunc("/"+API_VERSION+"/vms/{name}/cmd",s.vmCommand).Methods（"POST")

dleFunc("/"+API VERSION+"/vms/{name}/files",s.vmFileUpload).Methods("POST")

dleFunc("/"+API_VERSION+"/vms/{name}/files",s.vmFileDownload).Methods("GET")

andleFunc("/"+API VERSION+"/health",s.healthCheck).Methods("GET")

slide-008.jpg

Slide text:

Arrakis - API

arrakis-py-sdk

arrakis-client

Arrakis Restserver

slide-009.jpg

Slide text:

Arrakis: A Quick Tour Of Linux Sandboxing

Linux Execution Modei

A Thread ls the smabest unn of execution. Each Thead has a task struct that's n the Scheduier run queue. Process" conssts of matiple thyeads, Treads" In a process have differeni T..

Why? - My app necos Python38, foo b version 12, your server has Python 3.9, 36, 312 and doosn't have foo Rd. Packago sn opp's dependancios along wth ks coro business logic Run, arbarsr.. Containers: Why? And What?: 全

Containers: Security Story

After al Containers run as native processos on iop of tha kernel Any malcious or buggy procest can attack the kemel and gain root Once t becomes root it can do'sriythirg d want.

Containers: Sandboxing

:JaT containers by rostricting CAPabitos and syscias they can cal Reduce attack surface Con rostrict thg anack surtace by restricting the Unux CAPabltios atowed to a process such as..

Containers Virtualization.

VintusHzation provdes another primaivo to run arbitrary code on your mochlnos Esch Vu has rts own guest user space and guest kerme, Doesn't drocty stsck the host kern

Kvm - Hypevisor In the form of a kernet mnodule that leverages VT-x to run swich CPU context and run VMs Vmu - Taks to kvm to spawn VMs and aso manaes emtuated Linux Virtualization

slide-010.jpg

Slide text:

Process 1 Process.2 08x0 4u! xor. ebx, ebx

IThread MThread

ioctl: Syscall Userspace

/dev/kvm: /dev/sda1 Kernel (os)

Execution model of a typical Linux System

: A*Trread' Is the snaBost una of erecuton. Each -Thread' has a task struet that's n thio Schodutor run quoue..

"Processe consists of mutupie thresds. -Thresds" n process huve different TDs but tame Pio, thare page tablo snd other th'ngs. Bound by shared resources Kermel provides prirtegeo' access to hardware. lf not gated, buggy or masclous code can wreak harroc.

Specisal nstructions to switeh to kernel mode / supervisor mode and lnvoke a syscal for prvteged sccess.:

slide-011.jpg

Slide text:

Containers: Why? And What?

slide-012.jpg

Slide text:

100

Mount Namespace Pid Pid:101 Pid 102

/foo:Userspace

Kerne!.

Linux Container with Pid and Mount Namespace

Why? - My app needs Pyhon3.8, foo tb verslon 1.2, your server has Python 3.9, 3.6, 3.12 and doesnt have foo Eb.

ckage an spp' deperdenciet blong with itt core business logic:

On Linior' a Contairer is = Coatctoni of Nomespoces of diferent resources (Process, Mourit, Net etc.). Rur arbltrry uso code on your machhne nsixe a contalner you only ie resources hn your nrmespsce snd not ounside.

Resource siocation hsppens via CGroups ("how much memory or Cpu to abocate ths namespoce??) Outide tho contther you soe ovoryhing insldo your chldren ramespacos?

slide-013.jpg

Slide text:

Container Pid Namespace Pid 1 Parent Root Namespaces

Pid 100 P!d

Pid 2 Mount Wamespace Pid 3 Pid 101 102 Pid

/foo Userspace

Kernel

Linux Container with Pid and Mount Wamespace

slide-014.jpg

Slide text:

Mount Namespace Pid 2 Pid. "Pid 101 P!d. 102

/foo.:Userspace

Kernel.

Linux Container with Pid and Mount Namespace'

Ipp nhods Python3.8, foo Eb vorslon 1.2, your sorrer has Python 3.9, 3.6, 3.12 and dootn't havd foo ib.:

kage an app's dependercles along wth hs core business logic:

Run ar bltrary user code on your machine. On Lnunt s Container ks a Colecton of Nsmespsces of differont resources (Process, Mourit, Not otc.).

nside a contahner you only see resources n jour namespace and not ouside..

Cucide the contener you see eveythhg nslde your chldren namespuces

Rtsource alocaicn hoppens via CGroups ('how much rmemory or CPu to sbocire thhs namespace?)

.$..

Contminenk Securkty Story

slide-015.jpg

Slide text:

Containers: Security Story

Root Namespace Userspace

Container

Malicious or Buggy Process Pid Namespace Pid 100 Trusted Process:Pid: 200

:'syscall ioctl Attack Surface Syscall: ioctl

Kernel:

Container processes can attack the kernel

Ater s Containers run ss nativo procosses on top ot thg kernel:

.: Ary mallclous or buggy procass can attack the kerre and galn root..

slide-016.jpg

Slide text:

Container Root Namespace Userspace

Malicious or Buggy Process Pid Namespace: Pid 100 Trusted Process Pid: 200

syscall ioctl Attack Surface Syscall. ioct!.

Kernel

Container processes can attack the kernel

Ater s Containers run ss nstlve processes on top of the kernel'.

Ary malclous cr buggy process can atsck the kornel and gain rool.: 、

+. Onco it becormes root k can do arything r warts..

slide-017.jpg

Slide text:

Arrakis: A Quick Tour Of Linux Sandboxing:

Unax Execution Model

Containerte: Why? And What?'

AinmCotL

Containerr: Sardbadng

ora tuaai l iruao t

Contsinert - Vrtunzston

Uinn Vrtumizadon ：：

K·hype

Horestit morigred by fim turta vytes Shocat te dtir by yot: Orty mo Mleroyhs vs Traditoru Vms

slide-018.jpg

Slide text:

ontainers: Sandboxing

: Joie containers by rosricting CAPsbuities and syscalls thay can cat

Roduco attock surfsco

Userspace

Root Namespace:

Malicious or Buggy Process Container # minijail0 -s /usr/share/minijail0/$(uname -m)/cat.policy -\ /bin/cat /proc/self/seccomp_filter

Pid Namespace Syscall ioctl mmap2: arg2. =a 0x0 munmap: 1 8h07 == 1640 11 9182 == 1640 11 0 == 16xo 11 89172 == 1640 u2do close: 1 # open with O LARGEFILEjO_RDONLYlO_NONBLOcK or some combination.

Reduce attack surface by using seccomp filters'

Con resirfct the stuock surface by restrkcting tho Lirunc CAPsbiltos sowed to a procos such as CAP_NET, CAP.SYS_ ADMIN etc.

slide-019.jpg

Slide text:

Userspace

Root Namespace

Malicious'or Buggy Process Container # minijailo -S /usr/share/minijailo/$(uname -m)/cat.policy --\ /bin/cat /eroc/self/seccomp_filter

Pid Namespace: Syscall. P!d ioctl. mmap2:arg2 == 0x0 open: arg1 == 32768 Il arg1 == 0 Il arg1 == 34816 Il arg1 == 2048 L:dowunm # open with O_LARGEFILEIO_RDONLYlO_NOvBLOck or some combination. close: 1

:Kernel.

Reduce attack surface by using seccomp filters

Can restrlct thie attack surface by restrictlng the Lirux CAPabaltes atowed to a process such ss CAP_NET, CAP_SYS_ADMN etc.

: These controt whst syscots a process cani call:: Slmlarty vre can uso seccomp frers to rostict what syscats a procoss can coll

'Minleil' developed by ChrcmeOs provdes an easy lrerface lo do boh.

slide-020.jpg

Slide text:

Containers → Virtualization

slide-021.jpg

Slide text:

Virtual Machine P2 Host Userspace Virtual Machine Monitor (vMA) (qenu / crosvm / firecracker) Create VM Host: Some chent On:

Guest Userspace" P3. running.: the vM Thread:

virtio

net driver. block driver Block Device Net Device'

Guest Kernel. VMRESUAE VMEXIT

Hypervisor /dev/kvn Host Kernel

Linux Virtualization: Key Components.

:: Kym - Hypervisor in the form of a keme module that haveraget VT-x to run tirach CPU contoxt and run VM!..

VMm - Tatka to kvm to spaum Vmt and atso riansges omubhed davicos that servo VM requoss.

slide-022.jpg

Slide text:

P2 frtual Machine Monitor (VMM) / crosvm / firecracker) Create VM

Guest Userspace: P3: the VM running Thread

net driver: block driver Bock Device Net Device

. Guest Kernel VNRESUAE VMEXIT

: Hypervisor.: /dev/kvm.... Host Kernel.

Linux Virtualization: Key Components.

. To chents VhM tread Is blocked snd rns as *guest VMr, When VM needs soring ft eouts cut via VMEXITS and se sevced snd VAREsUMED. Vum - Tatks to kum io sparn Vms and atso manages erutated devices that serve VM requests Ksm - Hypervisor in the form of a kenol modulo thiat hrverages VT-x to run swich CpU context and rn VMs':

VMEXITS and VhRESUMES are exptrsre, W/e thould try to rminlmlre them

slide-023.jpg

Slide text:

MicroVMs vs Traditional VMs

HEcrovMs powerod by Rust based VhMs

Sandboxing Emulated Devices.

[emu; Jailed Architecture crosvm

Block Net

Net'

1

Seccomg Seccow

Aou Aq nso eg pnous

: 1. Onty msjor archtecturos and emulated devicos supported

Loss code paths -) Boots Fast.

:.

slide-024.jpg

Slide text:

MicroyMs powered by Rust based VhiMs

Sandboxing Emulated Devices:

: qemu: Jailed Architecture crosvm

Block Net

Block Net:

Mimi Seccomp Minijail. (seccomp)

: Shouid bo cloor by now' *

Onty major archhectures snd emulated derces supported

2. Less blost -> smaber memory footprint vs trscions! VhuMs. 3. Microvu' - Roflocton of the VuM rathor than the running guost, Less code paths -> Boots Fast

slide-025.jpg

Slide text:

sowg xooq itey' -': Fast snapshots - Just cump the entire guest memory': -': cioud-hypervsor isn't at the wti'm of Amszon's Frecracker.. Securty as the flrst parameter since mukd-tenant code eoxiecution in m'nd o Had hotphugging and snapshot restora when Arakds started

gVhscr possbty a good in-betwaen cption betweon Contairers and MleroyMs.

Bared on the stcurty gueranees noeded:--、：

Base Project crosvm Desktop focused

Fork Fork

firecracker cloud-hypervisor.

- Serverless loads: - VM Manager daemon - AwS Lambda out of the box. -. Hotplugging devices -. More general Huoddns Ndy - - Snapshot support purpose enterprise VMM

slide-026.jpg

Slide text:

Arrakis MicroVM Runtime

Linux Server

arrakis-py-sdk REST API Arrakis Restserver

REST API

arrakis-client cloud-hypervisor process cloud-hypervisor process

VNC Server Code Server VNC Server Code Server

Arrakis MicroVM Sandbox Arrakis MicroVM Sandbox

Userspace

Kernel /dev/kve /dev/kvm

slide-027.jpg

Slide text:

Arrakis: Storage Using OverlayFS

OverlayFs To. Protect The Rootfs Of Each Sandbox.

..Arakis Sandbox. Is /usr/bin' Arrakis Sandbox' Is /usr/bin

Uerspace Userspact

r overlayfs /usr/in/k riv overloryfs. Tusr/bin/e..

ro rootfs /ar/in/a: ro rcotfs Tusr/bin/a

Kerre! Kerre!

: Shared Rootfs ext9 Lnage

Each sandbox with it's own Rw layer: and a shared RO layer.

slide-028.jpg

Slide text:

niasabeigis:siyr

Assoon asweboot a sandbox,"initinsidesetsup the overlayfs as detailedbelow.

/bin/busybox mkdir -p $[NEWROOT} /bin/busybox mkdir-p ${woRk} echo"Creating upper and work directories in SwRITABLE Rw" /bin/busybox mkdir -p ${UPPER} /bin/busybox mkdir -p ${NEWROOT_LOWER_RO}

/bin/busyboxmkdir-p ${NEWROOT_WRITABLE_RW]

echo“Mountingoverlayfsto $NEWRooT" SINEWROOT} if[$？-neO ];then /bin/busybox mount-t overlay overlay -o lowerdir=${LOwER_RO},upperdir=${UPPER},workdir=$[wORK} exec /bin/busybox sh# Drop to shell for debugging echo"Error mounting overlayfs!"

else return1 echo "Overlayfs mounted successfully."

fi

slide-029.jpg

Slide text:

vNc Server: rans apoy: vvc Server Code Server.

Arrakis MicroVM Sandbox: Arrakis MicroVM Sandbox:

Cloud-Hypervisor Process cloud-Hypervisor Process:

Disk Device

Per Va Rw. Layer Per vn Rw Layer

Userspace Base Read-Only Rootfs

Kernel. /dev/kvn:: /dev/vm

slide-030.jpg

Slide text:

Arrakis: Networking

1. Tap Devices: Each sandbox gets a unique tap network device

2. Linux Bridge: All tap devices are connected to a Linux bridge on the host

3. Port Forwarding: Automatic port forwarding from the host to the sandbox

slide-031.jpg

Slide text:

description string,portForwardDesc string)（portForward,error)

hostPort, err:= s.portAllocator.AllocatePort()

cmd:= exec.Command（ "nat", "-t"， "iptables", "-A”

"PREROUTING"

"tcp", "--dport",

strconv.Itoa(int(hostPort)), "-j", "DNAT" "--to-destination",

fmt.Sprintf("%s:%d",vmIP, guestPort),

err =cmd.Run（)

slide-034.jpg

Slide text:

Arrakis: Updated Architecture

slide-035.jpg

Slide text:

86 apt-get clean A:

86:rm -rf. /var/lib/apt/lists/x

# Install Chrome.:

https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb 86 \: RUN wget. -0 /tmp/google-chrome-stable_current_amd64.deb # Any failure here will be fixed by the. install -f command below.

dpkg. -i /tmp/google-chrome-stable_current_amd64.deb ll true 86 \

apt-get clean 86 rm -rf /var/lib/apt/lists/*- rm. -rf /tmp/google-chrome-stable_current_amd64.deb 86 \ apt-get update 86 apt-get install. -f -y:86 \

RUN curl: -fsSL https://deb.nodesource.com/setup 22.x -o nodesource_setup:sh 8f # Install Node.js and npm

sudo -E bash nodesource_setup.sh 86 \

rm: -f nodesource_setup.sh 86 \ apt-get.clean: 86 rm -rf /var/lib/apt/lists/x: sudo apt-get:install -y: nodejs: 86

slide-036.jpg

Slide text:

Arrakis: Chrome

slide-039.jpg

Slide text:

Zero Overhead Sandbox Branching

- Zero overhead branching

- Btrfs powered incremental snapshots

- Memory dump compression at snapshot

- Multi stage plans can fail

- Restarting a deep DAG is expensive

- Backtrack + replan + execute much faster at scale

- Can cache key sandbox milestones across plans for reuse

slide-040.jpg

Slide text:

VMM.puso0: VhM.snapshot0 Pershst rw overlayfs iaryer. Recume the VMM

Teec (s aserrer)'srupitoruteta comtistiComnts strint, tzpihtte strl

I+pir?.I+f+ft+rtt+lr+d I+u+at t+ Los++t ra tilt Ioi tt".

Csp, orr'? 'sartetea.tiatuta() piaa,*-, m.apictioi otitw'.+ie.w(cur) 》 Pae't+ tt (irr +) t+is I + irrelhfte.

I! foshre + r+h Iho v +rh If. yupubot (o!ls. tfer focf?:( rrimalt - ma.ipitleat.beresttAt,trsmr'oratn)

Nu. itatat + yattaurel relt

jtateletoluabest: m ath.>eia(tutpastr,.ststeruioiitiioam) i/ Coy the sihtefel eil te the swmuNt eirmteryt'lime we yupaNt aorsh'l ue.rait.

1+ orr.ait (: J).tafet coriimg stetefut eiaa tt saapubet' dirrtery") Irt*+ coyytitkw.ttetetolotitpata, atatwtuthltoit): "4sttio='! ttthfuloliot..1.+y+. a,utat+fulplhprth,.

retws'nit, tat.trrorte raile 1o topy tialotot tik iout dirxtent' t', rr)

slide-041.jpg

Slide text:

Arrakis: Claude Desktop Vibe Coding

- Create a Google Docs clone where multiple people can join by specifying the name of the doc

- Use NodeJS and run the app on port 5737 in the background.

slide-042.jpg

Classification audit: raw/sources/slide-ai-classification/dense/wsFd22SL1s8/audit.json