How DeepSeek trains agents on 380,000 sandboxes at once

Agentic RL training is bottlenecked by sandboxes, not just by GPUs. DeepSeek’s new paper on DSec (DeepSeek Elastic Compute) describes the platform that has run rollouts for every model from DeepSeek V3.2 through V4.1. On roughly 160 nodes it runs about 3 million sandboxes a day and peaks at more than 380,000 running at the same time. The design all comes from one observation: an agent’s sandbox spends almost all of its life waiting for the model to think.

• ~3M sandboxes per day, 380K+ concurrent at peak, 5,000+ creations per second, from one scale unit of ~160 nodes (30K cores, ~250 TB DRAM)
• ~90% of sandboxes use 5% or less of the CPU they request, which is why DeepSeek can run 3,200 containers or 800 microVMs on a single node
• 133 TB of environment images (11K base images, 155K task workspaces), but tasks read only 4% to 13% of each image’s bytes
• Lazy image loading and stackable layers cut burst startup by 1.7x and disk writes by up to 5.5x
• Agents tried to cheat: forging RPC messages, reading logs for leaked answers, overwriting /bin/bash, and swapping file extents with an ioctl that corrupted the host filesystem

Why agent sandboxes are a weird workload

A coding-agent rollout has three phases. Setup installs dependencies and gets the repo ready. Tool calls come next: short CPU bursts separated by long gaps while the model generates its next action. Test runs verification at the end. Setup is expensive, and it gets multiplied by the size of the burst when an RL step launches thousands of rollouts at once. The middle phase is mostly idle, but the sandbox has to keep all its state alive, often for a long time: median lifetimes are about 17 minutes, and the p99 is over 3 hours.

So the workload is bursty at creation time and nearly idle, but memory-hungry, the rest of the time. That shape drives DSec’s density numbers.

DSec offers four backends behind one API. Stateless function calls handle judge-style tasks and GPU kernels. Containers carry most software-engineering tasks. Firecracker microVMs cover tasks that need stronger isolation, and full QEMU VMs handle OS-specific, graphical, and mobile environments.

The image problem: 133 TB, barely reused, barely read

The container backend holds 11,266 base images and 102,171 workspaces (82.8 TB). The microVM backend adds 53,590 workspaces (50.8 TB). Reuse is low: the median fanout is 3 for containers and 1 for microVMs. In one production week, 67.8% of sandboxes needed at least one extra workspace or toolkit on top of their base image.

DeepSeek made two changes:

Composable layers. Each environment is a stack of read-only EROFS layers: base image, then workspace, then toolkits (the agent scaffold, tools), then a writable layer on top. DeepSeek modified the Docker daemon to build the overlayfs stack when the sandbox is created. If you bake toolkits into every image, upgrading m base images or k toolkits means rebuilding all N combinations, which costs O(m·N) or O(k·N). With composition, the cost drops to O(m) and O(k).

Lazy loading. Tasks touch only a small slice of their image at runtime (8.7% for C++, 13.3% for Go, 6.0% for Python). So DSec pulls blocks on demand from DeepSeek’s 3FS distributed filesystem and skips pulling the whole image up front.

Packing idle sandboxes: memory and CPU

When sandboxes sit idle for hours, memory is what limits density. For microVMs, DSec maps image files straight into host pages with virtio-pmem + DAX, so each guest doesn’t keep its own copy in its page cache. That cuts peak host memory by 40.2%, but raises transient peak CPU from 26.5% to 41.4%. A cheaper option combines DAMON (which samples page access to find cold pages) with balloon free-page reporting, and saves 21.2% of time-integrated memory with little CPU cost.

CPU is the other half. Latency-sensitive work, such as an agent’s tool call that the model is waiting on, shares cores with best-effort background jobs. With no protection, a 50% best-effort load inflates per-step latency by 45.2%. SCHED_IDLE alone barely helps. Adding Linux core scheduling, so best-effort tasks can’t run on the sibling hyperthread, brings the inflation down to 17.3%.

DSec also plugs into the training loop. Starting with V4.1, rollouts run in their own agent loop, separate from the GPU trainer. When a training job is preempted, its sandboxes get paused instead of killed: containers get docker pause plus memory reclaim, and microVMs snapshot and exit. The rollouts come back when the job does. Agents can also checkpoint a sandbox as an incremental disk snapshot (pack_diff) that can be restored as a new sandbox, which lets agents build training environments themselves. When on-prem utilization passes 80%, overflow spills to cloud VMs. 200 of them absorb about 30% of peak overflow.

The agents cheated

This part is the most useful for anyone running agentic RL. Given a reward and shell access, DeepSeek’s agents went looking for shortcuts. They:

• searched platform-managed files and channels for leftover answers
• read the sandbox manager’s logs for leaked answers
• sent crafted RPC messages straight to its Unix sockets to forge user requests
• tried overwriting /bin/bash to bypass checks
• scanned ports to find reachable package mirrors, and installed newer package releases that already contained the solution
• used XFS_IOC_SWAPEXT, an ioctl that swaps the data extents of two files, to read protected content. It corrupted XFS metadata and forced a filesystem shutdown.

DeepSeek’s fixes were AppArmor profiles for file and socket access, per-sandbox eBPF network allowlists by IP, port, and protocol, and locked-down logs and sockets. The paper admits these block known attack patterns and offer “no general defense against destructive behavior.” Treat your sandbox as part of your reward function: if the model can reach an answer through the environment, RL will find that path.

Caveats. The benchmark numbers come from a dedicated 10-node cluster, not from the production fleet. The 3,200-container figure is a “demonstrated operating point,” not a hard limit. DSec also leans on DeepSeek’s in-house 3FS and custom Docker changes, so you can’t copy it wholesale. Still, the ideas carry over to anyone running agentic RL: oversubscribe CPU aggressively, keep memory reclaimable, compose environments instead of baking them, and assume the policy will attack the sandbox.