Serverless Runner

Repository

I built this high-performance serverless execution engine to explore the boundaries of low-latency virtualization and distributed systems. By leveraging WebAssembly for sandboxing and implementing advanced patterns like async micro-batching and module caching, the system achieves over 15,000 requests per second in a Kubernetes environment.

Wasm Execution Engine

The core of the platform is a secure, sandboxed execution environment powered by Wasmtime. To prevent malicious or runaway code from affecting the host, I implemented strict resource limits, including memory caps (64MB) and fuel-based instruction metering (100M instructions). The use of memory pipes for stdin and stdout ensures that communication between the host and the guest is entirely in-memory, minimizing I/O overhead.

1pub async fn run_wasm(engine: &Engine, module: Module, input: Vec<u8>) -> AppResult<(i32, String)> {
2    let stdout_pipe = MemoryOutputPipe::new(1024 * 1024); // 1MB buffer
3    let stdin_pipe = MemoryInputPipe::new(input);
4
5    let mut wasi_ctx_builder = WasiCtxBuilder::new();
6    wasi_ctx_builder.stdout(stdout_pipe.clone()).stdin(stdin_pipe);
7
8    let state = HostState {
9        wasi: wasi_ctx_builder.build_p1(),
10        limits: StoreLimitsBuilder::new()
11            .memory_size(64 * 1024 * 1024)
12            .instances(1)
13            .build(),
14    };
15
16    let mut store = Store::new(engine, state);
17    store.limiter(|s| s);
18    store.set_fuel(100_000_000)?;
19
20    let mut linker = Linker::new(engine);
21    add_to_linker(&mut linker, |s: &mut HostState| &mut s.wasi)?;
22
23    let instance = linker.instantiate_async(&mut store, &module).await?;
24    let func = instance.get_typed_func::<(), ()>(&mut store, "_start")?;
25
26    func.call_async(&mut store, ()).await?;
27    Ok((0, String::from_utf8_lossy(&stdout_pipe.contents()).to_string()))
28}

Async Micro-Batching

Database persistence is often the hardest bottleneck in high-throughput systems. To overcome this, I decoupled the request path from the database write. Each execution result is dropped into a tokio::sync::mpsc channel. A background worker then aggregates these results and flushes them in batches using the PostgreSQL UNNEST operator. This allows the system to insert hundreds of records in a single round-trip, drastically reducing the transaction overhead.

1async fn flush_batch(pool: &PgPool, batch: &mut Vec<DbMessage>) {
2    if batch.is_empty() { return; }
3
4    let mut ids = Vec::new();
5    let mut function_names = Vec::new();
6    // ... extract other fields
7
8    let query = "
9        INSERT INTO executions (id, function_name, status_code, stdout_snippet, duration_ms, error_message)
10        SELECT * FROM UNNEST($1::uuid[], $2::varchar[], $3::integer[], $4::text[], $5::bigint[], $6::text[])
11    ";
12
13    sqlx::query(query)
14        .bind(&ids)
15        .bind(&function_names)
16        // ... bind other fields
17        .execute(pool)
18        .await?;
19}

Module Caching

To eliminate the "Cold Start" problem common in serverless platforms, I implemented a global module cache using DashMap. Instead of re-compiling Wasm binaries for every request—which can take 50ms or more—the compiled wasmtime::Module is stored in memory. This reduces the instantiation overhead to less than 1ms, transforming a CPU-intensive compilation step into a simple hash map lookup.

1pub struct AppState {
2    pub db_pools: Vec<PgPool>,
3    pub wasm_engine: Engine,
4    pub batcher: Batcher,
5    pub module_cache: DashMap<String, Module>,
6}
7
8// In the request handler:
9let module = if let Some(m) = state.module_cache.get(&function_name) {
10    m.clone()
11} else {
12    let m = Module::from_file(&state.wasm_engine, path)?;
13    state.module_cache.insert(function_name.clone(), m.clone());
14    m
15};

Performance Results

The final architecture was deployed to a Kubernetes cluster with 32 replicas of the runner service and a sharded PostgreSQL backend. Through intensive benchmarking with oha, the system reached a peak throughput of 14,972 RPS with a P99 latency of only 31.5ms. This represents a 90x improvement over the initial baseline, demonstrating the power of combining Rust's safety and performance with modern distributed system patterns.