Junhua (Drew) Zhu

Software Engineer · Cloud & AI Infrastructure

15+ years building software at scale. Now on a mission to democratize AI — by building the infrastructure that makes it faster, cheaper, more reliable, and accessible to everyone.

About

I'm a software engineer at Meta with 15+ years of experience building large-scale systems. I believe the biggest barrier to AI's potential isn't the models — it's the infrastructure beneath them. That's what I work on.

My work spans two sides of the same mission:

AI Infra— Making AI faster, cheaper, and more reliable to run. GPU clusters, distributed training, inference optimization, and model serving at scale.
Infra AI— Making infrastructure smarter. Agentic systems that self-heal, auto-remediate, and reason about infra the way an oncall engineer would.

Together, they form a flywheel: better infrastructure makes AI more accessible, and smarter AI makes infrastructure more efficient. That's the cycle I'm building toward — democratizing AI by driving down the cost and complexity of running it.

This mission is bigger than one person. I write and share what I learn at infra.bea.ai to build a community of infrastructure engineers who want to shape this future together. We call ourselves #TheLastProgrammers — not out of fear, but out of clarity. Yes, AI will write more and more of the code. But someone still needs to build the systems that make AI reliable, efficient, and safe. That's our job. That's our edge. And the more of us who understand these systems deeply, the faster we get to a world where AI is accessible to everyone.

Now

›Exploring the full AI infra stack — from silicon to intelligence — and building a systematic guide to help more engineers into this space
›Exploring agentic AI infrastructure — agent memory systems, sandboxed execution, and multi-agent coordination protocols
›Building clouda-sandbox — open-source serverless sandboxes on AWS Lambda MicroVMs for AI agents
›Publishing weekly deep dives on AI infra topics — KV cache, disaggregated inference, distributed training patterns
›Sharing learnings on LinkedIn and X (just started) with the AI infrastructure community

Work

Writing

I run infra.bea.ai and write weekly on LinkedIn. Recent series and posts:

→

MoE Inference Visualized

How expert routing actually works inside a real model — visualizing gating, load balancing, and sparse activation in Mixture-of-Experts inference.

→

Visualizing Every Step of LLM Inference

Inside a 64M parameter model — every matrix multiply, attention head, and token prediction visualized end to end.

→

Building GPT from Scratch

10-part series — from ~500 lines of PyTorch on a MacBook to distributed training across 1,000 GPUs. Data parallel, FSDP, tensor parallel, disaggregated inference.

→

Agent Sandboxes: The Cloud Compute Paradigm Shift

How cloud compute is evolving from human-triggered to agent-controlled — EC2 → Lambda → Lambda MicroVMs. E2B, Firecracker, and building clouda-sandbox.

→

Agent Memory Deep Dive

How every major coding agent handles memory — file-based, virtual memory, graph-based, dual-path with decay. Why the field is pre-convergence.

→

Agent Landscape: 50+ Open-Source Projects Analyzed

The 5 things every production agent needs — orchestration, durable state, memory, sandboxed execution, and coordination protocols.

→

The KV Cache Problem

The same caching problems we've solved for decades (Redis, buffer pools, CDNs) — now THE bottleneck in LLM serving. Paging, tiered placement, compression on GPU memory.

→

Prefill vs Decode: Why One GPU Can't Do Both

You wouldn't run OLTP and OLAP on the same database. So why are we doing exactly that with LLM inference? Disaggregated serving and adaptive routing.

Education

Hunan University

B.S. & M.S. in Computer Science and Software Engineering · Changsha, China

2002 — 2009

Get in Touch

I'm always happy to chat about AI infrastructure, distributed systems, agentic AI, or anything at the intersection.

junhuazhu@bea.ai LinkedIn