Banghua Zhu, co-founder and CTO of RadixArk — the company behind SGLang — on starting with inference, RL at ten-thousand-GPU scale, and how to run an open-source company.

Banghua Zhu is co-founder and CTO of RadixArk, the company behind the open-source inference engine SGLang. At Tsinghua he ranked first in his Electronic Engineering department. He then earned his PhD in EECS at Berkeley under professor Michael Jordan, working on machine learning theory, statistics, and reinforcement learning theory. With his advisor he co-founded Nexusflow AI, an enterprise-agent and post-training startup acquired by NVIDIA around mid-2025, where he became tech lead of NeMo-RL. In December 2025 he left to co-found RadixArk with his long-term collaborator Ying Sheng, SGLang's co-creator, who gave up even more at xAI to become CEO.

RadixArk builds the infrastructure under large language models: SGLang serves them in production, and Miles, its open-source reinforcement-learning framework, post-trains them. Adopters include Google, Microsoft, Oracle, NVIDIA, AMD, Alibaba, Tencent, and ByteDance; the company also provides infra support for Thinking Machines and collaborates with DeepSeek. The larger ambition: give every company and developer the same quality of infrastructure that frontier labs build for themselves. In May 2026 (five months after founding), RadixArk announced a $100M seed led by Accel at a valuation over $400M.

In our in-depth conversation, we discuss:

  1. What RadixArk ships: SGLang for inference, Miles for RL post-training, and the $100M seed
  2. Why demand runs inference > RL > pre-training, and why RadixArk starts with inference
  3. Cursor post-training Kimi: how inference traffic becomes the next model's training data
  4. The two 2023 cold emails (to John Schulman and to Ying Sheng) that pulled a theory PhD into LLMs
  5. Watching GPT-3.5 solve LeetCode while his advisor Michael Jordan stayed skeptical — and deciding to bet on LLMs anyway
  6. Day-0 support for DeepSeek-V4: ShadowRadix, sparse-attention work, half the core-dev team
  7. Why RL environments (function-calling latency, CPU scheduling, reward design) are the overlooked infra
  8. Nexusflow AI: making PPO work in open source with a 7B model, then the NVIDIA acquisition
  9. What GPUs do to talent: frontier labs, YOLO runs, and undergrads beating PhDs
  10. Taste as criticism: why the creator of Linux is his model, and what coding agents do to CS careers

My biggest takeaways from this conversation:

  1. Start where the demand is: inference first, RL next, pre-training maybe.

    Few companies can afford to pre-train. More want RL post-training, because it is cheaper and targets the agent tasks they care about. Everyone who touches AI needs inference. Zhu's ordering is blunt: "pre-training is less than RL, which is less than inference". So the products are sequenced against the ladder — "we are starting with inference first and then gradually moving to RL infra", with pre-training infra as the eventual third step.

  2. Your inference traffic is the next model's training data.

    The bet behind the business: "if you do a lot of inference it naturally generates a very large amount of data" — especially private, vertical data that RL can exploit. Cursor post-training Kimi's model is the template: improve a proprietary model on your own agent tasks, serve it, collect data, repeat. Zhu describes the loop from the provider's side: "They use my inference, and get data during inference". He expects the pattern to spread from frontier labs to AI-native companies to Fortune 500s.

  3. RLHF was crude but worked — which meant huge room for improvement.

    In early 2023 Zhu had never touched NLP — "I didn't even know GPT-2 or GPT-3 existed" — but GPT-3.5 writing LeetCode solutions shocked him. As an RL theorist he inspected OpenAI's post-training algorithms and "found them to be very ad hoc, not very principled ... so I knew this might be something hastily put together". Michael Jordan read the same evidence as more OpenAI hype. Zhu read crude-but-working as enormous upside under scaling, and committed within months.

  4. Day-0 support for a new model costs half the core team.

    SGLang's reputation for Day-0 support (a new model runs on it the day it is released) exists because the team pays for it: DeepSeek-V4's architectural innovations consumed half the core-dev team's time, producing ShadowRadix, a prefix-caching mechanism built for the model's hybrid attention, plus sparse-attention optimizations and several rounds of performance gains after launch. That grind is why Zhu's own early bet against SGLang was wrong. He thought no one would switch from vLLM to a latecomer; Ying Sheng's first-principles view — "if your inference engine is fast enough ... users will choose it" — won.

  5. The RL bottleneck is the environment, not the GPUs.

    An RL environment is where the model practices (executing function calls across dockers) and how it is scored (the reward). Both halves are now the constraint: on the systems side, "the bottleneck, even the entire training bottleneck is often in function calling" — latency, concurrency, CPU scheduling; on the design side, "how to design the reward, is an art". Frontier labs can only make models stronger by building complex environments where models still fail, so demand for this layer keeps rising. It is also exactly where Zhu, cautiously, sees the US pulling ahead: Chinese teams are strong on the infrastructure itself — "People in China are actually very strong at infrastructure" — but data is more limited there, and if the US keeps investing more in complex agentic environments, "China will fall behind a bit".

  6. PPO wasn't broken; it was under-tuned.

    In 2023, OpenAI said PPO (its reinforcement-learning algorithm) worked, but nearly everyone else who tried it failed and retreated to simpler methods like DPO. Zhu argued the opposite at conferences: "There are many tricks in it, and if you can do all these tricks well then the upper limit of your PPO will be very high". The failures had two unglamorous causes. One is infrastructure, where careless implementation leaves "hidden pitfalls everywhere" and wrong conclusions follow. The other is PPO's pile of untuned knobs (the critic, the discount factors, every hyperparameter) in a search space too large for systematic sweeps, so tuning ran on validated intuition. His concrete example: a freshly initialized critic will corrupt the language model beyond repair, so you first warm the critic up on some data, mathematically incorrect but necessary, and only then start PPO. The payoff was Starling, a 7B model trained with RLAIF (RLHF with AI raters): "we might be the only one at the time to get PPO working in open source", and it scored very high on Chatbot Arena. The bet was settled in his favor, and the team's tricks fed into Llama 2 and Llama 3 post-training, even as the field later moved to simpler algorithms like GRPO.

  7. Nexusflow was right about agents — just a year too early.

    The 2023 thesis was to use RL for function calling and verifiable tasks. But buyers kept asking whether an agent was just RAG (retrieval-augmented generation), and the team spent its time explaining that "an agent is actually a model that can take action". Zhu's lesson: "doing the right thing it must be at the right timing" — a correct prediction delivered early means educating a market instead of selling into it. The exit still worked: DeepSeek proving RL mattered made NVIDIA want more people doing RL.

  8. Your skills are capped by the GPUs you can access.

    Two to four nodes teach you nothing about trillion-parameter training; at thousands or tens of thousands of cards, fault tolerance, accuracy matching, and fleet-wide utilization are different problems entirely. Hence frontier-lab people seem rare: "people are shaped by the GPUs they get to use". Over-trained caution can even hurt — "Even undergraduates sometimes do better than PhDs", because the best runs often come from intuition-guided YOLO launches, not exhaustive ablations. His advice to students follows: go where the resources are; $20,000 a month of Claude Code tokens teaches what an advisor's grant cannot.

  9. The software is open source; reliability at ten-thousand-GPU scale is the product.

    RadixArk open-sources everything people use day to day. The extreme-scale layer is different: an open-source community cannot rent ten thousand cards to verify a framework, so commercial contracts fund the testing that makes the infra reliable — "Miles has already experienced thousand-card level testing". The extreme optimizations stay commercial not as a moat trick but because "even if this part is open-sourced not many people will use it". Revenue becomes the mechanism that democratizes access to scale.

  10. An agent's output is only as good as your ability to criticize it.

    Zhu screens for people who feel discomfort at bad code. His exhibit is the creator of Linux — "the ancestor of early Agent usage", who used people as agents and gatekept quality by rejecting everything below his standard. The same trait now decides how much you can get out of machines: "If you think the agent's code is already good then your upper bound will be at the level of the agent's code". Coding agents make taste more valuable, not less: the person who knows why 500 lines should be 100 gets more out of every agent.