Host: Uncle Moon
Guest: Banghua Zhu

Source: English subtitles. Dialogue is grouped into paragraphs, retaining the conversation and opening highlights.

Speaker labels are inferred from context and have not been checked against the audio throughout. A dagger (†) marks passages with uncertain attribution. Timestamps come from the subtitles; short exchanges within one subtitle may share a timestamp.

The source English subtitles contain repetitions and some phrases that appear earlier or later than in the Chinese subtitles. Their wording is retained here, with punctuation and paragraph breaks added.

Contents

Highlights

Banghua Zhu (Guest) (00:00:00)

Even undergraduates can do better than PhDs.

Uncle Moon (Host) (00:00:02)

acquired by NVIDIA in August or September last year, and then started a second company in December last year.

Banghua Zhu (Guest) (00:00:07)

It's like they proposed some algorithms that were better than OpenAI's RLHF at the time— slightly better algorithms. Once it reaches for example, a thousand or even ten thousand card level, In fact, for the system, the requirements for the entire infra are completely different.

Uncle Moon (Host) (00:00:17)

To be honest, after meeting enough people, you'll find that integrity and kindness are truly important.

Banghua Zhu (Guest) (00:00:21)

People in China are actually very strong at infrastructure.

Uncle Moon (Host) (00:00:26)

From my perspective, it's a person with extremely strong judgment meeting a person with extremely strong judgment— super strong judgment.

Banghua Zhu (Guest) (00:00:31)

So I'm particularly good at exams. For example, whether it was the college entrance exam or at Tsinghua, I was basically always first in my department.

Leaving NVIDIA and the vision for RadixArk

Uncle Moon (Host) (00:00:40)

Hello everyone, this is Uncle Moon. Today we invited Zhu Banghua, the CTO of the company behind RadixArk SGLang, for an interview. RadixArk recently raised $100 million and is on a roll. Let Banghua introduce himself first.

Banghua Zhu (Guest) (00:00:56)

Hello everyone. I'm Zhu Banghua, and now I'm the co-founder and CTO of RadixArk. Before that, I did my undergraduate degree in the Department of Electronic Engineering at Tsinghua. Then at Berkeley, I did my PhD in EECS. After graduation, I founded a company Nexusflow AI, which mainly focuses on enterprise agents, post-training and reinforcement learning. Then this company was acquired by NVIDIA around the middle of last year. I joined NVIDIA to be the tech lead for NeMo-RL— tech lead. And recently, my long-term collaborator Ying Sheng and I founded RadixArk. Nice to meet you all.

Uncle Moon (Host) (00:01:36)

Last year, it was acquired by NVIDIA around August or September, and then in December last year, he started a second company. Because I've started businesses before, I know this wave definitely lost a lot of money from the acquisition. What motivated you to start a second business even after losing so much money?

Banghua Zhu (Guest) (00:01:55)

I think the most important thing is the community's technology and people. And also the company's mission and what it wants to do. And also the company's mission and what it wants to do. I think these two points combined are what really attracted me. We can discuss this in more detail later.

Uncle Moon (Host) (00:02:10)

Can you reveal how much money you gave up? How much money did you give up?

Banghua Zhu (Guest) (00:02:15)

I can't really say the specifics, but it's probably enough to live a comfortable life. However, it's definitely not as much as what our boss Ying gave up from xAI.

Uncle Moon (Host) (00:02:24)

Okay, let's officially start today's interview. Let's talk about the startup phase, okay? Let's talk about RadixArk. This company has been very popular recently, having just raised $100 million, and is valued at over 400 million. They have many collaborations with domestic companies like DeepSeek and so on, and also many collaborations with Thinking Machines in North America many collaborations. Can you introduce what RadixArk does? What is your vision?

Banghua Zhu (Guest) (00:02:52)

RadixArk is essentially doing one thing. Essentially, it's taking Frontier AI Lab-level, or even better, AI infrastructure, to democratize it, so that more companies, developers can use it more open, more reliable, and efficient AI infra. And then after it's done well, share it with everyone. So essentially, we are an AI Infra company. And the core is around large language models training and inference systems. And on the training side, we are currently working on an RL reinforcement learning framework, which is Miles. And on the inference side, we are making an open-source inference open-source inference framework called SGLang. And SGLang and Miles have a lot of open-source adoption. Including several top companies in North America, such as Google, Microsoft, Oracle, NVIDIA, AMD, etc, also Alibaba, Tencent, ByteDance, etc, In fact, everyone has very deep cooperation.

Uncle Moon (Host) (00:03:55)

You just mentioned that you do both inference and reinforcement learning. When you founded this company, how did you decide on these two directions, to start together?

Banghua Zhu (Guest) (00:04:04)

Yes. Because we are an infrastructure construction company, So we hope to cover the entire AI life cycle well. This actually includes the current AI's most relevant Infra is pre-training to post-training and finally to inference. That is, first train the model, and then use it for inference. And for pre-training, actually, the companies that need this, because the cost is very high, so there are not many companies that want to do pre-training. And post-training, which is RL, actually has a greater demand, because compared to pre-training, its cost is lower. At the same time, you can also use RL to improve the model very well in a specific task, or an agent task you are interested in to improve the model very, very well. And of course, inference is the biggest demand, because all, basically everywhere, if you need to use AI, you need to talk to AI and use an inference engine to do inference— engine to do inference.

Banghua Zhu (Guest) (00:05:00)

So from this demand, pre-training is less than RL, which is less than inference. So we are starting with inference first and then gradually moving to RL infra. and even step by step, if possible, we'll also do pre-training infra.

Uncle Moon (Host) (00:05:14)

You just said that you helped many companies plan to do post-training, which is reinforcement learning. Because some time ago, there was a company called, Cursor, they used Kimi's model for post-training. Do you think this approach will become mainstream? Is that right? Is that right? Because you chose this business model.

Banghua Zhu (Guest) (00:05:32)

We think, our vision or our bet is that these kinds of things will happen more and more. This is because before, everyone had so many open-source models, most people's needs were to directly interact with these open-source models. But with more and more complex Agentic tasks constantly emerging, In fact, if you do a lot of inference, it naturally generates a very large amount of data. And this data is very helpful for further improving the model's capabilities. Especially some private data, some, private vertical sector data is very useful. For RL, it's very useful. So eventually, if everyone considers it from a cost-reduction and efficiency perspective, they would definitely prefer to train the model first, train it to a more satisfactory state. and then do inference. So like Cursor, they will, for example, further improve Kimi's capabilities within Cursor, That is, on Cursor's own agent-related capabilities, and then do inference. Yes.

Uncle Moon (Host) (00:06:35)

So, from your perspective, this is a good closed loop. Right?

Banghua Zhu (Guest) (00:06:40)

Yes.

Uncle Moon (Host) (00:06:40)

I help them improve the ability of their private models— the capabilities of their proprietary models. They use my inference, and get data during inference. Continue to help them improve the capabilities of their private models. So, for these two directions—

Banghua Zhu (Guest) (00:06:49)

Yes, you could say that.

Uncle Moon (Host) (00:06:50)

what do you think about the business model and future development? How do you view this issue?

Banghua Zhu (Guest) (00:06:54)

Right. Currently, the most direct and biggest demand, especially commercial demand, is still on the inference side. I think reinforcement learning, or so-called post-training for agents, is actually just starting. More and more people are realizing that you might need to train the model. This initially only happened in frontier labs. Everyone would say, "I need to train a model," and then provide it for inference. Then it gradually spread to NewLabs and AI-native companies, like Cursor. And then later, we think it will also gradually spread to companies that are further away from AI, traditional enterprises, Including Fortune 500 companies. If they realize that the AI capabilities they need are beyond what current AI can provide, They might choose to do some post-training themselves and then do inference.

Uncle Moon (Host) (00:07:45) †

Okay. we can discuss these topics like Neo Labs later, okay? I actually—

Banghua Zhu (Guest) (00:07:52)

This just now—

SGLang and the move into language models

Uncle Moon (Host) (00:07:53)

And because I'm quite familiar with the SGLang community were you involved from the beginning? Or did you gradually join when the company was founded later?

Banghua Zhu (Guest) (00:08:00)

I've always been— until I joined the company, I wasn't really part of the SGLang community. Yes, this is actually a longer story. Ying Sheng and Lianmin Zheng— Ying Sheng and Lianmin Zheng— and they are the two co-creators of SGLang. when we were PhD students, And at that time, I was in my fifth year of my PhD I was doing my PhD. When GPT first came out, Um, before that, I had been working on Machine Learning Theory, and Statistics, This Reinforcement Learning Theory, etc. And then the moment I used ChatGPT, I felt that this might be a groundbreaking thing. So at the end of 2022, beginning of 2023, I decided to do something related to language models and GPT. So at that time, I was looking for what kind of direction I should take. So at that time, I was actually very actively reaching out to many people, just wanting to collaborate with everyone, and understand what everyone was doing.

Banghua Zhu (Guest) (00:09:10)

And at that time, the outreach was quite responsive. Um, it was actually two groups of people. One was John Schulman. And one group was John Schulman. He was also like a senior from Berkeley. And at that time, I— we had written an RLHF theory paper, Um, which essentially proposed some algorithms that were slightly better than OpenAI's RLHF at the time, which proposed algorithms slightly better than OpenAI's RLHF. So I emailed John at the time and said: we might have a better algorithm we can discuss it together. And John replied very quickly. And John replied very quickly. LoRA, and even some other system issues. LoRA and even some other systems issues. And the other email was actually sent to Ying Sheng. I also sent another email to Ying Sheng. Because they had an early paper called FlexGen, which was about how to do LLM inference on a CPU— LLM Inference. It was a very early inference paper.

Banghua Zhu (Guest) (00:10:11)

I found it very interesting after reading it. So I directly reached out to Ying Sheng. Then Ying Sheng brought Lianmin Zheng, who was at Berkeley at the time, to join us. So the three of us started doing a lot— This collaboration in research. Later, they were both very innovative people. And at that time, they continuously worked on LMSYS— such a non-profit organization. And there are many open-source projects under it, including Vicuna, which was one of the earliest distilled language model, and it had tens of millions of downloads on Hugging Face at that time. Then they later created Chatbot Arena, which was also a very popular model benchmark comparison tool— Um, a tool. At that time, OpenAI, Anthropic, and xAI were all competing on this leaderboard. a tool. At that time, OpenAI, Anthropic, and xAI were all competing on this leaderboard to prove that their model was the strongest.

Banghua Zhu (Guest) (00:10:58)

Then, This later became LM Arena, and now it's a company worth about 1.5 billion. And later, Ying Sheng and Lianmin started SGLang. Ying Sheng and Lianmin began building SGLang. Um, I have to mention, Um, I have to mention that before SGLang, there was another— before SGLang, there was another competitor or another very popular open source inference engine called vLLM. And they actually did very well at that time. They actually did very well at that time, and they had been doing it for almost a year. And I personally wasn't particularly optimistic about SGLang at the time. Because I think if you're in a domain, where there's already a very good domain, to make a completely identical or very similar product, and an open-source software. It might not be able to convince others to use SGLang instead of vLLM. So at that time, I was actually working with my advisor on the first company, Nexusflow AI.

Banghua Zhu (Guest) (00:11:48)

And we didn't, I didn't— And we didn't— I— basically didn't participate in the SGLang project. But after those one or two years, SGLang went through various events, SGLang went through all kinds of ups and downs. And the community became more and more active, and many people were willing to contribute to SGLang, and even many people's— And the community became more and more active— careers or life paths have changed because of SGLang. many careers and lives changed because of SGLang. So around the end of last year, Ying Sheng and I talked about it at length again and caught up. And at the same time, everyone wants to do more than just inference, but also more things related to Frontier Infra, but also more things related to Frontier Infra, including training, including training, And this is probably more aligned with my expertise.

Banghua Zhu (Guest) (00:12:33)

So I decided around that time that I could just do more open source with everyone, and do more interesting frontier system research. just do more open source with everyone, and do more interesting frontier systems research.

Uncle Moon (Host) (00:12:47)

Wow, is this information so extensive?

Banghua Zhu (Guest) (00:12:48) †

No problem, no problem, no problem.

Uncle Moon (Host) (00:12:49) †

Okay, okay. That story was really, really long. John Schulman appeared. There was John Schulman, and LM Arena, and SGLang. then LM Arena, and SGLang, vLLM, and other communities. vLLM and other communities, right? Right? When I was listening to you just now, I found your reaction to be very, very quick. At that moment, Because at that time, I had just graduated with my PhD, I was also working at a company. but when GPT came out, many people around me were questioning it, "Oh, too many hallucinations," "and this thing might just be a flash in the pan," and so on. How certain were you that this would be the next big wave?

Banghua Zhu (Guest) (00:13:26)

Um, I actually— I had no contact with NLP at that time. I didn't even know GPT-2 or GPT-3 existed. I didn't even know GPT-2, GPT-3 existed. Yes, so—

Uncle Moon (Host) (00:13:36)

I don't know.

Banghua Zhu (Guest) (00:13:37)

Yes, so, So at that time, I thought, I tried GPT-3.5, and it wrote code— 5, it wrote code. It was actually written— It's very bad now. But at that time, I never thought that such a next token prediction machine could could, for example, for example, simple LeetCode questions, for example, simple LeetCode questions, it could write them out for you. It could write them out for you. This really shocked me at the time This really shocked me at the time. And my advisor at the time— And my advisor at the time, Michael Jordan— Michael Jordan, um, when I told him about this, he was actually quite skeptical. He thought that statistically, it was just a next token prediction problem, and it was no different from previous things. token prediction problem, it's no different from before. He thinks this might just be another OpenAI hype.

Banghua Zhu (Guest) (00:14:14)

But I personally feel, if at that stage, without much careful training, under the circumstances, it could already reach such a level, If you continue to scale, whether scaling back then it was scaling parameters, then scaling, more data, more responses length, like test time thinking, I think its potential is huge. So after trying it, I felt I had to commit to this direction.

Uncle Moon (Host) (00:14:44)

Wow, your understanding is too profound. At that moment, you could react and realize that this thing was still quite rough. And as long as it slowly scales more and becomes more refined, the results will definitely be better, and the upper limit might not be seen for a short time. You already had this understanding at that time.

Banghua Zhu (Guest) (00:15:00)

Well, to be precise, because we do theory, So we looked at it then, for example, some algorithms he used in post-training RL, we found them to be very, ad hoc, not very principled. There are many ways to improve the algorithm. But he didn't do that then, so I knew this might be something hastily put together. It's quite— But it really— I think what was surprising then— a surprising point was that such an RLHF algorithm, which seemed very trivial and straightforward, It actually made the model so natural in human interaction. This, I think, if you let me predict, I would predict the exact opposite result. I would think this algorithm couldn't work. Yes, I still find that very amazing.

Uncle Moon (Host) (00:15:48)

What you just said really shocked me, because how should I put it? Just some time ago, Yao Shunyu said in an interview, "Don't trust the old guys," and "have your own judgment," similar to that kind of statement. Anyway, you saw this so clearly at that moment, because you yourself have a lot— because you are a PhD in reinforcement learning, a PhD in reinforcement learning theory.

Banghua Zhu (Guest) (00:16:04)

Uh-huh.

Uncle Moon (Host) (00:16:08)

And you found it so rough, and the results were so good. If it's done right, or more refined, that would be amazing, right? That's the feeling, right?

Banghua Zhu (Guest) (00:16:14)

Yes, that's right.

Uncle Moon (Host) (00:16:15)

Wow, that's amazing. Haha, this, this, this, This wave should be led by people like you. Exactly. How should I put it? This wave should be led by people like you. Anyway, when I finished listening to that, that's how I felt. You accurately predicted future developments. And also with many current popular figures, you have interactions and communication. And then you went all in, right?

Banghua Zhu (Guest) (00:16:45)

Hmm, almost, but I actually think it's still very early. I think at such an early stage, everyone has a chance. So the game has just begun.

Uncle Moon (Host) (00:16:54)

Yes. In May 2026, you think the game has just begun.

Banghua Zhu (Guest) (00:16:58)

Yes, that's right.

Thinking Machines and supporting new models

Uncle Moon (Host) (00:16:59) †

Okay, let's expand on this topic later. Yes, OK, that's because— You just mentioned John Schulman as well. John Schulman is not only a co-founder of OpenAI, but also a co-founder of Thinking Machines, right? Then, Thinking Machines Lab recently released their new multimodal interaction model. And I also found on X that they interact a lot with you, and mentioned that you provided infrastructure support as well. Then, Can you tell us what kind of model do you think is different from OpenAI, Anthropic, and Gemini? In your opinion.

Banghua Zhu (Guest) (00:17:31)

Right. Hmm. Thinking Machines— They released this interaction model, and they actually, um, released many demonstration videos, which were indeed very impressive. It's a more real-time voice interaction model, which I understand they've always wanted to do, which is more multimodality That is new attempts at Human AI interaction. So I've always felt, I really like the culture of Thinking Machines. What I like about Thinking Machines is: let everyone allow researchers researchers to try various different things. And even if some of the results of these attempts may not be particularly good, but they will always have some experimental results that can eventually flourish into something that, for example, leads the next era. So I haven't played with this interaction model yet, but I think after seeing many of their demos, I'm really looking forward to trying it. This is not yet possible, but we can only say, please look forward to it.

Uncle Moon (Host) (00:18:24)

For example, in SGLang and during the collaboration with Thinking Machines, is there anything interesting you can share with everyone?

Banghua Zhu (Guest) (00:18:32)

I think first of all, it's already a very multi-modality model, meaning it has voice input and voice output.

Uncle Moon (Host) (00:18:35)

Understood. So, do you think this new interaction model for LLM infrastructure, will it change the paradigm for inference infrastructure such as SGLang?

Banghua Zhu (Guest) (00:18:46)

And this part, especially for for example, how to prepare such data, for example, you see, you will see— you will see Thinking Machines— Thinking Machines— Thinking Machines— It knows when to reply. And these are actually all in text it probably won't appear in such a very— And these are actually all in text natural multi-turn conversation format— um, natural multi-turn conversation format. So this is actually for. the entire data pipeline, it will have some very different things, very different from pure text. it will have some very different things. And at the same time, of course, this voice model, it actually includes the design of this token, and the design of the entire infra, for the support of the omni model, and the overall infra design, Support for omni models will actually be new challenges.

Uncle Moon (Host) (00:19:36)

Of course, not only with Thinking Machines. Everyone knows that SGLang is always able to support models on day one— provide day-one support. What kind of effort do you think was put in during this process? What kind of effort was actually put in? And throughout the process, as a CTO—

Banghua Zhu (Guest) (00:19:50)

Yes, building this ecosystem requires a lot— The coredev team's tireless efforts and support. And especially when very, very new architectures emerge, this applies to inference infra and even training infra, There will actually be more challenges. For example, DeepSeek V4, it actually made many innovations in model architecture. This means that when doing inference and training infra, everyone needs to make many new optimizations and changes for their specific architecture. For example, SGLang proposed some ShadowRadix. These are some changes for the prefix cache, specifically for DeepSeek V4, and optimization. And even some Hi Sparse for sparse attention optimization, etc, sparse attention optimizations, etc. So at that time, for example, the DeepSeek V4 Pro project, probably took everyone— this probably took half of the core-dev's time, even half of the company's people's energy, to support this.

Banghua Zhu (Guest) (00:20:52)

Yes, the results were also very rewarding, which is, for DeepSeek V4, the performance when it was first obtained and the final performance delivered by SGLang— they really optimized it many, many times.

Uncle Moon (Host) (00:21:03)

what do you think is your main role? What is the main role? Yes, everyone is so strong that this can be done spontaneously, right?

Banghua Zhu (Guest) (00:21:09)

Hmm. Currently, I do more training. And about DeepSeek V4, I actually joined quite late, so many of the early tasks, were often coordinated by other colleagues in the company. Yes, everyone is very reliable.

Uncle Moon (Host) (00:21:26)

Yes, everyone is so strong. This can be done spontaneously, right?

Banghua Zhu (Guest) (00:21:30)

Yes, that's right.

Uncle Moon (Host) (00:21:31)

OK. because you've collaborated with so many different large model companies in China and the US, what do you think is the biggest difference in experience?

Banghua Zhu (Guest) (00:21:38)

Hmm. I think in terms of experience, Chinese students, their Infra level is very high. And everyone probably writes code, and they write it very fast, very reliable. And here in the US, I think everyone's ability to do things from scratch— I think everyone's— I think it's more imaginative. Everyone is actually more— thinks more, has more— more crazy ideas, more crazy ideas, many of which may not work, many of which may not work. But once some of them work, they can create very new things. But once this— for example, when everyone knows how to do this, I think Chinese colleagues are very good at doing things from one to a hundred and making them perfect.

Model gaps and reinforcement learning environments

Uncle Moon (Host) (00:22:20)

Yes, I've interviewed many AI infra colleagues in Silicon Valley, and everyone says: if in a community, once a framework is basically set up, especially for an AI Infra project, If the number of Chinese people cannot exceed half, the code quality of this project will sharply decline. Do you agree with this statement? Yes. Let's continue to talk about this.

Banghua Zhu (Guest) (00:22:36)

Hmm. I can only say that there are many very strong Americans, but, I think in this era, because— Hmm, because AI— it's mainly about coding, And or some math capability, that is, people's ability to, understand well how to write the best system or make the best data and algorithms. And in this regard, I think Chinese people are better at it.

Uncle Moon (Host) (00:23:04)

the comparison we just mentioned, DeepSeek V4 and other open-source— open models like DeepSeek V4, including Thinking Machines, including Thinking Machines, Including Anthropic and these closed-source large models, Anthropic and other closed-source LLMs, how big do you think the gap between the models is? 6 months? 1 year? That is, the open-source models from these New Labs in the US and the closed-source models in the US are a year apart.

Banghua Zhu (Guest) (00:23:22)

Hmm, I actually think it depends on the region. If you're comparing Chinese open source and American closed source, I think it's about 6 months. If you're comparing American open source and American closed source, then it's probably closer to a year. Yes Ah, there aren't many New Labs in the US doing open source models yet.

Uncle Moon (Host) (00:23:36)

Yes. So, American open-source large models are 6 months behind Chinese open-source large models.

Banghua Zhu (Guest) (00:23:42)

and I'm just talking about the ones that have been released. For example, NVIDIA's Nemotron model. for example, NVIDIA's Nemotron model—

Uncle Moon (Host) (00:23:47)

I know this.

Banghua Zhu (Guest) (00:23:50)

For example, NVIDIA's Nemotron model.

Uncle Moon (Host) (00:23:52)

I know this.

Banghua Zhu (Guest) (00:23:53)

because we came from NVIDIA's Nemotron before, so everyone is still catching up. But the progress has been very, very fast recently, so maybe the distance— I think the gap is slowly narrowing. I think that's about right, yes. Yes.

Uncle Moon (Host) (00:24:06)

Chinese open-source large models are 6 months behind American closed-source large models. Is that roughly the magnitude, right? Because you've personally worked on these models.

Banghua Zhu (Guest) (00:24:14)

Hmm, yes, I think this also varies between China and the US.

Uncle Moon (Host) (00:24:16)

what do you think are the main reasons for these model differences? Infra, data, algorithm optimization? We just talked about the reinforcement learning environment.

Banghua Zhu (Guest) (00:24:23)

From the perspective of open-source large models, I think in China, everyone's Infra is actually quite good. Quite good, quite solid. But data might be a more limited area. I actually think that because in China, maybe— I actually feel that, because in China, maybe— For some reason, everyone actually for truly complex RL environments, investment might not be more than in the US. So, in the US, everyone keeps investing in this more complex thing agentic RL infra. China will fall behind a bit.

Uncle Moon (Host) (00:25:00)

can you elaborate on this part? So, do you think—

Banghua Zhu (Guest) (00:25:05)

Hmm. reinforcement learning environments are often overlooked, but also very important, part of AI infra. That is to say, especially now that agent environments are becoming more complex, often, the bottleneck, even the entire, training bottleneck is often in function calling. Its latency cannot be controlled, and even the concurrency is not high enough. This is why everyone suddenly realized CPU is very important, because often, for example, when doing code execution, These other multi-docker environments such CPU scheduling, has become increasingly important. And in this area, on one hand, AI infra also brings new challenges, the challenges this poses for AI infra. On the other hand, building truly high-quality environments, especially how to design the reward, is an art. So, being able to design the reward well— So, being able to design the reward well, and also allow the model to continue improving in complex environments, Even in multi-turn environments like Claude Code, in environments like Claude Code, These are all very open questions.

Banghua Zhu (Guest) (00:26:12)

So, if we can do all this data well— So, reinforcement learning still has a long way to go. Right Hmm, some are RL environment infra.

Uncle Moon (Host) (00:26:21)

for example, there are many reinforcement learning environment companies in Silicon Valley, such as E2B, I know. You think it's very necessary, right? I know.

Banghua Zhu (Guest) (00:26:26)

Yes. Some are specifically for building RL environments, the data for RL environments, which is the prompt and the corresponding reward. And then sell it to these frontier labs, and sell it to frontier labs. Right, I think maybe in the US, everyone wants to push the frontier better. Yes.

Uncle Moon (Host) (00:26:43)

This is also a strange point that among domestic entrepreneurs such companies are relatively rare. But in Silicon Valley, there are already many such companies, right?

Banghua Zhu (Guest) (00:26:51)

Right. for many frontier labs, the only way to make models stronger is to build more complex environments, where models currently fail, and then make their models succeed in these environments. So their demand for this part will be stronger. Yes, this is also one of our focuses. Hmm, when Nexusflow started, there were a few—

Uncle Moon (Host) (00:27:13)

So while you're developing your own RL Infra, you also collaborate with these companies to make your RL Infra and these environments better adapted, right?

Banghua Zhu (Guest) (00:27:23)

ideas at the time.

Starting Nexusflow and making PPO work

Uncle Moon (Host) (00:27:25)

Okay. So we've talked about SGLang. Let's go back to your first startup. Your first startup, Nexusflow AI. What was the trigger? How did you decide to start this with your advisor?

Banghua Zhu (Guest) (00:27:37)

Hmm, the first one— ideas at the time. Hmm, the first one— Nexusflow was founded around early to mid-2023 and I decided to change careers. And at that time, Hmm, one thing is, We made an open-source model called Starling. It was a 7B model, using a lot of open-source data. We made an open-source version based on RLAIF to create an open-source version of RLAIF. And scored very, very high on Chatbot Arena. And it really demonstrated how RLAIF algorithms can make the model— demonstrating how RLAIF could make the model— Right, and at the same time, the results would be more satisfying. So we thought at the time, that we might be the only one at the time to get PPO working in open source. And I think this thing, If you can this RL thing, If you do it very, very well, you can use the same insight to make agents better.

Banghua Zhu (Guest) (00:28:39)

So in 2023, we thought we should use RL for function calling. Use it for, even verifiable We thought we should use RL for function calling— some, its tasks like math, some, its tasks like math, Even coding, and even later, this, function calling. So this was one of the reasons for Nexusflow— opportunity. And then, later, we also tried to scale up to for example, 30B, which was quite large at the time, or even 70B models. And also helped various companies do this RL. This as a service is post-training enterprise agents. At that time, many people were very concerned about this topic. And there were also some very good early signals. So this was a major motivation for starting a business.

Uncle Moon (Host) (00:29:24)

In 2023, you got PPO working in your own environment, with your own data.

Banghua Zhu (Guest) (00:29:30)

At that time, it was because because OpenAI said they did PPO very well. But when many of us tried it, we found that PPO actually didn't work. At that time, more popular methods were, for example, DPO, some simpler versions, and at the same time, it didn't require, for example, a value function. But actually, our belief at the time was that this was actually more because you didn't really tune PPO well, tuned. There are many tricks in it, and if you can do all these tricks well, then the upper limit of your PPO will be very high. So I always had this belief, and at that time, at, various conferences, I would also battle with people. But eventually, it was proven that if you can do PPO well, its upper limit is very high.

Banghua Zhu (Guest) (00:30:12)

Of course, in the end, there were GRPO and other relatively simpler algorithms, relatively speaking simpler algorithms, and now gradually, fewer people are doing PPO, and more people are using GRPO. I think Infra actually has—

Uncle Moon (Host) (00:30:23)

So, what do you think was the most important factor? What was the most important factor? Was it your RL background? Or did you build better datasets? Better Infra, etc? Can you share?

Banghua Zhu (Guest) (00:30:33)

I think Infra actually has a relatively large weight in it, because there are many places where it is implementation and when training, it may make mistakes. If you do it— Infra is not a fancy thing, but if you don't do many things carefully, then there will be hidden pitfalls everywhere, and then you will draw completely different or wrong conclusions. And of course, on the other hand, PPO's— PPO needs to be tuned correctly. It has too many things to do. For example, how to train the critic, which is the value function? For example, the critic, How to train the value function? And how to design all these discount factors and all the hyperparameters in the entire PPO algorithm? all the hyperparameters in PPO? tuning parameters. Yes. Not really.

Uncle Moon (Host) (00:31:15)

This tuning experience, do you think it's something you accumulated over 5 years of study?

Banghua Zhu (Guest) (00:31:19)

because I used to do pure theory, so actually, so actually, starting to do Language Model at that time was also around the same time— also at this— also around the same time. It's not difficult.

Uncle Moon (Host) (00:31:28)

How did you finally tune it? How difficult was this process?

Banghua Zhu (Guest) (00:31:33)

it's not difficult. In fact, many times when doing this kind of model training, because the search space is too large, so it's actually hard to find this thing very principledly. And more often, it's like some intuition. If you feel that your intuition is correct, you just try in that direction. If you feel that your intuition— And if you feel that this direction— if you validate some of your assumptions, you will— if you— you will validate part of this intuition, and then you propose new intuitions, because the dimension of this problem is too high. So you can't just do a greedy search, because the dimension of this problem is too high. But instead, you need to use your own very, very intuition, to figure this out with your own intuition. very, very— to explore a path. That's roughly what it means to explore a path. Is that a bit too abstract? Right. For example—

Uncle Moon (Host) (00:32:16)

Right. It's a bit too abstract. Can you give a very precise example? Just one.

Banghua Zhu (Guest) (00:32:23)

For example, this critic, when training the value function, This is a very simple trick. If you directly initialize a critic randomly, or even if your— initialize a critic randomly, or even if your critic— or even if your— the critic is from the reward function initialized, Then add a hat, You'll find that the initial loss function is very high. It might start from single digits, two or three, but usually the loss, function is like 0.00something. Then you need to train for a while, and it will go down. But at this point, if you use this critic function to directly optimize your policy, which is the language model itself, it will cause the language model's weights to change significantly, and very— because the direction and update weights are incorrect, or the gradient itself may not be quite right.

Banghua Zhu (Guest) (00:33:14)

So a better approach, even though it's mathematically incorrect, is to first warm up the critic with some data, and train it for a while, letting the loss drop, for the current policy to a relatively low level, letting it converge. Then you use this critic to initialize the entire PPO training. There are many such tricks. but if you don't do this, for example, if you don't do this, it might destroy the model from the start. And you won't be able to fix it later. Yes, tricks like these.

Uncle Moon (Host) (00:33:46)

So you started in 2023, and through your intuition, you gradually figured out these tricks. You got it working, and finally, you were able to get a very high score on the LM Arena leaderboard, right?

Banghua Zhu (Guest) (00:33:58)

Yes, that was an interesting thing we did back then. Yes.

Uncle Moon (Host) (00:34:02)

Did you tune this 7B?

Banghua Zhu (Guest) (00:34:05)

Yes, it started with 7B.

Uncle Moon (Host) (00:34:07)

How long did it take to tune this 7B model?

Banghua Zhu (Guest) (00:34:11)

This actually didn't take too much time, maybe less than a month. But preparing the data took a long time, because at that time, there wasn't much multi-comparison data available open-source— multi-comparison data. So for one question, there are many different responses. And then this response also needs to be ranked by people or models. Because the concept of RLHF itself was very new, because RLHF was still a new concept. Yes, and then open-sourced this dataset and pipeline. I don't think so.

Uncle Moon (Host) (00:34:43)

Wow, that's really impressive. Because you said you had never touched LLMs before. But from knowing ChatGPT to starting a business and training such a model, RLHF model, it only took such a short time. Is there a bit of genius involved?

Banghua Zhu (Guest) (00:34:58)

This actually means that if you let someone now, for example, an undergraduate, if they have an idea, I think they can also implement it very quickly. And I actually think more of it is that at the beginning of 2023, it was a stage of blooming, where everything related to LLMs was very underexplored, everything— LLM-related was very underexplored. So at that time, doing inference engines, doing model evaluation, doing RL, you could produce a lot of interesting and impactful work.

Uncle Moon (Host) (00:35:29)

Yes, but at that time, through this work, you already foresaw, function calling, that agents would definitely become a trend, and you had already started helping some companies deploy these things, right?

Banghua Zhu (Guest) (00:35:39)

For example, we also did some including function calling benchmarks and also fun- function calling, Agent synthetic data generation pipelines. At that time, we also had a lot of discussions with the Llama team, and later, many tricks were used in Llama 2 and Llama 3's post-training.

Uncle Moon (Host) (00:35:54)

Is this also an important reason why you were eventually acquired by NVIDIA?

Banghua Zhu (Guest) (00:35:58)

I think NVIDIA's opportunity at that time was mainly due to the DeepSeek V3 release. It made everyone realize that RL is actually very important. Because before that, many people still thought that if they only did SFT or directly SFT'd GPT's output, they could make open-source models very good, and wouldn't need to do RL. But the DeepSeek V3 model proved that— But DeepSeek V3 proved that that RLVR training, even if you directly start from a pretrained checkpoint, Doing RLVR is enough, you can just do RLVR. including NVIDIA. So at that time, NVIDIA really hoped that everyone would do better, That is, more people would do RL. Yes Yes, RLHF is—

Uncle Moon (Host) (00:36:42)

Understood. You just mentioned two key terms. One is RLHF. The other is RLVR. Can you briefly explain these two concepts?

Banghua Zhu (Guest) (00:36:51)

Right. RLHF means reinforcement learning from Human Feedback. So its main idea and purpose— from Human Feedback. So it— Its main idea and purpose is to make the model's speech more human-like, and more easily liked by people. So his idea is to first learn what kind of response is more likable. And its method is to train a reward model. And this reward model is from one response generating many— one prompt generating many different responses, and then people rate them, which response is better, Which response is worse. Then, from this data, a scoring model is trained, then use that scoring model to guide the training of large language models. The ultimate goal is to make the model's speech more appealing. So if you chat with GPT, before, you might have felt that when you asked a question, it might have been very concise, and didn't give you enough information. But after GPT-3.5, it became more human-like, and its speech became more detailed.

Banghua Zhu (Guest) (00:37:53)

And in this way, everyone might like the given answer more. And as for RLVR, it's actually more like— It's— It stands for Reinforcement Learning with Verifiable Rewards. So it's this— It mainly targets areas that can be truly verified. For example, if you have a math problem, or a coding problem, then it can directly— you can directly calculate whether the math problem is correct, Or whether the code is well written, whether it is written correctly. So in this situation, this is another objective reward— reward. And through this, you can directly make better improvements in some more objective areas. Also recently, including more agentic tasks, which can all be verified, to see if the final goal has been achieved. And this method will also be more objective, that is, more objective.

Uncle Moon (Host) (00:38:43)

So, can I understand it this way? RLHF makes the model perform well in the chatbot aspect. But to truly become tool-oriented, becoming a coding agent, a web agent, etc, it truly needs a new paradigm, RLVR.

Banghua Zhu (Guest) (00:38:58)

Hmm, that's how it can be understood. Right. It's just that RLHF, for example, will also make a model now, The last stage of this— this training— makes it more aligned, for example, with human values, and, then making it safer, for example— all of that comes through RLHF. This kind of reward model signal will be given. Yes.

Startup timing, acquisition, and NVIDIA

Uncle Moon (Host) (00:39:18)

Understood. You just explained these two important RL paradigms. Okay, let's go back to the startup process. What was the most difficult moment for Nexusflow AI? What was the most difficult moment? How did you get through it?

Banghua Zhu (Guest) (00:39:30)

I think the first experience was too short, so it probably ended quickly. So there weren't any particularly difficult moments.

Uncle Moon (Host) (00:39:37) †

I understand, it was too smooth, right? Okay. So, in your first startup, did you learn any entrepreneurial insights, for example, cognitive aspects, that you want to apply to your second startup?

Banghua Zhu (Guest) (00:39:45)

Hmm, I think one idea is that, Hmm, many times, Hmm, doing the right thing, it must be at the right timing. Actually, it means that at that time, for example, in 2023, we often told everyone, When we were doing agents, people would ask what an agent is. Is it just— RAG is Retrieval Augmented Generation. Is it just that I search this text and then give you a response? So we actually had to spend a lot of time, time to educate the market, To tell everyone that an agent is actually a model that can take action. It can use tools, and it can use these tools well in multi-turn conversations. It can use tools, and use the tool effectively over multiple turns. The whole market wasn't particularly bad at that time. So this thing, although relatively speaking, we might have predicted the trend correctly, but it might have happened a bit later, but this might have happened a bit later.

Banghua Zhu (Guest) (00:40:29)

So because of this misalignment, it actually led to— meaning you might not be able to enter the hottest market right away, meaning, you might not be able to enter the hottest market right away. So on this matter— So on this matter, this is probably an interesting lesson we learned from our first startup— lesson. Hmm. This acquisition was not a surprise. I actually knew about it quite early lesson. Hmm.

Uncle Moon (Host) (00:40:43)

Because I've also started a business, I really agree with this statement. Entrepreneurship is about finding the right people at the right time to do the right thing, right?

Banghua Zhu (Guest) (00:40:49)

Yes, yes, exactly.

Uncle Moon (Host) (00:40:51)

Yes, so, many tech experts at the beginning, might be too visionary and enter the market too early. But they have to personally educate the market too much. Later, they found that it was actually the third or fourth to enter the market that survived, And lived well, right?

Banghua Zhu (Guest) (00:41:09)

Yes, it is. Yes, it's often like that.

Uncle Moon (Host) (00:41:11)

So, let's talk about the whole acquisition process. For example, on the day the offer was signed, what was your mental state like? And what did you do?

Banghua Zhu (Guest) (00:41:19)

Actually—

Uncle Moon (Host) (00:41:21) †

the 30B model was also trained.

Banghua Zhu (Guest) (00:41:22)

This acquisition was not a surprise. I actually knew about it quite early. And so, at that time, that day was relatively calm. And so, at that time, that day was relatively calm. And I just felt like it was a new beginning. And it was indeed at that time that I was very much looking forward to— I was eager to go from GPU-poor to GPU-rich.

Uncle Moon (Host) (00:41:40)

Yes, how many GPUs did you have when you started your business?

Banghua Zhu (Guest) (00:41:40)

Maybe two to four nodes, which was quite— two to four nodes was already a rather extravagant state. It was a very ambitious state.

Uncle Moon (Host) (00:41:49)

What, four nodes?

Banghua Zhu (Guest) (00:41:50)

H100, that's 32 cards. Yes.

Uncle Moon (Host) (00:41:52)

Ah, that's amazing.

Banghua Zhu (Guest) (00:41:54)

Yes, at that time, we could train 70B.

Uncle Moon (Host) (00:41:57)

After you were acquired, joining NVIDIA for those few months, what was your personal experience like? Can you tell us what this project is specifically for?

Banghua Zhu (Guest) (00:42:03)

NVIDIA is a very different experience. And it's also very interesting. I was mainly involved in the initial effort called NeMo Gym. Called NeMo Gym. The idea at the time was that NVIDIA is a very large organization. And actually, everyone has all sorts of good ideas and good data. We hope to create something similar to OpenAI Gym, an NVIDIA Gym, so that we can collect all the— and collect all the internal agentic data and ideas And then use it for training our own models. This started as an effort by our small team. But later, it gradually became a company-wide effort. But later, it gradually became a company-wide effort. It became a company-wide effort, meaning every team— Every team wanted to put its data into the Gym. And then these Gym data are used for training. Then, many people participated in this project. Everyone contributed various interesting reinforcement learning environments, And even some SFT data. I think that experience was very positive.

Banghua Zhu (Guest) (00:43:02)

The experience was very positive, and everyone felt that, even in such a large company, many people can work together on something interesting.

Uncle Moon (Host) (00:43:10)

Can you tell us what this project is specifically for?

Banghua Zhu (Guest) (00:43:13) †

Oh, right, NeMo Gym. NeMo Gym is mainly a— it's currently a— an Agent data collection. We eventually wanted to make it a more solid Agent Infra, and ensure that during post-training, you only need to interact with NeMo Gym. NeMo Gym will collect all the data, And help you decide how to train the current model with what kind of data, And how to optimize the entire agentic— the infra side. Of course, this is a rather ambitious target. Currently, it has only done the data collection part, this part. But it has already been a huge help for RL on the Nemotron model.

Uncle Moon (Host) (00:43:52) †

Okay, I understand. So, during those months at NVIDIA, what do you think was the biggest difference compared to your previous startup? What was the biggest difference?

Banghua Zhu (Guest) (00:43:59)

Well, there were too many cards at that time. Though it still wasn't quite a frontier AI lab, compared to startups, it's indeed a world of difference.

Uncle Moon (Host) (00:44:07)

So, for a top researcher like you, what does having many GPUs mean?

Banghua Zhu (Guest) (00:44:12)

Well, actually, many times, from a system perspective, your system— many systems— setting it up, if you train on a few nodes, it's actually quite easy. Or if you're training a model of about 100B, it's actually not that complicated. But once it reaches for example, trillion-level parameter count, Once it reaches for example, thousands or even tens of thousands of cards, The requirements for the entire system's infrastructure are completely different. You need to do a lot of different fault tolerance, and you need to ensure that after all these are done, your accuracy can still match— after all these are done, your accuracy can still match. and you need to ensure that, for example, the entire— the fleet's MFU is— the MFU across the fleet is relatively— quite good. So, the system challenges it needs to address are much greater. Right.

Uncle Moon (Host) (00:44:56)

So this can only be experienced with large-scale GPUs in big companies, and you didn't really experience it in your previous startup, right?

Banghua Zhu (Guest) (00:45:02)

Yes, that's why, I think many so-called frontier labs, everyone— everyone thinks that people from frontier labs will be a relatively rare talent. This is because people are trained by cards. That shows people are shaped by the GPUs they get to use. And many times, if you've touched many cards, you'll probably know how to train such a large-scale model, large-scale model. But if you only do very small-scale training, then your experience cannot generalize to large scale.

Uncle Moon (Host) (00:45:29)

What you said echoes Shunyu Yao's recent— an interview with Yao Shi Yu a while ago. As long as undergraduates are placed in this large-scale GPU environment, undergraduates can also do these things, right?

Banghua Zhu (Guest) (00:45:38)

I think so. Even undergraduates sometimes do better than PhDs. Because sometimes PhDs, after too much training, they might become too rigorous, or they might be reluctant to YOLO. But actually, many times, the truly good runs might come from YOLO.

Uncle Moon (Host) (00:45:54)

Let's explain in detail: what is YOLO? You previously—

Banghua Zhu (Guest) (00:45:57)

YOLO means that, I think, it's based on intuition. I think this run is good. Then I'll just launch it directly, and I won't say that: I have to be very meticulous, I do a lot of ablations. Because this is still the same as before, relying on intuition to make relatively correct choices, and then in this very high-dimensional search space, to find the best data and the combination of algorithms and infra. Right.

Uncle Moon (Host) (00:46:22)

So you also support giving enough GPUs to young people to let them YOLO, right?

Banghua Zhu (Guest) (00:46:27)

I think training is like this. Right.

Uncle Moon (Host) (00:46:30)

Understood. So, you started a new company so quickly. Your family, your co-founders, what do your NVIDIA colleagues think? Separately.

Banghua Zhu (Guest) (00:46:39)

Hmm. As for family, I— my wife and parents are also very supportive. Yes, to make them understand, that is to say, my motivation is not from a financial perspective. Instead, we feel that this is something very worthwhile to be done, and if it is done, it may have a very, very big impact. And former business partners, actually this— there were some attempts to keep me, but everyone understood very well. That is to say, how to say, it feels like the times are calling, it is indeed necessary to do something more interesting. Yes, and NVIDIA was very supportive as well. Yes. Hmm.

Uncle Moon (Host) (00:47:17)

The whole thing was a very exciting first startup, actually, right?

Banghua Zhu (Guest) (00:47:21)

It's very interesting. Yes, I gained a lot of experience.

From theory to systems

Uncle Moon (Host) (00:47:24)

Understood. So let's pause our startup discussion here. Let's move on to the second part, which is Banghua's research. Because Banghua is actually a PhD in reinforcement learning theory— PhD. This PhD circle has an unwritten pecking order, have you heard of it?

Banghua Zhu (Guest) (00:47:42)

Uh-huh.

Uncle Moon (Host) (00:47:43)

Those who do theory look down on those who do algorithms, and those who do algorithms look down on those who do infra. You know, everyone says that: how could a theory master suddenly start doing Infra? to start working on infra? And you keep emphasizing infra, infra, infra. What made you— Oh, so you're also the kind of researcher who hopes their work can impact the real world.

Banghua Zhu (Guest) (00:48:00)

Actually, I think this change happened after the GPT release. In the past, everyone used to do machine learning, the scale was relatively— relatively small. The most important infra might be the recommendation system's infra. The size of such models compared to current models, is actually many orders of magnitude smaller. This means that the problems within the infra are not that complex, not that big, it hasn't reached such a large scale. And in that era, in many cases, algorithms and theory could still get you a lot of model training and the entire development of machine learning. So at that time, especially when I first entered Berkeley, for example, for example, Mike Michael Jordan, who is my current— my boss at the time, or Martin J. Wainwright, were all like this for example, when I entered Berkeley, a supervisor everyone wanted to follow. Yes, and then, After LLMs emerged, this problem has undergone a fundamental change.

Banghua Zhu (Guest) (00:48:54)

Because LLMs kept getting larger, everyone found that as long as you scale data, scale this infra as- scale the entire model parameter size, it can make the model smarter and smarter. At this time, doing too much theory, saying I predict how smart it can become, this actually doesn't make much sense. Instead, the most difficult problem has become: how do I make this happen with the lowest cost? Because if I can train a very, very large model for 10 billion, but if I have very good infra, I can reduce this cost to 5 billion. This will have a much greater direct impact on the world than doing some theoretical proofs. It will be much larger. So it gradually became that what people who do Systems do will become more and more important. And so I think, essentially, it's a supply and demand issue. And so I think, essentially, it's a supply and demand issue.

Uncle Moon (Host) (00:49:42)

Oh, so, you are also the kind of researcher who hopes that what you do can have an impact on the real world.

Banghua Zhu (Guest) (00:49:51)

Yes, this is for sure.

Tsinghua, computational biology, and Berkeley

Uncle Moon (Host) (00:49:53)

Understood. I'm sure you have many stories from your Berkeley PhD. Let's start from the beginning. How did you get into Tsinghua? And what was your entire student life like at Tsinghua? What kind of research did you do at Tsinghua?

Banghua Zhu (Guest) (00:50:07)

Hmm. Yes, I entered Tsinghua. I am from Lianyungang, Jiangsu Province. At that time, northern Jiangsu was relatively backward, and then, so actually, not many people from one school could get into Tsinghua. Right, and—

Uncle Moon (Host) (00:50:21)

Not many people from Jiangsu get into Tsinghua, right?

Banghua Zhu (Guest) (00:50:23)

The competition in Jiangsu is fierce, and northern Jiangsu is relatively backward. Yes, and at that time, I was lucky enough to go to the Department of Electronic Engineering at Tsinghua. Yes, and then I did some communication-related research there. In my junior year, I went to Stanford for an exchange program. I did UGVR, which was a visiting program at Stanford. And that summer, I was with Stanford EE a professor named David Tse, who originally worked in communications. He's considered a pioneer in the field of communications. He did a lot of very foundational work. But at that time, his interest suddenly shifted to Computational Biology. That's computational biology. So, I worked with him on computational biology for a while. This led to a shift in my interests later on. When I applied, I applied for computational biology. Ah, I almost got an offer from Stanford Medical School, an offer. Yes.

Banghua Zhu (Guest) (00:51:16)

Then, But anyway, when I eventually entered Berkeley, it was also through computational biology. Yes.

Uncle Moon (Host) (00:51:25)

Then why? Hey, wait a minute, the amount of information is a bit overwhelming. First, let's talk about what UGVR is.

Banghua Zhu (Guest) (00:51:32)

UGVR is actually a— It's— Stanford's Undergraduate Research Program. At that time, several universities in China and Stanford co-organized it. to give everyone the opportunity to do summer research at Stanford, summer research. I'm very grateful to the UGVR teachers and sponsors for their help. Yes.

Uncle Moon (Host) (00:51:54)

Understood. Hmm. You said that not many people from northern Jiangsu got into Tsinghua. What do you think made you get into Tsinghua? What do you think made you get into Tsinghua? Was it genius intelligence? And studying day and night, or a good family environment, what do you think?

Banghua Zhu (Guest) (00:52:06)

Um. First of all, I don't consider myself particularly hardworking, and intellectually, I didn't do well in various competitions before. It wasn't very good. But I think I have a special point, which is: Um, my— If I use large language models as an example, my— Um, ability to handle long contexts is quite strong. For example, I'm very good at exams. For example, whether it was the college entrance exam or at Tsinghua, I was basically first in my department. But, you know, I'm the type of person who for example, if I had a solid-state physics exam the next day, I usually didn't study much before, and I was pretty laid-back. Then on the last night, I would go— or a day or two before, I would cram, and read through the book. Then the next day, I would take the exam, and I should do pretty well. But after the exam, I would forget everything.

Banghua Zhu (Guest) (00:52:56)

So, it's hard for me to internalize and train. The positions in my brain don't really change. the weights in my brain don't really change. I've basically completely forgotten what I learned in the electronics department. But if I were to study solid-state physics again now, I've basically completely forgotten what I learned in the electronics department. I might be able to pick it up again in two nights. I might be able to pick it up again. So I'm the kind of person who can handle a large context. But I basically can't train anything into my brain. I don't really understand it. Yes.

Uncle Moon (Host) (00:53:23)

Why is this? How was this done?

Banghua Zhu (Guest) (00:53:26)

Um, yes, it is.

Uncle Moon (Host) (00:53:28)

This is truly amazing. Right, so let's go back to that— your journey from Tsinghua to Berkeley. How did you start with computational biology, and how did you then switch from computational biology to the very famous Michael Jordan? the very famous Michael I. Jordan? That statistics department, right?

Banghua Zhu (Guest) (00:53:45)

Actually, this— The opportunity to do computational biology was also because I felt at the time— I got into computational biology because I thought— it's hard to say. Through Deep Learning, to do some genome prediction, genome assembly, and even how to do some— genome assembly, and even how to do some exploration of human and biological genes and medical aspects for humans, might have a more direct impact. it might have a more direct impact. So I was doing this at the time. I thought it would be more interesting to do this during my undergraduate studies. It was also during the early days when AlphaFold had just come out. So that period was very exciting. And I got to learn about statistics and machine learning theory because when I first entered Berkeley, it was my boss at the time, Jiao Jiantao. And he had just joined Berkeley as a faculty member.

Banghua Zhu (Guest) (00:54:38)

I knew him before because he was also an alumnus of our electronics department. Then I talked to him, asking if we wanted to do some research together. And he was doing statistics, information theory. He was a very strong person at the time. statistics, information theory, a very strong person at the time. So I collaborated with him on some statistics, Information theory related content. And at that time, there was another person, a faculty member who joined Berkeley around the same time, named Jacob Steinhardt. I collaborated with them a lot These statistics and this machine learning— These related papers. And later, through this opportunity. I gradually embarked on the theoretical path of no return.

Research culture, mentors, and defining work

Uncle Moon (Host) (00:55:22)

So what do you think is the biggest difference between your student life at Berkeley and Tsinghua? Personal experience.

Banghua Zhu (Guest) (00:55:28)

I think Tsinghua or domestic schools, people generally tend to— Everyone measures a person in a more low-dimensional way. Just a few points: for example, what's your GPA, and whether you've won any special awards, and how many social work activities you've participated in, etc. But when you start your PhD at Berkeley, you find that everyone has their own unique skills. And no one tried to reduce you to a single rank, to compare this person, he just has a higher score than this person. Instead, you can actually interact with all sorts of eccentric geniuses, and everyone actually has their own unique skills and strengths. And then you'll actually find— By spending time with people in that cohort, you'll find that you can learn many interesting things from everyone. Yes.

Uncle Moon (Host) (00:56:16)

Hmm, give an example of the most eccentric genius you've met at Berkeley.

Banghua Zhu (Guest) (00:56:21)

I can mention a faculty member, Jacob Steinhardt, whom I just mentioned. He's a very interesting teacher, I think. He's from the statistics department. And in his first year, he worked with me on many of his own PhD thesis projects called Robust Statistics. This translates to "robust statistical learning" in Chinese. Then we wrote a paper of nearly 200 pages, and submitted the paper to The Annals of Statistics. At that time, it was also the best journal in statistics. But in his second year, he— I originally wanted to continue collaborating with him, but he suddenly said his interests had shifted, and he wanted to do Science of Machine Learning. At that time, it seemed like metaphysics to me. He was doing interpretability and some language model evaluation at that time. At that time, I thought this thing doesn't make sense.

Banghua Zhu (Guest) (00:57:04)

His paper at the time It was about MMLU, and it was about asking language models common sense questions, and having the language model answer. What was this language model, I wondered? And having the language model answer. You're not doing theory properly, and you're going to do these things. I found this truly incomprehensible. And later, MMLU became the golden standard for large language model evaluation. And I think, actually, that now, in research, I think they are very far-sighted. I mainly think— In retrospect, I think they were truly visionary.

Uncle Moon (Host) (00:57:34)

So, for example, with these teachers, Michael Jordan, Jiao Jiantao, what do you think influenced you the most during your collaboration? At that time, I understand that deep learning was very popular, right?

Banghua Zhu (Guest) (00:57:42)

I mainly think, observe what kind of problems the top people in this field are thinking about, what everyone's taste is like, what kind of problems everyone thinks are important, and what kind of problems will even influence the future, for example, problems in five or ten years. But their judgment may not be correct, but they have their own set of logic. That is, how to use this to deduce what will happen in the future through their own logic What will happen in the future. I think this is probably the most important one, Skill, right. I think this is probably the most important one, Skill, right. Of course, in terms of technology, from them I learned a lot. Because I'm not a theoretical person at all, so basically, if it weren't for them, I wouldn't have been able to— I wouldn't even have been able to take the first step. Right Yes, that's what it was at the time, yes.

Uncle Moon (Host) (00:58:25)

Other students were all doing various deep learning.

Banghua Zhu (Guest) (00:58:29)

Well, it's actually fine.

Uncle Moon (Host) (00:58:31)

CVPR, and all sorts of other things. Publishing many top conference and journal papers. crazy. publishing top conference and journal papers. Did you feel any pressure then? During your PhD, Quantity not being important is a topic that everyone has been discussing.

Banghua Zhu (Guest) (00:58:41)

We also published some at that time. Yes, we were all just following the trend. And then, I think at that time, academia was slowly— it wouldn't particularly look at this How many papers or top conferences you published. But in the end, it still depends on what you actually— you, people don't look at the average or the number— you— People don't look at the average or the number, but at the maximum, which is your most impactful work. If your entire PhD only has one piece of work, one paper that everyone knows about, and everyone uses, then that's a star PhD. It's not about having 10 CVPR or NeurIPS papers every year to prove you're the best.

Uncle Moon (Host) (00:59:18)

Really, right? Quantity is not important, that's a topic everyone keeps bringing up Right?

Banghua Zhu (Guest) (00:59:23)

Yes, you must do your best.

Uncle Moon (Host) (00:59:24)

Yes. Just one star— star work is enough.

Banghua Zhu (Guest) (00:59:27)

Yes, it is.

Uncle Moon (Host) (00:59:28)

Then what do you think, for example, at that time, Collaborating so much with Michael Jordan, because he's also a grandmaster, Right? What do you think made him grow step by step to such a level?

Banghua Zhu (Guest) (00:59:41)

Yes, I think there were a few points that I found very admirable. One point is that Mike was quite old at the time, but he would attend the weekly group meetings. We would have a two-and-a-half-hour reading group. He would always show up, unless something very important came up. He would calm down and read— reading things he didn't understand and he wanted us to read together. And this— We mainly studied Optimization for those years, and even Economics, because Mike thought, human-AI interaction was very important. And in this, it's no longer a problem of machine-to-machine interaction, but how machines and humans collaborate. This problem essentially requires machines to understand humans This is essentially an economic problem, so he started reading economics books. We followed him for several years, studying econ theory. This including game theory, and also some auction theory. This was actually very interesting at the time. I even wrote many papers on this topic.

Banghua Zhu (Guest) (01:00:41)

And I think Mike's interests often shift. He started out with a PhD in psychology, then he gradually learned statistics. Then he started doing statistics, then machine learning, and now he's exploring economics again. So he himself is also someone who is constantly learning. But he never thought that being in machine learning meant he should only work on machine learning. On the contrary, he is willing to absorb new knowledge at his age. So I think this is what makes me very impressed— impressed. Even when I'm 60 or 70, I might not have his experience and mindset to keep learning, keep learning new things. Yes Keep learning new things. Yes.

Uncle Moon (Host) (01:01:23)

Was Hinton also a psychologist?

Banghua Zhu (Guest) (01:01:26)

I guess so.

Uncle Moon (Host) (01:01:26)

Yes, mm. Are these people the type who always stay curious, always keep learning? That kind of state.

Banghua Zhu (Guest) (01:01:32)

Yes, that's right. So I really, really admire Mike.

Uncle Moon (Host) (01:01:36)

Because if you don't enjoy the learning process, it's actually very painful. Because learning is like reshaping the neurons in your brain. Because learning is like reshaping the neurons in your brain, right?

Banghua Zhu (Guest) (01:01:44)

Yes, it is.

Uncle Moon (Host) (01:01:44)

And they can still be learning up to now, that's true. That's true. They are naturally the kind of people who should do research, right?

Banghua Zhu (Guest) (01:01:51)

Yes, that's right. I'm not that kind of person, because I forget things as soon as I read them. So, the books Michael and I read back then— But I still vaguely remember those philosophies and some of these principles behind them. I didn't—

Uncle Moon (Host) (01:02:07)

Yes. But even so, you're still constantly publishing papers, right?

Banghua Zhu (Guest) (01:02:11)

publish many papers. Yes, I prefer to calm down and write. Yes Hmm, yes, at that time, it was also this—

Faculty positions, PhDs, and research resources

Uncle Moon (Host) (01:02:18)

Well, at the end of your PhD, you basically start looking for a job. And at that time, you chose to go to the University of Washington to find a faculty position, right?

Banghua Zhu (Guest) (01:02:27)

um, our cohort, this batch of PhDs, everyone— applied to both industry and academia. In various— So, including me, including Yin, Lianmin, everyone— including Ying, and Lianmin— we all went on to Applied for a faculty position, yes.

Uncle Moon (Host) (01:02:48)

So in the end, you didn't go, and then you started a business, right?

Banghua Zhu (Guest) (01:02:52)

Still in a deferred state. Yes. Hmm Yes. Hmm.

Uncle Moon (Host) (01:02:54)

Yes, and the school was quite supportive of you, right?

Banghua Zhu (Guest) (01:02:57)

The school is quite supportive, yes.

Uncle Moon (Host) (01:02:59)

Understood. So this postponement of starting the job was because you saw the opportunity with ChatGPT, which we just talked about, right?

Banghua Zhu (Guest) (01:03:07)

Hmm, yes, that's right. In academia and industry, there are indeed significant differences in resources and other aspects. So, often, if what you want to do requires substantial resources, it might be difficult in academia, especially for a relatively young faculty member. It might be quite difficult. Yes.

Uncle Moon (Host) (01:03:24)

So if you were to give advice to students who just started their PhD, what advice would you give? Honestly, what advice would you give them?

Banghua Zhu (Guest) (01:03:32)

I think you shouldn't put too much pressure on yourself, because what you're doing now and what you might be doing five years from now are completely different. So I actually think that if you are now For example, in the past one or two or three years, you feel you have achieved nothing, I don't think that's a big problem. I just feel, for example, including me, it was also in my fifth year, I decided to change direction to achieve more noticeable results. And even Jacob, whom I just mentioned, I don't know why I've called him so many times today— But he also started doing it in his fifth year, truly relevant Robust Statistics papers, and also in his fifth year, he found Berkeley. When he was on the market, based on that fifth-year paper, he found a faculty position at Berkeley.

Banghua Zhu (Guest) (01:04:07)

So many people I know everyone is constantly exploring, And then slowly when they explore some good opportunities, and at that time, that one time or that one piece of work, made themselves very, very— That is to say, let themselves elevate the level of their work, right.

Uncle Moon (Host) (01:04:30)

Can it be summarized like this? Keep exploring, and find your interests, while waiting for that opportunity.

Banghua Zhu (Guest) (01:04:36)

Yes.

Uncle Moon (Host) (01:04:36)

there's a saying in games called "version hitting you in the face".

Banghua Zhu (Guest) (01:04:39)

Ah, yes, that's roughly what it means. Yes, but you also need to judge when the time is right for a new version.

Uncle Moon (Host) (01:04:44)

What does that mean?

Banghua Zhu (Guest) (01:04:46)

That is to say, for example, this wave of Language models, the truth is, many people would feel for example, that it's not relevant, Or it's not a particularly hot topic. or it might die down in a year. In that case, you might not invest in this direction, and you might miss out on some things.

Uncle Moon (Host) (01:05:06)

Understood, so when you see an opportunity, you still have to Seize it, yes. If there are students now struggling with whether to do a PhD, would you recommend industry or a PhD?

Banghua Zhu (Guest) (01:05:16)

Well, I think if there's a good opportunity, it's also worth considering going to places with more resources. Take a look. And because studying for a PhD is a long-term process. And if you're really interested, you can study at 30, 35, or even 50, 60 years old, and it's still fine to get a PhD. But on the contrary, I think now is an AI revolution era. I'm not sure if it will have as big an impact as the Industrial Revolution, but I think there will be big changes in the next few years, and transformation. Well, for example, the most intuitive example is that if you're at a company, you may have unlimited Claude Code, you can use 20,000 dollars or more of these tokens per month. You can explore how to make this agent work more efficiently for you. But if you're at school, if you spend 20,000 dollars a month, you might burn through all your advisor's funding.

Banghua Zhu (Guest) (01:06:05)

So I think from this perspective, it might be better to go to places with more resources to take a look, and see some of the cutting-edge AI stuff. I think it's actually quite important. Uh, yes, that's roughly what it means, yes. Yes.

Uncle Moon (Host) (01:06:21)

So your criterion is actually the abundance of resources, the number of GPUs, how much Claude Code is available, because these things can nurture people, right?

Banghua Zhu (Guest) (01:06:28)

Yes, it is.

Uncle Moon (Host) (01:06:30)

Yes, if you stay in school, if you don't have enough resources to support yourself, in the end, you won't develop the skills needed in industry, or even academia.

Banghua Zhu (Guest) (01:06:40)

Or especially for this AI and Language model direction, it is indeed so. I think it is, yes.

Uncle Moon (Host) (01:06:45)

So it goes back to the earliest topic, that people are nurtured by the environment and opportunities, right?

Banghua Zhu (Guest) (01:06:52)

Yes, yes, it is.

Uncle Moon (Host) (01:06:53)

Yes, it feels like once you've succeeded in starting a business, or people who have done well, it's hard to deny that statement, right?

Banghua Zhu (Guest) (01:06:59)

Uh, actually there are many—

Uncle Moon (Host) (01:07:01)

All right.

Continual learning, multimodality, and training

Uncle Moon (Host) (01:07:02)

Let's move on to the next big section, which is about future prospects. Let's start with the first one. The wave of large models, do you think it's just beginning? because you just mentioned it. So what do you think will be the most important issues in the next one or two years?

Banghua Zhu (Guest) (01:07:17)

if you look offline, there are many, many new labs spinning up. many are dedicated to solving continu-continuous learning or similar problems. And the definition of this problem itself is not particularly clear, what kind of— what kind of scenario and this paradigm can be defined as continual learning. And, for example, some people now say that Anthropic— they actually, in some sense, have also achieved continual learning, because they are now using Claude Code— continual learning, because they're now using Claude Code, quality, and then train stronger models, improve their data quality, and then train stronger models. And if there really is such a turning point, that this thing will exponentially explode, then there might be a company, that suddenly has a very big leap in intelligence. So this might be a very interesting question: will this turning point appear? So this might be a very interesting question: will this turning point appear? And if it does, who will reach this turning point first?

Uncle Moon (Host) (01:08:17)

So do you believe in this wave of self-evolution? You mean continuous learning?

Banghua Zhu (Guest) (01:08:21)

I hope he is. I believe it, or I want to believe it. Yes, but it might also be limited by for example, resources, for example, energy constraints. So even if it's achieved, it won't be able to explode to what we most want it to— reach the level of ASI.

Uncle Moon (Host) (01:08:40)

So what if we look further ahead? For example, in five years? So it's not just pure text.

Banghua Zhu (Guest) (01:08:44)

In five years, I think pure text language models might become less relevant— less relevant, and everyone will likely move towards native multimodality, even omni, meaning any modality, of modality, input and any modality output. And Robotics might also have more new progress. New progress.

Uncle Moon (Host) (01:09:06)

next is multimodal, and then embodied AI, robotics and such. Understood.

Banghua Zhu (Guest) (01:09:11)

I think pure text's scaling might, it— might not take so many years to fully understand. But for more modalities, the new data, new interaction, and new training paradigms it brings might have more new problems to solve. There are more and more new problems to solve.

Uncle Moon (Host) (01:09:28)

Understood. So the current training paradigm for large models is still pre-training, SFT, and post-training, right? How much potential do you think each stage of this still has to be unleashed? Yes, share your opinion.

Banghua Zhu (Guest) (01:09:41)

I think people from different labs from Frontier Labs— Yes, I think everyone has different ideas.

Uncle Moon (Host) (01:09:45)

Yes, so now—

Banghua Zhu (Guest) (01:09:47)

Yes, I think, on the pre-training side, Um, that is, how well the data is cleaned, this is very, very important. And this thing, currently, it seems there isn't one yet— hasn't hit a war yet. If you can clean the data better, you have more high-quality coding data, for example, for example, in the direction of coding, if you have more coding, high-quality coding data, then the model's capability boundary can actually be improved. pre-training's design generally determines how well you can do post-training. It actually determines a boundary. And it actually determines a boundary. And then, SFT or— And then, SFT or, post-training: small fix for pre-training, small fix.

Banghua Zhu (Guest) (01:10:30)

Or to allow the model to have some of this, Um, for example, simple instruction following, or some simple, Um, use slightly higher quality data to remove some noise from the pre-trained model, and then do the actual RL run or post-training to push the model's capabilities to the boundary, to the boundary defined by pre-training. And, even now, many people are doing on-policy distillation. It's about whether we can merge a few very good experts from different fields— merge, back into such a good, well-rounded generalist model, into a well-rounded generalist model in the end.

Open-source RL, large-scale training, and platforms

Uncle Moon (Host) (01:11:08)

for example, because you've been helping various companies with RL infra, how much room for improvement do you think this set of things still has? How much room for improvement is there? The gap is so big.

Banghua Zhu (Guest) (01:11:17)

You mean the RL infra itself? or just?

Uncle Moon (Host) (01:11:20)

Just RL.

Banghua Zhu (Guest) (01:11:22)

As for RL, I actually think, there's one point, which is that there's still a big difference between top AI Labs and open source. there's still a big gap between Frontier Labs and open source. So, on the open source side, there might actually be more worthwhile things to do. And it might be something that many companies don't have yet. And it might be something that many companies don't have yet. But if it does exist, people might have, more opportunities to be empowered to be closer to the front. People might have— line.

Uncle Moon (Host) (01:11:42)

How do you understand this part?

Banghua Zhu (Guest) (01:11:45)

a Frontier Lab-style setup.

Uncle Moon (Host) (01:11:45)

How should I understand this? So you think the open-source community doesn't have these conditions yet, right?

Banghua Zhu (Guest) (01:11:50)

for example, the reinforcement learning environment, its corresponding Infra and data quality, and also the various details of the RL Infra itself. And also the various details of the RL Infra itself. If you haven't verified it on a very large scale, then this Infra itself, may not be able to reliably string together, for example, a hundred-billion-parameter model— to train one, say, a trillion-parameter model. And often you need many cards to verify this. And then you can understand this by running many different cases, you have to run many different cases, you can understand whether it is truly reliable. Will I have some small problems? many corner cases we hadn't found before, it may cause some deviation in your accuracy, or even lead to wrong conclusions. Yes.

Uncle Moon (Host) (01:12:32)

So, for example, you are helping some companies complete the construction of ten-thousand-card post-training through RadixArk.

Banghua Zhu (Guest) (01:12:36)

This is also partially, why we hope to do this through RadixArk, a reason to get this done. Because in itself, actually, From an open-source community perspective, it's hard to support for example, a thousand-card testing, or ten-thousand-card testing. But through some business opportunities, you can actually let more people access these thousand and ten-thousand cards. This way, this access can be better democratized. Yes. Yes.

Uncle Moon (Host) (01:13:06)

and you will gradually release these things. You use RadixArk to help companies build post-training systems at ten-thousand-GPU scale. Yes. So with so many things being released, how do you make your business model…

Banghua Zhu (Guest) (01:13:19)

Miles has already experienced thousand-card level testing. It's actually very reliable. Of course, for ten thousand cards, it may require more fault tolerance, And more optimization of the acquisition layer. So this part, is still a direction everyone is working towards.

Uncle Moon (Host) (01:13:34)

more sustainable? So this is also a trick you use to ensure your business model, More sustainable.

Banghua Zhu (Guest) (01:13:40)

For this part, we consider that, we hope, to minimize the cost of accessing scenarios that people use in their daily lives. But in very extreme scenarios, the extreme optimizations it needs, sometimes, open-sourcing it won't really help the entire community. It will be very helpful.

Uncle Moon (Host) (01:13:59)

OK, so this is also a trick you use to ensure your business model.

Banghua Zhu (Guest) (01:14:03)

Hmm. It isn't necessarily a business-model tactic; it's just that even if this part is open-sourced, not many people will use it. Yes.

Uncle Moon (Host) (01:14:12)

OK, so, As just mentioned, for example, robots, whether in China or Silicon Valley, there are many robot companies. So for them, for example, your, mentioned SGLang inference engine— Is it helpful? What do you think?

Banghua Zhu (Guest) (01:14:31)

Yes, there will definitely be some. That is to say, it also depends on the specific use case of the robot. Hmm, if it is, Hmm, For example, some dialogue forms on robots, or you want the robot to do a VLA, and then let it perform some reasoning based on the VLA. These may require corresponding hardware, and this new model architecture, and this new, for example, its pre-feel— say, if its prefill… But eventually, many underlying logics of its inference engine, such as the kernel, and the compiler, can all be reused. The robot side is still quite early, yes.

Uncle Moon (Host) (01:15:08)

Yes. Do you have similar collaborations now?

Banghua Zhu (Guest) (01:15:11)

Hmm, possibly, but it also depends on how ultimate your ultimate goal is.

Uncle Moon (Host) (01:15:14)

Understood. So, looking long-term, for inference, What will be the ultimate outcome for Infra and RL Infra? Can it be understood that, in the future there will be a DataBricks or Snowflake-like Infra company, helping all kinds of great companies worldwide with Infra management, something like that?

Banghua Zhu (Guest) (01:15:38)

Hmm, possibly. I think at some state there will be such a company; it's more like a platform, and then to enable and empower more people to build frontier-level AI, build frontier-level, this AI, and even use frontier-level infrastructure to do and inference. But even after that, what will happen? I think that's actually more interesting. Is it possible that this will redefine— how to redefine— what will the agent look like then? And how will agents interact with people? And the interaction between agents, And agents, interaction. And what will be the difference between this agent and human interaction? Does it need to empower all these personal agents? And will all these personal agents' interactions, and the interface for human interaction, all need to be defined by one company? Then I think this might be some more interesting questions in the future. So on the one hand, RadixArk wants to do this, to become a platform.

Banghua Zhu (Guest) (01:16:31)

So one thing we at RadixArk want to do is: Infra First, Infra First frontier lab to explore more cutting-edge new problems. Yes. more cutting-edge new problems. Yes. It may not be several companies, it may not be several companies, but possibly in the end…

Uncle Moon (Host) (01:16:46)

I roughly understand. Recently, many companies, for example, those doing Agent auctions: I post a task, and each company builds its own Agent to take on such tasks. It's like outsourcing to Agents instead of people. Agent,

Banghua Zhu (Guest) (01:17:04)

uh-huh.

Uncle Moon (Host) (01:17:05)

And what you just said is: Is it possible for RadixArk? Such inference Infra or RL Infra to ultimately support, several companies' operations, right? operations, right?

Banghua Zhu (Guest) (01:17:18)

but possibly in the end, the way all companies in the world operate will be through agents. but possibly in the end, the way all companies in the world operate will be through Agent. At that time, At that time, it also needs a company similar to a power company to empower everyone, a company like that to empower everyone. Then we might hope, for example, that this company is a bit hard— say, that company is RadixArk. Of course, what you just mentioned. agents. But in fact, there was also the open cloud-derived community before, where agents hired people. agents. But there has also been this OpenClaw-derived community. So this, everyone is exploring new ways for agents and humans to interact. For example, an agent will say how much money I will pay. For example, an agent might say how much money I'm willing to pay. Then it says to send someone to help me do something in a place.

Banghua Zhu (Guest) (01:17:56) †

And this might be more interesting.

Uncle Moon (Host) (01:17:59) †

Like having people run errands.

Banghua Zhu (Guest) (01:17:59) †

Yes.

Uncle Moon (Host) (01:18:02)

So I roughly understand your vision: that in the future, human-Agent collaboration will be as natural as using electricity now. Then, RadixArk, such a company will be the power plant.

Banghua Zhu (Guest) (01:18:15)

Hmm, hopefully. Yes.

Uncle Moon (Host) (01:18:17)

Ah, this, vision, it's the first time I've heard you talk about it. So I feel it's quite grand.

Banghua Zhu (Guest) (01:18:24)

I think, This is just one of countless future possibilities. And this might not happen at all. It's also possible that what actually happens is, For example, at some point, Anthropic or OpenAI could dominate. And then the AI game ends. We actually hope to see an era where a hundred flowers bloom. Yes.

Uncle Moon (Host) (01:18:41)

Yes, if their model is unparalleled, they would be like the ultimate Matrix in the world; they are the ultimate Matrix-like entity in the world. Infra, right?

Banghua Zhu (Guest) (01:18:52) †

Yes, they are both a model company, and an Infra company.

Uncle Moon (Host) (01:18:54) †

then everyone just calls their APIs. And all agents exist within their matrix.

Banghua Zhu (Guest) (01:18:58)

Yes, this is one possible outcome. But we hope for a different outcome. Yes, there are more different agents, interacting— interact, and these agents all come from different companies. Then we become the people who empower these companies.

Uncle Moon (Host) (01:19:11)

So you joined this startup, the ultimate goal is to push further in this direction regardless of success or failure, you feel you've made your effort, right?

Banghua Zhu (Guest) (01:19:19)

Yes, that's right.

Uncle Moon (Host) (01:19:20)

Ah, this is truly the first time I've heard you say it. It's really, really great.

Banghua Zhu (Guest) (01:19:24)

This is quite metaphysical.

Uncle Moon (Host) (01:19:25)

Yes. But I still think I truly believe in one saying: because you believe, you see. Right? If you let this world try to go the way you want it to, Even if it fails, but at least it's a very interesting process Very interesting— a very meaningful process.

Banghua Zhu (Guest) (01:19:40)

Yes, it is. Yes.

Uncle Moon (Host) (01:19:42)

It might even inspire many people, right? Maybe you're not the successful one, but in the end, it inspires many to continue striving in this direction. Someone takes over, and they succeed.

Banghua Zhu (Guest) (01:19:52)

Yes, we think that kind of world would be more interesting. So we actually hope to be able to build such a world together. Yes.

Uncle Moon (Host) (01:19:59)

It does seem quite interesting.

Neo Labs, world models, and startup opportunities

Uncle Moon (Host) (01:20:02)

Hmm. Yes, as just mentioned, many new labs, right? Whether in China or the US, more and more companies are training large models themselves. What do you think of these Neo Labs?

Banghua Zhu (Guest) (01:20:15)

I actually think that this is the first step towards the vision we hope to see. That is, more and more people, are willing to challenge Frontier Labs. They won't think: OpenAI and Anthropic are the end game. On the contrary, they think, that we can now create new paradigms, to create new— AI. This kind of AI might not even be through the current pre-training, post-training paradigm. For example, what David Silver's company did. They said they wanted to do RL from scratch. This is a very interesting idea. Of course, I don't know if it will succeed in the end. But with more and more such attempts, everyone can make a new generation of changes in the paradigm. And we actually hope to empower more and more companies like these. Let them push the frontier of the times, and we work with them to promote this, to make them faster and more efficient to push this frontier forward.

Uncle Moon (Host) (01:21:07)

How should I put it? As a researcher myself, I've never encountered a situation like this before where capital is heavily pushing for the establishment of labs.

Banghua Zhu (Guest) (01:21:18)

Uh-huh.

Uncle Moon (Host) (01:21:19)

Right? Now we've entered a phase where as long as you're a good enough researcher, you can earn a high salary in industry to do research. From this perspective, I truly believe that science and technology are the primary productive forces, and then it's about diligently doing this research, and diligently exploring, diligently creating actual value— isn't it the best career path for a student? Is it for a student the best career path?

Banghua Zhu (Guest) (01:21:44)

Yes, I think so. In fact, everyone is betting on an AGI future now. So whether it's investors or companies, everyone is willing to spend more funds or costs to bet on such a thing. I think this is also very encouraging for many AI researchers. It's very encouraging. Yes.

Uncle Moon (Host) (01:22:01)

Yes, this is a good thing, right?

Banghua Zhu (Guest) (01:22:02)

It's really good.

Uncle Moon (Host) (01:22:03)

When has there ever been such a situation before? Right?

Banghua Zhu (Guest) (01:22:05)

Yes, that's right.

Uncle Moon (Host) (01:22:05)

These cards— whether it's NeoLabs' talent or the cards on NeoCloud. Right? All this money turned into talent and cards That's the feeling.

Banghua Zhu (Guest) (01:22:15)

Yes, that's right.

Uncle Moon (Host) (01:22:16)

Yes, whether it succeeds or not, you should do such things. Is that your opinion?

Banghua Zhu (Guest) (01:22:22)

I think so. Because even if you don't succeed this time, you might have another chance. But if you never try, you might never succeed. Yes, that's right.

Uncle Moon (Host) (01:22:30)

Yes, especially this wave from OpenAI and this wave from Anthropic, has already approached a trillion market value, right? Everyone has seen it. If you invested at the beginning a few hundred million or a billion, and then by chance there was a paradigm shift, It's very likely your 100 million became a trillion, right?

Banghua Zhu (Guest) (01:22:45)

Yes, and this opportunity is—

Uncle Moon (Host) (01:22:46)

Yes, this kind of attraction is also very strong, right?

Banghua Zhu (Guest) (01:22:50)

it really depends on the times. If you go back ten years, there might not have been such a big opportunity. On the contrary, if this thing— AI just came out— it's actually in a chaotic state. Heroes emerge in troubled times. So, at this time, it's even more important to fight for it. Even in the worst case, if nothing was achieved, I think you can still go back to a big company and retire. What was not done— Yes, I think the world model is also one.

Uncle Moon (Host) (01:23:16)

Hmm, this— Everyone must seriously ruminate on these words This is golden career advice. Okay. Yes, besides these, Neo Labs and, NeoCloud. And as for the increasingly popular world models, Have you learned about them or have your own opinions?

Banghua Zhu (Guest) (01:23:31)

which is an exploration of new modalities that— an exploration of new modalities. So I think it's very natural. At the same time, if you can do it very, very good, so-called word— the so-called world— model. Though how to define one is debated. But, if you can predict the physical world very well, But that is to say, if you can predict the physical world very well, physical simulation environments— physical— simulation environments. This is actually for robotics or for some, even in the gaming industry, there will be a very, very big one help. This is actually a completely different paradigm from language models. This is actually a completely different paradigm from language models. Of course, I haven't really worked on word models, but I do think this is also a very important point. Yes. Of course, I haven't really worked on world models, but I do think this is a very important point for the future.

Uncle Moon (Host) (01:24:14)

Yes, I find that when you look at these directions, it's similar to what you said about Michael Jordan. You just encourage everyone, go explore. You're not like many of those big V's, Focusing on a single direction: oh, this won't work, that won't work. But I found you just say, hey, everyone should try this, everyone should try that too, right?

Banghua Zhu (Guest) (01:24:32)

Because I know my opinion will be biased, if I— I really feel, for example, I felt at the time, for example, a very small point, for example, I thought DPO was not good. I think PPO is actually better. I won't say that this is set in stone, because I think this problem cannot be predicted by one person's strength. You must have very rigorous observations to really confirm this.

Exploration, judgment, and Arena evaluations

Uncle Moon (Host) (01:25:00)

Let's move to the wrap-up phase. Just looking back at your past, so brilliant, undergraduate, PhD, first startup, second startup. That is, what do you think you could have done better? For example, at the age of 25, for example, at the age of 25, what would you do more of, and what would you do less of?

Banghua Zhu (Guest) (01:25:20)

I actually feel that at that time, I might still try more new directions. If I hadn't tried, for example, I hadn't tried, communication, I haven't tried computer biology. I even looked at some blockchain theory for a while, or later econ theory, I think my understanding of some things would not be as complete as it is now. I roughly know what kind of problems communication needs to solve. I mean, I roughly know, for example, what kind of problems communication needs to solve, and what kind of problems computers need to solve, why these problems were important at that time, and now they may be less important or were not important at that time. But now it's important. So I actually think, that seeing the rise and fall of various research directions, It actually makes me more confident in judging what will happen next. Mm-hmm. Yes, I think that running a company—

Uncle Moon (Host) (01:26:10)

Yes, I could feel just now, you said your judgment is very, very accurate. Actually, I— Actually, I've been reflecting on this in the past, with my startup partners and with many friends. I found that, at a certain point, whether you can make that decision— That is the highest manifestation of a person's cognition.

Banghua Zhu (Guest) (01:26:28)

Hmm.

Uncle Moon (Host) (01:26:29)

Yes, I think your judgment is very, very accurate. And I also really believe what you just said, that after so many different explorations in the past you've developed this kind of judgment for things.

Banghua Zhu (Guest) (01:26:40)

Mm-hmm. Yes, I think that running a company and doing anything, actually, in the end, it's just, every decision, the result it brings. Yes, actually, I mean, I— I also judge a person by whether, when he makes decisions, he can always or often make the right decisions, whether he can always, or often make the right decisions. I think this is indeed a very important— I think this is indeed a very important aspect of a person's thinking. It's an aspect of a person's thinking. Right. Oh, I'm not interviewing It's a direction. Right. Oh, I'm not interviewing. For example, I really admire Ying Sheng.

Uncle Moon (Host) (01:27:09)

Could you elaborate? For example, during an interview, how can you tell whether someone will make the right decisions?

Banghua Zhu (Guest) (01:27:14)

Oh, I'm not interviewing. For example, I really admire Ying Sheng. Our CEO, I think he actually— For example, I really admire Ying Sheng. Our CEO, I think he actually, his judgment on many things— Our CEO, I think he actually, his judgment on many things. is very, very accurate. his judgment on many things. is very, very accurate. For example, when he did Arena. is very, very accurate. For example, when he did Arena, he probably had an idea before he did it. For example, when he did Arena, he probably had an idea before he did it. He said this is a very important thing. He probably had an idea before he started doing this. He said this is a very important thing, and it might be something everyone needs, and it might be something everyone needs. And sure enough, Orin, that thing became very, very popular. And sure enough, Arena really took off.

Banghua Zhu (Guest) (01:27:44)

And then, when he was doing SGLang later, my judgment was completely opposite to his. my judgment was completely opposite to his. And it turns out he was right. I think there's no need to spend time creating a brand new inference engine. But he might think that if this inference engine, from a first principle perspective, from a principle perspective, if your inference engine is fast enough, good enough, and widely supported enough, users will choose it. And it turns out he was right, and my judgment at that time was, was completely opposite, and completely wrong, and completely wrong. And he actually, very early on when he was doing S-chain— was completely opposite, and completely wrong. And he actually, very early on when he was doing S-chain, he pulled Chen Yang and many others to do this Aiyao thing together.

Banghua Zhu (Guest) (01:28:13) †

very early on, when he was working on SGLang, he brought Chenyang Zhao and many others together to work on RL, because he believed RL would be a direction worth betting on, and it turns out that's indeed where everyone is now reinforcement learning, especially for this agent, because of the agent, the demand for reinforcement learning itself is also increasing. the demand for reinforcement learning itself is also increasing. So because— So because SGLang and reinforcement learning integration— So because— SGLang and reinforcement learning integration are done very, very well. are done very, very well, so many more and more people who do training— so many more and more people who do training will also use SGLang. So I think— will also use SGLang. So I think— So I think, my co-founder— my co-founder— I think this is very rare and valuable. Yes. Oh, you're a bit— I think this is very rare and valuable. Yes.

Uncle Moon (Host) (01:28:50) †

Oh, you're a bit—

Banghua Zhu (Guest) (01:28:50) †

Mm-hmm. Yes.

Uncle Moon (Host) (01:28:52)

Oh, you're a bit—

Banghua Zhu (Guest) (01:28:52)

Mm-hmm. He is indeed very strong.

Uncle Moon (Host) (01:28:53)

Too modest. Actually, from my perspective, it's like a person with super strong judgment met a person with super, super strong judgment. Super, super strong. That's the feeling. Because you were, yes.

Banghua Zhu (Guest) (01:29:01)

Mm-hmm. He is indeed very strong. He is indeed very strong. Yes, I don't know myself. Yes, I don't know myself. Yes, I actually think that when everyone invests, they are actually investing in the original version of Arena.

Uncle Moon (Host) (01:29:06)

But my perspective is very strong, because at that time, doing Nexus, including being able to smooth out this RL tech stack, including so early— To figure out agent and function calling so early on. In that stage of 2023, Yes, and it's a non-large model PhD from reinforcement learning optimization. This is already very abnormal, I think. Hmm.

Banghua Zhu (Guest) (01:29:26)

Thanks.

Uncle Moon (Host) (01:29:27)

Yes, and actually, there was something just now that I can talk about further with everyone, namely, LM Arena, the leaderboard for large language models. Right? It recently received 150 million US dollars in funding, right? Right? That is, large models not only need to look at those professional things Math, long texts, those relatively objective rankings on the leaderboards. But it also needs a way for everyone to score, right? to seriously look at how the general public perceives and approves of this large model. So, it's also— started doing this very early, but, recently, it's been noticed by various powerful investors, and then, they invested.

Banghua Zhu (Guest) (01:30:05)

Yes, I don't know myself. Yes, I actually think that when everyone invests, they are actually investing in the original version of Arena. Because at that time, in that society— Because at that time, in that society, in that context. What everyone valued most was whether your chat itself would be liked by people. in that context. What everyone valued most was whether your chat itself would be liked by people. Because at that time, the model had not yet reached— What everyone valued most was whether your chat itself would be liked by people. Because at that time, the model had not yet reached, it couldn't— Because at that time, the model had not yet reached, it couldn't, it couldn't give you a good— it couldn't, it couldn't give you a good— it couldn't give you a good, high-quality response, high-quality response. But at this stage, everyone is slowly moving towards the frontier of intelligence.

Banghua Zhu (Guest) (01:30:33)

At this point, people don't care if they like the response, but rather if it can solve the most complex problems. Can it solve coding problems? This is now an indicator that Arena often cannot reflect. So, I personally feel that this— if Arena wants to go further, it may need to develop in this direction.

Uncle Moon (Host) (01:30:53)

Yes, yes, which is to build more environments and objectively look at this issue. Can it be solved by a certain large model, right?

Banghua Zhu (Guest) (01:31:00)

Yes, it's more about advancing the Intelligence Boundary, evaluation, not just through human judgment. This— emotion or human judgment can simply determine an evaluation like this.

Uncle Moon (Host) (01:31:11)

I feel they will definitely move in this direction, right?

Banghua Zhu (Guest) (01:31:14)

Yes, I hope so.

Coding agents and the future of software jobs

Uncle Moon (Host) (01:31:17)

Let's test our judgment again. This is to say, because you also— you're using Claude Code to write code every day now. What do you think of the programmer industry? Actually, I don't really like using the word "programmer", because it's too broad. Because there are indeed computer scientists, and engineers, and programmers, with different role divisions. So, what do you think the computer industry will become in terms of employment? Just a scenario. Let's make a vision.

Banghua Zhu (Guest) (01:31:45)

Hmm. I think because of coding agents it has actually caused two effects. One effect is that for junior software engineers, or recent CS graduates, it's indeed harder to find jobs. Because many times people find that agents can easily replace some more basic Computer Science Software Engineering related work. On the other hand, it will concentrate many resources on truly strong and tasteful people. These top-level System Engineer, or, Software Engineers, And even Researchers. Because agents will constantly amplify their own capabilities and accelerate their iteration speed. In this way, eventually, it might cause a change in the CS industry. In the past, everyone had to maintain very complex software. This might indeed be a hundred-person or thousand-person effort. But later, as coding agents become stronger, it might become a ten-person effort. But it requires ten of the strongest system people, The most tasteful people to get this done.

Banghua Zhu (Guest) (01:32:56)

And so I haven't quite figured out one thing: how should this educational gap be filled? Will schools in the future, their role be more like how to screen out those who can become the strongest, these— system managers or software managers, system engineers or software engineers? So I can't imagine what kind of future this will be. So this part, I can't imagine what kind of future it will be.

Uncle Moon (Host) (01:33:25)

Yes, I also think, just from the perspective of the computer industry, the visible future, job positions will basically be concentrated in the category you just mentioned.

Banghua Zhu (Guest) (01:33:35)

Yes, I think it's highly likely.

Uncle Moon (Host) (01:33:37)

Companies now— especially, you've also seen a trend: in Silicon Valley, it's about elite talent. If you hire thirty or forty people, this company can basically operate well, and has great potential. The more people you hire, it might even make people think your company has redundant staff, And lacks potential, right?

Banghua Zhu (Guest) (01:33:54)

Yes, it's more like AI agent native companies may have differences in various architectures and in their design. It may be different from previous companies from previous eras.

Uncle Moon (Host) (01:34:04)

Yes, its entire organizational structure, communication methods, and collaboration methods will all change, right?

Banghua Zhu (Guest) (01:34:08)

Yes, it is, yes.

Uncle Moon (Host) (01:34:09)

But at the same time, if you study computer science in China, everyone knows, it's a very broad major. Basically, the number of computer science students in every school ranks among the top, I think at least top five. Computer science. Yes, and then, it's really hard to see clearly with the naked eye. How can so many computer science graduates continue to survive in this industry?

Banghua Zhu (Guest) (01:34:34)

Yes, I don't know either.

Uncle Moon (Host) (01:34:36) †

Yes, that is—

Banghua Zhu (Guest) (01:34:36)

I think it's very— this is a difficult question, yes.

Uncle Moon (Host) (01:34:40)

Well, this is a difficult question.

Banghua Zhu (Guest) (01:34:41)

Anyway, If possible, hypothetically, if this AI still needs to develop further in all aspects, Then in many cases, perhaps so much supply will become some— um, for example, to make AI further progress in Data Labeling demand. Yes.

Uncle Moon (Host) (01:35:03)

This means more people might first engage in data labeling.

Banghua Zhu (Guest) (01:35:08)

Yes, but— But this, because data labeling is becoming more and more complex, so its threshold will also become higher and higher. It's just that you— if you are only labeling preference data now, actually for for frontier models, it doesn't help much. In fact, more often, everyone needs to label increasingly complex data. And this is indeed currently still a very human capital-heavy, an effort.

Uncle Moon (Host) (01:35:32)

But actually, no matter what, my personal experience is, if you really like computer science, You should just go all in. Because the big opportunities in this industry are still there. There. If you don't like it enough, and just want a job, this industry might become increasingly unfriendly. I think this judgment can be made.

Banghua Zhu (Guest) (01:35:50)

Yes, that's right. But actually, to take a step back, AI could also lead to, that is, a lot of so-called wealth, might actually become irrelevant. So, people might just do what they love. People will ultimately do what they love. If you truly enjoy CS and coding, you can continue.

Uncle Moon (Host) (01:36:10)

traditional programming.

Banghua Zhu (Guest) (01:36:11)

You can continue coding, just for interest, not for productivity or money. And if some people don't like CS, but because of AI, AI can empower them to do whatever they want. From another perspective, it's actually a good thing.

Uncle Moon (Host) (01:36:27)

Oh, this view is similar to what Musk said a while ago, that human society will eventually become one where everyone's wealth doesn't matter, everyone already has extreme material freedom. Everyone can do what makes them happy. Is that the feeling, right?

Banghua Zhu (Guest) (01:36:43)

I don't necessarily think financial freedom will be that close, but I think because of AI, because of AI, it will lower the barrier for people to do what they love. This threshold will be lowered. Because if, suppose I suddenly feel that computational biology is the future, I want to go back to biology, I can use AI to help me better onboard this direction, and let me cooperate with AI to lower the barrier for me to work in this field, the threshold for engaging in this field. I think this way, it will make it easier for people to change careers. Yes.

Uncle Moon (Host) (01:37:19)

Oh, I also interviewed some biology PhDs before at Stanford. they use Claude Code every day now for various data processing tasks, and when learning new directions, they use Claude Code for all of it.

Banghua Zhu (Guest) (01:37:30)

Yes, it is.

Uncle Moon (Host) (01:37:31)

Previously, for example, it might have taken a long time to learn programming, or someone might not have had the time to learn programming. Now, they're directly using Claude Code to do things in their own field. Now they can just use Claude Code to work in their own fields. Yes, actually, just from the computer science industry alone,

Banghua Zhu (Guest) (01:37:42)

Yes, it is. So overall, I think this tool will empower more and more, more people to do what they love, as long as it's used well.

Uncle Moon (Host) (01:37:50)

the number of personnel needed will be less and less, but in terms of broad coverage, AI gives people more freedom.

Banghua Zhu (Guest) (01:37:59)

Yes, it is, yes.

Uncle Moon (Host) (01:38:02)

Hey, actually, overall, Overall, It's still a better world, right?

Banghua Zhu (Guest) (01:38:09)

Well, I can only hope so.

Uncle Moon (Host) (01:38:12)

Why do I say I can only hope so?

Banghua Zhu (Guest) (01:38:14)

Well, it will indeed bring many realistic unemployment problems, because not everyone, for example, there are people, who enjoy doing computer science and writing code, but their abilities might not be that strong. But they just like the process. Such people, they might indeed face challenges like not being able to find a job. Any societal transformation will bring a lot of pain. This part is unavoidable, but it's also a rather tragic thing. But it's also quite a tragic thing. Yes.

Uncle Moon (Host) (01:38:48)

Indeed, it's possible. I personally see this as an industrial revolution. So I can see in some scenarios, in the process of transformation, many young friends are struggling to find jobs.

Banghua Zhu (Guest) (01:39:03)

Yes, it is.

Technical taste, hiring, and integrity

Uncle Moon (Host) (01:39:04)

Yes. As you mentioned, all kinds of resources and opportunities will be more concentrated on those with good taste and strong initiative. From your perspective, how do you screen such people? What is good taste?

Banghua Zhu (Guest) (01:39:16)

Actually, many times I feel, in— for example, during your interview process and when communicating with everyone, many times you'll see: For example, can they feel— can they find strong criticism in their daily lives regarding code or other, code or other, directions? Will they feel that when they see this thing— Will they feel that when they see this thing, written like this, I feel uncomfortable. And will they feel, that my code shouldn't be written like this, or my research shouldn't be done this way? I, many people actually, won't have strong criticism. They won't criticize things, and they'll think, "Oh, this is actually pretty good." I'll just accept it directly. But if you look at, for example, Linux, the one who wrote the Linux system, it's actually the ancestor of early Agent usage. It uses people as Agents. And if someone writes poorly, he'll scold them and just furiously rant online.

Banghua Zhu (Guest) (01:40:11)

But behind it, he's always gatekeeping a lot of code quality, gatekeeping. He really checks whether this thing, if it really doesn't meet his aesthetic, he will directly reject it. So, in fact, many times it's more about whether people can, in real life, Or in the process of writing code, or while writing code, what's wrong with this code, what exactly is wrong with the code, or often when discussing their usual work. We can feel it, that there are really people, there are really people, when we talked, He said, "When I look at our company's code now, I feel really uncomfortable. I want to rewrite it. I hope many parts of it can be improved. But we can't do that now." When we talked about this, we would feel quite happy. Then we'd delve deeper into the conversation to understand if the problems he saw were real problems, and if he had better solutions.

Banghua Zhu (Guest) (01:41:05)

Because only then can you, for example, when working with a prompt agent, you can criticize the agent, saying, "what you did is terrible. I need higher-quality code." If you think the agent's code is already good, then, your upper bound will be at the level of the agent's code. But if you can supervise the agent, you can tell it that this system should write code better than this. What you've written is far from what I imagined. And you can tell the agent what truly good code looks like. You can drive the agent to write truly good code.

Uncle Moon (Host) (01:41:37)

I think this part can be seen as a career development bible for young friends. Hmm. Yes, it's very, very— The ideas are very clear in guiding everyone on how to improve themselves at work. Actually, I've interviewed some people before who were doing well in open source. Those who do well in open source, including the person behind Linux. Hmm, all have these qualities.

Banghua Zhu (Guest) (01:41:58)

Yes, yes.

Uncle Moon (Host) (01:41:58)

Their qualities make them, how should I put it? In their careers, they are always several steps ahead of others, until others can't catch up. This is truly the case.

Banghua Zhu (Guest) (01:42:09)

Yes, it is.

Uncle Moon (Host) (01:42:09)

If you want to talk about it, honestly, it's about real intelligence. People with an IQ over 180 or 200 are rarely seen. Hmm. But mostly, with these subtle differences, it all depends on the taste you mentioned, and why isn't this beautiful? Why isn't this done well? Why can I write 500 lines of code for this? Can we write it in 100 lines and make it easier to understand? This kind of stubborn, obsessive-compulsive disorder. Hmm. That's a manifestation of high taste, right?

Banghua Zhu (Guest) (01:42:38)

Yes, that's right. We really lack this, and we appreciate such people. But I'm not really that kind of person, so I don't dare write code— even for SGLang, because Lianmin and Ying would definitely roast me. Right.

Uncle Moon (Host) (01:42:48)

You're too modest. Yes, but being able to identify such people, I think is also a very important quality. There are always talented people, but not always mentors. If some people think, "Oh, this person is difficult to work with, very stubborn." But from your perspective, you might think, "Oh, this person is truly striving for perfection." "Truly high quality, very high taste." Then he should indeed cooperate with you.

Banghua Zhu (Guest) (01:43:08)

Yes, many people are actually like this. They might not fit in well at large companies. People will think this person is too troublesome. And they say they disagree with everyone, blocking the progress of many things. But actually, we, including Lianmin and Ying at LMSYS, we are doing RadixArk. Actually, there's a mission behind it, which is to hope to allow truly capable people and those with ideas, and also those who are upright and kind, to be discovered, and to be recognized by the world, and to, to stand out in this big context. Yes.

Uncle Moon (Host) (01:43:38)

I particularly like what you just said about integrity and kindness, because we just talked about technical topics. To be honest, after meeting enough people, you'll find, Integrity and kindness are really important. Yes, this is something that when you,

Banghua Zhu (Guest) (01:43:48)

Hmm, yes, that's right. The fundamental character is very important.

Uncle Moon (Host) (01:43:50)

communicate with others, it's easy for you, or not so easy, after gaining some experience, you can judge their true nature. And based on that true nature, you can have some resonant communication, and then you can continue to do greater things. If there isn't this basic foundation, then everything else might be out of the question. If there isn't this basic foundation, then everything else might be out of the question. Yes. Ah, I totally agree with that point.

Banghua Zhu (Guest) (01:44:10)

Yes, yes, I think finding startup partners or finding this, even every person in the company, I think this point might actually be the most important.

Uncle Moon (Host) (01:44:17)

Hmm. I found that's why I always get along well with you. Maybe that's why. Thanks, thanks, thanks. Resonated, resonated, yes.

Banghua Zhu (Guest) (01:44:24)

Yes, I think I get along very well with you.

Uncle Moon (Host) (01:44:27) †

So next, I'm thinking,

Banghua Zhu (Guest) (01:44:28) †

Soul resonance.

Uncle Moon (Host) (01:44:29)

next is—

Advice for students and professionals

Uncle Moon (Host) (01:44:30)

Next, we'll, OK. Next, we'll move on to the last question. For— So we're moving on to the last question. For undergraduate students, for PhD students, For students who are currently working, give these three types of people your most sincere advice.

Banghua Zhu (Guest) (01:44:54)

Hmm. Yes, I think, Hmm, This can easily sound very vague. But I think, um, the world is changing very fast now, which actually leads to many times, um, the cost of trial and error, Actually, it will become lower, because no matter what you try, as long as it's in the right general big, big direction, it will eventually work out. So, some people, many people will struggle. For example, should I give up my PhD and do something I want to do? To try the direction that some startups are trying? Or should I give up many years of work experience I had before and try a completely different direction? But I think, many people are willing, the more you are willing to give up, the more you will gain later. And I think, um, especially in this rapidly changing era, it's those who can let go who will ultimately gain more.

Banghua Zhu (Guest) (01:46:00)

Yes, so I also encourage everyone, if there's something you really want to try, especially if you think, for example, why can this person do this, why can't I do it? Then I think you should probably give it a try. Maybe you'll find that person is not a big deal, and you might even do better than them. Yes.

Uncle Moon (Host) (01:46:16)

Ah, that's so well said. And what you say is very convincing, you know? It's like you're from a globally renowned team of mentors, Michael Jordan's side, gave up theory, gave up statistics, gave up optimization in this direction. Right? bravely chose this new large model, and so became a trendsetter in the era of large models, right? Brave to give up, so you can gain more, right?

Banghua Zhu (Guest) (01:46:37)

Yes. It's mainly because I didn't do well in theory back then.

Uncle Moon (Host) (01:46:40)

Too modest. Yes, anyway, thanks, Banghua, as the guinea pig for my first long podcast.

Banghua Zhu (Guest) (01:46:46) †

Thank you for the invitation.

Uncle Moon (Host) (01:46:48) †

Thanks, thanks.

Banghua Zhu (Guest) (01:46:48) †

Thank you, thank you.

Uncle Moon (Host) (01:46:49) †

It was a very pleasant chat.

Banghua Zhu (Guest) (01:46:50) †

Very pleasant.

Uncle Moon (Host) (01:46:50) †

Very pleasant.

† At least one speaker change or brief response in this passage needs audio verification.