Host: Uncle Moon
Guest: Tianyin Xu

Source: English subtitles. Dialogue is grouped into paragraphs, retaining the conversation and opening highlights.

Speaker labels are inferred from context and have not been checked against the audio throughout. A dagger (†) marks passages with uncertain attribution. Timestamps come from the subtitles; short exchanges within one subtitle may share a timestamp.

Opening excerpts have been placed in timestamp order; several appear among the main conversation in the source subtitle file.

Contents

Highlights

Tianyin Xu (Guest) (00:00:00)

I've always argued that best paper awards should be abolished—they're deeply misleading.

Uncle Moon (Host) (00:00:04)

That's a bold take. Aren't you worried about offending people?

Tianyin Xu (Guest) (00:00:06)

I actually don't really believe in AI, but I think people should be believed. These failures all came from AI gaming the reward and producing bogus proofs. There are simply too many bugs in production, and nowhere near enough people to fix them.

Uncle Moon (Host) (00:00:19)

in other words, how do you teach the operating systems course and why do students like this course so much?

Tianyin Xu (Guest) (00:00:24)

Because I think research is a really wonderful thing, right? Then I feel that he has no set standard for all beautiful things.

PhD Applications, Rejections, and a Year in Germany

Uncle Moon (Host) (00:00:34)

Conversations with 100 fascinating people in Silicon Valley. This is Uncle Moon. Today's guest is Tianyin Xu, a tenured professor at UIUC. He has won numerous best paper awards in computer systems, and is one of the finest systems researchers I know. He is currently on sabbatical, visiting UC Berkeley's SkyLab. Welcome, Tianyin.

Tianyin Xu (Guest) (00:00:54)

Hi, everyone. It's an honor to be on Uncle Moon's podcast and chat with you all.

Uncle Moon (Host) (00:00:58)

All right, let's dive in. From 2003 to 2010, you completed both your bachelor's and master's degrees at Nanjing University. Then, in 2010, you applied to PhD programs in the U.S. Reports say you were rejected by 24 schools. What exactly happened?

Tianyin Xu (Guest) (00:01:16)

So what happened? Honestly, I'm not entirely sure. A failure like that usually has many causes. But the root cause was probably that my track record wasn't strong enough. I applied in 2010, when the academic environment in China was very different from what it is today. I did get one interview, though, with the program where I eventually did my PhD. It was with the professor who later became my advisor. The interview lasted about 30 minutes, and I remember it vividly. He asked me five questions, and I couldn't answer a single one. A few days later, I received the rejection letter. Let me share three of those five questions. The whole thing was pretty embarrassing. First: why do you want to pursue a PhD? And ultimately, what kind of career do you want? Second: compared with your peers, what are your strengths? Third: what kind of research do you want to do? I'd never really thought about any of those questions.

Tianyin Xu (Guest) (00:02:14)

They weren't technical, so my answers were very weak. question at that time was, 'What will your future career development be?' I heard that at the time I read some guides saying that I heard American professors all like their future students to become professors, so at that time I said I wanted to go down the academic path and become a professor. Then my future advisor at that time asked me, he said, 'So what do you think it takes to become a professor?' I couldn't answer, and then I said, 'Well, if you do research well, shouldn't it naturally make you a professor?' Then he just laughed and said, 'It's not like that, actually.' Later, we moved on to the second question. The second question was, he asked, 'Compared to your peers, what are your strengths?' What are your technical strengths? Then I hadn't thought about this question either, and I was stunned for quite a while.

Tianyin Xu (Guest) (00:02:55)

In the end, an answer popped out, and I said that I work particularly hard. Then he laughed and said that 'working hard' is a default; it's not really an advantage, right, it's not a strength. He said all PhD students work very hard, and then he followed up with more questions. He asked what exactly your strengths are? Then I said I didn't have any advantages and then moved on to the next question. The next question was about what kind of research you want to do. At that time, my preparation was also very rough. I had read a lot of his papers and said that I especially liked his ABC paper and wanted to do those projects. Then he smiled again and said that if a professor puts a paper on their homepage, it means that the paper has already been completed. Otherwise, you wouldn't have put this paper on this website, right?

Tianyin Xu (Guest) (00:03:32)

He said he actually wanted to ask what kind of research I wanted to do, rather than doing a certain project, and I still couldn't answer. A few days later, I received a rejection letter. That was my first application experience. Honestly, I later got an offer from a pretty good school, but I didn't end up going, which also laid some groundwork for my second application later on. Anyway, after that,

Uncle Moon (Host) (00:03:51)

you took a gap year and chose to go to Europe, right?

Tianyin Xu (Guest) (00:03:55)

Yes, at that time, Nanjing University happened to have an exchange program. So I applied to a school in Germany called the University of Göttingen. It's the same story—I felt that the domestic environment back then was very different from now. Nowadays, I think students have many choices. Even in my own lab, there are many visiting students, but around 2010, going abroad was still not a very common thing. So opportunities weren't that many, and at that time there happened to be one opportunity, so I went.

Uncle Moon (Host) (00:04:22)

And during the time from Nanjing University to Germany, you were still playing football, and you even wrote on your homepage a football resume about how you played from Nanjing University all the way to Germany. How did that come about?

Tianyin Xu (Guest) (00:04:33)

I actually can't remember this clearly. I didn't even know there was such a page. But I indeed really like football, and I really like sports. I like sports. Competitive sports are much like research. You have to give them everything you've got. Both demand your very best. But sports are even more competitive: there is only one champion. Everyone else loses. Research is fiercely competitive too. Think about the papers PhD students submit to top conferences and journals each year. Acceptance rates are low, and everyone is brilliant and exceptionally hardworking. So how can a PhD student compete? At top conferences and journals, so few papers are accepted. Then everyone does research here. People are all very smart. Then very hard working. Probably the most hard-working. The smartest people work on it day and night. Then you as one. For example, a PhD student. You are just starting. How do you go to win the game?

Tianyin Xu (Guest) (00:05:18)

In competitive sports and, I think, doing research, mediocrity has no place. You either do very well, or you don’t do it at all. So I think there are very similar aspects between the two. That’s why I’ve always been very interested in sports—not just football, basketball, anything, I really like them. It’s just that later I stopped playing football because I was too busy, and also because I’m a bit stubborn, as I feel I wasn’t playing very well. When things are going well, I don't really want to kick anymore;

Uncle Moon (Host) (00:05:43)

if I do it, I want to do the best one, right?

Tianyin Xu (Guest) (00:05:44)

Right. It's just that my own mindset wasn't very good.

Uncle Moon (Host) (00:05:46)

You stayed in Germany for a year and finally got an offer from UCSD. During that year, what do you think you did correctly? And was it that your papers increased or that your recommendation letters got stronger?

Tianyin Xu (Guest) (00:05:54)

Honestly, I think nothing really changed; I just tried again, that's all.

Uncle Moon (Host) (00:05:57)

I think after just trying again and getting this offer, was there any story behind it?

Tianyin Xu (Guest) (00:06:03)

Because for me, I’ve always thought that I'm not a particularly smart person. Since I was young, I wasn’t especially smart either. For challenging things, the first time I do them, I definitely won’t do well. So for me, from my life experience, if I want to do something relatively challenging, I definitely have to fail many times. So actually, failing the first time isn’t a particularly big blow for me. And also, I’m not really someone who likes to settle; I’m not particularly the type of person who likes to compromise. I don’t really like doing something just because the previous task didn’t go well and then choosing the easier option. For example, when I was filling out my college application for the college entrance exam, I only applied to one school. At that time, I thought that if I didn’t get in, I would just retake the exam for another year, right? So the second time. The first application was not successful.

Tianyin Xu (Guest) (00:06:40)

Then I guess I’ll just apply once more. Try applying one more time. That's exactly what I was thinking at the time. Then, when I applied the second time. Later, didn't I also apply to the same school? And then finally went there too. And at that time, I still remember that interview. He/She is my advisor. His nickname is Wai Wai. Then WaiWai called me again at that time. At that time, I made a call on Skype. Then when he made the phone call. I was really nervous at that time. So the first time I applied, it didn’t go well. I thought I’d just apply again and try harder. That’s how I was thinking at the time. When I applied the second time, I ended up applying to the same school again, and eventually, I went there. I I still remember my second interview. YY, my future advisor, called me on Skype. I was nervous after the earlier rejection.

Tianyin Xu (Guest) (00:07:09)

I had carefully prepared answers to his earlier questions. But he didn't want to listen at that time. He asked me right then why I hadn't gone at that time. I didn't go to the school whose offer I received at that time. Then I told him some of my situation. Then he talked about me at that time. They started trying to persuade me to go to UCSD for a PhD. I was extremely shocked at that time. Then I also asked him. I said why this. But he didn’t want to listen. Instead, he asked me why I didn’t go to the school I had initially received an offer from. I explained my situation to him, and then he started persuading me to go to UCSD for a PhD. I was very shocked, and I even asked if he wanted to hear my prepared answers. want to listen. Then he told me that doing systems research is a bit different from theory research.

Tianyin Xu (Guest) (00:07:38)

In systems research, persistence is very important; it’s about being someone who isn’t afraid of failure. Then, if you're willing to keep trying, basically your systems research won't be particularly bad. So he told me that if you are willing to persist and willing to do this, then you've already passed my test. So I got this offer, and then I went. So when you ask me whether I improved during this time, I actually feel that I didn't improve much, I just persisted for another year. But

Uncle Moon (Host) (00:08:00)

as an observer, my feeling is just like what you said earlier. You are unwilling to settle and have high standards, and that means you are willing to try and willing to fail. The impression you give me is that this year you have proven this: you do not settle, and you want to accomplish this.

Tianyin Xu (Guest) (00:08:16)

It is possible—it's possible

Uncle Moon (Host) (00:08:17)

that if I look at various systems researchers, I find that this is one of the most valuable qualities of a systems researcher.

Tianyin Xu (Guest) (00:08:23)

I think it's essential.

Uncle Moon (Host) (00:08:24)

In my experience, it's something every systems researcher needs. Some research fields may depend on a flash of inspiration. Systems research, however, demands diligence, repeated failure, and the willingness to keep trying.

Tianyin Xu (Guest) (00:08:34)

Exactly. I completely agree.

Research Methods, Mentorship, and PCheck

Uncle Moon (Host) (00:08:37)

Then you joined YY's lab. Advisors have very different styles. Some guide students step by step, while others are very hands-off. How would you describe YY's style?

Tianyin Xu (Guest) (00:08:46)

He wasn't entirely hands-off, but he didn't hold your hand either. YY was already very senior at the time, yet he was an advisor who actively guided students. I think he would tell you how something ought to be done. this. Just now there was a question, for example, how does a PhD student think about this idea in the first few years, right? Then I remember that some of my previous practice was that a lot of times we would read that story, like how Newton came up with this, this, this Newton's law because he was hit by an apple and suddenly had an inspiration, right? Then YY is actually very opposed to this kind of process. He always emphasizes that an idea shouldn’t be something that comes from a sudden flash of inspiration, nor should it be something where you just sit under a tree and get hit by something and suddenly have an inspiration.

Tianyin Xu (Guest) (00:09:34)

It’s actually a very systematic process of exploration. What he emphasizes is starting from a problem: if you have an important problem, you start by trying to understand that problem, right? Go in-depth to understand this issue. After you go in-depth to understand this issue? So your solution is often obvious once you understand the problem; it can even be very simple and practical. This is a method of doing research that he emphasizes, and we all follow this method. This approach avoids many risks because once you deeply understand a problem, your answer naturally becomes very natural and very effective in solving the problem. You won't have a solution full of tricks from the start. Then I realized after spending a lot of time that this solution couldn't solve the problem either.

Tianyin Xu (Guest) (00:10:10)

It actually doesn't have such a risk, so actually I think that when doing research, because of this—at that time, in any case, I was very much in agreement with his, how should I put it? This is a principle, so I have always been practicing this practice, and I think it’s quite good and smooth.

Uncle Moon (Host) (00:10:26)

Continuously practicing this practice helped you get the PCheck. Could you share with We built a tool called PCheck. Could you explain what PCheck does?

Tianyin Xu (Guest) (00:10:35)

This became the second paper of my PhD. The idea is actually quite simple. Back then we were actually studying, I actually don't know how to translate it into Chinese, it's 'fault-tolerant systems,' which in Chinese might be called fault-tolerant system, right?

Uncle Moon (Host) (00:10:48)

Fault-tolerant system,

Tianyin Xu (Guest) (00:10:49)

fault-tolerant system. From our perspective at that time, what we were studying was a type of fault called misconfiguration. When there is a misconfiguration, the system obviously won't operate very well, right? At that time, YY asked a question which was: why is it that we have been studying fault-tolerant systems for so many years, and many of our systems claim to be fault-tolerant, yet when a fault like a misconfiguration occurs, the system just doesn't work, right? Because according to major companies, misconfiguration is a very important cause of system failure, so we started from this perspective. At that time, I did my first project, and the first project was to answer the question of why those mature production-level systems. They cannot handle faults like misconfiguration very well, and our approach is also very simple and crude, which is fault injection. Fault injection is a very mature technology, which means you try all kinds of different faults. Here, the fault refers to misconfiguration.

Tianyin Xu (Guest) (00:11:46)

Then you inject this fault into this system, so it is injected into this system, and then you observe how this system behaves. Then, during the process of doing this project? At that time, we were also very busy, that is, we had to do this kind of project to answer this question. Among them, there was one phenomenon that we found very interesting, which is that there are many misconfigurations. When you inject them into the system, the system actually doesn't show much behavior; it still works normally, right? It neither initiated a fault-tolerant procedure, nor did it say there was a fault, so why? Then actually the reasoning is very obvious: this system read these misconfigurations into the system's state, but it didn't use them. So why didn't it use these faults, that is, the misconfigurations?

Tianyin Xu (Guest) (00:12:25)

It is because these codes or these executions must be used only under a specific situation, and the most typical example is that when it is used, there is an external failure, that is, it triggers these misconfigurations, and when it is used, there is already a failure, and then I start a failure-recovery program. Then at this time I need to use some configuration, and then I find that this configuration is not good. Only then will it explode, and to us, this is like a time bomb, right? In this system, you're just like a time bomb that hasn't been triggered yet. Once you trigger it, the problem is usually very serious because when you're doing fault tolerance, that is, failure recovery, it's already a very critical time. At this point, discovering another failure—this kind of double failure—usually leads to a major disaster.

Tianyin Xu (Guest) (00:13:06)

When we were doing the first project, we didn't have the time to study this issue in depth because it was a different research problem, but the issue was there. Then for the second project, we said we would try to understand this issue and figure out how to solve it, so naturally the second project came

The Meaning of Best Paper Awards

Tianyin Xu (Guest) (00:13:22)

about.

Uncle Moon (Host) (00:13:22)

We can talk about this first project later. Let's talk about this best paper first. Are you surprised that this paper won the best paper?

Tianyin Xu (Guest) (00:13:29)

I was quite surprised at the time because I didn't know what the standard for the best paper was either. So I feel quite happy to be recognized in the field.

Uncle Moon (Host) (00:13:34)

What about him later? At that time I was surprised, and then?

Tianyin Xu (Guest) (00:13:38)

I have been in this field for a long time now, and last year, for example, at SOSP, I was also a member of the best paper review committee. Personally, I feel that the best paper thing is very random, very, that is, random and subjective, so now I completely feel that the best paper thing cannot fully represent the quality of a paper, it cannot represent it at all for me. I even feel that in recent years, for example, even the best papers in our field at SOSP and OSDI have quite a few issues for me, right?

Uncle Moon (Host) (00:14:04)

Are you so bold to speak without fear of offending people?

Tianyin Xu (Guest) (00:14:06) †

I am I really feel this way because I actually personally don’t know what the standard for the best paper should be? Then its process, generally speaking, for me, is like this: a white paper—when it gets this white paper, it means that the reviewer, right, in other words, a reviewer particularly likes your paper and has nominated your paper. And that's it, that's it. For me, it doesn't represent what the future impact of this paper will be. It also doesn't mean this is the highest quality paper here because quality itself can't really be measured, so pursuing whether a paper is the best paper is actually quite meaningless. Personally, I have always advocated for canceling the best paper award because I think it is very misleading, and how should I put it? I think what I mean is that everyone does research, like trying to get the best paper, right?

Tianyin Xu (Guest) (00:14:47)

Then I feel that having this called paper awards actually makes everyone not focus on doing research properly.

Uncle Moon (Host) (00:14:49)

I have heard a similar viewpoint before, and I strongly agree with it.

Tianyin Xu (Guest) (00:14:55) †

Right, if you remove the best paper, then how would everyone work? Then everyone would do the research they like the most, and I think that is the best state. Research really should, in essence, be like this.

Uncle Moon (Host) (00:15:04)

Right. And I also heard a saying that you don't need to look at the best paper, but the test-of-time paper because whether its citations are high after 10 years and its impact in various places is obvious.

Tianyin Xu (Guest) (00:15:14)

For me, all awards are not particularly meaningful; I'm not a person who believes in awards. Then, personally, I feel, for example, about academia in China. Actually, I’ve been really messed up by awards, so my ideal is an academic environment without these awards and other frustrating things. Every researcher just works on the problems they consider important, does it happily and joyfully, gets recognized, and then you just present the paper.

Uncle Moon (Host) (00:15:37)

Do you think I should share this concern?

Tianyin Xu (Guest) (00:15:40)

I'm really not worried about this, I really think so,

Uncle Moon (Host) (00:15:43)

so let's just cut it out, okay?

Tianyin Xu (Guest) (00:15:43)

Yes, I actually expressed my views to many people during the conference.

Misconfiguration and Systems for AI Agents

Uncle Moon (Host) (00:15:49)

Okay, right, you just said that this work is based Was it based on your 2013 paper, “Do Not Blame Users for Misconfigurations”?

Tianyin Xu (Guest) (00:15:55)

exactly, this work.

Uncle Moon (Host) (00:15:56)

I really like that work. This, this question is really good, don't blame the user for making a wrong configuration, right?

Tianyin Xu (Guest) (00:16:01)

Right.

Uncle Moon (Host) (00:16:02)

Then, what you were saying about this work is roughly that when I looked at the paper at the time, it just mentioned that a system was misconfigured. Engineers would all say this was a user operation error, but that paper of yours argued the opposite: if many users are unaware of the issue and make the same mistake, then there may be a problem with your system's design itself. That was roughly the motivation I got after reading it.

Tianyin Xu (Guest) (00:16:20)

Yes. Right. Exactly, I think you understood this paper.

Uncle Moon (Host) (00:16:21)

Then, in your paper, you mentioned that a user simply capitalized the letters in their name and then went through 75 rounds of communication and collected 10 logs. When you first saw this kind of problem, was it from the industry?

Tianyin Xu (Guest) (00:16:37)

This is a case from the industry. When I first saw this case, I was very shocked, but later I actually understood it because this company is actually a storage company, right? Then the storage company, actually very, their model was really troublesome. At that time, the company sold those—I remember this company didn't want to put their name on it, a bit embarrassing, but actually it was a very good company. Their product at that time was a storage box, right? It bundled software and hardware together, and they sold this storage solution, calling it a solution, to the users. Then the problem is that for a storage company, after a user has used your product and it malfunctions, it is impossible for them to send this company's box back to you. They can't even send all of their data back to you because this data is confidential. Many times, their clients are, for example, government agencies or banking departments, right?

Tianyin Xu (Guest) (00:17:21)

How can you send this data back and back and back again? So this diagnosis is really time-consuming and tedious. I can only tell you the few pieces of information I have, and then the person on your end has to guess and try to figure out what kind of problem you actually have. So the root cause might be very simple, but once there is a fault, it becomes a very expensive and very time-consuming process.

Uncle Moon (Host) (00:17:46)

Then during the whole process, you discovered 743 configuration error issues, and you also reported that a large number of these issues were confirmed and fixed by their developers. Which fix made you feel that I was not just publishing a paper, but really helping the industry solve a problem?

Tianyin Xu (Guest) (00:18:04)

I actually always felt at that time that because our starting point was system design, right? It's just a fault-tolerant system, and I think this is quite natural. I also don't feel that any particular patch or bug is especially important. Actually, I've never found discovering bugs that exciting, because I think there are too many bugs in the industry—so many that people can't fix them all. So discovering a few more bugs doesn't seem very meaningful to me. Of course, I understand that many papers, our evaluation is about proving that this solution is useful, and one of our evaluation metrics is to find new bugs. I think the more meaningful part of this paper is that it may tell some, especially open-source developers, when you are designing a system. You have to think about how users use it. I think there's a The work felt meaningful. At the time, our department had a professor named Stephen Savage.

Tianyin Xu (Guest) (00:18:55)

He is a very prominent security researcher in systems and networking. He was on our faculty. He organized a seminar, and I remember that one day a student proposed an idea at the seminar: use machine learning to detect misconfigurations. Stephen's immediate response was, “Why not build the system correctly?” He knew about our work, which took the same perspective. His first thought wasn't how machine learning could find more misconfigurations. Instead, he asked, “Why not do what Tianyin did and build the system correctly? Then you wouldn't need machine learning to find these problems.” That reframing may change how others approach the problem.

Uncle Moon (Host) (00:19:37)

What I especially like about this work is its relevance to today's age of AI agents. Back then, your paper argued that poorly designed systems create problems when people use them. Now AI agents are the users. If a system is still designed only for human users, then AI agents are bound to make mistakes.

Tianyin Xu (Guest) (00:19:58)

Right.

Uncle Moon (Host) (00:19:59)

So I'd like to ask: based on your earlier experience, do systems for AI agents need to be redesigned?

Tianyin Xu (Guest) (00:20:09)

Absolutely. Everyone now talks about AI-native systems. Yet many current systems weren't designed for AI agents as users. Treat an AI agent as a user, and the system is guaranteed to fail. We've even been building benchmarks recently. For example, one project is a collaboration with the TLA+ Foundation. In it, we ask an agent to construct a simple proof. Given a protocol, the agent writes a proof that the protocol is correct. Then you can could say from our experience that we found TLA+ to be a very mature toolchain. It has a proof system called TLAPS, which is the TLA+ Proof System, right? Then people have written a lot of proofs, and they have never encountered some strange problems. Their system's convexity is also very mature, so they want to create a benchmark.

Tianyin Xu (Guest) (00:20:56)

But if you put AI in, put an AI agent in, the agent will write all kinds of strange proofs, and it will seem like the thing has been proved, but actually it is not valid, and AI is especially good at reward hacking. Because when people write a proof, they really want to prove this, right? He really wants to prove that this protocol is correct, but AI doesn't actually really want to prove that the protocol is correct. AI just says, because the uncle wants to prove that the protocol is correct, so I try my best and can pass this proof system, which led to him discovering many of these strange bugs, so many colleagues from the TLA+ Foundation, like Markus. Or Stephan and the others started fixing all sorts of bugs that they had never even thought about before, all of which were caused by AI trying desperately to do reward hacking and then doing proof, which led to these problems.

Tianyin Xu (Guest) (00:21:43)

And I think that once an AI agent becomes a user, all systems should rethink how to handle this interface, in my opinion.

Uncle Moon (Host) (00:21:49)

I strongly agree with this view because I can observe that more and more companies in Silicon Valley are starting to invest in how to redesign the interface for AI agents, how AI agents should log in, how AI agents should ensure security, how AI agents should operate, and so on. I think it is an inevitable trend that AI agents will become the main force in work in the future.

Tianyin Xu (Guest) (00:22:09)

Yes.

Uncle Moon (Host) (00:22:09)

I think the biggest opportunity for systems researchers or some system engineers next is in this area.

Tianyin Xu (Guest) (00:22:14)

Honestly, I agree. I think this is a really good thing for systems research. In the past, if you wanted to fundamentally redesign a system, right? The cost is high, and not everyone may be willing to invest. But now, I think AI has a very positive impact on systems research. People are willing to consider this and feel it's necessary to redesign the system. I think this is a very good thing.

Uncle Moon (Host) (00:22:37)

In the system industry, I have always seen that it has to be behind products and algorithms. Because if you try to go too far ahead and create a system that cannot be deployed and applied on a large scale, then it actually has no value; the cost is too high and the ratio of input to output is too low, meaning the ROI is too low.

Tianyin Xu (Guest) (00:22:56)

Right.

Uncle Moon (Host) (00:22:57)

Now it is equivalent to the rapid development of AI, which has generated an infinite number of new major demands, forcing systems researchers and system engineers to continuously push forward the development of these new systems.

Tianyin Xu (Guest) (00:23:07)

Yes.

Uncle Moon (Host) (00:23:08)

I think this is a really good thing in the era of systems researchers.

Tianyin Xu (Guest) (00:23:11)

I think it is a very good thing. I think every systems researcher should seize this opportunity, because it includes, for example, in the past, if you were doing a system, right? Then you say that your system is very good, but it can only handle one type of workload. In fact, it's very hard to get, and even in a startup, it's hard to get funding. It's even hard to convince the judges that my system is good, and the judges will definitely reject your paper saying that this is not general. But now, it's like saying that I just make a system, and this system, for example, has very good IO performance. Many people would be very excited, right? I feel that your system is particularly useful, and its economic market is particularly large, so I think this is a very good thing.

Uncle Moon (Host) (00:23:46)

For example, if you were to work on a single aspect, like how to allow an agent to log in.

Tianyin Xu (Guest) (00:23:52)

Yes.

Uncle Moon (Host) (00:23:52)

Right, that agent login may seem minor, but because he considered all the possible reward hacking and ways to hack this system by different agents, he made it very solid. In this way, you can ensure the security of the entire system for the users.

Tianyin Xu (Guest) (00:24:03)

Right.

Uncle Moon (Host) (00:24:03)

What a good thing.

Tianyin Xu (Guest) (00:24:04)

Yes.

Uncle Moon (Host) (00:24:04)

I think this opportunity is indeed very significant. We can talk more in depth about it later when we discuss some of your recent research and

Mentors, Students, and Finding the Right Research Environment

Uncle Moon (Host) (00:24:11)

this AI system. Then going forward, when you enter the PhD stage and interact with your advisor, you have always been very grateful to your PhD advisor for constantly being supportive in various ways, and on your homepage, you said he is a rock-star advisor. Why is that? What exactly did he teach you?

Tianyin Xu (Guest) (00:24:25)

I think good mentors are all similar, while bad mentors each have their own problems, but good mentors are all similar. So I think he definitely taught me how to do research and then he told me how to think about certain problems correctly, right? Then I think one thing he does particularly well is that he really emphasizes deeply understanding problems, which is something I really, really appreciate. Also, he encourages thinking differently, and I think these are the things he emphasizes that I really appreciate. And I feel that he makes me constantly full of passion for research. I feel that research is a really wonderful thing, and I also especially like doing it.

Uncle Moon (Host) (00:24:59)

This is truly a top-level mentor,

Tianyin Xu (Guest) (00:25:01)

so I really think that having him as a mentor is actually very strict and very direct, but I especially like direct mentors because I don't really like some indirect advice. So I Your personalities were also a strong match. But I think the essential quality— what matters most— is a genuine interest and passion for research. That's what I admired most about him.

Uncle Moon (Host) (00:25:23)

Right. And he was also a very successful entrepreneur. I often see his talks on entrepreneurship online. He comes across as decisive, forceful, and demanding. But with you, the fit clearly worked. That's the impression I get.

Tianyin Xu (Guest) (00:25:38)

Fit may be the most important thing between an advisor and an advisee.

Uncle Moon (Host) (00:25:42)

He founded three companies, all quite successful. Did he ever discuss entrepreneurship with you?

Tianyin Xu (Guest) (00:25:48)

Yes. I listened to him talk about starting companies for three years. Right.

Uncle Moon (Host) (00:25:53)

What did you take away from it?

Tianyin Xu (Guest) (00:25:54)

I've always believed that research and startups share the same core. Both require innovation, and both demand the pursuit of excellence. These things are basically the same at their core. You need a team; it's very difficult for one person to accomplish something alone. You also need a good advisor who can tell you certain things, right? The difference is that research might be more particular about how to say it, how should I put it? It’s more about valuing intellectual merits, or rather, valuing expanding the human knowledge base. And I think doing a startup might require more consideration of marketing aspects, right? Things in the market, but I think there are really many commonalities between the two; it depends on what you like. Of course, there’s also the matter of timing, location, and people—whether you can start this company. There are many very good technologies, but if the time isn’t right, you still can’t start the company.

Tianyin Xu (Guest) (00:26:39)

But I think that there are very many similarities between the two.

Uncle Moon (Host) (00:26:42)

The feeling I get is that you’re the type of person who doesn’t want to start a company for the short term, right?

Tianyin Xu (Guest) (00:26:48)

Because this person is relatively free and easygoing, and I haven't thought about it too much yet.

Uncle Moon (Host) (00:26:52)

Yes, yes, although you don't want to start a company, you have gained a lot of successful entrepreneurial insights at a glance.

Tianyin Xu (Guest) (00:26:57)

Indeed, I was brainwashed, brainwashed for three or four years,

Uncle Moon (Host) (00:27:00)

and indeed there was no problem at all. In 2018, you completed your When you finished your PhD, you received faculty offers from five universities, and ultimately chose UIUC. Why UIUC?

Tianyin Xu (Guest) (00:27:07)

At the time, among the offers I had, UIUC was the fairly obvious choice. One reason was that my former advisor had been on the faculty at UIUC for seven years. He had many fond memories of the university, which certainly influenced me. More broadly, with any offer— whether in academia or industry— you need to consider what the place can offer you as a platform. What can you learn there? A faculty offer comes down to a few things. Can you find excellent collaborators? Are there good mentors? Are there senior colleagues you deeply respect and would want as mentors? Can you recruit outstanding students? UIUC checked all of those boxes. And Champaign suited my temperament very well. I really like Champaign. Now that I've come to the Bay Area, I still often miss Champaign and want to go back there. So I think UIUC is a pretty good choice,

Uncle Moon (Host) (00:28:08)

indeed different from others, haha. Most of the students I know who are familiar with Champaign feel that it's just a pure cornfield where you can't even stay for a day.

Tianyin Xu (Guest) (00:28:15)

Yes, it's a cornfield,

Uncle Moon (Host) (00:28:17)

so it's really a very pure area suitable for researchers.

Tianyin Xu (Guest) (00:28:18)

I think it's a very simple place without much else, and you can just stay there and think about some problems.

Uncle Moon (Host) (00:28:23)

I think now that you have been a professor for 8 years, what kind of students do you like the most?

Tianyin Xu (Guest) (00:28:33)

I think this question is too difficult. This question is actually an equivalent question, which is what kind of partner you want. And I think there is no answer to it. I once watched a documentary, right? It was directed by a documentary filmmaker. He interviewed many single men and women and asked them to describe what they wanted their future partner to be like. They would list very detailed descriptions, such as wanting someone who looks a certain way, has red hair, went to college, and so on. But five years later, the director interviewed them again and looked at how many of them already had partners and tried to match them, only to find that it was not the same at all. The spouse they eventually found was not at all like how they had described at the time, and I think it's the same with students—you can say it's really hard to get to know a person.

Tianyin Xu (Guest) (00:29:16)

Finally, it is said that you can work very closely with this person for five years and have a very close relationship, because I think the advisor-advisee relationship is a lifelong relationship. For me, it means thinking about what kind of person would ultimately be able to work out, that is, to establish a very personal relationship. Students like this can definitely work out, but if this relationship can't be established, I generally don't know how to work, that is, work with this student. Then I know this view is very controversial because many of my colleagues think this is fundamentally wrong. And I have a colleague whom I respect very much, at UIUC. His point is that you shouldn't establish a personal relationship with students.

Uncle Moon (Host) (00:29:54)

I've seen a lot of this kind of view, which is true,

Tianyin Xu (Guest) (00:29:57)

but this kind of thing really depends on how you think about it. But for me, I completely can't work closely with someone I can't establish a personal connection with—

Uncle Moon (Host) (00:30:06)

it feels like you can't even speak, right?

Tianyin Xu (Guest) (00:30:09)

Because I think, it's still that sentence, I feel doing research, especially very difficult research, is inherently a very hard thing, right? When you are in a difficult situation, you have to be very direct in saying some things. For example, I once watched a football match—in football matches, you can see those people, for example in the World Cup, when they play football, they are not playing while smiling. Especially those defenders, right? Many times you just have to yell loudly and then say whether you should handle the ball this way or not, or that you should kick long, or that you shouldn't kick long, or whatever, and then shout loudly, yell loudly, and get the team going. If every time you are smiling, that football team I think plays badly, and I think of it like doing research; every research is like an ambitious project.

Tianyin Xu (Guest) (00:30:50)

It definitely has very difficult moments, and at such times, how you can inspire the team to continue moving forward relies on a personal trust. Without trust, I personally don't know how to push this matter forward.

Uncle Moon (Host) (00:31:01)

In fact, what you just described reminded me of many scenes I have seen, for example, Gu AIling or Cristiano Ronaldo when they face certain challenges. Their eyes aren't happy; they are the kind of eyes that want to win,

Tianyin Xu (Guest) (00:31:16)

all of them twisted, right? For example, when you play in the NBA, look at how no one goes for a rebound with a smile on their face. Everyone is contorting their faces and giving their utmost effort to grab the rebound. I think research often has many moments like this too.

Uncle Moon (Host) (00:31:27)

The same goes for entrepreneurship. Many top entrepreneurs tell their teams that we are a sports team; we are here to win, not to fool around, and this is how we win. We are here to make the customers happy; we are not here to have fun ourselves.

Tianyin Xu (Guest) (00:31:45)

So, this kind of thing can only happen in situations of extreme trust and where a personal connection can be established. Right.

Uncle Moon (Host) (00:31:52)

So you're saying that you firmly believe there should be some good connection with a PhD.

Tianyin Xu (Guest) (00:31:55)

I think this is a very personal thing. This is my way; it doesn't represent a universal way. It's like the professor I just mentioned, who is also very successful. They think that everyone should just do things according to their own ideas. I think the best thing about the academic world is that everyone does things according to their own thoughts. There isn't a standard method here, so what suits you is the best for me. I can completely understand that point because the core of that point is that if you are too personal with a student, then the way you teach that student is actually biased, right? It is very biased by your own taste or preference, so it may not necessarily be the best for the student. On the contrary, if you have a very personal relationship with the student, when you think about the student, you might be able to give some very objective advice.

Tianyin Xu (Guest) (00:32:33)

But I can't do this myself because my personality is just not that kind of person. So for me, the best thing is to have a connection and be able to speak more directly about some things, right?

Uncle Moon (Host) (00:32:43)

It goes back to some of the topics we talked about at the very beginning, which is that people still need to be themselves. Being yourself makes it easier to be more consistent and makes doing things go more smoothly.

Tianyin Xu (Guest) (00:32:55)

Right.

Uncle Moon (Host) (00:32:56)

Is there a certain trait that makes you feel this person just shouldn't do research and makes you not want them to be a student?

Tianyin Xu (Guest) (00:33:03)

I feel like, I feel like, the word 'research'—if you break it down, what I tell the students is, what does 'research' mean? It's just 're' and 'search', right? It's this word, it's like this: research. 'Re' means doing something repeatedly, you do 're' many times. And 'search' is actually the process of searching. Anyone who is willing to continuously research, then they are a researcher, right? It is a very common practice, and then you can do research in the industry, you can also do research in academia, and you can do research anywhere, so I don't think research is a very narrow concept. Anyone can do research, of course, for me. The style of research in academia and industry is not quite the same, and what they pursue is also a bit different. So I think depending on what kind of research you want to do, you might need to choose an environment to do the research you want.

Tianyin Xu (Guest) (00:33:43)

For example, if you want to do a very imaginative research, right? You want to do a very theoretical research, and this research may not be very suitable to be practiced in some places. For example, didn't someone just get the Fields Medal a couple of days ago, right? Wang Hong and Deng Yu won the Fields Medal, and the research they did is very beautiful, right? It's very wonderful, but you must say that this research, if you're going to do it in, for example, a startup, then I think it might not be very suitable to do this kind of research. But in a startup, you can completely do research on your own product, and you will definitely reflect on it, right? Then go and think about how to improve or think about some fundamental issues, you're doing research. So I don't think there are people who cannot do research.

Tianyin Xu (Guest) (00:34:17)

Of course, whether to do research with me or not fundamentally comes down to whether the two of us can establish some connection and whether I can help this student. I think this is the most important question for me. So if I think some students may be particularly outstanding. Then I feel that I can't help this student, maybe I might not really want to admit this student as a PhD student, I think.

Uncle Moon (Host) (00:34:37)

Understood. What exactly do you mean by that kind of research being very high-dimensional? Everyone can do research. Researching the size of a button for a product in a startup is a kind of research because you need to 're' and 'search'.

Tianyin Xu (Guest) (00:34:48)

Right.

Uncle Moon (Host) (00:34:49)

Then you do mathematics in that school and proving the Goldbach conjecture is a kind of research.

Tianyin Xu (Guest) (00:34:53) †

Yes. I think the research you do at school is also very meaningful, right? Fundamental and applied research both matter.

Uncle Moon (Host) (00:34:58)

advanced way of thinking, meaning that as long as you seriously focus on innovation, pushing things forward, and optimizing, it all counts as research.

Tianyin Xu (Guest) (00:35:06)

Because back in 2017, didn't I spend a year at Facebook? Back in 2017, didn't I spend a year at Facebook? Facebook's systems research: it doesn't have a systems research department because Facebook doesn't believe in a systems research department completely separate from production. So all systems researchers are actually working on production services. So how do we do it? Actually, it's because Facebook is not a research company, right? We worked with many engineers to build some systems, and the way we conducted this research was that we built the system first, and then we reflected on it afterwards. We looked at why we did it that way at the time, possibly because there weren't other good ways to do it, right?

Tianyin Xu (Guest) (00:35:44)

Because when you are actually going live in production, you are really nervous and you don't really have much time to think about this, then you keep thinking about where this can be improved, where it can be done differently, or what the costs and benefits are. As long as you do this, it is research, and we also document all those production services that we have done. I ended up writing a paper and published it on OSDI.

Uncle Moon (Host) (00:36:01)

This kind of paper is my personal favorite — prolific and really flavorful, that's really how it is. So what

Starting a PhD and What AI Can Replace

Uncle Moon (Host) (00:36:08)

do you think a PhD student in the very early first or second year should do if they join your group or if you are mentoring them? Go read 100 papers and then jump in to work on a project with a fifth or sixth grader, or just think about it and write code by yourself over there.

Tianyin Xu (Guest) (00:36:25)

Right. Because I feel that a PhD is a very personal thing and it doesn't have a standard answer, so first of all, I don't think reading 100 papers is a very good practice. Then I remember my PhD advisor at the time was actually very opposed to reading papers to get ideas. He had a rather radical point of view: he thought that every paper is very biased and is just the author's own perspective, right? Actually, each paper is essentially a perspective, it's the author's own viewpoint. And you know, the current practice is that top conferences are very rigorous, so everyone uses a lot of techniques when writing papers and is not necessarily that honest. So, if you read a lot of papers and try to get ideas from them, it’s not a very good way to innovate. Sometimes, you probably shouldn’t even read them, at least that's what I personally think.

Tianyin Xu (Guest) (00:37:10)

I feel that many papers are like feed; if you consume too much feed, your brain becomes numb, and then you won’t be able to innovate. So I think the best way is not to read papers to get ideas, but to come up with ideas based on real problems. Here is a very important question: we need to solve this problem, right? Then, then how do we solve this problem? If this problem is important and difficult, it requires research, rather than just because there is a paper that won the best paper award and you decide to read someone else's limitations section and then write your next paper. I think this is not a good approach, so I actually do not advocate reading 100 papers from the beginning.

Uncle Moon (Host) (00:37:41)

That recent college graduate started reading for a PhD, and he doesn't have much experience in discovering really impressive

Tianyin Xu (Guest) (00:37:48)

I think this is part of an advisor's job. An advisor should give students good questions to pursue. on. Basically, I feel that all the PhD students I know started like this. Then, working on projects together with senior students, I think it's a very good practice. My former advisor did it this way, and now at Berkeley, basically every student does this when they enter Berkeley. And then Berkeley is more of a bigger lab, right? This professor-student interaction is not very hands-on most of the time, and actually, you end up learning more from your peers than from the professor. This was also the case for my own group. I think working with a senior student on some difficult problems is a really good way to start, and you can also learn a lot and build some connections. Then if you have some questions, you can just come up and do them.

Uncle Moon (Host) (00:38:36)

Right. In the face of such a situation, especially in the current AI era, you don't even need to interact with peers; you can just interact with agents every day and continuously produce a large amount of content. So what is the meaning of university then? What is the meaning of teaching?

Tianyin Xu (Guest) (00:38:49)

I personally feel that agents are still very difficult to replace—at least for now, agents are still hard to replace. Agents are quite a cold thing, I think. Then the agent is not really something that considerate or comprehensive. I think guidance from a person is still quite important, whether that person is a senior student or an advisor. I think the agent has many capabilities; it can improve your coding productivity. Right, the productivity of system building, and it can also help you come up with some ideas, but in the end, even if it helps you come up with ideas, you still need to figure out yourself, how should I put it? So basically, it's about understanding these ideas and figuring out which ones can be done and which ones can't.

Uncle Moon (Host) (00:39:25)

Overall, my impression after listening is that you have to trust people's judgment and taste, right? These things are irreplaceable by agents in a short period of time.

Tianyin Xu (Guest) (00:39:31)

I don't really feel that, I actually don't believe that in the future the world will be completely replaceable by agents. I still think that the future world will be a world of humans, but a large amount of labor will be taken over by agents.

Uncle Moon (Host) (00:39:40)

Even if AGI is realized, don't you think it will be like this?

Tianyin Xu (Guest) (00:39:46)

I don't know what your definition of AGI is? It's just what I think. First of all, personally I feel that thinking about what will happen after AGI is achieved is quite a waste of time. To me, it's like thinking about what to do after the apocalypse. It's the same thing, it's a fun thing to think about, but don't take it too seriously for me. And then if there's a student who is always panicking there, saying that AI will replace everything I've learned and it's all useless, then essentially you can't make progress, right? And on the contrary, I think we shouldn't overthink things. Let's just think a little about the future—what can agents do now? Then we can use current agents to improve our learning abilities and work skills. I think this is a more practical approach.

Tianyin Xu (Guest) (00:40:26)

Yes, because if in the end people become slaves to AI, then I believe human learning ability becomes even more important, otherwise how would we compete with AI?

Uncle Moon (Host) (00:40:37)

Skynet, right?

Tianyin Xu (Guest) (00:40:38)

Right, but if we, if we people are doing the right thing, then AI is a servant of humans. Essentially, it's still humans who decide how this world operates, right? So your own abilities, your own learning ability, and your own work capability are still very important.

Uncle Moon (Host) (00:40:50)

Just now, we went through a short half-hour interview, but I could feel from all your words that you are a natural systems researcher.

Tianyin Xu (Guest) (00:40:58)

No.

Uncle Moon (Host) (00:40:58)

It's just that this feeling is very obvious, very obvious, because this system really requires you to lower your head and think carefully about the tasks in your hands every day, doing them step by step. It works through doing; it doesn't require you to think about what will happen in 5 or 10 years every day. As long as you work on this system bit by bit, step by step, it can eventually be completed. With your personality traits, my goodness, you are really suited for working on systems.

Tianyin Xu (Guest) (00:41:19)

Research still needs to be done continuously, I think, anyway, I feel that you definitely have to do research, you absolutely need to be open-minded, and you need to know what is happening around you, right? But it is indeed necessary to have a main thread of

Writing and Reviewing Papers in the AI Era

Tianyin Xu (Guest) (00:41:32)

your own, I think.

Uncle Moon (Host) (00:41:33)

OK, so our next question is that many professors now say that theses need to be reformed, and that they no longer require the traditional sections. Some propose using only an abstract and introduction you want to tackle this problem, and then directly include the dialogue with the agent. What do you think of this opinion?

Tianyin Xu (Guest) (00:41:48)

I think this opinion might be suitable in some cases. It holds true under a specific context, right? I think it's because, for me, the key to a paper is what the contribution of this paper is, right? It's just that I don't feel like, for example, the Fields Medal winners I just mentioned, like Wang Hong and Deng Yu, right? I don't think their paper should be presented in this way. in other words, if the contribution of your work ultimately is an artifact, and this artifact is very useful, everyone can use it and uses it well, then perhaps your method is enough, and you have an abstract to tell others how this artifact is used. Then it later became an open-source project, and it might even be possible to implement an agent there. But a lot of research is actually a process of thinking, because I feel that the essence of research is understanding the world, right?

Tianyin Xu (Guest) (00:42:30)

Regarding your understanding of this matter, I think it shouldn't be entirely presented through AI. Leslie Lamport is a Turing Award winner, and many of our current works actually use some of the tools he developed. But there is one viewpoint of his that I really agree with: if you don't write something down, you actually can't fully think it through. So I think the writing process is a process that helps people truly understand this. Personally, I really don't think it should be skipped.

Uncle Moon (Host) (00:42:57)

I can't agree with this view at all because I also think that the main training in a PhD, this process of writing a paper, is precisely a process of training your brain.

Tianyin Xu (Guest) (00:43:08)

Right.

Uncle Moon (Host) (00:43:09)

You can't write clearly; this question of yours definitely hasn't been thought through.

Tianyin Xu (Guest) (00:43:12)

Yes. So I think a lot of fundamental things haven't been changed, and many viewpoints are brought up because the current review system is under a lot of pressure. Since AI has come out, everyone's productivity has increased, and there are a lot of paper submissions. This includes, for example, some traditional systems where the circles are very small, right? The system's community is particularly small, and in the past, there might have been only 200 submissions per year, but now it has become 1,000 submissions. If you continue to handle things according to the original organization, the pressure now is very high, and after the pressure increases, your system is actually not scalable. When your workload scales up, your system begins to break. I think what we should reflect on is this.

Tianyin Xu (Guest) (00:43:48)

How to make this system work well so that it can scale up to the current volume of submissions, rather than simplifying it into another system—for example, if you remove the writing part or the thinking part, I think the essence of the research is broken.

Uncle Moon (Host) (00:43:59)

I think that's indeed very reasonable; we really need to find a way to improve the system's scalability.

Tianyin Xu (Guest) (00:44:07)

Right.

Uncle Moon (Host) (00:44:08)

I’ve heard Terence Tao say before that mathematicians nowadays are different from mathematicians in the past. Previously, mathematicians had to come up with many ideas. Now ideas have become so easy, mathematicians have to constantly review and verify to validate these ideas. We don’t know what might happen next, like how this paper system will turn out. But we definitely need to undergo some reforms,

Tianyin Xu (Guest) (00:44:29)

and I completely agree with this. I think I completely agree, which is to say, we should think about what systems research should change in AI era. It should do it, right? Because I feel that AI has raised this bar. In the past, without AI, you might have been able to do some research, and the idea for this research wasn't particularly interesting, but it involved a lot of engineering work, or it required some special skills to accomplish. Now, this kind of research has become relatively difficult because AI can help you do these things. So what this means is that in AI era, the way you do research is different. It's about how you push the frontier because this frontier is different, and the way you push the frontier is different. But essentially, the nature of research, I think, hasn't changed. A good researcher is still a good researcher. Terence Tao is still Terence Tao, right?

Tianyin Xu (Guest) (00:45:12)

It's not like anyone uses AI and instantly becomes more powerful than Terence Tao. I don't think that's the case.

Uncle Moon (Host) (00:45:17)

It's obvious, especially now, that after having AI, those experts who can write papers and do research are more efficient.

Tianyin Xu (Guest) (00:45:25)

Right. I think AI can increase a person's productivity by 5 times, 10 times, or even 100 times. Some people say, 'I think so,' but the question is, what is your baseline? If your ability is 0, then multiplying by 100 is still 0.

Uncle Moon (Host) (00:45:33)

I also want to point out this: when we talk about this 100 times increase, it means that for someone who is already at 100 points, it becomes 10,000, but for someone at 1 point, it becomes 100. Oh my, the gap becomes even bigger.

Tianyin Xu (Guest) (00:45:42)

Yes.

Recruiting Students and Finding Real Problems

Uncle Moon (Host) (00:45:43)

Right, so let's go back to your research itself, from your PhD to you've won more best paper awards than almost anyone I know. received are among the most I have ever seen. So, how do you view these awards, even though you just kept saying you're not optimistic and that it doesn't matter?

Tianyin Xu (Guest) (00:45:58)

I think these are all just luck for me,

Uncle Moon (Host) (00:46:00)

but if a person's luck keeps happening repeatedly, I consider it to be a reflection of a kind of standard of ability,

Tianyin Xu (Guest) (00:46:05)

not that I think otherwise. I actually don't agree; I just feel that it was a matter of luck at the time. For example, in the past two years, I haven’t won as many awards for my papers, but I don’t think the quality of my papers has dropped that much.

Uncle Moon (Host) (00:46:19)

So looking back at having published so many good papers, how do you think it’s possible to continuously produce this kind of good research?

Tianyin Xu (Guest) (00:46:26)

I actually think doing research is very simple. If you hire the right student, they will naturally do top research. Doing top research is about a person's quality; with that quality, they can produce very good work. So the real issue here is how to hire the right person and then how to help this person understand the method of doing this work. As long as you do these two things well. After that, there are some results.

Uncle Moon (Host) (00:46:46)

I think let's break down this step. First, how do you recruit this person, and second, how do you bring this person in? Let's break down

Tianyin Xu (Guest) (00:46:54)

how to recruit this person. I think it's completely based on gut feelings for me because I'm not someone who thinks about things in detail. For example, when I recruited students, I just chatted with them for a while. Then see if there is any chemistry because personally I don’t believe I have it. I have a friend, this friend of mine is really interesting. He just wants, he uses dating apps, and after he uses dating apps, he often goes on dates with others, right? Then think about how to go there, and I asked him if dating apps are accurate? Because dating apps are basically where you initially arrange to meet someone and spend a night with them. In the end, whether this person will become your long-term partner, he says, even spending just one night isn’t acceptable, right?

Tianyin Xu (Guest) (00:47:36)

Then I think, as a student, you interview this student for half an hour, but could this person be someone you have to work with for the next five years? So I think this is too difficult, so it's more about seeing fate or chemistry, just looking at some gut feelings of yours.

Uncle Moon (Host) (00:47:45)

Let's break down the second half: the student is recruited, so how do you help him publish good papers?

Tianyin Xu (Guest) (00:47:49)

Publish papers, but the same sentence: that's not our goal, right?

Uncle Moon (Host) (00:47:55)

So how should we...

Tianyin Xu (Guest) (00:47:57)

I think it's about letting them do the research they want to do, because I think every student... His goals are also somewhat different. Some students might want to start a startup; he doesn't necessarily say they have to publish papers. I think that's also a very meaningful thing. But maybe the research he wants to do is different from a student who says they want to go to another school to become a professor or to make some breakthroughs in fundamental research. The ways to train these two people are different. It should be very different, so I don’t think there is a magic formula. I just think that if you have a very deep connection with this student, you should be able to understand what this student really wants. What does he like to do and what is his ultimate goal? So what things can make him very happy and what does he need?

Tianyin Xu (Guest) (00:48:34)

Then after you have sorted all these things out, I think you should be able to help this student, and I think anyone should be able to help this student. But the most difficult part is how you can establish this connection, how you can understand this student, and then help him find some problems or resources or something. Many times it is also very difficult; I think the hardest part is when the student says, 'I don't even know what to do.' I don't know what I like either. At that stage, you have the student try different directions. Then you ask which ones they genuinely enjoy, and use your experience to help them choose a topic.

Uncle Moon (Host) (00:49:01)

Listening to you describe it, I'm reminded of the classic master–apprentice tradition in research. You know? There's something beautiful about that kind of relationship. Today, many professor–student relationships feel more like employment. But in the older European and American tradition, the model was different: established scholars personally guided students through an apprenticeship. From what you've described, that kind of relationship is something AI agents cannot replace.

Tianyin Xu (Guest) (00:49:34)

Right.

Uncle Moon (Host) (00:49:34)

You understand the person.

Tianyin Xu (Guest) (00:49:36)

Exactly.

Uncle Moon (Host) (00:49:36)

And you can truly work with them.

Tianyin Xu (Guest) (00:49:37)

Right.

Uncle Moon (Host) (00:49:37)

Then together you can do meaningful work that advances technology.

Tianyin Xu (Guest) (00:49:40)

What could be better than that? It's genuinely enjoyable, I think. but it’s just about doing interesting things, right? Life is very happy,

Uncle Moon (Host) (00:49:46) †

really great. Even though you are very humble, always saying 'best paper', it doesn't matter, but on your homepage you said a sentence: 'fight dragons, not windmills', which means to conquer the evil dragons, not the windmills, right? How does that windmill facing the wind manifest in your interactions with students?

Tianyin Xu (Guest) (00:50:04)

I think that the evil dragon actually represents a real problem, while the windmill represents a false problem. So I believe that it is very important to solve a problem that is truly meaningful, and solving a false problem, I think, doesn't have much significance, especially for systems research, because systems research is something very close to practice. So if this problem is not very important. I think going to solve it might not be that meaningful.

Uncle Moon (Host) (00:50:27)

Two points: the first point is how you help students discover this problem; the second point is, in a cornfield in Illinois, Champaign, so far away from Silicon Valley, how do you discover the real problem?

Tianyin Xu (Guest) (00:50:36)

This is a very profound question. I think, although this... firstly, the second question is quite simple because I think... Even the most important problems in the field of systems are just a few problems; there aren't that many fundamental problems. But the manifestations of these fundamental problems in actual systems are varied. So personally, I always have a view that many people disagree with, but my view is that I think there is no such thing as a small problem, only some problems are not explored in depth. in other words, for me, there are only that many fundamental problems. Then there are some fundamental issues in this, and in the practical implementation of the system, its reflection may not be the same. Some problems may seem big, some problems may seem small, but if you dig deep enough, if it is a difficult or important problem, it will inevitably return to a very fundamental deep issue.

Tianyin Xu (Guest) (00:51:24)

For example, many reliability issues ultimately come down to a few fundamental problems, such as how to reason about concurrency, right? How to reason about asynchrony and then how to reason about some nondeterministic behavior. If you solve this problem, then the higher-level problems are all solved. But these problems are very difficult, so even if you have a very small problem, you can eventually make it very big. And actually, I suggest this to students. At the beginning, start with a relatively specific and smaller problem, rather than going for what we call 'boiling the ocean.' If you start too broadly, it will be difficult to handle. It's better to be focused with a specific issue, which will make it easier to manage. You want to dig as deep as possible, but just do your best, and work to whichever level you can, then move on to the next project. Can we go on to solve some more fundamental problems, right?

Tianyin Xu (Guest) (00:52:08)

This is the second one, this is your second question I think, and then the first question is, what is the first question?

Uncle Moon (Host) (00:52:13)

It's about how you help students find this most important question, right?

Tianyin Xu (Guest) (00:52:16)

this, as I said, depends on the student's interest. Some students may prefer theoretical questions, or some students might really enjoy reasoning-type things. Some students really like hacking; their greatest strength is being able to hack a very complex thing very clearly. At this time, based on their interests, you can help map it to something for them. Compare the more important issues in reality and then start working on those issues. These issues include some students in my own group who like to do formal verification, right? Formal verification means you need to define the specification, use the correct tools to write proofs, and then finally the entire system can be formally verified. Some students prefer the empirical method, which might include ourselves; I myself, when doing my PhD, looked at many empirical cases. Then think about how to solve these problems through some automatic methods.

Uncle Moon (Host) (00:53:04)

I like the second one. And when

Operating Systems Courses and CS Education

Uncle Moon (Host) (00:53:07)

you were at UIUC, you took on some responsibilities, like teaching the operating systems course. Regarding responsibility, your operating systems course, I saw online that students rated it as really awesome, like extremely impressive.

Tianyin Xu (Guest) (00:53:20)

There is nothing, nothing, nothing—

Uncle Moon (Host) (00:53:22)

it's just about how you teach the operating systems course and why students like this course so much.

Tianyin Xu (Guest) (00:53:29)

I feel like I don't know why students like it either. Maybe it's because I told a few jokes in class.

Uncle Moon (Host) (00:53:35)

I think I should tell everyone about it.

Tianyin Xu (Guest) (00:53:38)

I feel that this operating systems course doesn't really teach operating systems, at least for me, because at UIUC, the operating systems course is actually divided into two courses. One course is called system programming, and it basically teaches everything in operating systems except for the kernel. It's just that it doesn't teach kernel programming, but it teaches everything else, including concurrency, process control, memory control, all those things. This course is also very good. It was taught by a colleague of mine named Lawrence Angrave, who manages it, and it's very good. And for the major assignment, it might be like having you write a malloc, right? Then they have you write a shell and these kinds of things. Actually, this course has already covered most of the basic training. The next course is actually about kernel programming.

Tianyin Xu (Guest) (00:54:16)

But if you look at the students taking the kernel programming course, I think less than 10% of the students each year, maybe even less than 5%, I believe that after graduation, these students will go on to do it. For example, someone coming to the Bay Area to be a Linux developer—people like that—5% of them, 95% of them, after they graduate, they won't end up being a kernel developer. What they might do later is, for example, hacking Kubernetes or building some AI infrastructure, right? So what exactly did Linux kernel hacking teach him? So I think the reason why this is valuable to them in this course is not that it completely explains this one thing. Kernel hacking is a medium, but the skills they gain are how to manage this complexity. Because before this OS course, basically all the courses you took, for example when doing big assignments, you were building something from scratch, right? Like your data structures.

Tianyin Xu (Guest) (00:55:10)

You write a small data structure, in your algorithms class you write a small algorithm, and even previously you wrote a small malloc, wrote a small shell; they are all very small things, but what about our class? You are going to hack Linux, but Linux is a very complex and very dirty thing. For a student, in just one semester, it is actually very difficult to fully hack and understand it. So, in this process, how can you make progress? Actually, this skill set is very useful in a company, including for myself when I went to Facebook. I am facing a huge codebase, and then you want me to be able to make progress in a month and build some things. So I need to have the ability to abstract a lot of things, understand the complexity, and still be able to make progress in order to communicate, right?

Tianyin Xu (Guest) (00:55:48)

Then the documentation isn't there, and the code is all very hacky, and this is a problem that exists in Linux as well. Of course, Linux documentation is actually quite good, but some things in device drivers are not so, not so elegant. And I think that if a student, after finishing this course, can gain the confidence to face, as you just said, a fierce dragon or a world that is not so perfect. I think this is the biggest skill that this class teaches them, so I think teaching OS is meaningful, not that I want to train all 100 of them to become kernel hackers.

Uncle Moon (Host) (00:56:20)

I feel really deeply about this, haha. Having graduated and worked for three to four years, I really increasingly feel that if you are an undergraduate majoring in computer science, courses like OS, operating systems, and network systems really need to be studied well. What I regret most now is that I didn’t study these courses thoroughly enough because you will find that no matter how big data infrastructure, cloud infrastructure, or even AI infrastructure develop now. It hasn't escaped the original one, which is the graphical architecture, including the basic ideas of computing, storage, and networks, but? It's about how you make the complex things in your mind become abstract.

Tianyin Xu (Guest) (00:56:58)

Right.

Uncle Moon (Host) (00:56:58)

It is about becoming clear when working on a problem, such as taking on a major task at a company or even a big entrepreneurial task, breaking it down step by step into the smallest units, and then stacking these units back together step by step.

Tianyin Xu (Guest) (00:57:10)

Yes.

Uncle Moon (Host) (00:57:11)

I found that

Tianyin Xu (Guest) (00:57:11)

I completely agree

Uncle Moon (Host) (00:57:11)

that the greatest significance of these courses for me is actually that I think these courses teach engineering thinking, computer engineering thinking, which will not be affected by external influences. As long as you have this kind of thinking, no matter how AI develops, you will be able to find a highly valuable job in this society.

Tianyin Xu (Guest) (00:57:31)

I actually agree with what you said,

Uncle Moon (Host) (00:57:32)

that there are some arguments now. It means don't study CS anymore. CS has already collapsed. I don't know how to put it. I think maybe for some parts, like the simpler kinds of software engineering development, the demand is indeed getting smaller, but overall, for building these big kinds of projects, I think the CS discipline will still not disappear. This discipline will not disappear.

Tianyin Xu (Guest) (00:57:54)

I don't think it will disappear. Because I feel that such big statements are all comparative, and I find it hard to understand what the argument behind them is. I personally do feel that the CS job market isn't as big as it used to be because AI can take over a lot of tasks, right? In the past, even some very simple tasks required you to hire a CS undergraduate student to do them, but now AI can help you with that. So I think, if what we are worried about is that this market will shrink, I do think it will shrink. But if you say that OS will disappear, I don't think that's possible. It is still something that requires people, just like any other profession, it will still require people to do the work in the future.

Uncle Moon (Host) (00:58:31)

The entire current modern society is building large-scale computer systems.

Tianyin Xu (Guest) (00:58:36)

Right.

What Systems Work Can AI Do?

Uncle Moon (Host) (00:58:38)

Then let's talk about the relationship between AI and these systems next. You've been studying for over ten years. From what I see, the main thread of your papers is actually quite clear to me: how to find problems when systems have issues and how to fix them, mostly focusing on reliability and mostly on fault tolerance. in other words, there are now many kinds of AI that are essentially SRE. It's about letting AI debug this,

Tianyin Xu (Guest) (00:59:01)

which is particularly popular right now,

Uncle Moon (Host) (00:59:02)

super, super popular, even getting funding is popular, and then.

Tianyin Xu (Guest) (00:59:06)

Yes.

Uncle Moon (Host) (00:59:06) †

Is this the one that's popular in academia too?

Tianyin Xu (Guest) (00:59:08)

I wonder if it's not popular in academia? I don't think the fire affects that.

Uncle Moon (Host) (00:59:12)

Maybe we can talk about it later. So, what do you think these AIs are actually capable of doing in terms of human SRE work, and what can't they do? Where are the boundaries?

Tianyin Xu (Guest) (00:59:21)

Actually, I think the capabilities of AI now, right? Regarding the capabilities of those current frontier models, like Fable or that GPT-5.6 Soul, right? Then I think their abilities are already very strong, so basically I feel that for any system task, Siri task—I’m not saying, I mean any system task—if you tell AI how to do this task, AI is basically capable of doing it. So if the question is whether AI can do these tasks, I think it definitely can. The key question is how you go about doing it and what the cost is, I think. in other words, if we can lower this cost, right, and also if we can have a more general method to tell AI how to perform these tasks, then I think AI can definitely do these things well. I don't think AI would do poorly, and I even don't think AI lacks the abilities to do these things.

Uncle Moon (Host) (01:00:08)

So, what role does a human play in this?

Tianyin Xu (Guest) (01:00:10)

I think the role a person can play in this is what? I have always had a thought, and I don’t know if it’s a very stupid one, about guiding AI to do some right things. Because for me, AI is like a very smart person. It’s like in a company, let’s suppose your company has an enormous number of people, and all these people are very hard-working, you still need to tell them how to do the right thing. I think that's the role people should play right now: showing junior engineers what good work looks like. how to go about doing this, how to think about your trade-offs, and what you should do? How to decompose this task and then tackle it step by step, then let AI handle some of the repetitive work.

Tianyin Xu (Guest) (01:00:49)

But I don't think AI—at least with its current capabilities—if you just give it a very vague definition or specification, it can actually do it, but it may require many iterations, which would be very costly.

Uncle Moon (Host) (01:00:59)

And this introduces another problem. It's just that people go to guide these AI agents and then let them complete some tasks, and moreover, they are experienced and tasteful people. So the question arises: how can a fresh graduate become such a person? Then there is no such training guidance process, and

Tianyin Xu (Guest) (01:01:18)

I think this is a huge impact on current CS education. I think,

Uncle Moon (Host) (01:01:24)

let's imagine how we could change this to help current CS students gain this experience.

Tianyin Xu (Guest) (01:01:30)

I think that now students should use AI to do some learning and work. I actually don’t really understand why these things are considered difficult to learn now, because originally, as a senior engineer for example, you accumulated that expertise over years of hands-on learning. what they need to do is to... By guiding AI to do this, as a learning student, you just practice this. You practice your learning through AI and also practice your research through AI, then naturally you will reach this level.

Uncle Moon (Host) (01:02:05)

There are two problems: the first problem is that you are too junior. Some of your taste and guidance may not be enough, and then AI output might tell you some things that are not really high-quality or high-level. That's the first problem. The second problem is that these positions are becoming fewer and fewer. Originally, people would slowly progress through small positions, which would allow them to learn interpersonal communication and improve themselves.

Tianyin Xu (Guest) (01:02:34)

Right.

Uncle Moon (Host) (01:02:35)

So these two issues lead

Tianyin Xu (Guest) (01:02:36)

me to think that the first issue is what I mentioned before: I feel that people are still a very important element in this, and we shouldn't imagine an action that is completely without humans. So I think things like advice or mentorship will not be replaced by AI. At least, that's my point of view. in other words, if you have such a growth environment, you can still achieve something using AI. Very good quality, and then the market has shrunk, which is a very real problem, but actually I don't really know how to solve this problem.

Uncle Moon (Host) (01:03:05)

Then let's all reflect and discuss together, so that the audience can chat in the bullet comments and the comment section.

Tianyin Xu (Guest) (01:03:11)

Right.

SREGym and Simulating Production Failures

Uncle Moon (Host) (01:03:11)

You've recently begun working on AI for systems. One project I found especially interesting was your work on AI for SRE. You built SREGym, an environment where AI agents tackle site reliability problems. It contains 102 real-world IT operations scenarios, yet agents solve only 11.4 percent of them. What did you observe? Why could the agents solve so few? What were they missing?

Tianyin Xu (Guest) (01:03:48)

Those results were from about a year ago, I believe.

Uncle Moon (Host) (01:03:51)

Yes.

Tianyin Xu (Guest) (01:03:52)

First, I should point out that my students led this project. I didn't initiate it, so I can't claim it entirely as my own work. As for the original benchmark, I think today's models have largely solved those tasks. Run it again with the latest models, and I think they'd score close to 90 percent.

Uncle Moon (Host) (01:04:13)

That much progress in one year?

Tianyin Xu (Guest) (01:04:14) †

Yes, in just one year. AI capabilities have advanced dramatically. AI's capabilities, anyway, there was a big fluctuation at the end of last year, that is,

Uncle Moon (Host) (01:04:21) †

Fibonacci

Tianyin Xu (Guest) (01:04:21) †

is not correct, not 4.4, 4.6, 4.8.

Uncle Moon (Host) (01:04:23) †

Right.

Tianyin Xu (Guest) (01:04:24)

At the end of last year, I felt it was the end of last year and the beginning of this year, that is, the end of 2025 and the beginning of 2026, I felt there was an essential capability, an essential leap, and also, recently this year, anyway, a new wave of models came out, and I think its ability has increased again. I felt that during last summer, its ability was still average, and this work was actually done at the end of the year before last, I think. So at that time, AI capabilities were not that strong, and there were various problems, including very high reasoning hallucinations, and its context was not that large, so its capabilities were very limited. But now, these problems that existed before are actually very easy for AI to overcome. The fundamental issue here is that the problems we originally wrote were very simple.

Tianyin Xu (Guest) (01:05:08)

Because I think the practice of benchmark research is that it gradually builds the benchmark according to the capabilities of AI, right? Because a benchmark, a good benchmark, can measure the boundary of AI, so when AI capability is very weak, if you create an extremely difficult benchmark and AI cannot complete any of it, then this benchmark is actually not good. And of course, if an AI is very strong and can complete 90% of the benchmark, then this benchmark is basically a bit useless as well. So the best benchmark is when you can say that 30%-40% can be done by AI, and 60% AI cannot do, so it can keep challenging AI. Now, actually, the biggest challenge of our benchmark is this, and a very clear problem is that a benchmark itself is originally a small, small environment, right? As for us, we can only have a small environment.

Tianyin Xu (Guest) (01:05:50)

Although our benchmark can actually have some cases, for example, a system with three nodes or five nodes, it is still very different compared to hyperscale systems like Meta's. So AI is very strong in these small environments, but whether it is strong in these huge environments, I think, is not necessarily the case. But how do you scale a small one... How does the environment simulate a huge and complex environment? Which things should be simulated in this small environment and which things can you abstract out, and this is a point we pay quite a lot of attention to, and this is actually quite difficult, quite difficult.

Uncle Moon (Host) (01:06:23)

Right.

Tianyin Xu (Guest) (01:06:23)

Because since you go from a big system to a small system, you inevitably have to throw some things away. So what do you throw away, right? What I mean is that if the things you throw away are very important, then this small simulator of yours becomes useless. But if you want to keep everything, then essentially you can't turn a big system into a small one. Yet you need the small one because if you want to do RL, you can't do it if it's too complex or online. So I think this is one of the challenges of SRE. Then the coding benchmark doesn't have this problem because the coding benchmark is an offline thing—you just keep generating code, right? Then going to do it

Uncle Moon (Host) (01:06:53)

for SRE is too real-time, meaning that after you solve a problem, can it really solve the issue, fix the online bug, and get feedback? And the whole feedback process isn't offline, it's online.

Tianyin Xu (Guest) (01:07:05)

So what we want to solve now is this issue of fidelity.

Uncle Moon (Host) (01:07:06)

Can you explain this in detail?

Tianyin Xu (Guest) (01:07:10)

It's about how to simulate a specific production failure. For example, recently our school has a program, an undergraduate research experience program called the REU program, and this summer our program is letting those undergraduate students... What did he do? He just went to read a postmortem, this postmortem could be from those high-tech companies, like Cloudflare or something. Then they announced some failures, meaning they said that at a certain time they had a failure, and they put the detailed description of it online. Then the students, based on their description, implemented the failure in this relatively small simulated environment and tested AI, and we found that even though this was a small environment, it still didn't… It can completely represent this larger environment, but compared to the most basic failures the realistic failures you described pose a very different challenge for AI. when facing these more realistic failures.

Uncle Moon (Host) (01:07:58)

I heard two points of information: the first point is that I think students can indeed gain a lot of training in this process, and the whole school helps them to find it. He can get hands-on with some of the real problems of these big companies in advance. The second point is that in the process of abstracting it into a small simulator, this simulator can later be used for RL.

Tianyin Xu (Guest) (01:08:24)

Right.

Uncle Moon (Host) (01:08:25)

this, nowadays this kind of RL environment company simulator company is really popular in Silicon Valley,

Tianyin Xu (Guest) (01:08:30)

right?

Uncle Moon (Host) (01:08:31)

It's too hot, too hot. Any environment that can truly help this model improve its capabilities—issues that previous models couldn't solve can be solved after training on it—then this environment will be sold at a very high price.

Tianyin Xu (Guest) (01:08:43)

I see.

Uncle Moon (Host) (01:08:46)

This is something that has already become an unspoken convention in Silicon Valley. So, regarding this, moving forward, SRE. This environment will increasingly grow, I can feel it will grow more and more, and then AI needs to learn through such an environment to continuously study the ability to solve industrial-level production problems in the real world.

Tianyin Xu (Guest) (01:09:07)

Yes.

Uncle Moon (Host) (01:09:08)

That's great, it feels like UIUC is already ahead and has started training students.

AI-Assisted Formal Modeling

Uncle Moon (Host) (01:09:14)

My next question is about another project. Before this interview, you also published a new paper. I found that work fascinating. It combines AI agents with formal methods to improve testing.

Tianyin Xu (Guest) (01:09:25)

Right.

Uncle Moon (Host) (01:09:26)

Could you walk us through how it works?

Tianyin Xu (Guest) (01:09:30)

Sure, what we are actually doing with model checking is... what is the essence of model checking? The essence of model checking is that you have a very detailed low-level code, and then you need to write this code into a high-level abstraction, and this abstraction is described using a mathematical language. This mathematical language can have various forms, right? The language we are using, called TLA+, was written by Leslie Lamport, whom I mentioned earlier. And if you can describe this system using a formal language, then you can do a lot of formal analysis called formal reasoning. You can reason about many properties of this system, such as safety. Safety and liveness: safety obviously means that bad things will never happen, and liveness means that good things will eventually happen. But as long as you describe it as a formal model, you can check these properties. The most difficult part here is how do you?

Tianyin Xu (Guest) (01:10:22)

You're going to write a high-level description of a very low-level system, right? What about an abstraction? So this matter is very intellectually challenging, and for a PhD student, this could be four years of their time, four years of their life. Before AI, we had a project and a very talented student who is now a researcher at Microsoft Research; he spent a full five years doing this work. Why does he have to spend five years? It's because he has to learn this formal stuff from the beginning. TLA+, this formal stuff, is actually not taught in any undergraduate courses, right? This wasn’t taught in the course, right? You might spend a year or two to learn this set of things thoroughly, and then what happens after you have learned it? You also need to write a specification. We say that specification is a very overloaded term. Actually, you are writing a formal formal models and invariants.

Tianyin Xu (Guest) (01:11:08)

Invariants are the properties you want to preserve. To write them— to build a good model of a system— you must understand that system very deeply. Otherwise, you can't abstract it properly. You have to know the system inside and out. For a systems researcher, that should be a real system. Let's use an example. Uncle Moon, what's your favorite system? A distributed system?

Uncle Moon (Host) (01:11:28)

My favorite distributed system would be Google's Borg, or perhaps Kubernetes.

Tianyin Xu (Guest) (01:11:32)

Kubernetes. Okay. When we started, we used ZooKeeper. ZooKeeper is a consensus system. already very complex. It has a consensus protocol called ZAB. So, what about this student? He has to go understand all the details inside ZAB and be able to understand how ZooKeeper implements ZAB, and then understand all these things? He can go and write a formal model, and after he finishes writing it, your system is changing, your code is changing, and tomorrow there will be a new release. You still have to follow the evolution of the code to write a different model, right? This matter is very costly and also requires a lot of skill, so the student named Ruizé, Ruizé, spent four or four to five years writing all of these things, and after finishing writing, then? You can plug in all those formal and very good tools and then let them translate and then discover the problems inside.

Tianyin Xu (Guest) (01:12:15)

You can do model checking, and you can also do proof. Proof is a stronger guarantee, but this requires doing for 5 years, and in 5 years we only completed one system. But after we completed this system, then what? We solved a lot of problems in this system that hadn't been solved for many years, right? There was one issue, I remember, in ZooKeeper, and many people had studied it before and it was very complex. But through formal reasoning, it could clearly tell you what was going on. However, it was very expensive; it would require a PhD to spend five years to do this. But now, with AI, it's different. It turned into a five-hour job—

Uncle Moon (Host) (01:12:47)

is Is the improvement really that dramatic— down to just five hours?

Tianyin Xu (Guest) (01:12:50)

Yes, five hours. Five years of work became five hours, because Five hours of work because of AI — it doesn't need to learn all these formal specifications; it has already learned them. Now, frontier models can write TLA+ really well, just like writing in C or Python, very skillfully, right? Moreover, AI actually has a deep understanding of the system because it has already learned all the source code and the like. If you give it the code again, it can immediately learn from it, so all this time is automatically handled by AI, meaning it naturally already knows it. The only thing not solved is the previously more intellectual tasks. The most difficult part when you are modeling this system is how exactly you go about modeling it. Because if you still stick to that idea of wanting to keep everything, in the end, you won't be able to build this model, because you want everything.

Tianyin Xu (Guest) (01:13:32)

Your model will become large enough that you won't be able to do formalized things. But if the model is too simple, you won't discover any problems either. Deciding what should be modeled and what shouldn't used to rely on the experience of skilled veterans. Or rely on a very talented PhD student, on his intuition, his intelligence. Now we are handing this over to AI. We say, AI, you're quite smart, so can you help us figure this out by yourself and help us figure out the method for this? Because a mature open-source project has gone through many years, it actually knows which parts are tricky and which parts are unlikely to have problems, right? Then this is actually us. When human intuition comes into play, it's easy to make mistakes, so I want to model it more precisely. On the other hand, I think there’s no big problem in another area, so I won’t model it.

Tianyin Xu (Guest) (01:14:20)

So originally we were writing this model, and now we just tell AI that based on your experience, this kind of code is very difficult, very error-prone, and very easy to go wrong, so you should model it more carefully. That place may have never had any problems in history, so you don't need to model it. You just abstract it out, and AI actually does this very well. So after running it once, AI can actually write a very good model, and this model can conform to your interpretation. Once you get this whole system mature, you can leave the subsequent work to it. Traditionally, formal methods meant model checking or constructing proofs. Of course, the main challenge is that AI isn't human. It may engage in reward hacking. It can produce a model that looks correct, but isn't what you asked for.

Tianyin Xu (Guest) (01:15:03)

So you need ways to guide or even constrain AI, preventing reward hacking and helping it understand the system step by step, until it can complete the task on open-source distributed systems. It takes only five hours.

Uncle Moon (Host) (01:15:22)

That's a lot to take in. First, has productivity in this field really improved that dramatically? From five years to five hours?

Tianyin Xu (Guest) (01:15:29)

Yes. That's why AI for formal methods is now such a hot field. Formal methods themselves have existed for decades. I've always believed fundamental research matters. We can move this fast today because earlier researchers laid the groundwork. The road was already there. The reason it wasn't widely adopted in industry was the cost. It required highly specialized expertise, which made it expensive. It didn't fit Silicon Valley's “move fast and break things” culture. But at NASA, this approach makes perfect sense. They verify their systems, because sending a spacecraft to Mars only to have it fail would be extremely costly. They even avoid multithreaded code because concurrency is so difficult. They use single-threaded code and verify it formally before deploying it on missions like Mars Pathfinder. That approach doesn't work in Silicon Valley, where competition rewards speed. But once AI automates formal verification, the economics change completely. That's why so many AI-for-science startups are emerging.

The Cost of Proofs and the Limits of Specifications

Uncle Moon (Host) (01:16:28)

For a general audience, what does this mean for companies of all sizes?

Tianyin Xu (Guest) (01:16:34)

It is something that can help you solve some things that are very difficult to do through human reasoning alone. Now you can use some mathematical tools to help you verify or check certain behaviors in your system.

Uncle Moon (Host) (01:16:50)

Are these things normally impossible to find through repeated debugging and trial and error?

Tianyin Xu (Guest) (01:16:56)

Debugging is a reactive approach, meaning you only debug when a problem occurs, right? But our dream is to ensure that the failure never occurs. If we can formally verify the entire system, then that class of failure cannot happen. Verification means we have ruled it out. We have mathematically proven that the system will not suffer the failure we're targeting.

Uncle Moon (Host) (01:17:20)

Before, I understood why many companies found the return on investment too low. Formal verification took too much time.

Tianyin Xu (Guest) (01:17:27)

Right.

Uncle Moon (Host) (01:17:27)

It could be cheaper to learn through failure.

Tianyin Xu (Guest) (01:17:28)

Exactly.

Uncle Moon (Host) (01:17:29)

Deploy the system, and then

Tianyin Xu (Guest) (01:17:32)

buy insurance, if you have to.

Uncle Moon (Host) (01:17:34)

Right.

Tianyin Xu (Guest) (01:17:35)

If a failure happens only once every ten years, buy a large insurance policy. If it does happen, let the insurer cover the loss.

Uncle Moon (Host) (01:17:39)

Right.

Tianyin Xu (Guest) (01:17:39)

But now, instead of buying insurance, a formal proof can guarantee the failure won't occur for ten years— or even a hundred. make this mistake. So you don't need to buy insurance and you can happily run there.

Uncle Moon (Host) (01:17:48)

I had a colleague before who used to help some companies do this, for example, helping large cloud computing providers.

Tianyin Xu (Guest) (01:17:55)

Right.

Uncle Moon (Host) (01:17:55)

Every time it goes down, it's very expensive, right? Like that kind of EC2 going down would make half of the U.S. unable to access the internet. That means the whole service is down, so they have to maintain a team specifically to handle this.

Tianyin Xu (Guest) (01:18:08)

Before AI era, the most enthusiastic about this—and still the most enthusiastic now—is Microsoft because they have Azure, and reliability is very important. And then there's AWS.

Uncle Moon (Host) (01:18:15)

Basically, it's just these two companies. It is nurturing the team.

Tianyin Xu (Guest) (01:18:18)

Yes. But after having AI, I feel that this barrier has become very low, and then many people can do this.

Uncle Moon (Host) (01:18:23)

So what do you think will happen to the cost? How many tokens it would consume is to say that the issue of generating these things and the value produced in the end

Tianyin Xu (Guest) (01:18:33)

is more technical, because it depends on what you want to achieve. I think that if you want to do a complete proof, at least for now, it is still very expensive. Previously, we didn't work on that TLAPS thing, right? He had written this for five years before, and now he wants AI to write this proof. Then he ran a ZooKeeper system spec. A system spec is a specification that describes the system; it is not a specification that describes the protocol, so it is relatively low-level. Then he ran it at that time, and it cost $2,000 to run for 5 days. Well, I think this is still... A relatively expensive expensive thing, but when we chose to do this project at that time, we did not choose to do proof; we chose to do model checking. Model checking is relatively cheap and you can get feedback immediately.

Tianyin Xu (Guest) (01:19:13)

I thought we wanted to make a tool at that time, so I felt that such a tool would be more practical. It's somewhat more practical than doing a proof, or even more, more, more practical, at least at the present moment. But we also want to do some proof to make some investment for the future.

Uncle Moon (Host) (01:19:30)

To further understand it, many companies can start adding this during the process of developing their own systems;

Tianyin Xu (Guest) (01:19:37)

it's completely possible

Uncle Moon (Host) (01:19:39)

to use this system while developing.

Tianyin Xu (Guest) (01:19:42)

Yes.

Uncle Moon (Host) (01:19:42)

Go and continuously verify.

Tianyin Xu (Guest) (01:19:43)

Yes.

Uncle Moon (Host) (01:19:44)

In that case, he won't have any problems with every step.

Tianyin Xu (Guest) (01:19:45)

Yes.

Uncle Moon (Host) (01:19:46)

It's just because we all work on systems and know that premature optimization is the root of all evil.

Tianyin Xu (Guest) (01:19:50)

Right.

Uncle Moon (Host) (01:19:50)

Yes, but during the process of building the system, you start applying it right from the simplest MVP process, and then each time you add something, you put it in, and with each addition, it seems like a very beautiful scene.

Tianyin Xu (Guest) (01:20:02)

Yes.

Uncle Moon (Host) (01:20:03)

Especially now, AI agents, whether under someone's supervision or not, even if someone is watching them, you no longer look at their code. Many people do this, right? I directly threw out a requirement and immediately a software came out.

Tianyin Xu (Guest) (01:20:12)

Right.

Uncle Moon (Host) (01:20:13)

If, under this circumstance, we can provide a service that helps these various people check the messy AI systems they have written before going online, it seems that many problems could be avoided.

Tianyin Xu (Guest) (01:20:27)

Yes. I think the concept that everyone is talking about now is called SDD, which stands for specification-driven development.

Uncle Moon (Host) (01:20:32)

Right.

Tianyin Xu (Guest) (01:20:33)

in other words, you first write a formal specification, and then each time you let AI write something, you are able to prove it or verify that the code you wrote satisfies your specification. In this way, no matter what AI does, the final product you produce can be verified. So if this can be done, I think it can reduce a lot of AI-related problems, I think.

Uncle Moon (Host) (01:20:51)

Whether from an academic or entrepreneurial perspective, this is a very good idea.

Tianyin Xu (Guest) (01:20:55)

I think now there are many startups working on AI for formal methods; it just depends on what your anchor point is,

Uncle Moon (Host) (01:21:03)

applying various formal methods to systems built for the current AI agents.

Tianyin Xu (Guest) (01:21:10)

Right.

Uncle Moon (Host) (01:21:11)

Go verify it, go prove it. I find that it really requires constant communication with everyone. For someone like me who has never done this before, I was completely blank on this. Just now, you probably already explained it to everyone in the simplest language. Although I still have a lot I don't understand, I can sense that this is now feasible.

Tianyin Xu (Guest) (01:21:23)

But what I want to point out is that this is how I feel now. How should I say this? in other words, for one of the very difficult questions, you can see that there are now two voices, right? For example, right now there are two opinions in the field of reliability. The first opinion is that in AI era, the biggest problem with using AI to build systems is reliability and security.

Uncle Moon (Host) (01:21:43)

Yes.

Tianyin Xu (Guest) (01:21:43)

You can't guarantee it, and you are particularly anxious, you are particularly anxious,

Uncle Moon (Host) (01:21:48)

this matter is too alarming.

Tianyin Xu (Guest) (01:21:49)

But there is also a voice saying that now systems can be formally verified, and this process has already started, taking 5 years to be reduced to 5 hours or 5 minutes. So why is there such a huge gap here? Here, if the first thing is true, the second thing should not be true.

Uncle Moon (Host) (01:22:02)

Why is that?

Tianyin Xu (Guest) (01:22:03)

The reason lies in the fact that even if you can formally verify it, it only tells you that your system code is consistent with the specification you need; that is actually all it does. But I still think it is a very difficult problem, which is what kind of system you need and what your requirements are. What kind of specification you want, I think this matter is very difficult to solve completely relying on AI; it is hard to solve, and it requires experienced people to handle this matter.

Uncle Moon (Host) (01:22:30)

The higher-ups need someone to guide the completion of this specification.

Tianyin Xu (Guest) (01:22:32)

Yes, this matter is quite difficult because sometimes at the beginning, you want this system, but you yourself can't figure out what kind of system you actually want, right?

Uncle Moon (Host) (01:22:40)

Right.

Tianyin Xu (Guest) (01:22:40)

You might, at the beginning when you set up this system, say that mine is within this formal specification, there is something called a fault model, which means you envision how your faults will look, and then you can build a set of fault-tolerant systems. But maybe tomorrow or the day after, you find that there are some issues in your infrastructure. What about the other faults described by your fault model? So defining this specification is actually not that easy, I think

Uncle Moon (Host) (01:23:01)

it's too difficult, too difficult.

Tianyin Xu (Guest) (01:23:04)

Right.

Recovering from failures and handling agent mistakes

Uncle Moon (Host) (01:23:04)

So the first half of your entire career was spent designing these reliability, that operating system, making it harder for people to make mistakes, better helping people solve problems with these systems. Now you can see that you are gradually starting to design some things that connect with agents, that is, allowing agents or various AI tools to help people do this maintenance and build better systems. Some system stuff of agent-native,

Tianyin Xu (Guest) (01:23:29)

hmm,

Uncle Moon (Host) (01:23:31)

so what's your ultimate, for example, ideal? in other words, in the end, you might not be able to reach the state you hope the computer operating system will become.

Tianyin Xu (Guest) (01:23:43)

I think this question is quite deep, and I still believe it should start from the perspective of reliability. A reliable system is one where, in the future, at least 80% of its users are agents, and agents are not perfect—they can make mistakes, right? Then it might hallucinate, and then it might be off-base, so how can your system tolerate these things and also provide some feedback? Because the agent is also a very smart thing; as long as you give it feedback, it will do better in the second round. And I think there are There are many systems problems that still need to be solved. Without solving them, this vision won't become reality.

Uncle Moon (Host) (01:24:20)

I feel the same way. The future of these systems looks incredibly promising. But when you examine the details, you see how many hard problems remain— how many PhD students and systems engineers it will take to solve them one by one. It feels a little like the Manhattan Project.

Tianyin Xu (Guest) (01:24:36)

I think many of the core principles already exist. Then I don't know if you know have you heard of a project called ROC—

Uncle Moon (Host) (01:24:45)

No, I haven't.

Tianyin Xu (Guest) (01:24:46)

Around the year 2000, David Patterson, then a professor at Berkeley— a Turing Award winner and one of the pioneers of RISC-V— led a project called ROC: Recovery-Oriented Computing. Its central premise was this: computer systems inevitably fail. In other words, even if you spend all your resources on testing and formal verification, the system will still fail. So we should invest substantially in recovery, rather than assume a system will always be reliable. Instead, ask: what can we do when the system does fail? Can we build a system that recovers quickly after a crash? This is their big project. It is a collaboration project between Stanford and Berkeley. At that time, they received a lot of money from the NSF, and they had They had many ideas that could not be realized at the time. The first idea was to look at what makes a good system fail. Their approach was human-centered.

Tianyin Xu (Guest) (01:25:47)

They studied real production systems, and asked a fascinating question: when a system fails, is the hardware at fault? Is it the network? Or is it the people operating the system? When we think about fault-tolerant systems, we usually assume the hardware failed, a machine crashed, or an Intel processor suffered a bit flip. Something along those lines. But their empirical study found that the largest share of failures came from human error— operator error. People have to operate these systems, and the systems are complex. So people make mistakes, and those mistakes are often the leading cause of system outages. Jim Gray explored this issue. He later won the Turing Award and wrote a seminal paper on the subject. At the time, he was with a Silicon Valley company called Tandem Computers. Its product line was the NonStop system. The company aimed to build a system so robust that it would never stop— hence the name NonStop. Gray studied it in depth.

Tianyin Xu (Guest) (01:26:42)

The company had invested enormous effort in reliability, yet the system still failed. So he wrote a paper based on an empirical study. He pointed out that the company's core selling point was reliability. So in the end, when we sold it to the user and it crashed, what was the actual reason? Then he found the same problem, which was that the main reason was human operational error. It wasn't the hardware; the hardware wouldn't crash. The system software is already capable of fault tolerance, right? Then David Patterson, anyway, just pushed this matter forward a step, and he said that we cannot assume that users won't make mistakes; we should assume what to do when our system crashes. Then he had two very cute ideas. One idea is called system-wide undo, which means if a person makes a failed operation, how can I undo this? I'll redo it, right? Because people make mistakes, right?

Tianyin Xu (Guest) (01:27:24)

So if someone makes a mistake, I would do it again, again, just like when I first applied for a PhD and failed, I would reapply. But he thinks this system must be able to allow you to do this. If this system doesn’t let you do it, after failing once, how can you undo it? Because actually this matter is very difficult since there is no system to support you. For example, if you send an email, once you have sent it, how can you undo it, right? So it must have a very systematic mechanism to ensure that you can do this undo, and at that time they had a paper, and this matter was very difficult to generalize, so they made a special case. They built an email server, and they felt that an email server was a very difficult thing because sending emails or what?

Tianyin Xu (Guest) (01:27:56)

These things are very difficult to roll back, so they said we will focus on a relatively difficult problem and create an email server that can be undone. Then they had three, maybe five, principles. This was a paper they published at USENIX ATC at the time, presenting one of their cute ideas. Another cute idea is called microreboot. Then what he meant at that time was, when we have a problem in the system, what is the most routine action we take? It's restarting. In fact, we rarely, for example, if we have a blue screen, very rarely do we say, after I get a blue screen, I need to find out where this bug is? If we just restart directly and it can work again and again, then we can continue working, right? So 70% of recovery mechanisms are reboots, but rebooting is very expensive in industrial systems because you have caches and things like TLBs, right?

Tianyin Xu (Guest) (01:28:35)

You just rebooted, and all your states were lost, so this is very expensive, but it is a very effective method. So what should be done? So can I build this system into one where you don't need a global reboot but can do a microreboot, meaning you only reboot that one component instead of rebooting the entire memory or something like that? This was an idea they had at the time. They also created a case study back then, which was hard to generalize, so they just made a case study. But this has Today, that idea is reflected in microservices and serverless systems. You can microreboot a single microservice or restart one service. Services are restarted all the time. So the idea already exists in practice. In the age of AI agents, we can revisit those classic principles and reuse them directly. We can ask which ideas remain valuable and can help AI agents, and which we no longer need.

Tianyin Xu (Guest) (01:29:24)

Then we won't have to optimize for every old constraint, and can design a much better system for AI agents.

Uncle Moon (Host) (01:29:31)

Wow. I feel I really need to learn more about the history of computing, as Professor Juncheng suggested. That history contains a great deal of hard-won wisdom.

Tianyin Xu (Guest) (01:29:44)

Exactly.

Uncle Moon (Host) (01:29:45)

Applying that wisdom in this new era can lead to remarkable results. now, when you were talking about this, undoing this matter, designing a system — this system is used by an agent, and then every operation of the agent can be undone. What a beautiful set of systems.

Tianyin Xu (Guest) (01:29:59)

Yes. I think this is completely achievable. This is not something that is theoretically very difficult; it's just a matter of whether anyone is willing to do it. I think a big challenge of AI systems nowadays is that they are developing too fast, right? It’s rare for people to be willing to say that I should focus and spend a year or two building a system. I think this is a challenge. in other words, if your AI capabilities are constantly changing, or many of your paradigms are constantly changing, then indeed, the challenge is how you quickly build a system, be able to think it through clearly, and make it usable for future agent paradigms as well. I think this is a challenge—a real challenge.

Uncle Moon (Host) (01:30:34)

Moreover, previously everyone could feel that after Claude's upgrade, some aspects actually became less user-friendly. But this is a training process—you add data and then remove data. If this system is applied here, you can undo, right? You don't need to retrain, you can undo something, right? Each training session costs hundreds of millions of dollars, and this seems useful whether from the perspective of large AI training systems or small AI applications.

Tianyin Xu (Guest) (01:31:04)

Yes.

Uncle Moon (Host) (01:31:04)

But it's quite difficult; this requires dedicated practice.

Tianyin Xu (Guest) (01:31:06)

I think now is a very good opportunity because in the past, for example, twenty or thirty years ago, reliability and security were never a first-class principle, right? Because reliability and security don’t make money, only performance makes money, performance makes money. Basically, almost everyone building a system relies on the first thing, which is performance. If you improve performance, or features—features along with performance—these two things make money. As for reliability, that is, or security, you just worry about not losing money, so you can actually achieve it through many other methods, including, as I just said, buying insurance, buying some insurance. Or you could say that even though you write a lot of test cases, and the test cases are never comprehensive, at least you have a lot of confidence, so you have a company invest a lot in security and reliability, which many companies are unwilling to do.

Tianyin Xu (Guest) (01:31:53)

But I think the agent has turned this around because with you, if you let the agent take the lead in doing many things, including writing code. So reliability and security are the biggest issues, and they are not problems you can solve just by scaling test cases. So I think this is a great opportunity, and researchers can take advantage of it, especially reliability and security researchers, who can then obtain a lot of resources and do some more meaningful things.

Uncle Moon (Host) (01:32:17)

In fact, it's like this: nowadays, the people who hold resources in every company have also realized that having agents do things to raise the ceiling isn’t that difficult. Instead, the real challenge is how to continuously trust these agents, which makes everyone very anxious. Including when you are using some AI agents now, you will find that when it comes to high-value tasks related to money, filling out forms, or personal information, if you ask it to do these things, the official stance is that they do not recommend you do this because they do not want to take on this responsibility. But if someone can help solve the security issues generated by these systems, the value that this brings to everyone would be immeasurable.

Tianyin Xu (Guest) (01:32:52)

I think this is the current situation. For example, one of the hottest topics is trustworthy AI and so on.

Uncle Moon (Host) (01:32:56)

Yes, that's right. So

A Sabbatical at Berkeley

Uncle Moon (Host) (01:32:58)

recently, you also came to the Bay Area, to UC Berkeley for a visit. What kind of considerations led to this?

Tianyin Xu (Guest) (01:33:06)

I haven't given it much thought, because I'm not someone who plans very far ahead. I never make five-year or even three-year plans. I mostly think about what I want to do this year. My planning horizon is fairly short. I'm currently on sabbatical. American universities usually have a system under which, after six years of service, a professor may take a year-long sabbatical. The basic idea is that every institution has its own constraints and biases. If you stay in one place for too long, it becomes hard to think outside the box. You start taking everything for granted. A sabbatical encourages you to experience another environment, one very different from your own, and reflect on what you see. When you return to your university, you can adopt some of the good practices you've encountered and improve some of the bad ones. That's the purpose of the sabbatical system. I became eligible for one last year.

Tianyin Xu (Guest) (01:34:05)

I had originally planned to take it after completing six years at UIUC. But last year, the Trump administration made major cuts to NSF funding. And UIUC is a very large public university— one of the largest in the country. The steep cuts in federal funding put enormous pressure on the university. To be honest, there was a fair amount of anxiety in our department, and I felt that anxiety spread to many PhD students. They saw the funding cuts, alongside the impact of AI, and everyone was nervous. At the time, I felt it wouldn't be right for me to leave in the middle of that, so I decided not to take my sabbatical just yet. I postponed it for a year. Things felt more stable this year, so I decided to go ahead with the sabbatical. I didn't have a specific plan at first. Then one day I ran into my current host, Ion Stoica, and mentioned it to him.

Tianyin Xu (Guest) (01:34:59)

He said, “You're welcome to come to Berkeley.” I thought, “Sure, why not?” And that's how I ended up here. He was very thoughtful about it, though. He wanted to make sure I'd be happy here and that the visit would give me what I was looking for. So he invited me to a SkyLab retreat. It was much larger than I'd expected. There were people from industry and academia, as well as students, all presenting. I went, liked what I saw, and decided to come. There really wasn't much more planning than that.

Uncle Moon (Host) (01:35:26)

There was a lot packed into that answer. First, given the current climate, there is real anxiety at both the institutional and individual levels. It's hard not to be affected by upheaval on that scale.

Tianyin Xu (Guest) (01:35:40)

Yes, some things are simply beyond our control.

Uncle Moon (Host) (01:35:44)

Second, once your work reaches a certain level, it's easier to visit leading research groups.

Tianyin Xu (Guest) (01:35:49)

A sabbatical visitor doesn't place much burden on the host institution. Instead, you become a colleague they can exchange ideas with, so most universities are very welcoming. If any professors watching this are considering a sabbatical, don't worry too much about imposing. Reach out and ask. There are more opportunities than you might think. That's also one reason I chose Berkeley. Berkeley's research culture differs sharply from UIUC's. In some ways, they're opposites. It's not about one being right or wrong; they're simply very different. For a sabbatical, I wanted to go somewhere genuinely different. That made Berkeley an excellent choice. I could have visited a university more like UIUC. It might have been an outstanding school, but if the culture were too similar, what would be the point of going there?

Uncle Moon (Host) (01:36:31)

I've heard it said that a sign of real intellectual maturity is the ability to hold multiple viewpoints, methods, and worldviews in your mind.

Tianyin Xu (Guest) (01:36:42)

Exactly.

Uncle Moon (Host) (01:36:42)

You already have a strong methodology and the practical Midwestern research worldview. Now you want to see something different.

Tianyin Xu (Guest) (01:36:50)

Right.

Uncle Moon (Host) (01:36:51)

And given your personality, you didn't arrive with a fixed idea of what you had to accomplish. You simply came to explore.

Tianyin Xu (Guest) (01:36:56)

Yes. I think research is a beautiful thing. And beautiful things don't follow a single standard. No rule says everyone must follow Method A or Procedure A. What makes academia and research so appealing is that many schools of thought can flourish. People can use different methods and philosophies, or follow different principles, and still succeed. If the result is something beautiful that people value, then we should make room for many different styles. Berkeley's style is very different from UIUC's, and UIUC's style is also different from MIT's. That's a good thing. All of those approaches should be able to coexist. That's one reason sabbaticals exist: they encourage you to experience something different and exchange ideas across institutions.

Uncle Moon (Host) (01:37:45)

There's another memorable line: beautiful things have no single standard.

Tianyin Xu (Guest) (01:37:49)

Exactly.

Uncle Moon (Host) (01:37:50)

Let a hundred flowers bloom.

Tianyin Xu (Guest) (01:37:50)

Yes.

Uncle Moon (Host) (01:37:51)

I think that's part of human nature. Now that AI agents are so capable, if you want to live a happy life, you need to seek out things that cannot be standardized. People have been saying this a lot lately. What is AI best at? Standardized tasks— problems with clear questions and clear answers. What is AI worst at? Things that are difficult to define, things that resist standardization.

Tianyin Xu (Guest) (01:38:12)

I think you're absolutely right. Yes.

Academia, Industry, and Career Choices

Uncle Moon (Host) (01:38:16)

That wraps up our discussion of your research. Let's move on to some broader, more philosophical questions. During your PhD, you interned at several places—NetApp, Facebook, and Microsoft Research— and you've now spent many years in academia. What do you see as the biggest difference between industry and academia in computing?

Tianyin Xu (Guest) (01:38:45)

By industry, do you mean industry research? Otherwise, the comparison isn't really apples to apples. So what's the difference between industry and academic research?

Uncle Moon (Host) (01:38:53)

Either interpretation is fine. Answer it however you think makes sense.

Tianyin Xu (Guest) (01:38:55)

Here's how I see it. Industry and academia approach research differently. Both are forms of research, and share the same fundamental purpose, but they take very different forms. I think academia— especially now that we have AI— needs to focus even more on fundamental research. Academia has fewer resources now, partly because of government intervention. With fewer resources, you have to pursue truly fundamental questions. Academia's job is to push the frontier. You may push it today, and AI may learn from it tomorrow. But unless we keep pushing that frontier, AI will have nothing new to learn. That's what academia should do. That's what good academic research looks like. Industry research, by contrast, is applied research. It should be tested in practice. It has to be useful, and it should produce results in the near term, ideally with significant impact. So the two styles of research are very different. different, and the methods of carrying them out are also very different.

Tianyin Xu (Guest) (01:39:56)

This is why I think, for example, when a person is facing a choice, the first thing they deal with is the question of what kind of research they want to do. If someone says, 'I just want to do very practical research that can benefit many people.' Many people say tomorrow or next month, so I think industry research might be a very good goal. Some people say they just want to do a pie-in-the-sky research, right, just looking at the sky and imagining what the future world would be like. I think that kind of research is quite good at Princeton.

Uncle Moon (Host) (01:40:21)

Before that, Jun Cheng had a point. It means that when a PhD goes for an internship in the industry, they shouldn't do research. He said, Jun Cheng said that a PhD in the industry should be completely immersed in the industry's system every day, seeing what the system engineers around them are working on. What does he write? Then get the most frontline understanding of this system, and then go back to school to slowly refine it to see if you can make something better. What do you think of this viewpoint?

Tianyin Xu (Guest) (01:40:47)

I think this is a very good method. I think the reason why we should do internships is because I still strongly encourage students to intern. I think internships are also a very good thing because you change your environment and then see some different systems that you originally had no chance to be exposed to, right? And also, you work with different people, so I think if you go for an internship and still follow the school's way of doing things, that's obviously not a very good practice, right? So I think if you go to the industry or do an internship, you should do something different this summer, and then learn something. Of course, there are many ways to learn. I think, for example, in my first period, I applied my own research to the industry system, and I think this is a very good, very good method. You can also do that.

Tianyin Xu (Guest) (01:41:29)

It is about going into the industry to learn about their problems, and then the most important problem there. First, I have a student, Xudong. His first internship was at Walmart. I remember that in his first two weeks, he would just catch people and ask them, 'What is the most important problem in your company?' I want to address your most important questions, and then he received a dozen answers. Many people also couldn't frame it as a research question, but everyone would honestly tell him what they think is the most important or the hardest problem in their work. Then, at this point, you refine it yourself. I think this might be a bit similar to what Juncheng said,

Uncle Moon (Host) (01:41:59)

maybe too similar,

Tianyin Xu (Guest) (01:41:59)

but I think it's about understanding this, because now. Academia is no longer the academia of the 70s, right? Back in the 1970s, there might have only been five computers in the United States, all in schools, and the industry would immediately follow up on whatever the schools were doing. But now, you see, the industry is often completely ahead of academia. Even when I was doing my PhD, Google had already implemented systems like MapReduce and Bigtable and then published a paper. Then thousands of papers are optimizing MapReduce, and then publishing Bigtable; thousands of papers are optimizing Bigtable. So I think that many times industry is ahead of academia. Therefore, I think when you go into industry, you should do things that industry needs to do. Wasn't there an old saying about this before? When you go to Rome, you should do what the Romans do. I think this is right, I think.

Uncle Moon (Host) (01:42:42)

Totally agree. So if one of your students asks you when they graduate, 'I got two offers: one is to go to Google to work on this system, and the other is to go to a certain school to be a professor. How would you help me make a decision?'

Tianyin Xu (Guest) (01:42:59)

One is to go to Google and work on core systems.

Uncle Moon (Host) (01:43:02) †

Right.

Tianyin Xu (Guest) (01:43:02) †

Another option is to

Uncle Moon (Host) (01:43:03)

go to an elite school to be a professor.

Tianyin Xu (Guest) (01:43:06)

This choice is very personal. I think it has to do with what he and the other person are pursuing, including his financial situation, right? Or his need for certain things, and then his personality, I think all of this applies—he is not someone who can absolutely... Still, it's the same statement: there is no formula. He has to make a choice based on the different situations of each person, I think, including going into the industry to create a system. And when you create this system, what exactly are you doing, right? in other words, if you go to this factory to screw in screws, it is possible that the system is very pretty, but when you screw in the screws, you may not necessarily feel that it's pretty. So I think any experience, the first thing for me is that in this experience, what kind of things you can learn.

Tianyin Xu (Guest) (01:43:46)

I think this is the most important question, whether you go into academia or industry. If you go to an extremely beautiful place but don't learn anything, then I think it's still nothing, for me, right? What it means to interrogate yourself is asking, 'If I work in this position for five years, what kind of skills will I be able to learn, and what abilities will I be able to develop?' Once you answer this question, I think you will have a very clear next step. And many times, what you are even asking is, 'Who should I work with?' I think who you work with depends on your level, and on whether they are very enthusiastic. Working with people who are excited or very enthusiastic about technical things is different from working with people who might just want to get by. Even in a very big company, I feel like you can't really learn much from those people.

Tianyin Xu (Guest) (01:44:30)

So, whether you go to a startup, a big company, or academia, essentially it's about what you can learn. Of course, there are many other constraints as well, right? I think a big problem with academia is that everyone is very poor, so if you really need money and your family needs money, I think it might not be a particularly good choice.

Uncle Moon (Host) (01:44:49)

Anyway, we've basically covered everything, right? There are interests in certain professions, in working with certain people, and in various kinds of family situations.

Tianyin Xu (Guest) (01:44:57)

Right.

Uncle Moon (Host) (01:44:57)

Especially what you just said about working with very exciting people—I’ve heard countless statements from big shots about this, which means that the things this team is doing and the people in this team make me willing to join this company on the day I graduate even if they don’t pay me. I think everyone is just—in fact, finding a job is getting increasingly difficult, and everyone’s career development is a bit unclear. Climbing the ladder, going for a promotion, these things are all very vague, but a lot of fans in the background often ask questions about how to choose. Actually, I think there is only one standard, which is really just what you mentioned earlier: whether you can learn something and whether being with these people is exciting or not.

Tianyin Xu (Guest) (01:45:33)

Yes,

Uncle Moon (Host) (01:45:33)

especially in the early stages of your career, this is the most, most, most, most important.

Tianyin Xu (Guest) (01:45:38)

Right.

Uncle Moon (Host) (01:45:38)

The other things you just mentioned are very honest words. Some big companies or certain places may seem dazzling, but the small team isn't really up to par, and then you are really

Tianyin Xu (Guest) (01:45:50)

correct because I think that's the case. Academia is a very standardized place. The best schools in the U.S. and just an average school in the U.S. actually don't have much difference. As a faculty member, you just do those three things, right? One is teaching, one is doing research, and another is mentoring students, but in industry. Different teams in different companies vary greatly, so it's actually hard to summarize in one sentence whether you should take an industry job. You can do a very creative job in the industry, or you can do a very boring job. Therefore, you need to understand exactly what this job entails. A good way to do this is to talk with people in the team, ideally, and see what their thoughts are. Actually, once you talk to someone, you'll know whether you want to work with these people. Many times,

Uncle Moon (Host) (01:46:30)

this is the core reason, because one company has too many people, too many teams, and too many projects.

Tianyin Xu (Guest) (01:46:35)

Right.

Uncle Moon (Host) (01:46:35)

Can't you figure out what you want? Then you can't work with these people in a highly excited way every day, which I think is especially a waste of your life in the early stages of your career.

Tianyin Xu (Guest) (01:46:43)

Yes. My former PhD advisor once made an excellent point. He said that in computing, your first job is not your forever job. So don't agonize over your first job, or imagine it's a secure position you'll keep for life. Treat it as a learning process. If you learn— really learn— your next job will be better. If you don't, even a huge compensation package won't stop you from being laid off two years later.

Uncle Moon (Host) (01:47:09)

That's true. The computing industry changes incredibly fast. The pace over just the past five or six years has made that obvious. We don't even need to go back to the early internet, the dot-com bubble, or the mobile internet. Just look at AI era. In only two or three years, so much has changed. AI has gone from basic chatbots to writing code. Next, for all we know, robotics could take off. You might have happily built a chatbot when they first appeared, only to find yourself far behind this year. That's how much can change in two years.

Tianyin Xu (Guest) (01:47:36)

As for industry, this is just my personal view, and it may be biased, but I believe the most important thing is the ability to keep learning. The greatest danger in industry is taking a position and then stopping learning. With technology changing every day, you will inevitably be left behind. The advantage of academia is that it forces you to learn. As a faculty member, you have to advise students, apply for funding, take on all kinds of responsibilities, and teach. The environment compels you to keep learning. If you don't, you can't keep up. In industry, however, you may settle into a job without making a conscious effort to learn. You can remain comfortable for two years, but what happens after that? That's the problem.

Uncle Moon (Host) (01:48:17)

Right. Some say academia has better long-term prospects, while industry pays more early on. People use all kinds of criteria, but to me, those distinctions are simply too vague. After our conversation, one point stands out clearly: what matters is the person.

Tianyin Xu (Guest) (01:48:34)

Exactly.

Uncle Moon (Host) (01:48:35)

What matters is you. As long as you can keep learning, whether you're in academia, in industry, or anywhere else, you're likely to do well. position,

Tianyin Xu (Guest) (01:48:42)

yes, yes.

Uncle Moon (Host) (01:48:44)

I feel that, in the end, a person,

Tianyin Xu (Guest) (01:48:47)

for me, is essentially this. I think a high-quality person always has a way,

Uncle Moon (Host) (01:48:48)

always moves forward.

Tianyin Xu (Guest) (01:48:50)

I really believe that one of the reasons for this person is

Uncle Moon (Host) (01:48:55)

moving forward.

Tianyin Xu (Guest) (01:48:56)

I don't really believe in AI, but I think people should be believed.

Uncle Moon (Host) (01:49:00)

I've been doing this podcast for a while, and I also feel the same way. It's true that you have to always

Advice for PhD Students and Lifelong Learning

Uncle Moon (Host) (01:49:05)

believe in people. So the next question is, in the process you just described, everyone can also tell that you pay special attention to people, especially to the connections between people, and are particularly willing to guide a student hand in hand to understand their essence and lead them as a professor. So if we are facing this... With so many students in the audience, you definitely can't guide them hand by hand. If you were willing how they can develop into strong PhD researchers— how they can train themselves— what is the most practical advice you would give them?

Tianyin Xu (Guest) (01:49:40) †

Well, I think all advice is— all advice is bad advice. How should I put it? Three things matter most in research, especially during a PhD. To do well, you should find every way you can to get those three things right. First, find a problem you truly care about. That's the first thing. When you care deeply about a problem, the work doesn't feel exhausting. Research is hard, and only genuine interest can sustain you. When you wake up each morning and brush your teeth, look at yourself in the mirror. Are you excited to keep working on the problem? Or are you thinking, “Oh no, I have to go back to the office and work on this again”? So first, find a problem you love. That is genuinely difficult. It may take one, two, or even three years. But if you keep looking, you will eventually discover what kind of work you care about.

Tianyin Xu (Guest) (01:50:24)

Second, it is crucial to develop a sound approach to research. Research is not the same as engineering. Often, there is no shortcut. What matters is finding the right method. Speed itself isn't important. You can make up for speed with time. But with the right method, if your approach is sound, you will eventually get there. Third, keep sharpening your skill set. Once you have a problem to pursue, you can see which skills are useful and which are not. Your time and energy are limited. You cannot do everything. Once you know what you need to do, you can deliberately develop the skills required to do it well. Get those three things right, and I believe you can succeed. Systems research isn't pure theory. It doesn't require extraordinary genius. Systems research is not especially dependent on raw talent. If you get those three things right, you can do good work. I believe anyone can do systems research.

Uncle Moon (Host) (01:51:16)

You still call that bad advice? That sounds like perfect advice to me. Very much so.

Tianyin Xu (Guest) (01:51:20)

I just think everyone already knows these things, but putting them into practice is difficult.

Uncle Moon (Host) (01:51:24)

Perhaps it sounds a little abstract?

Tianyin Xu (Guest) (01:51:27)

Yes, very abstract. It's a cliché, really. The same is true of many things. For example, earlier we discussed starting a company, or writing a paper. How do you write a good paper? How do you build a good company? Hundreds of books have been written on both. But reading those books doesn't mean you can write a good paper or build a successful company. You still have to learn through practice. That's why I think you also need good mentors.

Uncle Moon (Host) (01:51:54)

to say about this is that I saw a saying before, which is that people may hear a lot of correct advice, but why do some people grow quickly while others grow slowly. You want to see the person who grows quickly, right? Finding a mentor who can provide timely feedback to him is crucial. In the process of entrepreneurship, whether you look at those successful entrepreneurs or those PhD students who do very well, it all comes down to being able to get the most timely feedback from a very experienced mentor. If people can't get this kind of thing, then they need to make an effort to find it.

Tianyin Xu (Guest) (01:52:28)

That's how I feel. And this mentor, I think he... It doesn't necessarily have to be a very big-name person, right?

Uncle Moon (Host) (01:52:33)

Right.

Tianyin Xu (Guest) (01:52:33)

It's really someone very famous; he is a person who is willing to invest in you, someone who is willing to spend time on you. I think the more someone is like this, the more important it is.

Uncle Moon (Host) (01:52:44)

How to get others to bet on you and then invest in you is another big topic. You can't really put it into words, you know? It's hard for everyone to see.

Tianyin Xu (Guest) (01:52:51) †

Yes. This is a topic that requires lifelong learning,

Uncle Moon (Host) (01:52:52) †

and it goes back to what we just talked about about lifelong learning and continuous growth. Without this energy, if you yourself don't have this energy.

Tianyin Xu (Guest) (01:53:01)

Right. You want to become a person worth investing in, right?

Uncle Moon (Host) (01:53:04)

Right.

Tianyin Xu (Guest) (01:53:05)

This is your point,

Uncle Moon (Host) (01:53:05)

it’s worth it to you, it’s worth other people spending time. Worth other people spending time. He doesn’t invest just by saying please. The kind of thing where you can guide me, it won’t happen. It will never happen.

Tianyin Xu (Guest) (01:53:14)

Yes.

Uncle Moon (Host) (01:53:15)

Only by constantly striving and improving yourself, this is good, good, good, very fantasy, all nonsense, whether the chicken or the egg came first, this matter

Tianyin Xu (Guest) (01:53:21)

is indeed correct,

Uncle Moon (Host) (01:53:23)

but maybe when I reach the 100th episode later I will tell everyone how to do this, but for now I have only done a few episodes so I don't dare to say. And there is another question. If students are willing to start a business, you would support them, and previously your mentor was a very successful entrepreneur, and you have some very successful cases around you.

Tianyin Xu (Guest) (01:53:44)

Right.

Uncle Moon (Host) (01:53:44)

So what if this student tells you from the beginning, 'OK, professor, after I finish this PhD, I want to become an entrepreneur'? How would you guide him?

Tianyin Xu (Guest) (01:53:51)

Then I would definitely tell this student that I have never started a company, so I probably can't help him much in terms of entrepreneurship, right? So I currently have a student who really wants to start a business, and my suggestion to him was that he might need some other connections or some other advice, so he also got some, well, some help, I think, but how should I put it? I just feel that there are two types of entrepreneurship. One is where you start a business based on your own research. For example, you develop a particularly interesting technique and then use this technique to start a business. In the process of developing this technique, I think it can still be somewhat helpful. Of course, if you say that your startup is completely based on taking an existing technique to create a company. Then I might not be a very good advisor, I think that's the

Uncle Moon (Host) (01:54:32)

Building a company around an existing technology or a new technical innovation is certainly one path. Many companies that emerged from Google started that way. But I once heard your advisor caution against it. He said you should avoid taking a hammer and looking for a nail. It's usually better to start with a real business need. That's another topic, though. Everyone has a different view, and there are certainly both successes and failures.

Tianyin Xu (Guest) (01:54:57)

Right.

Uncle Moon (Host) (01:54:57)

Now for the third question. Suppose a student wants to enter industry and knows from the outset that they want to run large-scale systems and build practical systems. How would you advise that student?

Tianyin Xu (Guest) (01:55:09)

Well, I think academia and industry require different skill sets. That said, I don't think academia and industry are fundamentally different. Doing creative work is rewarding in either one. What matters, ultimately, is whether the place you choose gives you an environment that helps you do the work you want to do. That place might be academia or it might be industry. But the two paths do require different skill sets. Some things matter little in academia but are crucial in industry. If a student knows from the beginning that they want to enter industry, the most important thing is to identify the skills that industry requires. Then, during the PhD or other graduate study, they should deliberately develop those skills and become a more competitive candidate. When it comes to publications, industry doesn't care much about how many papers you've published.

Uncle Moon (Host) (01:55:59) †

One paper may be enough to graduate. Spend the rest of your time coding, learning system design, and exploring

Tianyin Xu (Guest) (01:56:05) †

the latest frontier ideas.

Uncle Moon (Host) (01:56:07)

Right. AI, for example.

Tianyin Xu (Guest) (01:56:09)

If you want an academic career, you'll need to build a publication record

Uncle Moon (Host) (01:56:12)

and focus on that.

Tianyin Xu (Guest) (01:56:14)

Build it up.

Uncle Moon (Host) (01:56:14)

Exactly. We've talked with Tianyin for more than two hours today, and it's clear that he is a truly exceptional systems researcher. There's a saying in Silicon Valley: when someone starts a company, there needs to be founder–problem fit. The right person is suited to the right problem. That comes from something deep within them. After these past two hours, I can clearly see that in you. I've met many systems researchers, and I work in systems research myself, though I'm only average. But you really are a natural systems researcher.

Tianyin Xu (Guest) (01:56:47)

I don't believe anyone is born that way. You just need to enjoy the work. Research is a wonderful thing. I think everyone should try it, especially now that the barriers are lower. Research used to feel like something confined to an ivory tower, something only a small number of people could do. But with AI and modern technology, research should become a daily practice. At its heart, it's a process of reflection. So I think everyone should give it a try.

Uncle Moon (Host) (01:57:10)

All right. Thank you, Tianyin. I hope that after hearing this, everyone will do a little research every day.

Tianyin Xu (Guest) (01:57:16)

Exactly.

Uncle Moon (Host) (01:57:17)

It can make you more effective at work.

Tianyin Xu (Guest) (01:57:18)

Yes.

Uncle Moon (Host) (01:57:19)

And it can strengthen all kinds of skills.

Tianyin Xu (Guest) (01:57:20)

Absolutely. You can research something as everyday as how to build your abs or how to get in shape. That's research too, isn't it?

Uncle Moon (Host) (01:57:27)

It's an iterative process of trial and error.

Tianyin Xu (Guest) (01:57:28)

Exactly.

Uncle Moon (Host) (01:57:29) †

Thanks for the suggestion.

Tianyin Xu (Guest) (01:57:30) †

All right.

Uncle Moon (Host) (01:57:30)

Thank you.

† At least one speaker change or brief response in this passage needs audio verification.