中文精读
AI Infra 的第一道选择：先决定优化哪笔账
嘉宾：朱亦博，阶跃星辰联合创始人、前字节 AI Infra 负责人，曾在微软、Google 任职（当期身份）｜主持：曲凯
发布：2025-08-02｜时长：58分08秒｜整理：2026-10-18
官方单集
同一个模型，为什么换一套系统就会更贵、更慢，甚至限制最终能力？朱亦博的解释从一个容易被忽略的问题开始：团队究竟在优化什么？训练时把卡用满、让模型分数最高、让用户少等，可能是三笔不同的账。AI Infra，即支撑模型可靠、高效运行的基础设施，要把硬件、模型和任务放在一起权衡。
这期是2025年8月的从业者视角。竞争评价、技术路线与成本预测保留当时时点；尤其不能把“大家依赖基础设施”直接写成“每家应用公司都该自建”。
一、从后台到核心，取决于计算是否一开始就是主要约束
回听：01:07–07:11、08:40–11:20。 朱亦博把底层问题归纳为计算、通信和存储。GPU 之外，还要有网络、交换、存储、运维、资源调度及训练与推理框架。他借云计算的服务层次说明这些组件的位置，是解释框架，不是唯一行业分类法。
传统互联网小公司业务起步时，通常先用通用服务，规模大后才需要专业团队把性能推到更高水平。大模型公司从一开始就面对大量数据与计算需求，Infra 因而直接影响能做什么模型。他用搜索引擎早期处理海量网页的需求作类比：后台能力可能成为产品竞争的起点。
是否自建却仍应算规模账。他假设每月算力支出1亿元，改善10%就对应约1000万元价值。这是教学算例，并非阶跃成本披露。若应用公司使用量小，同样的优化比例未必覆盖一支团队的投入，云服务与模型服务可以提供基线。自建的门槛来自真实成本、控制要求与预期收益，不能从行业热度推出来。
二、用户等待、训练利用率与推理成本，不是同一个目标
回听：08:40–09:43、16:15–22:06。 用户问聊天产品一个问题，首字延迟决定多久看到回应；之后的输出速度与稳定性影响交互是否顺畅。系统可以有很高的总吞吐量，同时让单个用户等很久。因此产品体验不能只引用一个“每秒多少 token”的数字。
训练端常看 MFU，即实际计算相对硬件理论计算能力的利用程度。但朱亦博强调，漂亮的局部指标未必等于好的全局选择。他对 DeepSeek 的解释是：其面向推理成本的模型设计，在强化学习需要反复生成、评分时获得优势。这是嘉宾对行业路径的解释，未独立验证公司内部决策，也不能将成功归结为唯一原因。
他进一步区分输入处理 prefill 和逐步输出 decoding，并把 decoding 速度与成本视为当时的首要目标：它既影响线上成本，也影响训练获得反馈的速度。这个优先级有时间与任务条件。长文输入、多模态理解等任务可能有不同瓶颈，不能据此忽略输入端。对应用的编辑推论是：先明确用户最难忍受的等待在哪里，再选择系统指标，且把整项任务成功率一起看。
三、结构先定完再“交给系统优化”，可能已经错过最重要的选择
回听：22:06–28:41、47:54–51:51。 朱亦博认为，模型由算法、系统、数据共同决定。一个结构可能只带来小幅效果提升，却损失大量运行性能；也可能反过来。若算法团队先把结构定好，Infra 只负责让它跑得快，有些成本问题已经被锁进了设计。
他讲到一个未点名模型：参数或激活量看起来不大、榜单效果不错，但实际硬件运行效率可能比更大模型还差。原因在于“参数量—效果”不是“真实运行成本—效果”。通信、内存访问与实现方式会改变最后账单。这里保留其案例判断，不推断是哪家公司，也不把它当公开复测。
MOE（混合专家）则是成本约束反向推动架构的例子：每次只激活部分专家，有机会保留较大模型容量而降低使用开销。朱回忆团队早年从业务嫌模型太贵出发研究 MOE，强调系统人员可以参与设计。它并不意味着所有早用 MOE 的组织都更好，或某种结构天然更便宜；仍须在真实硬件和目标负载下比较。
小团队的潜在优势是共同讨论权衡，而非人少本身。让三方对同一结果负责，需要决策权与目标配合；只要求工程师更主动，不能自动修复组织阻力。
四、第三方的长期价值，要能影响上下游
回听：11:20–16:15、28:41–39:28。 如果模型和硬件人人都能拿到，中间层只做短期加速，就容易被同行追上，陷入价格竞争。朱亦博建议从业者更深入模型或硬件，参与联合设计。他用独占内容吸引用户的平台类比模型服务：通用聚合之外，还需要让用户留下的特有价值。
但联合设计也意味着承担路线风险。重点是能否参与改变结构，而非单纯押注某家合作伙伴。热门开源模型促进适配，同时也可能把大量工程投入绑定在现有架构；新结构出现后，一部分积累就需要重做。能立即解决的工程问题与还看不清路线的范式问题，应分开预期。
阶跃当期希望为国产芯片适配模型，构建模型与芯片互相推广的生态。免费商用是节目中的公司声明，具体使用必须另查对应版本和许可证，播客不能代替授权文本。朱也没有断言所有公司最终都必须自研芯片；他的判断是，如果某家把模型和硬件整体优化得很好，就会给对手带来压力。
五、视觉推理的价值，在于少丢掉空间与操作关系
回听：39:28–43:48。 识别照片里是什么，与判断如何从柜子中取出被挡住的物品，要求不同。后者需要看懂遮挡、位置与操作顺序：先移开什么，再拿什么。把画面先压成一段文字，可能丢失这些关系，所以朱强调直接利用视觉信息推理的意义。
他用机器人“大脑”和“小脑”作比喻：任务拆解与动作平衡是不同层次。图片定位、拍照解题也属于他列举的视觉推理场景，但能给出步骤不等于机器人已经能够安全执行；定位判断还涉及误判与隐私。这里的安全与隐私限制为编辑补充，避免把能力演示扩大成现场可靠性证明。
这也解释了为什么模型与系统应共同评估：一个更直接的输入路径可能改善信息保留，实际延迟、成本和任务正确率却仍需测量。公司对多模态领先程度与生成成本下降的表述，均是当期自评和预测。
六、能利用更多计算的方法，也要证明当前交付价值
回听：52:58–57:58。 朱引用《The Bitter Lesson》，强调长期能充分利用计算的方法可能超过短期技巧。对 Infra 而言，问题是系统与结构能否随规模持续受益，而不是只在某个固定配置上挤出一次优势。这不等于工程细节可以省略，硬件演进的访谈概括也不应当作严谨定律。
谈 Agent 时，他同时看到共生与挤压：应用先用工程把模型“踮脚”才能完成的任务做成体验，模型再可能吸收为原生能力；但模型公司也未必更早发现那些场景。他明确说看不清最终生态。不能把这段摘成“所有应用都将消失”，更实用的问题是：今天的工程优势是否产生了用户关系、流程可靠性或其他可继续累积的价值？后一问是编辑推论。
练习：给同一个任务画两条曲线
选一个高频任务，比较两套模型与系统方案。第一条曲线记录模型分数或质量；第二条在相同任务、并发与硬件条件下记录每成功任务成本，同时标注首字延迟、总耗时和失败恢复。把输入处理与输出阶段拆开，避免一个平均速度掩盖瓶颈。
然后让产品、算法与工程共同写一段取舍说明：哪项用户收益值得承担哪项成本？若只改善利用率却没有改善目标结果，应说明它的价值来自哪里。最后用真实月用量估算可节省金额，再决定采购、自建或继续观察。此练习为编辑整理，不是阶跃的内部验收流程。
来源与回听
据完整转写、官方简介与章节整理，未复听音频或复测模型。转写大量回答误标主持人，本文按问答语义提炼并以官方嘉宾信息校正；不确定的型号、参数不补数。成本算例、公司评述和商用声明均保留出处边界。
- 01:07：Infra 的层次与共同目标。
- 10:09：自建的规模账。
- 17:45：优化目标如何改变结果。
- 22:06：算法、系统、数据的共同权衡。
- 36:35：模型与芯片联合适配。
- 47:54：参数量与真实运行成本的区别。
- 55:43：长期计算能力与工程价值。
相关主题：AI 基础设施、模型评测、组织协同。延伸可听吴翼谈强化学习，以及季宇谈英伟达，理解训练吞吐与软硬件生态之间的联系。
English edition
AI Infrastructure Starts with Choosing Which Bill to Optimize
Guest: Zhu Yibo, cofounder of 阶跃星辰, former head of AI infrastructure at ByteDance, previously at Microsoft and Google, as introduced at release | Host: Qu Kai
Published: 2025-08-02 | Duration: 58:08 | Prepared: 2026-10-18
Official episode
Why can the same model become more expensive or slower under another system—and why can infrastructure limit its eventual capability? Zhu begins with a neglected question: what is the team optimizing? Maximizing hardware utilization, maximizing model scores, and minimizing the user’s wait can be three different calculations. AI infrastructure, the systems supporting reliable and efficient model operation, must weigh hardware, models, and tasks together.
This is a practitioner’s view from August 2025. Competitive judgments, technical directions, and cost forecasts retain that date. Depending on infrastructure does not imply that every application company should build its own.
1. Infrastructure becomes central when computation is an initial constraint
Listen: 01:07–07:11; 08:40–11:20. Zhu reduces the underlying problems to computation, communication, and storage. GPUs need networks, switches, storage, operations, resource scheduling, and training and inference frameworks. His comparison with cloud service layers is an explanatory framework, not the only possible industry taxonomy.
A small internet business can often begin with general services and hire specialists when its scale demands more. A foundation model company faces large data and computation requirements from the outset, making infrastructure part of what it can build. Zhu compares this with early search engines processing huge collections of webpages: background capabilities can become the starting point of product competition.
Building internally still requires a scale calculation. He imagines a monthly compute bill of RMB100 million, where a 10% improvement represents roughly RMB10 million in value. This is a hypothetical example, not disclosure of his company’s spending. A small application’s usage may be too low for the same percentage gain to cover a dedicated team. Cloud and model services provide a baseline. The threshold depends on actual costs, control needs, and expected returns, not sector excitement.
2. Waiting time, training utilization, and inference cost are different objectives
Listen: 08:40–09:43; 16:15–22:06. In a chat product, time to first token determines when the user sees a response. Subsequent speed and consistency determine whether the interaction feels smooth. A system can have high aggregate throughput while individual users wait a long time. A single tokens-per-second figure cannot establish the user experience.
Training teams often monitor MFU, a measure of actual model computation relative to theoretical hardware capability. Zhu argues that an attractive local metric does not guarantee a good overall choice. His interpretation of DeepSeek is that architecture oriented toward inference cost became advantageous when reinforcement learning required repeated generation and scoring. This is his explanation of an industry trajectory, not independent verification of internal decisions or a complete account of the company’s success.
He separates prefill, which processes input, from decoding, which produces output step by step. Decoding speed and cost are his priority at the time because they affect both online serving and the rate at which training receives feedback. That priority depends on the date and workload. Long documents or multimodal understanding may expose different bottlenecks. An editorial application is to identify the waiting that users most dislike before choosing the metric—and to evaluate whole-task success alongside it.
3. Handing a finished architecture to systems engineers can come too late
Listen: 22:06–28:41; 47:54–51:51. Zhu describes a triangle of algorithms, systems, and data. One architectural choice might offer a small quality improvement at a large performance cost, or the reverse. If algorithm researchers finish the structure before infrastructure is invited to accelerate it, some costs are already embedded.
He discusses an unnamed model that looks small in parameter count or active parameters and scores well, yet may run less efficiently than a larger model. A parameter-versus-quality chart is not an actual-cost-versus-quality chart. Communication, memory access, and implementation affect the bill. This remains Zhu’s example; the article neither identifies the company nor presents it as a public benchmark replication.
Mixture of experts, or MOE, illustrates cost constraints influencing architecture. Activating only some experts can retain a large capacity while reducing the work per use. Zhu recalls exploring MOE because business teams found a model too expensive, emphasizing that systems engineers can participate in design. This does not prove every early MOE adopter has a better organization, or that the architecture is inherently cheaper. It must be tested on actual hardware and workloads.
The potential advantage of a small team is shared discussion of tradeoffs, not small headcount itself. Joint responsibility requires aligned objectives and decision authority. Asking engineers to show more initiative cannot by itself remove organizational resistance.
4. Durable third-party value requires influence upstream or downstream
Listen: 11:20–16:15; 28:41–39:28. If everyone can access the same models and hardware, a middle layer offering temporary acceleration can be caught and pushed into price competition. Zhu advises practitioners to understand models or hardware deeply enough to join their design. He compares model services with platforms whose exclusive content gives users a reason to stay beyond generic aggregation.
Joint design also carries technical direction risk. The issue is participation in changing the architecture, rather than passively betting on a partner. Popular open models encourage adaptation while concentrating engineering investment around existing designs. A new architecture can force some of that work to be redone. Engineering problems with a visible route to completion and paradigm problems without one need different expectations.
At release, 阶跃星辰 sought to adapt models to Chinese chips and build mutual distribution benefits. Free commercial use was a company statement in the conversation. Actual use requires checking the corresponding version and license; a podcast is not an authorization document. Zhu also does not assert that every company must eventually build chips. His point is that a competitor integrating models and hardware particularly well could pressure others.
5. Visual reasoning matters when text loses spatial and operational relations
Listen: 39:28–43:48. Recognizing objects in a photo differs from deciding how to retrieve an obscured item from a cabinet. The latter requires understanding obstruction, location, and sequence: what moves first, and what can then be taken? Compressing the scene into text may discard those relationships, motivating direct reasoning from visual information.
Zhu’s robot “cerebrum and cerebellum” analogy separates task planning from balance and movement control. He also mentions locating a photograph and solving a photographed problem. Producing steps does not establish safe robot execution; location judgments introduce error and privacy concerns. These safety and privacy limits are editorial additions to prevent demonstrations from becoming claims of field reliability.
Models and systems therefore need joint assessment. A more direct input path may preserve information, while its actual latency, cost, and task accuracy still require measurement. Statements about multimodal leadership and future reductions in generation cost remain the company’s contemporary assessment and predictions.
6. Methods that benefit from more computation still need present delivery value
Listen: 52:58–57:58. Zhu invokes The Bitter Lesson: methods that exploit computation effectively may outperform short-lived tricks over time. For infrastructure, the question is whether a system and architecture keep benefiting as scale grows, rather than extracting a one-time advantage from a fixed setup. This does not eliminate engineering detail, and the conversation’s hardware history should not be treated as a rigorous physical law.
He sees both interdependence and pressure between agents and models. Application teams use engineering to make models “stand on tiptoe” to complete tasks; model providers may later absorb the capability natively. Yet providers may not discover those scenarios first. Zhu explicitly says he cannot see the final ecosystem. This cannot be reduced to a prediction that all applications disappear. An editorial question is whether today’s engineering advantage builds user relationships, reliable workflows, or other value that can accumulate.
Exercise: draw two curves for the same task
Choose a frequent task and compare two model-and-system options. One curve records model quality or scores. The other records cost per successful task under identical task, concurrency, and hardware conditions, with time to first token, total completion time, and failure recovery annotated. Separate input processing from output so an average speed does not conceal the bottleneck.
Ask product, algorithm, and engineering colleagues to write one shared tradeoff: which user benefit justifies which cost? If utilization improves without improving the target result, explain where its value comes from. Finally estimate savings using actual monthly usage before choosing procurement, internal development, or further observation. This is an editorial exercise, not the company’s internal acceptance procedure.
Sources and listening guide
Prepared from the complete transcript and official description and chapters, without replaying audio or testing models. Many transcript answers are mislabeled as the host; the synthesis follows question-and-answer meaning and official guest information. Uncertain hardware names and parameter counts are not reconstructed. Cost examples, company judgments, and commercial statements retain their source boundaries.
- 01:07: Infrastructure layers and shared objectives.
- 10:09: The scale calculation for internal investment.
- 17:45: How optimization objectives change outcomes.
- 22:06: Joint algorithm, system, and data tradeoffs.
- 36:35: Coordinating model and chip adaptation.
- 47:54: Parameters versus actual operating cost.
- 55:43: Long-term computation and engineering value.
Related themes: AI infrastructure, model evaluation, and organizational coordination. Continue with Wu Yi on reinforcement learning and Ji Yu on Nvidia for connections between training throughput and hardware-software ecosystems.