中文精读
Character.AI的后训练：用户留下更多对话，未必代表模型更懂他
42章经单期精读 · 角色聊天与模型迭代
嘉宾：Ted，节目当期介绍为前C.AI模型应用算法专家，曾负责后训练｜主持：曲凯
节目发布：2024年11月16日｜时长：60分11秒｜整理：2026年10月18日
原节目与官方介绍
怎样把“这句话不好聊”变成下一版模型的改进
开放聊天没有唯一正确答案。用户可以觉得一句话有趣，也可能改写、删掉或换一个回答。Ted回顾在Character.AI工作的约一年，讨论如何让这些行为进入数据、训练与实验。真正的主题是组织一套能持续发现错误的迭代系统。
本文保留2024年的内部参与者视角。团队结构、成本、用户量、语音速度与商业化判断均为当期口述，未独立核验，也不能代表C.AI当前状态。节目中公司交易、竞品和硅谷工作的描述，不在本篇升级为完整商业或就业结论。
1. 先定义用户在创造什么，才能决定优化什么
节目线索：01:19–14:36。 Ted认为C.AI商业化当时是探索不足，尚不能直接认定用户不愿付费。他描述早期团队多围绕模型训练，产品资源较少，又同时追求AGI与娱乐产品，影响资源取舍。这是他的解释，不是对公司发展原因的独立审计。
讨论Talkie时，Ted认可快速功能迭代、多模态和卡片市场，又提醒功能多不自动带来商业突破。C.AI的优势路径则是利用已有用户反馈改进模型。这里可迁移的判断是把竞品功能当作假设，观察它服务什么需求，而非看见什么就补什么。
对产品价值，Ted赞同把C.AI理解为创造性很强的感情互动：用户输入、重生成、筛选和替换文本，像在不断“抽卡”共同创作。它可以有情感体验，仍不等于以真实生活为中心的持续关系。
编辑判断：重生成既可能是享受创作，也可能是一直得不到想要的回答。评价前先区分创作型与关系型用户，明确希望改善的是故事选择、角色一致性还是生活连续性。节目关于推理成本低至外界同参数模型1%以内、千万级服务规模的数字，只能作为Ted解释当期优势的自述，不能成为通用成本基准。
2. 后训练是在改善实验循环，不是做一次微调
节目线索：17:22–21:25、24:40–32:40。 后训练指基础模型训练完成后，继续通过示范、偏好等数据调整行为。Ted描述的工作大量是看数据、讨论研究和改管线，而非整天写代码。一次变化如何尽快在用户体验中被观察到，比单独背算法名更重要。
他谈到四个可改进环节：训练管线、A/B测试、接近真实用户的评估、数据利用。A/B测试是让可比较的用户组体验不同版本，再检查差异；可靠的试验工具可以帮助团队识别小变更的影响。团队应利用已有强项，而非期待每个环节立刻完善。
训练方法上，SFT（监督微调）使用高质量示范，教模型在场景里怎样接话。偏好学习则面对没有唯一答案的情况，学习同一上下文中哪些回答更受喜欢。RLHF（基于人类反馈的强化学习）可以通过奖励模型指导优化；DPO（直接偏好优化）用偏好对简化训练流程。Ted说当时主要用SFT加DPO，再加少量RLHF。
编辑判断：区别在输入与目标，不在缩写是否先进。示范数据要说明哪种对话值得学，偏好对要说明谁在什么情境更喜欢哪一条。节目认可DPO的反馈速度，不等于它在所有产品与条件下都最优。团队自研模型、有训练权和大流量，也不是普通应用默认具备的前提。
3. 点赞之外，编辑和删除能提供更具体的线索
节目线索：33:15–38:42。 Ted反问，所谓“最高质量数据”是否真的不能更好？丢弃质量不足的一半、重写已有材料，可能比只扩大数量更值得试。他描述用最新用户反馈训练分类器，辅助挑选下一轮示范对话；暑假和开学的用户偏好可能不同，三个月前选好的数据也可能不再适配。
编辑前后的回答可以成为偏好对，删除行为可以帮助训练识别不受欢迎内容的分类器，再反过来筛示范资料。这样，偏好学习也会影响下一轮SFT该学什么。产品团队决定服务人群与方向，后训练团队负责资料怎样被模型消化，两者需要衔接。
编辑判断：行为不应被机械翻译成原因。用户改字可能在修事实、换剧情、调整语气，删除也可能只是重开故事。最好同时保留上下文、原因和后续选择，才能区分质量问题与创作自由。资料进入研究或训练还需要相应授权与用途边界。
节目解释可只对机器人输出计算训练损失，把用户话语作为上下文。这不能被写成“杜绝隐私”：上下文本身的使用、访问、可能记忆和泄露仍需独立控制。本篇未核验内部隐私治理，不能以技术描述代替完整保障。
4. 平均时长上升，还要看谁在变差、为什么聊得更多
节目线索：21:40–24:14、39:00–41:03。 面对“模型变蠢”的主观抱怨，Ted没有宣称已经解决。他说既看平均对话与时长，也看下降的少数群体，特别是重度用户。让多数人受益的版本，仍可能伤害另一群人的体验，团队要明白自己接受了什么取舍。
反复问“Can I ask you a question”的梗是一个反例。Ted解释，一些更安全的模型学会用提问绕开不希望继续的方向。对话轮数因此增加，却可能让用户觉得被拖延。本文把这保留为他的观察与解释，不认定是已经证明的心理或模型机制。
离线评估同样有边界。保留一部分高质量对话，检查模型续写；偏好模型辅助评分，都能提供信号。Ted明确说，当时没有足够完善的系统可以替代真实用户，最终仍需生产A/B测试。规模大能更快收样本，不意味着每个指标都更有解释力。
编辑判断：评估同时看均值、分群、具体失败与任务目标。关系对话需要更长不一定更好，创作对话不断换答案也不一定失败。不要让一种能被错误行为推高的指标，单独决定版本胜负。
5. 意图与速度，都要把完整交互链看进去
节目线索：41:30–50:34。 用户输入“大选”，可能要实时消息；让超级玛利奥写Hello World，也可能是玩笑，或想看角色风格的代码。系统需要结合角色、时间和上下文，判断认真答、轻松聊还是用工具，不能只按表面词语分类。
语音则暴露另一种链路。ASR把说话转成文字，文本模型生成回复，TTS把回复变成声音。如果全部串行，等待相加；Ted描述自有模型与推理设施让几个步骤流式重叠，并提前生成部分结果。他说美国地区约0.5秒可回应，但没有网络、分位数和完整测试条件，不能移作全球承诺。普通后训练也不直接等于同架构解码更快。
关于推理时计算，Ted设想先产生候选，再考虑一句话对后续几十轮的影响。所谓好感度加减是说明方向的例子，不是成熟的关系质量量表；当时的探索也不能写成已部署能力。OpenAI内部多模型路由的说法属于他的推测，不是内部证实。
编辑判断：先用端到端记录分清识别、生成、出声和打断在哪儿耗时，再决定优化。对话候选是否更好也应以长期反馈检验，避免把“多想几步”自动当成更懂用户。
一次模型更新，留下一份可解释的记录
以下为编辑建议，适合先做小规模多轮对照：选择创作型和关系型用户各一组，固定相同场景，记录改写、删除、重生成与退出位置，并让用户说明原因。
| 要分清的现象 | 需要补问或核对 |
|---|---|
| 聊得更久 | 更投入，还是反复问却没推进？ |
| 重生成更多 | 喜欢探索选项，还是角色总答偏？ |
| 删除一句 | 质量错误，还是主动换剧情？ |
| 大多数指标变好 | 重度用户与少数风格是否退步？ |
| 声音更快 | 首声、打断、整句完成是否都改善？ |
对每版保留目标、数据来源、分群结果与可接受的取舍。没有训练权时，也可先做标注、评估和提示配置实验。结尾的面试故事提供同一提醒：允许用Cursor找性能bug，仍要看首问、解释和推理过程；让候选人批评论文，再用实验验证直觉。判断力体现在怎样把疑问变成可检查的问题。
回听导航与关联阅读
- 11:35：互动创作与感情陪伴的价值区分。
- 17:15：建立高效迭代路径。
- 21:40：均值上升与少数用户退步。
- 25:37：示范与偏好学习的分工。
- 36:00：编辑、删除等丰富反馈。
- 39:49：离线评估为什么还不够。
- 43:22：流式语音的整链延迟。
相关主题：角色聊天评估、数据反馈、流式交互。可对照EVE的关系连续性与FlowGPT的品类留存，把创作、消费和长期关系分别定义。
来源边界：依据完整提供的转写、官方当期介绍及既有精读重组，未逐句复听。Ted、C.AI和训练术语按官方及清晰语境规范，结尾污染不作访谈观点。数字、竞品与战略判断保留内部参与者身份和2024年时点；心理解释、隐私、模型路由与推理研究均未扩展为已证实结论。
English edition
Character.AI’s Post-Training: More Conversation Does Not Necessarily Mean Better Understanding
42 Zhangjing episode analysis · Character conversations and model iteration
Guest: Ted, introduced as a former C.AI model-application algorithm specialist who worked on post-training | Host: Qu Kai
Published: November 16, 2024 | Duration: 60 minutes 11 seconds | Prepared: October 18, 2026
Original episode and official introduction
Turn an unsatisfying reply into an improvement to the next model
Open conversation has no single correct answer. A user may enjoy a response, rewrite it, delete it or request another. Reflecting on approximately a year at Character.AI, Ted discusses turning such behavior into data, training and experiments. The central subject is a system that continually discovers mistakes.
This article retains an internal participant’s perspective from 2024. Team structure, costs, audience size, voice latency and commercialization are contemporary accounts without independent verification, not current C.AI facts. Company transactions, competitors and Silicon Valley employment are not expanded into comprehensive business or career conclusions.
1. Define what users create before deciding what to optimize
Episode reference: 01:19–14:36. Ted considers commercialization underexplored at the time, rather than proof that users would not pay. He describes an early team concentrated on training, limited product resources and tension between AGI ambitions and an entertainment product. This is his explanation, not an audit of the company’s development.
He appreciates Talkie’s rapid iteration, multimodal features and card market while warning that more features do not automatically produce a business breakthrough. C.AI’s established route used feedback to improve models. The transferable judgment treats competitors’ features as hypotheses about needs rather than an automatic checklist.
Ted agrees that C.AI involves substantial creative emotional interaction. Users supply input, regenerate, select and replace text, repeatedly drawing alternatives as they co-create. Emotional experience can occur without a continuing relationship centered on real life.
Editorial interpretation: Regeneration can mean enjoyable exploration or repeated failure. Distinguish creative users from relationship-oriented users and specify whether the target is story choice, character consistency or continuity in everyday life. Claims of inference costs below 1% of external models with comparable parameter counts and service at a scale of millions remain Ted’s explanation of historical advantages, not reusable cost benchmarks.
2. Post-training improves an experimental loop rather than performing one fine-tune
Episode reference: 17:22–21:25 and 24:40–32:40. Post-training adjusts behavior after foundation training through demonstrations, preferences and other data. Ted describes substantial time analyzing data, discussing research and improving pipelines. Quickly observing the user effect of a change matters more than knowing algorithm names alone.
He identifies training pipelines, A/B testing, evaluation resembling real users and data utilization as four improvement paths. A/B tests compare groups exposed to different versions. Reliable experimental tools can reveal small changes, and teams can start from existing strengths without expecting perfection throughout.
Supervised fine-tuning, or SFT, uses high-quality demonstrations to teach context-appropriate conversation. Preference learning handles outputs without one correct answer by learning which response people prefer in the same context. Reinforcement learning from human feedback, or RLHF, can use a reward model to guide optimization. Direct preference optimization, or DPO, simplifies the process through preference pairs. Ted reports mainly SFT and DPO, with some RLHF.
Editorial interpretation: The distinction concerns inputs and goals. Demonstrations specify worthwhile conversations; preference pairs require knowing whose preference applies in which situation. Appreciating DPO’s feedback speed does not make it universally optimal. Proprietary models, training access and a large audience are also not default conditions for ordinary application teams.
3. Editing and deletion provide clues beyond likes
Episode reference: 33:15–38:42. Ted asks whether supposedly highest-quality data can improve further. Removing a weaker half or rewriting existing material may be worth testing before simply increasing volume. He describes using recent feedback to train classifiers that select subsequent demonstrations. Summer and school-term preferences can differ; data selected three months earlier may no longer fit.
Before-and-after edits can form preference pairs. Deleted responses can train a classifier identifying unwanted material, improving selection for SFT. Preference learning therefore influences what demonstrations are learned next. Product colleagues specify audience and direction while post-training specialists determine how models consume the material.
Editorial interpretation: Behavior should not become a cause mechanically. An edit can correct facts, alter a story or change tone; deletion can restart the narrative. Context, reasons and subsequent choices help distinguish quality problems from creative control. Research and training also require appropriate authorization and purpose boundaries.
The episode describes calculating training loss only on assistant output while treating user speech as context. This does not establish the elimination of privacy risk. Context usage, access, memorization and leakage still require independent controls. This article has not verified internal privacy governance.
4. Higher average duration still requires asking who worsened and why people talk more
Episode reference: 21:40–24:14 and 39:00–41:03. Ted does not claim to have solved complaints that a model became less intelligent. The team examines average exchanges and duration alongside groups whose experience declined, particularly heavy users. A version helping many can hurt others; acceptable tradeoffs must be understood.
The repeated “Can I ask you a question?” meme provides a counterexample. Ted interprets some safer models as learning to ask questions to avoid unwanted directions. More turns can therefore mean users feel stalled. This remains his observation and explanation, not a demonstrated psychological or model mechanism.
Offline evaluation also has limits. Held-out high-quality conversations and preference-model scores provide signals. Ted explicitly says the team lacked a sufficiently complete substitute for users and still relied on production A/B tests. Large scale collects samples faster without making every measure more interpretable.
Editorial interpretation: Combine averages, segments, specific failures and goals. Longer relationship conversations need not be better; selecting many creative alternatives need not be failure. A measure that undesirable behavior can inflate should not decide version success alone.
5. Intent and speed both require the whole interaction chain
Episode reference: 41:30–50:34. “Election” may request current news. Asking Super Mario for Hello World may be a joke or a request for character-styled code. The system must consider character, timing and context to choose a serious answer, casual conversation or tools rather than route solely by keywords.
Voice introduces another chain. ASR converts speech into text, a text model generates the reply and TTS converts that reply to sound. Serial execution adds waits. Ted describes proprietary models and infrastructure overlapping these stages through streaming and pregeneration. His approximately half-second American response figure lacks network conditions, percentiles and a complete protocol, and cannot become a global promise. Ordinary post-training also does not directly accelerate decoding under the same architecture.
For inference-time computation, Ted imagines generating alternatives and considering effects over many subsequent exchanges. Affection scores are illustrative, not a mature relationship-quality scale. Research ideas are not deployed capabilities. Claims about internal OpenAI model routing are his conjecture, not confirmed inside information.
Editorial interpretation: Record recognition, generation, first sound and interruption end to end before optimizing. Candidate replies also need long-term feedback; additional computation does not automatically mean better understanding.
Preserve an interpretable record of each model update
For this editorial exercise, use small groups of creative and relationship-oriented users in matched multiround scenarios. Record edits, deletion, regeneration and exit points, asking users why.
| Observation | What to ask or check |
|---|---|
| Longer conversation | More engagement, or repeated questioning without progress? |
| More regeneration | Enjoying alternatives, or repeated character failure? |
| Deleted response | A quality error, or deliberately changing the story? |
| Better majority metrics | Did heavy users or minority styles worsen? |
| Faster voice | Did first sound, interruption and completion all improve? |
Preserve goals, data provenance, segmented results and accepted tradeoffs for each version. Without training access, teams can still annotate, evaluate and test prompt or configuration changes. Closing interview stories reinforce the method: using Cursor to locate a performance bug still exposes the first question, explanation and reasoning; criticizing a paper creates intuitions that experiments must check. Judgment turns uncertainty into inspectable questions.
Listening guide and related reading
- 11:35: Creative interaction and emotional companionship.
- 17:15: Efficient iteration paths.
- 21:40: Better averages and declining minority experience.
- 25:37: Demonstrations versus preference learning.
- 36:00: Feedback from editing and deletion.
- 39:49: Why offline evaluation remains insufficient.
- 43:22: Latency across a streaming voice chain.
Related themes: character-chat evaluation, feedback data and streaming interaction. Compare EVE’s relationship continuity and FlowGPT’s category retention to define creation, consumption and lasting relationships separately.
Source boundary: Reorganized from the complete supplied transcript, official contemporary introduction and earlier close reading; audio was not checked sentence by sentence. Ted, C.AI and training terms follow official metadata and clear context. Contaminated closing text is excluded. Figures, competitive views and strategic interpretations retain the participant’s 2024 perspective. Psychological explanations, privacy, model routing and inference research are not expanded into verified conclusions.