42 Zhangjing episode analysis · Data quality and research feedback
Guest: 孟繁青, cofounder of Evolvent AI and former Kimi RL intern, as introduced at release | Host: Qu Kai
Published: August 8, 2026 | Duration: 59 minutes 29 seconds | Prepared: October 18, 2026
Official episode and introduction
Easier training does not make data quality easy to judge
Meng used a mature training platform inside a model company. Starting a business meant building platforms and finding a complementary opportunity. Turning training into a service simplifies data submission and execution without automatically making the data valuable.
The central mechanism connects task, environment, scoring, trajectory, training and reevaluation. The introduction says NUS PhD while Meng calls himself a doctoral student, so completion of the degree is not inferred. Rankings, research routes and commercial opportunities retain their August 2026 context. One benchmark cannot establish comprehensive leadership.
1. Agent evaluation enters environments, and scoring also needs acceptance
Episode reference: 01:38–07:09. Static tests resemble examinations; newer evaluations also test action. Agents are model systems using tools to advance tasks. Office work requires operating software such as Feishu and Notion, beyond producing answers. Simulated environments bring these interactions into tests.
First validate the environment: does the same input produce behavior consistent with the real software? One matching output checks one case rather than proving the entire environment correct. Then define multiple scoring points, such as which record changes and which constraints remain, rather than accepting a model’s statement that it finished.
Meng says people and agents still inspect scoring together. Editorial interpretation: Environments, tasks and graders can each be wrong. Passing may reflect exploiting a grader. Retain comparisons with real tasks, exceptions and human samples to prevent self-evaluation becoming a closed success narrative.
2. Useful data need training results and regression checks
Episode reference: 07:19–12:41. Training services let researchers concentrate more on data. Startups and platform builders still manage environments, resources and failures; internal convenience does not mean training has no entry cost everywhere.
Initial checks include answer correctness, cheating trajectories and suitable difficulty. Easy examples may add little signal; excessively difficult ones may teach little. These are filters. Training must then test target improvement while checking whether previous abilities, such as writing, deteriorate.
Meng observes agents mechanically repairing response formats, while valuable data insights still require research experience. Editorial interpretation: Counts, tidy formatting and effective learning signals differ. Buying or building data requires baselines, independent evaluation and old-task regression checks, beyond improvement on one score.
3. RSI needs a stated level of change
Episode reference: 22:08–25:17 and 29:51–35:10. Recursive self-improvement, or RSI, has no fully shared scope in the discussion. Qu separates revising an output, modifying the execution framework and training a stronger model. Meng places them within an environment-goal-action-feedback loop. Training from scratch and surpassing oneself is a stronger objective than automatically revising a document.
A harness organizes tools and context. Meng expects it to accompany a model and potentially become simpler, rather than disappear. Some automated research resembles smarter depth-first search: write a GPU kernel, score it, backtrack after poor results and choose a promising route. A kernel is a program executing computation on a GPU.
World models, value judgments and longer context explain his view of search efficiency. His claim that there is no separate RSI foundation model is a personal argument, not research consensus. Future model companies needing only roughly ten to twenty people is also an unverified scenario.
Editorial interpretation: State the target metric, allowed changes, feedback source and compute budget. Running continuously, editing its own code and obtaining a stronger model are different acceptance claims. A biased grader may simply produce increasingly effective optimization of that bias.
4. Strong models for construction and small models for operation require an economic test
Episode reference: 13:12–21:14 and 25:46–29:01. Meng offers a non-consensus account of Chinese models: resource limits encouraged pretraining architecture innovation, while accumulated post-training data creates a harder time gap. He explicitly does not know whether this is industry consensus and lacks knowledge of some internal company causes.
Alternating benchmark leaders do not imply equivalent experience or comprehensive superiority. Organization also affects iteration; competition cannot be reduced to cards or architecture alone.
For specialized tasks, he imagines strong models building workflows and improving smaller open models, with the latter operating daily at lower cost and latency. Privacy may influence the choice. Qu forecasts prices approaching free; Meng disagrees that this is inevitable because competition and commercial strategy also affect pricing.
Editorial interpretation: Compare total cost under equal task, quality, concurrency and maintenance conditions. Local models are not necessarily cheaper. A strong model generating a harness does not establish stable operation by a smaller one. Separate construction from continuing service costs.
5. External data can broaden distributions while commercial windows move
Episode reference: 35:28–42:29 and 46:25–48:14. Ordinary labeling, expert questions, synthetic environment trajectories and human work traces coexist. Stronger models demand more novel expert questions. Human traces still contain noise and usage restrictions; personal conversations do not automatically become saleable assets.
Model companies may buy externally to supplement internal biases even while generating their own data. As evaluations and tasks converge, capable customers may build internally. Meng sees trajectories that train and improve another model as a possible next opportunity. Taking many hours or a day to produce one trajectory establishes complexity rather than a guaranteed price or usefulness.
Distillation here involves external models providing training signals. Synthetic data’s value also includes environment, questions and scoring. Meng calls distillation an accelerator rather than the decisive factor in Chinese progress. That competitive and technical position grants no permission to train on arbitrary model outputs.
Editorial interpretation: Establish provenance, rights, privacy and contractual scope before quality. Synthetic material does not automatically remove rights questions, and unclear price units should not be reconstructed into quotations.
6. Long tasks link data, algorithms and infrastructure
Episode reference: 43:11–46:01 and 48:49–58:47. Meng illustrates the tail problem with a hundred tasks: ninety-nine finish in a minute, one takes a hundred minutes and the batch waits while resources sit idle. Longer, harder data can require scheduling, parallelization and algorithm changes.
“Algorithms are data; data are infrastructure” emphasizes interdependence rather than literal identity. If improved data reduce throughput drastically, examine effective improvement per unit of time rather than the impressiveness of one long trajectory.
Coding, science and automated research are contemporary directions, without independent checks of promoted scientific results. Meng favors startup model teams over the long term, emphasizing organizational speed when technologies are widely understood. Engineering-focused hiring reflects his team’s stage rather than a general rejection of academic ability.
Editorial interpretation: Record useful results and waiting bottlenecks together. Time saved through automation may be consumed by long-tail failures, manual repairs and environment maintenance, leaving no overall efficiency gain yet.
Exercise: send one trajectory through four checks
Choose a small task with readable results. Check whether its environment matches real operation, scoring matches the goal, the trajectory is correct without cheating and previous tasks regress after improvement. Record production and validation time together.
Retain old examples even when changing only prompts or knowledge. Before generating more data, identify which failure needs a new signal. This is editorial synthesis rather than Evolvent’s internal procedure or a promise of achieving RSI.
Listening guide and related reading
- 04:37: Work environments entering evaluation.
- 10:35: The final test of data quality.
- 12:48: A non-consensus pretraining thesis.
- 21:50: Which level RSI changes.
- 39:07: Internal versus external data.
- 43:00: Long tasks and infrastructure.
- 46:11: Synthetic data and distillation.
Related themes: agent evaluation, data sourcing and research throughput. Compare Wu Yi on RL systems and Zhu Yibo on joint optimization, testing capability, cost and feedback within one task.
Source boundary: The complete supplied transcript, earlier note and official introduction were read without sentence-by-sentence audio, model replication or contract inspection. Confirmed official names are used; uncertain versions, prices and scientific results are not reconstructed. Rankings, distillation’s contribution, RSI routes and organizational forecasts remain August 2026 views rather than independent technical acceptance.