505 posts, 2024-03 to 2026-05. From running 7B on a single 3060, to running Qwen 2.5 Coder 32B on a 4-node Mac mini M4 cluster (18 tok/s), running full DeepSeek on dual 512G M3 Ultra (20 tok/s), to the 2026 OpenCode black card that lets you use any model for $200.
One sentence:Models are no longer scarce; what is scarce is "the ability to integrate them in an engineering way".
0. Reading Guide
- Want to understand the 2024–2026 model evolution timeline → §1
- Want to do fine-tuning → §3 (Unsloth + GRPO hands-on)
- Want to know which hardware to choose → §4 (Mac mini / 3060 / Jetson / M4 Max)
- Want to understand the Scaling Law debate → §2.1
- Want to build model routing / multi-model committees → §5
- Want to learn about output length bottlenecks → §2.2
- Recommended tool recipes → §6
1. Model Evolution Timeline / Key Inflection Points
1.1 2024 H1: A hundred models bloom, the local revolution
Core events:
| Date | Model / Event | Key significance |
|---|---|---|
| 2024-03 | Tencent paper "More Agents Is All You Need" | Quantity compensates for capability / Llama2-13B × 15 votes beats Llama2-70B (GSM8K 35%→59%) |
| 2024-04 | Cohere Command R+ (104B, RAG-enhanced) | Lao Huang explicitly stated for the first time that "Command R+ is the first choice for knowledge bases, not GPT-4" |
| 2024-04 | NVIDIA Llama3-ChatQA-1.5-8B(Paper) | RAG-specialized fine-tuning on Llama3, GGUF available on LM Studio |
| 2024-04 | Meta Llama 3 8B / 70B open-sourced | 70B performance exceeds Claude Sonnet / Mistral Medium |
| 2024-04 | Mixtral 8x22B open-sourced | MoE architecture becomes mainstream |
| 2024-05 | OpenAI GPT-4o | Multimodal + free |
| 2024-05 | Alibaba Qwen 1.5-110B / Qwen 2 | Chinese open-source models enter the first tier |
| 2024-06 | Gemini 1.5 Pro large context | Lao Huang praised it early on, but it would take until 2.0 to become a mainstay |
| 2024-06 | Claude 3.5 Sonnet + Artifacts | Lao Huang: "I won't renew ChatGPT Plus next month" (2024-07-01) |
| 2024-07 | GPT-4o mini($0.15/M in) | Price slasher, made GraphRAG affordable to run |
| 2024-07 | Meta Llama 3.1 405B open-sourced, 128K context | Close to GPT-4o |
| 2024-08 | Zhipu GLM-4-9B fine-tuned to output 11,000 Chinese characters in one go | The output length bottleneck was broken(Lao Huang ran a xianxia novel test) |
| 2024-08 | xAI Grok 2 | Image generation + recognition |
Lao Huang's main stack from 2024-Q1 to Q2:
- Local: Mistral / Qwen 1.5-7B (Ollama)
- Embeddings: nomic-embed-text (local) / text-embedding-3-large (OpenAI)
- Cloud: GPT-4 + Command R+ (knowledge base)
1.2 2024 H2: The Cursor era + multimodal breakthroughs
| Date | Model / Event | Key significance |
|---|---|---|
| 2024-08 | Cursor goes mainstream | Built a Chrome summary plugin with zero code in 10 minutes (2024-08-31) |
| 2024-08 | Alibaba Qwen2-VL-2B / 7B / 72B | Domestic multimodal becomes usable, understands 20-minute videos |
| 2024-09 | DeepSeek V2.5 | Merged Chat + Coder |
| 2024-09 | OpenAI o1 / o1-mini | Slow thinking / reinforcement learning generates internal CoT |
| 2024-09 | Qwen 2.5 / Qwen2.5-Coder / Qwen2.5-Math | Alibaba full product family |
| 2024-10 | Alibaba Qwen 2.5 Coder 32B | Coding capability approaches the top tier |
| 2024-10 | NVIDIA Llama-3.1-Nemotron-70B | Defeats GPT-4o, 140+ models, second only to o1 |
| 2024-11 | Qwen 2.5 Coder 32B-Instruct | Mac mini M4 cluster 18 tok/s (see §4) |
| 2024-11 | Qwen 2.5-Turbo 1M context | First domestic million-level context model |
| 2024-11 | Pixtral Large / DeepSeek-R1-Lite | |
| 2024-11 | Thinking-Claude browser plugin | Performative thinking, the upper limit of prompt engineering |
| 2024-11 | M4 MAX real test: runs quantized 70B at 11 tok/s, 35W power consumption | |
| 2024-12 | Meta Llama 3.3-70B | Performance = 3.1-405B, cost 1/10 |
| 2024-12 | Microsoft Phi-4 14B | Math (GPQA / MATH) surpasses GPT-4o / Qwen 2.5-14B / Llama-3.3-70B |
| 2024-12 | Google Gemini 2.0 | Multimodal + tool calling + Agent / Lao Huang declared it "extremely awesome" |
| 2024-12 | Gemini 2.0 Flash Thinking | Slow-thinking multimodal |
| 2024-12 | DeepSeek R1 + APP launched (2025-01-12) | Chinese R1 goes viral |
Lao Huang switched his main stack in 2024-Q4:
- Main: Claude 3.5 Sonnet (coding) + Gemini 2.0 (chat/search)
- Local: Qwen 2.5-7B / 14B Q4
- Local server: Mac mini M4 (OLLAMA_KEEP_ALIVE=-1, paired with Enchanted iOS)
1.3 2025 H1: Reasoning model explosion + the DeepSeek moment
| Date | Model / Event | Key significance |
|---|---|---|
| 2025-01 | DeepSeek R1 APP launched | National-level |
| 2025-01 | Microsoft Phi-4(14B) | See above |
| 2025-01 | Kai-Fu Lee interview:Scaling Law is slowing down, startups have an extremely low success rate building ultra-large models | See §2.1 |
| 2025-01 | UC Berkeley DeepScaleR-1.5B: math surpasses o1-preview, $4500 A100 | Small-model specialized training paradigm |
| 2025-01 | DeepSeek-R1-1.5B runs on iPhone 12 mini (PocketPal) | On-device era |
| 2025-02 | Lao Huang used Unsloth GRPO to fine-tune Qwen2.5 3B | T4 / 1 hour / 9.9 vs 9.11 test passed (see §3) |
| 2025-02 | Cost of local deployment of full DeepSeek revealed | Not cheap, dual 512G M3 Ultra gets about 20 tok/s |
| 2025-02 | xAI Grok 3 | Lao Huang: "All-in-one trend, only mode distinctions, no model distinctions" |
| 2025-02 | GPT-4.5 | "Outrageously expensive" |
| 2025-02 | Claude 3.7 Sonnet | #1 in AI Coding, Cursor supports it immediately |
| 2025-03 | Tencent Hunyuan / Alibaba QwQ-32B | Reasoning models explode, QwQ-32B runs at 10 tok/s on M4 Pro 48G, stronger than DeepSeek-R1 32B |
| 2025-03 | DeepSeek-V3-0324 | Rock-bottom price + roughly Claude-3.7-level results at one-tenth the cost |
| 2025-03 | Google Gemma 3 27B single H100 | Between R1 and V3 |
| 2025-03 | Gemini 2.5 Pro | Lao Huang: "Amazing artifact for scanning codebases + article suggestions, Cursor subscription is an absolute steal" |
| 2025-04 | OpenAI GPT-image-1 API | Image models become API-based, integrated by Adobe/Figma/HeyGen/Wix |
| 2025-04 | Gemini 2.5 Flash hybrid reasoning | Thinking Budget configurable |
| 2025-05 | Claude 4 Sonnet | Proactively uses Sequential Thinking MCP, explains while working |
| 2025-05 | Claude 4 Opus(Mac version Prompt House) | "Far more efficient than human programmers" |
1.4 2025 H2: The Code Agent era + the rise of Kimi K2
| Date | Model / Event | Key significance |
|---|---|---|
| 2025-06 | GitHub Copilot Pro+ Claude integration | Lao Huang switched his main battlefield here (US residential IP is stable) |
| 2025-07 | Kimi K2 enters OpenRouter coding leaderboard at #4 | First domestic model to enter international coding Top 5 |
| 2025-07 | Anthropic Claude Code + Sub-agents | Haiku/Opus model tiers, see newtype · AI Coding Practice and Tool Stack Evolution |
| 2025-09 | OpenAI Codex | "Anthropic buys keywords to poach users" / finds bugs that Claude Code misses |
| 2025-10 | Anthropic Claude Skills | Lao Huang built Super Analyst / Super Writer |
| 2025-11 | Kimi K2 Thinking + Claude Code direct API connection | Triggered Lao Huang to build newtype AI OS(2025-11-09 post 1965) |
| 2025-11 | Domestic model Skills test: Kimi K2 > GLM-4.6 > Minimax M2 | Claude Code Router test |
| 2025-11 | Gemini 3 Pro | Lao Huang: "fast + thinking" binary choice fits product planning / Codex replaces Claude Code |
| 2025-11 | Claude Opus 4.5 | |
| 2025-12 | Gemini Deep Think Ultra tier $200 | Model committee takes over |
| 2025-12 | NotebookLM upgraded to Gemini 3 | Lao Huang declared it "the strongest knowledge base application" |
1.5 2026 Q1: OpenCode black card + domestic models become practical
| Date | Model / Event | Key significance |
|---|---|---|
| 2026-01 | OpenCode black card $200 for any model (sold out in seconds) | New API business form |
| 2026-01 | GPT-5.2 | Lao Huang switched all three Agents to GPT-5.2 |
| 2026-01 | Gemini "Personal Intelligence" | Gmail / Maps / Photos personalization |
| 2026-02 | Kimi K2.5(Highest-usage model on OpenRouter in 2026-02) | Main model used by OpenClaw |
| 2026-02 | Minimax M2.5 / GLM-5 / qwen3.5 runs on 256G memory | Localization to cut costs |
| 2026-03 | Perplexity Computer / Personal Computer $200 tier | Investment decision assistant |
| 2026-04 | Claude Opus 4.7(GitHub Copilot Pro+ promotion) | Lao Huang's main model |
| 2026-04 | DeepSeek V4 strong reasoning | Yu Bufan's test testimonial: "Produces relatively professional answers in professional-domain reasoning" |
| 2026-04 | GitHub pauses Copilot signups | Agents' compute consumption is unsustainable, stabilize existing users first |
| 2026-04 | Claude Identity verification required(passport + selfie) | Lao Huang announced switch to OpenCode |
| 2026-05 | Opus 4.7 / Sonnet 4.6 / Gemini 3.1 Pro Perplexity model committee | Three-way divergence + consensus analysis |
2. Three core debates
2.1 Does Scaling Law still hold?
Two camps:
Camp A: Scaling Law is slowing down (Kai-Fu Lee 2025-01 post 708)
- Only big tech can continue to build ultra-large models; startups have an extremely low success rate
- The models AI applications need are small enough, fast enough, cheap enough, capable enough, not necessarily the largest and most general
- If ultra-large models can't be made cheap, they don't need to be released; they can serve as internal "teachers" to train small models
- 2025 accelerates AI commercialization and also accelerates elimination
Camp B: Scaling Law still holds (Cohere CEO Aidan Gomez 2024-10 post 569)
- Scaling Law remains valid for a considerable period
- But the reality is that most companies' businesses cannot yet support GPT-4-level deployment costs
- Trend:General models and vertical models coexist, first use a general large model for prototyping → once it works, do vertical fine-tuning
Lao Huang's judgment (2024-08 post 436):
"For the underlying general large models, use whichever is good or cheap.For the upper-layer vertical small models, truly invest in fine-tuning and optimization to make them a moat. "
2.2 Output length is the real bottleneck (2024-08 post 466 / classic)
Test: Ask LLMs to write a 10,000-character xianxia novel
| Model | Performance |
|---|---|
| ChatGPT | Slacked off, only gave an outline + first chapter, and rushed an ending when prompted |
| Claude | Dutifully wrote it in chapters, 3000 characters per chapter |
| Fine-tuned GLM4-9B | Wrote 11,000 Chinese characters in one go (Colab A100) |
Lao Huang's view:
"What limits large model performance today is not context length (128K is already quite enough), but output length. An average of 2,000 Chinese characters and a maximum of 3,000 Chinese characters is really not enough. Zhipu has found a solution." (2024-08-28 post 468)
2.3 TPU vs GPU (2025-11 post 2055)
"In episode 132, I will explain the difference between TPUs and GPUs.Seeing so many people hyping that TPUs can replace GPUs is so ignorant. "
Lao Huang holds a long-term view on NVDA / NBIS / TSLA, and based on this judges there is still room to buy H100 / B200.
3. Unsloth + GRPO Fine-Tuning Hands-On (2025-02 post 801 / classic)
3.1 Core hands-on steps
GRPO fine-tuning on Qwen2.5 3B significantly improves logical reasoning ability.
- Platform: Google Colab T4 / done in 1 hour
- VRAM requirements:
- 1.5B model → 7G VRAM
- 7B / 14B models → 15GB VRAM
- Model upload: Automatically uploaded to Hugging Face huangyihe/qwen2.5-r (three GGUF versions: Q4/Q5/Q8)
- 9.9 vs 9.11 test:Original model had confused thinking; after fine-tuning it answered correctly with clear reasoning
3.2 Dataset format matters a lot (2025-02 post 807)
Unsloth uses the GSM8K dataset (math problems + detailed step-by-step solutions). The quality is significantly higher than ordinary one-question-one-answer formats.
Lao Huang's business insight:
"This Unsloth toolkit lowers the barrier to fine-tuning (small models only require 7GB of VRAM) and improves model capabilities. If you have your own methodology and corresponding use cases (for example, many business-focused content creators), it is very suitable to produce a corresponding dataset for fine-tuning. You can output a 3B model file, give it to your fans/users, and run it on mobile phones (PocketPal). This has huge commercial potential."
3.3 When to Fine-Tune? (2024-09 Post 478)
Besides giving models new knowledge, fine-tuning can also provide new capabilities:
- Llama has poor Chinese capabilities → fine-tune a Chinese version; RAG cannot solve this
- The Perplexity team fine-tunes open-source models to strengthen search capabilities
When not to fine-tune:
- If you only need knowledge-level improvements, try RAG first (the cost is much lower)
- For personal use, models below 7B are already capable enough; for standardized Markdown notes → RAG hit rate is sufficient
3.4 Horizontal Comparison of Fine-Tuning Tools (2024-09 Post 513)
| Tool | Features |
|---|---|
| Unsloth | Top recommendation, low VRAM usage, ready-to-use code |
| LlamaFactory | Low-code UI (Lao Huang "looked at it but never used it; the difficulty of fine-tuning is not in the code") |
| Hugging Face PEFT | Framework-level |
| XTuner / Axolotl | Used by foreigners/Western users |
3.5 Llamafile Concept (2024-04 post 25 / 2024-08 post 447)
"Maybe in the future every new book and every textbook will come standard with a small model. This model will be fine-tuned based on the book's content. Thanks to technologies like Llamafile, it can run as a single file, which is extremely convenient."
Three key thresholds:
- Deployment threshold (solved by Llamafile)
- Fine-tuning threshold (solved by Unsloth)
- Dataset preparation threshold(Unsolved, requires accumulation in each domain)
4. Full Record of Hardware Real-World Tests
4.1 Key Hardware Configurations / Benchmarks
| Hardware | Model / Test | Benchmark Score | Source |
|---|---|---|---|
| 3060 + 16GB RAM(Lao Huang's PC) | Qwen 1.5-7B Q4 | Smooth | 2024-04 Post 50 |
| Macbook Pro 2017 | Cannot run open-source models, can only use OpenAI API | - | 2024-04 Post 70 |
| Y7000P RTX 4060 8GB + 32GB RAM | qwen:14b Q4 | Sometimes fast, sometimes slow | 2024-05 Post 220 |
| Macbook Pro M3 16M | Ollama runs smoothly, Docker has chip incompatibility | Some features are limited | 2024-04 Post 127 |
| Self-purchased server: 32 cores + 256GB + dual 3090s | Llama 70B | Can run it | 2024-05 Post 147 (featured RAG post) |
| M4 MacBook Pro Review | 70B quantized | 11 tok/s, 35W power consumption | 2024-11 Post 596 |
| MacBook Pro M4 Pro 48GB + nano-texture display Lao Huang's main machine | QwQ-32B | 10 tok/s | 2025-02 Post 887 / 2025-03 Post 951 |
| MacBook Pro M4 Max 128G Daniel | DeepSeek R1 32B(Ollama) | 10-12 tok/s | 2025-02 Post 791 |
| DeepSeek R1 70B(Ollama) | 6 tok/s | Same as above | |
| Cluster of 4 Mac mini M4 + 1 MBP M4 Max(Lao Huang, using exo) | Qwen 2.5 Coder 32B | 18 tok/s | 2024-11 Post 607 |
| M3 Ultra dual 512GB(belongs to someone else) | Full-capacity DeepSeek 671B | 20 tok/s | 2025-03 Post 943 |
| Mac mini M4 24G(Lao Huang, 7K RMB on Xianyu) | Qwen 2.5 14B Q4 | 10 tok/s | 2024-12 Post 677 |
| iPhone 12 mini | DeepSeek-R1-1.5B Q4(PocketPal) | "Speed is okay" | 2025-01-31 Post 768 |
| iPhone 17 Pro | Qwen3-8B Q4(PocketPal) | 12 tok/s | 2025-09 Post 1778 |
| Jetson Nano Super 8G($249, launched in May; Lao Huang placed an order, 2025-02 post 774) | Small-sized models | Lower performance than M4 Mac mini and 3060 | 2025-01 Post 696 |
| GB10 / DGX Spark($3000,128G,NVDA GB10) | - | 1000T FP4, stackable | 2025-05 Post 1211 |
4.2 Hardware Purchase Recommendations (repeatedly stated by Lao Huang)
- Beginner starting out: MacBook Pro M4 Max (no-brainer choice)
- Budget of 100,000 RMB: Build a workstation, 4090 + i7 + 64GB
- Heavy usage: Mac mini M4 cluster (with national subsidy price) or rent Colab Pro
- Do not accept macOS: For running image tasks such as Stable Diffusion, prioritize NVIDIA graphics cards
4.3 Key Ollama Settings (lessons learned from pitfalls)
| Setting | Value | Effect |
|---|---|---|
OLLAMA_MODELS |
Custom folder | Do not place on C drive (Windows default) |
OLLAMA_KEEP_ALIVE |
-1 |
Do not release automatically, do not unload even after 5 minutes |
OLLAMA_HOST |
0.0.0.0 |
Listen globally, accessible over local area network |
| Model precision | Q4 (fast) / Q8 (slightly slower) | Q4 is sufficient for most scenarios |
| Context length | 32K-128K | Qwen 2.5 all have 128K context |
4.4 Ollama Advanced (2025-09 Post 1786 / 2025-08 Post 1595)
- Ollama client + built-in web search(2025-08)
- Ollama Turbo $20/month Cloud server running large models (2025-08)
- Ollama Cloud(2025-09) Supports deepseek-v3 / gpt-oss, etc.
- Directly call Hugging Face models(2024-10 Post 544)
5. Model Routing / Multi-Model Committee
5.1 n8n Model Selector (2026-02 Post 1604 / Classic)
Simple search workflow:
- gpt-4.1-mini makes the judgment: output "1" for simple requests, output "2" for complex requests (requiring Deep Research)
- Model Selector connects to two Perplexity models: sonar / sonar-deep-research
- AI Agent routes according to rules
"This setup is already very close to the 'intelligent model routing' I want."
5.2 Multiple Agents Each Assigned a Model (Lao Huang's current setup)
newtype OS / OpenCode configuration:
- Chief(Opus 4.7) — overall coordination, most expensive
- Deputy(Sonnet 4.5) — task assignment
- Researcher(Gemini 2.5 Pro) — large context of 1 million tokens
- Archivist(Haiku 4.5) — memory, cheap
- Writer / Editor(Sonnet 4.5) — mid-tier performance
5.3 Perplexity Model Committee (2026-03 Post 2560)
"Perplexity's 'model committee' feature is quite interesting. It dispatches three models (Opus 4.6, GPT-5.4, Gemini 3.1 Pro, all with Thinking enabled) to conduct research separately, then draws conclusions,to see what disagreements and consensus exist. "
5.4 Cherry Studio @ Multi-Model Parallel (2025-07 Post 1528)
Use the @ button to add multiple models for parallel output.Suitable for comparing different perspectives on the same question.
6. Current (2026-05) Recommended Setups
6.1 Regular Users (no development needs)
- Gemini Pro(Google One $20) — main tool, GBP/Maps/Photos personalization
- Grok 3 / 4.2(Twitter Blue Verified) — exclusive X data + straightforward style
- Knowledge base → NotebookLM (also included)
- Network: A US residential static IP is required
6.2 Creators (writing + content)
- GitHub Copilot Pro+ $39(Lao Huang's main tool)
- Claude Opus 4.7 / Sonnet 4.5
- Gemini 2.5 Pro
- OpenRouter API Backup (registered with a US credit card, bypassing direct purchase from Anthropic)
- Client:newtype OS / Codex / OpenCode
6.3 Serious Intelligence / Investment
- Perplexity Max $200(Advanced features such as Computer Use)
- Gemini Ultra $200(Deep Think)
- Cross-model verification: Model Committee (disagreements among three providers + consensus)
6.4 Local Experimentation (data privacy first)
- Ollama / LM Studio Dual installation
- Model combination: Qwen 2.5-14B Q4 (Chinese) + Gemma 3 27B (English)
- Embeddings: nomic-embed-text (local) / text-embedding-3-large (OpenAI)
- Hardware threshold: Mac mini M4 24GB minimum, ideal is MacBook Pro M4 Max 128GB
6.5 Fine-Tuning (vertical scenarios)
- Unsloth + Google Colab Pro($10/month)
- Dataset format: GSM8K style (question + solution steps + answer)
- Model selection: Qwen 2.5-1.5B / 3B (VRAM-friendly)
- Upload to Hugging Face → users download via PocketPal (runs on phone)
7. Counterintuitive Findings / Fixed Anchors
7.1 Model Evaluation Leaderboards vs Real-World Use
Lao Huang repeatedly says:You have to try them yourself to know which one is most suitable. Leaderboards can be referenced but must not be blindly trusted.
Examples:
- Qwen2 + nomic cannot retrieve results in GraphRAG, GPT-4o succeeds (2024-08 Post 442)
- glm-4.6 does not follow SOP when running Skills, K2 strictly adheres to it (2025-11 Post 1940)
- Gemini 2.0 scores well in evaluations, but its actual knowledge summaries are very bland → Lao Huang switched to Grok (2026-03 Post 2545)
7.2 There are many strong models, but none dominates everything right now
Lao Huang's summary in 2025-01 Post 765:
"There are many powerful models, but so far none has reached the point of dominating everything. So you must try them yourself to know which one suits you best."
7.3 Models and AI applications are two different things
"GPT-4o is a model, ChatGPT is an AI application. The official applications corresponding to these four models have many differences." (2025-01 Post 766)
NotebookLM is "strict mode Gemini", Drive Gemini is "lenient mode", and Gemini APP is "general + web-connected + fills in gaps". Same model, completely different product tuning.
7.4 RAG is an "open-book exam", fine-tuning is "attending cram school" (2024-04 Post 85 / classic analogy)
"Fine-tuning is like going to cram school; after you learn it, it's all yours, internalized. A knowledge base is like buying a pile of reference materials; you flip through them every time you need them, but you never actually learn them and make them your own."
7.5 Actual Cost of Pro Plans (Lao Huang 2025-07 Post 1718)
"Claude Code pays for itself in at most two days. I'm currently on the $100 tier, using Opus for analysis in Plan Mode, and often let it do ULTRA THINK. Using two to three full sessions a day is quite cost-effective."
Calculated at OpenRouter API prices, daily consumption of $100 is not uncommon (2025-07 Post 1543 Lao Huang "spent $100 in just two hours in the morning"). Therefore, the subscriptions from Anthropic / OpenAI / Google are selling at a huge loss to capture the market.
7.6 The business model of model subscriptions is unsustainable (2025-08 Post 1516)
"Claude models are hard currency right now. Whoever provides more model quota will have a popular product (Kiro). Whoever cannot bear the cost and starts cutting back will go downhill (Cursor)."
→ Later Cursor really couldn't hold on; Lao Huang unsubscribed in 2025-07 and switched to Claude Code. Later, after getting banned 4 times by Claude, he switched to Codex / OpenCode.
7.7 After model capabilities are pushed to the limit, data becomes the moat
"For the underlying general large models, use whichever is good or cheap.For the upper-layer vertical small models, truly invest in fine-tuning and optimizing them so they become a moat." (2024-08 Post 436)
→ Verification: The real moat of Lao Huang's newtype Profile is the Skills + Prompts library + content production SOP, not the model itself.
8. Common Pitfalls
8.1 Ollama downloads models to C drive by default on Windows
Set the OLLAMA_MODELS environment variable.
8.2 Llama series is poor at Chinese
Switch to Qwen series.
8.3 Ollama can only load GGUF format (.bin does not work)
.bin models downloaded from ModelScope cannot be used by Ollama.
8.4 LM Studio model files require two layers of folders
After downloading a model from Hugging Face, you need to create two layers of folders according to LM Studio rules; you cannot place them directly.
8.5 GraphRAG embedding models frequently throw errors when used locally
Running the index creation step with local large models (qwen2 + nomic) works, but the embedding stage often throws errors and requires hacky modifications openai_embeddings_llm.py(Post 398, 2024-07).
8.6 Cline using ollama locally for coding is slow and poor
The 3.5 Sonnet API is the proper partner for cline; small local models cannot keep up on coding capability (Post 593, 2024-11).
8.7 Ingesting PDF / PPT / Excel into the knowledge base
- PDF → Convert to Markdown first (Marker / MinerU / markitdown)
- PPT → First extract all text into md
- Excel → Don't even bother; current RAG systems are all poor at handling tables
8.8 Same issue: Chinese search vs English search results are worlds apart
Lao Huang, Post 693, 2024-12 Prompt:
Always reply in Chinese. Prefers English sources for online searches,
with Chinese sources as a last option. Always search the web for English
pages, even if questions are in Chinese. Translate questions to English
before searching.
9. Related wiki
- newtype Knowledge System and Full Life OS Evolution — Full evolution of the knowledge system (tool focus of this wiki)
- newtype · AI Tool Stack Practice and Evolution — Tool stack evolution (Ollama / LM Studio / AnythingLLM)
- newtype · AI Coding Practice and Tool Stack Evolution — Full pipeline: Cursor → Claude Code → Codex
- newtype · Agent Architecture and Multi-Agent Orchestration — Practical extension of the "More Agents Is All You Need" paper
- newtype · MCP Protocol and Ecosystem Practice — MCP ecosystem of Skills on Ollama Cloud / OpenRouter / Codex
- newtype · RAG Practice and Knowledge Base Evolution — RAG / GraphRAG / embedding model selection
Wiki source: 505 themed posts from newtype Knowledge Planet from 2024-03 to 2026-05, full scan, no sampling. Overlaps with the tool focus of newtype Knowledge System and Full Life OS Evolution and the timeline of newtype · AI Tool Stack Practice and Evolution; this wiki focuses on model evaluation, fine-tuning practice, and hardware real-world testing three dimensions.