Deeppin/ articles
Dev log · Engineering notes

Deeppin Engineering

From RAG, Context, and SmartRouter to deployment — what Deeppin is made of, component by component, and why.

  1. 05-012026

    §Stress-Testing SmartRouter: 2.59M Theoretical TPM — How Much Do We Actually Get?

    Drive all four SmartRouter groups (chat / merge / summarizer / vision) in parallel and the gap between theoretical and realized capacity becomes very visible. At ~600 tok/req: 275K TPM (11% of theoretical, RPM-bound). At ~5500 tok/req: 822K TPM (32%, TPM-bound). Three structural bottlenecks surfaced: vision has no fallback chain, SambaNova latency explodes to 100s under load, and Gemini is severely under-weighted in score routing.

  2. 04-182026

    §Classifying LLM Failures Into 6 Reasons: Actionable Alerts, Observable Quotas, Trusted Zero-Quota Checks

    A single errors counter tells you "something's wrong" but never whether your key broke, the provider is down, or it's just rate limiting. Deeppin labels LLM failures with one of six reasons, pairs them with a zero-quota key/catalog endpoint and timezone-aligned RPD resets, and turns "is this 429 noise or signal?" into a well-defined question.

  3. 04-182026

    §Mac × Telegram: How I build Deeppin solo from two entry points

    Claude Code has two entry points on my side: the Mac CLI and a Telegram bot. This post walks through how they split the work — Mac for depth, Telegram for reach — and why the phone side needs an explicit /readonly mode.

  4. 04-182026

    §Running Multiple Claude Agents on One Repo: git worktree for Isolation, rebase for Convergence

    Mac CLI plus a handful of Telegram chats, all pointing Claude at the same repo, none aware of each other — sharing one clone is guaranteed to blow up. git worktree pins each chat to its own branch and directory while keeping .git object storage shared, so disk cost is near-zero; pair that with rebase-onto-main at session start and PR merges at the end, and concurrent Claude agents go from "impossible" to "unremarkable".

  5. 04-172026

    §Deeppin's Monitoring Stack: How Prometheus + Grafana Got Wired Up

    From hand-rolled /debug endpoints to a real Prometheus + Grafana stack — what metrics are exposed, how to access them, and the gotchas we tripped on.

  6. 04-172026

    §Monitoring Observability SOP: Which Panel to Open First

    Building the dashboards was the easy part. Knowing when to look and what to look at is harder. This post lays out a 5-layer metric hierarchy and 4 incident scenarios, each with the exact PromQL and panel to open first.

  7. 04-162026

    §Deeppin System Components: What Every Part Does and How They Work Together

    From Nginx to Supabase, from SmartRouter to bge-m3 — a component-by-component breakdown of what each part is, why it exists, when it's invoked, and how data flows through it.

  8. 04-162026

    §SmartRouter: Multi-Provider Free Tier Stacking with Intelligent Routing

    Deeppin's SmartRouter stacks free-tier quotas from 5 providers (Groq, Cerebras, SambaNova, Gemini, OpenRouter) across 15 models. Real-time usage tracking and proactive scoring select the best (provider, model, key) slot before 429 errors occur.

  9. 04-162026

    §Zero-Downtime Deployment: Deeppin's CI/CD and Three-Layer Testing

    The complete pipeline from git push to production: unit tests → deploy → integration tests, three gates catching issues at every level. Plus how 200+ unit tests cover all backend logic with zero real dependencies.

  10. 04-152026

    §What Happens After You Send a Message: Full Data Path

    From the moment you click send to the first token appearing on screen — what Deeppin does at every step. Frontend, network, backend, context assembly, RAG, LLM, SSE, persistence, and the boundary between parallel and sequential work.

  11. 04-152026

    §Within-Thread Context: Sliding Window and Summary Prefix

    How a single conversation thread maintains long-term memory without blowing the token window — sliding window, write-time summaries, and two-phase truncation.

  12. 04-152026

    §Between-Thread Context: Ancestor Summary Chain and Anchors

    When a pin opens a sub-thread, how key information from ancestor conversations is passed down — ancestor chain traversal, depth-based token budgets, and anchor text preservation.

  13. 04-152026

    §How Three Context Layers Cooperate — and How They Compare to Mainstream Approaches

    How the sliding window, ancestor summary chain, and RAG semantic recall divide labor, compared against full history, fixed window, and RAG-only mainstream approaches.

  14. 04-152026

    §Write-Time Summary Maintenance: Why Not Compute on Read

    Deeppin updates thread summaries asynchronously at write time rather than computing them on demand during context assembly. The trade-offs and implementation details behind this design choice.

  15. 04-152026

    §RAG and Semantic Chunking: Why the Split Strategy Determines Retrieval Quality

    Fixed-size chunking is the most common source of RAG failure. Deeppin uses embedding-based semantic chunking: split by sentence first, then use vector distance to detect topic boundaries, ensuring each chunk is a complete semantic unit.

  16. 04-152026

    §Handling Long Text: From Chunking to RAG Injection

    When a user sends very long text or uploads a file, how the system chunks, embeds, stores, and later precisely recalls relevant passages in subsequent conversation.

  17. 04-152026

    §Giving Search Capability to Models That Can't Search

    How Deeppin uses a SearXNG + LLM pipeline to give any conversation model real-time web search capability — query detection, result filtering, and streaming output.

  18. 04-152026

    §SSE Full Pipeline: Every Hop from Groq to Browser

    The complete SSE transport chain — Groq streaming API, FastAPI async generators, Nginx proxy configuration, fetch ReadableStream, and state isolation under concurrent threads.

  19. 04-152026

    §Response Optimizations That Matter for User Experience

    From UUID pre-generation and SSE streaming to Nginx configuration and frontend rendering — every engineering detail that affects perceived speed.

  20. 04-152026

    §How Many Users Can This Free-Tier Stack Handle? Capacity Limits and Optimization Paths

    Deeppin runs entirely on free tiers: 5 LLM providers (Groq + Cerebras + SambaNova + Gemini + OpenRouter, stacked via SmartRouter), Oracle ARM, Supabase, Vercel. A component-by-component analysis of actual capacity limits, the current bottleneck, and the theoretical path to break through each one.

  21. 04-152026

    §Why pgvector Over a Dedicated Vector Database

    Pinecone, Weaviate, and Qdrant offer more features, but Deeppin chose pgvector. Co-location, RLS, zero extra services, Supabase free tier — a detailed analysis of every decision factor.