AI Engineer. I build multi-agent LLM systems for real enterprise users, and harden the AWS credentials, IAM boundaries, and multi-tenant authorization they run on. Two independent studies on cost-aware routing and failure attribution, code and data public. AI Engineer。我為真實企業用戶建構多代理 LLM 系統,並負責它們跑在 AWS 上的憑證、IAM 邊界與多租戶授權。兩項 LLM 系統的對照實驗獨立完成,程式碼與資料公開。
Systems running in a real firm, serving real staff, with real consequences when they break. Numbers, not adjectives.跑在真實事務所、服務真實員工、壞掉會有真實後果的系統。用數字說話。
These systems run inside an accounting firm, where a wrong answer costs a client and a leaked credential costs the firm. That constraint shaped how I build: route to the cheapest model that is actually good enough, fail closed rather than open, and measure the trade-off instead of asserting it.這些系統跑在會計事務所內部——一個錯誤答案的代價是客戶,一組外洩憑證的代價是整間公司。這個限制決定了我的做法:路由到真的夠好的最便宜模型、失敗時fail closed 而非 fail open、用量測證明取捨而不是用直覺斷言。
A three-mode router (local mock / on-prem Qwen / cloud Claude) delegating across 3 specialized agents and 11 MCP tools, with dual interfaces: conversational Chat and a form-based Quick UI. Keyword rules alone answer ~70% of queries in mock mode — a standalone third backend, which is also what keeps the whole flow end-to-end testable at zero API cost. Measured 80% task accuracy on Chat vs 60% on the Quick UI — the opposite of what I designed for.三模式路由(本地 mock / 地端 Qwen / 雲端 Claude)調度 3 個專責 Agent 與 11 個 MCP 工具,搭配雙介面:對話式 Chat 與表單式 Quick UI。規則式 mock 模式單靠關鍵字就答對 ~70% 查詢——它是獨立的第三種後端,也讓整條流程能在零 API 成本下端對端測試。實測 Chat 準確率 80%、Quick UI 只有 60%——跟我當初的設計賭注相反。
A deployable RAG template (LangGraph + Ollama + ChromaDB) built so the next client deployment is configuration, not a rewrite. The initial embedding model had no discriminative power on Chinese queries — everything scored nearly the same. Swapping to snowflake-arctic-embed2 restored retrieval quality. 14 unit tests + 6 end-to-end tests, with auth, CORS, and rate limiting wired in. Skeleton complete; first real-client trial pending.一套可重複部署的 RAG 模板(LangGraph + Ollama + ChromaDB),目標是讓下一次客戶部署是「改設定」而不是「重寫」。初期 embedding 模型對中文查詢完全沒有區分度——所有結果分數幾乎相同。改用 snowflake-arctic-embed2 後檢索品質恢復。含 14 個單元測試 + 6 個端對端測試,Auth/CORS/rate limit 已接。骨架完成,首個真實客戶測試待進行。
EGcloud runs the firm's billing and petty-cash workflow in production. Security baseline: IAM role + instance profile (no static keys), CORS narrowed from wildcard to an explicit allowlist, JWT auth with a multi-tenant multi-role model across 12 business tables, and API rate limiting. Then a real report: a lowest-tier user couldn't complete a reimbursement — 403. The petty-cash page writes to 4 tables, but 3 of them were dependent tables absent from INPUT_STORES, so an admin literally could not grant them.EGcloud 是事務所帳務與零用金流程的正式上線系統。安全基線:IAM role + instance profile(無靜態金鑰)、CORS 由萬用字元收斂為白名單、JWT 認證搭配涵蓋 12 張業務表的多租戶多角色模型、API rate limiting。接著來了真實回報:最低權限的使用者按不了「報銷完成」——403。零用金頁面實際會寫 4 張表,其中 3 張附屬表不在 INPUT_STORES 裡,管理員在畫面上根本勾不到。
The billing service (FastAPI + PostgreSQL on AWS) started rejecting every login. Root cause: RDS managed credentials rotate every 7 days, but the connection pool resolved them once at startup — and the container had been running for four straight weeks.帳務系統(FastAPI + PostgreSQL on AWS)登入全數回 500。根因:RDS 託管密碼每 7 天輪換,但連線池僅在啟動時解析一次——而容器已連續運行四週。
OperationalError, rebuild and retry) over the faster fix of disabling rotation — that one would have required changing the production master password, which is the higher-risk operation.關鍵取捨:我選了應用層自癒(每次建池重抓 Secrets Manager 憑證、捕捉 OperationalError 後自動重建重試),而不是「關掉輪換」這個更快的解——後者需要變更生產環境主密碼,風險更高。No percentage bars. Core means I've shipped production code with it and can debug it under pressure. Working means I've used it in a real project and would need a day to get back up to speed.不用百分比進度條。Core 表示我用它交付過正式環境的程式碼,出事時能自己除錯。Working 表示我在真實專案用過,但要一天時間重新熟悉。
The security row isn't a pivot. It's the part of the job I was already doing — hardening credentials and authorization on systems I also built — that I simply hadn't written down anywhere public until now.安全那一欄不是轉行。那本來就是我工作的一部分——在自己建的系統上做憑證與授權加固——只是一直沒有公開寫出來。
Systems I designed, built, and shipped.我設計、建構並交付的系統。
3-stage cascade router (Keyword → Embedding → LLM) tested on CLINC150 with 3 seeds. R4 Hybrid matches full-LLM accuracy (82.6% vs 82.9%, McNemar p > 0.3) while cutting 74% of LLM calls. Total experiment cost: $0.44. ACL workshop paper ready for arXiv.四階段級聯路由器(Keyword → Embedding → LLM → Hybrid),在 CLINC150 上以 3 組種子驗證。R4 Hybrid 準確率與 Full-LLM 無顯著差異(82.6% vs 82.9%,McNemar p > 0.3),同時減少 74% 的 LLM 呼叫。實驗總成本 $0.44。ACL workshop 論文已準備上 arXiv。
3 agents, 11 MCP tools, 3 orchestration modes. Pilot (n=6): Chat 80% accuracy / 6.6s avg vs. Quick UI 60% / 5.9s. The counterintuitive signal that structured input doesn't always win motivated a deeper investigation into Intent Density.3 個 Agent、11 個 MCP 工具、3 種編排模式。先導測試 (n=6):Chat 80% 準確率 / 平均 6.6s vs. Quick UI 60% / 5.9s。結構化輸入不一定更好的反直覺訊號,促使我深入研究意圖密度。
AI-powered decision system with a 7-member "parliament" of competing analyst personas. 6-agent pipeline (macro → screening → analysis → parliament → verdict), ~10K LOC across 13 Python modules. FastAPI dashboard with SSE streaming, Notion API integration for automated logging.AI 驅動的決策系統,核心是 7 位「議員」角色的對抗式辯論。6-agent 流水線(總經→選股→分析→議會→裁決),13 個 Python 模組共 ~10K LOC。FastAPI 儀表板搭配 SSE 串流,整合 Notion API 自動化紀錄。
56 commits, 14K LOC, 28 API endpoints. Multi-tenant platform with row-level security. Replaced a 4-hour Excel workflow, now completes in 10 minutes.56 次 commit、14K 行、28 個 API。多租戶平台搭配 row-level security。取代 4 小時的 Excel 流程,現在 10 分鐘完成。
~100 commits, 4 modules. Equity CTE traversal, Odoo ERP integration, work logs, case tracking. Used daily by 50+ staff.約 100 次 commit、4 個模組。股權 CTE 遍歷、Odoo ERP 整合、工作日誌、案件追蹤。每天 50+ 位同仁使用。
One-click POS→ERP pipeline. Reduced per-client monthly data entry from ~3 hours to ~5 minutes.一鍵 POS→ERP 流水線。每月客戶資料輸入從 3 小時縮至 5 分鐘。
End-to-end report automation. Cut monthly close from 2 days to ~30 minutes per client.端到端報表自動化。月結從 2 天縮短至約 30 分鐘。
Cross-conversation memory for AI assistants. Hot Cache pattern loads full context in ~170 tokens.AI 跨對話記憶技能。Hot Cache 模式用 ~170 tokens 載入全局。
First full-stack project at UTSA. React + Spring Boot with JWT auth and Docker deployment.在 UTSA 的第一個全端專案。React + Spring Boot,含 JWT 認證與 Docker 部署。
I started building things with my hands: 10 years of competitive robotics across WRO, FLL, FRC, VEX, and APRA, always as part of a team. In FRC we had 20+ members splitting into mechanical, electrical, programming, and strategy sub-teams; in WRO my team of three earned a WRO World Championship representing the USA. Those years taught me that orchestration matters more than any single component. A mediocre robot with excellent sub-system coordination beats a brilliant one with poor integration.我從動手做東西開始:橫跨 WRO、FLL、FRC、VEX、APRA 的十年機器人競賽經歷,每一場都是團隊作戰。FRC 隊伍 20+ 人分成機構、電控、程式、策略小組;WRO 三人小隊拿下代表美國出賽的WRO 世界賽。這些年教會我:協調比單一零件重要。子系統配合好的普通機器人,會贏過整合差的天才機器人。
The turning point was CMU's Robotics Feiyue Program in 2019. Walking through the Gates Center for Computer Science, seeing labs where robots learned from experience rather than following fixed rules, I realized the next frontier wasn't mechanical; it was intelligence. That's when I decided to study CS at UTSA, shifting from hardware systems to software, from robots to AI.轉折點是 2019 年的 CMU Robotics 飛躍計劃。走進 Gates 電腦科學大樓,看到實驗室裡的機器人不是按照固定規則運作,而是從經驗中學習,我意識到下一個前沿不是機械,而是智慧。那時我決定到 UTSA 讀資工,從硬體系統轉向軟體,從機器人轉向 AI。
At an accounting firm, I got to answer that question with real stakes. I designed and shipped a multi-agent LLM platform with hybrid task routing, dual-interface design, and RAG knowledge bases, serving 50+ daily users handling real financial data. Along the way I built 4 production systems from scratch.在會計師事務所,我得以在真實場景中回答這個問題。我設計並交付了一套多 Agent LLM 平台,包含混合任務路由、雙介面設計與 RAG 知識庫,每天服務50+ 位使用者處理真實財務資料。過程中從零打造了 4 套上線系統。
But building exposed gaps that engineering alone can't close. My mock router handles 70% of queries through keywords, but where exactly does the remaining 30% fail, and why does Claude succeed where rules don't? I tuned a 5-iteration agent loop by instinct, but I want to know the principled way to set that threshold. I can make agents work, but I want to understand why. These are the questions I'm actively working on, and the reason I started running controlled experiments alongside production work.但建構的過程暴露了工程手段無法填補的缺口。我的 mock 路由器靠關鍵字處理了 70% 的查詢,但剩下 30% 到底在哪裡失敗?為什麼 Claude 能在規則做不到的地方成功?我靠直覺調了 5 輪的 agent 迴圈上限,但我想知道設定這個閾值的理論依據。我能讓 Agent 運作,但我想理解為什麼。這些問題驅動我在生產工作之餘,開始進行系統性的對照實驗。
Paper readings, system reflections, and thoughts on where LLM agents are heading.論文閱讀、系統反思、以及對 LLM Agent 發展方向的想法。
I tested four routing strategies on CLINC150 across 3 random seeds (n=1,200 pooled LLM calls). A keyword→embedding→LLM cascade matches full-LLM accuracy (82.6% ± 1.2pp vs 82.9% ± 0.6pp, McNemar not significant in 3/3 seeds) while calling the LLM on only 26% of queries — a 74% LLM cost reduction with no accuracy loss.我在 CLINC150 上跨 3 個 random seed(總 pooled n=1,200)測試了四種路由策略。keyword→embedding→LLM 的 cascade 與全量 LLM 路由準確率相同(82.6% ± 1.2pp vs 82.9% ± 0.6pp,McNemar 在 3/3 seeds 皆 not significant),但只對 26% 查詢呼叫 LLM — LLM 成本降低 74%,準確率無損。
A meta-analysis of 100+ papers finds CoT mostly helps on math only. What this means for agent routing costs.一篇後設分析涵蓋 100+ 篇論文,發現 CoT 主要只在數學上有用。這對 agent 路由成本的意義。
Six-layer architecture, fail-closed tool design, three-tier memory, five-level compression, KAIROS daemon mode, and anti-distillation: a deep technical read.六層架構、fail-closed 工具設計、三層記憶、五級壓縮、KAIROS 守護程式模式、反蒸餾:一次深度技術解讀。
Agents benchmarked on real workplace tasks. Best model: 24%. Why that's both damning and expected.在真實工作任務上測試 agent。最佳模型:24%。為什麼這既令人失望又在預期中。
Adding a validator agent to Tree-of-Thought. The pattern matters more than the 5.6% gain.在思維樹上加了驗證 agent。這個模式比 5.6% 的增益更重要。
Two interfaces, same MCP tools, 6-task pilot. The UI was 39% faster on ambiguous queries.兩個介面、相同 MCP 工具、6 個任務先導測試。UI 在模糊查詢上快了 39%。
E2E audio models degrade on multi-turn dialogue. But is modularity inherently better, or just a crutch for weaker models?E2E 語音模型在多輪對話中退化。但模組化是天生更好,還是只是弱模型的暫時拐杖?
CoT-injected dialogue strategies work for sales. But does the approach generalize to domains with wider strategy trees?CoT 注入的對話策略在銷售中有效。但這個方法能泛化到策略樹更寬的領域嗎?
How a 100-commit internal platform with Odoo JSON-RPC integration led me to design a multi-agent orchestrator.一個 100 次提交的內部平台如何帶領我走向多 Agent 編排器的設計。
My orchestrator supports mock, local Qwen2.5:7b, and cloud Claude Sonnet. The latency gap is 5x. Here's the design.我的編排器支援 mock、本地 Qwen2.5:7b 和雲端 Claude Sonnet。延遲差距是 5 倍。以下是設計過程。
22 tax/accounting skills, a pure JSON RAG pipeline, and Claude Agent SDK. How I built a knowledge system without ML models.22 個稅務會計 skill、純 JSON 的 RAG pipeline、Claude Agent SDK。我如何不用 ML 模型就建構知識系統。
Microsoft's conversational multi-agent framework vs. my centralized orchestrator. Same problem, opposite design choices.微軟的對話式多 Agent 框架 vs. 我的集中式編排器。相同問題,相反的設計選擇。
I built a ReAct loop without knowing it had a name. Comparing my mock vs. Claude gap to the paper's ablation studies.我建了一個 ReAct 循環卻不知道它有名字。將我的 mock vs. Claude 差距與論文的消融研究做比較。
Interested in collaborating on LLM agent systems research. Open to discussion and feedback on my work.歡迎 LLM Agent 系統研究方面的合作邀請。歡迎交流與對我作品的回饋。