v1.5.0: SearchSubagent isolated search + UpstreamManifest cross-file contracts

Architecture upgrade to improve task completion for any model size:
- SearchSubagent: independent context window, parallel file reads, clean results only
- UpstreamManifest: deterministic AST extraction of signatures/constants, zero LLM
- PENCIL-style compression: subtask results compacted to structured artifacts
- Verifier pre-check: cross-file contract consistency before running tests
- Generator context slimmed: upstream_constraints + retry_hint injection

Code quality:
- orchestrator.py run() split into 5 private methods (410->253 lines)
- Full type annotations (Optional[DebugSubagent], Callable types)
- __all__ added to 7 core modules
- pyproject.toml: license fixed, ruff + mypy configured
- TUI: 30+ event icons, Server: /api/manifest endpoint
- STATUS.md rewritten: concise, English, professional format

451/451 tests green, 27 new tests for SearchSubagent+Manifest.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Val-sss
2026-05-06 17:08:08 +08:00
parent c072a40e11
commit d5ce4342f6
15 changed files with 818 additions and 493 deletions

485
STATUS.md
View File

@@ -1,321 +1,137 @@
# KWCode 项目状态记录
# KWCode Project Status
> 项目路径:D:\program\codeagent2604\kwcode
> GitHubhttps://github.com/val1813/kwcode
> 启动日期:2026-04-26
> 目标:本地模型 coding agent通过确定性专家流水线让本地模型达到最高任务完成率
> Path: D:\program\codeagent2604\kwcode
> GitHub: https://github.com/val1813/kwcode
> Started: 2026-04-26
> Goal: Local-model coding agent — maximize task completion rate via deterministic expert pipeline
---
## 当前状态:v1.4.0 (2026-05-06)
## Current: v1.5.0 (2026-05-06)
424/424 测试全绿。v1.4.0 新增多语言AST支持 + FastAPI Server + Textual TUI + VSCode插件。
451/451 tests green. Architecture upgrade: isolated search + cross-file contracts + PENCIL compression.
### v1.4.0 新增:多语言 + TUI + IDE兼容3 个模块)
### v1.5.0 — Isolated Search + Cross-File Contracts
理论来源XRAY MCP Server(ast-grep选型) + OpenCode(client/server分离) + CodeCompass(工具采用率)
Theory: WarpGrep (isolated search subagent) + CGM (graph-injected attention) + PENCIL (erase intermediate state) + SWE-ContextBench (context quality > model size)
**模块A: 多语言AST支持**
- `ast_engine/language_detector.py`7语言检测(Python/JS/TS/Go/Rust/Java/C#),项目标记文件识别
- `ast_engine/ast_grep_engine.py`:预定义查询模板(find_function/find_class/find_imports/find_method_call)LLM只填参数不写pattern
- `ast_engine/parser.py`TreeSitterParser扩展支持JS/TS/Go/Rust/Java可选依赖graceful fallback
- `ast_engine/graph_builder.py`SUPPORTED_EXTENSIONS动态扩展 + rig.json新增language_stats
- `experts/verifier.py`:多语言测试运行器(pytest/jest/go test/cargo test/mvn test) + 多语言语法检查
- `builtin_experts/golang/SKILL.md`Go领域知识并发/错误处理/测试)
- `builtin_experts/typescript/SKILL.md`TS领域知识类型系统/React/async
- `builtin_experts/rust/SKILL.md`Rust领域知识所有权/错误处理/tokio
- `builtin_experts/java/SKILL.md`Java领域知识Spring Boot/异常/Maven
**SearchSubagent** (`experts/search_subagent.py`)
- Independent context window — search noise never enters Generator
- Parallel file reads: ThreadPoolExecutor, 8 concurrent
- Returns only precise {file, start_line, end_line, content}
- Shadow TaskContext: Locator writes to shadow, main ctx stays clean
**模块B: FastAPI Server + Textual TUI**
- `server/app.py`FastAPI + SSE事件流端口7355
- POST /api/task → 提交任务返回task_id
- GET /api/task/{id}/events → SSE事件流
- GET /api/health, /api/status, /api/files, /api/file
- POST /api/rig/refresh → 重建rig.json
- `server/pipeline_factory.py`共享pipeline构建CLI和server复用
- `server/models.py`Pydantic模型(TaskRequest/TaskResponse/HealthResponse等)
- `tui/app.py`Textual TUI左文件树+右事件流+输入框自动启动server
- CLI新增`kwcode serve`命令 + `kwcode --tui`选项
**UpstreamManifest** (`core/upstream_manifest.py`)
- Deterministic AST extraction: Python ast module, regex fallback for others
- Tracks: function signatures, constants, import dependencies
- get_constraints_for_file() → injected into Generator prompt
- check_consistency() → Verifier pre-check, catches arg count / constant mismatches
- Zero LLM calls
**模块C: VSCode插件**
- `extension/src/extension.ts`插件入口命令注册文件保存触发RIG刷新
- `extension/src/server-client.ts`SSE客户端连接localhost:7355
- `extension/src/panel.ts`Webview面板事件渲染+任务输入
- 薄客户端架构不重复实现业务逻辑所有计算在server端
**PENCIL Compression + Contract Verification**
- task_compiler: _compact_subtask_result() keeps only signatures/constants/paths/test_status
- orchestrator: locator step uses SearchSubagent, verifier pre-checks contracts
- contract_violation error type triggers re-locate retry strategy
- Generator prompt receives upstream_constraints + retry_hint
**关键设计决策**
- ast-grep pattern绝对不让LLM生成只用QUERY_TEMPLATES预定义模板
- Server单例pipeline每个任务asyncio.to_thread()隔离
- EventBus事件直接推送到SSE QueueTUI/VSCode/CLI共享同一事件流
- 所有新依赖都是optionalmultilang/server/tui不影响现有安装
**Code Quality (this release)**
- orchestrator.py run() split into 5 private methods (410→253 lines)
- Full type annotations: Optional[DebugSubagent], Callable[[str,str],None]
- __all__ added to 7 core modules
- pyproject.toml: license fixed, ruff + mypy configured
- TUI: 30+ event icons added (contract_violation, ab_test, replay, etc.)
- Server: /api/manifest endpoint, version 1.5.0
### v1.4.0 — Multi-Language + TUI + IDE
理论来源Dive into Claude Code(arXiv:2604.14228) + Wink(arXiv:2602.17037) + ARCS(arXiv:2504.20434) + SpecEyes(arXiv:2603.23483) + OPENDEV(arXiv:2603.05344) + Turn-Control(arXiv:2510.16786)
Theory: XRAY MCP Server + OpenCode + CodeCompass
**模块1: EventBus 统一事件总线** (`core/event_bus.py`)
- append-only 日志支持 replay/时间旅行调试
- on/off/emit 三个核心方法wildcard "*" 监听所有事件
- CLI 接入追加式渲染EVENT_ICONS 17种事件图标
- 7-language AST (Python/JS/TS/Go/Rust/Java/C#)
- ast-grep with QUERY_TEMPLATES (LLM never writes patterns)
- FastAPI server (port 7355) + SSE streaming
- Textual TUI (file tree + event log + input)
- VSCode extension (thin client, all logic server-side)
**模块2: ToolGateway 工具权限层** (`tools/tool_gateway.py`)
- 专家权限白名单generator只读不写verifier可写可执行
- 文件读缓存 + 脏标记(写后自动失效缓存)
- 所有工具调用通过 EventBus 可观测
### v1.3.0 — EventBus + Error Strategy + Cognitive Gate
**模块3: 错误策略路由** (`core/orchestrator.py` RETRY_STRATEGIES)
- 按 error_type 切换重试序列syntax/assertion/import/patch_apply/runtime/unknown
- import 错误:确定性修复器 `tools/import_fixer.py`不调LLM
- _build_retry_hint() 按错误类型生成精准重试提示
- _should_search() 按失败类型决定是否搜网络(不再统一 retry>=2 时搜)
Theory: Dive into Claude Code + Wink + ARCS + SpecEyes + OPENDEV + Turn-Control
**模块4: 认知门控 CognitiveGate** (`core/cognitive_gate.py`)
- patch 行数持续递减 → 边际收益递减 → 自动停止
- 连续输出相同行数 → 原地打转 → 自动停止
- 最后一次极小(≤3行) → 模型无从下手 → 自动停止
- EventBus: append-only log, replay, wildcard listeners
- ToolGateway: per-expert permissions, file cache with dirty tracking
- Error strategy routing: 6 error types → different retry sequences
- CognitiveGate: diminishing returns detection → auto-stop
- GraduatedCompactor: 3-layer progressive context compression
- Plan auto-trigger for hard tasks
- Worktree isolation (git worktree / tempdir fallback)
- Speculative Prefetch: background file pre-read
- SearchRouter: intent-aware routing (arXiv/S2/GitHub/PyPI/Open-Meteo)
- Wink self-repair: scope_creep / repetitive_fix / patch_miss / empty_output
**模块5: 上下文渐进压缩 GraduatedCompactor** (`core/context_pruner.py`)
- Layer 1 (70%): 裁剪 tool 输出冗余(>500 token 提取关键词)
- Layer 2 (85%): 复用 ContextPruner 压缩中间轮次
- Layer 3 (95%): 摘要化早期对话,只保留关键决策
### v1.2.0 — RIG Project Map
**模块6: Plan 自动触发** (`core/orchestrator.py`)
- hard 任务自动生成执行计划(不打断用户)
- 低中风险直接执行,高风险暂停确认
Theory: RIG + FastCode + CodeCompass
**模块7: Worktree 隔离** (`core/task_compiler.py` WorktreeManager)
- Git 项目git worktree 隔离
- 非 Git 项目tempdir + copytree
- cleanup() 支持 merge 回主分支
- export_rig(): full project index (exports/imports/routes/test coverage)
- upstream_summary structured dict for multi-task context passing
- ConsistencyChecker: deterministic frontend/backend API mismatch detection
- Gate/Locator prompt explicitly guided to query rig.json
**模块8: Speculative Prefetch** (`experts/locator.py`)
- Locator 完成后后台线程预读文件到内存
- 减少 Generator 阶段 IO 等待
### v1.1.0 — P0+P1+P2 Optimizations
**模块9: SearchRouter 意图感知搜索** (`search/search_router.py`)
- 零 key 默认可用arXiv / Semantic Scholar / GitHub REST / PyPI JSON / Open-Meteo
- 按意图路由research/code_solution/code_example/weather/library_doc/general
- 可选 Tavily key 提升通用搜索质量
- 错误驱动搜索接入 orchestrator
- Verifier structured output (_classify_error: 5 error types)
- Circuit breakers (syntax 1x, import immediate, same-type 3x streak)
- Gate confidence scoring (0.92/0.75/0.55)
- Experience Replay (BM25 similar trajectory lookup)
- Session continuity (SessionState, 5-turn KWCODE.md re-injection)
- Locator minimal context (function boundaries, 60-line cap, gap markers)
- Watchdog 300s timeout
- Gate accuracy stats (/stats command)
**模块10: Wink 自修复监控** (`core/wink.py`)
- scope_creep: easy任务定位>5文件 → 纠正
- repetitive_fix: 同类错误≥2次 → 换思路
- patch_miss: patch_apply失败 → 重新读文件
- empty_output: Generator无输出 → 简化任务
### v0.9.0 — DAG Compiler + Debug Subagent
**搜索层网络保护补丁**
- duckduckgo.py: DDG为主SearXNG为可选增强不自动拉起Docker
- search_augmentor.py: 全局 try/except 保护,任何网络问题返回空
- orchestrator.py: 搜索结果为空不阻塞,异常不中断重试流程
- config.yaml: search_enabled 开关(环境变量 KWCODE_SEARCH_ENABLED
- 内网/离线用户设置 false 永远不触发网络请求
- TaskCompiler: DAG scheduler, ThreadPoolExecutor + Kahn topological sort
- Debug Subagent: sys.settrace variable capture on failure
- Prompt Optimizer: trajectory → experience rules → YAML system_prompt
- Cross-Encoder search reranking
### v1.2.0 新增RIG侦察层Project Map
### Core Pipeline (v0.5.0v0.8.0)
理论来源RIG(arXiv:2601.10112) + FastCode(arXiv:2603.01012) + CodeCompass工具采用率研究
**RIG-1: export_rig() 仓库结构索引** (`ast_engine/graph_builder.py`)
- 扫描全项目Python文件提取exports/importsregex零LLM
- 检测Flask/FastAPI路由装饰器 → api_routes
- 匹配test_foo.py → foo.py → test_coverage
- 扫描.js/.ts文件检测axios/fetch调用 → frontend_api_calls
- 双文件输出:.kaiwu/rig.json完整索引+ .kaiwu/rig_summary.json精简骨架<5KB
- 精简骨架动态截断文件列表保证注入Gate/Locator不爆context
**RIG-2: upstream_summary结构化** (`core/context.py` + `core/task_compiler.py`)
- upstream_summary从str改为dict: {modified_files, diffs, new_symbols, broken_interfaces}
- TaskCompiler自动提取上游patch的文件/diff/新符号,结构化传递给下游子任务
- 新增_format_upstream_text()将结构化dict转为LLM可读文本注入prompt
- 下游子任务Locator可直接读取"哪些文件被改了",不靠模型猜
**RIG-3: ConsistencyChecker前后端一致性检查** (`experts/consistency_checker.py`)
- 基于rig.json做确定性集合对比不调LLM
- 输出backend_only/frontend_only/matched不一致清单
- check_with_details()带文件位置信息,可直接作为子任务输入
- format_for_subtask()生成可注入Generator的文本
**RIG-4: Gate/Locator prompt显式引导查rig** (`core/gate.py` + `experts/locator.py`)
- Gate prompt新增显式引导"优先参考.kaiwu/rig.json理解项目结构"
- Locator prompt新增{rig_context}占位符
- _load_rig_context()读取rig_summary.json不是完整rig.json注入路由/前端调用/测试覆盖摘要
- 解决CodeCompass发现的工具采用率42%问题模型不查图的根因是prompt没引导
### v1.1.0 新增P0+P1+P2 全量优化
**P2-1: Watchdog任务超时** (`core/orchestrator.py`)
- threading.Timer 300秒硬超时卡死任务自动终止
- 重试循环每轮检查watchdog状态
**P2-2: Gate路由准确率统计** (`stats/value_tracker.py` + `cli/main.py`)
- `get_gate_accuracy()` 按expert_type统计成功率/平均耗时/平均重试
- 新增 `/stats` CLI命令展示30天统计报告
**其他**
- 新增 CONTRIBUTING.md架构红线、PR标准、贡献类型指南
- README 贡献章节重写(精简+链接CONTRIBUTING.md
- 删除 vision_expert.py.bakvision_expert保留为辅助输入通道
- Gate → 6 pipeline routes (locator_repair/codegen/refactor/doc/office/chat)
- BM25+AST call graph two-phase location (zero LLM, milliseconds)
- Generator: original from file, LLM only generates modified
- Verifier: syntax check + pytest
- 3-stage retry + Reflection root cause analysis
- 5 deterministic tools (read_file/write_file/run_bash/list_dir/git)
- KWCODE.md project rules + /plan + Checkpoint + DocReader
- Model capability tiers (SMALL/MEDIUM/LARGE)
- Expert flywheel (trajectory → pattern → backtest → AB test → production)
- 3-layer memory (PROJECT.md/EXPERT.md/PATTERN.md)
- Office document generation (Excel/PPT/Word)
- MCP Router, context compression, CJK BM25
---
#### P0+P1 优化7文件 +362行
## Test Summary
**P0-1: Verifier结构化输出** (`experts/verifier.py`)
- `_classify_error()` 纯正则提取 error_type/error_file/error_line/error_message/failed_tests
- 5种错误类型syntax/assertion/import/runtime/patch_apply
- 所有错误路径统一返回结构化字段DebugSubagent不再需要自己解析
**P0-2: 熔断器+缩小scope** (`core/orchestrator.py`)
- syntax错误1次后直接熔断重试无意义
- import错误立即熔断+提示安装依赖
- 同类error_type连续3次→硬熔断
- 第2次失败自动缩小scope到第一个文件+函数
- 低置信度(<0.6)任务自动减少重试预算
**P0-3: Gate置信度输出** (`core/gate.py`)
- `_estimate_confidence()` 关键词信号强度评分0.92/0.75/0.55三档)
- 不覆盖expert_registry已有的confidence
- orchestrator消费低置信度减少max_retries
**P1-1: Experience Replay** (`flywheel/trajectory_collector.py` + `core/orchestrator.py` + `core/context.py`)
- `find_similar()` BM25检索历史成功轨迹复用已有rank-bm25依赖
- orchestrator.run()开头自动调用结果存入ctx.similar_trajectories
- 飞轮闭环:同类任务不走冷启动
**P1-2: Session内多轮连贯** (`cli/main.py`)
- SessionState类跟踪tasks/files_touched/turn_count
- `to_reminder()` 生成System Reminder注入Gate memory_context
- 每5轮重新注入KWCODE.md核心规则注意力衰减对抗
**P1-3: Locator最小上下文裁剪** (`experts/locator.py`)
- 函数边界识别indent-baseddef到下一个同级def
- 去掉纯注释行docstring限3行
- 60行/函数上限gap marker标记不连续区域
- 文件路径header + 行号前缀
### v0.9.0 新增
**DAG 任务编译器TaskCompiler**
- `core/task_compiler.py`:轻量 DAG 调度器ThreadPoolExecutor + Kahn 拓扑排序
- 支持串行(依赖链)和并行(独立任务)混合执行
- 依赖上下文注入:前置任务结果自动追加到后续任务输入
- 零新依赖12 个测试覆盖(串行/并行/菱形DAG/环检测)
**Debug Subagent运行时调试子代理**
- `experts/debug_subagent.py`:基于 Debug2Fix 论文Microsoft 2026
- verifier 失败后用 sys.settrace 非侵入式捕获目标行变量值
- LLM 决定调试策略(断点位置+变量列表fallback 到 pytest --tb=long
- 调试结果注入 generator retry prompt让重试拿到真实运行时数据
- 15 个测试覆盖
**Prompt Optimizer飞轮优化 YAML system_prompt**
- `flywheel/prompt_optimizer.py`:分析成功轨迹 → Opus/Sonnet API 生成经验规则
- 规则追加到专家 YAML 的 system_prompt`## 经验规则(自动生成)`
- 替代已删除的 Python 代码优化器,方向正确:优化知识而非代码
**Reflexion 持久化**(保留)
- REFLECTION.md 结构化写入 + /plan 风险提示注入
**Cross-Encoder 搜索重排**(保留)
- BM25 后追加 Cross-Encoder 精排FLEX-2 自动降级
### v0.8.0 已移除(方向错误)
- ~~ExpertBase 继承体系~~Python专家把知识和执行逻辑混在一起
- ~~BugFixExpert.py~~(应该是 YAML 知识载体,不是 Python 类)
- ~~SelfImprovingOptimizer~~(优化 Python 代码 → 改为优化 YAML prompt
- ~~Registry Python 专家加载~~(回退到纯 YAML
### 已完成功能清单
**MVP 核心流水线**
- Gate LLM分类 → 6种流水线路由locator_repair/codegen/refactor/doc/office/chat
- BM25+AST调用图两阶段定位零LLM毫秒级
- Generator 从文件读originalLLM只生成modified
- Verifier 语法检查 + pytest 自动验证
- 三阶段重试(正常→从错误出发→最小化修改)+ Reflection根因分析
- 5个确定性工具read_file/write_file/run_bash/list_dir/git
**P1 四大功能v0.5.0**
- KWCODE.md 项目规则文件(分段加载+按任务类型注入+token上限15%
- /plan 计划模式 + 三档风险评估High/Medium/Low基于历史失败记录
- Checkpoint 文件快照git stash主路径+文件复制兜底+失败自动还原+降级建议)
- 非代码文件读取PDF/Word/MD/TXT + BM25Plus段落匹配 + Locator自动注入
**P2 三大功能v0.6.0**
- 模型能力自适应SMALL/MEDIUM/LARGE三档策略自动检测小模型强制plan
- 飞轮可见性通知专家投产Panel+积累进度+里程碑,不打断任务)
- 价值量化仪表盘SQLite本地统计+kwcode stats命令+启动周报)
**搜索模块重构v0.6.1**
- 四级内容提取管道trafilatura→newspaper→readabilipy→soup质量评分选最佳
- SearXNG + DDG 并行搜索ThreadPoolExecutor + URL去重合并
- kwcode setup-search 一键安装SearXNG
**意图感知搜索v0.6.2**
- 意图分类器增强5类意图+关键词扩充+LLM fallback语义分类
- ChatExpert搜索门控follow-up/推理不搜索,实时数据始终搜索)
- QueryGenerator按意图生成更精准搜索词
- BM25Plus搜索结果重排
**UI全面优化v0.7.0**
- 删掉所有机器内部信息logger只写~/.kwcode/kwcode.logwarnings静默
- 执行过程spinner动画rich.progresstransient=True完成后消失
- 完成后用户友好结果摘要(修改文件+改动bullet+测试结果)
- 重影大字KAIWU Header + 状态栏深色背景
- kwcode setup-search 一键安装SearXNG搜索引擎
**其他**
- 专家注册表15个预置专家YAML + .kwx导入导出
- 专家飞轮三道门轨迹→模式检测→回测→AB测试→投产
- 3层记忆系统PROJECT.md/EXPERT.md/PATTERN.md
- Office文档生成Excel/PPT/Word
- MCP Routerkwcode serve-mcp
- 上下文压缩(纯算法,头尾保留+中间关键词提取,<10ms
- 中文分词BM25DocReader CJK tokenizer
- /api 命令(临时/永久切换任意OpenAI兼容API
### E2E 验收结果2026-04-28gemma3:4b
- P1: KWCODE.md注入✓ /plan风险评估✓ Checkpoint还原✓ DocReader注入✓ (8/8)
- P2: 模型自适应(4b→SMALL)✓ 飞轮通知✓ ValueTracker✓ (6/6)
- 集成: Gate分类✓ Chat流水线✓ Codegen流水线(2.9s)✓ (3/3)
### 测试统计
| 类别 | 数量 | 状态 |
|------|------|------|
| 核心单元测试 | 38 | PASS |
| 回归测试 | 173 | PASS |
| P1 功能测试 | 33 | PASS |
| P2 功能测试 | 21 | PASS |
| 搜索重构测试 | 19 | PASS |
| 意图搜索测试 | 19 | PASS |
| E2E 真实模型 | 17 | PASS |
| RIG模块测试 | 29 | PASS |
| TaskCompiler测试 | 12 | PASS |
| 多语言模块测试 | 51 | PASS |
| Server/TUI测试 | 16 | PASS |
| **合计** | **424** | **全绿** |
### 待做
1. SQLite 跨 session 查询spec §7.1 kaiwu.db
2. 12 个预置专家完整 benchmark目前只跑了 BugFix+TestGen
3. 实时数据API提示注入codegen涉及天气/股价时prompt注入免费API信息避免模型编造假数据
4. ~~多语言AST支持~~ ✅ v1.4.0 已完成
5. pip publish 到 PyPI
6. install.ps1 / install.sh 一键安装脚本
### 已知问题
- qwen3-vl:8b 所有输出在thinking字段content为空已加thinking提取
- reasoning模型Gate调用慢8x multiplier
- SearXNG需要Docker Desktop无Docker时降级到DDG
- codegen生成网页时模型可能用Math.random()假数据需实时数据API提示注入
| Category | Count | Status |
|----------|-------|--------|
| Core unit tests | 38 | PASS |
| Regression tests | 173 | PASS |
| P1 feature tests | 33 | PASS |
| P2 feature tests | 21 | PASS |
| Search refactor | 19 | PASS |
| Intent search | 19 | PASS |
| E2E real model | 17 | PASS |
| RIG modules | 29 | PASS |
| TaskCompiler | 12 | PASS |
| Multi-language | 51 | PASS |
| Server/TUI | 16 | PASS |
| SearchSubagent+Manifest | 27 | PASS |
| **Total** | **451** | **All green** |
---
## 文件结构
## File Structure
```
kwcode/
@@ -324,49 +140,64 @@ kwcode/
├── STATUS.md
└── kaiwu/
├── cli/
│ ├── main.py # REPL + EventBus追加式渲染 + spinner + 结果摘要
│ ├── status_bar.py # 状态栏(4档自适应) + TokPerSecEstimator
│ └── onboarding.py # 首次启动引导
│ ├── main.py # REPL + EventBus rendering + spinner + summary
│ ├── status_bar.py # Status bar (4-tier adaptive)
│ └── onboarding.py # First-run onboarding
├── core/
│ ├── event_bus.py # [v1.3] 统一事件总线(append-only日志+replay)
│ ├── cognitive_gate.py # [v1.3] 认知门控(patch行数递减检测)
│ ├── wink.py # [v1.3] Wink自修复监控(偏离检测+纠正注入)
│ ├── gate.py # LLM任务分类 → 专家知识叠加
│ ├── orchestrator.py # 确定性流水线 + 错误策略路由 + Plan自动触发
│ ├── context.py # TaskContext数据类
│ ├── task_compiler.py # DAG调度器 + WorktreeManager隔离
│ ├── planner.py # /plan计划模式 + 风险评估
│ ├── checkpoint.py # 文件快照(git stash/文件复制)
│ ├── kwcode_md.py # KWCODE.md分段加载+注入
│ ├── model_capability.py # 模型三档自适应(SMALL/MEDIUM/LARGE)
│ ├── context_pruner.py # 上下文压缩 + GraduatedCompactor 3层渐进压缩
│ ├── network.py # 网络探测+代理配置
── sysinfo.py # 系统信息+VRAM监控
│ ├── event_bus.py # Unified event bus (append-only + replay)
│ ├── cognitive_gate.py # Diminishing returns detection
│ ├── wink.py # Self-repair monitor
│ ├── gate.py # LLM task classification → expert routing
│ ├── orchestrator.py # Deterministic pipeline + error strategy routing
│ ├── context.py # TaskContext dataclass
│ ├── task_compiler.py # DAG scheduler + WorktreeManager
│ ├── upstream_manifest.py # [v1.5] Cross-file contract tracking (zero LLM)
│ ├── planner.py # /plan mode + risk assessment
│ ├── checkpoint.py # File snapshot (git stash / file copy)
│ ├── kwcode_md.py # KWCODE.md segmented loading
│ ├── model_capability.py # Model tier detection (SMALL/MEDIUM/LARGE)
│ ├── context_pruner.py # Context compression + GraduatedCompactor
── network.py # Network detection + proxy config
│ └── sysinfo.py # System info + VRAM monitoring
├── experts/
│ ├── locator.py # BM25+调用图定位 + DocReader注入 + Speculative Prefetch
│ ├── generator.py # 代码生成(original从文件读LLM只写modified)
│ ├── verifier.py # 语法检查 + pytest
│ ├── search_augmentor.py # 搜索增强 + BM25重排 + 网络保护
│ ├── consistency_checker.py # 前后端接口一致性检查(确定性不调LLM)
│ ├── chat_expert.py # 聊天(搜索门控follow-up/推理不搜)
── office_handler.py # Office文档生成
├── search/
├── search_router.py # [v1.3] 意图感知搜索路由(arxiv/S2/GitHub/PyPI/Open-Meteo)
│ ├── duckduckgo.py # DDG主+SearXNG可选 + search_enabled开关
│ ├── extraction_pipeline.py # 四级内容提取
│ ├── intent_classifier.py # 意图感知(5类+LLM fallback)
│ ├── query_generator.py # 按意图生成搜索词
│ ├── content_fetcher.py # 薄封装→extraction_pipeline
│ └── quality_filter.py # 域名黑白名单
├── knowledge/doc_reader.py # PDF/Word/MD读取 + CJK分词BM25
├── flywheel/ # 轨迹→模式→生成→AB测试→投产
├── registry/ # 专家注册表 + .kwx打包
├── notification/ # 飞轮通知(expert_born/progress/milestone)
├── stats/ # 价值量化(SQLite)
├── memory/ # 三层记忆(PROJECT/EXPERT/PATTERN)
├── ast_engine/ # tree-sitter AST + 调用图(SQLite)
│ ├── locator.py # BM25+graph location + DocReader + Prefetch
│ ├── search_subagent.py # [v1.5] Isolated search (independent context)
│ ├── generator.py # Code generation (original from file, LLM writes modified)
│ ├── verifier.py # Syntax + pytest + cross-file contract check
│ ├── search_augmentor.py # Search augmentation + BM25 rerank
│ ├── consistency_checker.py # Frontend/backend API consistency (deterministic)
── chat_expert.py # Chat (search gating)
│ └── office_handler.py # Office document generation
├── search/ # Intent-aware search routing
├── knowledge/ # PDF/Word/MD reader + CJK BM25
├── flywheel/ # Trajectory → pattern → generation → AB test
├── registry/ # Expert registry + .kwx packaging
├── notification/ # Flywheel notifications
├── stats/ # Value tracking (SQLite)
├── memory/ # 3-layer memory system
├── ast_engine/ # tree-sitter AST + call graph
├── server/ # FastAPI + SSE (port 7355)
├── tui/ # Textual TUI
├── mcp/ # MCP Router
├── llm/ # Ollama + llama.cpp双后端
├── tools/ # 5个确定性工具 + ToolGateway + import_fixer
└── tests/ # 357个测试
├── llm/ # Ollama + llama.cpp backends
├── tools/ # 5 deterministic tools + ToolGateway
└── tests/ # 451 tests
```
---
## TODO
1. CLI refactor: split main.py (1861 lines) into cli/commands/
2. Comments: unify to English across all modules
3. Expert-level EventBus emit (file reads, function locations, test results)
4. SQLite cross-session queries
5. Full expert benchmark (12 presets)
6. pip publish to PyPI
7. install.ps1 / install.sh one-click install
## Known Issues
- qwen3-vl:8b outputs in thinking field, content empty (thinking extraction added)
- Reasoning models slow on Gate (8x multiplier)
- SearXNG requires Docker Desktop, degrades to DDG without it

View File

@@ -6,6 +6,8 @@ Each expert reads from and writes to specific fields only.
from dataclasses import dataclass, field
from typing import Optional
__all__ = ["TaskContext"]
@dataclass
class TaskContext:
@@ -71,3 +73,9 @@ class TaskContext:
# Experience Replay: similar successful trajectories from history
similar_trajectories: list = field(default_factory=list)
# SearchSubagent: cross-file constraints injected into Generator prompt
upstream_constraints: str = ""
# Retry hint: error-specific guidance injected into Generator prompt
retry_hint: str = ""

View File

@@ -15,6 +15,8 @@ import logging
logger = logging.getLogger(__name__)
__all__ = ["EventBus"]
class EventBus:
"""

View File

@@ -16,6 +16,8 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
__all__ = ["Gate"]
GATE_SYSTEM = "你是任务分类器。只返回JSON不要有其他内容。"
GATE_PROMPT = """分析用户输入返回分类JSON。

View File

@@ -7,7 +7,11 @@ RED-5: Max 3 retries, hardcoded.
import logging
import time
import threading
from typing import Optional
from typing import Callable, Optional, TYPE_CHECKING
if TYPE_CHECKING:
from kaiwu.experts.debug_subagent import DebugSubagent
from kaiwu.experts.vision_expert import VisionExpert
from kaiwu.core.context import TaskContext
from kaiwu.core.event_bus import EventBus
@@ -24,6 +28,8 @@ from kaiwu.memory.kaiwu_md import KaiwuMemory
from kaiwu.registry.expert_registry import ExpertRegistry
from kaiwu.tools.executor import ToolExecutor
from kaiwu.flywheel.trajectory_collector import TrajectoryCollector
__all__ = ["PipelineOrchestrator"]
from kaiwu.flywheel.pattern_detector import PatternDetector
from kaiwu.flywheel.ab_tester import ABTester
from kaiwu.core.checkpoint import Checkpoint
@@ -97,13 +103,13 @@ class PipelineOrchestrator:
office_handler: OfficeHandlerExpert,
tool_executor: ToolExecutor,
memory: KaiwuMemory,
registry: ExpertRegistry | None = None,
trajectory_collector: TrajectoryCollector | None = None,
ab_tester: ABTester | None = None,
chat_expert: ChatExpert | None = None,
debug_subagent=None,
vision_expert=None,
bus: EventBus | None = None,
registry: Optional[ExpertRegistry] = None,
trajectory_collector: Optional[TrajectoryCollector] = None,
ab_tester: Optional[ABTester] = None,
chat_expert: Optional[ChatExpert] = None,
debug_subagent: Optional["DebugSubagent"] = None,
vision_expert: Optional["VisionExpert"] = None,
bus: Optional[EventBus] = None,
):
self.locator = locator
self.generator = generator
@@ -124,9 +130,7 @@ class PipelineOrchestrator:
self.bus = bus or EventBus()
self._wink = WinkMonitor()
self._cognitive_gate = CognitiveGate()
# SearchSubagent: isolated context window for code search
self._search_subagent = SearchSubagent(locator, tool_executor)
# UpstreamManifest: cross-file contract tracking (per-task lifecycle)
self._manifest = UpstreamManifest()
def run(
@@ -134,11 +138,11 @@ class PipelineOrchestrator:
user_input: str,
gate_result: dict,
project_root: str,
on_status=None,
on_status: "Optional[Callable[[str, str], None]]" = None,
no_search: bool = False,
skip_checkpoint: bool = False,
pre_search_results: str = "",
image_paths: list = None,
image_paths: Optional[list[str]] = None,
) -> dict:
"""
Execute the expert pipeline.
@@ -169,6 +173,9 @@ class PipelineOrchestrator:
expert_system_prompt=gate_result.get("system_prompt", ""),
)
# Reset manifest for each new top-level task
self._manifest.clear()
# 处理图片路径
if image_paths:
ctx.image_paths = list(image_paths)
@@ -194,78 +201,13 @@ class PipelineOrchestrator:
else:
ctx.expert_system_prompt = kwcode_rules
# chat类型直接回复不走AB测试/搜索/重试
if expert_type == "chat":
self._emit(on_status, "chat", "聊天模式")
if self.chat_expert:
result = self.chat_expert.run(ctx)
else:
ctx.generator_output = {"explanation": "我是KWCode专注于代码任务。", "patches": []}
result = {"passed": True}
elapsed = time.time() - start_time
return {
"success": True,
"context": ctx,
"error": None,
"elapsed": elapsed,
}
# chat/vision类型早期返回
simple_result = self._handle_simple_type(ctx, expert_type, start_time, on_status)
if simple_result is not None:
return simple_result
# vision类型图片处理任务
if expert_type == "vision":
self._emit(on_status, "vision", "图片处理模式")
if self.vision_expert and ctx.image_paths:
result = self.vision_expert.run(ctx)
explanation = result.get("output", "").strip()
ctx.generator_output = {
"explanation": explanation,
"patches": [],
"metadata": {"vision": result.get("metadata", {})},
}
elapsed = time.time() - start_time
success = result.get("success", False)
return {
"success": success,
"context": ctx,
"error": None if success else explanation or "图片处理失败",
"elapsed": elapsed,
}
else:
ctx.generator_output = {"explanation": "图片处理功能需要配置Vision专家", "patches": []}
elapsed = time.time() - start_time
return {
"success": False,
"context": ctx,
"error": "Vision专家未配置或未提供图片",
"elapsed": elapsed,
}
# Gate 3: AB test — check if a candidate expert should be used for this task
ab_candidate_name = None
ab_used_new = False
if self.ab_tester and expert_type != "chat":
candidate_def = self.ab_tester.should_use_candidate(expert_type)
if candidate_def:
ab_candidate_name = candidate_def["name"]
ab_used_new = True
# Override gate_result to use the candidate expert's pipeline
gate_result = {
**gate_result,
"expert_name": ab_candidate_name,
"route_type": "expert_registry",
"pipeline": candidate_def.get("pipeline", []),
"system_prompt": candidate_def.get("system_prompt", ""),
}
self._emit(on_status, "ab_test", f"AB测试使用候选专家 {ab_candidate_name}")
else:
# Check if any candidate is in AB testing for this type (baseline run)
for name, info in self.ab_tester._candidates.items():
if (info["status"] == "ab_testing"
and info["expert_def"].get("type") == expert_type
and len(info["ab_results"]) < 10):
ab_candidate_name = name
ab_used_new = False
self._emit(on_status, "ab_test", f"AB测试基线对照候选 {name}")
break
# Gate 3: AB test
ab_candidate_name, ab_used_new, gate_result = self._setup_ab_test(gate_result, expert_type, on_status)
# Use custom pipeline from expert registry if available, else default
if gate_result.get("route_type") == "expert_registry" and "pipeline" in gate_result:
@@ -275,49 +217,8 @@ class PipelineOrchestrator:
self._emit(on_status, "gate", f"任务类型:{expert_type} | 难度:{gate_result.get('difficulty', '?')}")
# ── Experience Replay: find similar successful trajectories ──
if self.trajectory_collector and expert_type not in ("chat", "office", "vision"):
try:
similar = self.trajectory_collector.find_similar(user_input, expert_type, k=3)
if similar:
ctx.similar_trajectories = similar
best = similar[0]
self._emit(on_status, "replay",
f"发现相似成功案例:{best.get('user_input', '')[:40]}")
except Exception as e:
logger.debug("Experience replay failed (non-blocking): %s", e)
# codegen任务如果涉及实时数据首次就触发搜索不等失败重试
if expert_type == "codegen" and not no_search and self._needs_realtime_data(user_input):
try:
self._emit(on_status, "search", "检测到实时数据需求,预搜索...")
results = self.search_augmentor.search(ctx)
if results:
ctx.search_results = results
ctx.search_triggered = True
self._emit(on_status, "search_done", f"搜索完成,注入{len(results)}字参考信息")
except Exception as e:
logger.debug("Pre-search failed (网络保护,不阻塞): %s", e)
# ── Plan 自动触发hard 任务自动生成计划(不打断用户)──
if (gate_result.get("difficulty") == "hard"
and expert_type not in ("chat", "office", "vision")
and not ctx.subtask_results):
try:
from kaiwu.core.planner import Planner
from kaiwu.memory import pattern_md
planner = Planner(
locator=self.locator,
pattern_md_module=pattern_md,
llm=self.generator.llm,
)
plan = planner.generate_plan_steps(user_input, gate_result, project_root)
if plan and len(plan) > 1:
ctx.execution_plan = plan
self._emit(on_status, "plan_generated", f"自动生成 {len(plan)} 步计划")
self.bus.emit("plan_generated", {"steps": len(plan), "msg": f"生成 {len(plan)} 步计划"})
except Exception as e:
logger.debug("Auto-plan failed (non-blocking): %s", e)
# Experience replay + pre-search + plan
self._prepare_context(ctx, gate_result, expert_type, user_input, project_root, no_search, on_status)
# ── Checkpoint: snapshot before execution (skip in multi-task to avoid race) ──
checkpoint = Checkpoint(project_root)
@@ -354,33 +255,9 @@ class PipelineOrchestrator:
if success:
elapsed = time.time() - start_time
checkpoint.discard() # Clean up snapshot on success
# Reviewer: 需求对齐审查(非阻塞,不影响成功判定)
review_result = self._do_review(ctx, on_status)
# Save to memory on success (with elapsed for expert/pattern tracking)
self.memory.save(project_root, ctx, elapsed=elapsed)
# Update expert registry stats
expert_name = gate_result.get("expert_name")
if expert_name and self.registry:
self.registry.update_stats(expert_name, success=True, latency=elapsed)
# Flywheel: record trajectory and detect patterns (non-blocking)
self._record_trajectory(ctx, True, elapsed, on_status)
# Gate 3: record AB test result if this task is part of an AB test
self._record_ab_result(ab_candidate_name, ab_used_new, True, elapsed, on_status)
# P2: Value tracking (local SQLite)
self._record_value(project_root, gate_result, True, elapsed, ctx)
# P2: Milestone check
self._check_milestone(on_status)
# Reflexion持久化成功时也记录注意事项
self._persist_reflection(project_root, ctx, gate_result, success=True)
return {
"success": True,
"context": ctx,
"error": None,
"elapsed": elapsed,
}
return self._record_success(ctx, project_root, gate_result,
ab_candidate_name, ab_used_new, elapsed,
checkpoint, on_status)
ctx.retry_count += 1
error_detail = ""
@@ -506,6 +383,173 @@ class PipelineOrchestrator:
_watchdog.cancel() # Clean up watchdog timer
elapsed = time.time() - start_time
return self._record_failure_result(ctx, project_root, gate_result,
ab_candidate_name, ab_used_new, max_retries,
elapsed, checkpoint, checkpoint_saved, on_status)
def _handle_simple_type(self, ctx: TaskContext, expert_type: str, start_time: float, on_status) -> Optional[dict]:
"""Handle chat and vision early returns. Returns result dict or None to continue."""
# chat类型直接回复不走AB测试/搜索/重试
if expert_type == "chat":
self._emit(on_status, "chat", "聊天模式")
if self.chat_expert:
result = self.chat_expert.run(ctx)
else:
ctx.generator_output = {"explanation": "我是KWCode专注于代码任务。", "patches": []}
result = {"passed": True}
elapsed = time.time() - start_time
return {
"success": True,
"context": ctx,
"error": None,
"elapsed": elapsed,
}
# vision类型图片处理任务
if expert_type == "vision":
self._emit(on_status, "vision", "图片处理模式")
if self.vision_expert and ctx.image_paths:
result = self.vision_expert.run(ctx)
explanation = result.get("output", "").strip()
ctx.generator_output = {
"explanation": explanation,
"patches": [],
"metadata": {"vision": result.get("metadata", {})},
}
elapsed = time.time() - start_time
success = result.get("success", False)
return {
"success": success,
"context": ctx,
"error": None if success else explanation or "图片处理失败",
"elapsed": elapsed,
}
else:
ctx.generator_output = {"explanation": "图片处理功能需要配置Vision专家", "patches": []}
elapsed = time.time() - start_time
return {
"success": False,
"context": ctx,
"error": "Vision专家未配置或未提供图片",
"elapsed": elapsed,
}
return None
def _setup_ab_test(self, gate_result: dict, expert_type: str, on_status) -> tuple:
"""Setup AB test. Returns (ab_candidate_name, ab_used_new, gate_result)."""
ab_candidate_name = None
ab_used_new = False
if self.ab_tester and expert_type != "chat":
candidate_def = self.ab_tester.should_use_candidate(expert_type)
if candidate_def:
ab_candidate_name = candidate_def["name"]
ab_used_new = True
# Override gate_result to use the candidate expert's pipeline
gate_result = {
**gate_result,
"expert_name": ab_candidate_name,
"route_type": "expert_registry",
"pipeline": candidate_def.get("pipeline", []),
"system_prompt": candidate_def.get("system_prompt", ""),
}
self._emit(on_status, "ab_test", f"AB测试使用候选专家 {ab_candidate_name}")
else:
# Check if any candidate is in AB testing for this type (baseline run)
for name, info in self.ab_tester._candidates.items():
if (info["status"] == "ab_testing"
and info["expert_def"].get("type") == expert_type
and len(info["ab_results"]) < 10):
ab_candidate_name = name
ab_used_new = False
self._emit(on_status, "ab_test", f"AB测试基线对照候选 {name}")
break
return (ab_candidate_name, ab_used_new, gate_result)
def _prepare_context(self, ctx: TaskContext, gate_result: dict, expert_type: str,
user_input: str, project_root: str, no_search: bool, on_status) -> None:
"""Experience replay + pre-search + plan generation."""
# ── Experience Replay: find similar successful trajectories ──
if self.trajectory_collector and expert_type not in ("chat", "office", "vision"):
try:
similar = self.trajectory_collector.find_similar(user_input, expert_type, k=3)
if similar:
ctx.similar_trajectories = similar
best = similar[0]
self._emit(on_status, "replay",
f"发现相似成功案例:{best.get('user_input', '')[:40]}")
except Exception as e:
logger.debug("Experience replay failed (non-blocking): %s", e)
# codegen任务如果涉及实时数据首次就触发搜索不等失败重试
if expert_type == "codegen" and not no_search and self._needs_realtime_data(user_input):
try:
self._emit(on_status, "search", "检测到实时数据需求,预搜索...")
results = self.search_augmentor.search(ctx)
if results:
ctx.search_results = results
ctx.search_triggered = True
self._emit(on_status, "search_done", f"搜索完成,注入{len(results)}字参考信息")
except Exception as e:
logger.debug("Pre-search failed (网络保护,不阻塞): %s", e)
# ── Plan 自动触发hard 任务自动生成计划(不打断用户)──
if (gate_result.get("difficulty") == "hard"
and expert_type not in ("chat", "office", "vision")
and not ctx.subtask_results):
try:
from kaiwu.core.planner import Planner
from kaiwu.memory import pattern_md
planner = Planner(
locator=self.locator,
pattern_md_module=pattern_md,
llm=self.generator.llm,
)
plan = planner.generate_plan_steps(user_input, gate_result, project_root)
if plan and len(plan) > 1:
ctx.execution_plan = plan
self._emit(on_status, "plan_generated", f"自动生成 {len(plan)} 步计划")
self.bus.emit("plan_generated", {"steps": len(plan), "msg": f"生成 {len(plan)} 步计划"})
except Exception as e:
logger.debug("Auto-plan failed (non-blocking): %s", e)
def _record_success(self, ctx: TaskContext, project_root: str, gate_result: dict,
ab_candidate_name, ab_used_new: bool, elapsed: float,
checkpoint, on_status) -> dict:
"""Record success: memory, registry, trajectory, AB, value, milestone, reflection."""
checkpoint.discard() # Clean up snapshot on success
# Reviewer: 需求对齐审查(非阻塞,不影响成功判定)
review_result = self._do_review(ctx, on_status)
# Save to memory on success (with elapsed for expert/pattern tracking)
self.memory.save(project_root, ctx, elapsed=elapsed)
# Update expert registry stats
expert_name = gate_result.get("expert_name")
if expert_name and self.registry:
self.registry.update_stats(expert_name, success=True, latency=elapsed)
# Flywheel: record trajectory and detect patterns (non-blocking)
self._record_trajectory(ctx, True, elapsed, on_status)
# Gate 3: record AB test result if this task is part of an AB test
self._record_ab_result(ab_candidate_name, ab_used_new, True, elapsed, on_status)
# P2: Value tracking (local SQLite)
self._record_value(project_root, gate_result, True, elapsed, ctx)
# P2: Milestone check
self._check_milestone(on_status)
# Reflexion持久化成功时也记录注意事项
self._persist_reflection(project_root, ctx, gate_result, success=True)
return {
"success": True,
"context": ctx,
"error": None,
"elapsed": elapsed,
}
def _record_failure_result(self, ctx: TaskContext, project_root: str, gate_result: dict,
ab_candidate_name, ab_used_new: bool, max_retries: int,
elapsed: float, checkpoint, checkpoint_saved: bool,
on_status) -> dict:
"""Record failure: checkpoint restore, memory, registry, trajectory, AB, value, reflection."""
# ── Checkpoint: restore on failure ──
if checkpoint_saved:
restored = checkpoint.restore()
@@ -557,10 +601,11 @@ class PipelineOrchestrator:
ctx.relevant_code_snippets = search_result["code_snippets"]
# Inject upstream constraints for Generator
if search_result.get("upstream_constraints"):
ctx._upstream_constraints = search_result["upstream_constraints"]
ctx.upstream_constraints = search_result["upstream_constraints"]
files = search_result["relevant_files"]
funcs = search_result["relevant_functions"]
self._emit(on_status, "locator_done", f"文件:{', '.join(files[:3])} | 函数:{', '.join(funcs[:3])}")
func_str = ', '.join(funcs[:3]) if funcs else "(文件级修改)"
self._emit(on_status, "locator_done", f"文件:{', '.join(files[:3])} | 函数:{func_str}")
elif step == "generator":
self._emit(on_status, "generator", "生成patch...")

View File

@@ -20,6 +20,8 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
__all__ = ["TaskCompiler", "WorktreeManager", "CycleError"]
MAX_PARALLEL_WORKERS = 4

View File

@@ -16,6 +16,8 @@ from typing import Optional
logger = logging.getLogger(__name__)
__all__ = ["UpstreamManifest"]
class UpstreamManifest:
"""

View File

@@ -318,7 +318,7 @@ class GeneratorExpert:
prompt = self._build_retry_prompt(ctx, fpath, original, task_desc, search_ctx)
# Inject upstream constraints from SearchSubagent (cross-file contracts)
upstream_constraints = getattr(ctx, '_upstream_constraints', "")
upstream_constraints = ctx.upstream_constraints
if upstream_constraints:
prompt += f"\n\n## 跨文件契约(必须遵守)\n{upstream_constraints}"
@@ -327,9 +327,8 @@ class GeneratorExpert:
prompt += f"\n\n## 相关文档参考\n{ctx.doc_context[:800]}"
# Inject retry_hint if available
retry_hint = getattr(ctx, 'retry_hint', "")
if retry_hint:
prompt += f"\n\n## 重试提示\n{retry_hint}"
if ctx.retry_hint:
prompt += f"\n\n## 重试提示\n{ctx.retry_hint}"
system = self._build_system(ctx)

View File

@@ -26,6 +26,8 @@ from kaiwu.tools.executor import ToolExecutor
logger = logging.getLogger(__name__)
__all__ = ["SearchSubagent", "SearchResult"]
# Max parallel file reads per batch
MAX_PARALLEL_READS = 8
# Max lines per snippet returned to Generator

View File

@@ -34,13 +34,13 @@ def create_app(
from kaiwu.server.models import (
TaskRequest, TaskResponse, HealthResponse,
StatusResponse, FileContent,
StatusResponse, FileContent, ManifestResponse,
)
from kaiwu.server.pipeline_factory import build_pipeline
app = FastAPI(
title="KwCode Server",
version="1.3.0",
version="1.5.0",
description="KwCode coding agent HTTP API with SSE streaming",
)
@@ -78,7 +78,7 @@ def create_app(
async def health():
return HealthResponse(
status="ok",
version="1.3.0",
version="1.5.0",
model=ollama_model or "local",
project_root=project_root,
)
@@ -286,4 +286,16 @@ def create_app(
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@app.get("/api/manifest", response_model=ManifestResponse)
async def get_manifest():
"""Get current UpstreamManifest state (cross-file contracts)."""
manifest = orchestrator._manifest
sigs = manifest.get_all_signatures()
consts = manifest.get_all_constants()
return ManifestResponse(
signatures=sigs,
constants=consts,
file_count=len(sigs),
)
return app

View File

@@ -33,7 +33,7 @@ class TaskResult(BaseModel):
class HealthResponse(BaseModel):
"""Health check response."""
status: str = "ok"
version: str = "1.3.0"
version: str = "1.5.0"
model: str = ""
project_root: str = ""
@@ -61,3 +61,10 @@ class FileContent(BaseModel):
content: str
language: str = ""
lines: int = 0
class ManifestResponse(BaseModel):
"""UpstreamManifest state response."""
signatures: dict[str, dict[str, str]] = Field(default_factory=dict)
constants: dict[str, dict[str, str]] = Field(default_factory=dict)
file_count: int = 0

View File

@@ -0,0 +1,363 @@
"""
Tests for SearchSubagent and UpstreamManifest.
Covers: isolated context, parallel reads, manifest extraction, contract checking.
"""
import pytest
from unittest.mock import MagicMock, patch
from kaiwu.core.upstream_manifest import UpstreamManifest
from kaiwu.experts.search_subagent import SearchSubagent, SearchResult
from kaiwu.core.context import TaskContext
# ══════════════════════════════════════════════════════════════════════
# UpstreamManifest Tests
# ══════════════════════════════════════════════════════════════════════
class TestUpstreamManifest:
def setup_method(self):
self.manifest = UpstreamManifest()
def test_extract_python_signatures(self):
patches = [{
"file": "utils.py",
"original": "",
"modified": "def reset_password(token: str, new_pass: str) -> bool:\n return True\n",
}]
self.manifest.update(patches)
sigs = self.manifest.get_all_signatures()
assert "utils.py" in sigs
assert "reset_password" in sigs["utils.py"]
assert "token: str" in sigs["utils.py"]["reset_password"]
assert "-> bool" in sigs["utils.py"]["reset_password"]
def test_extract_constants(self):
patches = [{
"file": "config.py",
"original": "",
"modified": "API_BASE = 'https://api.example.com/v2'\nMAX_RETRIES = 3\n",
}]
self.manifest.update(patches)
consts = self.manifest.get_all_constants()
assert "config.py" in consts
assert "API_BASE" in consts["config.py"]
assert "MAX_RETRIES" in consts["config.py"]
def test_extract_imports(self):
patches = [{
"file": "views.py",
"original": "",
"modified": "from kaiwu.core.context import TaskContext\nimport os\n\ndef handler():\n pass\n",
}]
self.manifest.update(patches)
# Should track dependency
assert "views.py" in self.manifest._imports
assert any("TaskContext" in imp for imp in self.manifest._imports["views.py"])
def test_get_constraints_for_file(self):
# First, register upstream signatures
patches_upstream = [{
"file": "kaiwu/core/context.py",
"original": "",
"modified": "class TaskContext:\n def __init__(self, user_input: str = ''):\n self.user_input = user_input\n",
}]
self.manifest.update(patches_upstream)
# Then register a file that imports from upstream
patches_downstream = [{
"file": "views.py",
"original": "",
"modified": "from kaiwu.core.context import TaskContext\n\ndef handler():\n ctx = TaskContext(user_input='test')\n",
}]
self.manifest.update(patches_downstream)
constraints = self.manifest.get_constraints_for_file("views.py")
# Should mention the upstream contract
assert "kaiwu/core/context.py" in constraints or constraints == ""
def test_check_consistency_no_violations(self):
# Register a function signature
self.manifest._signatures["utils.py"] = {
"process": "def process(data: list, mode: str)"
}
self.manifest._dependency_graph["main.py"] = ["utils.py"]
# Code that calls process correctly
code = "result = process(my_data, 'fast')\n"
violations = self.manifest.check_consistency("main.py", code)
assert violations == []
def test_check_consistency_too_many_args(self):
# Register: process takes 2 params (data, mode)
self.manifest._signatures["utils.py"] = {
"process": "def process(data: list, mode: str)"
}
self.manifest._dependency_graph["main.py"] = ["utils.py"]
# Code that calls process with 3 args
code = "result = process(my_data, 'fast', True)\n"
violations = self.manifest.check_consistency("main.py", code)
assert len(violations) == 1
assert "3" in violations[0] and "2" in violations[0]
def test_check_consistency_constant_redefinition(self):
self.manifest._constants["config.py"] = {"API_BASE": "'https://api.example.com'"}
self.manifest._dependency_graph["main.py"] = ["config.py"]
# Code redefines constant with different value
code = "API_BASE = 'https://wrong-url.com'\nprint(API_BASE)\n"
violations = self.manifest.check_consistency("main.py", code)
assert len(violations) >= 1
assert "API_BASE" in violations[0]
def test_to_compact_summary(self):
patches = [{
"file": "a.py",
"original": "",
"modified": "MAX_SIZE = 100\ndef foo(x: int) -> str:\n return str(x)\n",
}]
self.manifest.update(patches)
summary = self.manifest.to_compact_summary()
assert "signatures" in summary
assert "constants" in summary
assert "imports" in summary
def test_clear(self):
self.manifest._signatures["a.py"] = {"foo": "def foo()"}
self.manifest.clear()
assert self.manifest._signatures == {}
assert self.manifest._constants == {}
def test_async_function_extraction(self):
patches = [{
"file": "api.py",
"original": "",
"modified": "async def fetch_data(url: str, timeout: int = 30) -> dict:\n return {}\n",
}]
self.manifest.update(patches)
sigs = self.manifest.get_all_signatures()
assert "fetch_data" in sigs["api.py"]
assert "async def" in sigs["api.py"]["fetch_data"]
def test_regex_fallback_for_non_python(self):
patches = [{
"file": "main.go",
"original": "",
"modified": "func ProcessData(input []byte, mode string) error {\n return nil\n}\n\nMAX_BUFFER = 4096\n",
}]
self.manifest.update(patches)
sigs = self.manifest.get_all_signatures()
assert "main.go" in sigs
assert "ProcessData" in sigs["main.go"]
consts = self.manifest.get_all_constants()
assert "MAX_BUFFER" in consts.get("main.go", {})
def test_multiple_patches_accumulate(self):
self.manifest.update([{
"file": "a.py",
"original": "",
"modified": "def foo() -> int:\n return 1\n",
}])
self.manifest.update([{
"file": "b.py",
"original": "",
"modified": "def bar(x: str) -> str:\n return x\n",
}])
sigs = self.manifest.get_all_signatures()
assert "foo" in sigs["a.py"]
assert "bar" in sigs["b.py"]
def test_count_params_with_self(self):
"""self/cls should not count as parameters."""
count = UpstreamManifest._count_params("def method(self, x: int, y: int)")
assert count == 2
def test_count_params_with_kwargs(self):
"""*args/**kwargs means we can't determine exact count."""
count = UpstreamManifest._count_params("def func(x, *args, **kwargs)")
assert count is None
def test_count_args_nested(self):
"""Nested function calls shouldn't split on inner commas."""
count = UpstreamManifest._count_args("foo(1, 2), bar(3)")
assert count == 2 # Two top-level args
# ══════════════════════════════════════════════════════════════════════
# SearchSubagent Tests
# ══════════════════════════════════════════════════════════════════════
class TestSearchSubagent:
def setup_method(self):
self.mock_locator = MagicMock()
self.mock_tools = MagicMock()
self.subagent = SearchSubagent(self.mock_locator, self.mock_tools)
def test_search_returns_clean_results(self):
"""SearchSubagent should return structured results, not raw locator state."""
self.mock_locator.run.return_value = {
"relevant_files": ["src/main.py", "src/utils.py"],
"relevant_functions": ["process", "validate"],
"edit_locations": ["src/main.py:L10-20"],
"method": "bm25_graph",
}
self.mock_tools.read_file.return_value = "def process(data):\n return data\n"
ctx = TaskContext(user_input="fix the process function", project_root="/tmp/proj")
result = self.subagent.search(ctx)
assert result["relevant_files"] == ["src/main.py", "src/utils.py"]
assert result["relevant_functions"] == ["process", "validate"]
assert result["method"] == "bm25_graph"
assert "code_snippets" in result
def test_search_does_not_modify_original_ctx(self):
"""SearchSubagent must not pollute the original TaskContext."""
self.mock_locator.run.return_value = {
"relevant_files": ["a.py"],
"relevant_functions": ["foo"],
"edit_locations": [],
"method": "bm25_graph",
}
self.mock_tools.read_file.return_value = "def foo():\n pass\n"
ctx = TaskContext(user_input="test", project_root="/tmp")
# These should remain None after search
assert ctx.locator_output is None
self.subagent.search(ctx)
# Original ctx should NOT be modified by search
assert ctx.locator_output is None
def test_search_with_manifest_constraints(self):
"""Should include upstream constraints when manifest is provided."""
self.mock_locator.run.return_value = {
"relevant_files": ["views.py"],
"relevant_functions": ["handler"],
"edit_locations": [],
"method": "llm_fallback",
}
self.mock_tools.read_file.return_value = "def handler():\n pass\n"
manifest = UpstreamManifest()
manifest._signatures["utils.py"] = {"process": "def process(data: list) -> dict"}
manifest._dependency_graph["views.py"] = ["utils.py"]
ctx = TaskContext(user_input="fix handler", project_root="/tmp")
result = self.subagent.search(ctx, manifest)
assert "upstream_constraints" in result
assert "process" in result["upstream_constraints"]
def test_search_locator_returns_nothing(self):
"""Should return empty results gracefully when locator finds nothing."""
self.mock_locator.run.return_value = None
ctx = TaskContext(user_input="fix something", project_root="/tmp")
result = self.subagent.search(ctx)
assert result["relevant_files"] == []
assert result["code_snippets"] == {}
assert result["method"] == "none"
def test_parallel_read_handles_errors(self):
"""File read errors should not crash the subagent."""
self.mock_locator.run.return_value = {
"relevant_files": ["good.py", "bad.py"],
"relevant_functions": ["foo"],
"edit_locations": [],
"method": "bm25_graph",
}
def side_effect(path):
if "bad" in path:
return "[ERROR] File not found"
return "def foo():\n return 42\n"
self.mock_tools.read_file.side_effect = side_effect
ctx = TaskContext(user_input="test", project_root="/tmp")
result = self.subagent.search(ctx)
# Should have snippet for good.py but not bad.py
assert "good.py" in result["code_snippets"]
assert "bad.py" not in result["code_snippets"]
def test_extract_precise_snippet_with_functions(self):
content = """import os
def helper():
return 1
def target_func(x, y):
result = x + y
if result > 10:
return result * 2
return result
def another():
pass
"""
snippet = self.subagent._extract_precise_snippet(
content, ["target_func"], "test.py"
)
assert "target_func" in snippet
assert "x + y" in snippet
# Should NOT include unrelated functions in full
assert "def another" not in snippet or "..." in snippet
def test_extract_precise_snippet_no_functions(self):
"""When no functions specified, return first N lines."""
content = "line1\nline2\nline3\n"
snippet = self.subagent._extract_precise_snippet(content, [], "test.py")
assert "line1" in snippet
def test_merge_ranges(self):
ranges = [(1, 10), (8, 20), (25, 30)]
merged = SearchSubagent._merge_ranges(ranges)
assert merged == [(1, 20), (25, 30)]
def test_merge_ranges_with_gap(self):
"""Ranges within 3 lines should merge."""
ranges = [(1, 10), (12, 20)] # gap of 2
merged = SearchSubagent._merge_ranges(ranges)
assert merged == [(1, 20)]
def test_find_function_range(self):
lines = [
"import os",
"",
"def foo(x):",
" return x + 1",
"",
"def bar():",
" pass",
]
start, end = SearchSubagent._find_function_range(lines, "foo")
assert start == 2
assert end == 5 # up to but not including def bar
def test_find_function_range_not_found(self):
lines = ["def other():", " pass"]
start, end = SearchSubagent._find_function_range(lines, "nonexistent")
assert start == -1
assert end == -1
class TestSearchResult:
def test_to_dict(self):
r = SearchResult(
file="src/main.py",
start_line=10,
end_line=25,
content="def foo():\n pass",
function_name="foo",
)
d = r.to_dict()
assert d["file"] == "src/main.py"
assert d["start_line"] == 10
assert d["end_line"] == 25
assert d["function_name"] == "foo"

View File

@@ -57,7 +57,7 @@ class TestServerHealth:
assert resp.status_code == 200
data = resp.json()
assert data["status"] == "ok"
assert data["version"] == "1.3.0"
assert data["version"] == "1.5.0"
assert data["model"] == "test-model"
asyncio.run(_test())
@@ -222,7 +222,7 @@ class TestServerModels:
from kaiwu.server.models import HealthResponse
resp = HealthResponse(model="qwen3-8b", project_root="/tmp")
assert resp.status == "ok"
assert resp.version == "1.3.0"
assert resp.version == "1.5.0"
def test_file_content(self):
from kaiwu.server.models import FileContent

View File

@@ -51,6 +51,42 @@ EVENT_ICONS = {
"task_completed": "",
"task_error": "",
"keepalive": "",
# v1.5: SearchSubagent + UpstreamManifest events
"contract_violation": "⚠️",
"locator": "🔍",
"locator_done": "🔍",
"locator_fail": "🔍",
"generator": "⚙️",
"generator_done": "⚙️",
"generator_fail": "⚙️",
"verifier": "✔️",
"verifier_done": "✔️",
"verifier_fail": "✔️",
"reflection": "💭",
"debug": "🐛",
"debug_done": "🐛",
"replay": "📼",
"ab_test": "🧪",
"ab_test_record": "🧪",
"ab_graduated": "🎓",
"ab_archived": "📦",
"flywheel": "",
"checkpoint": "💾",
"import_fix": "🔧",
"office": "📄",
"office_done": "📄",
"office_fail": "📄",
"search": "🌐",
"search_done": "🌐",
"suggest": "💡",
"warning": "⚠️",
"watchdog": "⏱️",
"low_confidence": "⚠️",
"review": "👁️",
"review_done": "👁️",
"review_gap": "⚠️",
"chat": "💬",
"vision": "👁️",
}
DEFAULT_SERVER_URL = "http://127.0.0.1:7355"

View File

@@ -8,7 +8,7 @@ version = "1.4.0"
description = "KwCode - Local-model coding agent with MoE expert pipeline"
requires-python = ">=3.10"
readme = "README.md"
license = "MIT"
license = {text = "MIT"}
keywords = ["coding-agent", "local-llm", "ollama", "code-generation"]
classifiers = [
"Development Status :: 4 - Beta",
@@ -81,3 +81,17 @@ include = ["kaiwu*"]
[tool.setuptools.package-data]
"kaiwu.builtin_experts" = ["*.yaml"]
[tool.ruff]
line-length = 100
target-version = "py311"
[tool.ruff.lint]
select = ["E", "F", "W", "I"]
ignore = ["E501"]
[tool.mypy]
python_version = "3.11"
warn_return_any = true
warn_unused_configs = true
ignore_missing_imports = true