mirror of
https://github.com/val1813/kwcode.git
synced 2026-09-03 06:34:30 +08:00
feat: v1.7.0 KAIJU架构借鉴 — bounded context + sub-task decomposition + DetailedLogger
三个核心改动： 1. Generator bounded context: 只传当前函数代码+相关failing tests，不传完整历史 2. 存根任务sub-task decomposition: 多pass函数逐个独立实现，每个函数独立context 3. DetailedLogger完整流水线日志: 不截断LLM输入输出，记录各节点IO和工程决策 附带修复： - OpenAI兼容API检测: localhost非标准端口探测/api/tags判断是否Ollama - LLM Backend on_call钩子: 每次调用自动触发回调记录 513 tests passed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Val-sss
26
CHANGELOG.md
26
CHANGELOG.md
@@ -4,6 +4,32 @@ All notable changes to KWCode are documented here.
|
||||
|
||||
---
|
||||
|
||||
## [1.7.0] - 2026-05-07
|
||||
|
||||
### KAIJU架构借鉴 + DetailedLogger + Bounded Context
|
||||
|
||||
**核心理念**：借鉴KAIJU三个具体机制——bounded context per node、sub-task decomposition、完整流水线日志。
|
||||
|
||||
### Added
|
||||
|
||||
- **DetailedLogger完整流水线日志**（`kaiwu/audit/detailed_logger.py`）：每个任务生成独立JSON日志到`logs/`目录，记录LLM完整prompt/output（不截断）、各节点输入输出、工程决策（重试/熔断/搜索）。环境变量`KWCODE_DETAIL_LOG_DIR`可配置输出目录
|
||||
- **LLM Backend on_call钩子**：每次LLM调用自动触发回调，记录完整messages和response到DetailedLogger
|
||||
- **存根任务sub-task decomposition**（`_run_stub_decomposed`）：多个pass函数不再一次性让LLM实现，而是逐函数独立调用，每个函数独立context+独立失败筛选，一个函数失败不影响其他
|
||||
- **`_find_stub_functions`**：精确检测文件中的stub函数（pass/.../ return None/raise NotImplementedError）
|
||||
- **`_filter_relevant_failures`**：按函数名/文件名筛选相关测试失败，只给LLM看它需要的信息
|
||||
|
||||
### Changed
|
||||
|
||||
- **Generator bounded context**：`_generate_modified`不再注入全部8条structured_failures和完整retry_hint，而是只传与当前函数相关的失败（通过函数名/文件名匹配筛选），retry_hint截断到300字符
|
||||
- **OpenAI兼容API检测修复**：localhost非标准端口（如kaiwu部署器11435）现在通过探测`/api/tags`判断是否Ollama，不再错误地走`/api/chat`导致404
|
||||
- **Gate LLM调用记录**：`_llm_minimal_classify`中增加debug日志记录prompt和raw output
|
||||
|
||||
### Fixed
|
||||
|
||||
- **kaiwu部署器（llama.cpp server）兼容**：`_detect_openai_compat`对`127.0.0.1:11435`正确返回True（OpenAI兼容），不再当作Ollama
|
||||
|
||||
---
|
||||
|
||||
## [1.6.2] - 2026-05-07
|
||||
|
||||
### 执行反馈深度升级 + 存根任务修复
|
||||
|
||||
@@ -10,13 +10,13 @@
|
||||
[](https://python.org)
|
||||
[]()
|
||||
[](https://github.com/val1813/kwcode/actions/workflows/test.yml)
|
||||
[]()
|
||||
[]()
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
> **v1.6.2 已发布！** 执行反馈深度升级：结构化测试失败解析 + TraceCoder历史教训累积 + whole_file写入修复。安装命令：
|
||||
> **v1.7.0 已发布！** KAIJU架构借鉴：Generator bounded context + 存根任务sub-task decomposition + DetailedLogger完整流水线日志。安装命令：
|
||||
>
|
||||
> ```bash
|
||||
> pip install kwcode
|
||||
@@ -30,6 +30,7 @@
|
||||
|
||||
| 日期 | 内容 |
|
||||
|------|------|
|
||||
| 05-07 | **v1.7.0** KAIJU架构借鉴：Generator bounded context(只传当前函数+相关测试) + 存根任务sub-task decomposition(逐函数独立实现) + DetailedLogger完整流水线日志(不截断LLM输入输出) + OpenAI兼容API检测修复(kaiwu部署器兼容) |
|
||||
| 05-07 | **v1.6.2** 执行反馈深度升级：结构化测试失败解析(parse_test_failures) + TraceCoder历史教训累积(attempt_history) + whole_file写入修复(存根任务不再patches=0) + 完整审计日志(llm_calls/node_io) + pytest -v详细输出 |
|
||||
| 05-07 | **v1.6.1** 架构收敛：删除WholeFileImplExpert/DependencyFixExpert，纯确定性机制驱动pipeline。Generator增强(upstream_constraints注入system prompt + retry_hint携带上次代码 + tier=small填空框架)。License改为Apache-2.0。513 tests green |
|
||||
| 05-07 | **v1.6.0** MoE确定性架构：GapDetector(11种GapType,零LLM) + ExecutionStateTracker(回归检测) + EnvProber(工具链/依赖自动修复) + Gate确定性优先路由 + 63个专项诊断测试 |
|
||||
|
||||
@@ -10,8 +10,7 @@
|
||||
[](https://python.org)
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
|
||||
[]()
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
184
kaiwu/audit/detailed_logger.py
Normal file
184
kaiwu/audit/detailed_logger.py
Normal file
@@ -0,0 +1,184 @@
|
||||
"""
|
||||
DetailedLogger: 完整不截断的流水线日志记录器。
|
||||
|
||||
每个任务生成一个 JSON 文件，记录：
|
||||
- LLM 完整 prompt/output（不截断）
|
||||
- 各节点（Gate/Locator/Generator/Verifier）的输入输出
|
||||
- 工程机制决策（重试策略、搜索、熔断等）
|
||||
|
||||
输出目录：环境变量 KWCODE_DETAIL_LOG_DIR，默认为项目 logs/ 目录。
|
||||
设为空字符串可禁用。
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 默认日志目录（相对于源码根目录，开发调试用）
|
||||
_DEFAULT_LOG_DIR = Path(__file__).resolve().parent.parent.parent / "logs"
|
||||
|
||||
|
||||
def _get_log_dir() -> Optional[Path]:
|
||||
"""获取详细日志输出目录。返回 None 表示禁用。"""
|
||||
env_val = os.environ.get("KWCODE_DETAIL_LOG_DIR")
|
||||
if env_val is not None:
|
||||
if env_val == "":
|
||||
return None # 显式禁用
|
||||
return Path(env_val)
|
||||
return _DEFAULT_LOG_DIR
|
||||
|
||||
|
||||
class DetailedLogger:
|
||||
"""单任务详细日志记录器。非阻塞，所有操作失败静默。"""
|
||||
|
||||
def __init__(self, user_input: str = "", model: str = "unknown"):
|
||||
self._enabled = True
|
||||
self._log_dir = _get_log_dir()
|
||||
if self._log_dir is None:
|
||||
self._enabled = False
|
||||
return
|
||||
|
||||
self._start_time = time.time()
|
||||
self._task_id = datetime.now().strftime("%Y-%m-%d_%H%M%S")
|
||||
self._user_input = user_input
|
||||
self._model = model
|
||||
self._timeline: list[dict] = []
|
||||
self._metadata: dict = {}
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self._enabled
|
||||
|
||||
def set_metadata(self, **kwargs):
|
||||
"""设置任务级元数据（expert_type, difficulty, routing_source 等）。"""
|
||||
if not self._enabled:
|
||||
return
|
||||
self._metadata.update(kwargs)
|
||||
|
||||
def log_llm(self, caller: str, prompt: str, system: str,
|
||||
raw_output: str, tokens: Optional[dict] = None,
|
||||
elapsed_ms: float = 0, messages: Optional[list] = None):
|
||||
"""
|
||||
记录一次 LLM 调用（完整不截断）。
|
||||
|
||||
Args:
|
||||
caller: 调用方标识（gate/generator/verifier/reflection 等）
|
||||
prompt: 完整 prompt 文本（非 chat 模式时）
|
||||
system: system prompt
|
||||
raw_output: LLM 原始输出
|
||||
tokens: {"input": N, "output": N}
|
||||
elapsed_ms: 调用耗时毫秒
|
||||
messages: chat 模式的完整 messages 列表
|
||||
"""
|
||||
if not self._enabled:
|
||||
return
|
||||
try:
|
||||
entry = {
|
||||
"time": datetime.now().strftime("%H:%M:%S.%f")[:-3],
|
||||
"elapsed_s": round(time.time() - self._start_time, 2),
|
||||
"type": "llm_call",
|
||||
"caller": caller,
|
||||
"input": {},
|
||||
"output": raw_output,
|
||||
"elapsed_ms": round(elapsed_ms, 1),
|
||||
"tokens": tokens or {},
|
||||
}
|
||||
if messages:
|
||||
entry["input"]["messages"] = messages
|
||||
else:
|
||||
entry["input"]["system"] = system
|
||||
entry["input"]["prompt"] = prompt
|
||||
self._timeline.append(entry)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def log_node(self, stage: str, input_data: dict, output_data: dict,
|
||||
detail: str = ""):
|
||||
"""
|
||||
记录一个流水线节点的输入输出。
|
||||
|
||||
Args:
|
||||
stage: 节点名称（gate/locator/generator/verifier/search 等）
|
||||
input_data: 节点接收的输入
|
||||
output_data: 节点产出的输出
|
||||
detail: 可选的补充说明
|
||||
"""
|
||||
if not self._enabled:
|
||||
return
|
||||
try:
|
||||
entry = {
|
||||
"time": datetime.now().strftime("%H:%M:%S.%f")[:-3],
|
||||
"elapsed_s": round(time.time() - self._start_time, 2),
|
||||
"type": "node_io",
|
||||
"stage": stage,
|
||||
"input": input_data,
|
||||
"output": output_data,
|
||||
}
|
||||
if detail:
|
||||
entry["detail"] = detail
|
||||
self._timeline.append(entry)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def log_decision(self, stage: str, decision: str, reason: str = "",
|
||||
context: Optional[dict] = None):
|
||||
"""
|
||||
记录一个工程决策（重试策略选择、熔断、搜索触发等）。
|
||||
|
||||
Args:
|
||||
stage: 决策发生的阶段
|
||||
decision: 决策内容
|
||||
reason: 决策原因
|
||||
context: 相关上下文数据
|
||||
"""
|
||||
if not self._enabled:
|
||||
return
|
||||
try:
|
||||
entry = {
|
||||
"time": datetime.now().strftime("%H:%M:%S.%f")[:-3],
|
||||
"elapsed_s": round(time.time() - self._start_time, 2),
|
||||
"type": "decision",
|
||||
"stage": stage,
|
||||
"decision": decision,
|
||||
"reason": reason,
|
||||
}
|
||||
if context:
|
||||
entry["context"] = context
|
||||
self._timeline.append(entry)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def write(self, expert_type: str = "unknown", success: bool = False):
|
||||
"""任务结束时写入日志文件。非阻塞。"""
|
||||
if not self._enabled:
|
||||
return
|
||||
try:
|
||||
self._log_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
record = {
|
||||
"task_id": self._task_id,
|
||||
"user_input": self._user_input,
|
||||
"model": self._model,
|
||||
"expert_type": expert_type,
|
||||
"success": success,
|
||||
"total_elapsed_s": round(time.time() - self._start_time, 2),
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
**self._metadata,
|
||||
"timeline": self._timeline,
|
||||
}
|
||||
|
||||
filename = f"{self._task_id}_{expert_type}.json"
|
||||
filepath = self._log_dir / filename
|
||||
filepath.write_text(
|
||||
json.dumps(record, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
logger.debug("DetailedLog written: %s", filepath)
|
||||
except Exception as e:
|
||||
logger.debug("DetailedLogger write failed (non-blocking): %s", e)
|
||||
@@ -94,7 +94,7 @@ try:
|
||||
from importlib.metadata import version as _pkg_version
|
||||
VERSION = _pkg_version("kwcode")
|
||||
except Exception:
|
||||
VERSION = "1.6.2"
|
||||
VERSION = "1.7.0"
|
||||
|
||||
# ── Shadow/重影大字 KAIWU ──
|
||||
_KAIWU_SHADOW = [
|
||||
|
||||
@@ -110,3 +110,6 @@ class TaskContext:
|
||||
|
||||
# 审计日志引用（Generator等专家通过此记录LLM调用）
|
||||
_audit_logger: "Any" = None
|
||||
|
||||
# DetailedLogger引用（完整不截断的流水线日志）
|
||||
_detailed_logger: "Any" = None
|
||||
|
||||
@@ -263,6 +263,9 @@ class Gate:
|
||||
max_tokens=30,
|
||||
temperature=0.0,
|
||||
)
|
||||
# DetailedLogger: 记录 Gate LLM 调用（通过 llm._on_llm_call 已自动记录，
|
||||
# 这里额外记录解析结果到 logger）
|
||||
logger.debug("[gate_llm] prompt=%s, raw=%s", prompt[:200], raw[:200])
|
||||
|
||||
# 尝试解析JSON响应
|
||||
json_str = self._extract_json(raw)
|
||||
|
||||
@@ -45,6 +45,7 @@ from kaiwu.flywheel.strategy_stats import StrategyStats
|
||||
from kaiwu.flywheel.user_pattern_memory import UserPatternMemory
|
||||
from kaiwu.telemetry.client import TelemetryClient
|
||||
from kaiwu.audit.logger import AuditLogger
|
||||
from kaiwu.audit.detailed_logger import DetailedLogger
|
||||
from kaiwu.core.model_capability import detect_model_tier, STRATEGIES, ModelTier
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -178,6 +179,28 @@ class PipelineOrchestrator:
|
||||
start_time = time.time()
|
||||
self._audit.start()
|
||||
|
||||
# ── DetailedLogger：完整流水线日志 ──
|
||||
model_name = getattr(self, '_model_name', 'unknown')
|
||||
self._detailed = DetailedLogger(user_input=user_input, model=model_name)
|
||||
self._detailed.set_metadata(
|
||||
project_root=project_root,
|
||||
gate_result=gate_result,
|
||||
)
|
||||
# 设置 LLM 回调，自动记录每次调用
|
||||
if self._detailed.enabled:
|
||||
def _llm_hook(messages, raw_output, elapsed_ms):
|
||||
self._detailed.log_llm(
|
||||
caller="llm_backend",
|
||||
prompt="",
|
||||
system="",
|
||||
raw_output=raw_output,
|
||||
elapsed_ms=elapsed_ms,
|
||||
messages=messages,
|
||||
)
|
||||
llm = getattr(self.generator, 'llm', None)
|
||||
if llm and hasattr(llm, '_on_llm_call'):
|
||||
llm._on_llm_call = _llm_hook
|
||||
|
||||
# 任务级超时看门狗
|
||||
TASK_TIMEOUT_S = 300 # 单任务最长5分钟
|
||||
_watchdog_triggered = threading.Event()
|
||||
@@ -201,6 +224,8 @@ class PipelineOrchestrator:
|
||||
)
|
||||
# 注入审计日志引用，让Generator等专家能记录LLM调用
|
||||
ctx._audit_logger = self._audit
|
||||
# 注入 DetailedLogger 引用到 ctx，让 Generator 等专家能直接写入
|
||||
ctx._detailed_logger = self._detailed
|
||||
|
||||
# 模型能力等级注入ctx
|
||||
ctx.model_tier = self._model_tier.value
|
||||
@@ -331,6 +356,10 @@ class PipelineOrchestrator:
|
||||
sequence = EXPERT_SEQUENCES.get(expert_type, ["generator", "verifier"])
|
||||
|
||||
self._emit(on_status, "gate", f"任务类型：{expert_type} | 难度：{gate_result.get('difficulty', '?')} | 路由：{ctx.routing_source}")
|
||||
# DetailedLogger: gate 决策
|
||||
self._detailed.log_node("gate",
|
||||
{"user_input": user_input[:500]},
|
||||
{"expert_type": expert_type, "difficulty": gate_result.get("difficulty", "?"), "routing_source": ctx.routing_source, "confidence": gate_result.get("confidence", 0), "sequence": sequence})
|
||||
|
||||
# ── MoE专家选择（已移除独立Expert类，统一走pipeline） ──
|
||||
moe_expert = self._select_moe_expert(ctx, expert_type)
|
||||
@@ -468,6 +497,11 @@ class PipelineOrchestrator:
|
||||
|
||||
self._emit(on_status, "retry", f"第{ctx.retry_count}次尝试失败：{error_detail[:100]}")
|
||||
self.bus.emit("retry", {"count": ctx.retry_count, "error": error_detail[:100]})
|
||||
# DetailedLogger: 记录重试决策
|
||||
self._detailed.log_decision("retry",
|
||||
decision=f"retry #{ctx.retry_count}, strategy={retry_strategy.get('sequence', [])}",
|
||||
reason=f"error_type={current_error_type}, detail={error_detail[:200]}",
|
||||
context={"error_type_streak": ctx._error_type_streak, "attempt_record": attempt_record})
|
||||
|
||||
# 第2次重试前反思：让LLM分析失败原因
|
||||
if ctx.retry_count == 1 and ctx.verifier_output and ctx.generator_output:
|
||||
@@ -566,6 +600,8 @@ class PipelineOrchestrator:
|
||||
ctx.generator_output = {"explanation": "我是KWCode，专注于代码任务。", "patches": []}
|
||||
result = {"passed": True}
|
||||
elapsed = time.time() - start_time
|
||||
self._detailed.log_node("chat", {"user_input": ctx.user_input[:500]}, {"explanation": (ctx.generator_output or {}).get("explanation", "")[:500]})
|
||||
self._detailed.write(expert_type="chat", success=True)
|
||||
return {
|
||||
"success": True,
|
||||
"context": ctx,
|
||||
@@ -723,6 +759,8 @@ class PipelineOrchestrator:
|
||||
self._record_flywheel(ctx, gate_result, True)
|
||||
# 审计日志
|
||||
self._audit.write(ctx, elapsed, True, getattr(self, '_model_name', 'unknown'))
|
||||
# DetailedLogger: 写入成功日志
|
||||
self._detailed.write(expert_type=gate_result.get("expert_type", "unknown"), success=True)
|
||||
return {
|
||||
"success": True,
|
||||
"context": ctx,
|
||||
@@ -763,6 +801,8 @@ class PipelineOrchestrator:
|
||||
self._record_flywheel(ctx, gate_result, False)
|
||||
# 审计日志
|
||||
self._audit.write(ctx, elapsed, False, getattr(self, '_model_name', 'unknown'))
|
||||
# DetailedLogger: 写入失败日志
|
||||
self._detailed.write(expert_type=gate_result.get("expert_type", "unknown"), success=False)
|
||||
return {
|
||||
"success": False,
|
||||
"context": ctx,
|
||||
@@ -779,6 +819,7 @@ class PipelineOrchestrator:
|
||||
search_result = self._search_subagent.search(ctx, self._manifest)
|
||||
if not search_result or not search_result.get("relevant_files"):
|
||||
self._emit(on_status, "locator_fail", "定位失败")
|
||||
self._detailed.log_node("locator", {"user_input": ctx.user_input[:200]}, {"result": None}, detail="定位失败")
|
||||
return False
|
||||
# 将干净结果传给ctx（Generator只看到这些）
|
||||
ctx.locator_output = {
|
||||
@@ -795,17 +836,26 @@ class PipelineOrchestrator:
|
||||
funcs = search_result["relevant_functions"]
|
||||
func_str = ', '.join(funcs[:3]) if funcs else "（文件级修改）"
|
||||
self._emit(on_status, "locator_done", f"文件：{', '.join(files[:3])} | 函数：{func_str}")
|
||||
# DetailedLogger: locator 节点
|
||||
self._detailed.log_node("locator",
|
||||
{"user_input": ctx.user_input[:200], "gap_type": str(ctx.gap.gap_type.value) if ctx.gap and hasattr(ctx.gap, 'gap_type') else ""},
|
||||
{"files": files, "functions": funcs, "method": search_result.get("method", ""), "upstream_constraints": search_result.get("upstream_constraints", "")[:500]})
|
||||
|
||||
elif step == "generator":
|
||||
self._emit(on_status, "generator", "生成patch...")
|
||||
result = self.generator.run(ctx)
|
||||
if not result:
|
||||
self._emit(on_status, "generator_fail", "生成失败")
|
||||
self._detailed.log_node("generator", {"files": (ctx.locator_output or {}).get("relevant_files", [])}, {"result": None}, detail="生成失败")
|
||||
return False
|
||||
n_patches = len(result.get("patches", []))
|
||||
self._emit(on_status, "generator_done", f"生成{n_patches}个patch")
|
||||
# 用新patch更新manifest（跨文件追踪）
|
||||
self._manifest.update(result.get("patches", []))
|
||||
# DetailedLogger: generator 节点
|
||||
self._detailed.log_node("generator",
|
||||
{"files": (ctx.locator_output or {}).get("relevant_files", []), "functions": (ctx.locator_output or {}).get("relevant_functions", [])},
|
||||
{"patch_count": n_patches, "files_modified": [p.get("file", "") for p in result.get("patches", [])], "explanation": result.get("explanation", "")})
|
||||
|
||||
elif step == "verifier":
|
||||
self._emit(on_status, "verifier", "验证中...")
|
||||
@@ -826,15 +876,22 @@ class PipelineOrchestrator:
|
||||
"error_message": detail[:200],
|
||||
"failed_tests": [],
|
||||
}
|
||||
self._detailed.log_node("verifier", {"patch_count": len((ctx.generator_output or {}).get("patches", []))}, {"passed": False, "error_type": "contract_violation", "detail": detail[:300]})
|
||||
return False
|
||||
result = self.verifier.run(ctx)
|
||||
if not result or not result.get("passed"):
|
||||
detail = result.get("error_detail", "unknown") if result else "no result"
|
||||
self._emit(on_status, "verifier_fail", f"验证失败：{detail[:80]}")
|
||||
self._detailed.log_node("verifier",
|
||||
{"patch_count": len((ctx.generator_output or {}).get("patches", []))},
|
||||
{"passed": False, "error_type": result.get("error_type", "") if result else "", "error_detail": detail[:1000], "tests_passed": result.get("tests_passed", 0) if result else 0, "tests_total": result.get("tests_total", 0) if result else 0})
|
||||
return False
|
||||
tp = result.get("tests_passed", 0)
|
||||
tt = result.get("tests_total", 0)
|
||||
self._emit(on_status, "verifier_done", f"语法OK | 测试：{tp}/{tt}")
|
||||
self._detailed.log_node("verifier",
|
||||
{"patch_count": len((ctx.generator_output or {}).get("patches", []))},
|
||||
{"passed": True, "tests_passed": tp, "tests_total": tt})
|
||||
|
||||
elif step == "office":
|
||||
self._emit(on_status, "office", "生成Office文档...")
|
||||
|
||||
@@ -217,11 +217,12 @@ class GeneratorExpert:
|
||||
if self._is_test_generation_task(ctx):
|
||||
return self._run_test_generation(ctx, files)
|
||||
|
||||
# ── whole_file scope: 存根实现，LLM返回完整文件 ──
|
||||
# ── whole_file scope: 存根实现 ──
|
||||
# Sub-task decomposition: 逐函数独立实现，每个函数独立context
|
||||
if ctx.gap and hasattr(ctx.gap, 'gap_type'):
|
||||
from kaiwu.core.gap_detector import GapType
|
||||
if ctx.gap.gap_type in (GapType.NOT_IMPLEMENTED, GapType.STUB_RETURNS_NONE):
|
||||
return self._run_whole_file(ctx, files)
|
||||
return self._run_stub_decomposed(ctx, files, funcs)
|
||||
|
||||
# For each file+function pair, extract original and generate modified
|
||||
# Deduplicate: only patch each (file, function) once
|
||||
@@ -367,7 +368,8 @@ class GeneratorExpert:
|
||||
return any(kw in lower for kw in _WEB_KEYWORDS)
|
||||
|
||||
def _generate_modified(self, ctx: TaskContext, fpath: str, original: str, task_desc: str) -> Optional[str]:
|
||||
"""Ask LLM to generate modified code. Hashline primary, full-function fallback."""
|
||||
"""Ask LLM to generate modified code. Hashline primary, full-function fallback.
|
||||
Bounded context: only pass current function + relevant failing tests."""
|
||||
search_ctx = ""
|
||||
if ctx.search_results:
|
||||
search_ctx = f"参考资料：\n{ctx.search_results}"
|
||||
@@ -392,38 +394,55 @@ class GeneratorExpert:
|
||||
if ctx.doc_context:
|
||||
prompt += f"\n\n## 相关文档参考\n{ctx.doc_context[:800]}"
|
||||
|
||||
# 注入初始测试失败信息（让LLM第一次就看到具体报错）
|
||||
# ── Bounded context: 只注入与当前函数相关的 failing tests ──
|
||||
# 从 structured_failures 中筛选与当前函数/文件相关的条目
|
||||
initial_failure = getattr(ctx, 'initial_test_failure', '')
|
||||
|
||||
# 结构化失败信息（精确告诉LLM每个测试为什么失败）— 首次和重试都注入
|
||||
structured = (ctx.verifier_output or {}).get("structured_failures", [])
|
||||
if not structured and initial_failure:
|
||||
from kaiwu.core.test_parser import parse_test_failures
|
||||
structured = parse_test_failures(initial_failure)
|
||||
if structured:
|
||||
lines = ["## 必须修复的测试失败（精确信息）"]
|
||||
for f in structured[:8]:
|
||||
name = f.get("test_name", "?")
|
||||
expected = f.get("expected", "")
|
||||
actual = f.get("actual", "")
|
||||
snippet = f.get("snippet", "")
|
||||
err_type = f.get("error_type", "")
|
||||
if expected and actual:
|
||||
lines.append(f"- {name}: 期望 {expected}，实际 {actual}")
|
||||
elif err_type and snippet:
|
||||
lines.append(f"- {name}: {err_type}: {snippet[:120]}")
|
||||
elif snippet:
|
||||
lines.append(f"- {name}: {snippet[:120]}")
|
||||
else:
|
||||
lines.append(f"- {name}")
|
||||
prompt += "\n\n" + "\n".join(lines)
|
||||
elif initial_failure and ctx.retry_count == 0:
|
||||
# fallback: 没解析出结构化信息时给raw output
|
||||
prompt += f"\n\n## 当前测试失败\n{initial_failure[:800]}"
|
||||
|
||||
# Inject retry_hint if available
|
||||
if structured:
|
||||
# 提取当前函数名（从 original 的第一行 def/class 获取）
|
||||
current_func = self._extract_func_name_from_code(original)
|
||||
# 筛选与当前函数/文件相关的失败
|
||||
relevant_failures = self._filter_relevant_failures(structured, current_func, fpath)
|
||||
if relevant_failures:
|
||||
lines = ["## 必须修复的测试失败（精确信息）"]
|
||||
for f in relevant_failures[:5]:
|
||||
name = f.get("test_name", "?")
|
||||
expected = f.get("expected", "")
|
||||
actual = f.get("actual", "")
|
||||
snippet = f.get("snippet", "")
|
||||
err_type = f.get("error_type", "")
|
||||
if expected and actual:
|
||||
lines.append(f"- {name}: 期望 {expected}，实际 {actual}")
|
||||
elif err_type and snippet:
|
||||
lines.append(f"- {name}: {err_type}: {snippet[:120]}")
|
||||
elif snippet:
|
||||
lines.append(f"- {name}: {snippet[:120]}")
|
||||
else:
|
||||
lines.append(f"- {name}")
|
||||
prompt += "\n\n" + "\n".join(lines)
|
||||
elif not relevant_failures and structured:
|
||||
# 没有精确匹配时，给前3条作为上下文（但不是全部）
|
||||
lines = ["## 相关测试失败"]
|
||||
for f in structured[:3]:
|
||||
name = f.get("test_name", "?")
|
||||
snippet = f.get("snippet", "")
|
||||
lines.append(f"- {name}: {snippet[:100]}" if snippet else f"- {name}")
|
||||
prompt += "\n\n" + "\n".join(lines)
|
||||
elif initial_failure and ctx.retry_count == 0:
|
||||
# fallback: 没解析出结构化信息时给精简的 raw output
|
||||
prompt += f"\n\n## 当前测试失败\n{initial_failure[:500]}"
|
||||
|
||||
# Inject retry_hint: 只传一句话总结，不传完整历史
|
||||
if ctx.retry_hint:
|
||||
prompt += f"\n\n## 重试提示\n{ctx.retry_hint}"
|
||||
# 截断 retry_hint，避免注入过多历史
|
||||
hint = ctx.retry_hint
|
||||
if len(hint) > 300:
|
||||
hint = hint[:300] + "..."
|
||||
prompt += f"\n\n## 重试提示\n{hint}"
|
||||
|
||||
system = self._build_system(ctx)
|
||||
|
||||
@@ -449,6 +468,41 @@ class GeneratorExpert:
|
||||
logger.warning("Generator: all candidates identical to original or empty")
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _extract_func_name_from_code(code: str) -> str:
|
||||
"""从代码片段中提取函数/类名。"""
|
||||
for line in code.split("\n"):
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("def "):
|
||||
# def func_name(...)
|
||||
name = stripped[4:].split("(")[0].strip()
|
||||
return name
|
||||
if stripped.startswith("class "):
|
||||
name = stripped[6:].split("(")[0].split(":")[0].strip()
|
||||
return name
|
||||
return ""
|
||||
|
||||
@staticmethod
|
||||
def _filter_relevant_failures(failures: list, func_name: str, file_path: str) -> list:
|
||||
"""筛选与当前函数/文件相关的测试失败。"""
|
||||
if not func_name:
|
||||
return failures[:5] # 无法确定函数名时返回前5条
|
||||
|
||||
relevant = []
|
||||
fname_lower = func_name.lower()
|
||||
fpath_base = file_path.rsplit("/", 1)[-1].rsplit("\\", 1)[-1] if file_path else ""
|
||||
|
||||
for f in failures:
|
||||
test_name = f.get("test_name", "").lower()
|
||||
snippet = f.get("snippet", "").lower()
|
||||
# 测试名包含函数名，或 snippet 中提到函数名/文件名
|
||||
if (fname_lower in test_name or
|
||||
fname_lower in snippet or
|
||||
(fpath_base and fpath_base.lower() in snippet)):
|
||||
relevant.append(f)
|
||||
|
||||
return relevant
|
||||
|
||||
def _try_hashline(self, ctx: TaskContext, fpath: str, original: str,
|
||||
task_desc: str, search_ctx: str) -> Optional[str]:
|
||||
"""Try Hashline anchor-based editing. Returns modified code or None."""
|
||||
@@ -853,11 +907,22 @@ class GeneratorExpert:
|
||||
output_tokens=output_tokens,
|
||||
engineering_actions={},
|
||||
)
|
||||
# DetailedLogger: 完整不截断记录
|
||||
detailed = getattr(ctx, '_detailed_logger', None)
|
||||
if detailed and detailed.enabled:
|
||||
detailed.log_llm(
|
||||
caller=caller,
|
||||
prompt=prompt,
|
||||
system=system,
|
||||
raw_output=raw_output or '',
|
||||
tokens={"input": len(prompt) // 4 + len(system) // 4, "output": len(raw_output or '') // 4},
|
||||
)
|
||||
except Exception:
|
||||
pass # 非阻塞
|
||||
|
||||
def _run_whole_file(self, ctx: TaskContext, files: list[str]) -> Optional[dict]:
|
||||
"""whole_file scope：LLM返回完整文件内容，直接write_file，不走apply_patch。"""
|
||||
"""whole_file scope：LLM返回完整文件内容，直接write_file，不走apply_patch。
|
||||
Fallback for _run_stub_decomposed when decomposition fails."""
|
||||
patches = []
|
||||
explanation_parts = []
|
||||
|
||||
@@ -917,3 +982,178 @@ class GeneratorExpert:
|
||||
}
|
||||
ctx.generator_output = result
|
||||
return result
|
||||
|
||||
def _run_stub_decomposed(self, ctx: TaskContext, files: list[str], funcs: list[str]) -> Optional[dict]:
|
||||
"""
|
||||
Sub-task decomposition for stub tasks.
|
||||
Instead of asking LLM to implement all pass functions at once,
|
||||
decompose into per-function subtasks with independent bounded context.
|
||||
Each function gets its own LLM call with only its code + relevant tests.
|
||||
Falls back to _run_whole_file if decomposition fails.
|
||||
"""
|
||||
patches = []
|
||||
explanation_parts = []
|
||||
|
||||
for fpath in files[:3]:
|
||||
if "test" in fpath.lower():
|
||||
continue
|
||||
|
||||
# Read file content
|
||||
if self.tools:
|
||||
content = self.tools.read_file(fpath)
|
||||
else:
|
||||
content = ctx.relevant_code_snippets.get(fpath, "")
|
||||
if not content or content.startswith("[ERROR]"):
|
||||
continue
|
||||
|
||||
# Find all stub functions (pass/... body)
|
||||
stub_funcs = self._find_stub_functions(content)
|
||||
if not stub_funcs:
|
||||
continue
|
||||
|
||||
# If only 1-2 stubs, or file is small, use whole_file (simpler)
|
||||
if len(stub_funcs) <= 2 or len(content.split("\n")) < 50:
|
||||
result = self._run_whole_file(ctx, [fpath])
|
||||
if result:
|
||||
return result
|
||||
continue
|
||||
|
||||
# ── Per-function decomposition ──
|
||||
initial_failure = getattr(ctx, 'initial_test_failure', '')
|
||||
structured_failures = (ctx.verifier_output or {}).get("structured_failures", [])
|
||||
if not structured_failures and initial_failure:
|
||||
from kaiwu.core.test_parser import parse_test_failures
|
||||
structured_failures = parse_test_failures(initial_failure)
|
||||
|
||||
file_patches = []
|
||||
for func_name, func_code in stub_funcs:
|
||||
# Build bounded context for this single function
|
||||
relevant_tests = self._filter_relevant_failures(
|
||||
structured_failures, func_name, fpath
|
||||
)
|
||||
|
||||
test_info = ""
|
||||
if relevant_tests:
|
||||
lines = []
|
||||
for f in relevant_tests[:3]:
|
||||
name = f.get("test_name", "?")
|
||||
expected = f.get("expected", "")
|
||||
actual = f.get("actual", "")
|
||||
snippet = f.get("snippet", "")
|
||||
if expected and actual:
|
||||
lines.append(f"- {name}: 期望 {expected}，实际 {actual}")
|
||||
elif snippet:
|
||||
lines.append(f"- {name}: {snippet[:120]}")
|
||||
else:
|
||||
lines.append(f"- {name}")
|
||||
test_info = "\n\n## 相关测试\n" + "\n".join(lines)
|
||||
|
||||
prompt = (
|
||||
f"任务：{ctx.user_input[:200]}\n\n"
|
||||
f"实现以下函数（来自 {fpath}）：\n"
|
||||
f"```\n{func_code}\n```\n\n"
|
||||
f"只输出实现后的完整函数代码（从def开始）。\n"
|
||||
f"保持原始缩进和签名不变，只替换pass为实现。\n"
|
||||
f"纯代码，无markdown，无解释。"
|
||||
f"{test_info}"
|
||||
)
|
||||
|
||||
if ctx.search_results:
|
||||
prompt += f"\n\n参考资料：\n{ctx.search_results[:500]}"
|
||||
|
||||
system = self._build_system(ctx)
|
||||
|
||||
raw = self.llm.generate(prompt=prompt, system=system, max_tokens=2048, temperature=0.0)
|
||||
self._log_llm_call(ctx, f"generator_stub_{func_name}", prompt, system, raw)
|
||||
modified = self._clean_code_output(raw)
|
||||
|
||||
if modified and modified.strip() != func_code.strip():
|
||||
modified = modified.strip("\n")
|
||||
modified = self._align_indentation(func_code, modified)
|
||||
file_patches.append({
|
||||
"file": fpath,
|
||||
"original": func_code,
|
||||
"modified": modified,
|
||||
})
|
||||
explanation_parts.append(f"{fpath}:{func_name}")
|
||||
|
||||
if file_patches:
|
||||
patches.extend(file_patches)
|
||||
|
||||
if not patches:
|
||||
logger.debug("Stub decomposition produced no patches, falling back to whole_file")
|
||||
return self._run_whole_file(ctx, files)
|
||||
|
||||
result = {
|
||||
"patches": patches,
|
||||
"explanation": f"Stub impl: {', '.join(explanation_parts)}",
|
||||
}
|
||||
ctx.generator_output = result
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _find_stub_functions(content: str) -> list[tuple[str, str]]:
|
||||
"""
|
||||
Find all stub functions (body is just 'pass', '...', or 'return None') in file content.
|
||||
Returns list of (func_name, full_function_code) tuples.
|
||||
"""
|
||||
lines = content.split("\n")
|
||||
stubs = []
|
||||
i = 0
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
stripped = line.lstrip()
|
||||
if stripped.startswith("def "):
|
||||
func_name = stripped[4:].split("(")[0].strip()
|
||||
indent_level = len(line) - len(stripped)
|
||||
start_idx = i
|
||||
|
||||
# Find end of function
|
||||
end_idx = i + 1
|
||||
while end_idx < len(lines):
|
||||
l = lines[end_idx]
|
||||
if l.strip() == "":
|
||||
end_idx += 1
|
||||
continue
|
||||
current_indent = len(l) - len(l.lstrip())
|
||||
if current_indent <= indent_level and l.strip():
|
||||
break
|
||||
end_idx += 1
|
||||
|
||||
func_code = "\n".join(lines[start_idx:end_idx]).rstrip()
|
||||
|
||||
# Check if body is stub (skip docstrings and comments)
|
||||
in_docstring = False
|
||||
clean_body = []
|
||||
for bl in lines[start_idx + 1:end_idx]:
|
||||
s = bl.strip()
|
||||
if not s:
|
||||
continue
|
||||
if s.startswith('"""') or s.startswith("'''"):
|
||||
if s.count('"""') >= 2 or s.count("'''") >= 2:
|
||||
continue # single-line docstring
|
||||
in_docstring = not in_docstring
|
||||
continue
|
||||
if in_docstring:
|
||||
continue
|
||||
if s.startswith("#"):
|
||||
continue
|
||||
clean_body.append(s)
|
||||
|
||||
is_stub = (
|
||||
len(clean_body) == 0 or
|
||||
(len(clean_body) == 1 and clean_body[0] in (
|
||||
"pass", "...", "return None",
|
||||
"raise NotImplementedError",
|
||||
"raise NotImplementedError()",
|
||||
))
|
||||
)
|
||||
|
||||
if is_stub:
|
||||
stubs.append((func_name, func_code))
|
||||
|
||||
i = end_idx
|
||||
else:
|
||||
i += 1
|
||||
|
||||
return stubs
|
||||
|
||||
@@ -73,6 +73,8 @@ class LLMBackend:
|
||||
self._total_output_tokens: int = 0
|
||||
self._call_count: int = 0
|
||||
self._token_budget: int = 0 # 0 = unlimited
|
||||
# DetailedLogger 回调：每次 LLM 调用后触发（由 orchestrator 设置）
|
||||
self._on_llm_call = None # Callable(caller, messages, raw_output, tokens, elapsed_ms)
|
||||
|
||||
# Prefer native llama.cpp if model_path provided and library available
|
||||
if model_path and HAS_LLAMA_CPP:
|
||||
@@ -122,7 +124,14 @@ class LLMBackend:
|
||||
# If it's not localhost at all, assume OpenAI-compatible
|
||||
if "localhost" not in url_lower and "127.0.0.1" not in url_lower:
|
||||
return True
|
||||
return False
|
||||
# For localhost on non-standard ports, probe /api/tags to detect Ollama
|
||||
try:
|
||||
resp = httpx.get(f"{url.rstrip('/')}/api/tags", timeout=3)
|
||||
if resp.status_code == 200 and "models" in resp.text:
|
||||
return False # It's Ollama
|
||||
except Exception:
|
||||
pass
|
||||
return True # Not Ollama, assume OpenAI-compatible
|
||||
|
||||
@property
|
||||
def token_usage(self) -> dict:
|
||||
@@ -211,6 +220,20 @@ class LLMBackend:
|
||||
self._last_elapsed = _time.perf_counter() - t0
|
||||
if self._tps_estimator:
|
||||
self._tps_estimator.record(result, self._last_elapsed)
|
||||
# DetailedLogger 回调
|
||||
if self._on_llm_call:
|
||||
try:
|
||||
messages = []
|
||||
if system:
|
||||
messages.append({"role": "system", "content": system})
|
||||
messages.append({"role": "user", "content": prompt})
|
||||
self._on_llm_call(
|
||||
messages=messages,
|
||||
raw_output=result,
|
||||
elapsed_ms=self._last_elapsed * 1000,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
return result
|
||||
|
||||
def _generate_native(
|
||||
@@ -281,6 +304,16 @@ class LLMBackend:
|
||||
self._last_elapsed = _time.perf_counter() - t0
|
||||
if self._tps_estimator:
|
||||
self._tps_estimator.record(result, self._last_elapsed)
|
||||
# DetailedLogger 回调
|
||||
if self._on_llm_call:
|
||||
try:
|
||||
self._on_llm_call(
|
||||
messages=messages,
|
||||
raw_output=result,
|
||||
elapsed_ms=self._last_elapsed * 1000,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
return result
|
||||
|
||||
def _chat_ollama(
|
||||
|
||||
0
logs/.gitkeep
Normal file
0
logs/.gitkeep
Normal file
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "kwcode"
|
||||
version = "1.6.2"
|
||||
version = "1.7.0"
|
||||
description = "KwCode - Local-model coding agent with MoE expert pipeline"
|
||||
requires-python = ">=3.10"
|
||||
readme = "README.md"
|
||||
|
||||
Reference in New Issue
Block a user