feat(roundtable): 第二/三步模型自动容错(出问题自动换下一个模型)
模型出问题时按 config.yaml models[] 顺序自动切到下一个模型继续,并告知用户 「『X』模型异常 → 已切换『Y』继续」。前台实时 + 后台挂起两条路径都覆盖。 - 中间件兜底/熔断 AIMessage 打 LLM_ERROR_MARKER 标记,is_llm_error_message 可识别 - roundtable_model_fallback: fallback_model_chain 候选链 + reason_text + 总开关 env ROUNDTABLE_MODEL_FALLBACK - 触发: 异常(start_run失败/idle卡死) / 兜底错误文案 / 交付为空(仅seat/leader) - 后台 InProcessRoundtableGateway._run_turn_with_fallback 包住每轮 run - 前台 _special_run/_leader_run 收尾检测失败→model_switch 帧+换模型重流 - 结构化抽取 flow_extract round1 astream 同款换模型循环 - Step2 单席位异常隔离: 单席位失败不拖垮整批/整链/整个作业 - 时间线插「系统提示」气泡 + toast + 诊断 model_switch 事件 - Tests: test_roundtable_model_fallback.py Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
ae2706dad3
commit
d383b0e9dd
18 files changed
+1033
-174
No files matched your search
@@ -0,0 +1,104 @@
|
||||
"""圆桌会商「模型自动容错(换下一个模型)」共享助手。
|
||||
|
||||
会商的第二步(leader / seat)与第三步(summary / report / dashboard / structure)单轮执行时,
|
||||
若所用模型出现问题(抛异常 / idle 卡死 / 中间件兜底错误文案 / 交付为空),应自动切到
|
||||
**下一个可用模型**继续,并把「『X』模型异常 → 已切换『Y』模型」告知用户。
|
||||
|
||||
本模块只放**与执行路径无关**的纯逻辑,供前台(``multi_agent.py``)与后台
|
||||
(``roundtable_inprocess_gateway.py``)两条路径共用:
|
||||
|
||||
- ``model_fallback_enabled()``:总开关(env ``ROUNDTABLE_MODEL_FALLBACK``,默认开);
|
||||
- ``fallback_model_chain(current)``:按 ``config.yaml`` ``models[]`` 顺序给出候选模型链
|
||||
``[当前模型, …其余]``(去重保序);
|
||||
- ``reason_text(reason)``:把失败原因码翻译成简短中文(用于用户提示);
|
||||
- 复用 harness 的 ``is_llm_error_message`` 识别中间件兜底文案。
|
||||
|
||||
放在 app 层(依赖 ``deerflow.config`` + harness 的中间件常量,皆为 app→harness 合法方向)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
|
||||
from deerflow.agents.middlewares.llm_error_handling_middleware import (
|
||||
is_llm_error_message as is_llm_error_message, # re-export for callers
|
||||
)
|
||||
from deerflow.config import get_app_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 失败原因码 → 用户可见简短中文(用于「『X』模型<原因>,已切换…」提示)。
|
||||
_REASON_TEXT: dict[str, str] = {
|
||||
"quota": "额度不足/账单异常",
|
||||
"auth": "鉴权失败/无访问权限",
|
||||
"busy": "服务繁忙",
|
||||
"transient": "网络/连接异常",
|
||||
"upstream": "上游响应异常",
|
||||
"circuit": "连续失败被熔断",
|
||||
"generic": "调用失败",
|
||||
"exception": "调用异常",
|
||||
"timeout": "长时间无响应(卡死)",
|
||||
"empty": "未产出有效内容",
|
||||
}
|
||||
|
||||
|
||||
def reason_text(reason: str | None) -> str:
|
||||
"""把失败原因码翻译成简短中文(未知码 → 「调用失败」)。"""
|
||||
if not reason:
|
||||
return "调用失败"
|
||||
return _REASON_TEXT.get(reason, "调用失败")
|
||||
|
||||
|
||||
def model_fallback_enabled() -> bool:
|
||||
"""模型自动容错总开关(默认开;设 ``ROUNDTABLE_MODEL_FALLBACK=0/false/off`` 关闭)。
|
||||
|
||||
关闭时 ``fallback_model_chain`` 只返回单个候选(原模型),即恢复旧行为:模型失败就失败,
|
||||
不再尝试其它模型。
|
||||
"""
|
||||
raw = (os.getenv("ROUNDTABLE_MODEL_FALLBACK", "1") or "1").strip().lower()
|
||||
return raw not in {"0", "false", "no", "off"}
|
||||
|
||||
|
||||
def _max_attempts() -> int:
|
||||
"""最多尝试的模型个数(含首个)。``ROUNDTABLE_MODEL_FALLBACK_MAX``,0/未设 = 不限(试完全部)。"""
|
||||
try:
|
||||
return max(0, int(os.getenv("ROUNDTABLE_MODEL_FALLBACK_MAX", "0") or "0"))
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
|
||||
def _config_model_names() -> list[str]:
|
||||
try:
|
||||
cfg = get_app_config()
|
||||
return [m.name for m in (cfg.models or []) if getattr(m, "name", None)]
|
||||
except Exception: # noqa: BLE001 — 拿不到配置(内存后端/测试桩)时退化为空
|
||||
logger.debug("fallback_model_chain: get_app_config().models unavailable", exc_info=True)
|
||||
return []
|
||||
|
||||
|
||||
def fallback_model_chain(current: str | None) -> list[str]:
|
||||
"""按 ``config.yaml`` ``models[]`` 顺序构造候选模型链:``[当前模型, …其余]``,去重保序。
|
||||
|
||||
- ``current`` 为空/None → 从 ``models[0]`` 起(即后端默认模型),后接其余;
|
||||
- ``current`` 在配置内 → 它排第一,其余按配置顺序补齐;
|
||||
- ``current`` 不在配置内(自定义/未知名)→ 仍排第一,再补齐全部配置模型;
|
||||
- 总开关关闭 → 只返回单个候选(``current`` 或默认模型),等同旧行为;
|
||||
- 拿不到任何配置模型 → 返回 ``[current]``(current 为空则空列表,调用方自行退化为不换模型)。
|
||||
"""
|
||||
all_names = _config_model_names()
|
||||
cur = (current or "").strip()
|
||||
ordered: list[str] = []
|
||||
if cur:
|
||||
ordered.append(cur)
|
||||
for n in all_names:
|
||||
if n and n not in ordered:
|
||||
ordered.append(n)
|
||||
if not ordered:
|
||||
return [cur] if cur else []
|
||||
if not model_fallback_enabled():
|
||||
return ordered[:1]
|
||||
cap = _max_attempts()
|
||||
if cap > 0:
|
||||
ordered = ordered[:cap]
|
||||
return ordered
|
||||
Reference in new issue
Block a user