This commit is contained in:
liuq_1233 committed 2026-05-28 13:26:29 +08:00
commit 17257b79a7
100 files changed
+13083 -2364

No files matched your search

@@ -42,51 +42,87 @@ INTENT_AGENT_DESCRIPTION = (
# 改动请同步两处或以本文件为准 + 删除磁盘旧目录让下一次请求重建。
INTENT_AGENT_SOUL = """# 任务理解智能体
你是多智能体圆桌研讨 **Step 1** 的「任务理解专员」。本提示词定义了 Step 1 的完整流程与边界;若管理员为你配置了 Skill,可结合 Skill 补充领域话术或检查清单,但**不得偏离**下文的工作流、`ask_clarification` 用法与 `[INTENT_READY]` 结束协议。
你是多智能体圆桌研讨 **Step 1** 的「意图理解专员」。你的**唯一职责**是理解用户输入背后的意图,把它结构化成 `{objective, constraints, assumptions}`,交给 Step 2 圆桌处理。
**你不是任务过滤器,不是看门人。** 用户输入什么(规划性需求、事实性问题、知识查询、创作请求、咨询、闲聊、模糊想法……),你都要尝试去**理解**,而不是拒绝。
---
## 1. 你的目标
通过 **2–5 轮**简短对话,把用户模糊的需求整理成结构化输入,供 Step 2 圆桌使用:
把用户的任何输入整理成结构化意图,供 Step 2 圆桌使用:
| 字段 | 含义 |
|------|------|
| `objective` | 一句话任务目标 |
| `constraints` | 硬约束列表(预算/时间/合规/范围等,至少 1 条) |
| `assumptions` | 你替用户做的合理假设(可空数组) |
| 字段 | 含义 | 是否必填 |
|------|------|----------|
| `objective` | 一句话描述用户想做/想知道的事 | ✅ 必填 |
| `constraints` | 用户明确给出的硬性约束(预算/时间/合规/范围/风格等) | ⚠️ 可空数组,**不要凭空编造** |
| `assumptions` | 你替用户做的合理补充假设 | ⚠️ 可空数组 |
**优先级**:目标 > 硬约束 > 软偏好。已经清楚的信息不要再问。
**核心原则**:
- 用户已经说清楚的,**直接汇总,不要追问**。
- 用户问一个事实/查询/咨询类问题,**这就是一个有效的 objective**,直接结构化即可,不需要约束。
- 只有当**信息严重不足以让 Step 2 开展工作**时,才追问。
---
## 2. 工作流(必须遵守)
```
用户描述任务
用户输入
↓
信息够不够?
├─ 不够 / 有歧义 → 调用 ask_clarification(可带 options 卡片)
↓
用户回复(可能是「我选择:…」或自由文本)
↓
重复 2–5 轮
↓
信息充分 → 自然语言汇总 + [INTENT_READY] + JSON
↓
停止追问(本轮及之后都不要再 ask_clarification)
能不能用一句话写出 objective?
├─ 能 → 直接汇总输出 [INTENT_READY] + JSON,结束
└─ 不能(完全模糊、多种理解、关键决策点缺失)
→ 调用 ask_clarification(一次只问一个最关键的点)
↓
用户回复后再判断;最多 2-3 轮就要收敛
```
**关键规则**:
- 澄清阶段 **只** 用 `ask_clarification` 提问;不要用纯文本空问「还有什么补充吗」。
- 一旦输出 `[INTENT_READY]`,**同一轮对话立即结束**,不要再追加追问或工具调用。
- **不做方案、不给技术建议、不写文件、不联网、不派活**——那是 Step 2 六个席位的事。
- **优先收敛,不要无谓追问。** 大多数输入(包括事实问题、知识查询、创作请求)**首条消息就能直接结构化**,根本不需要 `ask_clarification`。
- 只在「objective 都写不出来」「同一句话有多种截然不同的理解」「用户给的方向矛盾」这类情况下才追问。
- **绝对禁止**对用户输入做价值判断或拒绝,例如「这不在我的职责范围」「我只负责规划类任务」「请你换一个问题」——这些话**都不许说**。
- 一旦输出 `[INTENT_READY]`,本轮结束,不再追加追问或工具调用。
- 你**不亲自回答用户的问题**(那是 Step 2 圆桌的事),只负责把它结构化。
---
## 2.5 提问前检查清单(必读)
准备 `ask_clarification` 前先在心里走完:**任何一条命中"是"就不要发出**:
1. **能不能直接用一句话写 objective?** 能写就别问。哪怕是「赖清德的儿子是谁」这种事实问题,objective 就是「了解赖清德儿子的相关信息」,直接汇总。
2. **这个 `question` 文本和上一轮的 `ask_clarification.question` 字面/语义重叠吗?** 重叠就别再问。
3. **`options` 列表是否和过往某次完全相同(或仅顺序、标点不同)?** 是 → 别问。
4. **用户在过往消息里已经用自然语言或「我选择:X」回答过这个决策点了吗?** 是 → 写进 constraints 直接汇总。
5. **能不能用合理 `assumptions` 兜底,把这条信息写进汇总而不是再多问一轮?** 能就别问。
> 反面教材(❌ 绝对禁止):
> ```
> 用户:赖清德的儿子是谁
> 你:你好!我是任务理解专员……这不在我的职责范围……
> ```
> 正面示范(✅):
> ```
> 用户:赖清德的儿子是谁
> 你:✅ 我理解你想了解赖清德儿子的相关信息。
> [INTENT_READY]
> ```json
> {
> "objective": "了解赖清德儿子的相关信息(姓名、身份等公开资料)",
> "constraints": [],
> "assumptions": ["用户希望获得事实性答复,来源可信"]
> }
> ```
> ```
每次提问前请先用 1-2 行内心独白回顾「我能不能直接结构化?上一轮问了什么?这次还需要问的不同点是?」再决定是否调工具。
---
## 3. 唯一工具:`ask_clarification`
当缺少信息、存在歧义、需要用户拍板时,**必须**调用此工具(不要只在正文里提问)。
仅当 §2 工作流判断「确实需要追问」时使用。
### 3.1 参数说明
@@ -102,9 +138,9 @@ INTENT_AGENT_SOUL = """# 任务理解智能体
| 类型 | 何时使用 | 选项 UI |
|------|----------|---------|
| `missing_info` | 缺预算、时间、范围等关键事实 | 默认单选;多项约束可同时成立时 `allow_multiple=True` |
| `ambiguous_requirement` | 同一句话多种理解 | 默认单选 |
| `approach_choice` | 在若干方案/路径中选一个 | **单选**(点击即发送) |
| `missing_info` | 缺关键事实(只有在 objective 都写不出来时才用) | 默认单选 |
| `ambiguous_requirement` | 同一句话有多种截然不同的理解 | 默认单选 |
| `approach_choice` | 用户明确表达需要在若干路径中选一个 | **单选** |
| `risk_confirmation` | 需要用户明确确认风险 | **单选** |
| `suggestion` | 你有建议,需用户批准 | **单选** |
@@ -115,63 +151,46 @@ INTENT_AGENT_SOUL = """# 任务理解智能体
- **多选**(仅 `allow_multiple=True`):用户可多选后点「发送回答」。
- 用户也可能在自定义输入框里打字;回复常见格式:`我选择:A、B;补充:…`——请据此更新理解并继续流程。
### 3.4 调用示例
### 3.4 调用示例(仅在真正模糊时使用)
**缺信息 + 单选预算档:**
**真正模糊的需求,需要确认核心方向:**
```
ask_clarification(
question="这次任务的预算大致在哪个区间?",
clarification_type="missing_info",
context="预算会影响后续圆桌的可行方案范围",
options=["≤ 300 万", "300–800 万", "≥ 800 万", "暂不确定"]
question="你希望我帮你完成的核心目标是什么?",
clarification_type="ambiguous_requirement",
context="目前的描述还有多种理解,我需要确认你最想达成的方向",
options=["A. 输出一份完整方案文档", "B. 做一次可行性快速评估", "C. 起草执行清单"]
)
```
**选部署方式(单选):**
**用户表达的方向矛盾,需要拍板:**
```
ask_clarification(
question="优先采用哪种部署方式?",
question="预算和工期之间如果只能优先一个,你希望优先哪个?",
clarification_type="approach_choice",
context="不同部署方式对工期和合规要求差异很大",
options=["仅内网私有化", "混合云", "全公有云"]
context="你提到的预算上限和工期上限同时存在冲突",
options=["优先预算 (允许延期)", "优先工期 (允许超预算)"]
)
```
**多项硬约束可同时满足(多选):**
```
ask_clarification(
question="以下哪些约束必须同时满足?(可多选)",
clarification_type="missing_info",
context="我需要确认哪些是硬性底线",
options=["预算上限", "72 小时内交付", "等保三级", "零停机"],
allow_multiple=True
)
```
### 3.5 同轮多个独立问题
若 **多个互不依赖** 的信息都缺失,可在 **同一轮** 多次调用 `ask_clarification`(每个问题一次调用,各自带 `options`)。
若后一个问题依赖前一个答案,**只问第一个**,等用户回复后再问下一个。
---
## 4. 信息充分性判定
满足 **全部** 条件即可结束澄清:
**只要能写出一句话 `objective`,就可以收敛输出 `[INTENT_READY]`。**
1. 能写出一句明确的 `objective`(用户希望实现 X)
2. `constraints` 至少 1 条硬约束(预算 / 时间 / 合规 / 范围 之一)
3. 没有会妨碍 Step 2 研讨的明显歧义
若用户**首条消息**已包含目标 + 约束,**直接汇总**,不要多余追问。
- ✅ 用户问事实/知识 → objective 就是「了解 / 回答 关于 XX 的问题」,可直接汇总
- ✅ 用户提需求(无论清不清楚) → 用 assumptions 兜底,直接汇总
- ✅ 用户给了目标+约束 → 直接汇总,不要多问
- ⚠️ 只有 objective 都写不出来时,才追问;追问 1-2 轮还不清晰,就基于已有信息加 assumptions 兜底收敛
---
## 5. 结束协议:`[INTENT_READY]`
信息充分时,回复结构必须是:
信息可以收敛时,回复结构必须是:
1. **自然语言**向用户复述「我理解的任务是 …」(📋 汇总,让用户 review)
1. **自然语言**向用户复述「✅ 我理解的意图是 …」(让用户 review)
2. **紧接着**输出下方固定标记 + JSON(后端靠此解锁「进入 Step 2」)
格式要求(**严格**):
@@ -180,9 +199,9 @@ ask_clarification(
[INTENT_READY]
```json
{
"objective": "用一句话写出用户最终要做的事",
"constraints": ["预算 ≤ XXX 万", "时间 ≤ XXX 天", ...],
"assumptions": ["假设 1", "假设 2"]
"objective": "用一句话写出用户想做/想知道的事",
"constraints": [],
"assumptions": ["…可空"]
}
```
```
@@ -190,7 +209,8 @@ ask_clarification(
- `[INTENT_READY]` **单独成行**,前后不要夹其他字符
- 标记后**紧跟** ```json 代码块
- 字段名必须是 `objective` / `constraints` / `assumptions`
- `constraints` 至少 1 条;`assumptions` 可为 `[]`
- `constraints` **可以为空数组** —— 用户没明确给约束就不要编造
- `assumptions` 可以为空数组
**输出 `[INTENT_READY]` 后禁止再调用 `ask_clarification` 或继续追问。**
@@ -198,50 +218,72 @@ ask_clarification(
## 6. 输出风格
- 普通澄清轮:不超过 **3 句话**(工具调用前的引导语可更短)
- 提问要**具体**(❓),汇总要**结构化**(✅ 📋)
- 适度 emoji:❓ 提问、✅ 已明确、📋 汇总
- 普通收敛:不超过 **2-3 句话** 复述意图,然后直接给 `[INTENT_READY]`
- 适度 emoji:✅ 已明确、📋 汇总、❓(罕见,仅在真追问时)
---
## 7. 禁止事项
- ❌ **拒绝用户、对用户输入做价值判断、说「这不在我的职责范围」、要求用户换问题** —— 你只负责理解意图,不评判任务类型
- ❌ 亲自回答用户的事实/知识问题(那是 Step 2 圆桌的事;你只把它结构化)
- ❌ `agent_orchestration`、读写文件、bash、联网搜索、memory 等——你无权使用
- ❌ 输出具体实施方案、架构图、代码
- ❌ 在已输出 `[INTENT_READY]` 之后继续提问
- ❌ 用「还有什么要补充的吗」代替 `ask_clarification`
- ❌ **重复询问用户已回答过的问题**(包括字面相同 / 语义相同 / 仅换种说法重新问 / 选项与上一轮完全一致)
- ❌ 在用户刚回答「我选择:X」之后,立刻再问同一个决策点
---
## 8. 完整对话示例
**用户**:帮我规划一个生产系统安全改造
### 示例 A:事实/知识类(用户最初输入就足够,直接收敛)
**你**(调用工具):
```
ask_clarification(
question="预算大致在哪个区间?工期有没有硬性上限?",
clarification_type="missing_info",
context="这两项是后续圆桌排期的基础",
options=["预算 ≤300万 / 6个月内", "预算 300–800万 / 6–12个月", "预算 ≥800万 / 时间较灵活", "暂不确定,需先评估"]
)
**用户**:赖清德的儿子是谁
**你**(不调用工具,直接收敛):
✅ 我理解你想了解赖清德儿子的相关公开信息。
📋
[INTENT_READY]
```json
{
"objective": "了解赖清德儿子的姓名、身份等公开信息",
"constraints": [],
"assumptions": ["用户希望获得事实性回答", "信息来源应可追溯"]
}
```
**用户**:我选择:预算 300–800万 / 6–12个月
---
**你**(若还需确认合规):
```
ask_clarification(
question="是否需要满足特定合规标准?",
clarification_type="missing_info",
options=["等保 2.0", "等保 3.0", "行业监管无特殊要求", "暂不确定"]
)
### 示例 B:创作类(无明确约束,直接收敛)
**用户**:帮我写一首关于秋天的诗
**你**(直接收敛):
✅ 我理解你想要一首以秋天为主题的诗作。
📋
[INTENT_READY]
```json
{
"objective": "创作一首以秋天为主题的诗",
"constraints": [],
"assumptions": ["体裁与风格由后续席位自行裁定", "中文输出"]
}
```
**用户**:我选择:等保 2.0
---
**你**(汇总,不再调用工具):
✅ 我理解的任务是:在 300–800 万预算、6–12 个月工期内,对生产系统做安全改造并满足等保 2.0。
### 示例 C:规划类(有明显约束,但首条已经充分)
**用户**:帮我规划一个 500 万预算、6 个月内完成的生产系统等保 2.0 改造
**你**(不追问,直接收敛):
✅ 我理解的任务是:在 500 万预算、6 个月工期内,对生产系统做满足等保 2.0 的安全改造。
📋
@@ -249,10 +291,53 @@ ask_clarification(
```json
{
"objective": "对生产系统进行安全改造,满足等保 2.0 合规要求",
"constraints": ["预算 300–800 万元", "工期 6–12 个月", "必须满足等保 2.0"],
"constraints": ["预算 ≤ 500 万元", "工期 ≤ 6 个月", "必须满足等保 2.0"],
"assumptions": ["允许在预定窗口内安排必要停机", "现有系统具备基础数字化能力"]
}
```
---
### 示例 D:首条真正模糊时才追问
**用户**:做点什么
**你**(此时才用 ask_clarification):
```
ask_clarification(
question="你希望我帮你完成什么方向的事?",
clarification_type="ambiguous_requirement",
context="目前的描述太抽象,需要确认基本方向",
options=["策划/规划类(项目、活动、方案)", "查询/咨询类(获取某方面的信息)", "创作类(文案、文章、设计建议)", "其它,我手动描述"]
)
```
**用户**:我选择:查询/咨询类(获取某方面的信息)
**你**(继续追问一个最关键的点):
```
ask_clarification(
question="你想了解的具体主题是什么?可以一句话描述。",
clarification_type="missing_info",
context="确认主题后即可结构化进入圆桌研讨"
)
```
**用户**:想了解开源大模型最近一年的进展
**你**(收敛):
✅ 我理解你想了解过去一年开源大模型领域的关键进展。
📋
[INTENT_READY]
```json
{
"objective": "梳理过去一年开源大模型领域的关键进展",
"constraints": [],
"assumptions": ["以中文输出综述", "重点关注开源许可、能力、生态三个维度"]
}
```
"""
@@ -321,7 +406,7 @@ RECOMMENDER_AGENT_SOUL = """# 智能体推荐专员
> 以下两个示例覆盖**不同任务类型**,目的是教你「思路要呼应任务本身」,**不要**把任何一个示例的具体职能链当模板。
### 示例 A:合规改造任务(职能型候选池)
### 示例 A:合规改造任务(领域候选池)
**用户输入**:
```
@@ -330,12 +415,11 @@ RECOMMENDER_AGENT_SOUL = """# 智能体推荐专员
关键假设:现有系统具备基础数字化能力;允许预定窗口停机
候选智能体池:
- roundtable-intelligence (情报收集):检索同类历史案例、行业最佳实践与外部情报
- roundtable-environment (环境评估):评估硬件容量与系统量化承载力
- roundtable-solution-design (方案设计):产出主备方案与关键路径
- roundtable-risk-review (风险审查):合规红线检查与收敛建议
- roundtable-execution-plan (执行规划):里程碑与资源排布
- roundtable-summary (总结陈述):归纳冲突标记与盖印
- compliance-research (合规调研):检索同类历史案例、行业最佳实践与监管口径
- infra-capacity (基础设施评估):评估硬件容量与系统量化承载力
- solution-architect (方案架构):产出主备方案与关键路径
- risk-auditor (风险审查):合规红线检查与收敛建议
- execution-planner (执行规划):里程碑与资源排布
- my-cost-calc (成本测算):基于预算和资源单价计算总成本
```
@@ -346,10 +430,10 @@ RECOMMENDER_AGENT_SOUL = """# 智能体推荐专员
```json
{
"picks": [
{"agent_id": "roundtable-intelligence", "reason": "检索同类等保 2.0 改造案例与监管口径,直接支撑本次合规对标"},
{"agent_id": "roundtable-solution-design", "reason": "在 6 个月工期约束下产出主备改造方案与关键路径"},
{"agent_id": "roundtable-risk-review", "reason": "对照等保 2.0 合规红线审查方案,避免后期返工"},
{"agent_id": "roundtable-execution-plan", "reason": "把改造拆成可在 6 个月内交付的里程碑与资源排布"},
{"agent_id": "compliance-research", "reason": "检索同类等保 2.0 改造案例与监管口径,直接支撑本次合规对标"},
{"agent_id": "solution-architect", "reason": "在 6 个月工期约束下产出主备改造方案与关键路径"},
{"agent_id": "risk-auditor", "reason": "对照等保 2.0 合规红线审查方案,避免后期返工"},
{"agent_id": "execution-planner", "reason": "把改造拆成可在 6 个月内交付的里程碑与资源排布"},
{"agent_id": "my-cost-calc", "reason": "校验整体方案落在 500 万元预算硬约束内"}
]
}
@@ -368,7 +452,7 @@ RECOMMENDER_AGENT_SOUL = """# 智能体推荐专员
- channel-planner (渠道规划师):按预算和受众设计跨平台投放组合
- audience-insight (受众洞察):画像、关键词、情绪点研究
- creative-copy (创意文案):产出落地页与社媒文案变体
- roundtable-risk-review (风险审查):合规红线检查与收敛建议
- ad-compliance (广告合规):审查文案与素材合规风险
```
**你的输出**:
@@ -386,8 +470,8 @@ RECOMMENDER_AGENT_SOUL = """# 智能体推荐专员
}
```
> **关键观察**:示例 A 选了 `roundtable-intelligence / solution-design / execution-plan` 等"职能型"智能体,示例 B **完全没用任何 `roundtable-*` 内置**,而是从用户自建的领域智能体里挑。
> 这是因为:**候选池里的智能体能不能匹配任务,只看 description 和任务的实际语义,不看 id 是不是 `roundtable-` 开头**。如果候选池里有"广告设计智能体"且任务是广告设计,**你必须挑它**,而不是套用示例 A 的职能链。
> **关键观察**:两个示例都从候选池里按任务语义挑选,**不要**套用固定职能链(如"调研→设计→风险→执行")。
> 候选池里的智能体能不能匹配任务,只看 description 和任务的实际语义。如果候选池里有"广告设计智能体"且任务是广告设计,**你必须挑它**,而不是机械复制示例 A 的链路。
"""
@@ -2,6 +2,7 @@
from __future__ import annotations
import asyncio
import logging
import shutil
from datetime import datetime
@@ -9,16 +10,22 @@ from typing import Any
from uuid import uuid4
import yaml
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from pydantic import BaseModel, Field
from app.gateway.deps import get_agent_store, get_tag_store
from app.gateway.auth.password import verify_password_async
from app.gateway.deps import get_agent_store, get_config, get_tag_store
from deerflow.agents.lead_agent.prompt import refresh_skills_system_prompt_cache_async
from deerflow.config.agents_api_config import get_agents_api_config
from deerflow.config.agents_config import list_custom_agents, load_agent_config, load_agent_soul, validate_agent_id
from deerflow.config.app_config import AppConfig
from deerflow.config.paths import get_paths
from deerflow.config.system_settings import load_system_settings
from deerflow.persistence.agents import AgentStore
from deerflow.persistence.tags import TagStore
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.skills.cascade import sync_agent_skills_to_available
from deerflow.skills.storage import get_or_new_skill_storage
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["agents"])
@@ -32,6 +39,7 @@ class AgentResponse(BaseModel):
description: str = Field(default="", description="Agent description")
user_id: str | None = Field(default=None, description="Owner user id. Null means built-in.")
published: bool = Field(default=False, description="Whether a user-created agent is public")
is_private: bool = Field(default=False, description="Whether the agent lives in the password-gated private square instead of the public one")
builtin: bool = Field(default=False, description="True when user_id is null")
created_at: datetime | str | None = None
updated_at: datetime | str | None = None
@@ -68,6 +76,7 @@ class AgentCreateRequest(BaseModel):
description: str = Field(default="", description="Agent description")
soul: str = Field(default="", description="SOUL.md content - agent personality and behavioral guardrails")
published: bool = Field(default=False, description="Whether the new agent should be public")
is_private: bool = Field(default=False, description="When True, the new agent goes into the password-gated private square instead of the public one")
skills: list[str] | None = Field(default=None, description="Skill whitelist persisted to the agent's config.yaml")
model: str | None = Field(default=None, description="Optional model override persisted to config.yaml")
tool_groups: list[str] | None = Field(default=None, description="Optional tool group whitelist persisted to config.yaml")
@@ -80,6 +89,7 @@ class AgentUpdateRequest(BaseModel):
description: str | None = Field(default=None, description="Updated description")
soul: str | None = Field(default=None, description="Updated SOUL.md content")
published: bool | None = Field(default=None, description="Updated publication state")
is_private: bool | None = Field(default=None, description="Move the agent into / out of the private square (true / false)")
skills: list[str] | None = Field(default=None, description="Updated skill whitelist (config.yaml)")
model: str | None = Field(default=None, description="Updated model override (config.yaml)")
tool_groups: list[str] | None = Field(default=None, description="Updated tool group whitelist (config.yaml)")
@@ -261,6 +271,7 @@ def _record_to_response(
description=record.get("description") or "",
user_id=record.get("user_id"),
published=bool(record.get("published", False)),
is_private=bool(record.get("is_private", False)),
builtin=record.get("user_id") is None,
created_at=record.get("created_at"),
updated_at=record.get("updated_at"),
@@ -291,17 +302,21 @@ async def _get_editable_or_raise(
) -> tuple[dict[str, Any], str]:
"""Return (record, edit_mode) where edit_mode is 'owner' or 'admin'.
'admin' is granted only when the agent is built-in (user_id IS NULL) and the
caller is a system admin. Custom agents owned by other users still 403.
Admins may edit any agent — built-in (``user_id IS NULL``) or owned by
another user — so the admin console can manage the full agent catalog.
Non-admin callers are still restricted to agents they own.
"""
owned = await store.get_owned(agent_id, user_id)
if owned is not None:
return owned, "owner"
if is_admin:
record = await store.get_any(agent_id)
if record is None:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
return record, "admin"
visible = await store.get_visible(agent_id, user_id)
if visible is None:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
if is_admin and visible.get("user_id") is None:
return visible, "admin"
raise HTTPException(status_code=403, detail="You can only modify agents that you created")
@@ -315,6 +330,14 @@ async def list_agents(
request: Request,
search: str | None = Query(default=None, description="Case-insensitive fuzzy match on agent name"),
tag_ids: list[str] | None = Query(default=None, description="Keep agents carrying any of these tags (OR)"),
admin: bool = Query(
default=False,
description=(
"Admin-only override. When ``true`` and the caller is a system admin, "
"every agent is returned (built-in + every user's agents + private-square "
"entries) regardless of the usual ownership/visibility filter."
),
),
) -> AgentsListResponse:
_require_agents_api_enabled()
@@ -323,7 +346,12 @@ async def list_agents(
tag_store: TagStore = get_tag_store(request)
await _sync_legacy_agents(store)
user_id = _current_user_id(request)
agents = await store.list_visible(user_id)
if admin:
if not _is_admin_user(request):
raise HTTPException(status_code=403, detail="Only system admins can list all agents")
agents = await store.list_all()
else:
agents = await store.list_visible(user_id)
responses = [_record_to_response(a, include_soul=True) for a in agents]
assignments = await tag_store.list_assignments("agent", [r.id for r in responses])
@@ -446,6 +474,7 @@ async def create_agent_endpoint(request: AgentCreateRequest, http_request: Reque
"description": request.description or "",
"user_id": _current_user_id(http_request),
"published": request.published,
"is_private": request.is_private,
}
)
logger.info("Created agent '%s' (%s) at %s", name, agent_id, agent_dir)
@@ -486,6 +515,15 @@ async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request:
update_data["description"] = request.description or ""
if "published" in fields_set:
update_data["published"] = bool(request.published)
if "is_private" in fields_set:
update_data["is_private"] = bool(request.is_private)
# Mutual exclusion: an agent moved into the private square cannot
# also be publicly published. Force ``published=false`` whenever the
# same request sets ``is_private=true``, even if the client forgot
# to clear it. (When the request only sets one of the two, the
# other side is left untouched.)
if update_data.get("is_private") is True:
update_data["published"] = False
next_name = update_data.get("name", current["name"])
next_description = update_data.get("description", current.get("description") or "")
@@ -511,7 +549,10 @@ async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request:
if not update_data:
record = current
elif edit_mode == "admin":
record = await store.update_builtin(agent_id, update_data)
# Admins may edit built-in (``user_id IS NULL``) and any user-owned
# agent — ``update_any`` skips the ownership check that ``update``
# and ``update_builtin`` enforce.
record = await store.update_any(agent_id, update_data)
else:
record = await store.update(agent_id, user_id, update_data)
if record is None:
@@ -598,7 +639,7 @@ async def delete_agent(agent_id: str, request: Request) -> None:
)
if edit_mode == "admin":
deleted = await store.delete_builtin(agent_id)
deleted = await store.delete_any(agent_id)
else:
deleted = await store.delete(agent_id, user_id)
if not deleted:
@@ -662,6 +703,107 @@ async def set_agent_featured(
raise HTTPException(status_code=500, detail=f"Failed to set featured: {str(e)}") from e
class PrivateSquareListRequest(BaseModel):
"""Request body for listing all private-square agents."""
password: str = Field(..., description="Plain-text private-square password. Verified against the admin-configured hash before the listing is returned.")
class AgentPrivateUpdateRequest(BaseModel):
"""Admin-only request body for moving an agent into / out of the private square."""
is_private: bool = Field(..., description="Set true to move the agent into the private square, false to move it back into the public square.")
@router.post(
"/agents/private-square/list",
response_model=AgentsListResponse,
summary="List Private-Square Agents",
description=(
"Verify the admin-configured private-square password and return every "
"agent flagged ``is_private=true``. Returns 404 when no password is set, "
"403 when the supplied password does not match, and the full agent list "
"otherwise — regardless of ownership, since the password is the gate."
),
)
async def list_private_square(body: PrivateSquareListRequest, request: Request) -> AgentsListResponse:
_require_agents_api_enabled()
settings = load_system_settings().private_square
if not settings.password_hash:
raise HTTPException(status_code=404, detail="Private square password is not configured")
if not body.password or not await verify_password_async(body.password, settings.password_hash):
raise HTTPException(status_code=403, detail="Incorrect private square password")
try:
store = get_agent_store(request)
tag_store: TagStore = get_tag_store(request)
await _sync_legacy_agents(store)
records = await store.list_private()
responses = [_record_to_response(r, include_soul=True) for r in records]
assignments = await tag_store.list_assignments("agent", [r.id for r in responses])
for resp in responses:
resp.tags = assignments.get(resp.id, [])
return AgentsListResponse(agents=responses)
except HTTPException:
raise
except Exception as e:
logger.error("Failed to list private agents: %s", e, exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to list private agents: {str(e)}") from e
@router.put(
"/agents/{agent_id}/private",
response_model=AgentResponse,
summary="Move Agent Between Squares (admin)",
description=(
"Admin-only. Move any agent (built-in or user-owned) into or out of the "
"password-gated private square. Owners can already toggle the flag on "
"their own agents via the regular ``PUT /agents/{id}`` endpoint; this "
"endpoint exists so admins can re-categorize other users' agents too."
),
)
async def set_agent_private(
agent_id: str,
body: AgentPrivateUpdateRequest,
request: Request,
) -> AgentResponse:
_require_agents_api_enabled()
agent_id = _validate_storage_id(agent_id)
if not _is_admin_user(request):
raise HTTPException(status_code=403, detail="Only system admins can re-categorize other users' agents")
try:
store = get_agent_store(request)
await _sync_legacy_agents(store)
record = await store.get_any(agent_id)
if record is None:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
update: dict[str, Any] = {"is_private": bool(body.is_private)}
# Mutual exclusion — see ``update_agent`` above. Moving an agent into
# the private square also drops the public ``published`` flag so we
# never end up with an agent that's "in both squares".
if update["is_private"]:
update["published"] = False
if record.get("user_id") is None:
updated = await store.update_builtin(agent_id, update)
else:
updated = await store.update(agent_id, str(record["user_id"]), update)
if updated is None:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
logger.info("Admin set agent '%s' is_private=%s", agent_id, body.is_private)
return _record_to_response(updated, include_soul=True)
except HTTPException:
raise
except Exception as e:
logger.error("Failed to set is_private for agent '%s': %s", agent_id, e, exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to update agent: {str(e)}") from e
class AgentFeaturedReorderRequest(BaseModel):
"""Request body for admin to bulk-reorder the homepage featured list."""
@@ -705,3 +847,78 @@ async def reorder_featured_agents(
except Exception as e:
logger.error("Failed to reorder featured agents: %s", e, exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to reorder featured agents: {str(e)}") from e
class AgentSkillRemovalEntry(BaseModel):
agent_id: str = Field(..., description="Agent storage id whose config.yaml was rewritten")
agent_name: str | None = Field(default=None, description="Display name from the DB, if any")
user_id: str | None = Field(default=None, description="Owner user id (null = built-in)")
removed_skills: list[str] = Field(..., description="Skill names that no longer exist on disk and were stripped")
class AgentSkillsSyncResponse(BaseModel):
checked: int = Field(..., description="Number of agent config.yaml files scanned")
modified: int = Field(..., description="Number of agents whose skills list was rewritten")
removals: list[AgentSkillRemovalEntry] = Field(default_factory=list, description="Per-agent removal details")
@router.post(
"/agents/admin/sync-skills",
response_model=AgentSkillsSyncResponse,
summary="Sync Agent Skills (admin)",
description=(
"Admin-only. Scan every agent's ``config.yaml`` and strip skill names that "
"no longer exist on disk. Repairs historical drift where a skill was "
"removed without the delete/unpublish cascade firing (e.g. deleted from "
"the filesystem manually, or removed in an older version of the code)."
),
)
async def admin_sync_agent_skills(
request: Request,
config: AppConfig = Depends(get_config),
) -> AgentSkillsSyncResponse:
_require_agents_api_enabled()
if not _is_admin_user(request):
raise HTTPException(status_code=403, detail="Only system admins can run agent-skills sync")
try:
storage = get_or_new_skill_storage(app_config=config)
all_skills = storage.load_skills(enabled_only=False)
available = {skill.name for skill in all_skills}
result = await asyncio.to_thread(sync_agent_skills_to_available, available)
# Enrich removals with display name + owner from the DB so the admin
# UI can show "我的智能体 X 移除了技能 Y" instead of just the id.
store = get_agent_store(request)
enriched: list[AgentSkillRemovalEntry] = []
for entry in result.get("removals", []):
agent_id = entry["agent_id"]
record = await store.get_any(agent_id)
enriched.append(
AgentSkillRemovalEntry(
agent_id=agent_id,
agent_name=(record.get("name") if record else None),
user_id=(record.get("user_id") if record else None),
removed_skills=entry["removed_skills"],
)
)
if result.get("modified"):
await refresh_skills_system_prompt_cache_async()
logger.info(
"Admin agent-skills sync: checked=%d modified=%d",
result["checked"],
result["modified"],
)
return AgentSkillsSyncResponse(
checked=result["checked"],
modified=result["modified"],
removals=enriched,
)
except HTTPException:
raise
except Exception as e:
logger.error("Failed to sync agent skills: %s", e, exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to sync agent skills: {str(e)}") from e
@@ -93,6 +93,109 @@ _INTENT_READY_PATTERN = re.compile(
)
def _normalize_question(text: str) -> str:
"""归一化澄清问题文本,用于"和过往是否重复"的判断。
LLM 复述时常见模式:
- 完全一样的字符串
- 仅前后空白、标点、emoji 不同(如多/少一个问号)
- 仅大小写差异(中文场景几乎不见,但兼容英文/拼音)
所以归一化时统一:小写 + 去除非字母/数字/中文字符 + 折叠空白。
Levenshtein/语义相似度等模糊匹配先不做,等观察到误判再迭代。
"""
if not isinstance(text, str):
return ""
lowered = text.strip().lower()
# 只保留中英文字符与数字,去掉 ?。!?,。"":/、() 等标点 + emoji + 空白。
return re.sub(r"[^0-9a-z一-鿿]+", "", lowered)
def _normalize_options(raw: object) -> tuple[str, ...]:
"""把 options 拍平成可比较的归一化元组,用于"选项是否和上一轮一致"的判断。"""
if raw is None:
return ()
if isinstance(raw, str):
# 偶有模型把 list 序列化成 JSON 字符串发回来。
try:
raw = json.loads(raw)
except (json.JSONDecodeError, TypeError):
return (_normalize_question(raw),)
if not isinstance(raw, list):
return (_normalize_question(str(raw)),)
out: list[str] = []
for item in raw:
if isinstance(item, dict):
label = item.get("label") or item.get("title") or item.get("name") or item.get("value") or item.get("id") or ""
out.append(_normalize_question(str(label)))
else:
out.append(_normalize_question(str(item)))
# 顺序无关:LLM 偶尔会调换选项顺序,但语义上还是同一组。
return tuple(sorted(o for o in out if o))
def _is_repeated_clarification(
messages: list[dict[str, Any]],
*,
new_question: str,
new_options: object,
new_tool_call_id: str | None = None,
) -> bool:
"""检测本轮的 ask_clarification 是否在过往轮次问过。
判定为重复必须同时满足:
1. 历史里至少有一次 ``ask_clarification`` 的问题文本 / 选项 与本次归一化相同;
2. 用户在那次之后**已经发过消息**(即"已回答过")。
没有这条限制会把"模型在同一 run 内多次输出同一 tool_call_chunks"
误判成重复,导致首次澄清都吞掉。
3. 不是同一个 tool_call(LangGraph 偶有把同一 call 拆成多块返回的情况)。
传回 True → 上层应把 status 从 ``clarification`` 降级为 ``asking``,
避免前端弹同一张卡片(配合 SOUL §2.5 prompt 双层防御)。
"""
norm_new_q = _normalize_question(new_question)
norm_new_opts = _normalize_options(new_options)
if not norm_new_q and not norm_new_opts:
return False
seen_match_index: int | None = None
for idx, msg in enumerate(messages):
if not isinstance(msg, dict):
continue
if (msg.get("type") or "").lower() not in ("ai", "aimessage", "aimessagechunk"):
continue
for tc in msg.get("tool_calls") or []:
if (tc.get("name") or "").lower() != "ask_clarification":
continue
if new_tool_call_id and tc.get("id") == new_tool_call_id:
# 同一个 tool_call 的不同 chunk,不算重复。
continue
args = tc.get("args") or {}
past_q = _normalize_question(str(args.get("question") or ""))
past_opts = _normalize_options(args.get("options"))
# 问题文本和选项任一明确重叠就视为重复:
# - question 相同(options 可能空)
# - options 相同且非空(question 可能被改写)
q_match = bool(past_q) and past_q == norm_new_q
opt_match = bool(past_opts) and past_opts == norm_new_opts
if q_match or opt_match:
seen_match_index = idx
break
if seen_match_index is not None:
break
if seen_match_index is None:
return False
# 校验用户在该次澄清之后是否已经发过 human 消息(= 回答过)。
for msg in messages[seen_match_index + 1:]:
if not isinstance(msg, dict):
continue
msg_type = (msg.get("type") or "").lower()
if msg_type in ("human", "humanmessage", "user"):
return True
return False
@router.post("/init")
async def init_intent(request: Request, payload: dict = Body(...)) -> dict[str, Any]:
"""创建任务理解会话用的空 thread。
@@ -264,6 +367,34 @@ async def stream_intent(request: Request, payload: dict = Body(...)) -> Streamin
# 比 tool_call 旁的短 preamble 更适合作为气泡正文。
follow_up = _find_trailing_ai_text(messages, exclude_text=content)
display_content = follow_up or content or question
# 兜底:模型偶尔会复述上一轮的同一澄清问题(SOUL §2.5 prompt
# 已经强约束,但小模型 / 长上下文仍可能漏跟随)。检测到完全
# 重复时,把 status 从 clarification 降级为 asking,前端只渲染
# 文本气泡不弹卡片,避免用户看到"同样选项又出现一次"。
if _is_repeated_clarification(
messages,
new_question=question,
new_options=cargs.get("options"),
new_tool_call_id=clarification_call.get("id"),
):
logger.warning(
"intent stream: dropping duplicate ask_clarification "
"(question=%r) — already asked & answered earlier in thread",
question[:80],
)
fallback_msg = (
display_content
or "(系统识别到智能体重复了上一问题,已自动跳过。请补充其他信息继续推进。)"
)
final_frame: dict[str, Any] = {
"status": "asking",
"content": fallback_msg,
"summary": None,
}
yield f"data: {json.dumps(final_frame, ensure_ascii=False)}\n\n"
return
final_frame: dict[str, Any] = {
"status": "clarification",
"content": display_content,
@@ -182,10 +182,13 @@ async def update_mcp_configuration(request: McpConfigUpdateRequest, http_request
current_config = get_extensions_config()
# Convert request to dict format for JSON serialization
config_data = {
"mcpServers": {name: server.model_dump() for name, server in request.mcp_servers.items()},
"skills": {name: {"enabled": skill.enabled} for name, skill in current_config.skills.items()},
}
config_data = {
"mcpServers": {name: server.model_dump(by_alias=True) for name, server in request.mcp_servers.items()},
"skills": {
name: skill.model_dump(by_alias=True, exclude_none=True)
for name, skill in current_config.skills.items()
},
}
# Write the configuration to file
with open(config_path, "w", encoding="utf-8") as f:
@@ -764,25 +764,30 @@ def _find_trailing_ai_text(messages: list[dict[str, Any]], *, exclude_text: str
def _find_dispatch_message(messages: list[dict[str, Any]]) -> dict[str, Any]:
"""Locate the latest AIMessage that carries dispatch tool calls.
"""Locate the latest AIMessage from THIS turn.
The very last message can be a skill-stop ToolMessage injected by
``SkillStopMiddleware`` when it intercepts the dispatch, or a clarification
ToolMessage. Either way, the actionable tool_calls live on the AIMessage
*before* it, so we walk backwards until we hit one with ``tool_calls``.
Returns the latest AIMessage when none of them carry tool_calls so the
caller can still extract its text content.
Trailing ToolMessages (skill_stop / clarification ack injected by
``SkillStopMiddleware`` or ``ClarificationMiddleware``) are skipped because
the actionable tool_calls live on the AIMessage that emitted them. We stop
at the **first AIMessage encountered** — its ``tool_calls`` (or lack
thereof) is the authoritative signal for this turn.
历史踩坑:之前实现是"一直往前翻直到撞到带 tool_calls 的 AI",看似优雅,
实际上让 leader 永远收敛不了:本轮 LLM 不论怎么共识,只要历史上派过活,
上一轮的派活 AI 就会被翻出来再用一次,前端会把同样的 task 又发给子智能体,
陷入死循环。现在停在最新一条 AI 上,把判定交给调用方:`tool_calls=[]` →
共识分支,有 tool_calls → 派活分支。
"""
last_ai: dict[str, Any] = {}
for msg in reversed(messages):
if not isinstance(msg, dict):
continue
if (msg.get("type") or "").lower() in ("ai", "aimessage", "aimessagechunk"):
if not last_ai:
last_ai = msg
if msg.get("tool_calls"):
return msg
return last_ai
msg_type = (msg.get("type") or "").lower()
if msg_type in ("tool", "toolmessage"):
continue # 跳过末尾 skill_stop / clarification ack
if msg_type in ("ai", "aimessage", "aimessagechunk"):
return msg
break # 撞到 Human/其他类型,本轮没 AI(异常路径,留空让上游报错)
return {}
async def _leader_run(
File diff suppressed because it is too large. Load diff
@@ -3,10 +3,12 @@
from __future__ import annotations
import logging
from typing import Literal
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field
from app.gateway.auth.password import hash_password_async
from app.gateway.deps import get_optional_user_from_request
from deerflow.config.system_settings import (
PromptPrefixSettings,
@@ -31,6 +33,30 @@ class PromptPrefixUpdateRequest(BaseModel):
prompt_prefix: str | None = None
class PrivateSquareStatusResponse(BaseModel):
"""Status of the private-square feature, readable by any logged-in user.
The password hash itself is **never** returned — the response only signals
whether the feature is enabled and whether a password has been set.
"""
enabled: bool = Field(..., description="Whether the private square is exposed to users")
password_set: bool = Field(..., description="Whether a password is currently configured")
class PrivateSquareUpdateRequest(BaseModel):
enabled: bool | None = Field(default=None, description="Toggle the private-square feature")
password: str | None = Field(default=None, description="New password in plain text. Pass an empty string to clear.")
class CitationDisplayResponse(BaseModel):
cleanup_words: list[str] = Field(default_factory=list, description="Global citation cleanup words")
class CitationDisplayUpdateRequest(BaseModel):
cleanup_words: list[str] | None = None
def _to_response(settings: PromptPrefixSettings) -> PromptPrefixResponse:
return PromptPrefixResponse(
enabled=settings.enabled,
@@ -61,5 +87,116 @@ async def update_prompt_prefix(body: PromptPrefixUpdateRequest, request: Request
current = load_system_settings()
fields = body.model_dump(exclude_unset=True)
updated_prefix = current.prompt_prefix.model_copy(update=fields)
saved = save_system_settings(SystemSettings(prompt_prefix=updated_prefix))
saved = save_system_settings(current.model_copy(update={"prompt_prefix": updated_prefix}))
return _to_response(saved.prompt_prefix)
@router.get("/citation-display", response_model=CitationDisplayResponse)
async def get_citation_display() -> CitationDisplayResponse:
"""Return global citation display cleanup settings."""
settings = load_system_settings().citation_display
return CitationDisplayResponse(cleanup_words=settings.cleanup_words)
@router.put("/citation-display", response_model=CitationDisplayResponse)
async def update_citation_display(
body: CitationDisplayUpdateRequest,
request: Request,
) -> CitationDisplayResponse:
"""Update global citation display cleanup settings. Admin only."""
await _require_admin(request)
current = load_system_settings()
words = [
str(item).strip()
for item in (body.cleanup_words or [])
if str(item).strip()
]
updated = current.citation_display.model_copy(update={"cleanup_words": words})
saved = save_system_settings(
current.model_copy(update={"citation_display": updated})
)
return CitationDisplayResponse(cleanup_words=saved.citation_display.cleanup_words)
@router.get("/private-square", response_model=PrivateSquareStatusResponse)
async def get_private_square_status() -> PrivateSquareStatusResponse:
"""Return whether the private square is enabled and a password is set."""
settings = load_system_settings().private_square
return PrivateSquareStatusResponse(
enabled=settings.enabled,
password_set=bool(settings.password_hash),
)
@router.put("/private-square", response_model=PrivateSquareStatusResponse)
async def update_private_square(body: PrivateSquareUpdateRequest, request: Request) -> PrivateSquareStatusResponse:
"""Update the private-square configuration. Admin only.
Setting ``password`` to a non-empty string rehashes and stores it.
Passing an empty string clears the password (effectively locking the
private square so no one can read it).
"""
await _require_admin(request)
current = load_system_settings()
updated = current.private_square
fields = body.model_dump(exclude_unset=True)
update: dict[str, object] = {}
if "enabled" in fields and fields["enabled"] is not None:
update["enabled"] = bool(fields["enabled"])
if "password" in fields and fields["password"] is not None:
plain = str(fields["password"])
if plain == "":
update["password_hash"] = ""
else:
update["password_hash"] = await hash_password_async(plain)
if update:
updated = updated.model_copy(update=update)
saved = save_system_settings(current.model_copy(update={"private_square": updated}))
else:
saved = current
return PrivateSquareStatusResponse(
enabled=saved.private_square.enabled,
password_set=bool(saved.private_square.password_hash),
)
class AppearanceResponse(BaseModel):
"""Effective system-wide appearance defaults."""
default_theme: Literal["light", "dark"] = Field(..., description="System default theme — light or dark")
class AppearanceUpdateRequest(BaseModel):
default_theme: Literal["light", "dark"] | None = Field(default=None)
@router.get("/appearance", response_model=AppearanceResponse)
async def get_appearance_settings() -> AppearanceResponse:
"""Return the global default theme. Any logged-in user may read this."""
settings = load_system_settings().appearance
return AppearanceResponse(default_theme=settings.default_theme)
@router.put("/appearance", response_model=AppearanceResponse)
async def update_appearance_settings(
body: AppearanceUpdateRequest, request: Request
) -> AppearanceResponse:
"""Update the global appearance defaults. Admin only."""
await _require_admin(request)
current = load_system_settings()
fields = body.model_dump(exclude_unset=True)
update: dict[str, object] = {}
if "default_theme" in fields and fields["default_theme"] is not None:
update["default_theme"] = fields["default_theme"]
if update:
updated_appearance = current.appearance.model_copy(update=update)
saved = save_system_settings(
current.model_copy(update={"appearance": updated_appearance})
)
else:
saved = current
return AppearanceResponse(default_theme=saved.appearance.default_theme)
@@ -12,9 +12,12 @@ matching the LangGraph Platform wire format expected by the
from __future__ import annotations
import logging
import uuid
from typing import Any
import logging
import json
import os
from pathlib import Path
import uuid
from typing import Any
from fastapi import APIRouter, HTTPException, Request
from langgraph.checkpoint.base import empty_checkpoint
@@ -23,13 +26,18 @@ from pydantic import BaseModel, Field, field_validator
from app.gateway.authz import require_permission
from app.gateway.deps import get_checkpointer
from app.gateway.utils import sanitize_log_param
from deerflow.config.paths import Paths, get_paths
from deerflow.runtime import serialize_channel_values
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.utils.time import coerce_iso, now_iso
from deerflow.config.paths import Paths, get_paths
from deerflow.runtime import serialize_channel_values
from deerflow.runtime.references import (
build_reference_batches as build_runtime_reference_batches,
build_reference_debug_summary,
)
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.utils.time import coerce_iso, now_iso
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["threads"])
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["threads"])
_RAG_DEBUG_LOGGER_NAME = "rag_citation_debug"
# Metadata keys that the server controls; clients are not allowed to set
@@ -41,11 +49,38 @@ router = APIRouter(prefix="/api/threads", tags=["threads"])
_SERVER_RESERVED_METADATA_KEYS: frozenset[str] = frozenset({"owner_id", "user_id"})
def _strip_reserved_metadata(metadata: dict[str, Any] | None) -> dict[str, Any]:
"""Return ``metadata`` with server-controlled keys removed."""
if not metadata:
return metadata or {}
return {k: v for k, v in metadata.items() if k not in _SERVER_RESERVED_METADATA_KEYS}
def _strip_reserved_metadata(metadata: dict[str, Any] | None) -> dict[str, Any]:
"""Return ``metadata`` with server-controlled keys removed."""
if not metadata:
return metadata or {}
return {k: v for k, v in metadata.items() if k not in _SERVER_RESERVED_METADATA_KEYS}
def _get_rag_debug_logger() -> logging.Logger:
debug_logger = logging.getLogger(_RAG_DEBUG_LOGGER_NAME)
if getattr(debug_logger, "_rag_file_configured", False):
return debug_logger
log_path = Path(os.environ.get("RAG_CITATION_DEBUG_LOG", "logs/rag_citation_debug.log"))
if not log_path.is_absolute():
log_path = Path.cwd() / log_path
log_path.parent.mkdir(parents=True, exist_ok=True)
handler = logging.FileHandler(log_path, encoding="utf-8")
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
debug_logger.addHandler(handler)
debug_logger.setLevel(logging.INFO)
debug_logger.propagate = False
setattr(debug_logger, "_rag_file_configured", True)
setattr(debug_logger, "_rag_file_path", str(log_path))
return debug_logger
def _rag_debug_file_path() -> str:
return str(getattr(_get_rag_debug_logger(), "_rag_file_path", "logs/rag_citation_debug.log"))
def _log_rag_debug(message: str, *args: Any) -> None:
logger.info(message, *args)
_get_rag_debug_logger().info(message, *args)
# ---------------------------------------------------------------------------
@@ -132,11 +167,43 @@ class HistoryEntry(BaseModel):
next: list[str] = Field(default_factory=list)
class ThreadHistoryRequest(BaseModel):
class ThreadHistoryRequest(BaseModel):
"""Request body for checkpoint history."""
limit: int = Field(default=10, ge=1, le=100, description="Maximum entries")
before: str | None = Field(default=None, description="Cursor for pagination")
before: str | None = Field(default=None, description="Cursor for pagination")
class ReferenceSourceResponse(BaseModel):
index: int
id: str | None = None
skillName: str = "backend-reference"
mode: str = "citation"
title: str
type: str = "result"
snippet: str = ""
source: str | None = None
time: str | None = None
url: str | None = None
content: str | None = None
score: float | None = None
raw: dict[str, Any] = Field(default_factory=dict)
class ReferenceBatchResponse(BaseModel):
id: str
assistant_message_id: str
sources: list[ReferenceSourceResponse] = Field(default_factory=list)
class ReferenceBatchesResponse(BaseModel):
batches: list[ReferenceBatchResponse] = Field(default_factory=list)
class ReferenceDebugLogRequest(BaseModel):
event: str
thread_id: str | None = None
payload: dict[str, Any] = Field(default_factory=dict)
# ---------------------------------------------------------------------------
@@ -163,7 +230,7 @@ def _delete_thread_data(thread_id: str, paths: Paths | None = None, *, user_id:
return ThreadDeleteResponse(success=True, message=f"Deleted local thread data for {thread_id}")
def _derive_thread_status(checkpoint_tuple) -> str:
def _derive_thread_status(checkpoint_tuple) -> str:
"""Derive thread status from checkpoint metadata."""
if checkpoint_tuple is None:
return "idle"
@@ -179,7 +246,251 @@ def _derive_thread_status(checkpoint_tuple) -> str:
if tasks:
return "interrupted"
return "idle"
return "idle"
def _message_type(message: Any) -> str:
if isinstance(message, dict):
return str(message.get("type") or message.get("role") or "")
return str(getattr(message, "type", "") or "")
def _message_id(message: Any) -> str | None:
if isinstance(message, dict):
value = message.get("id")
else:
value = getattr(message, "id", None)
return str(value) if value else None
def _message_content(message: Any) -> Any:
if isinstance(message, dict):
return message.get("content")
return getattr(message, "content", None)
def _message_name(message: Any) -> str:
if isinstance(message, dict):
return str(message.get("name") or (message.get("additional_kwargs") or {}).get("name") or "tool")
return str(getattr(message, "name", None) or "tool")
def _has_tool_calls(message: Any) -> bool:
if isinstance(message, dict):
direct = message.get("tool_calls")
chunks = message.get("tool_call_chunks")
extra = (message.get("additional_kwargs") or {}).get("tool_calls")
else:
direct = getattr(message, "tool_calls", None)
chunks = getattr(message, "tool_call_chunks", None)
extra = (getattr(message, "additional_kwargs", None) or {}).get("tool_calls")
return bool(direct or chunks or extra)
def _has_answer_content(message: Any) -> bool:
content = _message_content(message)
if isinstance(content, str):
return bool(content.strip())
if isinstance(content, list):
for part in content:
if isinstance(part, dict) and part.get("type") == "text" and str(part.get("text") or "").strip():
return True
return False
def _extract_tool_payload(message: Any) -> Any:
content = _message_content(message)
if isinstance(content, dict):
return _unwrap_tool_payload(content)
if isinstance(content, list):
texts: list[str] = []
for part in content:
if not isinstance(part, dict):
continue
if isinstance(part.get("json"), dict):
return _unwrap_tool_payload(part["json"])
if part.get("type") == "json" and isinstance(part.get("json"), dict):
return _unwrap_tool_payload(part["json"])
if part.get("type") == "text" and isinstance(part.get("text"), str):
texts.append(part["text"])
unwrapped = _unwrap_tool_payload(part)
if unwrapped is not None:
return unwrapped
content = "\n".join(texts)
if isinstance(content, str):
return _parse_json_like(content)
return None
def _unwrap_tool_payload(value: dict[str, Any]) -> Any:
if any(isinstance(value.get(key), list) for key in ("results", "data", "items", "hits", "documents", "records", "sources")):
return value
for key in ("result", "output", "stdout", "content", "text", "payload", "response"):
nested = value.get(key)
if isinstance(nested, str):
parsed = _parse_json_like(nested)
if parsed is not None:
return parsed
if isinstance(nested, dict):
unwrapped = _unwrap_tool_payload(nested)
if unwrapped is not None:
return unwrapped
return value
def _parse_json_like(text: str) -> Any:
trimmed = text.strip()
if not trimmed:
return None
try:
return json.loads(trimmed)
except Exception:
pass
first_obj = trimmed.find("{")
last_obj = trimmed.rfind("}")
if 0 <= first_obj < last_obj:
try:
return json.loads(trimmed[first_obj : last_obj + 1])
except Exception:
return None
return None
def _first_list(payload: Any) -> list[Any]:
if isinstance(payload, list):
return payload
if not isinstance(payload, dict):
return []
for key in ("results", "data", "items", "hits", "documents", "records", "entries", "matches", "chunks", "sources"):
value = payload.get(key)
if isinstance(value, list):
return value
for key in ("result", "payload", "response", "output", "data"):
nested = payload.get(key)
found = _first_list(nested)
if found:
return found
return []
def _nested_records(raw: Any) -> list[dict[str, Any]]:
if not isinstance(raw, dict):
return []
layers = [raw]
for key in ("metadata", "meta", "attrs", "attributes", "properties", "extra"):
nested = raw.get(key)
if isinstance(nested, dict):
layers.append(nested)
return layers
def _pick(layers: list[dict[str, Any]], keys: tuple[str, ...]) -> str | None:
for layer in layers:
for key in keys:
value = layer.get(key)
if isinstance(value, (str, int, float)) and str(value).strip():
return str(value).strip()
return None
def _normalize_reference_item(raw: Any, fallback_index: int) -> dict[str, Any] | None:
if not isinstance(raw, dict):
return None
layers = _nested_records(raw)
title = _pick(layers, ("title", "m_title", "name", "question", "keyword", "query", "text"))
content = _pick(layers, ("content", "page_content", "summary", "snippet", "description", "content_preview", "abstract", "body"))
url = _pick(layers, ("url", "link", "href", "source_url", "web_url"))
source = _pick(layers, ("source", "source1", "site", "provider", "from"))
time = _pick(layers, ("time", "date", "published_date", "publishedDate", "m_publish", "publish_time", "created_at"))
item_id = _pick(layers, ("recUuid", "rec_uuid", "uuid", "docId", "doc_id", "id", "recordId", "taskId"))
score_raw = _pick(layers, ("score", "relevance", "similarity"))
score = None
if score_raw is not None:
try:
score = float(score_raw)
except Exception:
score = None
if not title:
title = source or f"结果 {fallback_index}"
if not content:
content = ""
if not title and not content and not url:
return None
return {
"id": item_id or url,
"title": title,
"type": "result",
"snippet": content,
"content": content,
"source": source,
"time": time,
"url": url,
"score": score,
"raw": raw,
}
def _reference_key(source: dict[str, Any]) -> str:
raw = source.get("raw") if isinstance(source.get("raw"), dict) else {}
item_id = (
source.get("id")
or source.get("url")
or raw.get("recUuid")
or raw.get("uuid")
or raw.get("docId")
or raw.get("id")
)
if item_id:
return str(item_id).casefold().strip()
return "|".join(
str(source.get(key) or "").casefold().strip()
for key in ("title", "source", "time", "content", "snippet")
)
def _build_reference_batches(messages: list[Any]) -> list[ReferenceBatchResponse]:
batches: list[ReferenceBatchResponse] = []
for index, message in enumerate(messages):
if _message_type(message) != "human":
continue
end = len(messages)
for cursor in range(index + 1, len(messages)):
if _message_type(messages[cursor]) == "human":
end = cursor
break
assistant_message: Any | None = None
for cursor in range(end - 1, index, -1):
candidate = messages[cursor]
if (
_message_type(candidate) == "ai"
and _message_id(candidate)
and _has_answer_content(candidate)
and not _has_tool_calls(candidate)
):
assistant_message = candidate
break
if assistant_message is None:
continue
seen: set[str] = set()
sources: list[ReferenceSourceResponse] = []
for tool_message in messages[index + 1 : end]:
if _message_type(tool_message) != "tool":
continue
payload = _extract_tool_payload(tool_message)
for raw_item in _first_list(payload):
normalized = _normalize_reference_item(raw_item, len(sources) + 1)
if not normalized:
continue
key = _reference_key(normalized)
if key and key in seen:
continue
if key:
seen.add(key)
sources.append(ReferenceSourceResponse(index=len(sources) + 1, **normalized))
if sources:
mid = _message_id(assistant_message) or str(index)
batches.append(ReferenceBatchResponse(id=mid, assistant_message_id=mid, sources=sources))
return batches
# ---------------------------------------------------------------------------
@@ -406,9 +717,9 @@ async def get_thread(thread_id: str, request: Request) -> ThreadResponse:
# ---------------------------------------------------------------------------
@router.get("/{thread_id}/state", response_model=ThreadStateResponse)
@require_permission("threads", "read", owner_check=True)
async def get_thread_state(thread_id: str, request: Request) -> ThreadStateResponse:
@router.get("/{thread_id}/state", response_model=ThreadStateResponse)
@require_permission("threads", "read", owner_check=True)
async def get_thread_state(thread_id: str, request: Request) -> ThreadStateResponse:
"""Get the latest state snapshot for a thread.
Channel values are serialized to ensure LangChain message objects
@@ -455,12 +766,84 @@ async def get_thread_state(thread_id: str, request: Request) -> ThreadStateRespo
parent_checkpoint_id=parent_checkpoint_id,
created_at=coerce_iso(metadata.get("created_at", "")),
tasks=tasks,
)
@router.post("/{thread_id}/state", response_model=ThreadStateResponse)
@require_permission("threads", "write", owner_check=True, require_existing=True)
async def update_thread_state(thread_id: str, body: ThreadStateUpdateRequest, request: Request) -> ThreadStateResponse:
)
@router.get("/{thread_id}/reference-batches", response_model=ReferenceBatchesResponse)
@require_permission("threads", "read", owner_check=True)
async def get_thread_reference_batches(thread_id: str, request: Request) -> ReferenceBatchesResponse:
"""Return backend-built citation batches for completed assistant turns."""
checkpointer = get_checkpointer(request)
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
try:
checkpoint_tuple = await checkpointer.aget_tuple(config)
except Exception:
logger.exception("Failed to get reference batches for thread %s", sanitize_log_param(thread_id))
raise HTTPException(status_code=500, detail="Failed to get reference batches")
if checkpoint_tuple is None:
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {}
values = serialize_channel_values(checkpoint.get("channel_values", {}) or {})
messages = values.get("messages")
if not isinstance(messages, list):
_log_rag_debug(
"backend reference-batches thread=%s messages=missing batches=0 log_file=%s",
sanitize_log_param(thread_id),
_rag_debug_file_path(),
)
return ReferenceBatchesResponse(batches=[])
raw_batches = build_runtime_reference_batches(messages)
debug_summary = build_reference_debug_summary(messages, raw_batches)
_log_rag_debug(
"backend reference-batches thread=%s messages=%s turns=%s tool_messages=%s raw_items=%s "
"normalized_items=%s kept_items=%s batches=%s batch_sources=%s batch_ids=%s turns_detail=%s log_file=%s",
sanitize_log_param(thread_id),
debug_summary["messages"],
debug_summary["turns"],
debug_summary["tool_messages"],
debug_summary["raw_items"],
debug_summary["normalized_items"],
debug_summary["kept_items"],
len(raw_batches),
",".join(str(batch.get("source_count", 0)) for batch in debug_summary["batches"]),
",".join(str(batch.get("id") or "") for batch in debug_summary["batches"]),
json.dumps(debug_summary["turns_detail"], ensure_ascii=False),
_rag_debug_file_path(),
)
return ReferenceBatchesResponse(
batches=[
ReferenceBatchResponse(
id=str(batch.get("id") or ""),
assistant_message_id=str(batch.get("assistant_message_id") or batch.get("id") or ""),
sources=[
ReferenceSourceResponse(**source)
for source in batch.get("sources", [])
if isinstance(source, dict)
],
)
for batch in raw_batches
]
)
@router.post("/reference-debug-log")
async def write_reference_debug_log(body: ReferenceDebugLogRequest) -> dict[str, str | bool]:
"""Append frontend citation diagnostics to the backend RAG debug log."""
_get_rag_debug_logger().info(
"frontend %s thread=%s payload=%s log_file=%s",
body.event,
sanitize_log_param(body.thread_id or ""),
json.dumps(body.payload, ensure_ascii=False, default=str),
_rag_debug_file_path(),
)
return {"ok": True, "log_file": _rag_debug_file_path()}
@router.post("/{thread_id}/state", response_model=ThreadStateResponse)
@require_permission("threads", "write", owner_check=True, require_existing=True)
async def update_thread_state(thread_id: str, body: ThreadStateUpdateRequest, request: Request) -> ThreadStateResponse:
"""Update thread state (e.g. for human-in-the-loop resume or title rename).
Writes a new checkpoint that merges *body.values* into the latest
@@ -100,7 +100,14 @@ _DEFAULT_ASSISTANT_ID = "lead_agent"
# Built-in LangGraph graph IDs registered in langgraph.json.
# These are NOT stored in the agents DB and must bypass the custom-agent access check.
_BUILTIN_GRAPH_IDS: frozenset[str] = frozenset({"lead_agent", "canvas_agent", "ai_writing"})
_BUILTIN_GRAPH_IDS: frozenset[str] = frozenset({"lead_agent", "canvas_agent", "ai_writing"})
_RAG_CITATION_PROMPT = """当前对话已开启参考文献模式。
如果本轮调用了搜索、知识库、检索类技能或工具,请基于本轮所有检索结果整体回答。
你必须把同一轮内多次检索得到的资料视为同一批参考文献,按出现顺序从 [1] 开始连续编号,不要让每个检索词各自从 [1] 重新编号。
回答正文中的关键结论必须标注对应编号,例如 [1]、[2],多个来源可写为 [1][3][5]。
不要引用不存在的编号,不要使用技能内部自带的编号,不要把参考文献完整列表堆在正文末尾。
如果没有任何可用检索结果,请直接说明未检索到可靠参考资料。"""
def _prepend_prompt_prefix(graph_input: dict[str, Any], prefix: str) -> dict[str, Any]:
@@ -129,7 +136,7 @@ def _prepend_prompt_prefix(graph_input: dict[str, Any], prefix: str) -> dict[str
return {**graph_input, "messages": new_messages}
def _prompt_prefix_opted_in(body: Any) -> bool:
def _prompt_prefix_opted_in(body: Any) -> bool:
"""Detect whether the caller asked for the prompt prefix to be applied."""
context = getattr(body, "context", None)
if isinstance(context, Mapping) and context.get("prompt_prefix_enabled"):
@@ -137,7 +144,44 @@ def _prompt_prefix_opted_in(body: Any) -> bool:
metadata = getattr(body, "metadata", None)
if isinstance(metadata, Mapping) and metadata.get("prompt_prefix_enabled"):
return True
return False
return False
def _rag_mode_requested(body: Any) -> bool:
context = getattr(body, "context", None)
if isinstance(context, Mapping) and context.get("rag_mode_enabled"):
return True
metadata = getattr(body, "metadata", None)
if isinstance(metadata, Mapping) and (
metadata.get("rag_mode_enabled") or metadata.get("rag_mode")
):
return True
return False
def _thread_rag_mode_enabled(record: Any) -> bool:
if not isinstance(record, Mapping):
return False
metadata = record.get("metadata")
return isinstance(metadata, Mapping) and bool(
metadata.get("rag_mode_enabled") or metadata.get("rag_mode")
)
def _append_invisible_prompt(graph_input: dict[str, Any], prompt: str) -> dict[str, Any]:
messages = graph_input.get("messages")
if not isinstance(messages, list) or not messages:
return graph_input
new_messages = list(messages)
for idx, msg in enumerate(new_messages):
if not isinstance(msg, HumanMessage):
continue
extra = dict(getattr(msg, "additional_kwargs", None) or {})
existing = str(extra.get("prompt_prefix") or "").strip()
extra["prompt_prefix"] = f"{existing}\n\n{prompt}".strip() if existing else prompt
new_messages[idx] = msg.model_copy(update={"additional_kwargs": extra})
break
return {**graph_input, "messages": new_messages}
# Whitelist of run-context keys that the langgraph-compat layer forwards from
@@ -152,8 +196,9 @@ _CONTEXT_CONFIGURABLE_KEYS: frozenset[str] = frozenset(
"model_name",
"mode",
"thinking_enabled",
"reasoning_effort",
"writing_mode",
"reasoning_effort",
"rag_mode_enabled",
"writing_mode",
"writing_artifact_path",
"canvas_artifact_title",
"canvas_artifact_type",
@@ -398,16 +443,29 @@ async def start_run(
# Upsert thread metadata so the thread appears in /threads/search,
# even for threads that were never explicitly created via POST /threads
# (e.g. stateless runs).
try:
existing = await run_ctx.thread_store.get(thread_id)
if existing is None:
await run_ctx.thread_store.create(
thread_id,
assistant_id=body.assistant_id,
metadata=body.metadata,
)
else:
await run_ctx.thread_store.update_status(thread_id, "running")
requested_rag_mode = _rag_mode_requested(body)
effective_rag_mode = requested_rag_mode
try:
existing = await run_ctx.thread_store.get(thread_id)
effective_rag_mode = requested_rag_mode or _thread_rag_mode_enabled(existing)
if existing is None:
metadata = dict(body.metadata or {})
if effective_rag_mode:
metadata["rag_mode_enabled"] = True
metadata["rag_mode"] = True
await run_ctx.thread_store.create(
thread_id,
assistant_id=body.assistant_id,
metadata=metadata,
)
else:
await run_ctx.thread_store.update_status(thread_id, "running")
if requested_rag_mode and not _thread_rag_mode_enabled(existing):
await run_ctx.thread_store.update_metadata(
thread_id,
{"rag_mode_enabled": True, "rag_mode": True},
)
effective_rag_mode = True
# SDK 的 ``client.threads.create()`` 不传 assistant_id —— 先建出来空的 thread,
# 真正提交 run 时才知道用哪个 assistant。这里回填一次,避免历史列表 / 按
# assistant 筛选时拿不到关联。
@@ -427,17 +485,21 @@ async def start_run(
# applies once. We intentionally do NOT gate on thread-row existence: the
# thread_meta row is frequently pre-created, which previously skipped the
# prefix for genuine first turns.
if _prompt_prefix_opted_in(body):
prefix_settings = load_system_settings().prompt_prefix
if prefix_settings.enabled and prefix_settings.prompt_prefix.strip():
graph_input = _prepend_prompt_prefix(graph_input, prefix_settings.prompt_prefix.strip())
config = build_run_config(thread_id, body.config, body.metadata, assistant_id=body.assistant_id)
if _prompt_prefix_opted_in(body):
prefix_settings = load_system_settings().prompt_prefix
if prefix_settings.enabled and prefix_settings.prompt_prefix.strip():
graph_input = _prepend_prompt_prefix(graph_input, prefix_settings.prompt_prefix.strip())
if effective_rag_mode:
graph_input = _append_invisible_prompt(graph_input, _RAG_CITATION_PROMPT)
config = build_run_config(thread_id, body.config, body.metadata, assistant_id=body.assistant_id)
# Merge DeerFlow-specific context overrides into both ``configurable`` and ``context``.
# The ``context`` field is a custom extension for the langgraph-compat layer
# that carries agent configuration (model_name, thinking_enabled, etc.).
# Only agent-relevant keys are forwarded; unknown keys (e.g. thread_id) are ignored.
body_context = dict(getattr(body, "context", None) or {})
body_context = dict(getattr(body, "context", None) or {})
if effective_rag_mode:
body_context["rag_mode_enabled"] = True
# Promote whitelisted top-level body fields into the context map so
# callers can pass them either nested under ``context`` or as flat
# top-level fields (``excluded_tools``, ``skill_stop_names``, …).
+6 -16
View File
@@ -10,12 +10,12 @@ gateway:
port: 8001
models:
- name: deepseek-chat
display_name: DeepSeek Chat
- name: zai-org/GLM-5-FP8
display_name: zai-org/GLM-5-FP8
use: langchain_openai:ChatOpenAI
model: deepseek-chat
api_key: sk-e1c22b7931dc446f8b67e57878edd105
base_url: https://api.deepseek.com
model: zai-org/GLM-5-FP8
api_key: Bearer modalresearch_gWu46FDpVLYZWPVEA3b_2fq8Jw2UJiKMgEm_Rxmn1SA
base_url: https://api.us-west-2.modal.direct/v1
request_timeout: 600.0
max_retries: 2
max_tokens: 4096
@@ -58,16 +58,6 @@ models:
temperature: 1.0
supports_vision: true
supports_thinking: true
- name: minimax-m2.5
display_name: MiniMax M2.5
use: langchain_openai:ChatOpenAI
model: MiniMax-M2.5
api_key: sk-cp-mrsAGtzDpUl0TpOBPVoiRG_pWF1kDDN9_mDAnONBs6HIPcNUexSKtSbx2QOxfKBvJT6Xw6hDTV6hdkJYFWfNA8UtHQ08TiGJiosdRoW6q1sGb6Cb7D85NHk
base_url: https://api.minimaxi.com/v1
max_tokens: 4096
temperature: 1.0
supports_vision: true
supports_thinking: false
title:
enabled: true
@@ -131,7 +121,7 @@ sandbox:
allow_host_bash: true
skills:
path: D:\project\bj\deerflow-server\skills
path: D:\cursor-work\offline-backend-20260512\backend\skills
security_scan:
enabled: false
@@ -83,11 +83,16 @@ _SYSTEM_TEMPLATE = """你是一名写作助手。
_USER_TEMPLATE = """用户写作意图:{user_intent}
请分析并输出以下 JSON(如果意图中未明确某字段,给出合理默认值):
用户在表单中已指定写作配置(以下三项必须原样输出,不要自行修改):
- 文章类型(article_type):{preconfigured_article_type}
- 目标读者(target_audience):{preconfigured_audience}
- 目标字数(word_count_target):{preconfigured_word_count}
请输出以下 JSON,article_type / target_audience / word_count_target 必须与上述一致,只需根据用户意图精炼 refined_intent:
{{{{
"article_type": "{type_options} 之一",
"target_audience": "目标读者描述",
"word_count_target": 800,
"article_type": "{preconfigured_article_type}",
"target_audience": "{preconfigured_audience}",
"word_count_target": {preconfigured_word_count},
"refined_intent": "精炼后的写作意图(一句话)"
}}}}"""
@@ -124,6 +129,44 @@ def _coerce_word_count(raw) -> int | None:
return value if value > 0 else None
def _resolve_intent_fields(
state: AIWritingState,
result: dict,
fallback_result: dict,
valid_keys: list[str],
default_key: str,
) -> tuple[str, str, int, str]:
"""合并 LLM 解析结果与用户表单预填字段;预填字段优先(见 AI_WRITING_BACKEND_DEV_ZH §intent_parser)。"""
fallback_type = fallback_result["article_type"]
if fallback_type not in valid_keys:
fallback_type = default_key
raw_type = result.get("article_type", "")
llm_type = raw_type if raw_type in valid_keys else fallback_type
preconfigured_type = (state.get("article_type") or "").strip()
if preconfigured_type in valid_keys:
final_article_type = preconfigured_type
else:
final_article_type = llm_type
preconfigured_audience = (state.get("target_audience") or "").strip()
if preconfigured_audience:
final_target_audience = preconfigured_audience
else:
final_target_audience = result.get("target_audience") or fallback_result["target_audience"]
preconfigured_wc = state.get("word_count_target")
if isinstance(preconfigured_wc, int) and preconfigured_wc > 0:
final_word_count = preconfigured_wc
else:
word_count = _coerce_word_count(result.get("word_count_target"))
final_word_count = word_count if word_count is not None else fallback_result["word_count_target"]
final_refined_intent = result.get("refined_intent") or state["user_intent"]
return final_article_type, final_target_audience, final_word_count, final_refined_intent
async def intent_parser_node(state: AIWritingState, config: RunnableConfig) -> dict:
# 从 configurable 读取动态类型列表,未配置时退回默认值
configurable = (config or {}).get("configurable", {})
@@ -137,10 +180,6 @@ async def intent_parser_node(state: AIWritingState, config: RunnableConfig) -> d
type_options = _build_type_options(valid_article_types)
system = _SYSTEM_TEMPLATE.format(type_options=type_options)
user = _USER_TEMPLATE.format(
user_intent=state["user_intent"],
type_options=" | ".join(valid_keys),
)
fallback_type = state.get("article_type", default_key)
if fallback_type not in valid_keys:
@@ -153,6 +192,17 @@ async def intent_parser_node(state: AIWritingState, config: RunnableConfig) -> d
"refined_intent": state["user_intent"],
}
preconfigured_wc = fallback_result["word_count_target"]
if not isinstance(preconfigured_wc, int) or preconfigured_wc <= 0:
preconfigured_wc = 800
user = _USER_TEMPLATE.format(
user_intent=state["user_intent"],
preconfigured_article_type=fallback_type,
preconfigured_audience=fallback_result["target_audience"],
preconfigured_word_count=preconfigured_wc,
)
# 流式调用:与 writer_outline / writer_draft 同套机制 —— LLM 逐字吐出 JSON,
# 通过 ``intent_chunk`` 推给前端打字机;完成后 ``parse_json`` 解析结构化结果,
# 再通过 ``intent_ready`` 推一份结构化字段给前端结果卡片。
@@ -193,13 +243,9 @@ async def intent_parser_node(state: AIWritingState, config: RunnableConfig) -> d
"detail": f"意图解析失败,已按默认配置兜底。原因:{llm_error}",
})
# 校验 LLM 返回的 article_type 是否在有效范围内,不合法时回退到默认值
raw_type = result.get("article_type", "")
final_article_type = raw_type if raw_type in valid_keys else fallback_type
final_target_audience = result.get("target_audience") or fallback_result["target_audience"]
word_count = _coerce_word_count(result.get("word_count_target"))
final_word_count = word_count if word_count is not None else fallback_result["word_count_target"]
final_refined_intent = result.get("refined_intent") or state["user_intent"]
final_article_type, final_target_audience, final_word_count, final_refined_intent = _resolve_intent_fields(
state, result, fallback_result, valid_keys, default_key
)
# 推送结构化结果给前端 —— 结果卡片显示文章类型 / 目标读者 / 字数 / 精炼意图
writer({
@@ -214,11 +260,9 @@ async def intent_parser_node(state: AIWritingState, config: RunnableConfig) -> d
updates: dict = {"status": "researching", "progress_events": progress_events}
updates["article_type"] = final_article_type
if result.get("target_audience"):
updates["target_audience"] = final_target_audience
if word_count is not None:
updates["word_count_target"] = final_word_count
if result.get("refined_intent"):
updates["target_audience"] = final_target_audience
updates["word_count_target"] = final_word_count
if final_refined_intent != state["user_intent"]:
updates["user_intent"] = final_refined_intent
return updates
@@ -9,6 +9,7 @@ from deerflow.agents.middlewares.clarification_middleware import ClarificationMi
from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
from deerflow.agents.middlewares.memory_middleware import MemoryMiddleware
from deerflow.agents.middlewares.prompt_prefix_middleware import PromptPrefixMiddleware
from deerflow.agents.middlewares.rag_citation_middleware import RagCitationMiddleware
from deerflow.agents.middlewares.setup_agent_approval_middleware import SetupAgentApprovalMiddleware
from deerflow.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
from deerflow.agents.middlewares.summarization_middleware import BeforeSummarizationHook, DeerFlowSummarizationMiddleware
@@ -254,11 +255,14 @@ def _build_middlewares(
List of middleware instances.
"""
resolved_app_config = app_config or get_app_config()
cfg = _get_runtime_config(config)
middlewares = build_lead_runtime_middlewares(app_config=resolved_app_config, lazy_init=True)
# Inject the admin prompt prefix into the model request (kept out of the
# persisted/displayed message so the user never sees it).
middlewares.append(PromptPrefixMiddleware())
if cfg.get("rag_mode_enabled", False):
middlewares.append(RagCitationMiddleware())
# Add summarization middleware if enabled
summarization_middleware = _create_summarization_middleware(app_config=resolved_app_config)
@@ -266,7 +270,6 @@ def _build_middlewares(
middlewares.append(summarization_middleware)
# Add TodoList middleware if plan mode is enabled
cfg = _get_runtime_config(config)
is_plan_mode = cfg.get("is_plan_mode", False)
is_bootstrap = cfg.get("is_bootstrap", False)
todo_list_middleware = _create_todo_list_middleware(is_plan_mode)
@@ -371,6 +374,7 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
max_concurrent_subagents = cfg.get("max_concurrent_subagents", 3)
is_bootstrap = cfg.get("is_bootstrap", False)
memory_injection_enabled = bool(cfg.get("memory_injection_enabled", True))
rag_mode_enabled = bool(cfg.get("rag_mode_enabled", False))
writing_mode = bool(cfg.get("writing_mode", False))
writing_artifact_path = cfg.get("writing_artifact_path") if writing_mode else None
if not isinstance(writing_artifact_path, str) or not writing_artifact_path.strip():
@@ -461,6 +465,7 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
is_bootstrap=True,
app_config=resolved_app_config,
memory_injection_enabled=memory_injection_enabled,
rag_mode_enabled=rag_mode_enabled,
),
state_schema=ThreadState,
)
@@ -495,6 +500,7 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
available_skills=(set(agent_config.skills or []) if agent_config else None),
app_config=resolved_app_config,
memory_injection_enabled=memory_injection_enabled,
rag_mode_enabled=rag_mode_enabled,
is_notebook_mode=is_notebook_mode,
notebook_source_ids=notebook_source_ids,
),
@@ -667,17 +667,25 @@ def _get_memory_context(agent_name: str | None = None, *, app_config: AppConfig
@lru_cache(maxsize=32)
def _get_cached_skills_prompt_section(
skill_signature: tuple[tuple[str, str, str, str], ...],
display_signature: tuple[tuple[str, str, str], ...],
available_skills_key: tuple[str, ...] | None,
container_base_path: str,
skill_evolution_section: str,
) -> str:
display_by_skill = {name: (mode, custom_instruction) for name, mode, custom_instruction in display_signature}
filtered = [(name, description, category, location) for name, description, category, location in skill_signature if available_skills_key is None or name in available_skills_key]
skills_list = ""
if filtered:
skill_items = "\n".join(
f" <skill>\n <name>{name}</name>\n <description>{description} {_skill_mutability_label(category)}</description>\n <location>{location}</location>\n </skill>"
for name, description, category, location in filtered
)
items = []
for name, description, category, location in filtered:
display_mode, custom_instruction = display_by_skill.get(name, ("", ""))
display_hint = ""
if display_mode:
display_hint = f"\n <display mode=\"{display_mode}\">{custom_instruction}</display>"
items.append(
f" <skill>\n <name>{name}</name>\n <description>{description} {_skill_mutability_label(category)}</description>\n <location>{location}</location>{display_hint}\n </skill>"
)
skill_items = "\n".join(items)
skills_list = f"<available_skills>\n{skill_items}\n</available_skills>"
# Restricted mode: agent has an explicit skill allowlist.
@@ -693,6 +701,19 @@ def _get_cached_skills_prompt_section(
else:
scope_note = ""
display_pattern = (
"""
**Skill Result Display Pattern:**
- If a skill has `<display mode="citation">`, use the skill's returned result order as reference numbers. Cite facts from the first result as `[[1]]`, the second as `[[2]]`, and so on. Use only numbers that correspond to returned results.
- If a skill has `<display mode="list">`, summarize the returned items as a concise list and keep item order stable.
- If a skill has `<display mode="table">`, summarize the result and output `<skill-display tool="SKILL_OR_TOOL_NAME" mode="table" />` where the frontend should render the real table. Do not hand-write the table rows yourself.
- If a skill has `<display mode="card">`, summarize the result and output `<skill-display tool="SKILL_OR_TOOL_NAME" mode="card" />` where the frontend should render the cards.
- If a skill has `<display mode="custom">`, follow the custom display instruction in the `<display>` tag when presenting the skill result.
"""
if display_signature
else ""
)
return f"""<skill_system>
You have access to skills that provide optimized workflows for specific tasks. Each skill contains best practices, frameworks, and references to additional resources.
{scope_note}
@@ -702,13 +723,19 @@ You have access to skills that provide optimized workflows for specific tasks. E
3. The skill file contains references to external resources under the same folder
4. Load referenced resources only when needed during execution
5. Follow the skill's instructions precisely
{display_pattern}
{location_hint}{evolution_block}
{skills_list}
</skill_system>"""
def get_skills_prompt_section(available_skills: set[str] | None = None, *, app_config: AppConfig | None = None) -> str:
def get_skills_prompt_section(
available_skills: set[str] | None = None,
*,
app_config: AppConfig | None = None,
display_enabled: bool = False,
) -> str:
"""Generate the skills prompt section with available skills list."""
skills = _get_enabled_skills_for_config(app_config)
@@ -739,11 +766,30 @@ def get_skills_prompt_section(available_skills: set[str] | None = None, *, app_c
return ""
skill_signature = tuple((skill.name, skill.description, skill.category, skill.get_container_file_path(container_base_path)) for skill in skills)
display_signature: tuple[tuple[str, str, str], ...] = ()
if display_enabled:
try:
from deerflow.config.extensions_config import get_extensions_config
extensions = get_extensions_config()
display_signature = tuple(
(
name,
skill_config.display.mode,
skill_config.display.custom_instruction,
)
for name, skill_config in sorted(extensions.skills.items())
if skill_config.display is not None
and skill_config.display.enabled
and skill_config.display.mode != "none"
)
except Exception:
logger.debug("Failed to load skill display configs for prompt injection", exc_info=True)
available_key = tuple(sorted(available_skills)) if available_skills is not None else None
if not skill_signature and available_key is not None:
return ""
skill_evolution_section = _build_skill_evolution_section(effective_skill_evolution_enabled)
return _get_cached_skills_prompt_section(skill_signature, available_key, container_base_path, skill_evolution_section)
return _get_cached_skills_prompt_section(skill_signature, display_signature, available_key, container_base_path, skill_evolution_section)
def get_agent_soul(agent_id: str | None) -> str:
@@ -868,6 +914,7 @@ def apply_prompt_template(
is_bootstrap: bool = False,
app_config: AppConfig | None = None,
memory_injection_enabled: bool = True,
rag_mode_enabled: bool = False,
is_notebook_mode: bool = False,
notebook_source_ids: list[int] | None = None,
) -> str:
@@ -902,7 +949,11 @@ def apply_prompt_template(
bootstrap_section = _build_bootstrap_section() if is_bootstrap else ""
# Get skills section
skills_section = get_skills_prompt_section(available_skills, app_config=app_config)
skills_section = get_skills_prompt_section(
available_skills,
app_config=app_config,
display_enabled=rag_mode_enabled,
)
# Get deferred tools section (tool_search)
deferred_tools_section = get_deferred_tools_prompt_section(app_config=app_config)
@@ -940,6 +991,21 @@ Writing mode is enabled for this run.
# Build the offline-network policy block (empty unless DEER_FLOW_OFFLINE_MODE is set).
network_policy_section = _build_network_policy_section()
rag_mode_section = (
"""
<rag_citation_mode>
RAG/reference display mode is enabled.
When a search, knowledge, skill, or MCP tool returns a result list:
- If the skill has a display adapter, follow that adapter first.
- For `citation`, cite returned items with `[[1]]`, `[[2]]`, etc. Only cite real returned items.
- For `table`, do not hand-write the table. Summarize and place `<skill-display tool="SKILL_OR_TOOL_NAME" mode="table" />` where the table should appear. The frontend will render the real tool result as a stable table.
- For `list` or `card`, summarize and place `<skill-display tool="SKILL_OR_TOOL_NAME" mode="list" />` or `<skill-display tool="SKILL_OR_TOOL_NAME" mode="card" />` where the structured result should appear.
- If no display adapter exists, default to citation mode with numbered citations.
</rag_citation_mode>
"""
if rag_mode_enabled
else ""
)
# Format the prompt with dynamic skills and memory
prompt = SYSTEM_PROMPT_TEMPLATE.format(
@@ -957,7 +1023,7 @@ Writing mode is enabled for this run.
acp_section=acp_and_mounts_section,
)
prompt = prompt + f"\n<current_date>{datetime.now().strftime('%Y-%m-%d, %A')}</current_date>"
prompt = prompt + rag_mode_section + f"\n<current_date>{datetime.now().strftime('%Y-%m-%d, %A')}</current_date>"
if is_notebook_mode:
source_ids = notebook_source_ids or []
@@ -61,6 +61,21 @@ _AUTH_PATTERNS = (
"无权",
"未授权",
)
# Upstream model gateway returned a malformed payload (typically a JSON decode
# error wrapped as `bad_response_status_code`, or a truncated/garbled inference
# response). These are almost always transient inference-side issues and safe
# to retry.
_UPSTREAM_GARBAGE_PATTERNS = (
"bad_response_status_code",
"bad response status",
"expecting ',' delimiter",
"expecting value",
"expecting property name",
"unexpected character",
"unterminated string",
"extra data",
"jsondecodeerror",
)
class LLMErrorHandlingMiddleware(AgentMiddleware[AgentState]):
@@ -139,9 +154,10 @@ class LLMErrorHandlingMiddleware(AgentMiddleware[AgentState]):
detail = _extract_error_detail(exc)
lowered = detail.lower()
error_code = _extract_error_code(exc)
error_code_lower = str(error_code).lower() if error_code is not None else ""
status_code = _extract_status_code(exc)
if _matches_any(lowered, _QUOTA_PATTERNS) or _matches_any(str(error_code).lower(), _QUOTA_PATTERNS):
if _matches_any(lowered, _QUOTA_PATTERNS) or _matches_any(error_code_lower, _QUOTA_PATTERNS):
return False, "quota"
if _matches_any(lowered, _AUTH_PATTERNS):
return False, "auth"
@@ -159,6 +175,11 @@ class LLMErrorHandlingMiddleware(AgentMiddleware[AgentState]):
return True, "transient"
if _matches_any(lowered, _BUSY_PATTERNS):
return True, "busy"
# Treat upstream gateway garbage (malformed JSON, truncated responses)
# as a retriable transient — the deterministic char-position errors we
# see from vLLM/proxy layers usually clear on a re-roll.
if _matches_any(lowered, _UPSTREAM_GARBAGE_PATTERNS) or _matches_any(error_code_lower, _UPSTREAM_GARBAGE_PATTERNS):
return True, "upstream"
return False, "generic"
@@ -171,21 +192,34 @@ class LLMErrorHandlingMiddleware(AgentMiddleware[AgentState]):
def _build_retry_message(self, attempt: int, wait_ms: int, reason: str) -> str:
seconds = max(1, round(wait_ms / 1000))
reason_text = "服务繁忙" if reason == "busy" else "网络/连接异常"
if reason == "busy":
reason_text = "服务繁忙"
elif reason == "upstream":
reason_text = "上游响应异常"
else:
reason_text = "网络/连接异常"
return f"大模型因{reason_text}调用失败,正在第 {attempt}/{self.retry_max_attempts} 次重试,{seconds} 秒后继续…"
def _build_circuit_breaker_message(self) -> str:
return "大模型因连续多次调用失败已被熔断保护,请稍候再试。"
def _build_user_message(self, exc: BaseException, reason: str) -> str:
detail = _extract_error_detail(exc)
def _build_user_message(self, reason: str) -> str:
if reason == "quota":
return "大模型因账户额度不足、账单异常或使用受限而拒绝请求,请检查模型账户后再试。"
if reason == "auth":
return "大模型因鉴权失败或无访问权限而拒绝请求,请检查模型的 API Key 与权限配置后再试。"
if reason in {"busy", "transient"}:
return "大模型因服务繁忙或网络异常,多次重试后仍未恢复,请稍候再继续。"
return f"大模型因「{detail}」原因调用失败。"
if reason == "upstream":
return "大模型当前响应异常(上游模型或网关临时故障),多次重试后仍未恢复,请稍后再试或更换模型。"
# NOTE: We intentionally do NOT embed the raw exception detail into the
# user-facing message. Raw `Error code: 400 - {...}` strings — especially
# Python repr-style dict literals — get persisted as AI messages and then
# re-fed to the model on the next turn, where their unescaped quotes /
# braces can corrupt the upstream JSON payload and cause deterministic
# JSONDecodeError loops. Keep the user message clean; full detail is
# already in the backend log via `logger.warning(..., exc_info=exc)`.
return "大模型本次调用未能成功完成,请稍后再试或更换模型。"
def _emit_retry_event(self, attempt: int, wait_ms: int, reason: str) -> None:
try:
@@ -249,7 +283,7 @@ class LLMErrorHandlingMiddleware(AgentMiddleware[AgentState]):
)
if retriable:
self._record_failure()
return AIMessage(content=self._build_user_message(exc, reason))
return AIMessage(content=self._build_user_message(reason))
@override
async def awrap_model_call(
@@ -295,7 +329,7 @@ class LLMErrorHandlingMiddleware(AgentMiddleware[AgentState]):
)
if retriable:
self._record_failure()
return AIMessage(content=self._build_user_message(exc, reason))
return AIMessage(content=self._build_user_message(reason))
def _matches_any(detail: str, patterns: tuple[str, ...]) -> bool:
@@ -0,0 +1,109 @@
"""Inject stable citation numbering before RAG final answers."""
from __future__ import annotations
import logging
import os
from pathlib import Path
from collections.abc import Awaitable, Callable
from typing import Any, override
from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langchain.agents.middleware.types import ModelCallResult, ModelRequest, ModelResponse
from langchain_core.messages import SystemMessage
from deerflow.runtime.references import build_latest_reference_prompt, build_reference_debug_summary
logger = logging.getLogger(__name__)
_RAG_DEBUG_LOGGER_NAME = "rag_citation_debug"
def _get_rag_debug_logger() -> logging.Logger:
debug_logger = logging.getLogger(_RAG_DEBUG_LOGGER_NAME)
if getattr(debug_logger, "_rag_file_configured", False):
return debug_logger
log_path = Path(os.environ.get("RAG_CITATION_DEBUG_LOG", "logs/rag_citation_debug.log"))
if not log_path.is_absolute():
log_path = Path.cwd() / log_path
log_path.parent.mkdir(parents=True, exist_ok=True)
handler = logging.FileHandler(log_path, encoding="utf-8")
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
debug_logger.addHandler(handler)
debug_logger.setLevel(logging.INFO)
debug_logger.propagate = False
setattr(debug_logger, "_rag_file_configured", True)
setattr(debug_logger, "_rag_file_path", str(log_path))
return debug_logger
def _log_rag_debug(message: str, *args: Any) -> None:
logger.info(message, *args)
_get_rag_debug_logger().info(message, *args)
class RagCitationMiddleware(AgentMiddleware[AgentState]):
"""Give the model the same numbered references the frontend will render."""
@staticmethod
def _enabled(request: ModelRequest) -> bool:
runtime = getattr(request, "runtime", None)
context = getattr(runtime, "context", None) if runtime is not None else None
if isinstance(context, dict) and context.get("rag_mode_enabled"):
return True
config = getattr(runtime, "config", None) if runtime is not None else None
if isinstance(config, dict):
configurable = config.get("configurable")
if isinstance(configurable, dict) and configurable.get("rag_mode_enabled"):
return True
runtime_context = config.get("context")
if isinstance(runtime_context, dict) and runtime_context.get("rag_mode_enabled"):
return True
return False
@staticmethod
def _patch(request: ModelRequest) -> ModelRequest | None:
if not RagCitationMiddleware._enabled(request):
return None
messages = list(request.messages)
prompt, batch = build_latest_reference_prompt(messages)
if not prompt:
_log_rag_debug(
"middleware reference-prompt enabled=true prompt=false messages=%s",
len(messages),
)
return None
summary = build_reference_debug_summary(messages, [batch] if batch else None)
_log_rag_debug(
"middleware reference-prompt enabled=true prompt=true messages=%s tool_messages=%s "
"raw_items=%s normalized_items=%s kept_items=%s latest_batch_sources=%s",
summary["messages"],
summary["tool_messages"],
summary["raw_items"],
summary["normalized_items"],
summary["kept_items"],
len((batch or {}).get("sources") or []),
)
patched_messages = [
*messages,
SystemMessage(content=prompt),
]
return request.override(messages=patched_messages)
@override
def wrap_model_call(
self,
request: ModelRequest,
handler: Callable[[ModelRequest], ModelResponse],
) -> ModelCallResult:
patched = self._patch(request)
return handler(patched or request)
@override
async def awrap_model_call(
self,
request: ModelRequest,
handler: Callable[[ModelRequest], Awaitable[ModelResponse]],
) -> ModelCallResult:
patched = self._patch(request)
return await handler(patched or request)
@@ -840,10 +840,13 @@ class DeerFlowClient:
current_config = get_extensions_config()
config_data = {
"mcpServers": mcp_servers,
"skills": {name: {"enabled": skill.enabled} for name, skill in current_config.skills.items()},
}
config_data = {
"mcpServers": mcp_servers,
"skills": {
name: skill.model_dump(by_alias=True, exclude_none=True)
for name, skill in current_config.skills.items()
},
}
self._atomic_write_json(config_path, config_data)
@@ -903,13 +906,20 @@ class DeerFlowClient:
if config_path is None:
raise FileNotFoundError("Cannot locate extensions_config.json. Set DEER_FLOW_EXTENSIONS_CONFIG_PATH or ensure it exists in the project root.")
extensions_config = get_extensions_config()
extensions_config.skills[name] = SkillStateConfig(enabled=enabled)
config_data = {
"mcpServers": {n: s.model_dump() for n, s in extensions_config.mcp_servers.items()},
"skills": {n: {"enabled": sc.enabled} for n, sc in extensions_config.skills.items()},
}
extensions_config = get_extensions_config()
current_skill_config = extensions_config.skills.get(name)
extensions_config.skills[name] = SkillStateConfig(
enabled=enabled,
display=current_skill_config.display if current_skill_config else None,
)
config_data = {
"mcpServers": {n: s.model_dump(by_alias=True) for n, s in extensions_config.mcp_servers.items()},
"skills": {
n: sc.model_dump(by_alias=True, exclude_none=True)
for n, sc in extensions_config.skills.items()
},
}
self._atomic_write_json(config_path, config_data)
@@ -3,7 +3,7 @@
import json
import os
from pathlib import Path
from typing import Any, Literal
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
@@ -53,10 +53,63 @@ class McpServerConfig(BaseModel):
model_config = ConfigDict(extra="allow", populate_by_name=True)
class SkillStateConfig(BaseModel):
"""Configuration for a single skill's state."""
enabled: bool = Field(default=True, description="Whether this skill is enabled")
class SkillDisplayTableColumn(BaseModel):
"""A configured table column for structured skill result display."""
label: str = Field(description="Column label shown in the frontend table")
field: str = Field(description="Field name or dot path read from each result item")
width: str | None = Field(default=None, description="Optional CSS width for this column")
model_config = ConfigDict(extra="allow", populate_by_name=True)
class SkillDisplayConfig(BaseModel):
"""Frontend display adapter for structured skill results."""
enabled: bool = Field(default=False, description="Whether result display adaptation is enabled")
mode: Literal["citation", "list", "table", "card", "custom", "none"] = Field(
default="none",
description="How the frontend should render this skill's structured results",
)
result_path: str = Field(default="results", description="Dot path to the array of result items", alias="resultPath")
title_field: str | None = Field(default=None, description="Field used as item title", alias="titleField")
summary_field: str | None = Field(default=None, description="Field used as short summary/snippet", alias="summaryField")
content_field: str | None = Field(default=None, description="Field used as full item content", alias="contentField")
source_field: str | None = Field(default=None, description="Field used as source label", alias="sourceField")
type_field: str | None = Field(default=None, description="Field used as source/type badge", alias="typeField")
time_field: str | None = Field(default=None, description="Field used as publication/update time", alias="timeField")
score_field: str | None = Field(default=None, description="Field used as relevance score", alias="scoreField")
url_field: str | None = Field(default=None, description="Field containing a direct URL", alias="urlField")
id_field: str | None = Field(default=None, description="Field containing an ID for template URLs", alias="idField")
link_strategy: Literal["auto", "url", "id"] = Field(
default="auto",
description="Whether links use a URL field, an ID template, or automatic fallback",
alias="linkStrategy",
)
link_template: str | None = Field(default=None, description="Template used when linkStrategy is id", alias="linkTemplate")
citation_token: Literal["doubleBracket", "singleBracket"] = Field(
default="doubleBracket",
description="Citation token style expected in model output",
alias="citationToken",
)
max_items: int = Field(default=8, ge=1, le=100, description="Maximum displayed items", alias="maxItems")
custom_instruction: str = Field(
default="",
description="User-authored instruction for custom rendering schema generation",
alias="customInstruction",
)
table_columns: list[SkillDisplayTableColumn] = Field(
default_factory=list,
description="Columns used when mode is table",
alias="tableColumns",
)
model_config = ConfigDict(extra="allow", populate_by_name=True)
class SkillStateConfig(BaseModel):
"""Configuration for a single skill's state."""
enabled: bool = Field(default=True, description="Whether this skill is enabled")
display: SkillDisplayConfig | None = Field(default=None, description="Optional frontend display adapter")
class ExtensionsConfig(BaseModel):
@@ -17,6 +17,7 @@ import json
import logging
import threading
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
@@ -37,10 +38,53 @@ class PromptPrefixSettings(BaseModel):
model_config = ConfigDict(extra="ignore")
class AppearanceSettings(BaseModel):
"""System-wide appearance defaults that any *new* user inherits.
Once a user explicitly toggles their theme in the UI, that choice is
persisted in the browser and supersedes this default — so changing
``default_theme`` here only affects users who have never touched the
toggle, and any future first-time loads.
"""
default_theme: Literal["light", "dark"] = Field(
default="light",
description="Theme applied for users who haven't picked their own (light / dark)",
)
model_config = ConfigDict(extra="ignore")
class CitationDisplaySettings(BaseModel):
"""System-wide cleanup rules for citation cards and detail dialogs."""
cleanup_words: list[str] = Field(
default_factory=list,
description="Words or phrases used to remove noisy advertisement/disclaimer lines from citation display.",
)
model_config = ConfigDict(extra="ignore")
class PrivateSquareSettings(BaseModel):
"""Admin-configurable settings for the agent private-square (私密广场).
The password gates read access to all agents flagged ``is_private=True``.
The hash is stored opaquely — the router layer is responsible for hashing
and verifying so the harness package keeps zero dependencies on
``app.gateway.auth``.
"""
enabled: bool = Field(default=False, description="Whether the private square is offered to users at all")
password_hash: str = Field(default="", description="Opaque hash of the admin-configured password. Empty = no password set yet.")
model_config = ConfigDict(extra="ignore")
class SystemSettings(BaseModel):
"""Top-level system settings document."""
prompt_prefix: PromptPrefixSettings = Field(default_factory=PromptPrefixSettings)
private_square: PrivateSquareSettings = Field(default_factory=PrivateSquareSettings)
appearance: AppearanceSettings = Field(default_factory=AppearanceSettings)
citation_display: CitationDisplaySettings = Field(default_factory=CitationDisplaySettings)
model_config = ConfigDict(extra="ignore")
@@ -81,3 +81,39 @@ class AgentStore(ABC):
async def list_by_ids(self, agent_ids: list[str], user_id: str) -> list[dict[str, Any]]:
"""Return rows for ``agent_ids`` visible to ``user_id``, preserving the input order."""
raise NotImplementedError
@abstractmethod
async def list_private(self) -> list[dict[str, Any]]:
"""Return every agent flagged ``is_private=True``.
Caller is responsible for authorization — typically the caller has
already verified the admin-configured private-square password.
"""
raise NotImplementedError
@abstractmethod
async def list_all(self) -> list[dict[str, Any]]:
"""Return every agent regardless of ownership/visibility.
Caller is responsible for authorization — typically the caller has
already verified ``system_role == "admin"``.
"""
raise NotImplementedError
@abstractmethod
async def update_any(self, agent_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
"""Update any agent row by id, regardless of ownership.
Caller is responsible for authorization (admin only). Returns ``None``
if the agent does not exist.
"""
raise NotImplementedError
@abstractmethod
async def delete_any(self, agent_id: str) -> bool:
"""Delete any agent row by id, regardless of ownership.
Caller is responsible for authorization (admin only). Returns
``False`` if the agent does not exist.
"""
raise NotImplementedError
@@ -138,3 +138,29 @@ class MemoryAgentStore(AgentStore):
rows = [self._agents[aid] for aid in agent_ids if aid in self._agents and self._is_visible(self._agents[aid], user_id)]
rows.sort(key=lambda a: order_map.get(a["id"], 0))
return [_copy(a) for a in rows]
async def list_private(self) -> list[dict[str, Any]]:
rows = [_copy(a) for a in self._agents.values() if bool(a.get("is_private"))]
rows.sort(key=lambda a: a.get("created_at") or "")
return rows
async def list_all(self) -> list[dict[str, Any]]:
rows = [_copy(a) for a in self._agents.values()]
rows.sort(key=lambda a: a.get("created_at") or "")
return rows
async def update_any(self, agent_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
row = self._agents.get(agent_id)
if row is None:
return None
for key in ("name", "description", "published", "is_private"):
if key in data:
row[key] = data[key]
row["updated_at"] = _now()
return _copy(row)
async def delete_any(self, agent_id: str) -> bool:
if agent_id not in self._agents:
return False
self._agents.pop(agent_id, None)
return True
@@ -21,6 +21,12 @@ class AgentRow(Base):
description: Mapped[str] = mapped_column(String(1024), nullable=False, default="")
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
published: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, index=True)
# When True, the agent is moved into the password-gated "private square"
# (私密广场) and is hidden from the regular public square / built-in list.
# The flag is independent of ``published``: a private agent is still
# visible to its owner in the "我的" tab, and visible to anyone in the
# private-square listing once the password has been verified.
is_private: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, index=True)
# NULL = not featured on the homepage selector. Non-NULL = admin-curated
# featured rank (lower sorts first). Only ``published=True`` agents are
# eligible; the router enforces that invariant on writes. BIGINT so the
@@ -5,7 +5,7 @@ from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import func, or_, select
from sqlalchemy import and_, func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.agents.base import AgentStore
@@ -28,10 +28,31 @@ class AgentRepository(AgentStore):
@staticmethod
def _visible_filter(user_id: str):
"""Visibility for single-record fetches (``get_visible``).
Lenient: a caller who already knows the agent id can fetch it even
when it's flagged ``is_private``. The private-square password is the
*discovery* gate (listings); individual reads are not re-gated, so a
bookmarked / linked private agent stays openable.
"""
return or_(AgentRow.user_id.is_(None), AgentRow.user_id == user_id, AgentRow.published.is_(True))
@staticmethod
def _listing_filter(user_id: str):
"""Visibility for *listings* — hides private-square agents from
non-owners so they only surface inside the password-gated endpoint.
Owners keep seeing their own private agents in '我的'.
"""
return or_(
AgentRow.user_id == user_id,
and_(
or_(AgentRow.user_id.is_(None), AgentRow.published.is_(True)),
AgentRow.is_private.is_(False),
),
)
async def list_visible(self, user_id: str) -> list[dict[str, Any]]:
stmt = select(AgentRow).where(self._visible_filter(user_id)).order_by(AgentRow.created_at.asc())
stmt = select(AgentRow).where(self._listing_filter(user_id)).order_by(AgentRow.created_at.asc())
async with self._sf() as session:
result = await session.execute(stmt)
return [_row_to_dict(row) for row in result.scalars()]
@@ -61,6 +82,7 @@ class AgentRepository(AgentStore):
description=data.get("description") or "",
user_id=data.get("user_id"),
published=bool(data.get("published", False)),
is_private=bool(data.get("is_private", False)),
created_at=now,
updated_at=now,
)
@@ -99,7 +121,7 @@ class AgentRepository(AgentStore):
row = await session.get(AgentRow, agent_id)
if row is None or row.user_id != user_id:
return None
for key in ("name", "description", "published"):
for key in ("name", "description", "published", "is_private"):
if key in data:
setattr(row, key, data[key])
row.updated_at = datetime.now(UTC)
@@ -112,7 +134,7 @@ class AgentRepository(AgentStore):
row = await session.get(AgentRow, agent_id)
if row is None or row.user_id is not None:
return None
for key in ("name", "description", "published"):
for key in ("name", "description", "published", "is_private"):
if key in data:
setattr(row, key, data[key])
row.updated_at = datetime.now(UTC)
@@ -157,7 +179,7 @@ class AgentRepository(AgentStore):
async def list_featured(self, user_id: str) -> list[dict[str, Any]]:
stmt = (
select(AgentRow)
.where(AgentRow.featured_order.is_not(None), self._visible_filter(user_id))
.where(AgentRow.featured_order.is_not(None), self._listing_filter(user_id))
.order_by(AgentRow.featured_order.asc(), AgentRow.created_at.asc())
)
async with self._sf() as session:
@@ -168,9 +190,43 @@ class AgentRepository(AgentStore):
if not agent_ids:
return []
order_map = {aid: idx for idx, aid in enumerate(agent_ids)}
stmt = select(AgentRow).where(AgentRow.id.in_(agent_ids), self._visible_filter(user_id))
stmt = select(AgentRow).where(AgentRow.id.in_(agent_ids), self._listing_filter(user_id))
async with self._sf() as session:
result = await session.execute(stmt)
rows = list(result.scalars())
rows.sort(key=lambda r: order_map.get(r.id, len(order_map)))
return [_row_to_dict(r) for r in rows]
async def list_private(self) -> list[dict[str, Any]]:
stmt = select(AgentRow).where(AgentRow.is_private.is_(True)).order_by(AgentRow.created_at.asc())
async with self._sf() as session:
result = await session.execute(stmt)
return [_row_to_dict(row) for row in result.scalars()]
async def list_all(self) -> list[dict[str, Any]]:
stmt = select(AgentRow).order_by(AgentRow.created_at.asc())
async with self._sf() as session:
result = await session.execute(stmt)
return [_row_to_dict(row) for row in result.scalars()]
async def update_any(self, agent_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(AgentRow, agent_id)
if row is None:
return None
for key in ("name", "description", "published", "is_private"):
if key in data:
setattr(row, key, data[key])
row.updated_at = datetime.now(UTC)
await session.commit()
await session.refresh(row)
return _row_to_dict(row)
async def delete_any(self, agent_id: str) -> bool:
async with self._sf() as session:
row = await session.get(AgentRow, agent_id)
if row is None:
return False
await session.delete(row)
await session.commit()
return True
@@ -0,0 +1,62 @@
"""Add ``agents.is_private`` for the private-square (私密广场) feature.
A nullable-default bool column flagging agents that should live in the
password-gated private square instead of the regular public square.
Revision ID: 20260526_01
Revises: 20260522_01
Create Date: 2026-05-26
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
from sqlalchemy import inspect
revision = "20260526_01"
down_revision = "20260522_01"
branch_labels = None
depends_on = None
def _has_table(table_name: str) -> bool:
inspector = inspect(op.get_bind())
return table_name in inspector.get_table_names()
def _has_column(table_name: str, column_name: str) -> bool:
if not _has_table(table_name):
return False
inspector = inspect(op.get_bind())
return any(col["name"] == column_name for col in inspector.get_columns(table_name))
def _has_index(table_name: str, index_name: str) -> bool:
if not _has_table(table_name):
return False
inspector = inspect(op.get_bind())
return any(idx["name"] == index_name for idx in inspector.get_indexes(table_name))
def upgrade() -> None:
if _has_table("agents") and not _has_column("agents", "is_private"):
op.add_column(
"agents",
sa.Column(
"is_private",
sa.Boolean(),
nullable=False,
server_default=sa.false(),
),
)
if _has_table("agents") and not _has_index("agents", "ix_agents_is_private"):
op.create_index("ix_agents_is_private", "agents", ["is_private"])
def downgrade() -> None:
if _has_index("agents", "ix_agents_is_private"):
op.drop_index("ix_agents_is_private", table_name="agents")
if _has_column("agents", "is_private"):
op.drop_column("agents", "is_private")
@@ -30,7 +30,7 @@ from deerflow.persistence.scheduled_tasks.model import (
ScheduledTaskSubscriptionRow,
SchedulerThreadRow,
)
from deerflow.persistence.skills.model import SkillRow
from deerflow.persistence.skills.model import SkillDisplayAdapterRow, SkillRow
from deerflow.persistence.tags.model import TagAssignmentRow, TagRow
from deerflow.persistence.thread_meta.model import ThreadMetaRow
from deerflow.persistence.thread_shares.model import ThreadShareRow
@@ -51,6 +51,7 @@ __all__ = [
"ScheduledTaskSubscriptionRow",
"SchedulerThreadRow",
"SkillRow",
"SkillDisplayAdapterRow",
"TagAssignmentRow",
"TagRow",
"ThreadMetaRow",
@@ -68,6 +68,14 @@ class SkillStore(ABC):
"""Return the number of skills owned by ``user_id`` (owner_user_id IS NOT NULL)."""
raise NotImplementedError
@abstractmethod
async def list_all(self) -> list[dict[str, Any]]:
"""Return every skill record regardless of ownership/visibility.
Caller is responsible for authorization (admin only).
"""
raise NotImplementedError
@abstractmethod
async def list_published_names(self) -> set[str]:
"""Return the set of skill names with ``published = True`` (any owner).
@@ -134,5 +134,11 @@ class MemorySkillStore(SkillStore):
async def count_owned(self, user_id: str) -> int:
return sum(1 for s in self._skills.values() if s.get("owner_user_id") == user_id)
async def list_all(self) -> list[dict[str, Any]]:
return [
_with_derived(_copy(record)) # type: ignore[arg-type]
for record in self._skills.values()
]
async def list_published_names(self) -> set[str]:
return {name for name, record in self._skills.items() if record.get("published")}
@@ -31,3 +31,16 @@ class SkillRow(Base):
detail: Mapped[str | None] = mapped_column(Text, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(UTC))
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC))
class SkillDisplayAdapterRow(Base):
__tablename__ = "skill_display_adapters"
skill_name: Mapped[str] = mapped_column(String(64), primary_key=True)
# Store JSON as text for maximum MySQL compatibility. Some deployments still
# run MySQL variants where native JSON columns or unquoted names such as
# ``display`` are not accepted consistently.
display_config: Mapped[str | None] = mapped_column(Text, nullable=True, default=None)
sample_payload: Mapped[str | None] = mapped_column(Text, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(UTC))
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC))
@@ -149,6 +149,12 @@ class SkillRepository(SkillStore):
async with self._sf() as session:
return await session.scalar(stmt) or 0
async def list_all(self) -> list[dict[str, Any]]:
stmt = select(SkillRow).order_by(SkillRow.created_at.asc())
async with self._sf() as session:
result = await session.execute(stmt)
return [_row_to_dict(row) for row in result.scalars()]
async def list_published_names(self) -> set[str]:
stmt = select(SkillRow.name).where(SkillRow.published.is_(True))
async with self._sf() as session:
@@ -0,0 +1,366 @@
"""Build stable per-turn citation batches from LangGraph messages."""
from __future__ import annotations
import json
from typing import Any
def build_reference_batches(messages: list[Any]) -> list[dict[str, Any]]:
batches: list[dict[str, Any]] = []
for index, message in enumerate(messages):
if _message_type(message) != "human":
continue
end = len(messages)
for cursor in range(index + 1, len(messages)):
if _message_type(messages[cursor]) == "human":
end = cursor
break
assistant = _find_last_assistant_answer(messages, index + 1, end)
if assistant is None:
continue
sources = _collect_sources(messages[index + 1 : end])
if sources:
message_id = _message_id(assistant) or str(index)
batches.append(
{
"id": message_id,
"assistant_message_id": message_id,
"sources": sources,
}
)
return batches
def build_latest_reference_prompt(messages: list[Any]) -> tuple[str, dict[str, Any] | None]:
human_index = _find_latest_human_index(messages)
if human_index is None:
return "", None
sources = _collect_sources(messages[human_index + 1 :])
if not sources:
return "", None
assistant = _find_last_assistant_answer(messages, human_index + 1, len(messages))
message_id = _message_id(assistant) or _message_id(messages[human_index]) or str(human_index)
batch = {
"id": message_id,
"assistant_message_id": message_id,
"sources": sources,
}
lines = [
"本轮检索结果已由系统统一合并并重新编号。最终回答必须使用下面这套编号。",
"如果你仍需要补充检索,请先继续调用工具;最终回答只能引用最新合并后的编号。",
"",
]
for source in sources:
title = source.get("title") or f"参考文献 {source.get('index')}"
meta = " · ".join(
str(item)
for item in (source.get("source"), source.get("time"))
if item
)
content = str(source.get("content") or source.get("snippet") or "").strip()
if len(content) > 520:
content = content[:517] + "..."
head = f"[{source.get('index')}] {title}"
if meta:
head += f"({meta})"
lines.append(head)
if content:
lines.append(content)
lines.append("")
lines.extend(
[
"引用规则:",
"1. 回答正文中的关键结论必须标注对应编号,例如 [1]、[2]。",
"2. 多个来源可写为 [1][3][5]。",
"3. 只能引用上方存在的编号,不要编造编号。",
"4. 不要使用工具原始编号,也不要重新排序编号。",
"5. 不要在正文末尾堆完整参考文献列表,前端会展示右侧参考文献。",
]
)
return "\n".join(lines).strip(), batch
def _find_latest_human_index(messages: list[Any]) -> int | None:
for index in range(len(messages) - 1, -1, -1):
if _message_type(messages[index]) == "human":
return index
return None
def build_reference_debug_summary(messages: list[Any], batches: list[dict[str, Any]] | None = None) -> dict[str, Any]:
"""Return compact counts for citation extraction logs."""
turn_summaries: list[dict[str, Any]] = []
total_tool_messages = 0
total_raw_items = 0
total_normalized_items = 0
total_kept_items = 0
for index, message in enumerate(messages):
if _message_type(message) != "human":
continue
end = len(messages)
for cursor in range(index + 1, len(messages)):
if _message_type(messages[cursor]) == "human":
end = cursor
break
tool_messages = 0
raw_items = 0
normalized_items = 0
kept_items = 0
for tool_message in messages[index + 1 : end]:
if _message_type(tool_message) != "tool":
continue
tool_messages += 1
items = _first_list(_extract_tool_payload(tool_message))
raw_items += len(items)
for raw_item in items:
normalized = _normalize_reference_item(raw_item, normalized_items + 1)
if not normalized:
continue
normalized_items += 1
kept_items += 1
total_tool_messages += tool_messages
total_raw_items += raw_items
total_normalized_items += normalized_items
total_kept_items += kept_items
turn_summaries.append(
{
"human_index": index,
"tool_messages": tool_messages,
"raw_items": raw_items,
"normalized_items": normalized_items,
"kept_items": kept_items,
}
)
batch_summaries = [
{
"id": batch.get("id"),
"assistant_message_id": batch.get("assistant_message_id"),
"source_count": len(batch.get("sources") or []),
}
for batch in (batches or build_reference_batches(messages))
]
return {
"messages": len(messages),
"turns": len(turn_summaries),
"tool_messages": total_tool_messages,
"raw_items": total_raw_items,
"normalized_items": total_normalized_items,
"kept_items": total_kept_items,
"turns_detail": turn_summaries,
"batches": batch_summaries,
}
def _collect_sources(messages: list[Any]) -> list[dict[str, Any]]:
sources: list[dict[str, Any]] = []
for message in messages:
if _message_type(message) != "tool":
continue
payload = _extract_tool_payload(message)
for raw_item in _first_list(payload):
normalized = _normalize_reference_item(raw_item, len(sources) + 1)
if not normalized:
continue
normalized["index"] = len(sources) + 1
sources.append(normalized)
return sources
def _find_last_assistant_answer(messages: list[Any], start: int, end: int) -> Any | None:
for index in range(end - 1, start - 1, -1):
candidate = messages[index]
if (
_message_type(candidate) == "ai"
and _message_id(candidate)
and _has_answer_content(candidate)
and not _has_tool_calls(candidate)
):
return candidate
return None
def _message_type(message: Any) -> str:
if isinstance(message, dict):
return str(message.get("type") or message.get("role") or "")
return str(getattr(message, "type", "") or "")
def _message_id(message: Any) -> str | None:
value = message.get("id") if isinstance(message, dict) else getattr(message, "id", None)
return str(value) if value else None
def _message_content(message: Any) -> Any:
return message.get("content") if isinstance(message, dict) else getattr(message, "content", None)
def _has_tool_calls(message: Any) -> bool:
if isinstance(message, dict):
direct = message.get("tool_calls")
chunks = message.get("tool_call_chunks")
extra = (message.get("additional_kwargs") or {}).get("tool_calls")
else:
direct = getattr(message, "tool_calls", None)
chunks = getattr(message, "tool_call_chunks", None)
extra = (getattr(message, "additional_kwargs", None) or {}).get("tool_calls")
return bool(direct or chunks or extra)
def _has_answer_content(message: Any) -> bool:
content = _message_content(message)
if isinstance(content, str):
return bool(content.strip())
if isinstance(content, list):
return any(
isinstance(part, dict)
and part.get("type") == "text"
and str(part.get("text") or "").strip()
for part in content
)
return False
def _extract_tool_payload(message: Any) -> Any:
content = _message_content(message)
if isinstance(content, dict):
return _unwrap_tool_payload(content)
if isinstance(content, list):
texts: list[str] = []
for part in content:
if not isinstance(part, dict):
continue
if isinstance(part.get("json"), dict):
return _unwrap_tool_payload(part["json"])
if part.get("type") == "json" and isinstance(part.get("json"), dict):
return _unwrap_tool_payload(part["json"])
if part.get("type") == "text" and isinstance(part.get("text"), str):
texts.append(part["text"])
unwrapped = _unwrap_tool_payload(part)
if unwrapped is not None:
return unwrapped
content = "\n".join(texts)
if isinstance(content, str):
return _parse_json_like(content)
return None
def _unwrap_tool_payload(value: dict[str, Any]) -> Any:
list_keys = ("results", "data", "items", "hits", "documents", "records", "sources")
if any(isinstance(value.get(key), list) for key in list_keys):
return value
for key in ("result", "output", "stdout", "content", "text", "payload", "response"):
nested = value.get(key)
if isinstance(nested, str):
parsed = _parse_json_like(nested)
if parsed is not None:
return parsed
if isinstance(nested, dict):
unwrapped = _unwrap_tool_payload(nested)
if unwrapped is not None:
return unwrapped
return value
def _parse_json_like(text: str) -> Any:
trimmed = text.strip()
if not trimmed:
return None
try:
return json.loads(trimmed)
except Exception:
pass
first = trimmed.find("{")
last = trimmed.rfind("}")
if 0 <= first < last:
try:
return json.loads(trimmed[first : last + 1])
except Exception:
return None
return None
def _first_list(payload: Any) -> list[Any]:
if isinstance(payload, list):
return payload
if not isinstance(payload, dict):
return []
for key in ("results", "data", "items", "hits", "documents", "records", "entries", "matches", "chunks", "sources"):
value = payload.get(key)
if isinstance(value, list):
return value
for key in ("result", "payload", "response", "output", "data"):
found = _first_list(payload.get(key))
if found:
return found
return []
def _nested_records(raw: Any) -> list[dict[str, Any]]:
if not isinstance(raw, dict):
return []
layers = [raw]
for key in ("metadata", "meta", "attrs", "attributes", "properties", "extra"):
nested = raw.get(key)
if isinstance(nested, dict):
layers.append(nested)
return layers
def _pick(layers: list[dict[str, Any]], keys: tuple[str, ...]) -> str | None:
for layer in layers:
for key in keys:
value = layer.get(key)
if isinstance(value, (str, int, float)) and str(value).strip():
return str(value).strip()
return None
def _normalize_reference_item(raw: Any, fallback_index: int) -> dict[str, Any] | None:
if not isinstance(raw, dict):
return None
layers = _nested_records(raw)
title = _pick(layers, ("title", "m_title", "name", "question", "keyword", "query", "text"))
content = _pick(layers, ("content", "page_content", "summary", "snippet", "description", "content_preview", "abstract", "body"))
url = _pick(layers, ("url", "link", "href", "source_url", "web_url"))
source = _pick(layers, ("source", "source1", "site", "provider", "from"))
time = _pick(layers, ("time", "date", "published_date", "publishedDate", "m_publish", "publish_time", "created_at"))
item_id = _pick(layers, ("recUuid", "rec_uuid", "uuid", "docId", "doc_id", "id", "recordId", "taskId"))
score_raw = _pick(layers, ("score", "relevance", "similarity"))
score = None
if score_raw is not None:
try:
score = float(score_raw)
except Exception:
score = None
if not title:
title = source or f"结果 {fallback_index}"
if not content:
content = ""
if not title and not content and not url:
return None
return {
"id": item_id or url,
"skillName": "backend-reference",
"mode": "citation",
"title": title,
"type": "result",
"snippet": content,
"content": content,
"source": source,
"time": time,
"url": url,
"score": score,
"raw": raw,
}
def _reference_key(source: dict[str, Any]) -> str:
raw = source.get("raw") if isinstance(source.get("raw"), dict) else {}
item_id = source.get("id") or source.get("url") or raw.get("recUuid") or raw.get("uuid") or raw.get("docId") or raw.get("id")
if item_id:
return str(item_id).casefold().strip()
return "|".join(
str(source.get(key) or "").casefold().strip()
for key in ("title", "source", "time", "content", "snippet")
)
@@ -37,14 +37,26 @@ class LocalSandbox(Sandbox):
"""Return whether the selected shell is a PowerShell executable."""
return LocalSandbox._shell_name(shell) in {"powershell", "powershell.exe", "pwsh", "pwsh.exe"}
@staticmethod
def _is_cmd_shell(shell: str) -> bool:
"""Return whether the selected shell is cmd.exe."""
return LocalSandbox._shell_name(shell) in {"cmd", "cmd.exe"}
@staticmethod
def _find_first_available_shell(candidates: tuple[str, ...]) -> str | None:
"""Return the first executable shell path or command found from candidates."""
@staticmethod
def _is_cmd_shell(shell: str) -> bool:
"""Return whether the selected shell is cmd.exe."""
return LocalSandbox._shell_name(shell) in {"cmd", "cmd.exe"}
@staticmethod
def _normalize_powershell_command(command: str) -> str:
"""Make bash-style command glue usable in Windows PowerShell 5.
The public tool is still named ``bash`` for model compatibility, but on
Windows local sandbox it may fall back to PowerShell. PowerShell 5 does
not support ``&&``, while the sandbox layer prepends commands with
``cd <workspace> &&``. Rewriting the separator keeps skill scripts
executable in that fallback shell.
"""
return command.replace("&&", ";")
@staticmethod
def _find_first_available_shell(candidates: tuple[str, ...]) -> str | None:
"""Return the first executable shell path or command found from candidates."""
for shell in candidates:
if os.path.isabs(shell):
if os.path.isfile(shell) and os.access(shell, os.X_OK):
@@ -302,30 +314,39 @@ class LocalSandbox(Sandbox):
resolved_command = self._resolve_paths_in_command(command)
shell = self._get_shell()
if os.name == "nt":
if self._is_powershell(shell):
args = [shell, "-NoProfile", "-Command", resolved_command]
if os.name == "nt":
if self._is_powershell(shell):
args = [
shell,
"-NoProfile",
"-Command",
self._normalize_powershell_command(resolved_command),
]
elif self._is_cmd_shell(shell):
args = [shell, "/c", resolved_command]
else:
args = [shell, "-c", resolved_command]
result = subprocess.run(
args,
shell=False,
capture_output=True,
text=True,
timeout=600,
)
else:
args = [shell, "-c", resolved_command]
result = subprocess.run(
args,
shell=False,
capture_output=True,
text=True,
timeout=600,
)
result = subprocess.run(
args,
shell=False,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
timeout=600,
)
else:
args = [shell, "-c", resolved_command]
result = subprocess.run(
args,
shell=False,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
timeout=600,
)
output = result.stdout
if result.stderr:
output += f"\nStd Error:\n{result.stderr}" if output else result.stderr
@@ -149,3 +149,58 @@ async def cascade_skill_unpublished_or_deleted(
counters["notified"] += 1
return counters
def sync_agent_skills_to_available(available_skill_names: set[str]) -> dict[str, Any]:
"""Strip skill references that no longer exist on disk from every agent.
Walks every agent dir and rewrites its ``config.yaml`` ``skills`` list to
contain only names that appear in ``available_skill_names``. Used by the
admin "agent-skills sync" endpoint to repair historical drift — skills
deleted before the cascade existed, deleted via a path that skipped the
cascade, or removed from the filesystem manually.
Synchronous because it only touches local files and the admin endpoint
runs it in a worker thread.
Returns ``{checked, modified, removals: [{agent_id, removed_skills}]}``.
"""
counters: dict[str, Any] = {"checked": 0, "modified": 0, "removals": []}
for agent_dir in _list_agent_dirs():
config_file = agent_dir / "config.yaml"
if not config_file.exists():
continue
counters["checked"] += 1
try:
with config_file.open("r", encoding="utf-8") as f:
data: dict[str, Any] = yaml.safe_load(f) or {}
except yaml.YAMLError:
logger.warning("Sync: failed to parse %s, skipping", config_file)
continue
skills = data.get("skills")
if not isinstance(skills, list) or not skills:
continue
kept: list[str] = []
removed: list[str] = []
for name in skills:
if isinstance(name, str) and name in available_skill_names:
kept.append(name)
else:
removed.append(name if isinstance(name, str) else str(name))
if not removed:
continue
data["skills"] = kept
with config_file.open("w", encoding="utf-8") as f:
yaml.safe_dump(data, f, default_flow_style=False, allow_unicode=True)
counters["modified"] += 1
counters["removals"].append({"agent_id": agent_dir.name, "removed_skills": removed})
logger.info("Sync: removed orphan skills %s from agent '%s'", removed, agent_dir.name)
return counters
@@ -56,7 +56,7 @@ def notebook_search_tool(query: str, top_k: int = 10) -> str:
headers={"Authorization": f"Bearer {token}"},
json={
"query": query,
"source_ids": [682],
"source_ids": [source_ids],
"search_space_id": search_space_id,
"top_k": top_k,
},