Merge branch 'ji' into dhk-1

Resolve column_additions_mysql.sql conflict: keep square_id columns from master and roundtable_drafts.task_id from dhk-1.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
董宏凯andCursor committed 2026-06-11 17:40:08 +08:00
commit bcc3d617a0
142 files changed
+14953 -2383

No files matched your search

@@ -19,24 +19,30 @@
并配合 ``SkillStopMiddleware`` 在 ``agent_orchestration`` 调用后停图,便于网关从
AIMessage.tool_calls 解析派活列表。
- ``seat`` —— 子智能体交付:开 thinking + medium reasoning;**可写文件**
(``write_file`` / ``str_replace`` 产出 md 等成品文档),但禁派活 / 澄清 /
展示文件 / 看图 / 联网搜索 / **``bash``**。禁 ``bash`` 的原因:圆桌席位是
「文档产出者」,弱模型爱写 ``.py`` 脚本再跑出产物(乱且抓不到预览),与
``report`` 角色禁 ``bash`` 同理;让它**直接 write_file 写成品文档**。
(``write_file`` / ``str_replace`` 产出 md 等成品文档)+ **可跑 ``bash``**
(2026-06 放开:业务需要跑知识库检索 / Python 辅助分析)。禁派活 / 澄清 /
展示文件 / 看图 / 联网搜索。``report`` 通过 ``[*_SEAT_EXCLUDED, ...]`` 继承本角色,
所以 Step3 报告也跟着拿到 bash;只有 ``leader`` 和 ``seat_parallel`` 仍各自显式禁。
历史警示(保留):弱模型用 ``bash`` 跑 ``python xxx.py > out.md`` 生成产物会
**绕过前端虚拟预览**(预览只拦截 ``write_file.args.content``)→ 沙箱里有文件但
Step2 右侧预览空白。席位 SOUL 必须明确「bash 只用于辅助查询拿数据,**最终交付
必须用 write_file 写 .md 落盘**,不要让脚本生成最终产物」。
- ``seat_parallel`` —— **DAG 快速(fast)模式的并行席位(方案 A,2026-06)**:**禁写文件**
(``write_file`` / ``str_replace``,产物走正文)+ 禁 ``bash`` + 禁「打断 / 越权」类,
(``write_file`` / ``str_replace``,产物走正文)+ **禁 ``bash``** + 禁「打断 / 越权」类,
**放开 ``web_search`` 与技能**(搜索不碰 sandbox;技能靠 ``read_file`` SKILL.md)——
否则并行席位几乎调不动工具、产生不了步骤(步骤条不显示)。
并行席位同时跑 bash 会在共享 sandbox 里抢路径 / 占进程数;且 fast 模式整体不
写文件,bash 跑出来的产物既不能预览也没意义。
「写文件产出 md」归 **沙箱(sandbox)模式**(走 ``seat`` policy,单列 + 沙箱跟随预览);
fast 模式快、双列、产物走正文 —— 用户明确要求 fast 不生成文件。详见
``docs/roundtable-dag-orchestration.md`` §4 / §12.1 / §12.8 / §12.10。
与 ``seat`` 的差异:``seat`` 禁 ``web_search``(旧「圆桌一律禁搜索」),
``seat_parallel`` **放开搜索**(本期诉求)。
- ``report`` —— Step3 报告绘制:在 ``seat``(已禁 ``bash``)基础上**额外禁
``str_replace``**:
* ``bash`` 已在 ``seat`` 禁 → 杜绝写 Python 脚本再跑出 HTML(脚本产物落在
outputs,但前端虚拟预览只抓 ``write_file`` 的 ``args.content``,抓不到脚本
产物 → 预览停在半截 ``write_file``,正是「没写完就去生成 py 文件」的现象);
``seat_parallel`` **放开搜索**(本期诉求);``seat`` 已放开 ``bash``,本角色仍禁。
- ``report`` —— Step3 报告绘制:在 ``seat`` 基础上**额外禁 ``str_replace``**:
* 继承 ``seat`` 的 bash 放开(2026-06)—— Step3 也可跑 bash 做辅助查询,同
``seat`` 警示:**结果必须 write_file 落盘**,否则前端虚拟预览会空白
(预览只抓 ``write_file.args.content``,抓不到脚本产物 → 预览停在半截
``write_file``,正是「没写完就去生成 py 文件」的现象)。
* 禁 ``str_replace`` → 强制返工用整文件 ``write_file`` 重写
(``str_replace`` 没有 ``content`` 字段,会让前端虚拟预览读到 undefined
而清空沙箱)。
@@ -72,16 +78,21 @@ _LEADER_EXCLUDED: list[str] = [
"hindsight_retain",
"read_peer_delivery", # 总控靠 _collect_delivery_status 直读各席位,不需要这个席位侧工具
]
# seat:席位**可以写文件**(write_file / str_replace)产出 md 等成品文档,但**禁 bash** ——
# 圆桌席位是「文档产出者」,不该跑 shell/python 脚本(弱模型爱写 .py 脚本再跑生成产物,既乱又
# 抓不到产物预览,与 report 角色禁 bash 同理)。禁 web_search(圆桌串行一律禁搜索,旧决策)。
# seat:席位**可以写文件**(write_file / str_replace)产出 md 等成品文档,**bash 放开**
# (2026-06 决策,业务需要 bash 跑知识库检索 / Python 辅助分析)。禁派活 / 澄清 /
# 展示文件 / 看图 / 联网搜索。``report`` 通过 ``[*_SEAT_EXCLUDED, ...]`` 继承本角色,
# 所以 Step3 报告跟着拿到 bash;只有 leader / seat_parallel 仍各自显式禁。
#
# **历史警示(保留)**:弱模型用 bash 跑 ``python xxx.py > out.md`` 生成最终产物会
# 绕过前端虚拟预览(只拦 write_file.args.content) → 沙箱里有文件但 Step2 右侧预览
# 空白。必须靠席位 SOUL 强约束「bash 只用于辅助查询拿数据,最终交付必须用 write_file
# 写 .md 落盘」。
_SEAT_EXCLUDED: list[str] = [
"web_search",
"ask_clarification",
"present_files",
"view_image",
"agent_orchestration",
"bash",
]
# 并行席位 = **快速(fast)模式**(双列):**禁写文件**(产物走正文)+ 禁跑命令 +「打断/越权」类。
# **禁 web_search** —— 离线内网部署无外网,席位检索一律走「技能(skill)」(靠 read_file 读
@@ -20,7 +20,6 @@ import os
import re
from collections.abc import AsyncIterator
from contextvars import ContextVar
from difflib import SequenceMatcher
from hashlib import sha256
from time import perf_counter
from typing import Any
@@ -83,7 +82,7 @@ _LEADERBOARD_DAILY_SNAPSHOT_LIMIT = max(
)
_LEADERBOARD_TODAY_TTL_SECONDS = max(
30.0,
float(os.getenv("ADMIN_LEADERBOARD_TODAY_TTL_SECONDS", "300")),
float(os.getenv("ADMIN_LEADERBOARD_TODAY_TTL_SECONDS", "43200")),
)
_LEADERBOARD_QUERY_TIMEOUT_MS = max(
500,
@@ -269,15 +268,6 @@ class AdminLeaderboardDimension(BaseModel):
last_seen_at: str = ""
class AdminLeaderboardQuestion(BaseModel):
text: str
user_id: str = ""
email: str = ""
thread_id: str = ""
run_id: str = ""
asked_at: str = ""
class AdminLeaderboardTrendPoint(BaseModel):
date: str
runs: int = 0
@@ -314,8 +304,6 @@ class AdminLeaderboardResponse(BaseModel):
models: list[AdminLeaderboardDimension] = Field(default_factory=list)
agents: list[AdminLeaderboardDimension] = Field(default_factory=list)
run_statuses: list[AdminLeaderboardDimension] = Field(default_factory=list)
recent_questions: list[AdminLeaderboardQuestion] = Field(default_factory=list)
frequent_questions: list[AdminLeaderboardQuestion] = Field(default_factory=list)
trends: list[AdminLeaderboardTrendPoint] = Field(default_factory=list)
@@ -330,7 +318,6 @@ class AdminUserAnalyticsResponse(BaseModel):
models: list[AdminLeaderboardDimension] = Field(default_factory=list)
agents: list[AdminLeaderboardDimension] = Field(default_factory=list)
run_statuses: list[AdminLeaderboardDimension] = Field(default_factory=list)
recent_questions: list[AdminLeaderboardQuestion] = Field(default_factory=list)
trends: list[AdminLeaderboardTrendPoint] = Field(default_factory=list)
@@ -1014,129 +1001,6 @@ async def _query_model_rankings(
]
async def _query_recent_questions(
session,
since: datetime,
until: datetime,
*,
user_id: str | None,
limit: int,
exclude_user_ids: set[str] | None = None,
include_scheduled: bool = True,
include_admins: bool = True,
include_failed: bool = True,
cleanup_system_questions: bool = True,
) -> list[AdminLeaderboardQuestion]:
exclude_user_ids = exclude_user_ids or set()
stmt = (
select(RunRow.first_human_message, RunRow.user_id, UserRow.email, RunRow.thread_id, RunRow.run_id, RunRow.created_at)
.outerjoin(UserRow, UserRow.id == RunRow.user_id)
.where(_run_time_filter(since, until), RunRow.first_human_message.is_not(None), RunRow.first_human_message != "")
.order_by(RunRow.created_at.desc())
.limit(limit)
)
if user_id:
stmt = stmt.where(RunRow.user_id == user_id)
stmt = _apply_run_scope(
stmt,
user_column=RunRow.user_id,
run_id_column=RunRow.run_id,
exclude_user_ids=exclude_user_ids,
include_admins=include_admins,
include_failed=include_failed,
include_scheduled=include_scheduled,
)
rows = (await _execute_leaderboard_query(session, "recent_questions", stmt)).all()
questions: list[AdminLeaderboardQuestion] = []
for r in rows:
text = _clean_question_text(r[0]) if cleanup_system_questions else str(r[0] or "").strip()
if cleanup_system_questions and not _is_displayable_question(text):
continue
questions.append(
AdminLeaderboardQuestion(
text=text[:500],
user_id=r[1] or "",
email=r[2] or "",
thread_id=r[3] or "",
run_id=r[4] or "",
asked_at=_iso(r[5]),
)
)
return questions
async def _query_frequent_questions(
session,
since: datetime,
until: datetime,
limit: int,
exclude_user_ids: set[str] | None = None,
include_scheduled: bool = True,
include_admins: bool = True,
include_failed: bool = True,
cleanup_system_questions: bool = True,
) -> list[AdminLeaderboardQuestion]:
exclude_user_ids = exclude_user_ids or set()
stmt = (
select(
RunRow.first_human_message,
RunRow.user_id,
UserRow.email,
RunRow.created_at,
)
.outerjoin(UserRow, UserRow.id == RunRow.user_id)
.where(_run_time_filter(since, until), RunRow.first_human_message.is_not(None), RunRow.first_human_message != "")
.order_by(RunRow.created_at.desc())
.limit(min(max(limit * 20, 200), 1000))
)
stmt = _apply_run_scope(
stmt,
user_column=RunRow.user_id,
run_id_column=RunRow.run_id,
exclude_user_ids=exclude_user_ids,
include_admins=include_admins,
include_failed=include_failed,
include_scheduled=include_scheduled,
)
rows = (await _execute_leaderboard_query(session, "frequent_questions", stmt)).all()
groups: list[dict] = []
for r in rows:
text = _clean_question_text(r[0]) if cleanup_system_questions else str(r[0] or "").strip()
if cleanup_system_questions and not _is_displayable_question(text):
continue
key = _question_similarity_key(text)
if not key:
continue
target = None
for group in groups:
score = SequenceMatcher(None, key, group["key"]).ratio()
if score >= 0.72 or key in group["key"] or group["key"] in key:
target = group
break
if target is None:
groups.append({"key": key, "text": text, "count": 1, "user_id": r[1] or "", "email": r[2] or "", "asked_at": r[3]})
else:
target["count"] += 1
if r[3] and (target.get("asked_at") is None or r[3] > target["asked_at"]):
target.update({"text": text, "user_id": r[1] or "", "email": r[2] or "", "asked_at": r[3]})
groups.sort(key=lambda item: (item["count"], _iso(item.get("asked_at"))), reverse=True)
questions: list[AdminLeaderboardQuestion] = []
for group in groups[:limit]:
text = group["text"][:460]
if group["count"] > 1:
text = f"{text}(相似问题 {group['count']} 次)"
questions.append(
AdminLeaderboardQuestion(
text=text,
user_id=group["user_id"],
email=group["email"],
asked_at=_iso(group.get("asked_at")),
)
)
return questions
async def _query_trends(
session,
since: datetime,
@@ -1306,10 +1170,6 @@ async def _build_user_leaderboard_response(
"include_scheduled": include_scheduled,
"include_admins": include_admins,
}
question_common = {
**common,
"cleanup_system_questions": settings.cleanup_system_questions,
}
overview = await _query_overview(session, since_dt, until_dt, **common)
await _leaderboard_pause()
users = await _query_user_leaderboard(session, since_dt, until_dt, **common)
@@ -1326,10 +1186,6 @@ async def _build_user_leaderboard_response(
await _leaderboard_pause()
run_statuses = await _query_run_dimensions(session, since_dt, until_dt, user_id=None, field="status", limit=limit, **common)
await _leaderboard_pause()
recent_questions = await _query_recent_questions(session, since_dt, until_dt, user_id=None, limit=limit, **question_common)
await _leaderboard_pause()
frequent_questions = await _query_frequent_questions(session, since_dt, until_dt, limit=limit, **question_common)
await _leaderboard_pause()
trends = await _query_trends(session, since_dt, until_dt, **common)
return AdminLeaderboardResponse(
@@ -1347,8 +1203,6 @@ async def _build_user_leaderboard_response(
models=models,
agents=agents,
run_statuses=run_statuses,
recent_questions=recent_questions,
frequent_questions=frequent_questions,
trends=trends,
)
@@ -1572,95 +1426,6 @@ def _merge_dimensions_from_daily(
return sorted(merged_items, key=lambda item: (item.count, _sort_datetime(item.last_seen_at)), reverse=True)[:limit]
def _merge_recent_questions_from_daily(
responses: list[AdminLeaderboardResponse],
limit: int,
) -> list[AdminLeaderboardQuestion]:
questions: list[AdminLeaderboardQuestion] = []
seen: set[str] = set()
for response in responses:
for question in response.recent_questions:
key = question.run_id or f"{question.user_id}:{question.asked_at}:{question.text}"
if key in seen:
continue
seen.add(key)
questions.append(question)
questions.sort(key=lambda item: _sort_datetime(item.asked_at), reverse=True)
return questions[:limit]
_FREQUENT_QUESTION_COUNT_RE = re.compile(r"(相似问题\s+(\d+)\s+次)$")
def _split_frequent_question_count(text: str) -> tuple[str, int]:
match = _FREQUENT_QUESTION_COUNT_RE.search(text)
if not match:
return text, 1
return _FREQUENT_QUESTION_COUNT_RE.sub("", text).strip(), max(1, int(match.group(1)))
def _merge_frequent_questions_from_daily(
responses: list[AdminLeaderboardResponse],
limit: int,
) -> list[AdminLeaderboardQuestion]:
groups: list[dict] = []
for response in responses:
for question in response.frequent_questions:
base_text, count = _split_frequent_question_count(question.text)
key = _question_similarity_key(base_text)
if not key:
continue
target = None
for group in groups:
score = SequenceMatcher(None, key, group["key"]).ratio()
if score >= 0.72 or key in group["key"] or group["key"] in key:
target = group
break
if target is None:
groups.append(
{
"key": key,
"text": base_text,
"count": count,
"user_id": question.user_id,
"email": question.email,
"thread_id": question.thread_id,
"run_id": question.run_id,
"asked_at": question.asked_at,
}
)
else:
target["count"] += count
if _sort_datetime(question.asked_at) > _sort_datetime(target.get("asked_at")):
target.update(
{
"text": base_text,
"user_id": question.user_id,
"email": question.email,
"thread_id": question.thread_id,
"run_id": question.run_id,
"asked_at": question.asked_at,
}
)
groups.sort(key=lambda item: (item["count"], _sort_datetime(item.get("asked_at"))), reverse=True)
questions: list[AdminLeaderboardQuestion] = []
for group in groups[:limit]:
text_value = group["text"][:460]
if group["count"] > 1:
text_value = f"{text_value}(相似问题 {group['count']} 次)"
questions.append(
AdminLeaderboardQuestion(
text=text_value,
user_id=group["user_id"],
email=group["email"],
thread_id=group["thread_id"],
run_id=group["run_id"],
asked_at=group["asked_at"],
)
)
return questions
def _merge_trends_from_daily(responses: list[AdminLeaderboardResponse]) -> list[AdminLeaderboardTrendPoint]:
trends_by_date: dict[str, AdminLeaderboardTrendPoint] = {}
for response in responses:
@@ -1719,8 +1484,6 @@ def _merge_daily_leaderboard_payloads(
models=_merge_dimensions_from_daily(responses, "models", limit),
agents=_merge_dimensions_from_daily(responses, "agents", limit),
run_statuses=_merge_dimensions_from_daily(responses, "run_statuses", limit),
recent_questions=_merge_recent_questions_from_daily(responses, limit),
frequent_questions=_merge_frequent_questions_from_daily(responses, limit),
trends=_merge_trends_from_daily(responses),
)
@@ -2267,8 +2030,6 @@ async def _build_leaderboard_report_prompt(
models = snapshot.models
agents = snapshot.agents
run_statuses = snapshot.run_statuses
recent_questions = snapshot.recent_questions
frequent_questions = snapshot.frequent_questions
trends = snapshot.trends
top_users = snapshot.users_by_activity[:12]
context = {
@@ -2280,8 +2041,6 @@ async def _build_leaderboard_report_prompt(
"top_models": [m.model_dump() for m in models[:12]],
"agents": [a.model_dump() for a in agents[:12]],
"run_statuses": [s.model_dump() for s in run_statuses[:12]],
"frequent_questions": [q.model_dump() for q in frequent_questions[:30]],
"recent_questions": [q.model_dump() for q in recent_questions[:40]],
"trends": [t.model_dump() for t in trends[-21:]],
}
return (
@@ -2289,12 +2048,10 @@ async def _build_leaderboard_report_prompt(
"报告需要有结论,不要只罗列数字;请使用 Markdown 标题、列表和表格,让前端可以直接渲染。"
"重点要求:"
"1) 你是专门负责分析用户行为的专家,不要自称企业 AI 应用运营分析师;"
"2) 优先突出用户最近常用问题、最近关注的实时问题、事件、主题和业务方向,必须引用 recent_questions 中的近期问题作为依据;"
"3) 单独分析 frequent_questions,归纳高频关注簇,说明这些高频问题反映出的需求强度和重复频率;"
"4) 结合 top_users、top_skills、top_tools、top_models 和 trends,说明使用频率、使用增长、常用能力和重点用户行为画像;"
"5) 可以提及使用强度、关注变化和行为机会,但不要把报告重点放在系统本身的问题、故障、失败率或回答质量缺陷上;"
"6) 给管理员 3-5 条可执行建议,建议应围绕用户关注主题、资源配置、内容供给、培训引导和高频能力优化。"
"输出结构:摘要、最近关注、高频问题与频率、重点用户行为、常用能力、趋势观察、行动建议。"
"2) 结合 top_users、top_skills、top_tools、top_models 和 trends,说明使用频率、使用增长、常用能力和重点用户行为画像;"
"3) 可以提及使用强度、关注变化和行为机会,但不要把报告重点放在系统本身的问题、故障、失败率或回答质量缺陷上;"
"4) 给管理员 3-5 条可执行建议,建议应围绕资源配置、内容供给、培训引导和高频能力优化。"
"输出结构:摘要、重点用户行为、常用能力、趋势观察、行动建议。"
"\n\n统计数据如下:\n"
f"{json.dumps(context, ensure_ascii=False, default=str)}"
)
@@ -2452,10 +2209,6 @@ async def get_user_analytics(
"include_scheduled": settings.include_scheduled,
"include_admins": settings.include_admins,
}
question_common = {
**common,
"cleanup_system_questions": settings.cleanup_system_questions,
}
since_dt, until_dt, range_days = _resolve_range(days, since, until)
async with session_factory() as session:
await _set_analytics_read_timeout(session)
@@ -2470,7 +2223,6 @@ async def get_user_analytics(
models = await _query_run_dimensions(session, since_dt, until_dt, user_id=user_id, field="model", limit=limit, **common)
agents = await _query_run_dimensions(session, since_dt, until_dt, user_id=user_id, field="agent", limit=limit, **common)
run_statuses = await _query_run_dimensions(session, since_dt, until_dt, user_id=user_id, field="status", limit=limit, **common)
recent_questions = await _query_recent_questions(session, since_dt, until_dt, user_id=user_id, limit=limit, **question_common)
trends = await _query_trends(session, since_dt, until_dt, user_id=user_id, **common)
return AdminUserAnalyticsResponse(
@@ -2484,7 +2236,6 @@ async def get_user_analytics(
models=models,
agents=agents,
run_statuses=run_statuses,
recent_questions=recent_questions,
trends=trends,
)
@@ -18,10 +18,10 @@ from app.gateway.auth.password import verify_password_async
from app.gateway.deps import get_agent_store, get_config, get_tag_store
from deerflow.agents.lead_agent.prompt import refresh_skills_system_prompt_cache_async
from deerflow.config.agents_api_config import get_agents_api_config
from deerflow.config.agents_config import list_custom_agents, load_agent_config, load_agent_soul, validate_agent_id
from deerflow.config.agents_config import CommonQuestion, list_custom_agents, load_agent_config, load_agent_soul, validate_agent_id
from deerflow.config.app_config import AppConfig
from deerflow.config.paths import get_paths
from deerflow.config.system_settings import load_system_settings
from deerflow.config.system_settings import load_system_settings, normalize_square_id, resolve_default_square_id
from deerflow.persistence.agents import AgentStore
from deerflow.persistence.tags import TagStore
from deerflow.runtime.user_context import get_effective_user_id
@@ -42,6 +42,7 @@ class AgentResponse(BaseModel):
owner_username: str | None = Field(default=None, description="Publisher username (email local part). Null for built-in agents.")
published: bool = Field(default=False, description="Whether a user-created agent is public")
is_private: bool = Field(default=False, description="Whether the agent lives in the password-gated private square instead of the public one")
square_id: str = Field(default="", description="Publish square (发布广场) the agent is filed under. Empty string = the default square. Independent of is_private.")
builtin: bool = Field(default=False, description="True when user_id is null")
created_at: datetime | str | None = None
updated_at: datetime | str | None = None
@@ -53,6 +54,11 @@ class AgentResponse(BaseModel):
tool_groups: list[str] | None = None
skills: list[str] | None = None
common_questions: list[CommonQuestion] | None = Field(
default=None,
description="Recommended questions shown on the agent chat home / sidebar (常用问题). Each item is {title, prompt}; legacy string lists are coerced.",
)
tags: list[dict] = Field(default_factory=list, description="Tags assigned to this agent")
featured_order: int | None = Field(
@@ -98,9 +104,11 @@ class AgentCreateRequest(BaseModel):
soul: str = Field(default="", description="SOUL.md content - agent personality and behavioral guardrails")
published: bool = Field(default=False, description="Whether the new agent should be public")
is_private: bool = Field(default=False, description="When True, the new agent goes into the password-gated private square instead of the public one")
square_id: str | None = Field(default=None, description="Publish square (发布广场) to file the agent under. Omit / empty = the default square.")
skills: list[str] | None = Field(default=None, description="Skill whitelist persisted to the agent's config.yaml")
model: str | None = Field(default=None, description="Optional model override persisted to config.yaml")
tool_groups: list[str] | None = Field(default=None, description="Optional tool group whitelist persisted to config.yaml")
common_questions: list[CommonQuestion] | None = Field(default=None, description="Recommended questions (常用问题) persisted to config.yaml. Each item is {title, prompt}; legacy strings are coerced.")
class AgentUpdateRequest(BaseModel):
@@ -111,9 +119,11 @@ class AgentUpdateRequest(BaseModel):
soul: str | None = Field(default=None, description="Updated SOUL.md content")
published: bool | None = Field(default=None, description="Updated publication state")
is_private: bool | None = Field(default=None, description="Move the agent into / out of the private square (true / false)")
square_id: str | None = Field(default=None, description="Publish square (发布广场) to file the agent under. Empty string = the default square.")
skills: list[str] | None = Field(default=None, description="Updated skill whitelist (config.yaml)")
model: str | None = Field(default=None, description="Updated model override (config.yaml)")
tool_groups: list[str] | None = Field(default=None, description="Updated tool group whitelist (config.yaml)")
common_questions: list[CommonQuestion] | None = Field(default=None, description="Updated recommended questions (常用问题, config.yaml). Each item is {title, prompt}; legacy strings are coerced.")
class AgentSelectorResponse(BaseModel):
@@ -217,6 +227,7 @@ def _write_agent_config(
skills: list[str] | None = None,
model: str | None = None,
tool_groups: list[str] | None = None,
common_questions: list[CommonQuestion] | None = None,
) -> None:
agent_dir = get_paths().agent_dir(agent_id)
agent_dir.mkdir(parents=True, exist_ok=True)
@@ -232,19 +243,36 @@ def _write_agent_config(
config_data["model"] = model
if tool_groups is not None:
config_data["tool_groups"] = tool_groups
if common_questions is not None:
# Persist as plain dicts (dropping blank rows) so config.yaml stays a
# clean mapping rather than a pydantic repr. Accepts already-coerced
# CommonQuestion models or raw dicts/strings.
normalized = [
q if isinstance(q, CommonQuestion) else CommonQuestion.model_validate(q)
for q in common_questions
]
config_data["common_questions"] = [
q.model_dump() for q in normalized if q.prompt.strip()
]
with open(config_file, "w", encoding="utf-8") as f:
yaml.dump(config_data, f, default_flow_style=False, allow_unicode=True)
def _load_config_extras(agent_id: str) -> dict[str, Any]:
"""Read skills/model/tool_groups from the agent's config.yaml, tolerant of missing files."""
"""Read skills/model/tool_groups/common_questions from the agent's config.yaml, tolerant of missing files."""
empty = {"skills": None, "model": None, "tool_groups": None, "common_questions": None}
try:
cfg = load_agent_config(agent_id)
except (FileNotFoundError, ValueError):
return {"skills": None, "model": None, "tool_groups": None}
return dict(empty)
if cfg is None:
return {"skills": None, "model": None, "tool_groups": None}
return {"skills": cfg.skills, "model": cfg.model, "tool_groups": cfg.tool_groups}
return dict(empty)
return {
"skills": cfg.skills,
"model": cfg.model,
"tool_groups": cfg.tool_groups,
"common_questions": cfg.common_questions,
}
def _write_agent_soul(agent_id: str, soul: str) -> None:
@@ -404,7 +432,7 @@ def _record_to_response(
can happen lazily when the agent is actually opened.
"""
soul: str | None = None
extras: dict[str, Any] = {"skills": None, "model": None, "tool_groups": None}
extras: dict[str, Any] = {"skills": None, "model": None, "tool_groups": None, "common_questions": None}
if include_extras:
_ensure_agent_dir_from_record(record)
extras = _load_config_extras(record["id"])
@@ -418,6 +446,7 @@ def _record_to_response(
user_id=record.get("user_id"),
published=bool(record.get("published", False)),
is_private=bool(record.get("is_private", False)),
square_id=str(record.get("square_id") or ""),
builtin=record.get("user_id") is None,
created_at=record.get("created_at"),
updated_at=record.get("updated_at"),
@@ -425,6 +454,7 @@ def _record_to_response(
model=extras["model"],
tool_groups=extras["tool_groups"],
skills=extras["skills"],
common_questions=extras.get("common_questions"),
featured_order=record.get("featured_order"),
)
@@ -559,6 +589,10 @@ async def list_agents(
default=None,
description="Keep only agents published by this user id (the card's publisher). Built-in agents never match.",
),
square: str | None = Query(
default=None,
description="Keep only agents filed under this publish square (发布广场). Omit for all squares.",
),
admin: bool = Query(
default=False,
description=(
@@ -579,6 +613,11 @@ async def list_agents(
raise HTTPException(status_code=403, detail="Only system admins can list all agents")
paginate = scope is not None or page is not None
square_id = square.strip() if square and square.strip() else None
square_is_default = False
if square_id:
sq = load_system_settings().publish_squares
square_is_default = square_id == resolve_default_square_id(sq)
if paginate:
eff_page = page or 1
rows, total = await store.list_paginated(
@@ -591,6 +630,8 @@ async def list_agents(
admin=admin,
sort=sort,
owner=owner,
square_id=square_id,
square_is_default=square_is_default,
)
responses = await _build_list_responses(store, tag_store, user_id, rows)
return AgentsListResponse(agents=responses, total=total, page=eff_page, page_size=page_size)
@@ -608,6 +649,12 @@ async def list_agents(
responses = [r for r in responses if any(t.get("id") in wanted for t in r.tags)]
if owner and owner.strip():
responses = [r for r in responses if r.user_id == owner.strip()]
if square_id:
responses = [
r
for r in responses
if (r.square_id or "") == square_id or (square_is_default and not (r.square_id or ""))
]
responses = _sort_agent_responses(responses, sort)
return AgentsListResponse(agents=responses, total=len(responses), page=1, page_size=len(responses))
except HTTPException:
@@ -725,6 +772,7 @@ async def create_agent_endpoint(request: AgentCreateRequest, http_request: Reque
skills=request.skills,
model=request.model,
tool_groups=request.tool_groups,
common_questions=request.common_questions,
)
_write_agent_soul(agent_id, request.soul)
@@ -736,6 +784,8 @@ async def create_agent_endpoint(request: AgentCreateRequest, http_request: Reque
"user_id": _current_user_id(http_request),
"published": request.published,
"is_private": request.is_private,
# Normalize to a valid configured square id (default when unset).
"square_id": normalize_square_id(request.square_id, load_system_settings().publish_squares),
}
)
# Mirror skills/tool_groups/model into the listing cache so the gallery
@@ -782,6 +832,10 @@ async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request:
update_data["published"] = bool(request.published)
if "is_private" in fields_set:
update_data["is_private"] = bool(request.is_private)
if "square_id" in fields_set:
# Normalize to a valid configured square id (default when unset /
# unknown), so the row never holds a dangling square reference.
update_data["square_id"] = normalize_square_id(request.square_id, load_system_settings().publish_squares)
# Mutual exclusion: an agent moved into the private square cannot
# also be publicly published. Force ``published=false`` whenever the
# same request sets ``is_private=true``, even if the client forgot
@@ -797,8 +851,9 @@ async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request:
next_skills = request.skills if "skills" in fields_set else existing_extras["skills"]
next_model = request.model if "model" in fields_set else existing_extras["model"]
next_tool_groups = request.tool_groups if "tool_groups" in fields_set else existing_extras["tool_groups"]
next_common_questions = request.common_questions if "common_questions" in fields_set else existing_extras["common_questions"]
config_dirty = bool(update_data) or any(key in fields_set for key in ("skills", "model", "tool_groups"))
config_dirty = bool(update_data) or any(key in fields_set for key in ("skills", "model", "tool_groups", "common_questions"))
if config_dirty:
_write_agent_config(
agent_id,
@@ -807,6 +862,7 @@ async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request:
skills=next_skills,
model=next_model,
tool_groups=next_tool_groups,
common_questions=next_common_questions,
)
# Keep the listing cache in lock-step with config.yaml on every write.
await store.replace_skills(agent_id, next_skills)
@@ -13,9 +13,12 @@ from __future__ import annotations
import logging
import mimetypes
import re
import tempfile
from pathlib import Path
from typing import Any
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
from fastapi import APIRouter, BackgroundTasks, File, Form, HTTPException, Request, UploadFile
from fastapi.responses import FileResponse
from pydantic import BaseModel, Field
@@ -26,6 +29,12 @@ from deerflow.knowledge.service import EmptyThreadError
from deerflow.runtime import serialize_channel_values
from deerflow.runtime.references import build_reference_batches
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.utils.file_conversion import CONVERTIBLE_EXTENSIONS, convert_file_to_markdown
# Plain-text/markdown files are read directly; binary documents are converted.
_TEXT_EXTENSIONS = {".md", ".markdown", ".txt", ".text", ""}
# Bound the in-memory read so a knowledge import can't OOM the gateway.
_MAX_IMPORT_BYTES = 25 * 1024 * 1024
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/knowledge", tags=["knowledge"])
@@ -299,6 +308,80 @@ async def ingest_threads_batch(body: BatchIngestRequest, request: Request) -> di
return {"ok": True, "processed": len(items), "created": created, "skipped": skipped, "failed": failed, "llm_fallback": llm_fallback, "items": items}
async def _extract_upload_text(file: UploadFile) -> tuple[str, str]:
"""Read an uploaded file and return ``(filename, markdown_text)``.
Plain-text/markdown is decoded directly; PDF/Office documents are converted
to Markdown via the shared converter. Raises ``HTTPException`` on empty /
oversized / unsupported / unparseable files.
"""
name = file.filename or "document"
ext = Path(name).suffix.lower()
raw = await file.read()
if not raw:
raise HTTPException(status_code=400, detail="文件为空")
if len(raw) > _MAX_IMPORT_BYTES:
raise HTTPException(status_code=413, detail=f"文件过大,最大支持 {_MAX_IMPORT_BYTES // (1024 * 1024)}MB")
if ext in _TEXT_EXTENSIONS:
text = raw.decode("utf-8", errors="ignore")
elif ext in CONVERTIBLE_EXTENSIONS:
# Convert in an isolated temp dir; a fixed safe name avoids path issues
# while preserving the extension so the converter detects the type.
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp) / f"import{ext}"
tmp_path.write_bytes(raw)
md_path = await convert_file_to_markdown(tmp_path)
if md_path is None:
raise HTTPException(status_code=422, detail=f"无法解析文件:{name}")
text = md_path.read_text(encoding="utf-8", errors="ignore")
else:
raise HTTPException(status_code=415, detail=f"不支持的文件类型:{ext or '未知'}(支持 md/txt/pdf/word/ppt/excel)")
if not text.strip():
raise HTTPException(status_code=422, detail="文件内容为空或无法提取文本")
return name, text
@router.post("/ingest/file")
@require_auth
async def ingest_file(
request: Request,
file: UploadFile = File(...),
title: str | None = Form(default=None),
tags: str | None = Form(default=None),
status: str = Form(default="approved"),
model_name: str | None = Form(default=None),
) -> dict[str, Any]:
"""Import an uploaded file into the knowledge base as wiki note(s).
Converts the file to Markdown, distills it via the wiki-capture LLM pipeline
(same as thread capture), and writes the resulting declarative note(s) into
the shared vault. ``tags`` is a comma/space-separated string.
"""
service = get_knowledge_service(request)
name, text = await _extract_upload_text(file)
tag_list = [t for t in re.split(r"[,,\s]+", tags or "") if t]
try:
result = await service.capture_file(
content_md=text,
filename=name,
title=(title or None),
tags=tag_list,
status=status,
created_by=get_effective_user_id(),
model_name=(model_name or None),
)
except EmptyThreadError:
raise HTTPException(status_code=422, detail="文件内容无可沉淀的知识")
return {
"note": result["note"],
"created": result["created"],
"notes": result.get("notes", [result["note"]]),
"llm_fallback": result.get("llm_fallback", False),
}
@router.post("/ingest/search")
@require_auth
async def ingest_search(body: IngestSearchRequest, request: Request) -> dict[str, Any]:
@@ -62,13 +62,6 @@ def _anonymize(resp: AdminLeaderboardResponse) -> AdminLeaderboardResponse:
user.email = _mask_email(user.email)
user.user_id = hashed(user.user_id)
for bucket in (resp.recent_questions, resp.frequent_questions):
for question in bucket:
question.email = _mask_email(question.email)
question.user_id = hashed(question.user_id)
question.thread_id = ""
question.run_id = ""
return resp
@@ -166,6 +166,7 @@ class SkillResponse(BaseModel):
owner_user_id: str | None = Field(default=None, description="Custom skill owner (null = built-in or legacy)")
owner_username: str | None = Field(default=None, description="Publisher username (email local part). Null for built-in skills.")
published: bool = Field(default=False, description="Whether the owner has made this skill visible to other users")
square_id: str = Field(default="", description="Publish square (发布广场) the skill is filed under once published. Empty string = the default square.")
builtin: bool = Field(default=False, description="True when owner_user_id is null (no human owner)")
detail: str | None = Field(default=None, description="Human-authored long-form detail text (Markdown), editable by owner/admin")
display: dict[str, Any] | None = Field(default=None, description="Optional frontend display adapter")
@@ -308,12 +309,14 @@ def _skill_to_response(
builtin = skill.category != SkillCategory.CUSTOM
detail: str | None = None
name_zh: str | None = None
square_id = ""
if ownership is not None:
owner_user_id = ownership.get("owner_user_id")
published = bool(ownership.get("published", False))
builtin = bool(ownership.get("builtin", owner_user_id is None))
detail = ownership.get("detail")
name_zh = ownership.get("name_zh")
square_id = str(ownership.get("square_id") or "")
display = None if display_override is _UNSET else display_override
if display_override is _UNSET:
@@ -342,6 +345,7 @@ def _skill_to_response(
source=entry.get("source", "upload"),
owner_user_id=owner_user_id,
published=published,
square_id=square_id,
builtin=builtin,
detail=detail,
display=display,
@@ -772,6 +776,7 @@ async def list_skills(
tag_store: TagStore = Depends(get_tag_store),
search: str | None = Query(default=None, description="Case-insensitive fuzzy match on skill name or Chinese name (name_zh)"),
tag_ids: list[str] | None = Query(default=None, description="Keep skills carrying any of these tags (OR)"),
square: str | None = Query(default=None, description="Keep only skills filed under this publish square (发布广场). Omit for all squares."),
sort: str = Query(
default="time_desc",
pattern="^(time_desc|time_asc|activity_desc|activity_asc|usage_desc|usage_asc|name_asc|name_desc)$",
@@ -831,6 +836,16 @@ async def list_skills(
)
responses = await _attach_and_filter_tags(tag_store, responses, search, tag_ids)
responses = await _attach_owner_usernames(responses)
square_id = square.strip() if square and square.strip() else None
if square_id:
from deerflow.config.system_settings import load_system_settings, resolve_default_square_id
square_is_default = square_id == resolve_default_square_id(load_system_settings().publish_squares)
responses = [
r
for r in responses
if (r.square_id or "") == square_id or (square_is_default and not (r.square_id or ""))
]
responses = _sort_skill_responses(responses, sort)
return SkillsListResponse(skills=responses)
except HTTPException:
@@ -1366,12 +1381,14 @@ class SkillPublishRequest(BaseModel):
"""Request body for toggling a custom skill's published flag."""
published: bool = Field(..., description="Whether to publish (True) or unpublish (False)")
square_id: str | None = Field(default=None, description="Publish square (发布广场) to file the skill under when publishing. Omit / empty = the default square.")
class SkillPublishResponse(BaseModel):
name: str
owner_user_id: str | None
published: bool
square_id: str = ""
@router.put(
@@ -1393,10 +1410,18 @@ async def update_custom_skill_published(
record, mode = await _get_editable_skill_or_raise(skill_store, skill_name, user_id, is_admin=is_admin)
was_published = bool(record.get("published", False))
# When publishing, also file the skill into the chosen square (normalized
# to a valid configured square id; default when unset). Only one square
# per skill — a single column, so it can never be "published twice".
update_fields: dict[str, Any] = {"published": body.published}
if body.published:
from deerflow.config.system_settings import load_system_settings, normalize_square_id
update_fields["square_id"] = normalize_square_id(body.square_id, load_system_settings().publish_squares)
if mode == "admin":
updated = await skill_store.update_admin(skill_name, {"published": body.published})
updated = await skill_store.update_admin(skill_name, update_fields)
else:
updated = await skill_store.update(skill_name, user_id, {"published": body.published})
updated = await skill_store.update(skill_name, user_id, update_fields)
if updated is None:
raise HTTPException(status_code=500, detail="Failed to persist published state")
@@ -1419,6 +1444,7 @@ async def update_custom_skill_published(
name=updated["name"],
owner_user_id=updated.get("owner_user_id"),
published=bool(updated.get("published", False)),
square_id=str(updated.get("square_id") or ""),
)
except HTTPException:
raise
@@ -2762,14 +2788,15 @@ class CuratorConfigResponse(BaseModel):
class CuratorConfigRequest(BaseModel):
"""Payload for updating skill_evolution overrides.
"""Payload for updating user-facing curator settings in skill_evolution.json.
All fields are required so the frontend always sends a complete config;
backend writes the entire payload to skill_evolution.json atomically.
All other settings (enabled, timeouts) are global-only and controlled via config.yaml.
"""
enabled: bool = Field(..., description="Whether the agent can create and modify skills under skills/custom.")
curator: CuratorSubConfig
interval_hours: int = Field(..., ge=1, description="Hours between curator runs.")
stale_after_days: int = Field(..., ge=1, description="Days of inactivity before a skill is marked stale.")
archive_after_days: int = Field(..., ge=1, description="Days of inactivity before a stale skill is archived.")
model_name: str | None = Field(default=None, description="Model for the curator LLM review; null = primary model.")
class CuratorLogRecord(BaseModel):
@@ -2785,11 +2812,7 @@ class CuratorLogsResponse(BaseModel):
def _build_curator_config_response(evo) -> CuratorConfigResponse:
"""Pure helper so both the GET route and the post-PUT refresh share one shape.
``evo`` is a ``SkillEvolutionConfig`` instance (already merged with per-user
overrides if any).
"""
"""Pure helper so both the GET route and the post-PUT refresh share one shape."""
return CuratorConfigResponse(
enabled=evo.enabled,
curator=CuratorSubConfig(
@@ -2805,8 +2828,7 @@ def _build_curator_config_response(evo) -> CuratorConfigResponse:
@router.get("/curator/config", response_model=CuratorConfigResponse, summary="Get Skill Evolution Config")
async def get_curator_config(request: Request) -> CuratorConfigResponse:
"""Return the current curator config for the authenticated user (yaml
defaults + that user's overrides)."""
"""Return the current curator config for the authenticated user (yaml defaults + per-user overrides)."""
try:
from deerflow.config.skill_evolution_runtime import load_skill_evolution_config_for_user
@@ -2819,21 +2841,22 @@ async def get_curator_config(request: Request) -> CuratorConfigResponse:
@router.put("/curator/config", response_model=CuratorConfigResponse, summary="Update Skill Evolution Config")
async def update_curator_config(body: CuratorConfigRequest, request: Request) -> CuratorConfigResponse:
"""Persist per-user overrides to ``users/{user_id}/skill_evolution.json``
and return the effective merged config.
"""Persist the three user-facing curator thresholds to the per-user ``skill_evolution.json``.
The payload is validated against the ``SkillEvolutionConfig`` shape; unknown
fields are rejected.
Other settings (enabled, model, timeouts) are global and controlled via config.yaml.
"""
try:
from deerflow.config.skill_evolution_config import SkillEvolutionConfig
from deerflow.config.skill_evolution_runtime import load_skill_evolution_config_for_user, save_user_overrides
user_id = _current_user_id(request)
payload = body.model_dump()
# Round-trip through SkillEvolutionConfig to ensure persisted shape stays valid.
SkillEvolutionConfig.model_validate(payload)
save_user_overrides(user_id, payload)
save_user_overrides(user_id, {
"curator": {
"interval_hours": body.interval_hours,
"stale_after_days": body.stale_after_days,
"archive_after_days": body.archive_after_days,
"model_name": body.model_name,
},
})
return _build_curator_config_response(load_skill_evolution_config_for_user(user_id))
except HTTPException:
raise
@@ -14,8 +14,12 @@ from deerflow.config import get_app_config
from deerflow.config.system_settings import (
LeaderboardSettings,
PromptPrefixSettings,
PublishSquare,
PublishSquaresSettings,
SystemSettings,
list_publish_squares,
load_system_settings,
resolve_default_square_id,
save_system_settings,
)
@@ -219,6 +223,88 @@ async def update_private_square(body: PrivateSquareUpdateRequest, request: Reque
)
class PublishSquareItem(BaseModel):
id: str = Field(..., description="Stable square id")
name: str = Field(..., description="Display name / tab label")
description: str = Field(default="", description="Optional description")
class PublishSquaresResponse(BaseModel):
"""The configured publish squares plus the resolved default.
Readable by any logged-in user — squares are pure category tabs with no
access control, so the gallery needs the list to render its sub-tabs and
the publish flow needs it to populate the square picker.
"""
squares: list[PublishSquareItem] = Field(default_factory=list)
default_square_id: str = Field(..., description="Square id used when a publisher picks none")
class PublishSquaresUpdateRequest(BaseModel):
squares: list[PublishSquareItem] | None = Field(default=None, description="Full ordered replacement list of squares")
default_square_id: str | None = Field(default=None, description="Which square id is the publish default")
def _publish_squares_response(settings: PublishSquaresSettings) -> PublishSquaresResponse:
squares = list_publish_squares(settings)
return PublishSquaresResponse(
squares=[PublishSquareItem(id=s.id, name=s.name, description=s.description) for s in squares],
default_square_id=resolve_default_square_id(settings),
)
@router.get("/publish-squares", response_model=PublishSquaresResponse)
async def get_publish_squares() -> PublishSquaresResponse:
"""Return the configured publish squares (readable by any logged-in user)."""
return _publish_squares_response(load_system_settings().publish_squares)
@router.put("/publish-squares", response_model=PublishSquaresResponse)
async def update_publish_squares(body: PublishSquaresUpdateRequest, request: Request) -> PublishSquaresResponse:
"""Replace the publish-squares configuration. Admin only.
``squares`` is a full ordered replacement. Each square needs a non-empty
name; a missing id is auto-generated from a slug of the name. Duplicate ids
are de-duplicated (first wins). ``default_square_id`` must reference one of
the surviving squares, else it falls back to the first.
"""
from uuid import uuid4
await _require_admin(request)
current = load_system_settings()
fields = body.model_dump(exclude_unset=True)
update: dict[str, object] = {}
if "squares" in fields and fields["squares"] is not None:
normalized: list[PublishSquare] = []
seen_ids: set[str] = set()
for raw in body.squares or []:
name = (raw.name or "").strip()
if not name:
continue
sid = (raw.id or "").strip() or uuid4().hex[:8]
if sid in seen_ids:
continue
seen_ids.add(sid)
normalized.append(PublishSquare(id=sid, name=name, description=(raw.description or "").strip()))
update["squares"] = normalized
if "default_square_id" in fields and fields["default_square_id"] is not None:
update["default_square_id"] = str(fields["default_square_id"]).strip()
if update:
updated = current.publish_squares.model_copy(update=update)
# Re-anchor the default to a surviving square id.
if updated.squares and updated.default_square_id not in {s.id for s in updated.squares}:
updated = updated.model_copy(update={"default_square_id": updated.squares[0].id})
saved = save_system_settings(current.model_copy(update={"publish_squares": updated}))
else:
saved = current
return _publish_squares_response(saved.publish_squares)
class AppearanceResponse(BaseModel):
"""Effective system-wide appearance defaults."""