fix(会商第三步): 流程图入库限 rwfx-A + 修复跳过报告生成流程图出现双调度中卡片
- 入库功能仅 rwfx-A(businessCode 6BF)深链会商模式提供,其它业务深链生成流程图不再出现入库按钮/询问 - 流程图抽取串行化:auto 自动出图与用户「跳过报告直接生成流程图」手动出图竞态时,先中止上一轮、复用/移除被取代气泡,保证任意时刻只有一张流程图卡片 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
04e6d0aa00
commit
893f1e6933
19 files changed
+559
-51
No files matched your search
@@ -258,7 +258,7 @@ FastAPI application on port 8001 with health check at `GET /health`. Set `GATEWA
|
||||
| **Menu Overrides** (`/api/menu-overrides`) | Sidebar-menu overrides (菜单管理), **global/shared** (no user scoping; `updated_by` audit-only). The static menu tree (icons/routes/`adminOnly`) lives in the **frontend** code (`core/page-layout/sidebar-menu.ts`); iframe light-apps are injected at render time. This router persists a per-node **overlay** the frontend merges onto that tree: `parent_id` (re-parent, moves the whole subtree; NULL = top level), `sort_order` (reorder among siblings), `label` (rename; NULL keeps code label), `disabled` (停用 — hides the node **and its subtree**). A node with no row renders exactly as code defines it, so new code menus / newly-registered iframe apps appear automatically and are immediately configurable. `GET /` - list all overrides (open to any authenticated user); `PUT /` - **admin only**, atomically **replaces the whole set** (`{overrides: [...]}`); rejects re-parenting that would exceed **3 levels** or form a cycle (`_validate_depth`, HTTP 400). Backed by `deerflow.persistence.menu_overrides` (table `menu_overrides`, auto-created by `create_all`); store wired in `deps.py` as `app.state.menu_override_store`. Frontend: merge logic `core/page-layout/menu-overrides.ts` (`applyMenuOverrides` in `use-sidebar-menu.ts`), client `strategy-components/api/menu-overrides.ts`, admin page `pages/MenuManagementPage.tsx` (@dnd-kit drag-reorder/re-parent + 「移动到」parent selector + ↑↓ arrows + 改名/停用), entry in the bottom-left settings dropdown (`components/page-sidebar.tsx`, admin-only). Tests: `tests/test_menu_overrides.py`, `frontend-web/src/core/page-layout/menu-overrides.test.ts`. |
|
||||
| **Sensitive Words** (`/api/sensitive-words`) | Sensitive-word / desensitize map (敏感词管理 / 脱敏词), **global/shared** (no user scoping; `updated_by` audit-only). Each row maps an original `word` → `replacement`, with `enabled` (停用 parks a word without deleting) + `sort_order`. The **frontend** ships a seed map in code (`src/lib/desensitize.ts` `SEED_DESENSITIZE_MAP`) used as the default on a fresh deployment; once an admin saves, this table is the source of truth and the frontend loads it at startup (`DesensitizeWordsLoader` → `replaceDesensitizeMap`, **enabled** rows only). `GET /` - list all words (open to any authenticated user — every page desensitizes); `PUT /` - **admin OR the `lqq` account** (`_require_manager`: `system_role == "admin"` or email `@`-prefix `== "lqq"`), atomically **replaces the whole set** (`{words: [...]}`, de-duped by `word`, last wins). Backed by `deerflow.persistence.sensitive_words` (table `sensitive_words`, PK `word` `String(191)`, auto-created by `create_all`); store wired in `deps.py` as `app.state.sensitive_word_store`. Frontend: client `strategy-components/api/sensitive-words.ts`, page `pages/SensitiveWordsPage.tsx` (route `/page/strategy/admin/sensitive-words`, entry in the bottom-left settings dropdown — **gated on username `lqq`, not admin role**, present in both sidebar modes; 「导入种子词」 merges missing seed entries). Tests: `tests/test_sensitive_words.py`. |
|
||||
| **Dashboard Sessions** (`/api/dashboard-sessions`) | 「大屏绘制智能体」独立页面会话,**按外部 `task_id` 存取,不按 user 分权**(open read + shared writes;`user_id` audit-only)。一个 task 一行(upsert):`transcript`(聊天记录 JSON)+ `report_json`(最新结构化数据,可解析 = 「有效结果」)。`GET /by-task/{task_id}` - 取该 task 会话(无则 404);`PUT /by-task/{task_id}` - upsert(`{transcript?, report_json?}`,只覆盖传入项)。Backed by `deerflow.persistence.dashboard_sessions`(table `dashboard_sessions`, auto-created by `create_all`,无迁移);store wired in `deps.py` as `app.state.dashboard_session_store`。前端独立页 `roundtable-planning/pages/DashboardAgentPage.tsx`(路由 `/page/strategy/dashboard-agent?taskId=`,侧边栏「问答管理 → 大屏绘制」)让智能体分析登录拉取的业务 JSON → report-json → 渲染 `DashboardApp`;输入 seam `core/auth/dashboard-input.ts`(JSON 接口未就绪时回退 copTaskDetail 占位)。Tests: `tests/test_dashboard_sessions.py`。 |
|
||||
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), enabled, sort_order}`. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
|
||||
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, enabled, sort_order}`. `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` the **frontend** resolves at click time for `url` buttons (`source` = a `userInfo` field or `other`→literal `custom_value`) and appends to the target URL — mirrors the light-app login params; empty = no extra params. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
|
||||
| **Knowledge** (`/api/knowledge`) | Product knowledge base, **global/shared** (no user isolation; writes record `created_by`/`updated_by` for audit). `POST /ingest/thread/{thread_id}` - sediment one thread into a note (reads messages from the checkpointer, `threads:read` owner-checked); `POST /ingest/threads/batch` - batch-sediment the caller's recent threads (`skip_existing` via `.manifest.json` dedup). **Defaults to `background=true`**: the slow per-thread extraction loop is deferred to a FastAPI background task and the endpoint returns `{queued:true, processed:N}` immediately (each thread's transcript is truncated to `max_input_chars` before the LLM call). Running the loop inline for a large batch outlives the nginx gateway timeout (504), which the frontend's `parseResponse` surfaces as a generic "Request failed"; `background=false` keeps the blocking path (full per-thread `created/skipped/failed` tally) for small batches/scripts. Shared loop: `_run_batch_ingest`. **Optional model selection**: both ingest endpoints accept `model_name` (overrides `knowledge.extract_model_name` for that capture; threaded through `KnowledgeService.capture_thread` ? `extract_thread_llm_multi`). **Model-error surfacing**: when the chosen model produces nothing usable (timeout / API error / non-JSON) capture still **falls back to rule-based extraction and lands a draft** (`llm_fallback:true`); the single-thread (non-background) response and the batch response (`llm_fallback` count) carry this so the UI tells the user "???????". The "??????" chat button stays background/fire-and-forget (passes `model_name`, no per-save error toast); the batch-import dialog is synchronous and shows the fallback count. `POST /ingest/file` - **import an uploaded file** (multipart: `file` + optional `title`/`tags`/`status`/`model_name`) into wiki note(s): the file is converted to Markdown (md/txt read directly, pdf/word/ppt/excel via `convert_file_to_markdown`), distilled by the same wiki-capture LLM pipeline as thread capture (`source_type="file"`), and falls back to a verbatim **draft** note when the model returns nothing usable (`llm_fallback:true`); `POST /ingest/search` - sediment search/tool results as a reference note; `POST /notes` - manual note; `GET /notes` (keyword/tag/source_type/status + pagination); `GET/PUT/DELETE /notes/{id}` (DELETE is soft = archive, `?hard=true` removes the vault file); `POST /search` - `mode=keyword\|vector\|hybrid` (vector?keyword fallback when no embedding endpoint); `GET /notes/{id}/versions` - edit history; `GET /duplicates` + `POST /merge` - dedup; `POST /reindex` - rebuild vector index; `GET /graph` - product graph data (notes + entities, graph-blacklist filtered); `GET/POST /graph/blacklist` + `DELETE /graph/blacklist/{id}` - graph node blacklist (labels = entity names / note titles, case-insensitive match; blacklisted nodes and their edges are dropped from `/graph` and the wiki-export artifacts; table `knowledge_graph_blacklist`, auto-created); `POST /export/graph` + `GET /export/files/{graph.json\|graph.html\|graph.graphml\|cypher.txt}` - obsidian-wiki `wiki-export` artifacts (self-contained `graph.html` for iframe); `POST /resolve` `{targets:[?]}` + `GET /notes/{id}/wikilinks` - resolve `[[wikilink]]` targets to a note id / vault page / missing (powers clickable wikilinks); `GET /vault-page?path=` - read a raw vault meta/hub Markdown page (no DB note). `GET/POST /folders` + `PUT/DELETE /folders/{id}` - user-managed directories (notes carry a `folder` path; `GET /notes?folder=` filters a dir + sub-dirs); `GET/POST /extract-templates` + `GET/PUT/DELETE /extract-templates/{id}` - editable extraction-prompt templates (thread/document ingest accept `template_id`; capture/PUT-note accept `folder`). Backed by the `deerflow.knowledge` module (see below). |
|
||||
|
||||
Proxied through nginx: `/api/langgraph/*` ? LangGraph, all other `/api/*` ? Gateway.
|
||||
@@ -274,7 +274,9 @@ Seeding mirrors the roundtable pattern: verified `config.yaml` + `SOUL.md` ship
|
||||
|
||||
**Skill-driven retrieval (检索类型).** `material_source` has exactly two values: `general` (通用检索, default) and `notebook` (我的空间); legacy `knowledge_base`/`skill` are normalized to `general` by the researcher node. General retrieval is orchestrated from the **skills configured on the `ai-writing-researcher` agent** (`config.yaml → skills`, editable from the agent management page): the built-in `knowledge-base-search` skill (知识库检索) runs the native keyword fan-out web/intranet search (the old default path), while any other configured skill searches with a `description`-targeted query; skills run in configured order with early-stop at `max_materials`, and an empty/unresolvable skills config falls back to the built-in knowledge search so retrieval always works. The skill itself is seeded from `_ai_writing_seed_assets/skills/knowledge-base-search/SKILL.md` into `skills/public/` (never overwrites), and a pre-existing researcher `config.yaml` lacking the `skills` key is patched in place with the default (explicit `skills: []` is respected). **PAUSE-1 input append flow**: when the user types natural language into the material-confirm input (`re_search`/legacy `enable_skill_search` + `user_query`), the node matches the most relevant configured skill via `SkillPicker(candidates=…)` (skipped when only one skill is configured), searches with the user's words, and **appends** the new materials to the existing package (URL-dedup, original order preserved, no LLM re-rank/truncation, 100-item hard cap). **Skill-failure fault tolerance**: a configured skill that times out / raises / has all its keyword searches fail (vs. legitimately returning 0 results) no longer just silently moves on — the researcher node emits a `skill_search_step` of type `skill_error` (carrying the error `reason`), and if the configured skills collected **no** new materials at all it runs a `fallback` pass over other available skills (`_fallback_candidate_skills`: the built-in `knowledge-base-search` native path first, then other enabled catalog skills not yet tried) instead of erroring out. The frontend `SkillStepsCard` renders `skill_error` (rose) and `fallback` (amber) so the timeline shows "技能报错 → 改用其他技能搜集". Tests: `tests/test_ai_writing_material_source.py`.
|
||||
|
||||
**Skill-call timeout** is `AI_WRITING_SKILL_SEARCH_TIMEOUT_SECONDS` (env, default **120s**; `researcher.py` `_SKILL_SEARCH_TIMEOUT_SECONDS`); the DuckDuckGo provider's internal cap was raised 30s→120s so it doesn't undercut it.
|
||||
**Skill-call timeout** is hardcoded **300s (5 min)** — `researcher.py` `_SKILL_SEARCH_TIMEOUT_SECONDS` (per-skill wrapper) and `_skill_agent.py` `_REAL_AGENT_TIMEOUT_SECONDS` (whole real-agent skill run). The **final intranet ES fallback** (`IntranetSearchProvider`) gets a longer **600s (10 min)** default (`intranet_stub.py`; overridable via `ai_writing.intranet_search_timeout`) — that interface is the slow one, so our own fallback timeout is generous to avoid being blamed for "检不出".
|
||||
|
||||
**Partial-result harvest + query simplification (检不出/超时的容错).** When `researcher_skill_agent: true` runs a non-native skill via `run_skill_via_real_agent` (the deployment's real path — the skill, e.g. `knowledge-search-v2`, is invoked by the real agent through `bash python3 scripts/search.py "<query>"`), one writing run issues many queries and **some time out** (the skill script prints `{"success":false,"error":"请求超时…","count":0}`) while others succeed (`{"success":true,"count":30,"results":[…]}`). Two fixes so successful data is never dropped: (1) `_drive()` captures every **ToolMessage** stdout and `_harvest_tool_results()` parses the skill's JSON directly (`_try_load_json` tolerates a trailing `Exit Code: 1`; skips `success:false`; field-name union across `knowledge-search-v2`/intranet-ES shapes) — so even if the model's final `{materials:[]}` summary gives up (confused by the timeouts), the successful queries' real results are still collected; the function merges model materials + harvested results (dedup by url/recUuid/title, capped at `max_results`). (2) The model is instructed (in `_MATERIAL_OUTPUT_CONTRACT`) that a 0-result/timeout query must be **retried with a simpler, core query** (drop topic suffixes like 竞选/诉讼/政策, keep the core entity: 「特朗普竞选」→「特朗普」) and that any data already retrieved must always be kept. The same deterministic simplifier `search/base.py::simplify_query` is wired into `IntranetSearchProvider.search`: an empty first attempt retries once with the simplified query. Tests: `tests/test_ai_writing_skill_real_agent.py`, `tests/test_ai_writing_query_simplify.py`.
|
||||
|
||||
**Agentic skill execution (`ai_writing.researcher_skill_agent`, default false).** By default a configured non-native skill is *not* run agent-style — the node just turns its `description` into a query for the generic search provider. With this flag on, the directed-skill branch instead runs `nodes/_skill_agent.py::run_skill_search_agent`: it injects the skill's **SKILL.md** (read from `skill.skill_file`) alongside the researcher SOUL as the system prompt, binds a `search` tool to the researcher model, and runs a bounded (`_MAX_TURNS=4`) tool-call loop so the model drives the retrieval per the skill's instructions; the `SearchResult`s the tool returns are harvested directly as materials (no fuzzy text extraction). Requires a tool-calling model; if the model can't tool-call (returns no `tool_calls`) or the agent yields nothing, `_execute_skill` falls back to the legacy description→query search so materials are never empty. The built-in `knowledge-base-search` native concurrent path is unchanged. Tests: `tests/test_ai_writing_skill_agent.py`.
|
||||
|
||||
|
||||
@@ -35,6 +35,18 @@ router = APIRouter(prefix="/api/task-buttons", tags=["task-buttons"])
|
||||
# ── schemas ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TaskButtonLoginParam(BaseModel):
|
||||
"""url 外链追加的登录参数(与应用管理一致)。
|
||||
|
||||
运行时由前端从 ``userInfo`` 解析:``source`` 为 ``userInfo`` 字段名
|
||||
(如 ``accessToken`` / ``yUserId``),``other`` 时取 ``custom_value`` 字面值。
|
||||
"""
|
||||
|
||||
key: str = Field(default="", max_length=64)
|
||||
source: str = Field(default="accessToken", max_length=64)
|
||||
custom_value: str = Field(default="", max_length=512)
|
||||
|
||||
|
||||
class TaskButton(BaseModel):
|
||||
id: str = Field(..., min_length=1, max_length=128)
|
||||
business: str = Field(..., min_length=1, max_length=32)
|
||||
@@ -48,6 +60,8 @@ class TaskButton(BaseModel):
|
||||
id_kind: str = Field(default="task", max_length=16)
|
||||
# url 外链打开方式:blank=新窗口(默认)/ self=当前窗口。
|
||||
open_mode: str = Field(default="blank", max_length=16)
|
||||
# url 外链追加的登录参数列表(key + 取值来源)。
|
||||
login_params: list[TaskButtonLoginParam] = Field(default_factory=list)
|
||||
enabled: bool = True
|
||||
sort_order: int = 0
|
||||
|
||||
|
||||
+162
-23
@@ -53,19 +53,135 @@ def use_skill_search_agent() -> bool:
|
||||
_REAL_AGENT_MAX_TURNS = 16
|
||||
|
||||
# 真实智能体跑技能:整段运行(读 SKILL.md + 多轮工具调用)的超时上限,**硬编码 5 分钟**。
|
||||
# 内网检索很慢、技能常做多次检索,单次直连检索的 120s 不够整段用;这里写死,不依赖 env
|
||||
# (部署方不一定会配环境变量),保证内网慢检索至少有 5 分钟才被切断。
|
||||
# 内网检索很慢、技能常做多次检索,单次直连检索的 120s 不够整段用;5 分钟足够整段跑完
|
||||
# (含模型对超时检索词自行换简单词重试),不依赖 env(部署方不一定会配环境变量)。
|
||||
_REAL_AGENT_TIMEOUT_SECONDS = 300
|
||||
|
||||
_MATERIAL_OUTPUT_CONTRACT = (
|
||||
"你的任务:严格按上面挂载的检索技能(SKILL.md)的说明,用相应工具检索与「写作意图」相关的**真实素材**。\n"
|
||||
"- 必须按技能说明实际调用工具完成检索(读取知识库文件 / 调用内网检索接口 / MCP 等),**不得凭记忆编造**。\n"
|
||||
"- **某个检索词返回 0 条或报「请求超时」时,不要就此放弃**:请把它换成更核心、更简短的检索词再试一次"
|
||||
"(去掉「竞选/诉讼/政策/动向」等话题后缀,只保留核心实体,例如「特朗普竞选」→「特朗普」、"
|
||||
"「特朗普法律诉讼」→「特朗普」;多词时只留最关键的那个词)。\n"
|
||||
"- **只要有任何检索词成功拿到了数据,就必须把这些数据全部收下并输出**,绝不能因为别的检索词没出结果/超时就丢弃已经检索到的素材。\n"
|
||||
"- 检索完成后,**只输出一个 JSON 对象**(不要解释、不要 markdown 代码块、用英文双引号),把检索到的真实素材列出:\n"
|
||||
'{"materials": [{"title": "标题", "content": "素材正文摘录(尽量保留原文, <=500字)", "source": "来源", "url": "原文链接(可空)", "published_at": "日期(可空)"}]}\n'
|
||||
"- content 必须来自检索到的真实内容;若确实没检索到任何素材,输出 {\"materials\": []}。"
|
||||
"- content 必须来自检索到的真实内容;只有当**所有**检索词(含简化后重试)都没拿到任何素材时,才输出 {\"materials\": []}。"
|
||||
)
|
||||
|
||||
|
||||
# 技能 stdout 里「结果列表」常见的字段名(不同技能各异,做并集兼容)
|
||||
_RESULT_LIST_KEYS = ("results", "data", "items", "materials", "documents", "list", "hits")
|
||||
_TITLE_KEYS = ("title", "m_title", "name", "doc_title")
|
||||
_CONTENT_KEYS = ("content", "content_preview", "page_content", "text", "summary", "abstract")
|
||||
_SOURCE_KEYS = ("source", "source_category", "source1", "source_name", "kb_name")
|
||||
_DATE_KEYS = ("publish_time", "published_at", "m_publish", "date", "create_time")
|
||||
|
||||
|
||||
def _try_load_json(text: str):
|
||||
"""尽量从技能 stdout 解析出 JSON:先整体 loads,失败再截取首个平衡的 {…}/[…]。
|
||||
|
||||
技能脚本的输出常带尾巴(如 ``...}\n\nExit Code: 1``),整体 loads 会失败,故再用
|
||||
「首个左括号 → 最后一个对应右括号」截取主体重试。
|
||||
"""
|
||||
import json as _json
|
||||
|
||||
text = (text or "").strip()
|
||||
if not text:
|
||||
return None
|
||||
try:
|
||||
return _json.loads(text)
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
for open_c, close_c in (("{", "}"), ("[", "]")):
|
||||
start = text.find(open_c)
|
||||
end = text.rfind(close_c)
|
||||
if 0 <= start < end:
|
||||
try:
|
||||
return _json.loads(text[start : end + 1])
|
||||
except Exception: # noqa: BLE001
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _extract_result_items(obj) -> list:
|
||||
"""从解析出的 JSON 里取「结果数组」:顶层就是数组则直接用,否则按常见字段名找。"""
|
||||
if isinstance(obj, list):
|
||||
return obj
|
||||
if isinstance(obj, dict):
|
||||
for key in _RESULT_LIST_KEYS:
|
||||
v = obj.get(key)
|
||||
if isinstance(v, list):
|
||||
return v
|
||||
return []
|
||||
|
||||
|
||||
def _pick(d: dict, keys: tuple) -> str:
|
||||
for k in keys:
|
||||
v = d.get(k)
|
||||
if isinstance(v, str) and v.strip():
|
||||
return v.strip()
|
||||
if v not in (None, "") and not isinstance(v, (dict, list)):
|
||||
return str(v).strip()
|
||||
return ""
|
||||
|
||||
|
||||
def _harvest_tool_results(tool_outputs: list, skill_name: str, limit: int) -> list:
|
||||
"""**直接从技能 stdout 收割真实检索结果**(不依赖模型最后的 JSON 汇总)。
|
||||
|
||||
关键容错:一次写作会对同一技能跑多个检索词,部分检索词可能超时(``success:false``),
|
||||
但成功的那些(``success:true``)拿回的真实数据**必须照单全收**,不能因为别的检索词没出
|
||||
数据就整段丢弃。本函数逐条解析技能输出、跳过 ``success:false`` 的、把结果数组映射成
|
||||
SearchResult,按 url/recUuid/title 去重,截断到 limit。字段名做并集兼容(适配 knowledge-
|
||||
search-v2 的 ``{success,query,count,results:[{recUuid,title,url,content_preview,...}]}``
|
||||
及内网 ES 的 ``{m_title,content,recUuid,...}`` 等多种技能输出)。
|
||||
"""
|
||||
from ..search.base import SearchResult
|
||||
|
||||
harvested: list = []
|
||||
seen: set[str] = set()
|
||||
for raw in tool_outputs:
|
||||
obj = _try_load_json(raw)
|
||||
if obj is None:
|
||||
continue
|
||||
# 明确标记失败的(超时 / 报错)跳过,但不影响其它成功的输出
|
||||
if isinstance(obj, dict) and obj.get("success") is False:
|
||||
continue
|
||||
for it in _extract_result_items(obj):
|
||||
if not isinstance(it, dict):
|
||||
continue
|
||||
# 内网 ES 风格:真实字段藏在 metadata 里
|
||||
md = it.get("metadata") if isinstance(it.get("metadata"), dict) else {}
|
||||
merged = {**md, **it} if md else it
|
||||
content = _pick(merged, _CONTENT_KEYS)
|
||||
title = _pick(merged, _TITLE_KEYS)
|
||||
if not content and not title:
|
||||
continue
|
||||
url = _pick(merged, ("url",))
|
||||
rec = _pick(merged, ("recUuid", "rec_uuid", "id"))
|
||||
key = url or rec or title
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
try:
|
||||
score = float(merged.get("relevance_score") or 0.0)
|
||||
except (TypeError, ValueError):
|
||||
score = 0.0
|
||||
harvested.append(SearchResult(
|
||||
title=title or "(无标题)",
|
||||
content=content[:500],
|
||||
source=_pick(merged, _SOURCE_KEYS) or skill_name,
|
||||
published_at=_pick(merged, _DATE_KEYS),
|
||||
url=url,
|
||||
relevance_score=score or 0.75,
|
||||
source_type="skill",
|
||||
recUuid=rec,
|
||||
))
|
||||
if len(harvested) >= limit:
|
||||
return harvested
|
||||
return harvested
|
||||
|
||||
|
||||
def _summarize_tool_args(args: dict) -> str:
|
||||
"""把工具调用参数压成一句简短摘要,放进步骤条文案(取最能代表本次调用的字段)。"""
|
||||
if isinstance(args, dict):
|
||||
@@ -102,7 +218,7 @@ async def run_skill_via_real_agent(
|
||||
SearchResult。返回 ``(results, error_detail)``:results 为空且 error 非空 = 该技能失败
|
||||
(researcher 据此回退直连检索)。
|
||||
"""
|
||||
from langchain_core.messages import AIMessage
|
||||
from langchain_core.messages import AIMessage, ToolMessage
|
||||
|
||||
from deerflow.config import get_app_config
|
||||
from deerflow.subagents.config import SubagentConfig
|
||||
@@ -206,6 +322,9 @@ async def run_skill_via_real_agent(
|
||||
# 这样 custom 事件能被 LangGraph **实时**刷给前端 SSE;若在嵌套 astream 里直接 emit,
|
||||
# 事件会被攒到整段跑完才一次性吐出(后端在跑、前端却一直 loading)。
|
||||
final_text = ""
|
||||
# 技能每次工具调用的 stdout(如 search.py 打印的 {success,results} JSON),用于**直接收割**
|
||||
# 真实检索结果,不依赖模型最后是否把它们汇总进 {materials}。
|
||||
tool_outputs: list[str] = []
|
||||
seen_msg_ids: set[str] = set()
|
||||
step_queue: asyncio.Queue = asyncio.Queue()
|
||||
|
||||
@@ -229,6 +348,10 @@ async def run_skill_via_real_agent(
|
||||
text = _content_to_text(m.content)
|
||||
if text.strip():
|
||||
final_text = text
|
||||
elif isinstance(m, ToolMessage):
|
||||
out = _content_to_text(m.content)
|
||||
if out.strip():
|
||||
tool_outputs.append(out)
|
||||
finally:
|
||||
step_queue.put_nowait(None) # sentinel:跑完/出错都让 pump 收尾
|
||||
|
||||
@@ -255,29 +378,45 @@ async def run_skill_via_real_agent(
|
||||
logger.warning("skill real-agent[%s]: astream failed: %s", skill_name, e, exc_info=True)
|
||||
return [], f"技能检索失败: {e}"
|
||||
|
||||
# 1) 模型最终汇总的 materials(它可能已按相关度筛选/去重)
|
||||
model_results: list = []
|
||||
parsed = parse_json(final_text or "", {"materials": []})
|
||||
raw_materials = parsed.get("materials")
|
||||
if not isinstance(raw_materials, list):
|
||||
return [], ""
|
||||
if isinstance(raw_materials, list):
|
||||
for m in raw_materials:
|
||||
if not isinstance(m, dict):
|
||||
continue
|
||||
content = str(m.get("content") or "").strip()
|
||||
title = str(m.get("title") or "").strip()
|
||||
if not content and not title:
|
||||
continue
|
||||
model_results.append(SearchResult(
|
||||
title=title or "(无标题)",
|
||||
content=content[:500],
|
||||
source=str(m.get("source") or skill_name).strip() or skill_name,
|
||||
published_at=str(m.get("published_at") or "").strip(),
|
||||
url=str(m.get("url") or "").strip(),
|
||||
relevance_score=0.8,
|
||||
source_type="skill",
|
||||
))
|
||||
|
||||
results: list = []
|
||||
for m in raw_materials[:max_results]:
|
||||
if not isinstance(m, dict):
|
||||
# 2) **直接从技能 stdout 收割真实检索结果**——哪怕模型最后没汇总、或别的检索词超时,
|
||||
# 成功检索词拿回的数据也照单全收(用户诉求:拿到的数据必须接收,不能整段丢)。
|
||||
harvested = _harvest_tool_results(tool_outputs, skill_name, max_results)
|
||||
|
||||
# 3) 合并去重(模型汇总优先、技能原始结果补齐),按 url/recUuid/title 去重,截断到上限。
|
||||
merged: list = []
|
||||
seen_keys: set[str] = set()
|
||||
for r in model_results + harvested:
|
||||
key = (r.url or "").strip() or (r.recUuid or "").strip() or (r.title or "").strip()
|
||||
if key and key in seen_keys:
|
||||
continue
|
||||
content = str(m.get("content") or "").strip()
|
||||
title = str(m.get("title") or "").strip()
|
||||
if not content and not title:
|
||||
continue
|
||||
results.append(SearchResult(
|
||||
title=title or "(无标题)",
|
||||
content=content[:500],
|
||||
source=str(m.get("source") or skill_name).strip() or skill_name,
|
||||
published_at=str(m.get("published_at") or "").strip(),
|
||||
url=str(m.get("url") or "").strip(),
|
||||
relevance_score=0.8,
|
||||
source_type="skill",
|
||||
))
|
||||
return results, ""
|
||||
if key:
|
||||
seen_keys.add(key)
|
||||
merged.append(r)
|
||||
if len(merged) >= max_results:
|
||||
break
|
||||
return merged, ""
|
||||
|
||||
|
||||
def _read_skill_body(skill) -> str:
|
||||
|
||||
+3
@@ -818,6 +818,9 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
intranet = IntranetSearchProvider(
|
||||
base_url=intranet_url,
|
||||
knowledge_base_name=ai_writing_config.get("intranet_knowledge_base", "samples"),
|
||||
# 兜底接口超时可在 config.yaml 的 ai_writing.intranet_search_timeout 调大,
|
||||
# 默认 600s(10 分钟),给内网慢检索留足时间。
|
||||
timeout=ai_writing_config.get("intranet_search_timeout", 600),
|
||||
)
|
||||
fb_keywords = keywords[:keyword_count] or [intent]
|
||||
fb_gathered = await asyncio.gather(
|
||||
|
||||
+44
@@ -1,7 +1,51 @@
|
||||
import re
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from typing import List
|
||||
|
||||
# 复合检索词里常见的「话题/类别后缀」——核心实体后面跟的修饰词。内网 ES 对
|
||||
# 「实体+话题」这类长词常检不出(如「特朗普竞选」0 条),但「实体」本身能检出
|
||||
# (「特朗普」30 条)。剥掉这些后缀回退到核心实体重试,能救回大量检索词。
|
||||
_TOPIC_SUFFIXES = (
|
||||
"竞选", "诉讼", "法律", "政策", "主张", "纲领", "动向", "动态", "影响", "分析",
|
||||
"进展", "现状", "情况", "问题", "事件", "活动", "关系", "布局", "态势", "格局",
|
||||
"趋势", "背景", "历史", "观点", "评价", "报道", "新闻", "研究", "报告", "概况",
|
||||
"走势", "前景", "措施", "方案", "计划", "战略", "改革", "发展",
|
||||
)
|
||||
|
||||
|
||||
def simplify_query(query: str) -> str:
|
||||
"""把一个「检不出」的复合检索词简化成更核心、更短的检索词,供 0 结果时重试。
|
||||
|
||||
规则(无需大模型,纯启发式):
|
||||
1. 多词(含空格)→ 取第一个词(通常是核心实体),如「美国政局 特朗普」→「美国政局」;
|
||||
2. 单词复合 → 反复剥掉结尾的话题后缀,保留核心实体,如「特朗普法律诉讼」→「特朗普」、
|
||||
「特朗普竞选」→「特朗普」。
|
||||
返回空串 = 无法进一步简化(调用方据此不再重试,避免空转)。
|
||||
"""
|
||||
s = (query or "").strip()
|
||||
if not s:
|
||||
return ""
|
||||
# 1) 多词:取第一个词
|
||||
if re.search(r"\s", s):
|
||||
first = re.split(r"\s+", s)[0].strip()
|
||||
if first and first != s and len(first) >= 2:
|
||||
return first
|
||||
# 2) 单词复合:循环剥离结尾话题后缀
|
||||
changed = s
|
||||
while True:
|
||||
stripped = changed
|
||||
for suf in sorted(_TOPIC_SUFFIXES, key=len, reverse=True):
|
||||
if stripped.endswith(suf) and len(stripped) - len(suf) >= 2:
|
||||
stripped = stripped[: -len(suf)]
|
||||
break
|
||||
if stripped == changed:
|
||||
break
|
||||
changed = stripped
|
||||
if changed != s and len(changed) >= 2:
|
||||
return changed
|
||||
return ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class SearchResult:
|
||||
|
||||
+19
-4
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
import logging
|
||||
from typing import List
|
||||
|
||||
from .base import BaseSearchProvider, SearchResult
|
||||
from .base import BaseSearchProvider, SearchResult, simplify_query
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -27,13 +27,17 @@ class IntranetSearchProvider(BaseSearchProvider):
|
||||
self,
|
||||
base_url: str = "",
|
||||
knowledge_base_name: str = _DEFAULT_KB,
|
||||
timeout: int = 300,
|
||||
timeout: int = 600,
|
||||
**_ignored,
|
||||
):
|
||||
self.endpoint = base_url or _DEFAULT_ENDPOINT
|
||||
self.knowledge_base_name = knowledge_base_name or _DEFAULT_KB
|
||||
# 内网 ES 接口很慢,超时硬编码放宽到 5 分钟(不依赖 env 配置)
|
||||
self.timeout = timeout
|
||||
# 内网 ES 接口很慢,兜底检索超时放宽到 10 分钟(不依赖 env 配置);构造方可传更大值覆盖。
|
||||
# 「咱们先把自己的做好」:我方兜底超时给足,避免因我方过早超时被甩锅成「检不出」。
|
||||
try:
|
||||
self.timeout = max(int(timeout), 1)
|
||||
except (TypeError, ValueError):
|
||||
self.timeout = 600
|
||||
# 收集本次运行的检索异常,供 researcher 节点取出并提示用户(ES 波动等)
|
||||
self.errors: list[str] = []
|
||||
# 解析失败时收集的原始响应体,供前端「错误返回结果」折叠展示
|
||||
@@ -49,6 +53,17 @@ class IntranetSearchProvider(BaseSearchProvider):
|
||||
self.raw_responses.append(body)
|
||||
|
||||
async def search(self, query: str, max_results: int = 5, **kwargs) -> List[SearchResult]:
|
||||
results = await self._search_once(query, max_results)
|
||||
# 0 结果兜底:复合检索词常检不出(如「特朗普竞选」),换成更核心的检索词重试一次
|
||||
# (「特朗普竞选」→「特朗普」)。仅在首检为空、且能简化出不同的词时才重试。
|
||||
if not results:
|
||||
simpler = simplify_query(query)
|
||||
if simpler and simpler != query:
|
||||
logger.info("intranet search empty for %r, retry with simplified %r", query, simpler)
|
||||
results = await self._search_once(simpler, max_results)
|
||||
return results
|
||||
|
||||
async def _search_once(self, query: str, max_results: int = 5) -> List[SearchResult]:
|
||||
import httpx
|
||||
|
||||
payload = {
|
||||
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
"""Add login_params_json column to task_buttons.
|
||||
|
||||
按钮管理「外部链接」按钮新增「追加登录参数」能力(与应用管理一致):``url`` 外链在
|
||||
拼好地址后,可再依次追加 ``?{key}={value}``,``value`` 由前端运行时从 ``userInfo``
|
||||
按 ``source`` 解析(``other`` 时取字面 ``custom_value``)。整列以 JSON 存一个
|
||||
``[{key, source, custom_value}, ...]`` 列表,默认空 → 旧按钮行为不变。
|
||||
|
||||
ORM: ``TaskButtonRow.login_params_json = PortableJSON()``(SQLite/PG 原生 JSON,
|
||||
MySQL 落 TEXT)。``create_all`` 给全新库建列;已有库由启动期
|
||||
``_ensure_orm_columns_sync`` 补列。本 revision 保持 alembic 图一致并留档。
|
||||
|
||||
Revision ID: 20260625_01
|
||||
Revises: 20260620_02
|
||||
Create Date: 2026-06-25
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import inspect
|
||||
|
||||
|
||||
revision = "20260625_01"
|
||||
down_revision = "20260620_02"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _has_table(table_name: str) -> bool:
|
||||
inspector = inspect(op.get_bind())
|
||||
return table_name in inspector.get_table_names()
|
||||
|
||||
|
||||
def _has_column(table_name: str, column_name: str) -> bool:
|
||||
if not _has_table(table_name):
|
||||
return False
|
||||
inspector = inspect(op.get_bind())
|
||||
return any(col["name"] == column_name for col in inspector.get_columns(table_name))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
if _has_table("task_buttons") and not _has_column("task_buttons", "login_params_json"):
|
||||
op.add_column(
|
||||
"task_buttons",
|
||||
sa.Column("login_params_json", sa.Text(), nullable=True),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
if _has_column("task_buttons", "login_params_json"):
|
||||
op.drop_column("task_buttons", "login_params_json")
|
||||
+1
@@ -19,6 +19,7 @@ def _normalize(data: dict[str, Any], updated_by: str | None) -> dict[str, Any]:
|
||||
"id_param": str(data.get("id_param") or "id"),
|
||||
"id_kind": str(data.get("id_kind") or "task"),
|
||||
"open_mode": str(data.get("open_mode") or "blank"),
|
||||
"login_params": list(data.get("login_params") or []),
|
||||
"enabled": bool(data.get("enabled", True)),
|
||||
"sort_order": int(data.get("sort_order", 0)),
|
||||
"updated_by": updated_by,
|
||||
|
||||
+6
-1
@@ -29,7 +29,7 @@ from sqlalchemy import Boolean, Integer, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from deerflow.persistence.base import Base
|
||||
from deerflow.persistence.types import BeijingDateTime
|
||||
from deerflow.persistence.types import BeijingDateTime, PortableJSON
|
||||
|
||||
|
||||
class TaskButtonRow(Base):
|
||||
@@ -48,6 +48,11 @@ class TaskButtonRow(Base):
|
||||
id_kind: Mapped[str | None] = mapped_column(String(16), default="task")
|
||||
# ``url`` 外链打开方式:``blank``=新窗口(默认)/ ``self``=当前窗口。可空,读时归一化为 ``blank``。
|
||||
open_mode: Mapped[str | None] = mapped_column(String(16), default="blank")
|
||||
# ``url`` 外链追加的登录参数(与应用管理一致):在拼好的地址后再依次追加
|
||||
# ``?{key}={value}``,``value`` 由前端运行时从 ``userInfo`` 按 ``source`` 解析
|
||||
# (``other`` 时取 ``custom_value`` 字面值)。每项 ``{key, source, custom_value}``。
|
||||
# 旧行 / NULL 读时归一化为 ``[]``。
|
||||
login_params_json: Mapped[list] = mapped_column(PortableJSON(), default=list)
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
sort_order: Mapped[int] = mapped_column(Integer, default=0)
|
||||
updated_by: Mapped[str | None] = mapped_column(String(64))
|
||||
|
||||
+3
@@ -21,6 +21,8 @@ def _to_dict(row: TaskButtonRow) -> dict[str, Any]:
|
||||
data["id_param"] = data.get("id_param") or "id"
|
||||
data["id_kind"] = data.get("id_kind") or "task"
|
||||
data["open_mode"] = data.get("open_mode") or "blank"
|
||||
# JSON 列改名 login_params_json → login_params;旧行 / NULL 归一化为 []。
|
||||
data["login_params"] = data.pop("login_params_json", None) or []
|
||||
return data
|
||||
|
||||
|
||||
@@ -35,6 +37,7 @@ def _row_from(data: dict[str, Any], updated_by: str | None) -> TaskButtonRow:
|
||||
id_param=str(data.get("id_param") or "id"),
|
||||
id_kind=str(data.get("id_kind") or "task"),
|
||||
open_mode=str(data.get("open_mode") or "blank"),
|
||||
login_params_json=list(data.get("login_params") or []),
|
||||
enabled=bool(data.get("enabled", True)),
|
||||
sort_order=int(data.get("sort_order", 0)),
|
||||
updated_by=updated_by,
|
||||
|
||||
@@ -117,6 +117,12 @@ CALL _add_column_if_missing('knowledge_notes', 'folder', 'VARCHAR(512) NULL DEFA
|
||||
-- ORM: RoundtableJobRow.task_id = String(128), nullable, indexed。
|
||||
CALL _add_column_if_missing('roundtable_jobs', 'task_id', 'VARCHAR(128) NULL DEFAULT NULL');
|
||||
|
||||
-- ---- 按钮管理「外部链接」追加登录参数,对应 migration 20260625_01 ----------------
|
||||
-- login_params_json: url 外链追加的登录参数列表 JSON([{key,source,custom_value},...]),
|
||||
-- 由前端运行时从 userInfo 解析后拼到地址尾部;NULL/空 = 不追加(旧行为)。
|
||||
-- ORM: TaskButtonRow.login_params_json = PortableJSON()(MySQL 落 TEXT)。
|
||||
CALL _add_column_if_missing('task_buttons', 'login_params_json', 'TEXT NULL DEFAULT NULL');
|
||||
|
||||
-- ---- 清理临时存储过程 --------------------------------------------------------
|
||||
DROP PROCEDURE IF EXISTS _add_column_if_missing;
|
||||
|
||||
|
||||
Reference in new issue
Block a user