sync update from LAN repo (source dhk-1 @ 3e87be55)

This commit is contained in:
gozeros committed 2026-10-21 15:06:08 +08:00
1 parent ab6068a7c1
commit 42f2a26fa8
1482 files changed
+227759 -9584

No files matched your search

+635 -45
View File
@@ -4,6 +4,52 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
### Ordinary Q&A map visualization
The ordinary lead-agent prompt includes `_ORDINARY_QA_VISUALIZATION_SECTION` only for normal, non-bootstrap, non-writing, non-notebook, non-scheduled runs and defines the highest-priority output-carrier gate. Only an explicit current-request ask for HTML, a webpage/page, dashboard, large-screen view, or `.html` file permits reading `page-report-generation`, calling `page_report_template`, or writing HTML. “Draw a flowchart/chart/map/mind map” and “make a visualization” never imply HTML: ordinary flowcharts must be inline Mermaid, ordinary charts keep the established message-inline path, and requested maps emit strict JSON in a `map-json` fence. A mind map is emitted only for an explicit current-request ask for a mind map/knowledge map (or `analysis-mindmap`); ordinary analyses, summaries, outlines, flowcharts, and generic JSON must never auto-upgrade into one. Its strict `mindmap-json` contract is a bounded rooted tree (`id`, `parentId`, `label`, semantic `kind`, optional detail/source refs), targeting 12–24 nodes with a hard cap of 36, at most four layers including the center topic, at most six top-level dimensions and three children beneath every later node. Models must consolidate related facts into `detail`/`sourceRefs`, not make one node per report bullet, datum, paragraph, or reference. An unfinished dedicated `mindmap-json` fence (and compatible historical `mindmap` fence) remains an ordinary streaming code block; only the completed, valid object lazy-renders as a compact left-to-right message card. Clicking a parent node, or its +/− control at the outgoing connection point, expands/collapses its child tree and the canvas uses an explicit visible edge stroke. The full dialog keeps its close and view tools in the top-right and a layout-derived all-map preview in the bottom-right; the message preview also permits pan, wheel zoom, and zoom/reset controls. The message card's icon-only download control opens PNG/SVG/JSON choices. Generic JSON never invokes this renderer. Ambiguity defaults to inline output. Once the HTML gate passes, delivery is mandatory as a sandbox-generated `/mnt/user-data/outputs/*.html` artifact opened with `present_files`; a full inline `html` fence is forbidden as a substitute. The shared frontend `MarkdownContent` keeps its exact legacy branch when no map-shaped fence exists; dedicated `map-json` fences and complete generic `json` fences that contain map fields and pass the full map schema are validated and lazy-rendered from `components/workspace/visualizations/`. Map/loading/error cards use the full message-content width so their edges align with the step panel. Non-map and incomplete generic JSON retain the ordinary code-block path.
HTML artifacts are single-page by default. Omitted `preferred_page_count` is normalized to `1` during direct validation. A count greater than one is allowed only when the current user directly asks for multiple pages or gives a page count; content length and report chapter count never imply multi-page intent. Explicit multi-page intent without a count defaults to a three-page lightweight portal. It is one artifact HTML, not a folder of linked pages: `<main class="pages">` contains logical `.page` sections with unique `id`/`data-page`, and `href="#page-..."` navigation works in the sandboxed Blob preview through the injected anchor handler. Its validator requires the local Bootstrap/Alpine assets, `.pages`, multiple `data-page` sections and real anchor navigation: every actual `<a href="#page-...">` and every script-set `location.hash` must match a unique logical-page id, and each page must be reachable; comments or `#overview`/`#stations` data-page values cannot spoof this check. `MultiPageHtmlMiddleware` repeats that structural inspection at the `present_files` boundary, so a model that skips validate or writes custom tabs cannot expose a nonconforming portal. A user-uploaded background goes only on the requested logical `.page` in this mode (home page when unspecified), not on every page. When a template is explicitly requested, prepare strictly excludes profiles from the wrong page-count mode, persists `requested_page_count` and `page_mode`, and validate reloads them. Validation rejects multiple detected page views (`data-page`, tabpanels, `.report-page`, or `.view`) in single-page mode, or a single view when multi-page was explicitly requested.
Ordinary HTML generation is from scratch by default: do not read `page-template-library`, call catalog/prepare, or select a bundled layout unless the current user explicitly asks to use a template. The agent writes the final path directly under `/mnt/user-data/outputs/`, which lets frontend live preview open on the first `write_file`; it then validates and presents the same file without a bash/cp copy. Single-page HTML means a scrollable responsive document, not a fixed single-screen canvas. Direct validate derives `layout_mode` and page count from `page_brief` / `preferred_page_count`; it rejects `html/body overflow:hidden`, fixed 1920×1080 viewport scaling, undersized G6 containers, and long LR diagrams. Explicit template requests may use the shared `light-document` / `dark-document` profile backed by `report-document-responsive-template.html`; prepare creates its editable `working_copy` directly in outputs and persists the same constraints.
For an explicit multi-page HTML request without an explicit template request, `MultiPageHtmlMiddleware` rejects `page_report_template` catalog/prepare calls; the model must make one complete first `write_file`, then one validate. A failed validation returns patch-only repair guidance: retain the artifact and use the smallest `str_replace` edits for all listed CSS/navigation/JSON/DOM errors before one revalidation; do not recreate the page. `html_postprocess.maybe_fix_generated_html` also injects a marker-guarded dark Bootstrap compatibility layer into dark HTML during every write or replacement. It sets Bootstrap body/emphasis/secondary tokens and explicit readable foregrounds for `.card`, `.data-table` cells, `.modal-*`, and common form controls. Validation mirrors this by rejecting dark `.card`, `.data-table`, and modal markup without an explicit matching foreground color, so a preview cannot silently render black text on dark panels.
### Generated HTML artifact visualization runtimes
The same normal lead-agent prompt includes `_HTML_ARTIFACT_VISUALIZATION_SECTION`, but its contents are executable only after the explicit HTML-intent gate above passes; visualization content alone never implies this intent. New generated HTML explicitly loads browser distributions from Gateway `/api/public/html-runtime/`: `bootstrap.min.css`, `alpine.min.js`, `bootstrap.bundle.min.js`, `g6.min.js`, `echarts.min.js`, and then `deerflow-runtime.js`; it can also load the packaged country/region map catalog: `china-provinces`, `us-states`, `japan-prefectures`, and `taiwan-counties`. Bootstrap is the responsive layout/modal layer and Alpine is the small state/list-rendering layer; jQuery, React, Vue, Babel and public CDNs are not part of this artifact contract. The files are package resources under `packages/harness/deerflow/html_runtime/assets/` and ship with the harness wheel, so HTML execution does not depend on frontend Vite/node_modules assets, Leaflet, map tiles, or a public CDN. `config.yaml -> html_runtime.base_url` is the browser-reachable absolute runtime base written into generated source (localhost default; deployments must replace it with the real Gateway origin). Flowcharts use paired `data-df-flow` JSON blocks and AntV G6, not Mermaid; charts use `data-df-chart`; maps use ECharts Canvas-backed `data-df-geo` with the local GeoJSON catalog. `data-df-chart` must be strict JSON: no JavaScript callbacks or function-valued colors, with literal colors/data-item styles and string tooltip formatters instead. Dark HTML must set `data-df-theme="dark"` and explicitly give ECharts global/title/legend/axis/visible-series labels readable light text, including `nameTextStyle` for named axes. Map `description` values are plain text because the runtime escapes markup; click-to-filter or page-switch behavior listens to `deerflow:geo-map-click` and uses a valid `#page-...` anchor. `deerflow-runtime.js` waits until a hidden logical page has a real layout before creating ECharts and tracks later container changes with `ResizeObserver`, so map/chart canvases never depend on a browser-window resize for their first visible render. It initializes all three contracts, opens a detail popover for described regions/markers, and exposes catalog URLs through its public dynamic response. Map large screens use a centered map stage with narrow side rails and bottom summaries, rather than cards layered across a full-page map background. Frontend runtime replacement is bypassed whenever this explicit Gateway runtime is present. This remains separate from ordinary message rendering; ordinary `map-json` keeps its existing frontend tile renderer and `VITE_MAP_TILE_URL` configuration, and ordinary `flow-json`/`chart-json` must not be reintroduced.
User-uploaded raster images can be used as generated-HTML backgrounds through signed, unauthenticated `GET /api/public/html-images/{token}` URLs. `UploadsMiddleware` mints one URL per uploaded `.jpg/.jpeg/.png/.webp/.gif/.bmp` and injects it into `<uploaded_files>` as `Public URL`; the HTML contract requires that exact absolute URL in `background-image`, never a sandbox path, artifact API path, relative path, or base64 data URI. Single-page artifacts place it on `html/body`; the multi-page portal places it only on the requested logical page. Tokens are HMAC-signed, do not expire (the URL is written into durable HTML), and only grant read access to one file under the thread `uploads/` or `outputs/` directory. Engine: `deerflow.html_runtime.public_images`; router: `app/gateway/routers/html_images.py`; tests: `tests/test_html_images.py`.
The G6 runtime must not mutate author layout by auto-switching long LR graphs to TB or by increasing the target element height after render. Layout quality is decided before delivery: keep a flow to at most eight nodes where possible; otherwise split it or declare TB with at least 600px. Each flow gets its own document row and a parent that can grow naturally, preventing canvas overlap with later sections.
Markdown-to-HTML conversion must inventory source Mermaid/flowcharts, charts, and maps before editing. Direct `page_report_template validate` derives `required_visualizations` from its `required_features` and `page_brief`; an explicit-template prepare also persists a machine-readable `visualization_contract` in the per-output brief, which validate reloads and merges with its arguments. Required flowcharts fail validation unless the HTML contains `g6.min.js`, `deerflow-runtime.js`, a `.df-flow` container, and `data-df-flow`; embedded Mermaid and inline `new G6.Graph(...)` are errors. Required charts use ECharts + `data-df-chart`; required maps use ECharts + `data-df-geo`, an `.df-geo-map` container, and one of the four local GeoJSON catalog entries. Leaflet, `data-df-map`, map tiles, and manual `echarts.init` are validation errors for generated HTML. A failed report must be repaired and revalidated before `present_files`.
G6 nodes use light semantic rounded cards with matching borders, dark high-contrast text, centered type-prefixed labels, subtle shadows, and non-animated initial rendering (avoids a faded first frame). More than seven nodes automatically switch an `LR` request to layered `TB` layout and expand a short container to 720px, preventing long workflows from shrinking into illegible shapes; drag/zoom remain enabled.
### 问答记忆运行时总开关
通用/个人问答记忆由 `system_settings.json -> memory.enabled` 控制,默认 `false`,并且是该通用记忆域的权威总开关。关闭时 `lead_agent` 不挂载通用 `MemoryMiddleware`,工具注册不暴露 memory/hindsight 工具,prompt 不读取通用记忆,因此也不会触发 V1 的延迟提取模型调用。普通问答项目的项目级跨会话记忆是独立安全域,只受项目自身 `enabled` 控制,不得被这个通用总开关阻断。`config.yaml -> memory` 仅保留 provider/version/限额等参数,不能自行启用通用记忆。管理员通过 `GET/PUT /api/system-settings/memory` 和「设置 → 问答记忆」修改通用记忆开关;下一次创建的问答立即按新值构建。
### 项目空间(固定问答模式)
- `app/gateway/routers/analysis_projects.py`:用户私有项目 CRUD 和项目下新建/查询对话;更新请求不允许 `mode`,对话归属不通过可编辑 metadata 保存。
- 项目线程 metadata 的 `analysis_project_id` / `analysis_project_mode` / `analysis_project_name` 仅用于最近对话的类型、项目名与路由展示;创建、列表修复和每次运行都由服务端覆盖,授权与真实归属仍以 `analysis_project_threads` 绑定为准。
- `deerflow.persistence.analysis_projects`:`analysis_projects` / `analysis_project_threads` / additive `analysis_project_configs`,经 models 注册和 Gateway deps 初始化自动建表。配置表保存项目默认智能体、策略 Lens、报告结构快照及每条对话所选智能体,避免修改旧表破坏已有部署。
- `app/gateway/analysis_projects.py::prepare_project_run` 在共享 `start_run` 前置读取绑定、校验所有者及知识库、覆盖运行模式;运行知识范围是项目自身普通/助手知识库与该线程已绑定智能体普通知识库的去重并集,不能只取智能体配置而丢掉项目知识库;`authz.py` 同时保护直接读取线程。不要将 `analysis_project` 加入公共 context 白名单。
- `deerflow.agents.analysis_project` 提供三个模式的系统约束和固定方案模板;方案复用 deep_research_report 的 fixed structure,thread_harvester 在方案项目中额外采集服务端检索的引用。普通写作仍使用原有 adaptive 配置。
- 普通问答项目的跨会话记忆是独立安全域,权威 scope 为 `(owner_user_id, project_id)`,只信任 `analysis_project_threads` 的服务端绑定。Gateway 会清除客户端伪造的 `project_memory_scope` / `project_memory`,再由 `prepare_project_memory` 注入;`services.inject_validated_analysis_project_context` 同时禁用通用 `MemoryMiddleware` 与 memory/hindsight 工具。harness 只消费转义、有界、明确标为非指令数据的 `<project_memory>`,不得反向导入 `app.*`。
- 项目记忆只受项目自身 `enabled` 和 store readiness 门控;`system_settings.json -> memory.enabled` 只管理通用/个人记忆,即使它关闭,`prepare_project_memory`、成功回调入队、executor 与 dispatcher 仍须为已启用的 QA 项目工作。项目暂停会取消并围栏该项目 queued/running job。项目记忆异常必须降级为无记忆问答,不能污染普通运行配置。
- 项目记忆 job 的内部 `causal_generation` 保存在 `payload_json`(不得为此新增数据库列;兼容已由 `create_all` 建好的表),状态只按最新因果代次判断;一个全量重建的所有 job 共用 generation,任一失败使该批次失败。checkpoint 枚举完成前必须保持未 seal 的 `rebuilding`,避免先完成的部分 job 提前转 ready。dispatcher 通过 `claim_stalled_rebuilds` 恢复超过宽限期且没有活动 job 的 `rebuilding/recovering`,认领和重放期间召回必须 fail-closed。
- 持久化位于 `deerflow.persistence.analysis_project_memory`:`states` 保存项目设置和确定性快照,`items` 保存带来源、logical key、版本和可选 embedding 的原子记忆,`jobs` 保存脱敏后的幂等抽取任务。三个模型由统一 `Base.metadata.create_all` 自动建表,不需要新增 Alembic revision。
- 成功 run 只把最近完整 user/assistant exchange 入队,后台 executor 抽取并可调用共享 Wiki encoder。召回为 lexical + vector + importance/confidence/recency 混合排序;只有 embedding model/fingerprint 与 query encoder 完全匹配时使用向量,否则降级关键词。召回要排除当前 thread 的原子条目,并受 item/episode/token budget 限制;抽取任务中的 checkpoint 文本、项目名和背景在落库前必须脱敏。
- 对话编辑、单轮删除、清空、整线程删除以及直接 state 消息替换,必须先调用 `invalidate_project_memory_before_thread_mutation` 持久化失效并取消 lease,再修改 checkpoint;前置失效失败时拒绝修改。后续 rebuild 失败时保持 `rebuilding`,recall fail-closed。`start_rebuild` 删除旧的自动条目但保留手工或 pinned 条目;同 logical key 按 checkpoint `source_order` 收敛,不能由并发 worker 的完成顺序决定。dispatcher 只能按真实空槽 claim,过期任务达到 `max_attempts` 后必须终止。
- 管理 API:`GET/PATCH /api/analysis-projects/{id}/memory[/settings]`、`POST .../memory/rebuild`、`DELETE .../memory` 和 `.../memory/items*`;全部先走 `owned_project` 且只接受 `mode=qa`。前端 `ProjectMemoryPanel` 展示状态、快照、来源,并提供手工维护、固定、清空和重建。专项回归为 `tests/test_analysis_project_memory*.py` 加 `tests/test_analysis_projects.py`,必须同时覆盖内存与 SQLite、通用记忆关闭时 A 会话仍可沉淀到 B 会话召回、跨项目/跨用户隔离、项目暂停、来源失效、纠错顺序、租约与关键词降级。
- 方案生成首份报告对用户保持两步语义:「正常问答理解需求 → 结束问答并生成方案」。第一步应自然交流并先做语义级意图理解:复述真实目标、面向对象、可观察的成功结果与非目标/边界,推断必须标为暂定;后续消息增量更新整体理解,最新明确要求覆盖较早的冲突表述。仅在关键条件缺失时少量追问;信息足够后输出自然段式的简短「意图理解」,以及唯一的 `solution-intent-json` HTML 注释快照和末尾就绪标记,不得呈现为表单式访谈或额外确认阶段。快照的 `goal` / `audience` / `success_outcome` / `scope` / `constraints` / `deliverable` / `acceptance_criteria` 必须是有界非空字符串,`open_questions` 必须是有界字符串数组且只有为空时可就绪。`prepare_project_run` 删除客户端伪造的 `solution_requirements_confirmed` / `solution_intent`,只解析检查点中最新用户轮次后的最新助手回复;只有可读摘要、合法快照、末尾就绪标记和当前用户确认标记同时成立时才确认,并把归一化快照作为服务端权威 `solution_intent` 注入运行配置。`services.inject_validated_analysis_project_context` 负责绕过公共白名单安全注入这些服务端字段,确认后的系统提示把转义后的快照交给最终生成。已有 `report.md` 仅用于后续编辑的兼容解锁;`deep_research_report` 在首份方案未确认时必须拒绝执行。不要把该闸门退化为仅靠前端按钮或提示词的约定。
- 方案阶段的关键追问必须调用现有 `ask_clarification`:普通方案需求第一步至少给出 1 个最关键的可点击决策问题,一个调用只问一个决策点,提供 2–4 个稳定选项(含「其他/我补充」),总量限制 1–3 个,由 `ClarificationMiddleware` 继续输出既有 `{question, clarification_type, context, options, allow_custom, allow_multiple}` 载荷供前端点击。独立问题可同轮多次调用;不要用普通 Markdown 问句替代工具卡,也不要在用户回答前只输出快照/就绪标记。另有服务器校验的强制生成令牌 `<!-- solution-requirements:force -->`:只认本轮最后一条 user/human 消息,仍清除客户端伪造的 `solution_force_generate` / `solution_requirements_confirmed` / `solution_intent`。强制生成可复用 checkpoint 中最近一个合法但未就绪的快照(允许 `open_questions` 非空;没有则为 `None`),prompt 必须结合完整对话推断缺项、显式标注暂定假设并禁止再次澄清。确认/强制生成首报轮以 `WritingModeReportMiddleware(require_pipeline=True)` 做运行时兜底:纯文本“已生成”或澄清调用会被确定性改写为 `deep_research_report`;任何完成提醒/循环成功判定都必须同时看到固定成功工具结果和状态中真实登记的 `/mnt/user-data/outputs/report.md` artifact,被意图门禁拒绝的 ToolMessage 绝不能触发“已完成”提醒。当前用户轮次里只要管线返回结果却没有 artifact,就输出确定性的失败/重试提示并结束,禁止自动重试到循环上限;下一次用户点击「重新生成方案」才开始新调用。
- 方案项目最终落盘的 `report.md` 禁止 `[1]` / `【1】` 等数字引用标记。`deep_research_report_tool` 在 solution 模式下同时注入无编号引用要求并对最终 Markdown 做确定性清理:正文删除数字引用标记,参考来源改为无编号列表,代码围栏和行内代码保持原样;普通写作与深度研究不走该分支。后续方案修改的系统提示也必须维持这项约束。
- 前端 `frontend-web/src/pages/AnalysisProjectsPage.tsx` 和 `pages/analysis-projects/ProjectConversation.tsx` 嵌入现有 WorkspaceSidebar,复用消息流、输入框、研究进度和产物画布;策略项目默认打开真实消息流,用户问答、reasoning 与技能/工具执行步骤必须在运行中可见,结构化看板仍可一键切换。`StrategyBoard.tsx` 只解析对用户可见的「意图理解 / 多视角策略分析与辩论 / 完整策略方案」Markdown,不读取 reasoning/tool payload。完整策略方案以可导出的文档结构呈现。方案设置复用 `WorkflowConfigDialog` 管理报告结构。
- 回归测试:`tests/test_analysis_projects.py`(项目 CRUD、权限、模式锁定、运行配置、模板及记忆管理 API)和 `tests/test_analysis_project_memory*.py`(持久化、运行时、双后端可靠性与真实跨会话效果)。
DeerFlow is a LangGraph-based AI super agent system with a full-stack architecture. The backend provides a "super agent" with sandbox execution, persistent memory, subagent delegation, and extensible tool integration - all operating in per-thread isolated environments.
**Architecture**:
@@ -77,7 +123,7 @@ When making code changes, you MUST update the relevant documentation:
### Workflow Studio stream and canvas compatibility
- `app/gateway/workflow_agent_runner.py` is the boundary between LangGraph token chunks and durable workflow events. Provider reasoning in `AIMessage.additional_kwargs.reasoning_content`/`reasoning` is emitted to the UI as balanced inline `<think>` blocks; only visible answer content becomes the node's `text` output for downstream nodes. It must also emit every original `[AIMessage|ToolMessage, metadata]` tuple as `node.message.data.messages` with no whitelist, preview or length crop. Tool results and `additional_kwargs.clarification` are needed by the Studio's DeerFlow-compatible message UI and must survive durable SSE replay.
- `PatchedChatOpenAI` accepts provider response fields `reasoning_content` and `reasoning`, normalizing both into `AIMessage.additional_kwargs.reasoning_content`. For OpenAI-compatible models that declare `supports_thinking: true` without an explicit `when_thinking_enabled`, `create_chat_model` sends `extra_body.enable_thinking=true` whenever the frontend enables thinking. `app/gateway/workflow_agent_runner.py` is the boundary between LangGraph token chunks and durable workflow events. Provider reasoning in `AIMessage.additional_kwargs.reasoning_content`/`reasoning` is emitted to the UI as balanced inline `<think>` blocks; only visible answer content becomes the node's `text` output for downstream nodes. It must also emit every original `[AIMessage|ToolMessage, metadata]` tuple as `node.message.data.messages` with no whitelist, preview or length crop. Tool results and `additional_kwargs.clarification` are needed by the Studio's DeerFlow-compatible message UI and must survive durable SSE replay.
- An agent `ask_clarification` ToolMessage pauses the workflow at that agent node (`awaiting_input`), not as a terminal run. The pause records the source `nodeRunId`/`toolCallId`; resuming turns the submitted reply into the next user message on the same `wf-{run}-{node}` LangGraph thread. Keep the raw ToolMessage authoritative; do not invent a second card payload.
- `app/gateway/workflow_proposal_planner.py` converts the workflow controller's selected catalog roles into per-agent `mission`/`deliverable`/`scope`/`handoff` contracts. Only a selected visible `agentId` may contribute a contract; every omitted or malformed field falls back to a deterministic role-aware contract. Preserve this prompt-level worker boundary when adding strategies: researchers must not silently become final writers, and sequential roles must receive their upstream handoff binding plus an explicit next handoff.
- `app/gateway/routers/workflows_coze_compat.py` must return both `creator.self: true` and editable VCS metadata for an owned standalone workflow. Coze Playground derives its preview/read-only mode from those fields, so omitting `creator.self` disables node dragging even when the standalone page passes `readonly={false}`.
@@ -213,7 +259,7 @@ Independent of Workflow Studio and the existing roundtable. Plan: repo-root `doc
- **Config**: `config.yaml → report_collaboration` (`deerflow.config.report_collaboration_config.ReportCollaborationConfig`). Default **off** (`enabled` / `worker_enabled`).
- **Contracts**: `app/report_collaboration/contracts/` — snake_case Pydantic models matching `frontend-web/src/report-collaboration/api/types.ts`. SSE envelope uses camelCase (`eventId` / `sessionId` / `runId`). `node_run_id` is always `${node_id}-attempt{attempt}`.
- **Persistence**: `deerflow.persistence.report_collaboration` (`report_collaboration_*` tables, migrations `20260905_01` / `20260906_01` / `20260907_01`). SQL + memory stores; one active run per session via `active_dedupe_key`; command/message/run/version idempotency keys refuse cross-session reuse; event `seq` taken from the run row. Session delete removes children explicitly.
- **Gateway**: `/api/report-collaboration` (`app/gateway/routers/report_collaboration.py`). `enabled=false` → 503. Plan/run writes enqueue durable jobs and return immediately. Pre-run `POST /sessions/{id}/messages` runs a rule-based `IntentRouterAgent` + `RequirementResolver` (`app/report_collaboration/requirement/`): writes a requirement revision and clarification messages, never auto-starts planning. `POST /sessions/{id}/plan-requests` then runs `PlanProposalService` (`app/report_collaboration/planning/`): catalog projection (no SOUL/keys) → selector → 2–3 strategy RoleDemand graphs → assembler binds catalog agents/tools/quality gates → validator (acyclic DAG, artifact types, allowlisted tools). Select and start-run stay independent commands. `POST /runs` freezes role models, report-structure template and run budget. Rewrite / apply / restore are live (`reporting/`). `GET /runs/{id}/stream` is a read-only durable SSE tail (`execution/live_hub.py`): persist then publish, replay + `after_seq` / `Last-Event-ID`, heartbeat comments, token-delta coalesce before seq assignment. Disconnect does **not** cancel the run. Lifespan mounts `ReportCollaborationDispatcher` only when `enabled` **and** `worker_enabled` are both true; default both false.
- **Gateway**: `/api/report-collaboration` (`app/gateway/routers/report_collaboration.py`). `enabled=false` → 503. Pre-run intent dialogue uses the separately seeded built-in agent `report-collaboration-intent` through the shared real `/api/intent/init` + `/api/intent/stream` LangGraph thread path. The browser consumes upstream `messages-tuple` frames incrementally (text, provider reasoning and tool steps) and renders them in the normal `MessageList`; this path must never be replaced with `PlannerAgent.ainvoke()` or another wait-for-completion wrapper. `POST /sessions/{id}/intent-turns` receives only the completed streamed turn for durable mirroring; it does not invoke an intent model. An `INTENT_READY` terminal summary is converted to the report requirement and automatically fulfilled through `PlanProposalService`, with no first-plan confirmation click. The old pre-run `/messages` path returns `USE_STREAMING_INTENT`. `POST /sessions/{id}/plan-requests` remains for regeneration/compatibility. Workflow design after intent confirmation is LLM-generated, assembled against the safe agent catalog and validated (acyclic DAG, artifact types, allowlisted tools). Selecting a plan and starting a run remain independent commands. `POST /runs` freezes role models, report-structure template and run budget. Rewrite / apply / restore are live (`reporting/`). `GET /runs/{id}/stream` is a read-only durable SSE tail (`execution/live_hub.py`): persist then publish, replay + `after_seq` / `Last-Event-ID`, heartbeat comments, token-delta coalesce before seq assignment. Disconnect does **not** cancel the run. Lifespan mounts `ReportCollaborationDispatcher` only when `enabled` **and** `worker_enabled` are both true; default both false.
- **Runtime**: AgentScope adapters live in `app/report_collaboration/agentscope_runtime/` (not the publishable harness). `ConversationReplica` maps AgentScope 2.0.5 events 1:1 onto collaboration SSE types; TeamSay is `HINT_BLOCK`. `DeerFlowChatModelAdapter` wraps `create_chat_model` (no second Provider config); role models are frozen onto node `agent_snapshot` at `POST /runs` and missing/incapable models fail with 422 before the run is created. Fallback is recorded on the snapshot plus a durable `heartbeat` audit event. Nodes complete only after a validated artifact (`app/report_collaboration/execution/completion.py`). `TaskLedger` (`execution/ledger.py`) owns ready waves, attempts and terminal node status; AgentScope / coordinator suggestions never pick the next node. A writer-bearing run is `completed` only after `ReportFinalizer` publishes a formal version. `WaveExecutor` + `QualityRoleKernel` (`execution/quality_roles.py`) remain for fixtures: Verifier marks stale/contested claims, Analyst/Writer only cite supported IDs, Reviewer `revise` replans only the target write node plus review. Production worker injects `AgentScopeMemberKernel` (`agentscope_runtime/member_kernel.py`) with DeerFlow tools and per-run budget. `QualityGate` (`execution/quality_gate.py`) is the claim-level gate: conflicts/missing units, unverified facts, coverage gaps, and unmapped review targets fail before `validated`; `ArtifactBoard.supersede_previous` hides old attempts from downstream. `ReportCollaborationLiveHub` (`execution/live_hub.py`) fans out persisted envelopes to SSE viewers; `PublishingReportCollaborationStore` sanitizes secrets/privacy/hidden reasoning without truncating visible tool results, and writes redaction + event audits. Dispatcher (`execution/dispatcher.py`) atomically claims queued/recovering/expired-lease runs, heartbeats via `renew_lease`, two-phase cancels (`cancel_requested` → `finalize_cancel`), and recovers orphans (`execution/recovery.py`): keep completed work, mark leftover in-flight `interrupted`/`superseded`, open a new `planned` attempt. Late artifacts after cancel/reclaim are `superseded` and must not change terminal run status. Tests drive `dispatch_once()`; do not rely on the background loop. Per-run budgets (`security/budget.py`) fail-closed; per-user active-run caps are enforced at `create_run`.
- **Quality eval**: `app/report_collaboration/eval/` (`RC-BE-018`) — 20 annotated tasks (policy/market/enterprise/event/comparison ×4) with required angles, key facts, forbidden fabrications, templates and review rules. Deterministic scorer (no LLM judge) gates citation/angle/template/directed-mod; gold reports must pass, canned failures (including a roundtable-style essay) must fail and stay reproducible. Recommended runtime knobs live in `eval/thresholds.py` and match current `ReportCollaborationConfig` defaults (`worker_enabled` remains false). Tests: `tests/test_report_collaboration_eval.py`.
- **Runtime intervention**: `app/report_collaboration/interventions/` owns run-time natural-language commands. `RuntimeIntentRouterAgent` uses explicit rules first and a tool-free coordinator-model fallback only for ambiguity; its Pydantic decision cannot dispatch or mutate anything. `ImpactAnalyzer` validates server-side node/agent ownership, resolves angle-specific lineage, and enforces confidence/cost/parallel-branch confirmation (`intent_confirmation_threshold`, default `0.8`). `RuntimeCommandService` mirrors the human message, emits frozen command events, CAS-claims accepted commands, and applies them only through `TaskLedger` safe points. Rework persists command overlays, supersedes affected attempts/artifacts, creates monotonic attempts, and prevents late output from entering downstream. Waiting-node answers resume only those nodes. `rerun_node` without a legal target does not expand to the whole graph; requirement patches apply only after rework succeeds; stuck `executing` commands are reclaimed on takeover or lease timeout; `command.cancelled` is a durable SSE event. In-run `POST /sessions/{id}/messages` returns 409 (`RUN_ACTIVE`). Formal rewrite candidates and version apply/restore live in `reporting/` (`RC-BE-015`).
@@ -295,10 +341,14 @@ from deerflow.config import get_app_config
- Tools loaded via `get_available_tools()` - combines sandbox, built-in, MCP, community, and subagent tools
- System prompt generated by `apply_prompt_template()` with skills, memory, and subagent instructions
**agentfx built-in analyst + report convert skills.** `agentfx-analyst` (「任务研判报告助手」) is seeded from `app/gateway/routers/_agentfx_seed_assets/` before `_sync_legacy_agents`. After mandatory `web_search`, the agent writes each tool JSON to `/mnt/user-data/workspace/hits/qN.json` and runs **one** command — `task-report-build` (group `all`) — to fill the 14-category `task-reports.json` for the dashboard slots; the four per-page skills `task-report-{enemy,our,env,judge}` remain for re-running a single dashboard page. All five are thin CLIs over the shared stdlib engine `skills/public/task-report-lib/convert_lib.py` (no SKILL.md on the lib, so it never enters the skill list). The model must not `write_file` that JSON. Because the sandbox maps `/mnt/user-data/...` onto a host dir, a virtual absolute hits path can be `isdir`-true yet list empty on a Windows host — `resolve_hit_files` therefore tries the path as given, then the cwd-mapped sandbox path (`discover_user_data_root` + `resolve_sandbox_path` — bash `cd`s into the thread `workspace`, so `/mnt/user-data/outputs/task-reports.json` must land in that thread's `outputs/`, never `F:\mnt\...` on a Windows host), then the virtual-prefix-stripped relative form, then the bare dir name, and picks the first candidate that actually contains `.json`; a miss returns a `warnings` entry listing every tried form so the agent never has to write a debug script. Assemble of the 14 categories is the same function (`assemble_records`); the model must not copy or merge files. If the model is interrupted after convert, `task-reports.json` is already in outputs and the frontend import card also scans `GET /api/threads/{id}/artifacts` so入库 still works without `present_files`. Import remains `task-report-import`. Existing agent/skill directories are never overwritten. Tests: `tests/test_task_report_convert.py`, `tests/test_agentfx_seed.py`.
**agentfx built-in analyst + report convert skills.** `agentfx-analyst` (「任务研判报告助手」) is seeded from `app/gateway/routers/_agentfx_seed_assets/` before `_sync_legacy_agents`. After mandatory `web_search`, the agent writes each tool JSON to `/mnt/user-data/workspace/hits/qN.json`, writes task metadata as JSON data to the fixed `/mnt/user-data/workspace/task-context.json`, and runs **one constant command** — `task-report-build` (group `all`) — to fill the 14-category `task-reports.json` for the dashboard slots; untrusted task text never enters the shell command. The four per-page skills `task-report-{enemy,our,env,judge}` remain for re-running a single dashboard page. All five are thin CLIs over the shared stdlib engine `skills/public/task-report-lib/convert_lib.py` (no SKILL.md on the lib, so it never enters the skill list). The engine emits the exact category-specific shapes consumed by `sq-report.ts`, marks evidence-derived content with `[检索证据]`, rejects cross-task partial merges, and atomically writes Unicode JSON. The model must not `write_file` that JSON. Because the sandbox maps `/mnt/user-data/...` onto a host dir, `discover_user_data_root` + `resolve_sandbox_path` map virtual paths to the current thread's literal `user-data` root; hits, task context and output are canonically confined there, including symlink and Windows drive checks. `resolve_hit_files` may try confined mapped/relative/basename candidates and reports every miss so the agent never has to write a debug script. Assemble of the 14 categories is the same function (`assemble_records`); the model must not copy or merge files. If the model is interrupted after convert, `task-reports.json` is already in outputs and the frontend import card also scans `GET /api/threads/{id}/artifacts` so入库 still works without `present_files`. Import remains `task-report-import`. Existing agent/skill directories are never overwritten. Tests: `tests/test_task_report_convert.py`, `tests/test_agentfx_seed.py`.
**Forced-research built-in agent.** `forced-research-responder` (「强制检索输出助手」) is seeded from `app/gateway/routers/_forced_research_seed_assets/` before `_sync_legacy_agents`. `ForcedResearchMiddleware` is mounted only for this stable id. On every visible user turn it requires a successful information-gathering tool result before the model may finish; `tool_choice=required` is backed by an `after_model -> model` retry for gateways that ignore tool choice, with an eight-attempt failure cap that permits only an honest failure response. Skill discovery, reading `SKILL.md`, and writing a helper script do not satisfy the gate; web/knowledge/MCP results, non-skill uploaded-file reads, or successful Python skill-script execution do. Tool filtering removes `present_files`, `skill_manage`, task/schedule/memory/clarification surfaces. The middleware permits `write_file`/`str_replace` only for `/mnt/user-data/workspace/*.py` and permits `bash` only for `.py` execution without redirection/document paths, so skills can run scripts but the agent cannot create Markdown/document artifacts or write `outputs`. Tests: `tests/test_forced_research_agent.py`.
**Strategy-selection functional agents.** `app/gateway/routers/_strategy_selection_seed.py` seeds nine stable folder-backed agents from `_strategy_selection_seed_assets/` before `_sync_legacy_agents`: intent, governance/diplomacy/economy/security/public-opinion/law-policy angles, judge, and solution editor. Seeding is missing-directory-only and must never overwrite administrator changes. The global `strategy_agent_roles` registry classifies them independently of projects: intent/judge/solution are fixed, unique roles that may be configured but never added, disabled, reordered, or deleted; angle roles are a shared, ordered catalog that administrators may add, edit, disable (soft delete), restore, and reorder through `/api/strategy-agents`. Public `GET /api/strategy-agents` returns enabled identity/description metadata only; `include_disabled=true` or `include_config=true` is admin-only. Admin writes are `POST /api/strategy-agents/angles`, `PUT /api/strategy-agents/{id}`, and `DELETE /api/strategy-agents/{id}`; DELETE retains the folder-backed agent and role row to preserve historical references. Dynamic angles remain built-in folder-backed agents under the reserved `strategy-selection-angle-` id namespace, with `web` + `file:read`; a blank SOUL on create receives a safe role template. All strategy skill writes validate that names exist, are active, and are globally visible; generic agent CRUD cannot create or hard-delete reserved strategy IDs, and parallel execution rejects disabled roles. Angle configs reuse `policy-knowledge`; intent reuses `intent-clarification-policy`; the judge has no retrieval skill; the solution editor gets only `file:write`. Keep the SOUL contracts neutral and evidence-aware, with explicit de-escalation/civilian-protection framing and hard bans on operational combat guidance and targeted political manipulation. Tests: `tests/test_strategy_selection_seed.py`, `tests/test_strategy_agent_management.py`.
`POST /api/parallel-agents/run/stream` must remain observable during silent skill calls. Each running seat emits an `agent_progress` heartbeat every five seconds with the latest translated tool/reasoning/body phase and monotonic elapsed seconds; an upstream iterator that produces no frame for 180 seconds is closed and that seat emits `agent_error`, while the other seats continue. Set the terminal flag before queuing `agent_done` / clarification so a late heartbeat cannot reopen a completed card. The strategy workbench renders the intent Q&A history, readable skill execution steps and live heartbeat on the affected agent card; judge and synthesis traces use the same step translation. Tests: `tests/test_roundtable_model_fallback.py` and frontend `src/strategy-selection/lib.test.ts`.
**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
- Extends `AgentState` with: `sandbox`, `thread_data`, `title`, `artifacts`, `todos`, `uploaded_files`, `viewed_images`
- Uses custom reducers: `merge_artifacts` (deduplicate), `merge_viewed_images` (merge/clear)
@@ -330,7 +380,8 @@ Lead-agent middlewares are assembled in strict append order across `packages/har
15. **DeferredToolFilterMiddleware** - Hides deferred tool schemas from the bound model until tool search is enabled (optional)
16. **SubagentLimitMiddleware** - Truncates excess `task` tool calls from model response to enforce `MAX_CONCURRENT_SUBAGENTS` limit (optional, if `subagent_enabled`)
17. **LoopDetectionMiddleware** - Detects repeated tool-call loops; hard-stop responses clear both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer
18. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
18. **ContextWindowGuardMiddleware** - Final transient request-size guard; enforces the selected model's `max_input_tokens` after every context/tool injection
19. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
8a. **ToolMetricsMiddleware** - Appended immediately after `ToolErrorHandlingMiddleware` in `_build_runtime_middlewares`, so it sits *deepest* in the `wrap_tool_call` chain. Records one `tool_call_metrics` row per tool/skill/MCP invocation ? timing duration and bucketing failures into a coarse `error_category` (rate_limited / ip_blocked / auth_error / timeout / network_error / not_found / empty_result / tool_error). It catches both raised exceptions (before #8 converts them) and *self-handled* errors returned as error `ToolMessage`s (search tools that swallow HTTP 429/403). It also captures the targeted Agent Skill into `tool_call_metrics.skill_name`: the lead agent invokes a skill by `read_file`-ing its `/mnt/skills/<category>/<name>/SKILL.md` path (per the `<skill_system>` prompt block ? there is no dedicated "use skill" tool), so `skill_name` is recovered by scanning every tool call's string arguments for the `skills/{public,custom}/{name}/` path shape (`_SKILL_PATH_RE`), plus the explicit `name` argument of `skill_view`/`skill_manage`/`skill_archive`. `skill_view`-specific success detection handles its swallowed failures (`Skill 'x' not found.` returned as a plain string rather than raised). Regression coverage: `tests/test_tool_metrics_middleware.py`. Writes are best-effort fire-and-forget. **Per-run skill merging**: `SqlToolMetricsStore.record` upserts skill rows by `(run_id, skill_name)` instead of inserting one row per call ? a skill touched by several tool calls / retries inside one run collapses to a single row. Any success in the run wins (a skill that failed then finally succeeded is recorded as one `success`); a skill that only ever failed yields one `error` row whose `error_message` accumulates every attempt's message; durations are summed. Non-skill tool calls, and skill calls with no `run_id`, keep one row each. The read?merge?write is guarded by a per-event-loop `asyncio.Lock`. Surfaced by the admin `GET /api/admin/tool-metrics` router (which has a `kind=skill` filter and a per-skill `/summary` breakdown).
9. **PromptPrefixMiddleware** - Injects the admin-configured prompt prefix into the model request only. The prefix is tagged onto the first human message's `additional_kwargs['prompt_prefix']` by the Gateway (`app.gateway.services`) and prepended to content here at call time, so the persisted/displayed message stays prefix-free
10. **SummarizationMiddleware** (`DeerFlowSummarizationMiddleware`) - Context reduction when approaching token limits (optional, if enabled). **Non-destructive**: overrides the stock `before_model` (which persists a `RemoveMessage(REMOVE_ALL_MESSAGES)`) to a no-op and instead compresses **transiently** in `wrap_model_call` ? the model request is replaced with `[summary, *recent]` while the full original conversation stays in state (so the frontend keeps rendering the user's real Q&A). The `summary` message is `hide_from_ui` and never persisted; summaries are cached **per-thread in a process-wide, thread-safe LRU** (`_GLOBAL_SUMMARY_CACHE`, bounded to 512 threads, sticky boundary). The cache is module-level — **not** on the middleware instance — because `make_lead_agent` rebuilds a fresh middleware on every run; an instance cache would be cold each turn and force a blocking summary LLM call on every turn of a long conversation. Keyed by `thread_id` (resolved like `ThreadDataMiddleware`), so a summary computed on one turn is reused on the next, paying for an LLM call only on the first threshold crossing or a genuine re-compaction. **Threshold-first gating**: `_plan_compression` returns passthrough (no compression at all) whenever the *full* current conversation is still under the trigger — the cache/summary substitution only engages once the real context crosses the configured trigger. **Stickiness headroom** (`_apply_stickiness_headroom`, `_KEEP_HEADROOM_FRACTION=0.75`): after the `keep` policy picks a cutoff, the kept suffix is forced below ~75% of *every* trigger before summarizing. Without it a `keep` in *messages* (e.g. 20) can itself exceed a *token* trigger (e.g. 60000) when recent messages are large (tool outputs / pasted docs), leaving the "compressed" context still over the trigger so it re-summarizes on every follow-up question; the headroom makes the sticky boundary actually stick (re-compaction only after genuinely new content). When the genuine (blocking) summarize path runs, it emits a `{"type":"context_compacting","message":"正在压缩上下文…"}` event on LangGraph's `custom` stream channel (the cheap reuse path stays silent); the chat frontend's `onCustomEvent` renders it as a transient toast. Tests: `tests/test_summarization_middleware.py`
@@ -342,7 +393,8 @@ Lead-agent middlewares are assembled in strict append order across `packages/har
16. **DeferredToolFilterMiddleware** - Hides deferred tool schemas from the bound model until tool search is enabled (optional)
17. **SubagentLimitMiddleware** - Truncates excess `task` tool calls from model response to enforce `MAX_CONCURRENT_SUBAGENTS` limit (optional, if `subagent_enabled`)
18. **LoopDetectionMiddleware** - Detects repeated tool-call loops; hard-stop responses clear both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer
19. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
19. **ContextWindowGuardMiddleware** - Final transient request-size guard; enforces the selected model's `max_input_tokens` after every context/tool injection
20. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
### Configuration System
@@ -362,6 +414,24 @@ Configuration priority:
Config values starting with `$` are resolved as environment variables (e.g., `$OPENAI_API_KEY`).
`ModelConfig` also declares `use_responses_api` and `output_version` so OpenAI `/v1/responses` can be enabled explicitly while still using `langchain_openai:ChatOpenAI`.
`write_file_transport` defaults to `native` and remains available for backward
compatibility. For a gateway that streams ordinary text but buffers long tool
arguments, administrators should use Settings → File streaming compatibility.
That runtime switch applies `buffered_gateway` to every compatible configured
model from the next question without restarting the Gateway. The model still
selects the original `write_file`, narrowed to
path/description/append for that request. The adapter then starts a tool-free
ordinary-content body request inside the same logical ChatModel stream and
projects it onto canonical incremental `write_file` chunks. LangGraph and
ToolNode receive one original writer call; `begin_write_file` is neither exposed
nor executed in this mode. Only a normal terminal frame after at least one body
byte appends the boolean commit marker. Provider reasoning and a leading inline
`<think>` stay outside the file; empty bodies, truncation and unexpected tool
calls abort before sandbox I/O. Ordinary answers remain a single provider
request. Turning the switch off restores each model's configured/native
transport. Native models must remain byte-for-byte unchanged while the switch
is off. `artifact_renderer` and `text_bridge` exist for older configurations
and are not the recommended buffered-gateway transport.
**Extensions Configuration** (`extensions_config.json`):
@@ -392,17 +462,19 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + repeatable `tag_ids` multi-select filter, OR-matched; each item carries `tags` + owner-authored `detail`); `GET /{name}` - details; `PUT /{name}` - update enabled; `PUT /{name}/detail` - set the long-form `detail` text (custom skill: owner or admin; built-in/public: admin only ? materializes a `skills` row on demand); `GET /{name}/content` - raw SKILL.md source, restricted to an admin or the publisher (custom-skill owner); other users get 403 and see only description + detail; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data |
| **Agents** (`/api/agents`) | `GET /` - list built-in, owned, and published agents (optional `search` + repeatable `tag_ids` multi-select filter, OR-matched; each item carries `tags` and `featured_order`); `POST /` - create user-owned agent; `GET/PUT/DELETE /{id}` - read/update/delete by stable agent id; `GET /check` - validate non-empty display name (duplicates allowed); `GET /selector` - homepage agent picker (single admin-curated list, visibility-filtered, sorted by `featured_order` asc); `PUT /{id}/featured` - **admin** sets/clears the homepage `featured_order` (must be `published=true` to feature; passing `order: null` clears unconditionally); `PUT /featured/order` - **admin** bulk-reorders the homepage list (body `{agent_ids: [...]}` ? indices become 0..N-1) |
| **Tags** (`/api/tags`) | `GET /` - list tags (optional `tag_type` filter; open to all); `POST /` - create tag (admin); `PUT /{id}` / `DELETE /{id}` - update/delete tag (admin); `GET /assignments/{target_type}/{target_id}` - tags on a resource; `PUT /assignments/{target_type}/{target_id}` - replace a resource's tags (admin). Tags are typed `agent`/`skill`/`scheduled_task` and may only tag resources of their own type. |
| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); `GET /list` - list; `DELETE /{filename}` - delete |
| **Tags** (`/api/tags`) | `GET /` - list tags (optional `tag_type` filter; open to all); `POST /` - create tag (admin); `PUT /{id}` / `DELETE /{id}` - update/delete tag (admin); `GET /assignments/{target_type}/{target_id}` - tags on a resource; `PUT /assignments/{target_type}/{target_id}` - replace a resource's tags (admin). Tags are typed `agent`/`skill`/`scheduled_task`/`recommended_question`/`light_app` and may only tag resources of their own type. |
| **Upload policy** (`/api/upload-policy`) | Database-backed global singleton. `GET /` is authenticated; `PUT /` and `POST /reset` are admin-only atomic replacements with revision/audit metadata. Startup seeds the real database row with 20 files, 100 MiB total, documents/images 10 MiB each, spreadsheets 5 MiB, and large-file budgets 12k/32k chars. The validated row is also published to the harness runtime registry without introducing an `app.*` import. |
| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` reads the latest database policy, enforces category extensions/sizes/total/count, validates signatures/OOXML containers/text encoding, then commits the whole staged batch or rolls it back; optional document conversion uses `<original-name>.<ext>.md`. `GET /limits` returns the full effective policy plus compatibility fields; `GET /list` lists; `DELETE /{filename}` deletes. Uploaded raster images also receive a signed public HTML URL (`/api/public/html-images/{token}`) so generated pages can use them as `background-image`. |
| **Task Deep-link Threads** (`/api/task-deeplink-threads`) | Dedicated shared history index for the `3qfx` / `rwfx` / `xdfx` single-agent task workspaces. `GET /?task_id=&go_path=&limit=` performs an indexed `(task_id, go_path)` lookup in `task_deeplink_threads` and joins `threads_meta` by `thread_id` for the current title/status/timestamps; LangGraph checkpoints remain the source of truth for messages. Thread metadata carrying `taskRecordId + taskGoPath` is dual-written atomically by both SQL and memory thread stores and deleted with the thread. `3qfx/rwfx` use the task id as `taskRecordId`; `xdfx` uses the original action id while retaining the resolved real task id in `metadata.taskId`. `thread_type=task_deeplink` keeps these rows out of general recent chats; explicit `taskId` searches still opt out of that exclusion for compatibility. Alembic `20261019_01` is schema-only and intentionally does **not** backfill legacy task threads. Frontend hook: `core/threads/use-task-deeplink-threads.ts`. Tests: `packages/harness/tests/test_task_deeplink_threads.py`, `packages/harness/tests/test_thread_meta_task_lookup.py`. |
| **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
| **Threads search** (`POST /threads/search`, LangGraph-compat) | Conversation listing. **System threads are excluded by default**: rows whose `metadata.system` is truthy (scheduler runs, roundtable workers) never crowd the user's list; a caller whose `metadata` filter explicitly references `thread_type`/`system` (e.g. studio notebooks) opts out of the exclusion. New `query` field = case-insensitive title (`display_name`) substring search, used by the chats page's server-side pagination. `POST /threads/count` takes the same request body (limit/offset ignored) and returns `{total}` under identical filtering/exclusion semantics ? powers the chats page's ??/??? display without fetching rows (`ThreadMetaStore.count`). `ThreadMetaStore.search` gained `exclude_system`/`query` params; JSON-blob filters batch-scan until the requested `limit`/`offset` window is full (so pagination over a heavily-filtered set never under-fills), with a `thread_id` tie-breaker on the `updated_at DESC` ordering for stable OFFSET paging. Tests: `tests/test_thread_meta_search.py` |
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`) | `GET /{path}` - serve artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `GET /api/artifact-library` scans every current-user normal thread's authoritative `outputs/` directory (including files omitted from `present_files`), excludes uploads and image/audio/video media, joins the owning conversation title/agent route metadata, and exposes search/filter/pagination with a bounded 15-second per-user cache plus `refresh=true` bypass. Frontend route `/page/workspace/library` links each item back to the source chat with `?artifact=<path>` so the chat opens the existing artifact panel directly. |
| **Threads search** (`POST /threads/search`, LangGraph-compat) | Conversation listing. **System threads are excluded by default**: rows whose `metadata.system` is truthy (scheduler runs, roundtable workers) or whose `thread_type` is a known system type (`task-analysis` seat runs included; older seat rows are also matched by `task_analysis_seat`) never crowd the user's list; a caller whose `metadata` filter explicitly references `thread_type`/`system` (e.g. studio notebooks) opts out of the exclusion. New `query` field = case-insensitive title (`display_name`) substring search, used by the chats page's server-side pagination. `POST /threads/count` takes the same request body (limit/offset ignored) and returns `{total}` under identical filtering/exclusion semantics ? powers the chats page's ??/??? display without fetching rows (`ThreadMetaStore.count`). `ThreadMetaStore.search` gained `exclude_system`/`query` params; JSON-blob filters batch-scan until the requested `limit`/`offset` window is full (so pagination over a heavily-filtered set never under-fills), with a `thread_id` tie-breaker on the `updated_at DESC` ordering for stable OFFSET paging. Tests: `tests/test_thread_meta_search.py` |
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`, `/api/strategy-project-artifacts/{id}/download`) | `GET /{path}` streams thread artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `GET /api/threads/{id}/artifact-preview?path=` returns bounded text, using streaming statistics plus deterministic head/middle/tail samples above the database threshold. `GET /api/artifact-library` combines every current-user normal thread's authoritative `outputs/` files with completed strategy-selection syntheses projected from the owner-scoped `parallel_agent_tasks.messages` snapshot. Strategy items use `source_type=strategy`, carry `strategy_project_id` / `strategy_name` / `strategy_stale`, and explicitly distinguish `strategy_source_kind=strategy_selection` from an `analysis_project_mode=strategy` thread (`analysis_project`); virtual items also expose `download_url`, so clients never infer routing from sentinel thread/path values. Search/source filters cover both. `workflowStale=true` keeps the last completed historical synthesis visible and downloadable. The dedicated strategy download route revalidates current-user ownership and snapshot completion and serves Markdown without pretending the virtual item is a thread artifact. Uploads and image/audio/video media remain excluded. Library metadata is persisted in `artifact_library_items`; list/search/filter/count/pagination read only this user-scoped SQL index. A generation-fenced `artifact_library_sync_states` lease schedules low-concurrency filesystem/task reconciliation in the background on first/stale reads or `refresh=true`, keeps the last successful snapshot visible during refresh/failure, and reports `sync_status` / `syncing` / `indexed_at` / `sync_error` to the client. Files themselves remain authoritative on disk and are never copied into the database. |
| **Word export** (`POST /api/writing/export/docx`) | Shared Markdown → DOCX generator used by conversation, artifact, scheduled-task, Deep Research, and AI-writing download surfaces. `app/gateway/word_export.py` builds OOXML with no Python conversion dependency: A4 portrait, 3.7/3.5/2.8/2.6 cm margins, real multilevel heading numbering (`一、` / `(一)` / `1.`), two-character first-line indent on heading levels 1–3, formal body/table styles, page numbers beginning at 1 with odd pages left-aligned and even pages right-aligned, and embedded `app/gateway/assets/fonts/方正小标宋简体.ttf`. Before numbering it removes complete manual heading prefixes (including `2.1` / `2.1.1`) and drops Markdown horizontal rules from Word output. The ordinary-Q&A Lead Agent can emit the matching Markdown heading forms and avoid unordered bullet lists when the admin enables `prompt_prefix.ordinary_qa_markdown_format_enabled` in `system_settings.json`; the switch defaults off, while writing setup, writing mode, notebook, scheduled runs, and SOUL-driven custom agents keep their independent prompts. The font loader validates its internal family, SHA-256, and OpenType embedding permission (`OS/2.fsType`) before export. Tests: `tests/test_word_export.py`, `tests/test_es_query_routing_prompt.py`. |
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. When `custom_outline` is empty (the workbench default — the user did not pick a report structure), chat collection first calls `POST /sessions/{id}/intent-outline` after the collector turn ends: the chosen model plans a Markdown skeleton from the topic plus collected materials, the outline is frozen into the session snapshot and shown in the「开始撰写」form for editing. If that step is skipped or fails, `basic`/`quick` still call `generate_intent_outline` at write time, emit `plan_created` (`source: intent`), then write through the same shared writer; generation failure is a recoverable `intent_outline_fallback` warning and does not abort the report. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
| **Gateway auth** | Owner-scoped Deep Research routes await the platform's asynchronous `get_current_user` resolver before they read or write persistence, ensuring each session and job operation receives a concrete user ID. Deep Research artifacts are always resolved from a whitelisted filename via `/mnt/user-data/outputs/<name>` before mapping to a host path, preserving the sandbox guard on native Windows and Linux hosts. |
| **Suggestions** (`/api/threads/{id}/suggestions`) | `POST /` - generate follow-up questions; rich list/block model content is normalized before JSON parsing |
| **Thread Runs** (`/api/threads/{id}/runs`) | `POST /` - create background run; `POST /stream` - create + SSE stream; `POST /wait` - create + block; `GET /` - list runs; `GET /{rid}` - run details; `POST /{rid}/cancel` - cancel; `GET /{rid}/join` - join SSE; `GET /{rid}/messages` - paginated messages `{data, has_more}`; `GET /{rid}/events` - full event stream; `GET /../messages` - thread messages with feedback; `GET /../token-usage` - aggregate tokens |
| **Feedback** (`/api/threads/{id}/runs/{rid}/feedback`) | `PUT /` - upsert feedback; `DELETE /` - delete user feedback; `POST /` - create feedback; `GET /` - list feedback; `GET /stats` - aggregate stats; `DELETE /{fid}` - delete specific |
| **Answer feedback** (`/api/threads/{id}/messages/{mid}/feedback`) | Message-scoped `PUT`/`DELETE`, `GET /feedback-context`, and batched `POST /api/threads/{id}/feedback-contexts`; persists excellent/poor ratings, six stable poor-reason codes and optional detail idempotently per user/thread/message/agent. Legacy run-scoped feedback routes remain compatible. Admin-only `GET /api/agents/{agent_id}/feedback-score` uses the project-wide `system_role == "admin"` rule (and preserves auth-disabled mode) and returns a Bayesian user-feedback answer-quality score, raw task-success rate, weighted response-quality dimensions and reason counts; it is explicitly not a complete agent-capability evaluation. |
| **Runs** (`/api/runs`) | `POST /stream` - stateless run + SSE; `POST /wait` - stateless run + block; `GET /{rid}/messages` - paginated messages by run_id `{data, has_more}` (cursor: `after_seq`/`before_seq`); `GET /{rid}/feedback` - list feedback by run_id |
| **Open Chat** (`/api/open/chat`) | **Unauthenticated** open Q&A entry for external systems (the `/api/open/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES` + CSRF-exempt in `csrf_middleware.should_check_csrf`). `POST /chat` `{message \| messages[], model_name?, agent_name?, agent_id?, thinking_enabled=false, strip_think=true, thread_id?, recursion_limit=100}` → resolves the agent by **Chinese display name** (`agent_name`, matched against each `.deer-flow/agents/*/config.yaml` `name` via `_resolve_agent_id`; explicit `agent_id` wins if given), builds the lead agent via `make_lead_agent` (so the chosen **agent's** SOUL/tools/skills all apply), `ainvoke`s it once (stateless — no checkpointer, no thread persistence; runs under the `"default"` user bucket), and returns **only the final** `{answer, model_name, agent_name, agent_id, thinking_enabled, thread_id}` (no intermediate tool-call/process). **`POST /chat/stream`** is the **SSE streaming variant** (same request body) for **long tasks** — the blocking `/chat` produces no downstream bytes for the whole run, so a front nginx `proxy_read_timeout` (default 60s) cuts it off with a **504**; the stream keeps the connection warm (a `: keepalive` SSE comment every `_HEARTBEAT_SECONDS=15`s via a background producer-task + queue, plus `event: delta` AI-text increments and `event: progress`), and ends with **`event: result`** carrying the same final `{answer, …}` JSON (or `event: error`). Clients only need to read `event: result`; intermediate events are ignorable. Both endpoints share `_build_run_context` (config/agent/thread). `GET /agents` lists `{name, id}` so callers know what `agent_name` to send. **Thinking off is reliable**: when `thinking_enabled=false` the route also sets `thinking_force_disabled=true` so the lead agent calls `create_chat_model(force_disable_thinking=True)`, which injects **three** OpenAI-compatible "thinking off" shapes into `extra_body` — top-level `enable_thinking=false` (**SiliconFlow / DashScope hybrid models** like `deepseek-ai/DeepSeek-V4-*` on `api.siliconflow.cn` — the knob the nested `chat_template_kwargs` shape does NOT reach), `chat_template_kwargs.enable_thinking=false` (vLLM/Qwen native), and `thinking.type="disabled"` (Zhipu GLM) — even for models that only declare `supports_thinking: true`. For the **web-UI path** (which sends `thinking_enabled=false` but NOT `force_disable`), SiliconFlow models need `when_thinking_disabled: {extra_body: {enable_thinking: false}}` in their `config.yaml` entry (added to `deepseek-ai/DeepSeek-V4-Flash`) for thinking-off to take effect. Sets `is_scheduled_run=true` internally to skip the auto-title LLM call. **Generated-file return (`include_files=true`, default on)**: an agent that produces its report as a file (writes to `/mnt/user-data/outputs/` + `present_files`) returns only a short summary in `answer`; to recover the real deliverable both endpoints now also return `files: [{name, virtual_path, encoding(text|base64), content}]` — `_collect_output_files` scans the run's `state['thread_data']['outputs_path']` (the open thread is stateless so artifacts can't be fetched by thread, but the endpoint runs server-side and the sandbox already wrote the files to disk), text decoded as utf-8 else base64, capped by `_MAX_OUTPUT_FILES=20` / `_MAX_OUTPUT_FILE_BYTES=5MB`. `/chat` puts `files` on the response; `/chat/stream` puts it in the `event: result` payload. A run with no answer **and** no files now 502s ("…文本答案或文件"). Router `app/gateway/routers/open_chat.py`; standalone caller `test_open_chat_api.py` (repo root, streaming by default); tests `tests/test_open_chat.py`. |
| **Open 3qfx Ask** (`/api/open/3qfx/ask`) | **Unauthenticated** open entry that, given just a **任务id**, directly kicks off the 3qfx(三情分析)「第一次问答」as a **real persisted conversation** (sibling of `/api/open/chat`; same `/api/open/` auth-whitelist + CSRF-exempt). `POST /ask` `{task_id, message?, model_name?, thinking_enabled=false}` → (1) resolves the `3Q` agent from `app.state.business_mapping_store.get_mapping("3Q")` (the **same** agent the frontend deep-link `goPath=3qfx` uses; 503 if unconfigured); (2) **fetches the task detail server-side** (consumer `cop-task-detail`) and builds the opening text **identical to the frontend** `buildOpeningText` 3qfx branch — `任务名称:…\n任务内容:…\n任务id(task_id)为{task_id}` (an explicit `message` overrides and **skips** the fetch); (3) creates a fresh thread tagged `metadata.{taskId, agent_id}` (the exact filter `useTaskThreads` uses, and `metadata.taskId` makes it task-deeplink-shared / creator-path-resolved) and **starts a background persisted run** via `start_run(..., enforce_access=False)` (writes the checkpointer, runs under the `"default"` bucket); (4) **returns immediately** `{thread_id, run_id, agent_id, status, opening_text}` — the caller just needs "started", then watches the run in the 3qfx 任务工作区 conversation list. `start_run` gained a keyword-only `enforce_access` (default `True`); the open entry passes `False` so the admin-configured agent isn't rejected by the per-(missing)-login visibility check. **Consumer fetch config** lives in `config.yaml → task_deeplink`: `cop_task_detail_url` (full URL, appends `?taskId=`), `cop_task_detail_authorization` (default `Bearer admin`), `cop_task_detail_test` + `mock.cop_task_detail` (offline fake when the upstream is unreachable — on in this deployment). Router `app/gateway/routers/open_3qfx.py`; tests `tests/test_open_3qfx.py`. |
@@ -416,15 +488,15 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
| **Scheduled Tasks** (`/api/scheduled-tasks`) | Per-user cron-like agent tasks. `GET/POST /` - list/create owned tasks; `PATCH/DELETE /{id}` - update/delete (owner); `POST /{id}/{pause,resume,trigger}`; `GET /{id}/runs`, `GET /runs/{rid}`, run-file preview/edit/download; `GET /runs/{rid}/messages` - agent messages for the run's underlying agent run (used to inspect a stuck/running run; authorized via `_load_run`); `GET /runs/{rid}/html` - generated HTML document for an `html_page` run; `DELETE /runs/{rid}` - owner-scoped, with an admin fallback so admins can delete any user's run; publish/subscribe under `/{id}/{publish,unpublish,subscribe}` and `GET /published`. **Admin (cross-user):** `GET /admin/all` - every task; `PATCH/DELETE /admin/{id}` - edit/delete any task; `POST /admin/{id}/trigger`; `GET /admin/{id}/runs`. Admin routes require `system_role == "admin"`. |
| **HTML Page Favorites** (`/api/html-page-favorites`) | Bookmarked generated HTML pages, owner-scoped. `GET /` - list; `POST /` `{page_id, title?, description?, tags?}` - bookmark a `scheduled_html_pages` row (extracts a reusable style/layout summary via GLM5 at favorite time); a page can only be favorited once per user (`DuplicateFavoriteError` ? HTTP 409). `GET/DELETE /{id}` - read/delete (the create-task reference picker deletes favorites here). A favorite can be selected as a style reference when creating a new `html_page` task. |
| **Public Scheduled Tasks** (`/api/public/scheduled-tasks`) | **Unauthenticated** mirror for embedding. Adds `GET /runs/{rid}/html` and `GET /{task_id}/latest-html` for the `html_page` preview; the run result keeps `result.html` (only file paths are redacted). |
| **Light Apps** (`/api/light-apps`) | Light-app registry (????? / ????), **global/shared** (no user scoping; `created_by`/`updated_by` audit-only). Two kinds: `window_open` (launched in a new tab from the ???? card grid) and `iframe` (embedded under a parent sidebar menu via `mount_location` and dynamically injected into the sidebar). Both carry `route_params` ? a list of `{key, paramType: login\|theme, source, customValue, themeValue}` descriptors the **frontend** resolves at launch time (login params read from `localStorage.userInfo`; theme params use a literal value) and appends to `url` as query string. `GET /` - list (filters `app_type`/`mount_location`/`enabled_only`/`search`); `GET /{app_id}` - read one (both open to any authenticated user); `POST /`, `PUT /{app_id}`, `DELETE /{app_id}` - **admin only**; iframe apps must carry a `mount_location` (400 otherwise); the unique `name` collides as HTTP 409. Backed by `deerflow.persistence.light_apps` (table `light_apps`, auto-created by `create_all`); store wired in `deps.py` as `app.state.light_app_store`. Tests: `tests/test_light_apps.py`. |
| **Menu Overrides** (`/api/menu-overrides`) | Sidebar-menu overrides (菜单管理), **global/shared** (no user scoping; `updated_by` audit-only). The static menu tree (icons/routes/`adminOnly`) lives in the **frontend** code (`core/page-layout/sidebar-menu.ts`); iframe light-apps are injected at render time. This router persists a per-node **overlay** the frontend merges onto that tree: `parent_id` (re-parent, moves the whole subtree; NULL = top level), `sort_order` (reorder among siblings), `label` (rename; NULL keeps code label), `disabled` (停用 — hides the node **and its subtree**). A node with no row renders exactly as code defines it, so new code menus / newly-registered iframe apps appear automatically and are immediately configurable. `GET /` - list all overrides (open to any authenticated user); `PUT /` - **admin only**, atomically **replaces the whole set** (`{overrides: [...]}`); rejects re-parenting that would exceed **3 levels** or form a cycle (`_validate_depth`, HTTP 400). Backed by `deerflow.persistence.menu_overrides` (table `menu_overrides`, auto-created by `create_all`); store wired in `deps.py` as `app.state.menu_override_store`. Frontend: merge logic `core/page-layout/menu-overrides.ts` (`applyMenuOverrides` in `use-sidebar-menu.ts`), client `strategy-components/api/menu-overrides.ts`, admin page `pages/MenuManagementPage.tsx` (@dnd-kit drag-reorder/re-parent + 「移动到」parent selector + ↑↓ arrows + 改名/停用), entry in the bottom-left settings dropdown (`components/page-sidebar.tsx`, admin-only). Tests: `tests/test_menu_overrides.py`, `frontend-web/src/core/page-layout/menu-overrides.test.ts`. |
| **Compact Menu Overrides** (`/api/compact-menu-overrides`) | Compact/sidebar-summary left-nav overlay (简介模式菜单), **global/shared**. Two independent sets keyed by `menu_type` (`A` default / `B`); saving one type does not wipe the other. Type B rows use a `B::` node_id prefix so they stay unique on databases that still use `node_id` as the sole PK. `GET ?menu_type=` lists one type (open); `PUT {menu_type, overrides}` admin-only replaces that type. Frontend: `MenuManagementPage` 简介模式 type switcher + `runtime-config.js` `VITE_COMPACT_MENU_TYPE`. Tests: `tests/test_compact_menu_overrides.py`. |
| **Light Apps** (`/api/light-apps`) | Light-app registry (应用管理 / 应用列表), **global/shared** (no user scoping; `created_by`/`updated_by` audit-only). Two kinds: `window_open` (launched in a new tab from the 应用列表 card grid) and `iframe` (embedded under a parent sidebar menu via `mount_location` and dynamically injected into the sidebar). Both carry `route_params` — a list of `{key, paramType: login|theme, source, customValue, themeValue}` descriptors the **frontend** resolves at launch time (login params read from `localStorage.userInfo`; theme params use a literal value) and appends to `url` as query string. `GET /` - list (filters `app_type`/`mount_location`/`enabled_only`/`search` / repeatable `tag_ids` OR-match; each item carries `tags`); `GET /{app_id}` - read one (both open to any authenticated user); `POST /`, `PUT /{app_id}`, `DELETE /{app_id}` - **admin only**; iframe apps must carry a `mount_location` (400 otherwise); the unique `name` collides as HTTP 409. Tags reuse `/api/tags` with `tag_type=light_app` (not a 岗位 subscription type). Backed by `deerflow.persistence.light_apps` (table `light_apps`, auto-created by `create_all`); store wired in `deps.py` as `app.state.light_app_store`. Tests: `tests/test_light_apps.py`, `tests/test_light_apps_tags.py`. |
| **Menu Overrides** (`/api/menu-overrides`) | Sidebar-menu overrides (菜单管理). `menu_overrides` is the administrator-maintained shared **default**; `user_menu_override_configs` stores one complete personal snapshot per user. `GET /` returns `overrides` (effective), `default_overrides`, `user_overrides`, `has_user_override`, and `editable_scope`. `PUT /` is scope-aware: administrators replace the sparse default overlay, while ordinary users replace only their own complete snapshot; an empty personal payload (or `DELETE /`) restores defaults. Personal rows win for known `node_id`s and default rows absent from the snapshot flow through, so newly-added menus follow administrator configuration without overwriting existing personal choices. The static tree (icons/routes/`adminOnly`) remains frontend-owned; iframe light-app nodes are supported; re-parenting is validated against cycles and the three-level cap. Both tables are auto-created by `create_all`. Frontend: `core/page-layout/menu-overrides.ts`, `use-sidebar-menu.ts`, `strategy-components/api/menu-overrides.ts`, and `pages/MenuManagementPage.tsx`; the settings entry is available to every user and labels the administrator view as default management. Tests: `tests/test_menu_overrides.py`, `frontend-web/src/core/page-layout/menu-overrides.test.ts`. |
| **Compact Menu Overrides** (`/api/compact-menu-overrides`) | Compact/sidebar-summary menu overrides (简介模式菜单). `compact_menu_overrides` remains the administrator default for independent type `A` / `B` layouts; `user_compact_menu_override_configs` stores a complete snapshot per `(user_id, menu_type)`. Scope-aware `GET`/`PUT`/`DELETE` use the same effective/default/personal contract as the full menu: existing snapshot nodes stay personal and unseen new default nodes flow through. Type B default rows retain the historical `B::` storage prefix; public node ids never contain it. Frontend editing is fixed to the deployment's `runtime-config.js` `VITE_COMPACT_MENU_TYPE`. Tests: `tests/test_compact_menu_overrides.py`, `frontend-web/src/core/page-layout/compact-menu-overrides.test.ts`. |
| **Sensitive Words** (`/api/sensitive-words`) | Sensitive-word / desensitize map (敏感词管理 / 脱敏词), **global/shared** (no user scoping; `updated_by` audit-only). Each row maps an original `word` → `replacement`, with `enabled` (停用 parks a word without deleting) + `sort_order`. The **frontend** ships a seed map in code (`src/lib/desensitize.ts` `SEED_DESENSITIZE_MAP`) used as the default on a fresh deployment; once an admin saves, this table is the source of truth and the frontend loads it at startup (`DesensitizeWordsLoader` → `replaceDesensitizeMap`, **enabled** rows only). `GET /` - list all words (open to any authenticated user — every page desensitizes); `PUT /` - **admin OR the `lqq` account** (`_require_manager`: `system_role == "admin"` or email `@`-prefix `== "lqq"`), atomically **replaces the whole set** (`{words: [...]}`, de-duped by `word`, last wins). Backed by `deerflow.persistence.sensitive_words` (table `sensitive_words`, PK `word` `String(191)`, auto-created by `create_all`); store wired in `deps.py` as `app.state.sensitive_word_store`. Frontend: client `strategy-components/api/sensitive-words.ts`, page `pages/SensitiveWordsPage.tsx` (route `/page/strategy/admin/sensitive-words`, entry in the bottom-left settings dropdown — **gated on username `lqq`, not admin role**, present in both sidebar modes; 「导入种子词」 merges missing seed entries). Tests: `tests/test_sensitive_words.py`. |
| **Sentiment Agent Proxy** (`/api/sentiment-agent`) | BFF for the frontend 舆情分析 virtual agent. `POST /stream` accepts the AG-UI body plus `apiUrl`/`authUsername`/`authPassword` from `runtime-config.js`, forwards to that URL with TLS verify off, and byte-pipes upstream SSE. Connection/HTTP failures return the full error in `detail`; mid-stream failures emit AG-UI `RUN_ERROR`. Distinct from `skill-packages/sentiment-analysis.skill`. Tests: `tests/test_sentiment_agent_proxy.py`. |
| **Dashboard Sessions** (`/api/dashboard-sessions`) | 「大屏绘制智能体」独立页面会话,**按外部 `task_id` 存取,不按 user 分权**(open read + shared writes;`user_id` audit-only)。一个 task 一行(upsert):`transcript`(聊天记录 JSON)+ `report_json`(最新结构化数据,可解析 = 「有效结果」)。`GET /by-task/{task_id}` - 取该 task 会话(无则 404);`PUT /by-task/{task_id}` - upsert(`{transcript?, report_json?}`,只覆盖传入项)。Backed by `deerflow.persistence.dashboard_sessions`(table `dashboard_sessions`, auto-created by `create_all`,无迁移);store wired in `deps.py` as `app.state.dashboard_session_store`。前端独立页 `roundtable-planning/pages/DashboardAgentPage.tsx`(路由 `/page/strategy/dashboard-agent?taskId=`,侧边栏「问答管理 → 大屏绘制」)让智能体分析任务报告 JSON → report-json → 渲染 `DashboardApp`。Tests: `tests/test_dashboard_sessions.py`。 |
| **Task Reports** (`/api/task-reports`) | 任务级研判情报报告详情,**按 `task_id` 共享、不按 user 分权**(写入者只留 `updated_by` 审计)。`GET /by-task/{task_id}` 始终返回原 Consumer 兼容外壳 `{state:"200", msg:"操作成功!", data:[...]}`(未保存为 `data: []`);`POST /by-task/{task_id}` 首次保存,已有报告返回 409;`PUT /by-task/{task_id}` 原子替换完整 `data` 数组。`POST /import/by-task/{task_id}` 面向 TaskCOP 导入页,接收 `sq-report-mock.json` 兼容的 `data`/`records` 数组并完整覆盖目标任务;路径中的选定任务 id 优先于文件内的 `taskId`,避免误写。每一项严格是 `{id, taskId, createTime, updateTime, sessionId, contentJson, categoryType}`;`contentJson` 以 JSON **字符串**原样落在 `records_json`,避免破坏大屏按类目解析的输入格式。保存或导入非空 `data` 后,router 会通过 `app.state.taskcop_task_store.mark_analysis_completed` 将同 id 的 TaskCOP 任务升级至状态 `25`,使旧任务列表显示并筛选为“已完成”;普通保存时不存在匹配任务的报告仍可独立保存,导入则要求目标任务存在。Backed by `deerflow.persistence.task_reports`(table `task_reports`, auto-created by `create_all`);store wired in `deps.py` as `app.state.task_report_store`。前端客户端 `strategy-components/api/task-reports.ts`,`core/auth/dashboard-input.ts` 与 rwfx 3q 完成检查均直接读此接口,不再依赖 Consumer 或本地 sq-report mock。Tests: `tests/test_task_reports.py`。 |
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business/purpose/host), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, append_auth, auth_token_param, auth_name_param, enabled, sort_order}`. **Two independent, coexisting login-append mechanisms for `url` buttons** — both resolved frontend-side: (1) `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` resolved at click time (`source` = a `userInfo` field or `other`→literal `custom_value`) and appended to the target URL — mirrors the light-app login params; empty = nothing appended. (2) `append_auth` + `auth_token_param` + `auth_name_param` (columns, migration `20260625_02`, default off) — appends `?{auth_token_param}={token1}&{auth_name_param}={username}` so the target system can identify the current user; gated on **token login** — the frontend only stashes `token1` (the `?authToken=`) + the raw `getTokenInfo` username (`login.token1`/`login.username`) when login went through `/login/token`, so a non-token-login session appends neither. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
| **Fixed Questions** (`/api/fixed-questions`) | Administrator-managed exact-match fixed Q&A (问题配置), **global/shared**. Each row is `{id, question, answer, enabled, tokens_per_second, sort_order}`; stream speed is configurable per row from `1–1000 token/s` and defaults to `100`. `GET /` and whole-set atomic `PUT /` are both admin-only. At the start of an external chat run, `services._find_fixed_answer` checks the latest eligible plain-text human message with exact, case/space/punctuation-sensitive equality. Enabled matches switch the run to `make_fixed_answer_agent`: a local `BaseChatModel` tokenizes with `cl100k_base`, replays the answer in rate-limited token chunks, and keeps the existing LangGraph SSE `messages`/`values` protocol, run journal, checkpoint history, deterministic local title, and refresh behavior intact while no provider/model request or model-concurrency slot is used. File/multimodal/context-prefixed turns, `canvas_agent`, `ai_writing`, writing mode, resume commands, and internal orchestration/background child runs bypass matching. Storage failures fail open to the normal model path. Backed by `deerflow.persistence.fixed_questions` (table `fixed_questions`, SHA-256 indexed lookup plus exact original-string collision check, migrations `20260821_01` + `20260821_02`), wired as `app.state.fixed_question_store`. Frontend: `strategy-components/api/fixed-questions.ts`, admin page `pages/FixedQuestionsPage.tsx`, route `/page/strategy/admin/fixed-questions`, entries in both full-layout and compact-workspace settings menus. Tests: `tests/test_fixed_questions.py`. |
| **Fixed Questions** (`/api/fixed-questions`) | Administrator-managed exact-match fixed Q&A (问题配置), **global/shared**. Each row is `{id, question, answer, enabled, tokens_per_second, sort_order}`; stream speed is configurable per row from `1–1000 token/s` and defaults to `100`. `GET /` and whole-set atomic `PUT /` are both admin-only. At the start of an external chat run, `services._find_fixed_answer` checks the latest eligible plain-text human message with exact, case/space/punctuation-sensitive equality. Enabled matches switch the run to `make_fixed_answer_agent`: a local `BaseChatModel` uses a lossless standard-library-only word/CJK/punctuation splitter for rate-limited chunks. It must never initialize a model tokenizer such as `tiktoken.get_encoding`, because an empty tokenizer cache can trigger a public vocabulary download and break offline deployments. The local graph keeps the existing LangGraph SSE `messages`/`values` protocol, run journal, checkpoint history, deterministic local title, and refresh behavior intact while no provider/model request or model-concurrency slot is used. File/multimodal/context-prefixed turns, `canvas_agent`, `ai_writing`, writing mode, resume commands, and internal orchestration/background child runs bypass matching. Storage failures fail open to the normal model path. Backed by `deerflow.persistence.fixed_questions` (table `fixed_questions`, SHA-256 indexed lookup plus exact original-string collision check, migrations `20260821_01` + `20260821_02`), wired as `app.state.fixed_question_store`. Frontend: `strategy-components/api/fixed-questions.ts`, admin page `pages/FixedQuestionsPage.tsx`, route `/page/strategy/admin/fixed-questions`, entries in both full-layout and compact-workspace settings menus. Tests: `tests/test_fixed_questions.py`. |
| **Knowledge** (`/api/knowledge`) | Product knowledge base, **global/shared** (no user isolation; writes record `created_by`/`updated_by` for audit). `POST /ingest/thread/{thread_id}` - sediment one thread into a note (reads messages from the checkpointer, `threads:read` owner-checked); `POST /ingest/threads/batch` - batch-sediment the caller's recent threads (`skip_existing` via `.manifest.json` dedup). **Defaults to `background=true`**: the slow per-thread extraction loop is deferred to a FastAPI background task and the endpoint returns `{queued:true, processed:N}` immediately (each thread's transcript is truncated to `max_input_chars` before the LLM call). Running the loop inline for a large batch outlives the nginx gateway timeout (504), which the frontend's `parseResponse` surfaces as a generic "Request failed"; `background=false` keeps the blocking path (full per-thread `created/skipped/failed` tally) for small batches/scripts. Shared loop: `_run_batch_ingest`. **Optional model selection**: both ingest endpoints accept `model_name` (overrides `knowledge.extract_model_name` for that capture; threaded through `KnowledgeService.capture_thread` ? `extract_thread_llm_multi`). **Model-error surfacing**: when the chosen model produces nothing usable (timeout / API error / non-JSON) capture still **falls back to rule-based extraction and lands a draft** (`llm_fallback:true`); the single-thread (non-background) response and the batch response (`llm_fallback` count) carry this so the UI tells the user "???????". The "??????" chat button stays background/fire-and-forget (passes `model_name`, no per-save error toast); the batch-import dialog is synchronous and shows the fallback count. `POST /ingest/file` - **import an uploaded file** (multipart: `file` + optional `title`/`tags`/`status`/`model_name`) into wiki note(s): the file is converted to Markdown (md/txt read directly, pdf/word/ppt/excel via `convert_file_to_markdown`), distilled by the same wiki-capture LLM pipeline as thread capture (`source_type="file"`), and falls back to a verbatim **draft** note when the model returns nothing usable (`llm_fallback:true`); `POST /ingest/search` - sediment search/tool results as a reference note; `POST /notes` - manual note; `GET /notes` (keyword/tag/source_type/status + pagination); `GET/PUT/DELETE /notes/{id}` (DELETE is soft = archive, `?hard=true` removes the vault file); `POST /search` - `mode=keyword\|vector\|hybrid` (vector?keyword fallback when no embedding endpoint); `GET /notes/{id}/versions` - edit history; `GET /duplicates` + `POST /merge` - dedup; `POST /reindex` - rebuild vector index; `GET /graph` - product graph data (notes + entities, graph-blacklist filtered); `GET/POST /graph/blacklist` + `DELETE /graph/blacklist/{id}` - graph node blacklist (labels = entity names / note titles, case-insensitive match; blacklisted nodes and their edges are dropped from `/graph` and the wiki-export artifacts; table `knowledge_graph_blacklist`, auto-created); `POST /export/graph` + `GET /export/files/{graph.json\|graph.html\|graph.graphml\|cypher.txt}` - obsidian-wiki `wiki-export` artifacts (self-contained `graph.html` for iframe); `POST /resolve` `{targets:[?]}` + `GET /notes/{id}/wikilinks` - resolve `[[wikilink]]` targets to a note id / vault page / missing (powers clickable wikilinks); `GET /vault-page?path=` - read a raw vault meta/hub Markdown page (no DB note). `GET/POST /folders` + `PUT/DELETE /folders/{id}` - user-managed directories (notes carry a `folder` path; `GET /notes?folder=` filters a dir + sub-dirs); `GET/POST /extract-templates` + `GET/PUT/DELETE /extract-templates/{id}` - editable extraction-prompt templates (thread/document ingest accept `template_id`; capture/PUT-note accept `folder`). Backed by the `deerflow.knowledge` module (see below). |
Proxied through nginx: `/api/langgraph/*` ? LangGraph, all other `/api/*` ? Gateway.
@@ -540,6 +612,8 @@ Self-contained product knowledge base. **All knowledge capability lives in this
Custom agent metadata lives in the `agents` ORM table (`deerflow.persistence.agents`). `agents.id` is the stable runtime id and the filesystem directory name under `.deer-flow/agents/{id}/`; `agents.name` is display-only, can contain Chinese characters, and is not unique. `user_id IS NULL` marks built-in agents, user-owned agents are visible only to their owner unless `published = true`, and update/delete operations require ownership. The runtime accepts `agent_id` in context/configurable, and non-default `assistant_id` values are treated as agent ids after visibility checks.
Strategy-selection fan-out (`routers/parallel_agents.py`) must keep hidden reasoning separate per agent: provider reasoning blocks/fields and inline `<think>...</think>` text emit `reasoning_delta`, while visible answer text emits `delta` and only visible text may enter `agent_done.content`. Never concatenate `reasoning_delta` into a selectable strategy or generated deliverable. Inspect the final AI message with the shared `is_llm_error_message` detector and require substantive visible text before constructing a success result. A middleware fallback, empty/thinking-only response, or punctuation-only provider degeneration is a failed seat run: raise so the stream emits `agent_error`; never wrap it in `agent_done`. Regression coverage: `tests/test_roundtable_model_fallback.py`.
**Listing cache + pagination** ? to keep `GET /api/agents` fast, the gallery's per-card extras (skills + tool_groups) are denormalized into two cache tables instead of reading every agent's `config.yaml` on each request: `agent_skills` (relational, one row per (agent, skill) with `position`) and `agent_extras` (one row per agent: `tool_groups` JSON + `model` + `skills_synced_at`). `config.yaml` remains authoritative for the agent runtime; these tables are a read cache, write-through on create/update and backfilled from disk by `_sync_legacy_agents` (mtime-vs-`skills_synced_at` drift check). The list endpoint enriches via `AgentStore.get_extras_for([ids])` (two batched `IN` queries). Both tables are auto-created by `Base.metadata.create_all` (no migration needed). `GET /api/agents` supports **opt-in server-side pagination**: `scope` (`mine` = owned + favorited / `square` = built-in/published public), `page`, `page_size`, returning `{agents, total, page, page_size}`; omitting `scope`/`page` preserves the legacy "return everything" behavior. Name `search` matches the raw OR desensitized name (the frontend's static term?abbrev map is mirrored server-side in `deerflow/config/agent_name_desensitize.py`); `tag_ids` is an SQL OR-filter; `owner` keeps only agents published by that user id (the card's ??? ? built-in agents never match), powering the gallery's click-???-to-filter. `POST /api/agents/private-square/list` takes the same pagination params (incl. `owner`). Repository entry point: `AgentStore.list_paginated(...)`. Tests: `tests/test_agent_listing_cache.py::test_list_paginated_owner_filter_keeps_one_publisher`.
**Homepage selector (agent picker on the new-chat page)** ? single admin-curated list backed by `agents.featured_order` (nullable BIGINT). NULL = not featured. Only `published = true` agents can be featured (router enforces on writes); any published agent is eligible regardless of ownership. Cleared with `order: null`. The list rendered to end users is `WHERE featured_order IS NOT NULL` + visibility-filter, sorted by `featured_order ASC`. Admins reorder by sending the desired ids to `PUT /api/agents/featured/order` (assigns indices 0..N-1). BIGINT is used rather than INT so the management UI can seed initial values from `Date.now()` (overflows INT on MySQL).
@@ -559,18 +633,26 @@ frontend can hide or show the entry consistently; `PUT` is admin-only and
partially updates `reference_mode_enabled` and/or `cleanup_words`. The normal
workspace input box, iframe chat toolbar, send context, right reference panel,
and appearance settings all use this flag, while already locked citation
threads and selected LLMWiki knowledge bases still render their source
documents in the reference panel. Tests:
threads still render the generic reference panel. A non-empty selected
LLMWiki/assistant knowledge-space set instead carries an explicit Wiki reference
contract: the frontend filters every batch to those selected server-authored ids
and renders a flat Wiki table of contents plus inline Wiki reader, never the
generic reference-card component. Projects enable this only when their project
or bound-agent knowledge ids are actually non-empty. Tests:
`tests/test_prompt_prefix.py` and
`frontend-web/src/pages/iframe-chat-mode-buttons.test.ts`.
### Agent Module Tag Config (zzJC / zzXD 按类型聚合)
Backs the zzJC / zzXD agent-chat left sidebar (「按类型聚合智能体 + 聊天记录」). An admin picks which agent **tags** belong to a module; the module's left sidebar then shows only agents under those tags (empty/unconfigured = show all visible agents). Stored in `system_settings.json` → `agent_modules` (`AgentModuleSettings`: `modules: {module_key → [tag_id, ...]}`); pure helpers `get_module_tag_ids` / `set_module_tag_ids` (de-dupe + strip). Endpoints (in `app/gateway/routers/system_settings.py`): `GET /api/system-settings/agent-modules/{key}` (readable by any user — the sidebar needs it to filter), `PUT /api/system-settings/agent-modules/{key}` (admin only, full replacement of the tag-id list). Module keys are frontend-defined (`AGENT_MODULES` in `components/workspace/agents/agent-module-sidebar.tsx`, keyed by the module's landing agent id: `6bf`→`zzjc`, `10d69…`→`zzxd`). Tests: `tests/test_agent_modules.py`. Frontend: `core/system-settings` (`useAgentModuleConfig`/`useUpdateAgentModuleConfig`), admin `AgentModuleSettingsDialog` (gear button in the agent-chat header).
Backs the zzJC / zzXD agent-chat left sidebar (「按类型聚合智能体 + 聊天记录」). An admin picks which agent **tags** belong to a module. The module's sidebar then shows only agents under those tags (empty/unconfigured = show all visible agents). Stored in `system_settings.json` → `agent_modules` (`AgentModuleSettings`: `modules: {module_key → [tag_id, ...]}`); pure helpers `get_module_tag_ids` / `set_module_tag_ids` (de-dupe + strip). Endpoints (in `app/gateway/routers/system_settings.py`): `GET /api/system-settings/agent-modules/{key}` (readable by any user — the sidebar needs it to filter), `PUT /api/system-settings/agent-modules/{key}` (full replacement; admin only). Module keys are frontend-defined (`AGENT_MODULES` in `components/workspace/agents/agent-module-sidebar.tsx`). Tests: `tests/test_agent_modules.py`. Frontend: `core/system-settings` (`useAgentModuleConfig`/`useUpdateAgentModuleConfig`) and `AgentModuleSettingsDialog`.
### wx common-agent category
The fixed-deep-blue wx full-chat route (`#/page/workspace/wx/chats/new`) hides the global header and page sidebar while retaining its workspace common-agent panel. When its optional category is unset, the panel preserves the original recommendation logic (pinned → mine → square, deduplicated). A saved category selects one Agent Gallery category: `mine`, `sentiment`, or `square:<square_id>` (for example the configured `默认广场`). It is stored in `system_settings.json` → `wx_common_agents` (`WxCommonAgentsSettings`), validated against the effective publish-square list, and exposed by `GET`/`PUT /api/system-settings/wx-common-agents`. The `wx` account and administrators can update or clear it; all users can read it. The frontend uses `useWxCommonAgents` and displays the settings gear only to wx. Tests: `tests/test_wx_common_agents.py`.
### Business-mapping authorization
`/api/business-mapping` remains globally readable. Updates to 3Q, 6BF, and 7BF require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.
`/api/business-mapping` remains globally readable. Seeded rows include the main codes (3Q / 6BF / 7BF / 8BF / AFX) plus independent slots `3q-0…3q-6`, `6-1…6-4`, and `7-1…7-4` (empty agent/chain until configured on the 配置管理 page). Updates to 3Q, 6BF, 7BF, and those independent slots require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.
### Position-collaboration foundation
@@ -738,7 +820,7 @@ Two `TypeDecorator`s keep ORM columns correct across the `sqlite` / `postgres` /
### Schema Migrations / Manual Column-Add SQL (`scripts/sql/`)
Gateway startup runs Alembic through `app.gateway.app._run_db_migrations` after ORM table creation. The migration graph must always have unique revision ids and exactly one head; `tests/test_migration_graph.py` enforces both. Historical duplicate revision `20260821_01` is repaired by the `20260821_03` / `20260821_04` fixed-question chain and merge revision `20260825_01`. Startup DDL has a bounded timeout and failures are logged without preventing the app from serving.
Gateway startup runs Alembic through `app.gateway.app._run_db_migrations` after ORM table creation. The migration graph must always have unique revision ids and exactly one head; `tests/test_migration_graph.py` enforces both. Historical duplicate revision `20260821_01` is repaired by the `20260821_03` / `20260821_04` fixed-question chain and merge revision `20260825_01`. The duplicate Wiki revisions `20260916_01` / `20260920_01` are normalized by merge revision `20260916_03` and final reconciliation revision `20260920_02`; the latter must inspect the actual schema and idempotently ensure Wiki edges plus retrieval/incremental-sync columns so old databases stamped by either duplicate variant converge safely. Startup DDL has a bounded timeout and failures are logged without preventing the app from serving.
**Convention (required):** whenever code adds a field to an existing table (a new `mapped_column` on an ORM model), you must, in the same change:
1. Keep the matching alembic version file under `persistence/migrations/versions/` (guarded with `_has_table`/`_has_column`), and
@@ -755,6 +837,66 @@ MySQL index script synchronized.
### Sandbox System (`packages/harness/deerflow/sandbox/`)
**Runtime performance:** `AppConfig.runtime_performance` owns four independent
flags. `skill_view_reuse_enabled` and `skill_view_cache_enabled` default to true
when the keys are absent; set either to false and restart to restore a full
skill-view refresh on every tool call. `model_timing_nonblocking_enabled` and
`asyncio_thread_pool_enabled` stay false unless set. If application configuration
cannot be loaded, every switch stays off. `sandbox/skill_view_scope.py` wraps synchronous sandbox tools with a
per-invocation `ContextVar` snapshot, reset in `finally`. Reuse covers provider
initialization, path authorization from the discovered `enabled_roots`, command
translation and output masking; failed refreshes must not be retried by error
masking. A new tool call must validate again, including after skill offlining.
The optional `skill_view.py` cache validates fresh extensions bytes plus known
canonical/view file and directory metadata without the refresh lock on warm hits
of `refresh_skill_sandbox_view` itself. Sandbox tools do not use that full check
on the hot path: a bash command that does not name `/mnt/skills` does not stat
the catalogue; a command or read that names a skill reconciles only that skill
and any support library it declares. Skill-management file saves, deletions,
online/offline changes, skill deletion, and skill-address apply/rollback sync
that skill into the view immediately and publish the new extensions hash, so the
next command stays on the narrow path. A missing view or an extensions hash the
sync did not publish still falls back to the full rebuild. Direct edits outside
those writes stay stale until the next full rebuild. Offline skills are removed
from the view at disable time, which is what keeps a hidden shell path from
running them. Diagnostic samples are process-local and are appended only
after the sandbox tool call returns: each row stores the command text and the
skill names invoked by that call. The admin page is `/page/workspace/admin/skill-timing`
(`GET /api/admin/skill-view-diagnostics`). Recording never runs while the model
is still streaming, and a failed write cannot change the tool result.
Overlapping warm validations with the same cache entry and fresh extensions hash
are process-local single-flight; the flight is removed before followers wake, so a
later call validates again and a different extensions hash never joins. It is not
a TTL: mutations, support-library changes and target tampering invalidate the
snapshot; only rebuilds take the cross-process lock and publish the marker and
metadata manifest. Keep fixed mount paths, fail closed on invalid availability,
and never follow a target symlink while repairing or clearing the capability.
`SKILL_VIEW_CACHE` provides lock-wait/refresh/total timing and entry counts, not
skill contents. The full metadata check is O(number of known entries), not O(1);
the tool hot path above does not run it.
Model timing callback opt-in uses `run_inline=True` only for lightweight timing
and bounded enqueue; slow logging handlers run on a dedicated daemon, never on
the event loop or default tool executor. Queue overflow is best-effort diagnostic
loss with a `LLM_TIMING_LOG_DROPPED` summary. Other callbacks remain independent.
Regressions live in `packages/harness/tests/test_runtime_performance_config.py`,
`test_skill_view_tool_reuse.py`, `test_skill_view_cache.py`, and
`test_model_timing_nonblocking.py`. The first three flags do not increase worker
or thread counts. The fourth enables Gateway default-executor tuning in
`app/gateway/runtime_executor.py`, called from lifespan after config/logging setup
and before `langgraph_runtime`. Resolve nonempty `DEER_FLOW_ASYNCIO_THREAD_POOL_SIZE`
first, `DEERFLOW_IO_THREADS` second, and YAML `asyncio_thread_pool_size` last
(default 100, integer range 1..1024). Disabled means do not touch the executor,
even if an environment override is present. An invalid selected environment
value fails startup rather than silently reverting to a small pool. Log the
effective value/source/pid through `GATEWAY_ASYNCIO_EXECUTOR`; the AppConfig log
shows only the YAML value, not the final environment override. Uvicorn's owning
event loop cleans up the executor after runtime teardown: do not prematurely
shutdown it in lifespan, and never schedule its shutdown on itself with to_thread.
This does not modify AnyIO's separate pool, subagent event loops/executors,
Gateway worker count or upstream model quotas. Tests:
`packages/harness/tests/test_gateway_runtime_executor.py`. Restart after thread
pool changes; model timing instances also need restart for consistent behavior.
**Interface**: Abstract `Sandbox` with `execute_command`, `read_file`, `write_file`, `list_dir`
**Provider Pattern**: `SandboxProvider` with `acquire`, `get`, `release` lifecycle
**Implementations**:
@@ -766,6 +908,7 @@ MySQL index script synchronized.
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`, `deer-flow/skills/`
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
- Configured mount portability: `AppConfig._resolve_relative_mount_paths` resolves each relative `sandbox.mounts[*].host_path` against the loaded `config.yaml` directory before validation/provider construction, never against the process cwd. Keep repository-owned knowledge mounts relative (for example `knowledge/knowledge_base`); absolute paths remain unchanged, and sandbox providers always receive an absolute path.
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, using the persisted creator for every tracked thread.** The route's regular owner check still authorizes access first; creator resolution only selects the stable physical directory (`threads_meta.user_id`) and avoids a lost request context falling back to `users/default`. This also preserves cross-account task-deeplink conversations (rwfx/3qfx/xdfx) and roundtable system threads. Positive creator mappings are cached process-wide and DB/memory/missing-row failures fall back to the current login. It is wired at the thread-path consumers including `ThreadDataMiddleware.before_agent`, `present_file_tool`, uploads, artifact serving, and embedded `client.get_artifact`. **Rewrite-history recovery is metadata-only and job-owner scoped** (`list_by_session(..., user_id=current_user)`), so its GET/list endpoints intentionally skip redundant thread-owner checks and return an empty history instead of a history-only 404; the start/commit endpoints retain strict thread write authorization. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.
- **Durable document-rewrite commit:** `document_rewrite_pipeline.atomic_write_text` writes through a deliberately short sibling temporary filename so long Windows artifact paths stay below the path-length limit, and guarantees the target `outputs` parent exists. If the sandbox recreates that directory in the narrow commit window, it repairs the directory and retries; the original file is otherwise only changed through `os.replace`.
@@ -823,9 +966,24 @@ MySQL index script synchronized.
### Skills System (`packages/harness/deerflow/skills/`)
- **Cognitive situation-map runtime parity**: `cognitive-situation-map` has a
source runtime under `skills/public/cognitive-situation-map/assets/runtime/`
and a Gateway-served copy under
`packages/harness/deerflow/html_runtime/assets/cognitive-situation-map/`.
Keep their JavaScript, schema, and map atlas assets in sync. Built-in regions
include Taiwan Strait, South China Sea, the Philippines-centred surrounding
region (with its southern Sulu/Celebes approaches), First Island Chain, and
the U.S.--Pacific connection; every region is a four-quadrant atlas for the
terrain / maritime / tactical / briefing variants. `scripts/render.mjs`
permits a direct input `scenario.json` as the sole item in a fresh output
directory; all other non-empty output directories need a new path or explicit
overwrite authorization.
- **Location**: `deer-flow/skills/{public,custom}/`
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools)
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools). `name` and `description` are the minimal metadata contract; the frontend install dialog renders the shared `MINIMAL_SKILL_MD_EXAMPLE` containing only those two keys so optional fields are not mistaken for requirements. Keep `frontend-web/src/core/skill-example.test.ts` aligned with backend validation.
- **Loading**: `load_skills()` recursively scans `skills/{public,custom}` for `SKILL.md`, parses metadata, and reads enabled state from extensions_config.json. **A directory containing `SKILL.md` is a self-contained leaf** — discovery does not descend into its subdirectories, so a skill *package* that bundles its own `examples/`/`references/` sub-skills (each with a SKILL.md) surfaces as **one** skill, not a swarm of phantom entries.
- **Built-in seed source**: Product-owned defaults live under `app/gateway/routers/_*_seed_assets/skills/` and must be versioned even though runtime `skills/` trees are ignored. Seeders copy only when a target directory is absent and never overwrite administrator changes. Internal support packages such as `task-report-lib` deliberately omit `SKILL.md` and declare dependents in `.deerflow-skill-support.json`; keep `tests/test_seed_assets.py` and `tests/test_task_report_convert.py` synchronized with the shipped inventory and contracts.
- **Online/offline lifecycle**: reuse `Skill.enabled`; do not add a parallel lifecycle flag and do not substitute `published` or curator `state`. `PUT /api/skills/{name}` is a global switch: administrators may toggle every skill, while an ordinary user may toggle only a non-built-in `custom` skill whose persisted `owner_user_id` matches that user. It patches only `skills.<name>.enabled` in the raw `extensions_config.json`, preserving MCP placeholders and unrelated/unknown fields, then reloads configuration and refreshes the lead prompt cache. Writes use atomic replacement on ordinary files and an `fsync` in-place fallback for writable Docker single-file bind mounts; supported launchers instead keep the runtime file under a writable persisted config directory. Offline skills remain on disk and in the full catalogue (`load_skills(enabled_only=False)`), and existing agent/workflow references are deliberately retained so bringing the skill online restores them. Every model-facing path must use the enabled catalogue: prompt injection, `search_skills`, `skill_list`, `skill_view`, subagents, AI-writing/deep-research/roundtable discovery, and Workflow run-start resource validation. Direct sandbox access is enforced by a copied online-only view under writable `DEER_FLOW_HOME`; `/mnt/skills` in LocalSandbox and AIO/remote sandboxes must point only to that view, never to the canonical tree. AIO creation/reuse attests the view marker and fails closed when an older provisioner ignores the requested mount; explicitly declared internal support libraries (such as `task-report-lib`) enter the view only while a dependent skill is online. Path-sensitive `DEER_FLOW_*` variables and `PYTHONPATH` are removed from LocalSandbox bash while non-path settings such as `DEER_FLOW_GATEWAY_PORT` remain. LocalSandbox host bash still shares the backend UID and is not an OS isolation boundary; production requiring hostile-code isolation must use an attested AIO sandbox. Frontend controls are in the administrator 技能管理 list and, for owners, 「技能 → 技能列表 → 我的」; offline rows and retained agent references remain visible with an 已下线 badge and can be removed, while new selectors exclude them.
- **Discovery↔delete consistency**: a skill is keyed for discovery by its frontmatter `name`, but its on-disk directory name may differ (hand-dropped / mis-extracted package). `LocalSkillStorage._resolve_skill_dir(name, category)` resolves the *discovered* directory (canonical `<category>/<name>` fast-path, else scan-by-frontmatter-name); `custom_skill_exists`/`read_custom_skill`/`delete_custom_skill` all go through it, so **anything that gets listed can also be read and deleted** regardless of directory name (the old code assumed `custom/<name>/` and 404'd "Custom skill 'x' not found." on any non-canonical layout). Delete is path-guarded to never escape the `custom/` root. Tests: `tests/test_skill_storage_delete.py`.
- **Injection**: Enabled skills listed in agent system prompt with container paths
- **Configurable `es_query` routing prompt**: `config.yaml → skills.es_query_routing`
@@ -998,6 +1156,69 @@ scheduler runs once per deployment. Per run:
- Supports `supports_vision` flag for image understanding models
- Config values starting with `$` resolved as environment variables
- Missing provider modules surface actionable install hints from reflection resolvers (for example `uv add langchain-google-genai`)
- Generic `langchain_openai:ChatOpenAI` entries are transparently wrapped by
`PatchedChatOpenAI`. Besides preserving reasoning/tool-call metadata, the
patch recognizes compatible gateways that encode failures inside HTTP 200
payloads (`base_resp.status_code != 0` with null/empty `choices`) in both
streaming and non-streaming paths. It raises the stable
`OpenAICompatibleUpstreamError`; `LLMErrorHandlingMiddleware` retries
transient concurrency/inference failures, while genuine quota/balance
failures are classified as non-retriable and surfaced immediately. Embedded
vendor failures therefore cannot become empty successful assistant turns.
The same adapter repairs only bounded, information-preserving compatibility
deviations before LangChain conversion: a streamed full `message`,
choice-root tool calls, tool calls beside an empty `delta`, singleton tool
call objects, structured `function.arguments`, and missing/string indexes.
Tool argument assembly is invocation-local (never model-instance shared),
assigns a stable canonical index to staggered id-bearing parallel calls, and
distinguishes confirmed cumulative snapshots from standard OpenAI deltas;
once a call is identified as delta mode, brace fragments and all later bytes
pass through unchanged. Gemini `thought_signature` / `thoughtSignature`
(including `extra_content.google` spellings) is extracted before the SDK
drops extension fields, stored as dedicated id/index metadata, and restored
at the same wire location on the next request. Keep the concurrent-stream,
nested-brace, signature round-trip, and real-SDK structured-argument tests in
`packages/harness/tests/test_patched_openai_tool_args.py` when changing this
boundary.
- Model observability is content-free. `ModelTimingCallback` logs connection,
first chunk, completion/failure/cancellation latency against a sanitized
upstream origin and run id. `PatchedChatOpenAI` emits one
`OPENAI_TOOL_STREAM_SUMMARY` for tool/repair/error streams with shape counts
and first-tool-argument latency. `LLMErrorHandlingMiddleware` logs only stable
retry fields (`attempt`, `reason`, error type, status and provider code); raw
provider details remain available only for in-process classification and must
never enter exception strings, logs, or stack traces. Prompts, response text,
tool arguments, file contents, credentials and raw exception messages are
forbidden in these records. `GeneratorExit`/`CancelledError` is an INFO-level
`status=cancelled`, not a failed-call alarm.
Diagnostic handlers must not replace a successful result, the original model
exception or cancellation when they fail. Safe upstream exception strings
retain private classification detail only in memory: context recovery must
still recognize numeric vendor codes with context-length messages, and auth
classification must check structured codes as well as human descriptions.
- LangGraph run-creation streams must expose both `Content-Location` (canonical
run resource) and `Location` (the `/threads/{thread}/runs/{run}/stream`
reconnect path relative to the SDK's `/api` base). The CORS middleware must
continue exposing both headers. The frontend records each consumed SSE event
id before yielding it to React. Cursor ownership must match exactly one SDK
message accumulator: same-reader recovery supplies that cursor; a newly
mounted hook or a real thread switch must reconstruct from `-1`, not inherit
another reader's cursor. Shared API clients never rewrite that sentinel.
`useThreadStream` counts message-tuple
changes as business activity, probes a known active run only after 90 seconds
without a business frame, and stops only the local reader before rejoining.
Clean-EOF, transport-error, conflict and watchdog triggers share one
single-flight reconciliation so they cannot double-join or double-finalize.
- The default `MemoryStreamBridge` and `RunManager` are process-local. Gateway
streaming must run with one uvicorn worker: `Location`, `/join`, cancellation,
and `Last-Event-ID` recovery are separate HTTP requests and a different worker
cannot see the originating worker's live task or buffered events. A durable
checkpointer/RunStore stores history but is not a shared live stream/control
plane. `scripts/start_backend-online.sh` and
`offline-backend-20260512/start_backend.sh` must keep `GATEWAY_WORKERS=1` as
their safe default and reject larger values unless an operator explicitly accepts
the unsafe override. Multi-worker support requires shared implementations for
both event transport and live-run ownership; sticky routing is not a fix.
### Roundtable Model Fault Tolerance (会商模型自动容错 / 换下一个模型)
@@ -1014,9 +1235,11 @@ scheduler runs once per deployment. Per run:
### vLLM Provider (`packages/harness/deerflow/models/vllm_provider.py`)
- `VllmChatModel` subclasses `langchain_openai:ChatOpenAI` for vLLM 0.19.0 OpenAI-compatible endpoints
- Preserves vLLM's non-standard assistant `reasoning` field on full responses, streaming deltas, and follow-up tool-call turns
- Designed for configs that enable thinking through `extra_body.chat_template_kwargs.enable_thinking` on vLLM 0.19.0 Qwen reasoning models, while accepting the older `thinking` alias
- `VllmChatModel` subclasses `langchain_openai:ChatOpenAI` for vLLM OpenAI-compatible endpoints
- Thinking text is accepted from either `reasoning` (newer vLLM parsers) or `reasoning_content` (older vLLM / DeepSeek-style). The UI still reads canonical `additional_kwargs.reasoning_content`. Follow-up turns echo only the wire field that actually arrived, so other ChatOpenAI models are unchanged
- Shared helpers live in `deerflow/models/reasoning_fields.py` and are also used by `PatchedChatOpenAI` (the default `langchain_openai:ChatOpenAI` patch)
- Designed for configs that enable thinking through `extra_body.chat_template_kwargs.enable_thinking` on vLLM Qwen reasoning models, while accepting the older `thinking` alias
- Per-model switch: set only that entry's `use: deerflow.models.vllm_provider:VllmChatModel`. Do not change SiliconFlow / DeepSeek entries that already use `langchain_openai:ChatOpenAI`
### IM Channels System (`app/channels/`)
@@ -1126,7 +1349,47 @@ Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` me
### Conditional WeKnora LLMWiki
Ordinary knowledge-base cards expose an owner/admin-only name/description edit dialog in `WeKnoraKnowledgeWorkspace.tsx`, reusing `useLlmWikiActions().update` and `PATCH /api/llmwiki/knowledge-bases/{id}`. The route trims names, rejects blank or reserved conversation-deposit names, writes to WeKnora before updating local metadata, and retains existing write authorization. A successful remote list is authoritative for every list scope, including `personal`: absent remote IDs are hidden without deleting mappings/references. Upstream failures remain errors. The grid also filters cached `remote_status=missing` rows and no longer renders stale-record cleanup cards. Regression coverage: `tests/test_llmwiki_knowledge_management.py`.
**Selected knowledge-space policy:** if either `llmwiki_knowledge_base_ids` or
`assistant_knowledge_base_ids` is non-empty, Gateway sets
`knowledge_space_strict`. This is server policy, not a caller preference:
after ordinary context merging, Gateway overwrites both final
`configurable` and `context` with the authorized knowledge ids,
`knowledge_space_strict=true`, and the merged exclusion list, so a request
cannot pre-seed `false`, forged ids, or `excluded_tools=[]` to reopen external
retrieval. The lead agent must fail closed: no Web/browser
tools, MCP, external retrieval skills, ACP, deep-research pipeline, or
subagents. Only the pre-retrieved selected-space context and the read-only
`llmwiki_search` compatibility tool may retrieve knowledge; that tool supports
both ordinary and assistant Wiki stores and cannot accept IDs from the model.
Ordinary Wiki retrieval runs vector and local lexical channels concurrently,
uses RRF, reports raw vector threshold diagnostics, and continues serving a
coherent last-successful snapshot after a later sync is cancelled/failed.
`config.yaml` defaults the Wiki encoder to SiliconFlow `BAAI/bge-m3`; keep
`SILICONFLOW_API_KEY` outside Git. The frontend derives four starter questions
from the selected space names and hides generic recommendations while these
questions are present. Design and remaining graph/profile phases are in
`docs/KNOWLEDGE_SPACE_RETRIEVAL_AND_WIKI_EXPANSION_ZH.md`.
**Embedding ownership:** WeKnora's per-knowledge-base `embedding_model_id`
indexes uploaded source documents; `llmwiki.local_wiki_index.embedding` indexes
the generated Wiki-page mirror inside DeerFlow. Do not couple these two model
choices. `GET /api/llmwiki/embedding-models` returns a strict credential-free
projection of WeKnora Embedding models, and KB creation must validate and pass
the selected active model id. When the caller omits a choice, retain WeKnora's
active-default behavior. Existing KBs cannot change this binding in place, and
cross-KB WeKnora searches should use model-compatible knowledge bases.
Wiki-enabled knowledge bases accept two bounded, user-owned prompt extensions:
`wiki_config.content_instructions` (generated-page language, structure and
emphasis) and `wiki_config.extraction_instructions` (domain entities/concepts
to prioritize). The create/edit dialogs expose both values; Gateway accepts
only those two fields and `WeKnoraClient` merges edits into the remote
`wiki_config` so `synthesis_model_id`, extraction granularity and WeKnora-owned
runtime knobs are never dropped. WeKnora's upload/reparse Wiki ingest is the
actual consumer: saving instructions affects later ingests and does not itself
rewrite existing pages.
Ordinary knowledge-base cards expose an owner/admin-only name/description/Wiki-instructions edit dialog in `WeKnoraKnowledgeWorkspace.tsx`, reusing `useLlmWikiActions().update` and `PATCH /api/llmwiki/knowledge-bases/{id}`. The route trims names, rejects blank or reserved conversation-deposit names, writes to WeKnora before updating local metadata, and retains existing write authorization. Default test questions are derived from the authoritative remote display name. The editor omits `test_questions` unless the user actually changes that field, and `knowledge_profile.generated_test_questions_name` recognizes only the exact legacy four-question template as automatic; genuinely edited lists remain custom. `_mapping_view` must resolve both questions and `test_questions_customized` from a profile row containing the current remote name, so a rename immediately replaces old-name defaults without a migration. A successful remote list is authoritative for every list scope, including `personal`: absent remote IDs are hidden without deleting mappings/references. Upstream failures remain errors. The grid also filters cached `remote_status=missing` rows and no longer renders stale-record cleanup cards. Regression coverage: `tests/test_llmwiki_knowledge_management.py`, `tests/test_llmwiki_weknora.py`.
`llmwiki.weknora.api_base_url` is the provider switch: non-empty enables WeKnora, empty preserves the legacy LLMWiki UI. WeKnora is independently deployed and owns PostgreSQL/Redis/object storage/vector storage/parsers/models; DeerFlow config contains only `api_base_url` and optional `web_base_url`. DeerFlow deliberately sends no WeKnora credential, so the WeKnora endpoint must be restricted to a trusted network and allow DeerFlow traffic itself or through its reverse proxy. There is intentionally no `default_embedding_model_id`: `WeKnoraClient` queries `/api/v1/models` and uses WeKnora's active Embedding model when creating a KB; Wiki creation additionally resolves an active KnowledgeQA model inside WeKnora for synthesis. The native BFF is `app/gateway/routers/llmwiki.py`; ownership/publication mappings live in `deerflow.persistence.llmwiki`; the browser only sees DeerFlow mapping IDs. Knowledge-base owners publish immediately (`private` -> `published`) without a review queue, and can unpublish ordinary bases. The conversation-deposit KB is system-owned, always published, shared by all DeerFlow users, excluded from My Knowledge, and immutable through user/admin detail operations; recognition requires both the reserved name and `system` owner. Existing same-name remote mappings are adopted in place as the system deposit, preserving mapping ids and document links. Its user-facing description, generated Markdown source and tags use `cmzs` branding; list/detail reads synchronize stale remote descriptions, and search results, source previews and embedded-detail JSON recursively replace legacy brand text in historical conversation-deposit payloads. Deposit requests must reference a thread accessible to the current account. Personal knowledge-base list/search views are scoped to the current DeerFlow account even for admins; frontend LLMWiki caches include the live account id, and admin write power does not make other users' private KBs appear as owned. Only `system_role === "admin"` sees the WeKnora connection-config tab. Knowledge cards navigate to a dedicated WeKnora-style detail route. Wiki-enabled bases render and default to `Wiki -> documents -> graph`; standard bases omit Wiki and render `documents -> graph`. The detail page supports sequential multi-file upload with an aggregate result for writable ordinary bases, hides it for public read-only/system-deposit bases, and uses a generic loading label while the embed session starts. It also supports URL and published manual-Markdown imports, browses real Wiki pages/content, proxies authenticated original-file previews, pages through `/chunks/{knowledge_id}`, and renders the Wiki page graph from `/knowledgebase/{kb_id}/wiki/graph`; document details use a right sheet rather than the removed KB modal. The credential-free proxy uses one WeKnora service identity, so per-user separation is enforced by DeerFlow mappings while all remote KBs physically share that identity's WeKnora workspace. WeKnora iframe detail pages first call `/api/llmwiki/knowledge-bases/{id}/weknora/session` through the normal DeerFlow API auth path, then load `/platform/knowledge-bases/{weknora_id}` with a signed `deerflow_weknora_embed` cookie for `/platform`, `/assets`, `/locales`, root assets, and API calls. Injected fetch/XHR rewriting sends all iframe `/api/v1/*`, including colliding `/api/v1/auth/*`, through `/api/llmwiki/weknora-embed/api/v1/*`; Gateway auth/CSRF bypass only applies to those signed iframe proxy requests, the proxy still revalidates the mapped KB, and cookie security honors `X-Forwarded-Proto` for HTTPS reverse proxies. `llmwiki_search` is conditionally registered in `tools.py`, is read-only, and rechecks the current DeerFlow user, agent `llmwiki_knowledge_base_ids`, and thread-context selection on every call. Source previews prefer the cleaned Wiki/LLMWiki page text when a retrieved chunk maps to a Wiki page, with raw split chunks as the fallback. Agent bindings use an inline WeKnora-style searchable/grouped multi-select in the standard agent create/detail UI; new chats use the same interaction pattern as a compact composer dropdown, and selections are written to thread metadata/context and restored on reopen. The DeerFlow frontend is native React and the WeKnora frontend is not modified or embedded. Full deployment and behavior notes: `docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md`. Tests: `tests/test_llmwiki_weknora.py`.
@@ -1286,13 +1549,30 @@ link can become `#/login/<user>?password=…`. Regression coverage:
### File Upload
Multi-file upload with automatic document conversion:
- Endpoint: `POST /api/threads/{thread_id}/uploads`
- Supports: PDF, PPT, Excel, Word documents (converted via `markitdown`)
- Rejects directory inputs before copying so uploads stay all-or-nothing
- Reuses one conversion worker per request when called from an active event loop
- Files stored in thread-isolated directories
- Agent receives uploaded file list via `UploadsMiddleware`
Chat upload configuration is a database-backed global policy, not a static
`config.yaml` size tuple. `deps.py` creates and seeds the singleton row, the
Gateway loads it for every request, and `deerflow.uploads.policy` exposes only a
validated runtime copy to the harness. The administrator page under settings
edits or resets the same row. The built-in defaults are 20 files / 100 MiB total;
documents and images are 10 MiB per file, and spreadsheets are 5 MiB.
- Endpoint: `POST /api/threads/{thread_id}/uploads`; policy endpoints:
`GET/PUT /api/upload-policy`, `POST /api/upload-policy/reset`
- Supported policy formats: TXT/Markdown, PDF, DOC/DOCX, PPT/PPTX, JPG/JPEG/PNG,
and XLS/XLSX/CSV; the server restricts edits to parsers it can validate safely
- The whole batch is streamed to a private staging directory, checked for count,
total size, per-category size, extension, signature/container and encoding,
then atomically committed; validation or sandbox-sync failure restores old files
- Optional PDF/PPT/Excel/Word conversion always runs off the event loop with
bounded concurrency and a fixed extracted-text ceiling; companions are named
`report.docx.md`, with legacy `report.md` lookup retained for old uploads
- Files stay in thread-isolated directories; `UploadsMiddleware` injects bounded
JSON metadata plus small content samples, never unbounded bodies or repeated
converted companions
- Range-free large local reads return streaming full-file statistics and
deterministic head/middle/tail samples. Explicit line reads, `grep`, artifact
previews, image dimensions/derivatives and final model context all have hard caps
- Uploaded raster images also receive a signed public HTML URL (`/api/public/html-images/{token}`) so generated pages can use them as `background-image`
See [docs/FILE_UPLOAD.md](docs/FILE_UPLOAD.md) for details.
@@ -1322,7 +1602,25 @@ Automatic conversation summarization when approaching token limits:
when it has already been folded into the hidden summary. This keeps strict
OpenAI-compatible gateways from rejecting a tool-continuation call with
`No user query found in message`.
- LangChain keeps the agent prompt in `ModelRequest.system_message`, separate
from `ModelRequest.messages`. Compression must merge its hidden summary into
that dedicated slot and remove every `SystemMessage` from the ordinary
message list. Strict Qwen-style adapters accept only one system message at
index zero and otherwise return HTTP 400 (`system message must be at the
beginning`). Keep this normalization adapter-independent; the payload-level
OpenAI-compatible normalizer is a second line of defence and also merges
adjacent leading system messages.
- Keeps recent messages while summarizing older ones.
- **The model input hard guard is independent and always on when configured.**
Set `models[].max_input_tokens` (or `profile.max_input_tokens`) to the safe
input ceiling; `max_tokens` is output-only. `ContextWindowGuardMiddleware`
is mounted at the innermost end of both lead-agent and subagent model-call
chains, after memory/RAG/image/custom injections. It counts the complete
request (system prompt, messages/reasoning, tool schemas and response format),
then transiently removes old complete turns and truncates oversized bodies as
a last resort without mutating checkpoint state. Provider context-length
errors are non-retriable so a tokenizer mismatch cannot trigger repeated
submission of the same oversized request.
Tests: `tests/test_summarization_toggle.py` (gating + whitelist),
`tests/test_summarization_middleware.py` (cache behavior).
@@ -1561,9 +1859,11 @@ successful full scan, matching embedding fingerprint, no failed pages,
`ready_page_count == local_page_count`, and vectors for a non-empty library.
Ready mappings read DeerFlow's `llmwiki_wiki_pages` /
`llmwiki_wiki_vectors` / `llmwiki_wiki_sync_states` tables; incomplete mappings
call only WeKnora `/knowledgebase/{id}/wiki/pages?query=...` plus Wiki-page
detail. Never call `/knowledge-search` from internal search, RAG, or the
`llmwiki_search` tool, and never return raw chunks as the fallback. Core code lives under
in the Wiki-page retrieval service and `llmwiki_search` tool call only WeKnora
`/knowledgebase/{id}/wiki/pages?query=...` plus Wiki-page detail and never use
raw chunks as a silent fallback. The pre-answer RAG orchestrator separately
runs the explicit, authorized `/knowledge-search` raw-chunk channel described
under **Knowledge retrieval diagnostics** below. Core code lives under
`deerflow.integrations.weknora.local_index` and
`deerflow.persistence.llmwiki_index`; preserve the harness → app import
firewall. Gateway composition injects the store/search/sync services through
@@ -1577,8 +1877,10 @@ existing vectors.
Vectors are normalized little-endian float32 BLOBs. A page mirror and all of
its vectors are replaced in one transaction only after strict batched
embedding succeeds. Never delete the old generation before a remote embedding
call and never fall back to WeKnora raw chunks while the local feature flag is
enabled. NumPy matrices are per-mapping LRU snapshots invalidated by
call. The local Wiki-vector service must not silently substitute WeKnora raw
chunks when embedding fails; the pre-answer orchestrator's separately labelled,
authorized raw-chunk channel is independent rather than that fallback. NumPy
matrices are per-mapping LRU snapshots invalidated by
`index_revision`; large decode/matrix work runs through `asyncio.to_thread`.
`embedding.dimensions` is a local response-size invariant and fingerprint
input, not an outbound API field: fixed-dimension BGE/OpenAI-compatible
@@ -1594,6 +1896,50 @@ page is successfully rebuilt. `WikiRetrievalService` partitions mixed requests
per mapping, so ready libraries stay local while incomplete libraries use that
Wiki-only fallback.
Library-level overview requests (for example, “此知识库都讲了什么” or “这个知识空间有哪些内容”) are
recognized once by `local_index.overview` and shared by local vector and remote Wiki-page retrieval. A completed
local index returns representative page bodies without embedding the broad query or applying `min_similarity`; an
incomplete index asks the remote page API for its directory rather than attempting a wording match that can return
zero. In both paths directory, index, and summary pages are ineligible, and a page summary must never substitute
for its body. `AssistantKnowledgeRepository` implements the same user-facing overview semantics independently for
assistant knowledge bases.
**Knowledge retrieval diagnostics.** `app.gateway.llmwiki_rag.build_llmwiki_rag_context` is the single
pre-answer retrieval path for both normal Q&A and `POST /api/llmwiki/retrieval-debug`; do not build a second
debug-only ranking implementation. The debug endpoint accepts explicit ordinary and assistant knowledge-base ids
and returns the bounded `knowledge_retrieval_trace`: selected/authorized spaces, actual retrieval mode, channel
candidate counts, vector threshold diagnostics, warnings/fallback, zero-result reason, timings, and the exact
bounded evidence made available to the model. A normal run stores the same trace on the latest `HumanMessage`,
publishes it in the initial SSE `metadata` event, and exposes durable pre-retrieval plus model-driven
`llmwiki_search` tool traces through `GET /api/threads/{thread_id}/reference-batches` under `retrieval_traces`.
Keep trace results bounded to the same eight sources and 1400 characters per source as the answer prompt; never
expose provider credentials or unrelated library contents. If a client omits selection keys on a later turn,
recover the persisted thread selection; an explicitly present empty list must continue to clear the scope.
Ordinary pre-retrieval searches both generated Wiki page bodies/sections and authorized WeKnora raw document
chunks. The raw channel must be called only with remote ids derived from already-authorized DeerFlow mappings,
must reject every returned row whose remote id is not in that authorized map, and must allowlist/bound source
fields before adding them to prompts or traces. Run raw-chunk search concurrently with Wiki retrieval; one channel
failing marks the trace partial but must not discard successful evidence from the other. A ready local Wiki snapshot
uses vector + lexical channels with RRF; incomplete mappings use lexical Wiki recall and the remote Wiki-page API;
Wiki and raw result lists are fused without comparing provider score scales. Raw provider scores are channel scores,
not cosine similarity. Assistant knowledge runs vector and keyword/raw-chunk recall together and fuses both ranked
lists with weighted RRF; exact literal identifiers such as device or incident codes take priority, so imported raw
chunks remain searchable even when compatible Wiki vectors exist. This is not guaranteed pure-vector retrieval on
every turn. The same pre-answer orchestrator performs bounded graph expansion for ordinary Wiki links and imported
assistant entity relations whenever the selected repository exposes `get_adjacent_pages`. It uses up to four directly
recalled Wiki pages as
seeds (two per authorized mapping), traverses one hop by default and at most two hops for explicit multi-hop
wording or low-confidence vector diagnostics, inspects at most 24 adjacency rows, and accepts at most six pages
(three per mapping). Every returned page is rechecked against the authorized local mapping ids, must contain a
real eligible Wiki body, and carries a local-only slug/title path plus source/chunk provenance. Decayed walk scores
rank graph candidates; fixed graph slots merge them with Wiki/raw evidence without comparing unrelated score
scales. Trace fields include graph seeds/candidates/accepted/used, requested/executed hops, paths, depth reason and
`timings_ms.graph_expansion`; capability is dynamic. Graph failure marks the trace partial and must never discard
successful Wiki, raw-chunk, or assistant evidence. Assistant relation traversal resolves both endpoints through the
entities' `wiki_page_id`, and requires the relation, both entities, and both pages to be active and inside the same
selected assistant base; it must never follow cross-base relations.
The frontend must keep Wiki citations visibly distinct from raw-document
citations while hiding provider-brand wording from user-facing labels, status
text, and errors. Any click on a Wiki source card opens the shared right-side
@@ -1666,12 +2012,19 @@ conversation-deposit mapping. Never point this page at the authenticated
`/api/llmwiki` reads or weaken the published-only gate merely because the
mapping id is hard to guess.
Owners manually force a full sync/vectorization through `POST
/api/llmwiki/knowledge-bases/{id}/vectorize`; status is available through `GET
Owners manually request a metadata-first incremental sync/vectorization through
`POST /api/llmwiki/knowledge-bases/{id}/sync`; the older
`POST /api/llmwiki/knowledge-bases/{id}/vectorize` route is a compatibility
alias. Status is available through `GET
/api/llmwiki/knowledge-bases/{id}/wiki-index-status`. That response includes
aggregate `progress` and per-page `pages` details; current-scan progress is
derived from `active_sync_id` / `last_seen_sync_id`, while each page exposes its
index state, vector count, timestamps, and bounded error. The Wiki management
index state, vector count, timestamps, and bounded error. `last_sync` is the
persisted, lease-fenced terminal result and includes the sync id/mode/status and
added/updated/skipped/deleted/failed counters. Frequent progress polling must
pass `include_pages=false` so it reads only persisted aggregate state; the Wiki
management view keeps the default detailed response when it needs per-page
rows. The Wiki management
page must show the progress bar and a status badge beside every article. It
also exports all remote processed Wiki pages via `GET
/api/llmwiki/knowledge-bases/{id}/wiki/export.xlsx`. Directory export must keep
@@ -1761,8 +2114,15 @@ and shared entities/relations are retired only when their final contribution is
gone.
Assistant knowledge has one global full-library base (`知识梳理总库`) and any
number of admin-created custom bases. Ordinary users have read/search access and
administrators own all mutations. Assistant bases accept local documents independently
number of admin-created custom bases. Administrators always retain its management
surface and own all mutations. Day-to-day ordinary-user exposure is controlled by
`system_settings.json -> knowledge_base_product_version`: `v1` (the default) hides
assistant navigation, selectors and cross-links so ordinary users see only ordinary
knowledge; `v2` releases those surfaces again. The switch must not delete persisted
assistant bindings, and administrators remain able to configure assistant knowledge in
either mode. This is a recoverable product-surface switch rather than an assistant read
API authorization boundary; hidden assistant selections are omitted from new day-to-day
chat runs while their persisted bindings remain intact. Assistant bases accept local documents independently
of WeKnora via `/api/assistant-knowledge/imports/files`, as well as WeKnora export
packages or imports triggered from a WeKnora mapping. Local documents use the
Gateway `assistant_file_ingest` pipeline: persisted original files, chunked LLM Wiki
@@ -1774,12 +2134,76 @@ custom bases. Deletion removes the custom base's Wiki pages, vectors, entities,
relations, import audit rows, and package files; the global base is never
deletable because it owns full-library aggregation.
Assistant-knowledge reads keep the landing and article paths size-bounded.
`list_bases()` is metadata/statistics-only and must never call the lazy global
alignment pass: imports align their snapshot before commit, while content reads
and search retain `_ensure_base_aligned`. The Wiki directory calls
`GET /api/assistant-knowledge/wiki-pages?include_content=false`; repository
`load_only` projection must exclude both `content_markdown` and `search_text`
(response-only deletion is insufficient), and the frontend loads only the
active slug through `GET /wiki-pages/{slug}`. The explicit index sorts into the
first directory batch; that batch unlocks the reader immediately while later
500-row batches append in the background. A URL-provided slug must fetch in
parallel with the directory. An existing base with zero sources and zero Wiki
pages is a valid, reachable state: keep the Wiki management surface mounted and
show upload/package-import actions instead of requiring a document or redirecting
away. The ordinary Wiki empty reader must likewise link writable users to the
manual Wiki editor. Do not enable global pages, sources, or import-job queries
while the landing route has no `base_id`.
The assistant-base switcher is a hot metadata path. `list_bases()` must load all
four counters with one grouped statement rather than four serial counts per
base, and a metadata-only rename must not rescan content tables. Assistant
vector search validates base ids without loading counters, checks vector
fingerprint/dimensions before encoding, scans only vector blobs plus minimal
page identity in matrix batches, and hydrates bodies only for final top results.
Chinese lexical fallback must extract bounded meaningful subphrases rather than
treating the entire no-space question as one SQL token.
The shared WeKnora runtime client retains its authenticated session and caches
the upstream knowledge-base directory for 15 seconds, coalescing concurrent
scope requests. Knowledge-base and document mutations invalidate that cache;
generation checks prevent an older in-flight list response from repopulating it.
Local Wiki synchronization is metadata-first. A WeKnora page-list item that
contains the body is authoritative for hashing and must not trigger a detail
GET. For legacy list shapes without a body, only matching explicit `version` or
`updated_at` markers may skip the detail GET; missing markers must fail safe to
the detail API. A content-identical page still refreshes category/parent/path,
source/chunk references, page version and remote timestamps under the same
mapping lease, without replacing vectors. Missing pages are retired only after
a complete paginated scan and a final owner/sync-id lease fence. A bodyless
metadata-only scan is not a complete graph snapshot: if the provider graph call
also fails, preserve the prior graph rather than replacing it with partial
edges.
Both Gateway document APIs and successful content mutations through the
embedded WeKnora proxy schedule one per-mapping debounced retry sequence with
bounded 1/8/30/90/300-second backoff. Never cancel an in-flight sync to debounce
a newer mutation; wake the coordinator, update the local generation, and let
the database lease deduplicate workers. While WeKnora reports processing, poll
only the cheap knowledge-base state and retain both a post-processing scan and
one short quiet confirmation. Before retiring missing pages or replacing graph
edges, the sync service must re-read processing state; processing or an
unverifiable state produces a retryable partial result and preserves the prior
page/vector/graph snapshot. The periodic scheduler reads the remote
knowledge-base directory once per tick, skips unchanged/processing libraries
(including forced reconciliation), and retains a 10-minute fallback.
### Wiki package and local encoder invariants (2026-09)
Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_landing` (`wiki` default / `weknora`), with admin-only PUT `/api/system-settings/ordinary-knowledge-landing`. Explicit document/graph/article links bypass this default. Wiki reading and the embedded WeKnora view provide reciprocal navigation; `?view=weknora` explicitly selects the iframe and must not loop back to the default Wiki route.
- `knowledge_transfer.sync_to_global` is called after successful explicit local
Wiki vectorization; it skips conversation/system mappings. Do not enable a new
Wiki vectorization; it skips conversation/system mappings. Concurrent triggers
for one mapping are coalesced inside a Gateway process around the complete
check/create/import sequence. Cross-worker safety lives at the repository
transaction boundary: every assistant import takes the target base row
`FOR UPDATE` before graph alignment or dependent snapshot writes, preserving
one lock order across workers. Retry only recognized transient lock-wait,
deadlock and serialization-contention errors; let the failed transaction roll
back fully, then replay the whole package with bounded backoff. Cancellation
and non-contention failures must propagate unchanged. Do not enable a new
watcher or automatic remote-write scheduler as a side effect.
- `wiki_packages` exposes file-backed assistant exports and restores into empty
ordinary Wiki libraries. Validate vectors before any remote mutation; recreate
@@ -1801,8 +2225,8 @@ Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_
pages need backfill, not a forced rebuild of every page.
FastEmbed is a standard backend dependency. Encoder configuration, directory
upload and warmup live only in the administrator global settings dialog.
A runtime `system_settings.json` path override wins over config.yaml; uploaded
files are streamed into `.deer-flow/models/bge-m3`, never the source tree.
When the effective provider is local, its `system_settings.json` path override wins over config.yaml; uploaded
files are streamed into immutable `.deer-flow/models/bge-m3-revisions/<revision>` directories, never the source tree.
Encoder warmup is explicitly started from the settings UI/API and must never block application startup
or turn a model-load failure into a DeerFlow service failure. Local provider
is offline-only: require local_model_path or DEERFLOW_BGE_M3_MODEL_PATH and
@@ -1810,6 +2234,21 @@ Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_
`local_process_isolation=true` by default: inference runs in the bundled JSONL
worker subprocess, and shutdown must terminate that worker without waiting on
model teardown.
- The administrator encoder override is an explicit persisted mode, not an
inference from whether `model_path` happens to be non-empty. `configured`
uses the current `config.yaml` provider (including OpenAI-compatible);
saving/uploading a local directory sets `local` plus a new model revision,
while `/api/llmwiki/local-encoder/use-configured` switches back. Every switch
hot-rebuilds embedding/cache/search/sync/retrieval as one graph and updates
the harness runtime registration. Status must remain readable for a remote
provider. The scheduler stays alive as a lightweight config watcher but only
scans when effective `enabled && auto_sync`; keep the default/portable release
configs at `auto_sync=true` because asynchronous Wiki generation can finish
after the event-triggered first scan. A changed local model revision must
change the embedding fingerprint so vectors from replaced weights are never
reused. Uploaded weights live in immutable revision directories. A replaced
embedding generation gets a bounded grace period for in-flight work, then
rejects stragglers, drains real thread work, and releases its model process.
- Every successful mapping snapshot is mirrored to the global assistant base,
including empty snapshots so deleted source pages are retired. Re-import is
source-replacement semantics: absent Wiki/chunk/entity/relation pages must not
@@ -1818,6 +2257,13 @@ Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_
duplicate synthetic entity/relation articles. Store original Wiki metadata
separately from assistant workflow status. Count Wiki pages separately from
raw document chunks.
- Assistant graph retrieval resolves each active relation through both active
endpoint entities' `wiki_page_id` values and their active Wiki pages. Every
relation, entity, seed page, and neighbor page must match the requested
`base_id`; never traverse a corrupt cross-base edge. Pre-retrieval uses the
same bounded one/two-hop walk as ordinary Wiki, labels graph evidence as
assistant evidence, and lets a directly recalled non-seed page remain a
traversal frontier without duplicating it in the answer context.
- The global base aligns entities by normalized display name and aliases across
source records, and aligns content Wiki pages by normalized title. Do not use
source slug as the global canonical identity. Merge unique knowledge blocks
@@ -1832,6 +2278,21 @@ Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_
`tests/test_wiki_package_roundtrip.py`, `test_llmwiki_wiki_sync.py` and
`test_assistant_knowledge_repository.py`.
## Startup schema failure invariant
Gateway availability must not depend on successful startup DDL. ORM
`Base.metadata.create_all` (including database auto-create retries) logs schema
failures and continues with the existing engine/schema; do not restore a bare
`raise` that aborts FastAPI lifespan. Startup seed reads/writes must be
best-effort and use a process-local default where one exists. LangGraph
Checkpointer and Store own separate `setup()` tables, so `deps.langgraph_runtime`
must fall back to `InMemorySaver` / `InMemoryStore` if entering either persistent
context fails. The fallback keeps the API process available but is intentionally
non-durable; operators must still repair the logged database error and restart.
Regression coverage lives in `tests/test_ensure_orm_columns_sync.py` and must
exercise both ORM `create_all` failure and runtime-backend `setup()` failure.
## openGauss / GaussDB persistence invariant
`database.backend=gauss` is an application-ORM backend, parallel to `mysql`.
@@ -1844,3 +2305,132 @@ configured SQLite backend; the PostgreSQL checkpointer schema is not part of the
Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
names as timezone-aware PostgreSQL-family engines in portable types and
dialect-specific session settings.
## 2026-09-14 我的项目与知识库验收修复
- 简洁模式的「我的项目」列表、策略工作台和普通问答项目恢复工作区功能导航;窄屏提供展开菜单按钮。
- 普通问答先创建具名项目,再在项目内创建多个对话。列表读取 `/api/analysis-projects` 的 `qa` 项目,不再读取、标记或自动归入全局普通聊天。
- 项目设置更新保留既有知识库绑定。原「项目空间」与官方入口继续可用;我的项目下问答路由为 `qa/projects/groups/*`。
- 策略生成卡片统一先显示思考/执行过程,再显示正文;评判、方案及展开视图沿用相同顺序。
- 完成评判但尚未勾选策略的项目可以从方案列表进入,在工作台选择后生成;已保存方案在任务修改后仍可打开。旧列表请求不再覆盖新请求状态。
- 知识库管理页读取上游最新目录,避免远端删除后命中旧缓存;检索调用仍共享短期目录缓存。
- 管理员,或管理员已开放 `v2` 展示后的普通用户,访问尚未初始化的助手知识库时显示明确空态及刷新按钮。
验证:后端相关回归 273 项、前端策略/项目回归 31 项通过;真实 BAAI/bge-m3 编码、指定库向量检索与草稿隔离通过;本地隔离服务的问答项目创建/分组、真实问答、方案整理、助手库创建/编辑/文件入库/向量检索/详情均通过。完整验收记录见仓库 `docs/ji-knowledge-projects-acceptance.md`。不在源码或文档中保存接口密钥。
2026-09-14 日常使用复查:补修项目历史入口与移动侧栏、默认智能体同步、项目加载重试、助手知识检索过期响应隔离及无结果指引;更新 ScrollArea 的 React 19 兼容修复。后端再次 273 项通过,前端 33 项通过。范围及限制见仓库 docs/daily-usage-audit-20260914.md。
### 知识库扩展与统计检索(2026-09-20)
- `llmwiki_knowledge_bases.test_questions` / `retrieval_settings` 保存每库测试问题与检索策略;`NULL` 测试问题表示使用 `knowledge_profile.default_test_questions`,显式空数组表示禁用。列表/详情统一通过 `_mapping_view` 返回解析后的配置。
- `deerflow.integrations.weknora.local_index.advanced` 是问答与调试页共享的权威实现。实体扩展只允许一跳、数量有上限,来源为别名/Wiki 链接/Markdown 链接;扩展查询各自运行现有 Wiki-only 混合检索,再进行 RRF 融合。不得把扩展改成无限图遍历,也不得绕过当前用户的 mapping-id 授权。
- 时间区间与计数在后端确定性计算,模型不得自行数检索结果。优先按结构化 `source_refs` 的稳定来源标识去重;没有可识别来源记录时返回 `unit=wiki_page`。日期优先读取来源字段/元数据和 Wiki frontmatter,远端创建/更新时间只是带 `date_source` 的后备。区间统计排除无日期记录并返回 `undated_excluded_count`。
- `POST /api/llmwiki/knowledge-bases/{id}/retrieval-debug` 与 `llmwiki_search` 使用同一个 `advanced_search`。调试响应必须保留 query plan、statistics、references、diagnostics 和 request id,供 `/page/workspace/knowledge/{id}/retrieval-debug` 及浏览器 Network 面板排错。
- 全局与单库 retrieval-debug 页面、入口和 API 均仅管理员可用。前端隐藏不是权限边界;两个 POST 路由必须在参数解析后的任何检索、索引运行时重建或证据读取之前完成管理员校验。
- 回归测试:`tests/test_llmwiki_advanced_retrieval.py`;同时运行 `test_llmwiki_wiki_retrieval.py`、`test_llmwiki_vector_search.py`、`test_llmwiki_local_index_repository.py` 和 `test_llmwiki_weknora.py`。
### 知识库标题权重与引用切片(2026-09-22)
- 检索排序必须把原始用户查询的标题/别名命中作为高置信信号,不能让实体扩展词
冒充原始查询。统一实现位于 `deerflow.integrations.weknora.local_index.ranking`;
普通 Wiki 的向量、关键词、远端 Wiki、原文 chunk 以及助手知识库融合均保留
`title_match_score`,标题信号以有界乘数并入 RRF/相关性分数,不能作为绝对主排序键。
普通 Wiki 的全检索链标题加成由
`llmwiki.local_wiki_index.title_boost_enabled/weight` 控制;关闭或设为 0 必须恢复
原排序。
- 引用详情必须同时展示本次命中的切片和文档/Wiki 正文。`/api/llmwiki/sources/context`
返回 `focus_chunk_id` 及邻近 `chunks`;前端 `WeKnoraSourceDrawer` 使用精确焦点切片
作为详情顶部的「本次检索命中切片」,不能只重复文档级摘要。
- 对话页的后续问题保留生成结果,但只允许在消息列表底部渲染。`MessageList` 上报底部
状态,`ChatPage`、`AgentChatPage` 与分析项目对话通过 `InputBox` 的
`followupsDisplayAllowed` 控制;上滑隐藏,回到底部恢复。
### OpenAI-compatible stream regression coverage
- `PatchedChatOpenAI` must emit partial tool arguments immediately, before a complete JSON document exists.
`test_patched_openai_reasoning_streaming.py` replays actual SDK SSE through a gated local transport and
the Gateway message serializer, covering standard deltas, cumulative snapshots, empty tool headers,
and repeated complete tool names. Per-invocation/index name state must reset on a new call id; registered
complete names permit safe deduplication before arguments start, while legitimate name fragments remain intact.
- Empty `error: {}` is not an upstream error. Empty `delta` beside a `message.reasoning*` extension must not
discard reasoning; a valid delta field takes priority. The existing per-stream summary records only counts,
timing and allowlisted request flags (`incoming_reasoning_chunks`, `emitted_reasoning_chunks`,
`first_reasoning_ms`, `request_enable_thinking`, `repeated_tool_names`), not prompt/answer/argument bytes.
- Model-facing sandbox `write_file` executes once its full arguments are validated. Partial argument display
is a frontend virtual artifact, not incremental persistent writes. Do not confuse local replay with a
production upstream/20-user load test.
- `buffered_gateway` regression coverage must preserve the native external contract: the provider selects the
original metadata-only `write_file`, first confirmed body characters from the second ordinary-text request
appear immediately as canonical incremental `write_file` argument chunks, and LangGraph sees exactly one
original writer call. It must never publish or execute `begin_write_file`. Ordinary answers stay on one
provider request; renderer text never appears as assistant content; reasoning stays in the reasoning channel;
and empty/truncated output cannot reach file I/O. The isolated provider `content` channel is the typed
transaction boundary; do not add another model-authored JSON wrapper or delimiter. The older
`artifact_renderer` handshake retains its existing internal-control invariants. Unconfigured models remain
byte-for-byte on the native path.
### Thread operation timing
`app.gateway.operation_timing.OperationTiming` adds content-free phase logs and `Server-Timing` headers
to thread create/search, including existing-thread idempotent returns and HTTP error paths. Preserve direct
internal Python callers by keeping the injected Response optional. Checkpoint creation remains synchronous
from the endpoint's perspective; do not silently defer it and break immediate state reads. Phase time includes
saver lock/pool waits, not just SQL execution; handler time excludes pre-handler queueing. `run_agent` measures
the synchronous agent factory with `AGENT_BUILD_TIMING` without changing task ContextVars or MCP lifecycle.
Database archive/restore SQL is a separate DBA-operated handoff, not source code or an application migration.
Never wire database archiving into application startup or package it as part of this performance change.
### Uploaded-file context and bounded reads
- `deerflow.uploads.inspection` performs streaming whole-file statistics and
deterministic head/middle/tail sampling. It reads limits from the harness
runtime upload-policy registry and never imports `app.*`.
- `read_file` uses provider-level bounded range reads; both LocalSandbox and
AioSandbox implement them without fetching the complete file. A range-free
large local-file read returns the structured inspection payload.
- `UploadsMiddleware` keeps bounded file metadata/content samples in agent
state and prefixes a JSON-encoded `<uploaded_files>` manifest to the latest
real HumanMessage only in the transient model request. It never writes the
manifest into checkpoint history, and rebuilds legacy state from the upload
directory, so provider retries, tool loops and later follow-ups retain file
evidence. Generated Markdown companions are excluded as duplicate uploads.
- Only convertible Office/PDF originals may resolve Markdown sidecars. Plain
TXT/Markdown/CSV is sampled directly; same-stem user Markdown must never be
mistaken for the companion of a text or image upload.
- Converted companions use collision-free `<original-name>.<ext>.md` names and
extracted Markdown has a fixed character ceiling. Conversion always runs off
the event loop with bounded worker concurrency.
- `grep` scans configured upload sizes with total-byte, time, result and line
bounds. `ContextWindowGuardMiddleware` infers common provider window fields
when available. It never guesses a smaller window for an unprofiled model;
after a real provider context-length rejection it retries with progressively
bounded requests, prioritising the upload manifest and file-reading tools.
### Q&A tool and terminal-stream liveness (2026-09-28)
- `sandbox/skill_view.py` bounds warm-validation followers to 30 seconds. A cold catalogue rebuild may wait up to 120 seconds, and overlapping callers in the same process share that one rebuild (including when the metadata cache is off or the extensions file cannot be read) instead of each taking the process lock. The same thread may re-enter the view lock; it must not wait on itself. Timeout fails closed; never publish a stale view, remove a live leader, or bypass skill availability checks. Existing cache/reuse semantics remain intact.
- LocalSandbox commands close stdin, create an independent process group on POSIX, collect output in temporary files, and terminate the tree on timeout with bounded reaping. `SandboxConfig.command_timeout_seconds` is an optional finite positive number, default 600. Captured output is read with a 1 MiB per-stream memory budget; running commands have a sampled 16 MiB capture guard. AIO lock acquisition is bounded independently of remote request execution.
- Local text reads reject non-regular files after nonblocking open; range readers stop collecting an oversized current line at the output budget. Preserve the existing next-physical-line continuation convention.
- `runtime/runs/worker.py` gives terminal checkpoint inspection/status persistence individual 10-second budgets and final metadata cleanup one shared 10-second budget. Always publish the SSE terminator from the cleanup finally. These budgets must never detach executing graphs or rollback writes. Check cancellation and whether a newer run exists before further title/thread-status work; SQL already issued may not be reversible.
- Frontend terminal recovery checks send epoch/run id, clears the SDK reconnect marker before stopping the local reader, and then applies the checkpoint. Pending submit alone does not identify a new question. Ordinary SDK requests time out in 30 seconds while SSE retains its unlimited total duration.
- Regression and operational limits: [问答卡住排查与修复](../../docs/问答卡住排查与修复-20260928.md). This checkout contains no `config.example.yaml`; old config files remain valid without the new optional field.
### Concurrent skill refresh publication
`sandbox/skill_view.py` coalesces overlapping cold/invalidated refreshes by storage/runtime key and availability-config hash. Followers share one final tree validation, then recheck the current cache identity, config hash and capability marker before returning. They never reuse a completed flight, join a different permission generation, cancel a live leader after their 30-second timeout, or return the old capability after a failed refresh. The optimization does not cache filesystem metadata on a TTL or remove the remaining per-call scan cost. Regression: `packages/harness/tests/test_skill_view_refresh_coalescing.py`.
TodoMiddleware's internal HumanMessages carry `hide_from_ui=true` and `lc_source=todo`. The frontend's shared `isHiddenFromUIMessage` also recognizes the exact historical names `todo_reminder` and `todo_completion_reminder`; question navigation uses the same predicate. Never filter a genuine user's question just because its text quotes `<system_reminder>`. Regression: `test_todo_reminder_visibility.py` and frontend `todo-reminder-visibility.test.ts`.
Measured Windows test scope and remaining limitations are recorded in [本地问答与技能并发验收](../../docs/本地问答与技能并发验收-20260928.md). Frontend compact/agent onboarding mounts TDesign Guide only while a valid step is active: its idle resize listener otherwise dereferences an absent step. Completion/skip persistence and interrupted-tour behavior remain unchanged.
### Force-situation consumer skill authentication (2026-09-30)
The seven consumer-facing custom skill scripts (`fs-collect`, `fs-task-detail-info`,
`fs-timeline-info`, `fs-insight-info`, `fs-current-judgment-info`, `fs-target-info`,
and `fs-connection-info`) reserve line 1 for `# API_URL=...` and line 2 for
`# AUTHORIZATION=Bearer admin`. `monitor_api_common.monitor_authorization_from_second_line`
loads that exact second-line value; an explicit CLI `--authorization` remains the highest-priority
runtime override. Keep certificate verification disabled and proxies bypassed for these intranet
calls. The active sandbox skill mirror must be synchronized with the canonical custom skills.
Gateway detail writes emitted by `fs-collect` and `fs-detail-submit` log a token-masked request and
the corresponding response so an interrupted chain can be distinguished from a failed persistence
call without exposing the gateway bearer token.