Merge remote-tracking branch 'origin/master' into ji

# Conflicts:
#	.gitignore
#	frontend-web/src/components/workspace/artifacts/context.tsx
#	frontend-web/src/components/workspace/messages/message-group.tsx
#	offline-backend-20260512/backend/CLAUDE.md
#	offline-backend-20260512/backend/README.md
#	offline-backend-20260512/backend/app/gateway/app.py
#	offline-backend-20260512/backend/app/gateway/deps.py
#	offline-backend-20260512/backend/app/gateway/routers/llmwiki.py
This commit is contained in:
吉鑫升 committed 2026-08-24 14:21:19 +08:00
commit e818c24769
262 files changed
+29527 -5236

No files matched your search

+122 -13
View File
@@ -238,7 +238,7 @@ from deerflow.config import get_app_config
Lead-agent middlewares are assembled in strict append order across `packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py` (`build_lead_runtime_middlewares`) and `packages/harness/deerflow/agents/lead_agent/agent.py` (`_build_middlewares`):
1. **ThreadDataMiddleware** - Creates per-thread directories under the user's isolation scope (`backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); resolves `user_id` via `get_effective_user_id()` (falls back to `"default"` in no-auth mode); Web UI thread deletion now follows LangGraph thread removal with Gateway cleanup of the local thread directory
1. **ThreadDataMiddleware** - Creates per-thread directories under the creator's stable isolation scope (`backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); resolves persisted threads through `resolve_path_user_id(thread_id)` (falls back to `get_effective_user_id()` / `"default"` when metadata is unavailable); Web UI thread deletion now follows LangGraph thread removal with Gateway cleanup of the local thread directory
2. **UploadsMiddleware** - Tracks and injects newly uploaded files into conversation
3. **SandboxMiddleware** - Acquires sandbox, stores `sandbox_id` in state
4. **DanglingToolCallMiddleware** - Injects placeholder ToolMessages for AIMessage tool_calls that lack responses (e.g., due to user interruption), including raw provider tool-call payloads preserved only in `additional_kwargs["tool_calls"]`
@@ -342,10 +342,11 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
| **Public Scheduled Tasks** (`/api/public/scheduled-tasks`) | **Unauthenticated** mirror for embedding. Adds `GET /runs/{rid}/html` and `GET /{task_id}/latest-html` for the `html_page` preview; the run result keeps `result.html` (only file paths are redacted). |
| **Light Apps** (`/api/light-apps`) | Light-app registry (????? / ????), **global/shared** (no user scoping; `created_by`/`updated_by` audit-only). Two kinds: `window_open` (launched in a new tab from the ???? card grid) and `iframe` (embedded under a parent sidebar menu via `mount_location` and dynamically injected into the sidebar). Both carry `route_params` ? a list of `{key, paramType: login\|theme, source, customValue, themeValue}` descriptors the **frontend** resolves at launch time (login params read from `localStorage.userInfo`; theme params use a literal value) and appends to `url` as query string. `GET /` - list (filters `app_type`/`mount_location`/`enabled_only`/`search`); `GET /{app_id}` - read one (both open to any authenticated user); `POST /`, `PUT /{app_id}`, `DELETE /{app_id}` - **admin only**; iframe apps must carry a `mount_location` (400 otherwise); the unique `name` collides as HTTP 409. Backed by `deerflow.persistence.light_apps` (table `light_apps`, auto-created by `create_all`); store wired in `deps.py` as `app.state.light_app_store`. Tests: `tests/test_light_apps.py`. |
| **Menu Overrides** (`/api/menu-overrides`) | Sidebar-menu overrides (菜单管理), **global/shared** (no user scoping; `updated_by` audit-only). The static menu tree (icons/routes/`adminOnly`) lives in the **frontend** code (`core/page-layout/sidebar-menu.ts`); iframe light-apps are injected at render time. This router persists a per-node **overlay** the frontend merges onto that tree: `parent_id` (re-parent, moves the whole subtree; NULL = top level), `sort_order` (reorder among siblings), `label` (rename; NULL keeps code label), `disabled` (停用 — hides the node **and its subtree**). A node with no row renders exactly as code defines it, so new code menus / newly-registered iframe apps appear automatically and are immediately configurable. `GET /` - list all overrides (open to any authenticated user); `PUT /` - **admin only**, atomically **replaces the whole set** (`{overrides: [...]}`); rejects re-parenting that would exceed **3 levels** or form a cycle (`_validate_depth`, HTTP 400). Backed by `deerflow.persistence.menu_overrides` (table `menu_overrides`, auto-created by `create_all`); store wired in `deps.py` as `app.state.menu_override_store`. Frontend: merge logic `core/page-layout/menu-overrides.ts` (`applyMenuOverrides` in `use-sidebar-menu.ts`), client `strategy-components/api/menu-overrides.ts`, admin page `pages/MenuManagementPage.tsx` (@dnd-kit drag-reorder/re-parent + 「移动到」parent selector + ↑↓ arrows + 改名/停用), entry in the bottom-left settings dropdown (`components/page-sidebar.tsx`, admin-only). Tests: `tests/test_menu_overrides.py`, `frontend-web/src/core/page-layout/menu-overrides.test.ts`. |
| **Compact Menu Overrides** (`/api/compact-menu-overrides`) | Compact/sidebar-summary left-nav overlay (简介模式菜单), **global/shared**. Two independent sets keyed by `menu_type` (`A` default / `B`); saving one type does not wipe the other. Type B rows use a `B::` node_id prefix so they stay unique on databases that still use `node_id` as the sole PK. `GET ?menu_type=` lists one type (open); `PUT {menu_type, overrides}` admin-only replaces that type. Frontend: `MenuManagementPage` 简介模式 type switcher + `runtime-config.js` `VITE_COMPACT_MENU_TYPE`. Tests: `tests/test_compact_menu_overrides.py`. |
| **Sensitive Words** (`/api/sensitive-words`) | Sensitive-word / desensitize map (敏感词管理 / 脱敏词), **global/shared** (no user scoping; `updated_by` audit-only). Each row maps an original `word` → `replacement`, with `enabled` (停用 parks a word without deleting) + `sort_order`. The **frontend** ships a seed map in code (`src/lib/desensitize.ts` `SEED_DESENSITIZE_MAP`) used as the default on a fresh deployment; once an admin saves, this table is the source of truth and the frontend loads it at startup (`DesensitizeWordsLoader` → `replaceDesensitizeMap`, **enabled** rows only). `GET /` - list all words (open to any authenticated user — every page desensitizes); `PUT /` - **admin OR the `lqq` account** (`_require_manager`: `system_role == "admin"` or email `@`-prefix `== "lqq"`), atomically **replaces the whole set** (`{words: [...]}`, de-duped by `word`, last wins). Backed by `deerflow.persistence.sensitive_words` (table `sensitive_words`, PK `word` `String(191)`, auto-created by `create_all`); store wired in `deps.py` as `app.state.sensitive_word_store`. Frontend: client `strategy-components/api/sensitive-words.ts`, page `pages/SensitiveWordsPage.tsx` (route `/page/strategy/admin/sensitive-words`, entry in the bottom-left settings dropdown — **gated on username `lqq`, not admin role**, present in both sidebar modes; 「导入种子词」 merges missing seed entries). Tests: `tests/test_sensitive_words.py`. |
| **Dashboard Sessions** (`/api/dashboard-sessions`) | 「大屏绘制智能体」独立页面会话,**按外部 `task_id` 存取,不按 user 分权**(open read + shared writes;`user_id` audit-only)。一个 task 一行(upsert):`transcript`(聊天记录 JSON)+ `report_json`(最新结构化数据,可解析 = 「有效结果」)。`GET /by-task/{task_id}` - 取该 task 会话(无则 404);`PUT /by-task/{task_id}` - upsert(`{transcript?, report_json?}`,只覆盖传入项)。Backed by `deerflow.persistence.dashboard_sessions`(table `dashboard_sessions`, auto-created by `create_all`,无迁移);store wired in `deps.py` as `app.state.dashboard_session_store`。前端独立页 `roundtable-planning/pages/DashboardAgentPage.tsx`(路由 `/page/strategy/dashboard-agent?taskId=`,侧边栏「问答管理 → 大屏绘制」)让智能体分析任务报告 JSON → report-json → 渲染 `DashboardApp`。Tests: `tests/test_dashboard_sessions.py`。 |
| **Task Reports** (`/api/task-reports`) | 任务级研判情报报告详情,**按 `task_id` 共享、不按 user 分权**(写入者只留 `updated_by` 审计)。`GET /by-task/{task_id}` 始终返回原 Consumer 兼容外壳 `{state:"200", msg:"操作成功!", data:[...]}`(未保存为 `data: []`);`POST /by-task/{task_id}` 首次保存,已有报告返回 409;`PUT /by-task/{task_id}` 原子替换完整 `data` 数组。`POST /import/by-task/{task_id}` 面向 TaskCOP 导入页,接收 `sq-report-mock.json` 兼容的 `data`/`records` 数组并完整覆盖目标任务;路径中的选定任务 id 优先于文件内的 `taskId`,避免误写。每一项严格是 `{id, taskId, createTime, updateTime, sessionId, contentJson, categoryType}`;`contentJson` 以 JSON **字符串**原样落在 `records_json`,避免破坏大屏按类目解析的输入格式。保存或导入非空 `data` 后,router 会通过 `app.state.taskcop_task_store.mark_analysis_completed` 将同 id 的 TaskCOP 任务升级至状态 `25`,使旧任务列表显示并筛选为“已完成”;普通保存时不存在匹配任务的报告仍可独立保存,导入则要求目标任务存在。Backed by `deerflow.persistence.task_reports`(table `task_reports`, auto-created by `create_all`);store wired in `deps.py` as `app.state.task_report_store`。前端客户端 `strategy-components/api/task-reports.ts`,`core/auth/dashboard-input.ts` 与 rwfx 3q 完成检查均直接读此接口,不再依赖 Consumer 或本地 sq-report mock。Tests: `tests/test_task_reports.py`。 |
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, append_auth, auth_token_param, auth_name_param, enabled, sort_order}`. **Two independent, coexisting login-append mechanisms for `url` buttons** — both resolved frontend-side: (1) `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` resolved at click time (`source` = a `userInfo` field or `other`→literal `custom_value`) and appended to the target URL — mirrors the light-app login params; empty = nothing appended. (2) `append_auth` + `auth_token_param` + `auth_name_param` (columns, migration `20260625_02`, default off) — appends `?{auth_token_param}={token1}&{auth_name_param}={username}` so the target system can identify the current user; gated on **token login** — the frontend only stashes `token1` (the `?authToken=`) + the raw `getTokenInfo` username (`login.token1`/`login.username`) when login went through `/login/token`, so a non-token-login session appends neither. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business/purpose/host), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, append_auth, auth_token_param, auth_name_param, enabled, sort_order}`. **Two independent, coexisting login-append mechanisms for `url` buttons** — both resolved frontend-side: (1) `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` resolved at click time (`source` = a `userInfo` field or `other`→literal `custom_value`) and appended to the target URL — mirrors the light-app login params; empty = nothing appended. (2) `append_auth` + `auth_token_param` + `auth_name_param` (columns, migration `20260625_02`, default off) — appends `?{auth_token_param}={token1}&{auth_name_param}={username}` so the target system can identify the current user; gated on **token login** — the frontend only stashes `token1` (the `?authToken=`) + the raw `getTokenInfo` username (`login.token1`/`login.username`) when login went through `/login/token`, so a non-token-login session appends neither. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
| **Fixed Questions** (`/api/fixed-questions`) | Administrator-managed exact-match fixed Q&A (问题配置), **global/shared**. Each row is `{id, question, answer, enabled, tokens_per_second, sort_order}`; stream speed is configurable per row from `1–1000 token/s` and defaults to `100`. `GET /` and whole-set atomic `PUT /` are both admin-only. At the start of an external chat run, `services._find_fixed_answer` checks the latest eligible plain-text human message with exact, case/space/punctuation-sensitive equality. Enabled matches switch the run to `make_fixed_answer_agent`: a local `BaseChatModel` tokenizes with `cl100k_base`, replays the answer in rate-limited token chunks, and keeps the existing LangGraph SSE `messages`/`values` protocol, run journal, checkpoint history, deterministic local title, and refresh behavior intact while no provider/model request or model-concurrency slot is used. File/multimodal/context-prefixed turns, `canvas_agent`, `ai_writing`, writing mode, resume commands, and internal orchestration/background child runs bypass matching. Storage failures fail open to the normal model path. Backed by `deerflow.persistence.fixed_questions` (table `fixed_questions`, SHA-256 indexed lookup plus exact original-string collision check, migrations `20260821_01` + `20260821_02`), wired as `app.state.fixed_question_store`. Frontend: `strategy-components/api/fixed-questions.ts`, admin page `pages/FixedQuestionsPage.tsx`, route `/page/strategy/admin/fixed-questions`, entries in both full-layout and compact-workspace settings menus. Tests: `tests/test_fixed_questions.py`. |
| **Knowledge** (`/api/knowledge`) | Product knowledge base, **global/shared** (no user isolation; writes record `created_by`/`updated_by` for audit). `POST /ingest/thread/{thread_id}` - sediment one thread into a note (reads messages from the checkpointer, `threads:read` owner-checked); `POST /ingest/threads/batch` - batch-sediment the caller's recent threads (`skip_existing` via `.manifest.json` dedup). **Defaults to `background=true`**: the slow per-thread extraction loop is deferred to a FastAPI background task and the endpoint returns `{queued:true, processed:N}` immediately (each thread's transcript is truncated to `max_input_chars` before the LLM call). Running the loop inline for a large batch outlives the nginx gateway timeout (504), which the frontend's `parseResponse` surfaces as a generic "Request failed"; `background=false` keeps the blocking path (full per-thread `created/skipped/failed` tally) for small batches/scripts. Shared loop: `_run_batch_ingest`. **Optional model selection**: both ingest endpoints accept `model_name` (overrides `knowledge.extract_model_name` for that capture; threaded through `KnowledgeService.capture_thread` ? `extract_thread_llm_multi`). **Model-error surfacing**: when the chosen model produces nothing usable (timeout / API error / non-JSON) capture still **falls back to rule-based extraction and lands a draft** (`llm_fallback:true`); the single-thread (non-background) response and the batch response (`llm_fallback` count) carry this so the UI tells the user "???????". The "??????" chat button stays background/fire-and-forget (passes `model_name`, no per-save error toast); the batch-import dialog is synchronous and shows the fallback count. `POST /ingest/file` - **import an uploaded file** (multipart: `file` + optional `title`/`tags`/`status`/`model_name`) into wiki note(s): the file is converted to Markdown (md/txt read directly, pdf/word/ppt/excel via `convert_file_to_markdown`), distilled by the same wiki-capture LLM pipeline as thread capture (`source_type="file"`), and falls back to a verbatim **draft** note when the model returns nothing usable (`llm_fallback:true`); `POST /ingest/search` - sediment search/tool results as a reference note; `POST /notes` - manual note; `GET /notes` (keyword/tag/source_type/status + pagination); `GET/PUT/DELETE /notes/{id}` (DELETE is soft = archive, `?hard=true` removes the vault file); `POST /search` - `mode=keyword\|vector\|hybrid` (vector?keyword fallback when no embedding endpoint); `GET /notes/{id}/versions` - edit history; `GET /duplicates` + `POST /merge` - dedup; `POST /reindex` - rebuild vector index; `GET /graph` - product graph data (notes + entities, graph-blacklist filtered); `GET/POST /graph/blacklist` + `DELETE /graph/blacklist/{id}` - graph node blacklist (labels = entity names / note titles, case-insensitive match; blacklisted nodes and their edges are dropped from `/graph` and the wiki-export artifacts; table `knowledge_graph_blacklist`, auto-created); `POST /export/graph` + `GET /export/files/{graph.json\|graph.html\|graph.graphml\|cypher.txt}` - obsidian-wiki `wiki-export` artifacts (self-contained `graph.html` for iframe); `POST /resolve` `{targets:[?]}` + `GET /notes/{id}/wikilinks` - resolve `[[wikilink]]` targets to a note id / vault page / missing (powers clickable wikilinks); `GET /vault-page?path=` - read a raw vault meta/hub Markdown page (no DB note). `GET/POST /folders` + `PUT/DELETE /folders/{id}` - user-managed directories (notes carry a `folder` path; `GET /notes?folder=` filters a dir + sub-dirs); `GET/POST /extract-templates` + `GET/PUT/DELETE /extract-templates/{id}` - editable extraction-prompt templates (thread/document ingest accept `template_id`; capture/PUT-note accept `folder`). Backed by the `deerflow.knowledge` module (see below). |
@@ -482,6 +483,16 @@ Backs the zzJC / zzXD agent-chat left sidebar (「按类型聚合智能体 + 聊
### Position-collaboration foundation
### Roundtable business-chain enable/disable
`roundtable_chains.enabled` (BOOLEAN NOT NULL DEFAULT 1, migration `20260819_02`)
is the per-chain 启停 switch. New and existing chains default to **enabled**.
Owner or admin can flip it via `PUT /api/roundtable-chains/{id}` `{enabled}`.
Disabled chains stay visible in the management list (with an 启用/停用 filter)
and still show `updated_at`; the frontend hides them from the picker / public
entry rail so they cannot be selected into a roundtable. Tests:
`tests/test_roundtable_chain_stages.py` (`test_create_defaults_enabled_then_toggle`).
### Roundtable business-chain human validation
The existing `/api/roundtable-chains` `seats` JSON also accepts the optional
@@ -521,6 +532,17 @@ Built-in summary and action-plan completion also preserves a visible report
answer when the model misses its final `write_file` call: the router writes
that exact answer as the required Markdown artifact in the owning thread's
`outputs/` directory, so users receive both chat text and a downloadable file.
**Per-turn outputs write cap (弱模型防重复产物)**: ordinary position-node chats
and the summary builtin send run-context `position_artifact_cap: 1` (whitelisted
in `services.merge_run_context_overrides`). `PositionArtifactCapMiddleware`
then keeps at most one successful `/mnt/user-data/outputs/` `write_file` **and**
one `present_files` per user turn — truncates excess tool calls in the same
model response (and syncs raw provider `additional_kwargs.tool_calls`), rejects
further writes/presents after an `OK` / `Successfully presented files` or an
earlier sibling in the same batch, ends the turn when only excess calls remain,
and reminds the model to present once (or wrap up if already presented).
**Action-plan does not send the flag** (it must write report `.md` +
`action-plan-subtasks.json`). Tests: `tests/test_position_artifact_cap.py`.
### 岗位 (Position) Subscription System
@@ -635,7 +657,11 @@ Operators upgrading an existing DB run the whole file (`USE <db>;` then execute)
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`, `deer-flow/skills/`
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, NOT the current login.** Task-deeplink conversations (rwfx/3qfx/xdfx, `threads_meta.metadata.taskId`) are shared across all users (see the `taskId` escape branches in `thread_meta/sql.py`), but generated files/uploads live under a *per-user* dir. Resolving by current login meant account B opening a shared task conversation 404'd on the md/artifacts account A generated ("切换账号后生成的文件找不到"). `deerflow/runtime/thread_paths.py` (`resolve_path_user_id` sync / `aresolve_path_user_id` async) resolves a task-shared thread to its **creator** (`threads_meta.user_id`), so every account reads/writes the same physical directory; personal threads resolve to the current login unchanged (no behavior change). Wired at every thread-path seam: write source `ThreadDataMiddleware.before_agent` (drives all sandbox writes via `thread_data` paths), `present_file_tool`, `uploads/manager.get_uploads_dir` + `uploads_middleware` + uploads router, artifact serving (`path_utils.{a,}resolve_thread_virtual_path` → artifacts/skills routers), embedded `client.get_artifact`. Positive (taskId→creator) results are cached process-wide (immutable mapping); any failure / memory-mode store / missing row falls back to the current login. **Caveat**: files a *non-creator* wrote into a shared thread before this fix stay orphaned under that writer's dir (no data loss); all new writes consolidate under the creator. **Extended to 圆桌会商 system threads** (`metadata.thread_type == "roundtable"` — Step 2 seats/coordinator + Step 3 report/summary/dashboard/structure, created by `multi_agent.py` with `_ROUNDTABLE_THREAD_METADATA`): their Step2/Step3 sandbox artifacts (md/流程图/报告) must stay viewable when a历史草稿 is reopened or a taskId会商 draft is opened by another account, so `thread_paths._is_creator_shared_thread(metadata)` treats both `taskId` **and** `thread_type=="roundtable"` as creator-shared, and `ThreadMetaRepository.check_access` (the artifacts router's `owner_check`) bypasses the owner filter for both — roundtable thread ids are only surfaced inside the owner's draft, so this does not let unrelated accounts *discover* others' 会商 files. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, using the persisted creator for every tracked thread.** The route's regular owner check still authorizes access first; creator resolution only selects the stable physical directory (`threads_meta.user_id`) and avoids a lost request context falling back to `users/default`. This also preserves cross-account task-deeplink conversations (rwfx/3qfx/xdfx) and roundtable system threads. Positive creator mappings are cached process-wide and DB/memory/missing-row failures fall back to the current login. It is wired at the thread-path consumers including `ThreadDataMiddleware.before_agent`, `present_file_tool`, uploads, artifact serving, and embedded `client.get_artifact`. **Rewrite-history recovery is metadata-only and job-owner scoped** (`list_by_session(..., user_id=current_user)`), so its GET/list endpoints intentionally skip redundant thread-owner checks and return an empty history instead of a history-only 404; the start/commit endpoints retain strict thread write authorization. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.
- **Durable document-rewrite commit:** `document_rewrite_pipeline.atomic_write_text` writes through a deliberately short sibling temporary filename so long Windows artifact paths stay below the path-length limit, and guarantees the target `outputs` parent exists. If the sandbox recreates that directory in the narrow commit window, it repairs the directory and retries; the original file is otherwise only changed through `os.replace`.
- **Deep Research structural regeneration:** `POST /api/deep-research/sessions/{id}/report-variants` creates a distinct completed session from the source report and its selected evidence. It rebases both saved `[来源:source_id]` and legacy streamed `[[source:source_id]]` markers to cloned ids, and removes inherited markers with no selected clone. The frontend uses this endpoint only for “选择结构再生成”, then queues the durable writer with `operation=generate`; generation receives the confirmed outline and complete selected evidence set but never receives the inherited report as text to edit. The new report's completed card can subsequently start a separate in-place `AI 全文改写`. Requirement plans are deterministic and operation-specific, model reasoning is forwarded as live SSE before the first Markdown token, and generation prompts emit display-ready `[来源:id]` citations. Citation validation repairs only a unique near-complete `drs_src_*` prefix (provider-truncated suffix) and still rejects genuinely unknown ids. The report-only stream requests 8192 output tokens (except Codex Responses models, which reject token-limit kwargs) and rejects a malformed trailing citation instead of committing a provider-truncated report. Never restore the former second planning-model call unless its latency and streaming contract are redesigned together.
**Sandbox Tools** (in `packages/harness/deerflow/sandbox/tools.py`):
- `bash` - Execute commands with path translation and error handling
@@ -970,8 +996,6 @@ Per-user runtime overrides (whitelisted fields) live at `{base_dir}/users/{user_
Both endpoints are whitelisted in `auth_middleware._PUBLIC_EXACT_PATHS`. The shared `_get_or_create_user_by_username()` helper (in `routers/auth.py`) backs both `/login/username` and `/login/token`.
**Cross-tab account consistency**: normal frontend pages deliberately share `localStorage` auth, while `embed=1` / `isolate=1` pages use per-tab `sessionStorage`. Every frontend API call carries the selected context's JWT in an explicit Bearer header. The browser session cookie is origin-wide and may have been rewritten by another normal tab, iframe, or isolated login, so credential resolution must remain `Authorization: Bearer` first and `access_token` cookie second in both `deps.get_request_access_token()` and `langgraph_auth.authenticate()` (shared parser: `app/gateway/auth/token_source.py`). Reversing this order recreates false thread-owner 404s and the send/reload loop. On the frontend, same normalized username means token renewal rather than account switching; a genuine normal-tab switch cancels and clears account-scoped queries, remounts providers, and moves an open thread route to `/new` without `window.location.reload()`. Embedded/isolated pages ignore cross-tab storage changes. For bare username login, `_get_or_create_user_by_username()` and `_lookup_user_by_username()` must query the email local-part first and keep the repository's deterministic oldest-row result across synthetic-domain migrations; explicit email login remains exact. Regression coverage: `tests/test_health_auth_bypass.py`, `tests/test_username_login_domain.py`.
- `log_system` - Admin-only external log-system entry (`deerflow/config/log_system_config.py`): `{enabled, url, label}`. Surfaced read-only by `GET /api/system-settings/log-system` (admin only); the frontend renders it as a "????" item in the bottom-right settings menu (`workspace-nav-menu.tsx`) that opens `url` in a new tab. Disabled by default.
**`extensions_config.json`**:
@@ -986,6 +1010,10 @@ Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` me
The embedded detail proxy hides WeKnora's outer `.main > .aside_box` application sidebar and expands the route outlet, but deliberately keeps the Wiki page's internal directory/index sidebar.
**Embedded native knowledge-base switching:** The original WeKnora breadcrumb picker remains available inside the detail iframe. A signed 15-minute embed cookie carries only the current DeerFlow user's visible mappings (their own plus published bases); the proxied `/api/v1/knowledge-bases` response is filtered to that allowlist. Choosing another entry performs a full iframe navigation, rechecks the DeerFlow mapping and read/write permission, then reissues the cookie for the selected base. Never expose the upstream service-admin list directly or permit an arbitrary remote knowledge-base ID.
The independent enterprise-research workbench is **temporarily disabled**. Its source remains under `app/gateway/routers/enterprise_research.py` and `deerflow.persistence.enterprise_research`, but Gateway does not mount `/api/enterprise-research`, initialize its stores/executor, or register its ORM rows in `Base.metadata`. This ensures startup does not create or alter `enterprise_research_tasks` / `enterprise_research_report_jobs` on MySQL. It remains separate from `/api/deep-research`; do not re-enable it incidentally while changing the active Deep Research workflow. Regression coverage for its preserved implementation remains in `tests/test_enterprise_research_tasks.py`.
### Embedded Client (`packages/harness/deerflow/client.py`)
`DeerFlowClient` provides direct in-process access to all DeerFlow capabilities without HTTP services. All return types align with the Gateway API response schemas, so consumer code works identically in HTTP and embedded modes.
@@ -1116,6 +1144,15 @@ The router re-orders rows deterministically and only permits an enabled custom
theme as `default_theme`; removing or disabling the default safely falls back
to `light`. Regression coverage: `tests/test_appearance_theme.py`.
Compact top-bar shortcuts live in the same `appearance` setting. Their route
parameters support `login`, `theme`, and keyless `path`. A `path` parameter has
no query-key input; the operator selects its value source from the right-hand
dropdown and the frontend appends the resolved, URL-encoded value to the
external URL path (or Hash route) in configured order. Password stays an
ordinary `login` query parameter with the key `password`, so an external login
link can become `#/login/<user>?password=…`. Regression coverage:
`tests/test_appearance_theme.py`.
### File Upload
Multi-file upload with automatic document conversion:
@@ -1150,6 +1187,10 @@ Automatic conversation summarization when approaching token limits:
the config flag. `config.summarization.enabled` now defaults to **false**.
- `config.yaml` `summarization` still supplies the **tuning**: trigger types
(tokens, messages, or fraction of max input) and the keep policy.
- A compressed request must retain the latest non-empty `HumanMessage`, even
when it has already been folded into the hidden summary. This keeps strict
OpenAI-compatible gateways from rejecting a tool-continuation call with
`No user query found in message`.
- Keeps recent messages while summarizing older ones.
Tests: `tests/test_summarization_toggle.py` (gating + whitelist),
@@ -1268,8 +1309,16 @@ the terminal `lastJobId` in `usage_snapshot` so a refreshed conversation page
can replay its trace from the event log. Detailed mode uses stable targets (`introduction`,
`section:N`, `conclusion`) because section writers are concurrent;
multi-agent revisions must emit `report_reset` before their next report
version. Do not emit raw model reasoning or one persisted event per provider
token. The Vite page owns Word export through
version. Do not persist raw model reasoning or one persisted event per provider
token. Models that expose thinking inside ordinary `<think>...</think>` text
must be split statefully before a report delta, checkpoint, or Markdown
projection is created; an optional live-only `report_thinking` frame can drive
the message-list thought trace, while the sandbox receives answer Markdown
only. The post-report digest (`summarize_report`) must likewise forward
provider reasoning through live-only `summary_thinking` frames so the
chat timeline can render thought while waiting for the first `summary_delta`.
The Vite
page owns Word export through
`frontend-web/src/core/scheduled-tasks/markdown-docx.ts`; do not add Python
document-conversion dependencies merely to export text.
All non-streaming Deep Research control-plane calls (`complete()` for query /
@@ -1287,6 +1336,31 @@ The per-user Deep Research concurrency guard counts only session statuses
consume a slot; otherwise saved drafts make `POST /sessions/{id}/jobs` return
429 permanently. Regression: `test_saved_draft_sessions_do_not_consume_research_concurrency`.
`POST /api/deep-research/sessions/{id}/jobs` is retry-safe under MySQL
read/write splitting. Deep Research session/job/event/source/message writes
must serialise their changed row while the write transaction is still open
(`flush` for inserts, select-after-update **before** commit), never through a
post-commit `session.refresh()` or a fresh pooled read. Replica lag otherwise
turns a committed job into a misleading 500, can make dispatcher `claim_job()`
lose its snapshot, or can fail report/follow-up persistence after the model has
already completed. Always nudge the dispatcher for both a newly created job and
an idempotently reused queued job; otherwise a retry after such an interrupted
response returns the old job but leaves it waiting for the periodic scan. The
SSE replay generator must catch temporary event/job-store reads, emit a
recoverable warning, and keep the connection open so the next poll can repair
the cursor instead of surfacing an ASGI `ExceptionGroup`. Regressions:
`test_deep_research_writes_never_depend_on_post_commit_refresh`,
`test_job_create_never_reads_back_after_commit`,
`test_retry_of_existing_queued_job_nudges_dispatcher`, and
`test_stream_stays_open_when_durable_replay_has_a_transient_failure`.
At ordinary report completion, persist and confirm the session's
`report_markdown` projection **before** marking the Job `completed`. Retry a
short transient projection failure; if it cannot be confirmed, let the job
become failed rather than reporting a report that vanishes on refresh.
`test_report_is_persisted_before_job_is_marked_completed` and
`test_report_projection_failure_never_marks_job_completed` cover the ordering.
The chat-collection frontend must keep report-writing progress in the existing
workspace `MessageList` / `MessageGroup` tool-step renderer: inject only
virtual `deep_research_progress` tool calls for the durable phases, keep the
@@ -1312,6 +1386,22 @@ TDesign `Select` backed by `/api/models`; its explicit choice is sent as
while collection or report writing is active so one durable run cannot switch
models mid-stream.
Report structures (`/api/report-structures`) may store an optional
`retrieval_directions` list (检索方向) and an optional `retrieval_skills`
list (检索来源 / 指定技能). Empty / unset keeps today's collector
query-planning, default tools, and runner sub-query generation. A non-empty
directions list is frozen into `DeepResearchConfig.retrieval_directions`: the
chat collector's first user message gets a `<retrieval-directions>` block that
asks it to **split concrete retrievable terms along those facets** (never search
the direction labels themselves), and legacy `basic`/`detailed`/`multi_agent`/
`deep` runners pass the same constraint into query-planning instead of using
`课题 + 方向` as the search string. A non-empty skills list is frozen
into `DeepResearchConfig.retrieval_skills`: the chat collector's first user
message gets a `<retrieval-skills>` block, the collector run context receives
`extra_skills` so `skill_view`/`skill_list` can see those names, and legacy
material collection runs those skills instead of the default Q&A research-skill
trio. Helpers live in `deerflow.agents.deep_research.retrieval`.
Completed-report Q&A has two endpoints: retain `POST /sessions/{id}/chat` as the
legacy JSON response for compatibility, and use `POST /sessions/{id}/chat/stream`
for report-grounded Q&A SSE. The workbench must send a strong scoped edit
@@ -1366,11 +1456,10 @@ source drawer, which renders the complete Markdown page. When the local index
is disabled, the drawer reads the remote WeKnora Wiki-page endpoint directly
and must not surface the local-index-disabled response. A Wiki target must not
also load or render the legacy original-document preview, even when its citation
still carries a document id. The drawer supports
single-page Markdown and Word downloads, and its **Open in Wiki** link carries
both the DeerFlow knowledge-base mapping id and the exact `wikiSlug`. Wiki
search-result cards use this same drawer instead of navigating immediately.
Exact-page navigation uses the native
still carries a document id. The drawer supports single-page Markdown and Word
downloads, and its **Open in Wiki** link carries both the DeerFlow knowledge-base
mapping id and the exact `wikiSlug`. Wiki search-result cards use this same
drawer instead of navigating immediately. Exact-page navigation uses the native
`knowledge/{mapping_id}/wiki/view?wikiSlug=...` reader, not the WeKnora detail
iframe. Keep legacy `knowledge/{mapping_id}?tab=wiki&wikiSlug=...` links fast by
redirecting them in `LlmWikiKnowledgePage` before its runtime-loading gate.
@@ -1398,6 +1487,26 @@ external-enabled + index-enabled on every request and response, and hard-deny
the `system` / `对话沉淀` mapping. Never return remote WeKnora ids, provenance
refs, credentials, or vector bytes externally. Regression tests:
`tests/test_llmwiki_local_index_repository.py`,
`tests/test_llmwiki_wiki_sync.py`, `tests/test_llmwiki_vector_search.py`, and
`tests/test_llmwiki_wiki_sync.py`, `tests/test_llmwiki_vector_search.py`,
`tests/test_external_llmwiki_search.py`, and
`tests/test_llmwiki_wiki_retrieval.py`.
The sandbox whole-report rewrite may carry an optional `reportOutline` Markdown
template from the shared report-structure library. Persist the original template
in the durable input snapshot and append it as an explicit chapter-structure
constraint before requirements planning and prose generation; a template must
never be used as permission to invent facts absent from the current report.
`DocumentRewriteVersionRepository.create` backs the reversible before-image for
both artifact and whole-report rewrites. It must flush and serialize the new row
on the writer before commit, and must never call `session.refresh()` after
commit: a read/write-splitting MySQL endpoint can route that refresh to a
lagging replica, falsely report a version-save failure, and trigger the caller's
intentional restore of the rewritten document.
Version snapshots are an undo enhancement, not a condition for preserving an
already committed rewrite. If snapshot creation fails after the artifact/report
write, log the exception, complete the rewrite with
`versionSnapshotFailed: true`, and do not emit a user-facing failure or restore
the original. The sandbox uses that signal only to enable its ordinary manual
save fallback.