Merge remote-tracking branch 'origin/master' into ji
# Conflicts: # .gitignore # frontend-web/src/components/workspace/artifacts/context.tsx # frontend-web/src/components/workspace/messages/message-group.tsx # offline-backend-20260512/backend/CLAUDE.md # offline-backend-20260512/backend/README.md # offline-backend-20260512/backend/app/gateway/app.py # offline-backend-20260512/backend/app/gateway/deps.py # offline-backend-20260512/backend/app/gateway/routers/llmwiki.py
This commit is contained in:
commit
e818c24769
262 files changed
+29527
-5236
No files matched your search
@@ -238,7 +238,7 @@ from deerflow.config import get_app_config
|
||||
|
||||
Lead-agent middlewares are assembled in strict append order across `packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py` (`build_lead_runtime_middlewares`) and `packages/harness/deerflow/agents/lead_agent/agent.py` (`_build_middlewares`):
|
||||
|
||||
1. **ThreadDataMiddleware** - Creates per-thread directories under the user's isolation scope (`backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); resolves `user_id` via `get_effective_user_id()` (falls back to `"default"` in no-auth mode); Web UI thread deletion now follows LangGraph thread removal with Gateway cleanup of the local thread directory
|
||||
1. **ThreadDataMiddleware** - Creates per-thread directories under the creator's stable isolation scope (`backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); resolves persisted threads through `resolve_path_user_id(thread_id)` (falls back to `get_effective_user_id()` / `"default"` when metadata is unavailable); Web UI thread deletion now follows LangGraph thread removal with Gateway cleanup of the local thread directory
|
||||
2. **UploadsMiddleware** - Tracks and injects newly uploaded files into conversation
|
||||
3. **SandboxMiddleware** - Acquires sandbox, stores `sandbox_id` in state
|
||||
4. **DanglingToolCallMiddleware** - Injects placeholder ToolMessages for AIMessage tool_calls that lack responses (e.g., due to user interruption), including raw provider tool-call payloads preserved only in `additional_kwargs["tool_calls"]`
|
||||
@@ -342,10 +342,11 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
|
||||
| **Public Scheduled Tasks** (`/api/public/scheduled-tasks`) | **Unauthenticated** mirror for embedding. Adds `GET /runs/{rid}/html` and `GET /{task_id}/latest-html` for the `html_page` preview; the run result keeps `result.html` (only file paths are redacted). |
|
||||
| **Light Apps** (`/api/light-apps`) | Light-app registry (????? / ????), **global/shared** (no user scoping; `created_by`/`updated_by` audit-only). Two kinds: `window_open` (launched in a new tab from the ???? card grid) and `iframe` (embedded under a parent sidebar menu via `mount_location` and dynamically injected into the sidebar). Both carry `route_params` ? a list of `{key, paramType: login\|theme, source, customValue, themeValue}` descriptors the **frontend** resolves at launch time (login params read from `localStorage.userInfo`; theme params use a literal value) and appends to `url` as query string. `GET /` - list (filters `app_type`/`mount_location`/`enabled_only`/`search`); `GET /{app_id}` - read one (both open to any authenticated user); `POST /`, `PUT /{app_id}`, `DELETE /{app_id}` - **admin only**; iframe apps must carry a `mount_location` (400 otherwise); the unique `name` collides as HTTP 409. Backed by `deerflow.persistence.light_apps` (table `light_apps`, auto-created by `create_all`); store wired in `deps.py` as `app.state.light_app_store`. Tests: `tests/test_light_apps.py`. |
|
||||
| **Menu Overrides** (`/api/menu-overrides`) | Sidebar-menu overrides (菜单管理), **global/shared** (no user scoping; `updated_by` audit-only). The static menu tree (icons/routes/`adminOnly`) lives in the **frontend** code (`core/page-layout/sidebar-menu.ts`); iframe light-apps are injected at render time. This router persists a per-node **overlay** the frontend merges onto that tree: `parent_id` (re-parent, moves the whole subtree; NULL = top level), `sort_order` (reorder among siblings), `label` (rename; NULL keeps code label), `disabled` (停用 — hides the node **and its subtree**). A node with no row renders exactly as code defines it, so new code menus / newly-registered iframe apps appear automatically and are immediately configurable. `GET /` - list all overrides (open to any authenticated user); `PUT /` - **admin only**, atomically **replaces the whole set** (`{overrides: [...]}`); rejects re-parenting that would exceed **3 levels** or form a cycle (`_validate_depth`, HTTP 400). Backed by `deerflow.persistence.menu_overrides` (table `menu_overrides`, auto-created by `create_all`); store wired in `deps.py` as `app.state.menu_override_store`. Frontend: merge logic `core/page-layout/menu-overrides.ts` (`applyMenuOverrides` in `use-sidebar-menu.ts`), client `strategy-components/api/menu-overrides.ts`, admin page `pages/MenuManagementPage.tsx` (@dnd-kit drag-reorder/re-parent + 「移动到」parent selector + ↑↓ arrows + 改名/停用), entry in the bottom-left settings dropdown (`components/page-sidebar.tsx`, admin-only). Tests: `tests/test_menu_overrides.py`, `frontend-web/src/core/page-layout/menu-overrides.test.ts`. |
|
||||
| **Compact Menu Overrides** (`/api/compact-menu-overrides`) | Compact/sidebar-summary left-nav overlay (简介模式菜单), **global/shared**. Two independent sets keyed by `menu_type` (`A` default / `B`); saving one type does not wipe the other. Type B rows use a `B::` node_id prefix so they stay unique on databases that still use `node_id` as the sole PK. `GET ?menu_type=` lists one type (open); `PUT {menu_type, overrides}` admin-only replaces that type. Frontend: `MenuManagementPage` 简介模式 type switcher + `runtime-config.js` `VITE_COMPACT_MENU_TYPE`. Tests: `tests/test_compact_menu_overrides.py`. |
|
||||
| **Sensitive Words** (`/api/sensitive-words`) | Sensitive-word / desensitize map (敏感词管理 / 脱敏词), **global/shared** (no user scoping; `updated_by` audit-only). Each row maps an original `word` → `replacement`, with `enabled` (停用 parks a word without deleting) + `sort_order`. The **frontend** ships a seed map in code (`src/lib/desensitize.ts` `SEED_DESENSITIZE_MAP`) used as the default on a fresh deployment; once an admin saves, this table is the source of truth and the frontend loads it at startup (`DesensitizeWordsLoader` → `replaceDesensitizeMap`, **enabled** rows only). `GET /` - list all words (open to any authenticated user — every page desensitizes); `PUT /` - **admin OR the `lqq` account** (`_require_manager`: `system_role == "admin"` or email `@`-prefix `== "lqq"`), atomically **replaces the whole set** (`{words: [...]}`, de-duped by `word`, last wins). Backed by `deerflow.persistence.sensitive_words` (table `sensitive_words`, PK `word` `String(191)`, auto-created by `create_all`); store wired in `deps.py` as `app.state.sensitive_word_store`. Frontend: client `strategy-components/api/sensitive-words.ts`, page `pages/SensitiveWordsPage.tsx` (route `/page/strategy/admin/sensitive-words`, entry in the bottom-left settings dropdown — **gated on username `lqq`, not admin role**, present in both sidebar modes; 「导入种子词」 merges missing seed entries). Tests: `tests/test_sensitive_words.py`. |
|
||||
| **Dashboard Sessions** (`/api/dashboard-sessions`) | 「大屏绘制智能体」独立页面会话,**按外部 `task_id` 存取,不按 user 分权**(open read + shared writes;`user_id` audit-only)。一个 task 一行(upsert):`transcript`(聊天记录 JSON)+ `report_json`(最新结构化数据,可解析 = 「有效结果」)。`GET /by-task/{task_id}` - 取该 task 会话(无则 404);`PUT /by-task/{task_id}` - upsert(`{transcript?, report_json?}`,只覆盖传入项)。Backed by `deerflow.persistence.dashboard_sessions`(table `dashboard_sessions`, auto-created by `create_all`,无迁移);store wired in `deps.py` as `app.state.dashboard_session_store`。前端独立页 `roundtable-planning/pages/DashboardAgentPage.tsx`(路由 `/page/strategy/dashboard-agent?taskId=`,侧边栏「问答管理 → 大屏绘制」)让智能体分析任务报告 JSON → report-json → 渲染 `DashboardApp`。Tests: `tests/test_dashboard_sessions.py`。 |
|
||||
| **Task Reports** (`/api/task-reports`) | 任务级研判情报报告详情,**按 `task_id` 共享、不按 user 分权**(写入者只留 `updated_by` 审计)。`GET /by-task/{task_id}` 始终返回原 Consumer 兼容外壳 `{state:"200", msg:"操作成功!", data:[...]}`(未保存为 `data: []`);`POST /by-task/{task_id}` 首次保存,已有报告返回 409;`PUT /by-task/{task_id}` 原子替换完整 `data` 数组。`POST /import/by-task/{task_id}` 面向 TaskCOP 导入页,接收 `sq-report-mock.json` 兼容的 `data`/`records` 数组并完整覆盖目标任务;路径中的选定任务 id 优先于文件内的 `taskId`,避免误写。每一项严格是 `{id, taskId, createTime, updateTime, sessionId, contentJson, categoryType}`;`contentJson` 以 JSON **字符串**原样落在 `records_json`,避免破坏大屏按类目解析的输入格式。保存或导入非空 `data` 后,router 会通过 `app.state.taskcop_task_store.mark_analysis_completed` 将同 id 的 TaskCOP 任务升级至状态 `25`,使旧任务列表显示并筛选为“已完成”;普通保存时不存在匹配任务的报告仍可独立保存,导入则要求目标任务存在。Backed by `deerflow.persistence.task_reports`(table `task_reports`, auto-created by `create_all`);store wired in `deps.py` as `app.state.task_report_store`。前端客户端 `strategy-components/api/task-reports.ts`,`core/auth/dashboard-input.ts` 与 rwfx 3q 完成检查均直接读此接口,不再依赖 Consumer 或本地 sq-report mock。Tests: `tests/test_task_reports.py`。 |
|
||||
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, append_auth, auth_token_param, auth_name_param, enabled, sort_order}`. **Two independent, coexisting login-append mechanisms for `url` buttons** — both resolved frontend-side: (1) `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` resolved at click time (`source` = a `userInfo` field or `other`→literal `custom_value`) and appended to the target URL — mirrors the light-app login params; empty = nothing appended. (2) `append_auth` + `auth_token_param` + `auth_name_param` (columns, migration `20260625_02`, default off) — appends `?{auth_token_param}={token1}&{auth_name_param}={username}` so the target system can identify the current user; gated on **token login** — the frontend only stashes `token1` (the `?authToken=`) + the raw `getTokenInfo` username (`login.token1`/`login.username`) when login went through `/login/token`, so a non-token-login session appends neither. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
|
||||
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business/purpose/host), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, append_auth, auth_token_param, auth_name_param, enabled, sort_order}`. **Two independent, coexisting login-append mechanisms for `url` buttons** — both resolved frontend-side: (1) `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` resolved at click time (`source` = a `userInfo` field or `other`→literal `custom_value`) and appended to the target URL — mirrors the light-app login params; empty = nothing appended. (2) `append_auth` + `auth_token_param` + `auth_name_param` (columns, migration `20260625_02`, default off) — appends `?{auth_token_param}={token1}&{auth_name_param}={username}` so the target system can identify the current user; gated on **token login** — the frontend only stashes `token1` (the `?authToken=`) + the raw `getTokenInfo` username (`login.token1`/`login.username`) when login went through `/login/token`, so a non-token-login session appends neither. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
|
||||
| **Fixed Questions** (`/api/fixed-questions`) | Administrator-managed exact-match fixed Q&A (问题配置), **global/shared**. Each row is `{id, question, answer, enabled, tokens_per_second, sort_order}`; stream speed is configurable per row from `1–1000 token/s` and defaults to `100`. `GET /` and whole-set atomic `PUT /` are both admin-only. At the start of an external chat run, `services._find_fixed_answer` checks the latest eligible plain-text human message with exact, case/space/punctuation-sensitive equality. Enabled matches switch the run to `make_fixed_answer_agent`: a local `BaseChatModel` tokenizes with `cl100k_base`, replays the answer in rate-limited token chunks, and keeps the existing LangGraph SSE `messages`/`values` protocol, run journal, checkpoint history, deterministic local title, and refresh behavior intact while no provider/model request or model-concurrency slot is used. File/multimodal/context-prefixed turns, `canvas_agent`, `ai_writing`, writing mode, resume commands, and internal orchestration/background child runs bypass matching. Storage failures fail open to the normal model path. Backed by `deerflow.persistence.fixed_questions` (table `fixed_questions`, SHA-256 indexed lookup plus exact original-string collision check, migrations `20260821_01` + `20260821_02`), wired as `app.state.fixed_question_store`. Frontend: `strategy-components/api/fixed-questions.ts`, admin page `pages/FixedQuestionsPage.tsx`, route `/page/strategy/admin/fixed-questions`, entries in both full-layout and compact-workspace settings menus. Tests: `tests/test_fixed_questions.py`. |
|
||||
| **Knowledge** (`/api/knowledge`) | Product knowledge base, **global/shared** (no user isolation; writes record `created_by`/`updated_by` for audit). `POST /ingest/thread/{thread_id}` - sediment one thread into a note (reads messages from the checkpointer, `threads:read` owner-checked); `POST /ingest/threads/batch` - batch-sediment the caller's recent threads (`skip_existing` via `.manifest.json` dedup). **Defaults to `background=true`**: the slow per-thread extraction loop is deferred to a FastAPI background task and the endpoint returns `{queued:true, processed:N}` immediately (each thread's transcript is truncated to `max_input_chars` before the LLM call). Running the loop inline for a large batch outlives the nginx gateway timeout (504), which the frontend's `parseResponse` surfaces as a generic "Request failed"; `background=false` keeps the blocking path (full per-thread `created/skipped/failed` tally) for small batches/scripts. Shared loop: `_run_batch_ingest`. **Optional model selection**: both ingest endpoints accept `model_name` (overrides `knowledge.extract_model_name` for that capture; threaded through `KnowledgeService.capture_thread` ? `extract_thread_llm_multi`). **Model-error surfacing**: when the chosen model produces nothing usable (timeout / API error / non-JSON) capture still **falls back to rule-based extraction and lands a draft** (`llm_fallback:true`); the single-thread (non-background) response and the batch response (`llm_fallback` count) carry this so the UI tells the user "???????". The "??????" chat button stays background/fire-and-forget (passes `model_name`, no per-save error toast); the batch-import dialog is synchronous and shows the fallback count. `POST /ingest/file` - **import an uploaded file** (multipart: `file` + optional `title`/`tags`/`status`/`model_name`) into wiki note(s): the file is converted to Markdown (md/txt read directly, pdf/word/ppt/excel via `convert_file_to_markdown`), distilled by the same wiki-capture LLM pipeline as thread capture (`source_type="file"`), and falls back to a verbatim **draft** note when the model returns nothing usable (`llm_fallback:true`); `POST /ingest/search` - sediment search/tool results as a reference note; `POST /notes` - manual note; `GET /notes` (keyword/tag/source_type/status + pagination); `GET/PUT/DELETE /notes/{id}` (DELETE is soft = archive, `?hard=true` removes the vault file); `POST /search` - `mode=keyword\|vector\|hybrid` (vector?keyword fallback when no embedding endpoint); `GET /notes/{id}/versions` - edit history; `GET /duplicates` + `POST /merge` - dedup; `POST /reindex` - rebuild vector index; `GET /graph` - product graph data (notes + entities, graph-blacklist filtered); `GET/POST /graph/blacklist` + `DELETE /graph/blacklist/{id}` - graph node blacklist (labels = entity names / note titles, case-insensitive match; blacklisted nodes and their edges are dropped from `/graph` and the wiki-export artifacts; table `knowledge_graph_blacklist`, auto-created); `POST /export/graph` + `GET /export/files/{graph.json\|graph.html\|graph.graphml\|cypher.txt}` - obsidian-wiki `wiki-export` artifacts (self-contained `graph.html` for iframe); `POST /resolve` `{targets:[?]}` + `GET /notes/{id}/wikilinks` - resolve `[[wikilink]]` targets to a note id / vault page / missing (powers clickable wikilinks); `GET /vault-page?path=` - read a raw vault meta/hub Markdown page (no DB note). `GET/POST /folders` + `PUT/DELETE /folders/{id}` - user-managed directories (notes carry a `folder` path; `GET /notes?folder=` filters a dir + sub-dirs); `GET/POST /extract-templates` + `GET/PUT/DELETE /extract-templates/{id}` - editable extraction-prompt templates (thread/document ingest accept `template_id`; capture/PUT-note accept `folder`). Backed by the `deerflow.knowledge` module (see below). |
|
||||
|
||||
@@ -482,6 +483,16 @@ Backs the zzJC / zzXD agent-chat left sidebar (「按类型聚合智能体 + 聊
|
||||
|
||||
### Position-collaboration foundation
|
||||
|
||||
### Roundtable business-chain enable/disable
|
||||
|
||||
`roundtable_chains.enabled` (BOOLEAN NOT NULL DEFAULT 1, migration `20260819_02`)
|
||||
is the per-chain 启停 switch. New and existing chains default to **enabled**.
|
||||
Owner or admin can flip it via `PUT /api/roundtable-chains/{id}` `{enabled}`.
|
||||
Disabled chains stay visible in the management list (with an 启用/停用 filter)
|
||||
and still show `updated_at`; the frontend hides them from the picker / public
|
||||
entry rail so they cannot be selected into a roundtable. Tests:
|
||||
`tests/test_roundtable_chain_stages.py` (`test_create_defaults_enabled_then_toggle`).
|
||||
|
||||
### Roundtable business-chain human validation
|
||||
|
||||
The existing `/api/roundtable-chains` `seats` JSON also accepts the optional
|
||||
@@ -521,6 +532,17 @@ Built-in summary and action-plan completion also preserves a visible report
|
||||
answer when the model misses its final `write_file` call: the router writes
|
||||
that exact answer as the required Markdown artifact in the owning thread's
|
||||
`outputs/` directory, so users receive both chat text and a downloadable file.
|
||||
**Per-turn outputs write cap (弱模型防重复产物)**: ordinary position-node chats
|
||||
and the summary builtin send run-context `position_artifact_cap: 1` (whitelisted
|
||||
in `services.merge_run_context_overrides`). `PositionArtifactCapMiddleware`
|
||||
then keeps at most one successful `/mnt/user-data/outputs/` `write_file` **and**
|
||||
one `present_files` per user turn — truncates excess tool calls in the same
|
||||
model response (and syncs raw provider `additional_kwargs.tool_calls`), rejects
|
||||
further writes/presents after an `OK` / `Successfully presented files` or an
|
||||
earlier sibling in the same batch, ends the turn when only excess calls remain,
|
||||
and reminds the model to present once (or wrap up if already presented).
|
||||
**Action-plan does not send the flag** (it must write report `.md` +
|
||||
`action-plan-subtasks.json`). Tests: `tests/test_position_artifact_cap.py`.
|
||||
|
||||
### 岗位 (Position) Subscription System
|
||||
|
||||
@@ -635,7 +657,11 @@ Operators upgrading an existing DB run the whole file (`USE <db>;` then execute)
|
||||
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`, `deer-flow/skills/`
|
||||
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
|
||||
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
|
||||
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, NOT the current login.** Task-deeplink conversations (rwfx/3qfx/xdfx, `threads_meta.metadata.taskId`) are shared across all users (see the `taskId` escape branches in `thread_meta/sql.py`), but generated files/uploads live under a *per-user* dir. Resolving by current login meant account B opening a shared task conversation 404'd on the md/artifacts account A generated ("切换账号后生成的文件找不到"). `deerflow/runtime/thread_paths.py` (`resolve_path_user_id` sync / `aresolve_path_user_id` async) resolves a task-shared thread to its **creator** (`threads_meta.user_id`), so every account reads/writes the same physical directory; personal threads resolve to the current login unchanged (no behavior change). Wired at every thread-path seam: write source `ThreadDataMiddleware.before_agent` (drives all sandbox writes via `thread_data` paths), `present_file_tool`, `uploads/manager.get_uploads_dir` + `uploads_middleware` + uploads router, artifact serving (`path_utils.{a,}resolve_thread_virtual_path` → artifacts/skills routers), embedded `client.get_artifact`. Positive (taskId→creator) results are cached process-wide (immutable mapping); any failure / memory-mode store / missing row falls back to the current login. **Caveat**: files a *non-creator* wrote into a shared thread before this fix stay orphaned under that writer's dir (no data loss); all new writes consolidate under the creator. **Extended to 圆桌会商 system threads** (`metadata.thread_type == "roundtable"` — Step 2 seats/coordinator + Step 3 report/summary/dashboard/structure, created by `multi_agent.py` with `_ROUNDTABLE_THREAD_METADATA`): their Step2/Step3 sandbox artifacts (md/流程图/报告) must stay viewable when a历史草稿 is reopened or a taskId会商 draft is opened by another account, so `thread_paths._is_creator_shared_thread(metadata)` treats both `taskId` **and** `thread_type=="roundtable"` as creator-shared, and `ThreadMetaRepository.check_access` (the artifacts router's `owner_check`) bypasses the owner filter for both — roundtable thread ids are only surfaced inside the owner's draft, so this does not let unrelated accounts *discover* others' 会商 files. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.
|
||||
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, using the persisted creator for every tracked thread.** The route's regular owner check still authorizes access first; creator resolution only selects the stable physical directory (`threads_meta.user_id`) and avoids a lost request context falling back to `users/default`. This also preserves cross-account task-deeplink conversations (rwfx/3qfx/xdfx) and roundtable system threads. Positive creator mappings are cached process-wide and DB/memory/missing-row failures fall back to the current login. It is wired at the thread-path consumers including `ThreadDataMiddleware.before_agent`, `present_file_tool`, uploads, artifact serving, and embedded `client.get_artifact`. **Rewrite-history recovery is metadata-only and job-owner scoped** (`list_by_session(..., user_id=current_user)`), so its GET/list endpoints intentionally skip redundant thread-owner checks and return an empty history instead of a history-only 404; the start/commit endpoints retain strict thread write authorization. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.
|
||||
|
||||
- **Durable document-rewrite commit:** `document_rewrite_pipeline.atomic_write_text` writes through a deliberately short sibling temporary filename so long Windows artifact paths stay below the path-length limit, and guarantees the target `outputs` parent exists. If the sandbox recreates that directory in the narrow commit window, it repairs the directory and retries; the original file is otherwise only changed through `os.replace`.
|
||||
|
||||
- **Deep Research structural regeneration:** `POST /api/deep-research/sessions/{id}/report-variants` creates a distinct completed session from the source report and its selected evidence. It rebases both saved `[来源:source_id]` and legacy streamed `[[source:source_id]]` markers to cloned ids, and removes inherited markers with no selected clone. The frontend uses this endpoint only for “选择结构再生成”, then queues the durable writer with `operation=generate`; generation receives the confirmed outline and complete selected evidence set but never receives the inherited report as text to edit. The new report's completed card can subsequently start a separate in-place `AI 全文改写`. Requirement plans are deterministic and operation-specific, model reasoning is forwarded as live SSE before the first Markdown token, and generation prompts emit display-ready `[来源:id]` citations. Citation validation repairs only a unique near-complete `drs_src_*` prefix (provider-truncated suffix) and still rejects genuinely unknown ids. The report-only stream requests 8192 output tokens (except Codex Responses models, which reject token-limit kwargs) and rejects a malformed trailing citation instead of committing a provider-truncated report. Never restore the former second planning-model call unless its latency and streaming contract are redesigned together.
|
||||
|
||||
**Sandbox Tools** (in `packages/harness/deerflow/sandbox/tools.py`):
|
||||
- `bash` - Execute commands with path translation and error handling
|
||||
@@ -970,8 +996,6 @@ Per-user runtime overrides (whitelisted fields) live at `{base_dir}/users/{user_
|
||||
|
||||
Both endpoints are whitelisted in `auth_middleware._PUBLIC_EXACT_PATHS`. The shared `_get_or_create_user_by_username()` helper (in `routers/auth.py`) backs both `/login/username` and `/login/token`.
|
||||
|
||||
**Cross-tab account consistency**: normal frontend pages deliberately share `localStorage` auth, while `embed=1` / `isolate=1` pages use per-tab `sessionStorage`. Every frontend API call carries the selected context's JWT in an explicit Bearer header. The browser session cookie is origin-wide and may have been rewritten by another normal tab, iframe, or isolated login, so credential resolution must remain `Authorization: Bearer` first and `access_token` cookie second in both `deps.get_request_access_token()` and `langgraph_auth.authenticate()` (shared parser: `app/gateway/auth/token_source.py`). Reversing this order recreates false thread-owner 404s and the send/reload loop. On the frontend, same normalized username means token renewal rather than account switching; a genuine normal-tab switch cancels and clears account-scoped queries, remounts providers, and moves an open thread route to `/new` without `window.location.reload()`. Embedded/isolated pages ignore cross-tab storage changes. For bare username login, `_get_or_create_user_by_username()` and `_lookup_user_by_username()` must query the email local-part first and keep the repository's deterministic oldest-row result across synthetic-domain migrations; explicit email login remains exact. Regression coverage: `tests/test_health_auth_bypass.py`, `tests/test_username_login_domain.py`.
|
||||
|
||||
- `log_system` - Admin-only external log-system entry (`deerflow/config/log_system_config.py`): `{enabled, url, label}`. Surfaced read-only by `GET /api/system-settings/log-system` (admin only); the frontend renders it as a "????" item in the bottom-right settings menu (`workspace-nav-menu.tsx`) that opens `url` in a new tab. Disabled by default.
|
||||
|
||||
**`extensions_config.json`**:
|
||||
@@ -986,6 +1010,10 @@ Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` me
|
||||
|
||||
The embedded detail proxy hides WeKnora's outer `.main > .aside_box` application sidebar and expands the route outlet, but deliberately keeps the Wiki page's internal directory/index sidebar.
|
||||
|
||||
**Embedded native knowledge-base switching:** The original WeKnora breadcrumb picker remains available inside the detail iframe. A signed 15-minute embed cookie carries only the current DeerFlow user's visible mappings (their own plus published bases); the proxied `/api/v1/knowledge-bases` response is filtered to that allowlist. Choosing another entry performs a full iframe navigation, rechecks the DeerFlow mapping and read/write permission, then reissues the cookie for the selected base. Never expose the upstream service-admin list directly or permit an arbitrary remote knowledge-base ID.
|
||||
|
||||
The independent enterprise-research workbench is **temporarily disabled**. Its source remains under `app/gateway/routers/enterprise_research.py` and `deerflow.persistence.enterprise_research`, but Gateway does not mount `/api/enterprise-research`, initialize its stores/executor, or register its ORM rows in `Base.metadata`. This ensures startup does not create or alter `enterprise_research_tasks` / `enterprise_research_report_jobs` on MySQL. It remains separate from `/api/deep-research`; do not re-enable it incidentally while changing the active Deep Research workflow. Regression coverage for its preserved implementation remains in `tests/test_enterprise_research_tasks.py`.
|
||||
|
||||
### Embedded Client (`packages/harness/deerflow/client.py`)
|
||||
|
||||
`DeerFlowClient` provides direct in-process access to all DeerFlow capabilities without HTTP services. All return types align with the Gateway API response schemas, so consumer code works identically in HTTP and embedded modes.
|
||||
@@ -1116,6 +1144,15 @@ The router re-orders rows deterministically and only permits an enabled custom
|
||||
theme as `default_theme`; removing or disabling the default safely falls back
|
||||
to `light`. Regression coverage: `tests/test_appearance_theme.py`.
|
||||
|
||||
Compact top-bar shortcuts live in the same `appearance` setting. Their route
|
||||
parameters support `login`, `theme`, and keyless `path`. A `path` parameter has
|
||||
no query-key input; the operator selects its value source from the right-hand
|
||||
dropdown and the frontend appends the resolved, URL-encoded value to the
|
||||
external URL path (or Hash route) in configured order. Password stays an
|
||||
ordinary `login` query parameter with the key `password`, so an external login
|
||||
link can become `#/login/<user>?password=…`. Regression coverage:
|
||||
`tests/test_appearance_theme.py`.
|
||||
|
||||
### File Upload
|
||||
|
||||
Multi-file upload with automatic document conversion:
|
||||
@@ -1150,6 +1187,10 @@ Automatic conversation summarization when approaching token limits:
|
||||
the config flag. `config.summarization.enabled` now defaults to **false**.
|
||||
- `config.yaml` `summarization` still supplies the **tuning**: trigger types
|
||||
(tokens, messages, or fraction of max input) and the keep policy.
|
||||
- A compressed request must retain the latest non-empty `HumanMessage`, even
|
||||
when it has already been folded into the hidden summary. This keeps strict
|
||||
OpenAI-compatible gateways from rejecting a tool-continuation call with
|
||||
`No user query found in message`.
|
||||
- Keeps recent messages while summarizing older ones.
|
||||
|
||||
Tests: `tests/test_summarization_toggle.py` (gating + whitelist),
|
||||
@@ -1268,8 +1309,16 @@ the terminal `lastJobId` in `usage_snapshot` so a refreshed conversation page
|
||||
can replay its trace from the event log. Detailed mode uses stable targets (`introduction`,
|
||||
`section:N`, `conclusion`) because section writers are concurrent;
|
||||
multi-agent revisions must emit `report_reset` before their next report
|
||||
version. Do not emit raw model reasoning or one persisted event per provider
|
||||
token. The Vite page owns Word export through
|
||||
version. Do not persist raw model reasoning or one persisted event per provider
|
||||
token. Models that expose thinking inside ordinary `<think>...</think>` text
|
||||
must be split statefully before a report delta, checkpoint, or Markdown
|
||||
projection is created; an optional live-only `report_thinking` frame can drive
|
||||
the message-list thought trace, while the sandbox receives answer Markdown
|
||||
only. The post-report digest (`summarize_report`) must likewise forward
|
||||
provider reasoning through live-only `summary_thinking` frames so the
|
||||
chat timeline can render thought while waiting for the first `summary_delta`.
|
||||
The Vite
|
||||
page owns Word export through
|
||||
`frontend-web/src/core/scheduled-tasks/markdown-docx.ts`; do not add Python
|
||||
document-conversion dependencies merely to export text.
|
||||
All non-streaming Deep Research control-plane calls (`complete()` for query /
|
||||
@@ -1287,6 +1336,31 @@ The per-user Deep Research concurrency guard counts only session statuses
|
||||
consume a slot; otherwise saved drafts make `POST /sessions/{id}/jobs` return
|
||||
429 permanently. Regression: `test_saved_draft_sessions_do_not_consume_research_concurrency`.
|
||||
|
||||
`POST /api/deep-research/sessions/{id}/jobs` is retry-safe under MySQL
|
||||
read/write splitting. Deep Research session/job/event/source/message writes
|
||||
must serialise their changed row while the write transaction is still open
|
||||
(`flush` for inserts, select-after-update **before** commit), never through a
|
||||
post-commit `session.refresh()` or a fresh pooled read. Replica lag otherwise
|
||||
turns a committed job into a misleading 500, can make dispatcher `claim_job()`
|
||||
lose its snapshot, or can fail report/follow-up persistence after the model has
|
||||
already completed. Always nudge the dispatcher for both a newly created job and
|
||||
an idempotently reused queued job; otherwise a retry after such an interrupted
|
||||
response returns the old job but leaves it waiting for the periodic scan. The
|
||||
SSE replay generator must catch temporary event/job-store reads, emit a
|
||||
recoverable warning, and keep the connection open so the next poll can repair
|
||||
the cursor instead of surfacing an ASGI `ExceptionGroup`. Regressions:
|
||||
`test_deep_research_writes_never_depend_on_post_commit_refresh`,
|
||||
`test_job_create_never_reads_back_after_commit`,
|
||||
`test_retry_of_existing_queued_job_nudges_dispatcher`, and
|
||||
`test_stream_stays_open_when_durable_replay_has_a_transient_failure`.
|
||||
|
||||
At ordinary report completion, persist and confirm the session's
|
||||
`report_markdown` projection **before** marking the Job `completed`. Retry a
|
||||
short transient projection failure; if it cannot be confirmed, let the job
|
||||
become failed rather than reporting a report that vanishes on refresh.
|
||||
`test_report_is_persisted_before_job_is_marked_completed` and
|
||||
`test_report_projection_failure_never_marks_job_completed` cover the ordering.
|
||||
|
||||
The chat-collection frontend must keep report-writing progress in the existing
|
||||
workspace `MessageList` / `MessageGroup` tool-step renderer: inject only
|
||||
virtual `deep_research_progress` tool calls for the durable phases, keep the
|
||||
@@ -1312,6 +1386,22 @@ TDesign `Select` backed by `/api/models`; its explicit choice is sent as
|
||||
while collection or report writing is active so one durable run cannot switch
|
||||
models mid-stream.
|
||||
|
||||
Report structures (`/api/report-structures`) may store an optional
|
||||
`retrieval_directions` list (检索方向) and an optional `retrieval_skills`
|
||||
list (检索来源 / 指定技能). Empty / unset keeps today's collector
|
||||
query-planning, default tools, and runner sub-query generation. A non-empty
|
||||
directions list is frozen into `DeepResearchConfig.retrieval_directions`: the
|
||||
chat collector's first user message gets a `<retrieval-directions>` block that
|
||||
asks it to **split concrete retrievable terms along those facets** (never search
|
||||
the direction labels themselves), and legacy `basic`/`detailed`/`multi_agent`/
|
||||
`deep` runners pass the same constraint into query-planning instead of using
|
||||
`课题 + 方向` as the search string. A non-empty skills list is frozen
|
||||
into `DeepResearchConfig.retrieval_skills`: the chat collector's first user
|
||||
message gets a `<retrieval-skills>` block, the collector run context receives
|
||||
`extra_skills` so `skill_view`/`skill_list` can see those names, and legacy
|
||||
material collection runs those skills instead of the default Q&A research-skill
|
||||
trio. Helpers live in `deerflow.agents.deep_research.retrieval`.
|
||||
|
||||
Completed-report Q&A has two endpoints: retain `POST /sessions/{id}/chat` as the
|
||||
legacy JSON response for compatibility, and use `POST /sessions/{id}/chat/stream`
|
||||
for report-grounded Q&A SSE. The workbench must send a strong scoped edit
|
||||
@@ -1366,11 +1456,10 @@ source drawer, which renders the complete Markdown page. When the local index
|
||||
is disabled, the drawer reads the remote WeKnora Wiki-page endpoint directly
|
||||
and must not surface the local-index-disabled response. A Wiki target must not
|
||||
also load or render the legacy original-document preview, even when its citation
|
||||
still carries a document id. The drawer supports
|
||||
single-page Markdown and Word downloads, and its **Open in Wiki** link carries
|
||||
both the DeerFlow knowledge-base mapping id and the exact `wikiSlug`. Wiki
|
||||
search-result cards use this same drawer instead of navigating immediately.
|
||||
Exact-page navigation uses the native
|
||||
still carries a document id. The drawer supports single-page Markdown and Word
|
||||
downloads, and its **Open in Wiki** link carries both the DeerFlow knowledge-base
|
||||
mapping id and the exact `wikiSlug`. Wiki search-result cards use this same
|
||||
drawer instead of navigating immediately. Exact-page navigation uses the native
|
||||
`knowledge/{mapping_id}/wiki/view?wikiSlug=...` reader, not the WeKnora detail
|
||||
iframe. Keep legacy `knowledge/{mapping_id}?tab=wiki&wikiSlug=...` links fast by
|
||||
redirecting them in `LlmWikiKnowledgePage` before its runtime-loading gate.
|
||||
@@ -1398,6 +1487,26 @@ external-enabled + index-enabled on every request and response, and hard-deny
|
||||
the `system` / `对话沉淀` mapping. Never return remote WeKnora ids, provenance
|
||||
refs, credentials, or vector bytes externally. Regression tests:
|
||||
`tests/test_llmwiki_local_index_repository.py`,
|
||||
`tests/test_llmwiki_wiki_sync.py`, `tests/test_llmwiki_vector_search.py`, and
|
||||
`tests/test_llmwiki_wiki_sync.py`, `tests/test_llmwiki_vector_search.py`,
|
||||
`tests/test_external_llmwiki_search.py`, and
|
||||
`tests/test_llmwiki_wiki_retrieval.py`.
|
||||
|
||||
The sandbox whole-report rewrite may carry an optional `reportOutline` Markdown
|
||||
template from the shared report-structure library. Persist the original template
|
||||
in the durable input snapshot and append it as an explicit chapter-structure
|
||||
constraint before requirements planning and prose generation; a template must
|
||||
never be used as permission to invent facts absent from the current report.
|
||||
|
||||
`DocumentRewriteVersionRepository.create` backs the reversible before-image for
|
||||
both artifact and whole-report rewrites. It must flush and serialize the new row
|
||||
on the writer before commit, and must never call `session.refresh()` after
|
||||
commit: a read/write-splitting MySQL endpoint can route that refresh to a
|
||||
lagging replica, falsely report a version-save failure, and trigger the caller's
|
||||
intentional restore of the rewritten document.
|
||||
|
||||
Version snapshots are an undo enhancement, not a condition for preserving an
|
||||
already committed rewrite. If snapshot creation fails after the artifact/report
|
||||
write, log the exception, complete the rewrite with
|
||||
`versionSnapshotFailed: true`, and do not emit a user-facing failure or restore
|
||||
the original. The sandbox uses that signal only to enable its ordinary manual
|
||||
save fallback.
|
||||
Reference in new issue
Block a user