sync update from LAN repo (source master-yx @ cf505220)

This commit is contained in:
gozeros committed 2026-10-21 15:11:11 +08:00
1 parent e243ec3a46
commit 19d3cb96dc
114 files changed
+14787 -451

No files matched your search

+48 -2
View File
@@ -299,6 +299,8 @@ from deerflow.config import get_app_config
**Forced-research built-in agent.** `forced-research-responder` (「强制检索输出助手」) is seeded from `app/gateway/routers/_forced_research_seed_assets/` before `_sync_legacy_agents`. `ForcedResearchMiddleware` is mounted only for this stable id. On every visible user turn it requires a successful information-gathering tool result before the model may finish; `tool_choice=required` is backed by an `after_model -> model` retry for gateways that ignore tool choice, with an eight-attempt failure cap that permits only an honest failure response. Skill discovery, reading `SKILL.md`, and writing a helper script do not satisfy the gate; web/knowledge/MCP results, non-skill uploaded-file reads, or successful Python skill-script execution do. Tool filtering removes `present_files`, `skill_manage`, task/schedule/memory/clarification surfaces. The middleware permits `write_file`/`str_replace` only for `/mnt/user-data/workspace/*.py` and permits `bash` only for `.py` execution without redirection/document paths, so skills can run scripts but the agent cannot create Markdown/document artifacts or write `outputs`. Tests: `tests/test_forced_research_agent.py`.
**Strategy-selection functional agents.** `app/gateway/routers/_strategy_selection_seed.py` seeds nine stable folder-backed agents from `_strategy_selection_seed_assets/` before `_sync_legacy_agents`: intent, governance/diplomacy/economy/security/public-opinion/law-policy angles, judge, and solution editor. Seeding is missing-directory-only and must never overwrite administrator changes. The global `strategy_agent_roles` registry classifies them independently of projects: intent/judge/solution are fixed, unique roles that may be configured but never added, disabled, reordered, or deleted; angle roles are a shared, ordered catalog that administrators may add, edit, disable (soft delete), restore, and reorder through `/api/strategy-agents`. Public `GET /api/strategy-agents` returns enabled identity/description metadata only; `include_disabled=true` or `include_config=true` is admin-only. Admin writes are `POST /api/strategy-agents/angles`, `PUT /api/strategy-agents/{id}`, and `DELETE /api/strategy-agents/{id}`; DELETE retains the folder-backed agent and role row to preserve historical references. Dynamic angles remain built-in folder-backed agents under the reserved `strategy-selection-angle-` id namespace, with `web` + `file:read`; a blank SOUL on create receives a safe role template. All strategy skill writes validate that names exist, are active, and are globally visible; generic agent CRUD cannot create or hard-delete reserved strategy IDs, and parallel execution rejects disabled roles. Angle configs reuse `policy-knowledge`; intent reuses `intent-clarification-policy`; the judge has no retrieval skill; the solution editor gets only `file:write`. Keep the SOUL contracts neutral and evidence-aware, with explicit de-escalation/civilian-protection framing and hard bans on operational combat guidance and targeted political manipulation. Tests: `tests/test_strategy_selection_seed.py`, `tests/test_strategy_agent_management.py`.
**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
- Extends `AgentState` with: `sandbox`, `thread_data`, `title`, `artifacts`, `todos`, `uploaded_files`, `viewed_images`
- Uses custom reducers: `merge_artifacts` (deduplicate), `merge_viewed_images` (merge/clear)
@@ -540,6 +542,8 @@ Self-contained product knowledge base. **All knowledge capability lives in this
Custom agent metadata lives in the `agents` ORM table (`deerflow.persistence.agents`). `agents.id` is the stable runtime id and the filesystem directory name under `.deer-flow/agents/{id}/`; `agents.name` is display-only, can contain Chinese characters, and is not unique. `user_id IS NULL` marks built-in agents, user-owned agents are visible only to their owner unless `published = true`, and update/delete operations require ownership. The runtime accepts `agent_id` in context/configurable, and non-default `assistant_id` values are treated as agent ids after visibility checks.
Strategy-selection fan-out (`routers/parallel_agents.py`) must keep hidden reasoning separate per agent: provider reasoning blocks/fields and inline `<think>...</think>` text emit `reasoning_delta`, while visible answer text emits `delta` and only visible text may enter `agent_done.content`. Never concatenate `reasoning_delta` into a selectable strategy or generated deliverable. Inspect the final AI message with the shared `is_llm_error_message` detector and require substantive visible text before constructing a success result. A middleware fallback, empty/thinking-only response, or punctuation-only provider degeneration is a failed seat run: raise so the stream emits `agent_error`; never wrap it in `agent_done`. Regression coverage: `tests/test_roundtable_model_fallback.py`.
**Listing cache + pagination** ? to keep `GET /api/agents` fast, the gallery's per-card extras (skills + tool_groups) are denormalized into two cache tables instead of reading every agent's `config.yaml` on each request: `agent_skills` (relational, one row per (agent, skill) with `position`) and `agent_extras` (one row per agent: `tool_groups` JSON + `model` + `skills_synced_at`). `config.yaml` remains authoritative for the agent runtime; these tables are a read cache, write-through on create/update and backfilled from disk by `_sync_legacy_agents` (mtime-vs-`skills_synced_at` drift check). The list endpoint enriches via `AgentStore.get_extras_for([ids])` (two batched `IN` queries). Both tables are auto-created by `Base.metadata.create_all` (no migration needed). `GET /api/agents` supports **opt-in server-side pagination**: `scope` (`mine` = owned + favorited / `square` = built-in/published public), `page`, `page_size`, returning `{agents, total, page, page_size}`; omitting `scope`/`page` preserves the legacy "return everything" behavior. Name `search` matches the raw OR desensitized name (the frontend's static term?abbrev map is mirrored server-side in `deerflow/config/agent_name_desensitize.py`); `tag_ids` is an SQL OR-filter; `owner` keeps only agents published by that user id (the card's ??? ? built-in agents never match), powering the gallery's click-???-to-filter. `POST /api/agents/private-square/list` takes the same pagination params (incl. `owner`). Repository entry point: `AgentStore.list_paginated(...)`. Tests: `tests/test_agent_listing_cache.py::test_list_paginated_owner_filter_keeps_one_publisher`.
**Homepage selector (agent picker on the new-chat page)** ? single admin-curated list backed by `agents.featured_order` (nullable BIGINT). NULL = not featured. Only `published = true` agents can be featured (router enforces on writes); any published agent is eligible regardless of ownership. Cleared with `order: null`. The list rendered to end users is `WHERE featured_order IS NOT NULL` + visibility-filter, sorted by `featured_order ASC`. Admins reorder by sending the desired ids to `PUT /api/agents/featured/order` (assigns indices 0..N-1). BIGINT is used rather than INT so the management UI can seed initial values from `Date.now()` (overflows INT on MySQL).
@@ -766,6 +770,7 @@ MySQL index script synchronized.
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`, `deer-flow/skills/`
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
- Configured mount portability: `AppConfig._resolve_relative_mount_paths` resolves each relative `sandbox.mounts[*].host_path` against the loaded `config.yaml` directory before validation/provider construction, never against the process cwd. Keep repository-owned knowledge mounts relative (for example `knowledge/knowledge_base`); absolute paths remain unchanged, and sandbox providers always receive an absolute path.
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, using the persisted creator for every tracked thread.** The route's regular owner check still authorizes access first; creator resolution only selects the stable physical directory (`threads_meta.user_id`) and avoids a lost request context falling back to `users/default`. This also preserves cross-account task-deeplink conversations (rwfx/3qfx/xdfx) and roundtable system threads. Positive creator mappings are cached process-wide and DB/memory/missing-row failures fall back to the current login. It is wired at the thread-path consumers including `ThreadDataMiddleware.before_agent`, `present_file_tool`, uploads, artifact serving, and embedded `client.get_artifact`. **Rewrite-history recovery is metadata-only and job-owner scoped** (`list_by_session(..., user_id=current_user)`), so its GET/list endpoints intentionally skip redundant thread-owner checks and return an empty history instead of a history-only 404; the start/commit endpoints retain strict thread write authorization. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.
- **Durable document-rewrite commit:** `document_rewrite_pipeline.atomic_write_text` writes through a deliberately short sibling temporary filename so long Windows artifact paths stay below the path-length limit, and guarantees the target `outputs` parent exists. If the sandbox recreates that directory in the narrow commit window, it repairs the directory and retries; the original file is otherwise only changed through `os.replace`.
@@ -998,6 +1003,15 @@ scheduler runs once per deployment. Per run:
- Supports `supports_vision` flag for image understanding models
- Config values starting with `$` resolved as environment variables
- Missing provider modules surface actionable install hints from reflection resolvers (for example `uv add langchain-google-genai`)
- Generic `langchain_openai:ChatOpenAI` entries are transparently wrapped by
`PatchedChatOpenAI`. Besides preserving reasoning/tool-call metadata, the
patch recognizes compatible gateways that encode failures inside HTTP 200
payloads (`base_resp.status_code != 0` with null/empty `choices`) in both
streaming and non-streaming paths. It raises the stable
`OpenAICompatibleUpstreamError`; `LLMErrorHandlingMiddleware` retries
transient concurrency/inference failures, while genuine quota/balance
failures are classified as non-retriable and surfaced immediately. Embedded
vendor failures therefore cannot become empty successful assistant turns.
### Roundtable Model Fault Tolerance (会商模型自动容错 / 换下一个模型)
@@ -1128,6 +1142,15 @@ Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` me
### Conditional WeKnora LLMWiki
**Embedding ownership:** WeKnora's per-knowledge-base `embedding_model_id`
indexes uploaded source documents; `llmwiki.local_wiki_index.embedding` indexes
the generated Wiki-page mirror inside DeerFlow. Do not couple these two model
choices. `GET /api/llmwiki/embedding-models` returns a strict credential-free
projection of WeKnora Embedding models, and KB creation must validate and pass
the selected active model id. When the caller omits a choice, retain WeKnora's
active-default behavior. Existing KBs cannot change this binding in place, and
cross-KB WeKnora searches should use model-compatible knowledge bases.
Ordinary knowledge-base cards expose an owner/admin-only name/description edit dialog in `WeKnoraKnowledgeWorkspace.tsx`, reusing `useLlmWikiActions().update` and `PATCH /api/llmwiki/knowledge-bases/{id}`. The route trims names, rejects blank or reserved conversation-deposit names, writes to WeKnora before updating local metadata, and retains existing write authorization. A successful remote list is authoritative for every list scope, including `personal`: absent remote IDs are hidden without deleting mappings/references. Upstream failures remain errors. The grid also filters cached `remote_status=missing` rows and no longer renders stale-record cleanup cards. Regression coverage: `tests/test_llmwiki_knowledge_management.py`.
`llmwiki.weknora.api_base_url` is the provider switch: non-empty enables WeKnora, empty preserves the legacy LLMWiki UI. WeKnora is independently deployed and owns PostgreSQL/Redis/object storage/vector storage/parsers/models; DeerFlow config contains only `api_base_url` and optional `web_base_url`. DeerFlow deliberately sends no WeKnora credential, so the WeKnora endpoint must be restricted to a trusted network and allow DeerFlow traffic itself or through its reverse proxy. There is intentionally no `default_embedding_model_id`: `WeKnoraClient` queries `/api/v1/models` and uses WeKnora's active Embedding model when creating a KB; Wiki creation additionally resolves an active KnowledgeQA model inside WeKnora for synthesis. The native BFF is `app/gateway/routers/llmwiki.py`; ownership/publication mappings live in `deerflow.persistence.llmwiki`; the browser only sees DeerFlow mapping IDs. Knowledge-base owners publish immediately (`private` -> `published`) without a review queue, and can unpublish ordinary bases. The conversation-deposit KB is system-owned, always published, shared by all DeerFlow users, excluded from My Knowledge, and immutable through user/admin detail operations; recognition requires both the reserved name and `system` owner. Existing same-name remote mappings are adopted in place as the system deposit, preserving mapping ids and document links. Its user-facing description, generated Markdown source and tags use `cmzs` branding; list/detail reads synchronize stale remote descriptions, and search results, source previews and embedded-detail JSON recursively replace legacy brand text in historical conversation-deposit payloads. Deposit requests must reference a thread accessible to the current account. Personal knowledge-base list/search views are scoped to the current DeerFlow account even for admins; frontend LLMWiki caches include the live account id, and admin write power does not make other users' private KBs appear as owned. Only `system_role === "admin"` sees the WeKnora connection-config tab. Knowledge cards navigate to a dedicated WeKnora-style detail route. Wiki-enabled bases render and default to `Wiki -> documents -> graph`; standard bases omit Wiki and render `documents -> graph`. The detail page supports sequential multi-file upload with an aggregate result for writable ordinary bases, hides it for public read-only/system-deposit bases, and uses a generic loading label while the embed session starts. It also supports URL and published manual-Markdown imports, browses real Wiki pages/content, proxies authenticated original-file previews, pages through `/chunks/{knowledge_id}`, and renders the Wiki page graph from `/knowledgebase/{kb_id}/wiki/graph`; document details use a right sheet rather than the removed KB modal. The credential-free proxy uses one WeKnora service identity, so per-user separation is enforced by DeerFlow mappings while all remote KBs physically share that identity's WeKnora workspace. WeKnora iframe detail pages first call `/api/llmwiki/knowledge-bases/{id}/weknora/session` through the normal DeerFlow API auth path, then load `/platform/knowledge-bases/{weknora_id}` with a signed `deerflow_weknora_embed` cookie for `/platform`, `/assets`, `/locales`, root assets, and API calls. Injected fetch/XHR rewriting sends all iframe `/api/v1/*`, including colliding `/api/v1/auth/*`, through `/api/llmwiki/weknora-embed/api/v1/*`; Gateway auth/CSRF bypass only applies to those signed iframe proxy requests, the proxy still revalidates the mapped KB, and cookie security honors `X-Forwarded-Proto` for HTTPS reverse proxies. `llmwiki_search` is conditionally registered in `tools.py`, is read-only, and rechecks the current DeerFlow user, agent `llmwiki_knowledge_base_ids`, and thread-context selection on every call. Source previews prefer the cleaned Wiki/LLMWiki page text when a retrieved chunk maps to a Wiki page, with raw split chunks as the fallback. Agent bindings use an inline WeKnora-style searchable/grouped multi-select in the standard agent create/detail UI; new chats use the same interaction pattern as a compact composer dropdown, and selections are written to thread metadata/context and restored on reopen. The DeerFlow frontend is native React and the WeKnora frontend is not modified or embedded. Full deployment and behavior notes: `docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md`. Tests: `tests/test_llmwiki_weknora.py`.
@@ -1324,6 +1347,14 @@ Automatic conversation summarization when approaching token limits:
when it has already been folded into the hidden summary. This keeps strict
OpenAI-compatible gateways from rejecting a tool-continuation call with
`No user query found in message`.
- LangChain keeps the agent prompt in `ModelRequest.system_message`, separate
from `ModelRequest.messages`. Compression must merge its hidden summary into
that dedicated slot and remove every `SystemMessage` from the ordinary
message list. Strict Qwen-style adapters accept only one system message at
index zero and otherwise return HTTP 400 (`system message must be at the
beginning`). Keep this normalization adapter-independent; the payload-level
OpenAI-compatible normalizer is a second line of defence and also merges
adjacent leading system messages.
- Keeps recent messages while summarizing older ones.
Tests: `tests/test_summarization_toggle.py` (gating + whitelist),
@@ -1803,8 +1834,8 @@ Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_
pages need backfill, not a forced rebuild of every page.
FastEmbed is a standard backend dependency. Encoder configuration, directory
upload and warmup live only in the administrator global settings dialog.
A runtime `system_settings.json` path override wins over config.yaml; uploaded
files are streamed into `.deer-flow/models/bge-m3`, never the source tree.
When the effective provider is local, its `system_settings.json` path override wins over config.yaml; uploaded
files are streamed into immutable `.deer-flow/models/bge-m3-revisions/<revision>` directories, never the source tree.
Encoder warmup is explicitly started from the settings UI/API and must never block application startup
or turn a model-load failure into a DeerFlow service failure. Local provider
is offline-only: require local_model_path or DEERFLOW_BGE_M3_MODEL_PATH and
@@ -1812,6 +1843,21 @@ Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_
`local_process_isolation=true` by default: inference runs in the bundled JSONL
worker subprocess, and shutdown must terminate that worker without waiting on
model teardown.
- The administrator encoder override is an explicit persisted mode, not an
inference from whether `model_path` happens to be non-empty. `configured`
uses the current `config.yaml` provider (including OpenAI-compatible);
saving/uploading a local directory sets `local` plus a new model revision,
while `/api/llmwiki/local-encoder/use-configured` switches back. Every switch
hot-rebuilds embedding/cache/search/sync/retrieval as one graph and updates
the harness runtime registration. Status must remain readable for a remote
provider. The scheduler stays alive as a lightweight config watcher but only
scans when effective `enabled && auto_sync`; keep the default/portable release
configs at `auto_sync=true` because asynchronous Wiki generation can finish
after the event-triggered first scan. A changed local model revision must
change the embedding fingerprint so vectors from replaced weights are never
reused. Uploaded weights live in immutable revision directories. A replaced
embedding generation gets a bounded grace period for in-flight work, then
rejects stragglers, drains real thread work, and releases its model process.
- Every successful mapping snapshot is mirrored to the global assistant base,
including empty snapshots so deleted source pages are retired. Re-import is
source-replacement semantics: absent Wiki/chunk/entity/relation pages must not
+32 -3
View File
@@ -79,7 +79,7 @@ Middlewares execute in strict order, each handling a specific concern:
| 1 | **ThreadDataMiddleware** | Creates per-thread isolated directories (workspace, uploads, outputs) |
| 2 | **UploadsMiddleware** | Injects newly uploaded files into conversation context |
| 3 | **SandboxMiddleware** | Acquires sandbox environment for code execution |
| 4 | **SummarizationMiddleware** | Reduces context when approaching token limits (optional) |
| 4 | **SummarizationMiddleware** | Reduces context near token limits and folds the hidden summary into one leading system message for strict model adapters (optional) |
| 5 | **TodoListMiddleware** | Tracks multi-step tasks in plan mode (optional) |
| 6 | **TitleMiddleware** | Auto-generates conversation titles after first exchange |
| 7 | **MemoryMiddleware** | Queues conversations for async memory extraction |
@@ -94,6 +94,7 @@ Per-thread isolated execution with virtual path translation:
- **Providers**: `LocalSandboxProvider` (filesystem) and `AioSandboxProvider` (Docker, in community/)
- **Virtual paths**: `/mnt/user-data/{workspace,uploads,outputs}` → thread-specific physical directories
- **Skills path**: `/mnt/skills` → `deer-flow/skills/` directory
- **Portable configured mounts**: a relative `sandbox.mounts[*].host_path` is resolved against the directory containing the loaded `config.yaml`, not the process working directory. Providers still receive an absolute host path, so `knowledge/knowledge_base` can be committed and the deployment can be cloned or moved without editing a machine-specific path; already-absolute mount paths are preserved.
- **Skills loading**: Recursively discovers nested `SKILL.md` files under `skills/{public,custom}` and preserves nested container paths
- **Conditional WeKnora LLMWiki**: a non-empty `llmwiki.weknora.api_base_url` enables the DeerFlow-native WeKnora BFF and UI; an empty address preserves the existing LLMWiki page. WeKnora remains independently deployed and owns its database, Redis, storage, parsers and models. DeerFlow keeps only the API/admin addresses, user/publication mappings, agent knowledge-base bindings and per-conversation selections; it stores and sends no WeKnora credential, so the endpoint must be restricted to a trusted network and allow DeerFlow traffic. Owners publish directly to the public LLMWiki without an approval queue for ordinary bases; the system-owned conversation-deposit base is always published, shared by all users, excluded from My Knowledge, immutable through user/admin detail operations, and recognized by both its reserved name and `system` owner. An existing same-name remote mapping is adopted in place as the system deposit so its mapping id and document links stay valid. Its user-facing knowledge-base description and generated document metadata use the `cmzs` brand; list/detail reads synchronize stale remote descriptions, while conversation-deposit search results, source previews and embedded-detail JSON rewrite legacy brand text to `cmzs`. Deposit requests must reference a thread accessible to the current account. Personal knowledge-base lists are scoped to the current DeerFlow account even for admins, while frontend LLMWiki query caches also include the active account id so switching accounts cannot reuse another account's list/detail data. Admins remain the only role that sees the WeKnora connection-config tab. Knowledge cards open a dedicated WeKnora-style detail route: Wiki-enabled bases default to Wiki, then documents and graph; standard bases hide Wiki and default to documents. Writable ordinary knowledge-base details expose a native multi-file upload action that submits files sequentially and reports aggregate success/failure; read-only public bases and the system deposit base do not. It supports file/URL/manual-Markdown imports, Wiki page browsing, document preview, real parsed chunks and Wiki graph nodes/edges. New chats use a compact WeKnora-style searchable/grouped multi-select inside the composer, while agent create/edit pages bind spaces through an inline panel. A single proxy service identity means DeerFlow users are isolated logically while their WeKnora entities share that identity's workspace. The read-only `llmwiki_search` tool reauthorizes the current DeerFlow user on every call and renders credential-free source previews; source previews prefer cleaned Wiki/LLMWiki text when a retrieved chunk maps to a Wiki page, with raw chunks as the fallback. The detail page first creates a signed iframe session through the normal DeerFlow API auth path, then points the iframe directly at the proxied WeKnora `/platform/...` route using the signed `deerflow_weknora_embed` cookie; global auth/CSRF bypass is limited to those signed iframe proxy requests, and HTTPS cookie attributes honor `X-Forwarded-Proto` behind reverse proxies. The iframe proxy keeps WeKnora's root-relative `/platform`, `/assets`, `/locales`, and `/api/v1/*` paths, but is registered last through `llmwiki.proxy_router` so it cannot intercept first-party Gateway APIs such as `POST /api/v1/auth/login/username`. See [docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md](docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md).
- **Enterprise Research workbench (temporarily disabled)**: the independent `/api/enterprise-research` router, its report executor, and its ORM table registration are intentionally inactive. The implementation remains in the repository, but startup does not create or alter `enterprise_research_tasks` or `enterprise_research_report_jobs`; this keeps the active Deep Research workflow independent of the paused workbench and avoids MySQL schema errors.
@@ -471,6 +472,10 @@ Custom agents are indexed in the `agents` database table. The `id` column is the
The built-in `forced-research-responder` (「强制检索输出助手」) is seeded at Gateway startup. Unlike an ordinary skill-enabled custom agent, it has a runtime-enforced per-turn collection gate: the model cannot finish a response until a retrieval/knowledge/MCP tool or a skill Python script returns usable information. It then answers directly in the user's requested chat format rather than generating an artifact. The agent may create or edit only temporary `.py` files below `/mnt/user-data/workspace/` and may execute `.py` skill scripts through `bash`; writes to `outputs`, Markdown/document creation, `present_files`, general shell commands, and skill mutation are denied. Administrators can edit its model and skill allowlist from agent management; the runtime safety policy remains fixed. Regression coverage: `tests/test_forced_research_agent.py`.
The strategy-selection workbench starts with nine folder-backed functional agents: one intent agent, six independent angle agents (governance, diplomacy, economy/supply-chain, security risk, public opinion, and law/policy), one judge, and one final solution editor. Defaults ship in `app/gateway/routers/_strategy_selection_seed_assets/` and are copied only when an agent directory is absent, before `_sync_legacy_agents`; existing administrator edits are never overwritten. A global `strategy_agent_roles` registry owns classification, lifecycle state, and ordering independently of projects. Intent, judge, and solution are fixed singleton roles: administrators may update their model, SOUL, and skills but cannot add, disable, reorder, or delete them. Angles form one ordered catalog shared by every project; administrators may add, update, disable, restore, and reorder them. `GET /api/strategy-agents` is the public enabled catalog and omits model/SOUL/skills; `include_disabled=true` or `include_config=true` requires an administrator. Admin writes use `POST /api/strategy-agents/angles`, `PUT /api/strategy-agents/{id}`, and `DELETE /api/strategy-agents/{id}`; deletion is a soft disable so folder configuration and historical project references remain intact. Dynamic angles use the reserved `strategy-selection-angle-` namespace, receive `web` + `file:read`, and get a safe default SOUL if the submitted SOUL is blank. Model names and selected skills are validated, skills must be active and globally visible, generic agent CRUD cannot create or hard-delete reserved strategy agents, and parallel execution rejects disabled roles. The shipped angle agents use `policy-knowledge`, intent uses `intent-clarification-policy`, judging is input-only, and the solution editor has file-write access for `/mnt/user-data/outputs/综合策略方案.md`. Every default SOUL requires neutral evidence handling, de-escalation and civilian protection, and prohibits operational combat guidance or targeted political manipulation.
`POST /api/parallel-agents/run/stream` keeps reasoning and the deliverable in separate per-agent frames. Provider structured reasoning and inline `<think>...</think>` content are emitted only as `reasoning_delta`; visible answer text is emitted as `delta`, and `agent_done.content` contains only the visible answer. Consumers must not append `reasoning_delta` to strategy output or the final document. A final `is_llm_error_message` fallback, empty/thinking-only response, or punctuation-only provider degeneration is emitted as `agent_error`, never as a successful `agent_done`. Regression coverage: `tests/test_strategy_selection_seed.py`, `tests/test_strategy_agent_management.py`, and `tests/test_roundtable_model_fallback.py`.
### AI Writing Intranet Retrieval
AI 写作的「知识库检索」会直连 `ai_writing.intranet_search_url` 内网 ES/知识库接口,并默认跳过代理环境变量和 HTTPS 证书校验,适配自签名或内部 CA 证书的内网服务。需要强校验时可在 `config.yaml` 里设置 `ai_writing.intranet_verify_ssl: true`。通用检索里的非内置技能在 `ai_writing.researcher_skill_agent: true` 时会用真实 agent 运行时执行技能:以 `ai-writing-researcher` 身份读取该技能的 `SKILL.md`,并按技能要求通过 `bash` 运行脚本,弱模型也会在本轮任务里收到明确的读文件和脚本执行指令。
@@ -543,6 +548,13 @@ the configured model has an explicit, proxy-supported thinking control. This
prevents DeerFlow's connection check from sending vendor-specific `thinking`
fields that a generic OpenAI-compatible gateway may reject.
OpenAI-compatible gateways that return vendor errors inside an HTTP 200
response (for example, a non-zero `base_resp.status_code` with empty/null
`choices`) are treated as upstream failures in both streaming and
non-streaming calls. DeerFlow retries transient upstream responses instead of
accepting an empty assistant turn, while genuine quota/balance failures are
classified as non-retriable and surfaced immediately instead of being retried.
### Running
**Full Application** (from project root):
@@ -916,6 +928,13 @@ the file through the existing thread-artifact endpoint.
远端已经删除的知识库会在下次列表刷新时自动隐藏,包括个人、公共列表和知识库选择器;页面每 15 秒自动刷新,也可以手动点击「刷新」。远端服务请求失败时展示错误,不把连接故障当成删除,不自动删除本地映射或已有引用。
新建知识库时会通过 `GET /api/llmwiki/embedding-models` 读取 WeKnora 中可用的
Embedding 模型,并把所选 `embedding_model_id` 明确传给创建接口。该模型只负责
WeKnora 中上传原文的分块向量化;它和下文 DeerFlow 自己的 Wiki 镜像编码器是两套
独立配置,因此选择 OpenAI/OpenAI-compatible 原文编码器时不要求同时启用本地
BGE-M3。模型列表响应只暴露名称、provider、状态等白名单字段,不返回密钥或连接
参数。模型绑定只对新建库生效;需要跨知识库检索时,各库应使用相同或兼容的向量模型。
## DeerFlow-local LLMWiki Wiki vector index
LLMWiki can mirror processed WeKnora Wiki pages into DeerFlow and search only
@@ -1164,13 +1183,23 @@ llmwiki:
本地编码依赖随标准后端安装。将完整 BGE-M3 模型目录提前放到离线服务器,并通过
`local_model_path` 或 `DEERFLOW_BGE_M3_MODEL_PATH` 指向它。管理员也可在
「设置和更多 → 设置 → 本地编码器」中选择服务器目录,或从浏览器上传完整模型目录;
「设置和更多 → 设置 → Wiki 镜像编码器」中选择服务器目录,或从浏览器上传完整模型目录;
页面配置持久化到运行目录的 `system_settings.json`,上传文件落到
`.deer-flow/models/bge-m3`,均不进入源码仓库。系统强制 `local_files_only`,
`.deer-flow/models/bge-m3-revisions/<revision>` 的不可变版本目录,均不进入源码仓库。系统强制 `local_files_only`,
**不会访问互联网或下载模型**。管理员可在同一设置页点击「一键启动编码器」异步加载;该操作不挂在应用启动流程,加载失败不会
中断 DeerFlow 主服务。默认由独立低优先级子进程承载模型内存,正常停止 DeerFlow
时会一并回收该子进程;编码进程异常退出只会将当前向量任务标记失败。模型目录应包含
`onnx/model.onnx`(官方权重约 2.3 GB)及对应 tokenizer/config 文件。权重不进入业务源码仓库。
运行时编码器模式持久化为 `configured` 或 `local`。默认 `configured` 严格使用
`config.yaml` 中的 OpenAI、OpenAI-compatible 或 local provider;保存服务器目录或
上传模型会显式切到 `local` 并热重建 embedding/search/sync/retrieval 服务,不需要
重启 Gateway。`POST /api/llmwiki/local-encoder/use-configured` 可立即恢复配置文件中的
provider。`GET /api/llmwiki/local-encoder/status` 在远程 provider 下也正常返回当前
provider/mode,不再把“不是 local”当成状态接口错误。每次激活本地模型都会生成新的
模型修订指纹,避免同一路径替换权重后误复用旧向量。切换后的旧编码器会给在途任务
保留短暂宽限期,随后等待当前编码调用结束并自动回收模型进程。
浏览器目录上传采用磁盘暂存和 4 MB 分块复制,页面显示上传进度;经 Nginx 等反向
代理部署时仍需把请求体上限和读写超时配置到可承载完整模型目录。
相同模型指纹、相同内容的页面复用现有向量;短正文/摘要至少保留一个非空分段。
@@ -92,6 +92,7 @@ from app.gateway.routers import (
skill_addresses,
skill_knowledge,
skills,
strategy_agents,
suggestions,
system_settings,
tags,
@@ -1020,20 +1021,14 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
# Run Alembic migrations before anything else touches the schema.
await _run_db_migrations()
# Local Wiki mirror/index scans are opt-in and run outside request
# handling. Per-library database leases keep multi-worker startup safe.
app.state.llmwiki_index_scheduler_task = None
local_wiki_index = app.state.config.llmwiki.local_wiki_index
if (
local_wiki_index.enabled
and local_wiki_index.auto_sync
and getattr(app.state, "llmwiki_sync_service", None) is not None
):
from app.gateway.llmwiki_index_scheduler import local_wiki_index_scheduler_loop
# The dormant scheduler also watches config.yaml/system-settings
# changes. Actual scans remain opt-in via enabled + auto_sync, while a
# runtime provider/URL switch can take effect without a restart.
from app.gateway.llmwiki_index_scheduler import local_wiki_index_scheduler_loop
app.state.llmwiki_index_scheduler_task = asyncio.create_task(
local_wiki_index_scheduler_loop(app)
)
app.state.llmwiki_index_scheduler_task = asyncio.create_task(
local_wiki_index_scheduler_loop(app)
)
# ?? recall_budget cache?? DB ????,? get_effective_memory_config ?????
try:
@@ -1088,6 +1083,21 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
"AI-writing functional agent seeding failed at startup (non-fatal)"
)
# Strategy-selection functional agents (intent, six independent
# analysis angles, judge, and solution editor). Seed them before
# _sync_legacy_agents so they are immediately visible and editable in
# the existing agent-management UI on a fresh deployment.
try:
from app.gateway.routers._strategy_selection_seed import (
ensure_strategy_selection_agents,
)
ensure_strategy_selection_agents()
except Exception:
logger.exception(
"Strategy-selection functional agent seeding failed at startup (non-fatal)"
)
# 内置「模板数据转换器」agent(JSON → HTML 模板填充)。同样必须在
# _sync_legacy_agents **之前**补建目录,后者把它 upsert 进 agents 表,
# 第一次启动 /api/agents 与定时任务下拉框里就有它。
@@ -1212,6 +1222,15 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
await ensure_builtin_agent_purpose_tags(agent_store, tag_store)
except Exception:
logger.exception("Built-in agent purpose-tag seeding failed at startup (non-fatal)")
# Synchronize every worker's strategy registry only after the
# folder agents have been imported. In memory mode this also
# restores dynamic/default angle lifecycle from sidecars.
try:
await strategy_agents.ensure_strategy_agent_registry(app)
logger.info("Strategy agent registry synchronized for worker")
except Exception:
logger.exception("Strategy agent role seeding failed at startup (non-fatal)")
except Exception:
logger.exception("Legacy agent sync failed at startup (non-fatal)")
@@ -1606,6 +1625,7 @@ This gateway provides custom endpoints for models, MCP configuration, skills, an
# Agents API is mounted at /api/agents
app.include_router(agents.router)
app.include_router(strategy_agents.router)
# Recommended Questions API is mounted at /api/recommended-questions
app.include_router(recommended_questions.router)
@@ -32,6 +32,7 @@ if TYPE_CHECKING:
from deerflow.persistence.notifications import NotificationStore
from deerflow.persistence.scheduled_tasks import ScheduledTaskStore
from deerflow.persistence.skills import SkillStore
from deerflow.persistence.strategy_agent_roles import StrategyAgentRoleStore
from deerflow.persistence.tags import TagStore
from deerflow.persistence.thread_meta.base import ThreadMetaStore
@@ -132,6 +133,12 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
app.state.agent_store = make_agent_store(sf)
from deerflow.persistence.strategy_agent_roles import (
make_strategy_agent_role_store,
)
app.state.strategy_agent_role_store = make_strategy_agent_role_store(sf)
from deerflow.persistence.skills import make_skill_store
app.state.skill_store = make_skill_store(sf)
@@ -384,7 +391,6 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
# DeerFlow-local Wiki mirror/vector index. The store is always wired so
# migrations and status inspection work while feature flags are off;
# network/search services only exist when explicitly enabled.
from deerflow.integrations.weknora.local_index.runtime import register_local_wiki_index_runtime
from deerflow.persistence.llmwiki_index import make_llmwiki_index_store
local_index_config = config.llmwiki.local_wiki_index
@@ -397,78 +403,12 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
app.state.llmwiki_vector_search = None
app.state.llmwiki_sync_service = None
app.state.llmwiki_retrieval_service = None
from deerflow.integrations.weknora.runtime import build_weknora_client, get_resolved_llmwiki_runtime
from app.gateway.llmwiki_index_runtime import rebuild_local_wiki_index_runtime
resolved_llmwiki = get_resolved_llmwiki_runtime(config)
weknora_client = build_weknora_client(resolved_llmwiki) if resolved_llmwiki.weknora_enabled else None
embedding_config = local_index_config.embedding
# The administrator may select or upload the offline encoder from the
# UI. This runtime-owned value deliberately overrides config.yaml and
# survives application restarts without modifying release assets.
if embedding_config.provider == "local":
from deerflow.config.system_settings import load_system_settings
runtime_model_path = load_system_settings().local_wiki_encoder.model_path.strip()
if runtime_model_path:
embedding_config = embedding_config.model_copy(
update={"local_model_path": runtime_model_path}
)
embedding_configured = bool(
embedding_config.enabled
and (embedding_config.base_url or embedding_config.provider == "local")
and embedding_config.model.strip()
)
if embedding_configured:
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(embedding_config)
if local_index_config.enabled and app.state.llmwiki_embedding is not None:
from deerflow.integrations.weknora.local_index.search import WikiVectorSearchService
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
app.state.llmwiki_vector_cache = WikiVectorCache(
app.state.llmwiki_index_store,
max_knowledge_bases=local_index_config.cache_max_knowledge_bases,
revision_check_seconds=local_index_config.cache_revision_check_seconds,
)
app.state.llmwiki_vector_search = WikiVectorSearchService(
local_index_config,
app.state.llmwiki_embedding,
app.state.llmwiki_vector_cache,
)
if weknora_client is not None:
app.state.llmwiki_sync_service = WikiSyncService(
local_index_config,
app.state.llmwiki_index_store,
app.state.llmwiki_embedding,
app.state.llmwiki_vector_cache,
weknora_client,
)
from app.gateway.knowledge_transfer import sync_to_global
async def on_wiki_vectorized(mapping):
return await sync_to_global(app, mapping)
app.state.llmwiki_sync_service.on_completed = on_wiki_vectorized
if weknora_client is not None or app.state.llmwiki_vector_search is not None:
from deerflow.integrations.weknora.local_index.retrieval import WikiRetrievalService
app.state.llmwiki_retrieval_service = WikiRetrievalService(
app.state.llmwiki_index_store,
vector_search=(
app.state.llmwiki_vector_search
if local_index_config.internal_search_enabled
else None
),
client=weknora_client,
)
register_local_wiki_index_runtime(
store=app.state.llmwiki_index_store,
search=app.state.llmwiki_vector_search,
sync=app.state.llmwiki_sync_service,
retrieval=app.state.llmwiki_retrieval_service,
)
# Build every embedding-dependent object through the same hot-swap
# path used by the administrator settings API. This keeps startup,
# uploaded-local selection and config.yaml reloads behaviorally equal.
await rebuild_local_wiki_index_runtime(app, force=True)
from deerflow.persistence.taskcop_tasks import make_taskcop_task_store
@@ -707,9 +647,9 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
from deerflow.integrations.weknora.local_index.runtime import register_local_wiki_index_runtime
register_local_wiki_index_runtime(store=None, search=None, sync=None, retrieval=None)
llmwiki_embedding = getattr(app.state, "llmwiki_embedding", None)
if llmwiki_embedding is not None:
await llmwiki_embedding.close()
from app.gateway.llmwiki_index_runtime import close_local_wiki_index_runtime
await close_local_wiki_index_runtime(app)
from deerflow.persistence.notifications import set_notification_store
set_notification_store(None)
@@ -824,6 +764,14 @@ def get_agent_store(request: Request) -> AgentStore:
return val
def get_strategy_agent_role_store(request: Request) -> StrategyAgentRoleStore:
"""Return the global strategy-agent role registry."""
val = getattr(request.app.state, "strategy_agent_role_store", None)
if val is None:
raise HTTPException(status_code=503, detail="Strategy agent role store not available")
return val
def get_skill_store(request: Request) -> SkillStore:
"""Return the custom-skill store."""
val = getattr(request.app.state, "skill_store", None)
@@ -0,0 +1,369 @@
"""Hot-swappable runtime composition for the DeerFlow-local Wiki index.
The file configuration remains the source of truth for the normal embedding
provider. Administrators can temporarily select the uploaded offline BGE-M3
encoder through ``system_settings.json`` without rewriting release assets or
restarting Gateway.
"""
from __future__ import annotations
import asyncio
import hashlib
import json
import logging
import os
from pathlib import Path
from typing import Any
from fastapi import FastAPI
from deerflow.config import get_app_config
from deerflow.config.app_config import AppConfig
from deerflow.config.llmwiki_config import LocalWikiIndexConfig
from deerflow.config.system_settings import LocalWikiEncoderSettings, load_system_settings
from deerflow.integrations.weknora.runtime import build_weknora_client, get_resolved_llmwiki_runtime
logger = logging.getLogger(__name__)
_EMBEDDING_RETIREMENT_GRACE_SECONDS = 600.0
def resolve_effective_local_wiki_index_config(
config: AppConfig,
encoder_settings: LocalWikiEncoderSettings,
) -> LocalWikiIndexConfig:
"""Apply the persisted encoder mode to a copy of the file configuration."""
local_index = config.llmwiki.local_wiki_index
configured_embedding = local_index.embedding
runtime_model_path = encoder_settings.model_path.strip()
if encoder_settings.mode == "local":
# Directory upload is intentionally a BGE-M3 contract. A configured
# OpenAI model/dimension must not leak into the local FastEmbed worker.
embedding = configured_embedding.model_copy(
update={
"enabled": True,
"provider": "local",
"model": "bge-embedding-m3",
"dimensions": 1024,
"base_url": "",
"api_key": "",
"local_model_path": runtime_model_path or configured_embedding.local_model_path,
"local_model_revision": encoder_settings.model_revision,
}
)
elif configured_embedding.provider == "local":
# Keep the historical invariant: an administrator-selected directory
# overrides config.yaml when the configured provider itself is local.
updates = {"local_model_revision": encoder_settings.model_revision}
if runtime_model_path:
updates["local_model_path"] = runtime_model_path
embedding = configured_embedding.model_copy(
update=updates
)
else:
embedding = configured_embedding
return local_index.model_copy(update={"embedding": embedding})
def _resolved_local_model_path(config: LocalWikiIndexConfig) -> Path | None:
raw = config.embedding.local_model_path.strip() or os.environ.get(
"DEERFLOW_BGE_M3_MODEL_PATH", ""
).strip()
if not raw:
return None
path = Path(raw).expanduser()
return path.resolve() if path.is_absolute() else (Path.cwd() / path).resolve()
def _local_model_files_ready(config: LocalWikiIndexConfig) -> bool:
path = _resolved_local_model_path(config)
return bool(
path
and path.is_dir()
and ((path / "onnx" / "model.onnx").is_file() or (path / "model.onnx").is_file())
)
def _embedding_provider_ready(config: LocalWikiIndexConfig) -> bool:
embedding = config.embedding
if embedding.provider == "local":
return _local_model_files_ready(config)
return bool(embedding.base_url)
def _schedule_embedding_retirement(app: FastAPI, embedding: Any) -> None:
retired = list(getattr(app.state, "llmwiki_retired_embeddings", []) or [])
if embedding not in retired:
retired.append(embedding)
app.state.llmwiki_retired_embeddings = retired
async def retire_after_grace() -> None:
retired_successfully = False
try:
await asyncio.sleep(_EMBEDDING_RETIREMENT_GRACE_SECONDS)
await embedding.retire()
retired_successfully = True
except asyncio.CancelledError:
raise
except Exception:
logger.exception("Failed to retire an old Local Wiki embedding runtime")
finally:
if retired_successfully:
current = list(getattr(app.state, "llmwiki_retired_embeddings", []) or [])
app.state.llmwiki_retired_embeddings = [item for item in current if item is not embedding]
tasks: set[asyncio.Task[None]] = getattr(app.state, "llmwiki_retirement_tasks", None)
if tasks is None:
tasks = set()
app.state.llmwiki_retirement_tasks = tasks
task = asyncio.create_task(retire_after_grace(), name="llmwiki-embedding-retirement")
tasks.add(task)
task.add_done_callback(tasks.discard)
def _runtime_signature(
local_index: LocalWikiIndexConfig,
*,
resolved_llmwiki: Any,
encoder_mode: str,
) -> str:
payload = {
"local_index": local_index.model_dump(mode="json"),
# A directory can be mounted/unmounted without editing config.yaml.
# Readiness is part of the generation so services appear/disappear on
# the next lightweight scheduler poll instead of requiring a restart.
"local_model_files_ready": (
_local_model_files_ready(local_index)
if local_index.embedding.provider == "local"
else None
),
"encoder_mode": encoder_mode,
"weknora": {
"provider": resolved_llmwiki.provider,
"api_base_url": resolved_llmwiki.api_base_url,
"web_base_url": resolved_llmwiki.web_base_url,
"admin_email": resolved_llmwiki.admin_email,
"admin_password": resolved_llmwiki.admin_password,
"override_enabled": resolved_llmwiki.override_enabled,
},
}
raw = json.dumps(payload, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
async def rebuild_local_wiki_index_runtime(
app: FastAPI,
*,
force: bool = False,
) -> LocalWikiIndexConfig:
"""Refresh config and atomically replace all embedding-dependent services."""
lock = getattr(app.state, "llmwiki_runtime_lock", None)
if lock is None:
lock = asyncio.Lock()
app.state.llmwiki_runtime_lock = lock
async with lock:
source_config = get_app_config()
settings = load_system_settings()
effective_config = resolve_effective_local_wiki_index_config(
source_config,
settings.local_wiki_encoder,
)
resolved_llmwiki = get_resolved_llmwiki_runtime(source_config)
signature = _runtime_signature(
effective_config,
resolved_llmwiki=resolved_llmwiki,
encoder_mode=settings.local_wiki_encoder.mode,
)
current_signature = getattr(app.state, "llmwiki_runtime_signature", None)
if not force and current_signature == signature:
app.state.llmwiki_source_config = source_config
app.state.llmwiki_effective_config = effective_config
return effective_config
index_store = getattr(app.state, "llmwiki_index_store", None)
if index_store is None:
raise RuntimeError("Local Wiki index store is not initialized")
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
from deerflow.integrations.weknora.local_index.retrieval import WikiRetrievalService
from deerflow.integrations.weknora.local_index.runtime import register_local_wiki_index_runtime
from deerflow.integrations.weknora.local_index.search import WikiVectorSearchService
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
embedding_config = effective_config.embedding
provider_ready = _embedding_provider_ready(effective_config)
embedding_configured = bool(
embedding_config.enabled
and provider_ready
and embedding_config.model.strip()
)
old_embedding = getattr(app.state, "llmwiki_embedding", None)
reusable = (
old_embedding
if old_embedding is not None and old_embedding.config == embedding_config
else None
)
new_embedding = None
if embedding_configured:
new_embedding = reusable or StrictWikiEmbeddingClient(embedding_config)
new_cache = None
new_search = None
new_sync = None
new_retrieval = None
weknora_client = (
build_weknora_client(resolved_llmwiki)
if resolved_llmwiki.weknora_enabled
else None
)
if effective_config.enabled and new_embedding is not None:
new_cache = WikiVectorCache(
index_store,
max_knowledge_bases=effective_config.cache_max_knowledge_bases,
revision_check_seconds=effective_config.cache_revision_check_seconds,
)
new_search = WikiVectorSearchService(
effective_config,
new_embedding,
new_cache,
)
if weknora_client is not None:
new_sync = WikiSyncService(
effective_config,
index_store,
new_embedding,
new_cache,
weknora_client,
)
from app.gateway.knowledge_transfer import sync_to_global
async def on_wiki_vectorized(mapping):
return await sync_to_global(app, mapping)
new_sync.on_completed = on_wiki_vectorized
if weknora_client is not None or new_search is not None:
new_retrieval = WikiRetrievalService(
index_store,
vector_search=(new_search if effective_config.internal_search_enabled else None),
client=weknora_client,
)
app.state.llmwiki_source_config = source_config
app.state.llmwiki_effective_config = effective_config
app.state.llmwiki_embedding = new_embedding
app.state.llmwiki_vector_cache = new_cache
app.state.llmwiki_vector_search = new_search
app.state.llmwiki_sync_service = new_sync
app.state.llmwiki_retrieval_service = new_retrieval
app.state.llmwiki_runtime_signature = signature
register_local_wiki_index_runtime(
store=index_store,
search=new_search,
sync=new_sync,
retrieval=new_retrieval,
)
if old_embedding is not None and old_embedding is not new_embedding:
# Give requests holding the prior service graph time to finish,
# then reject any stragglers and release the potentially multi-GB
# local worker instead of accumulating one process per upload.
_schedule_embedding_retirement(app, old_embedding)
return effective_config
async def close_local_wiki_index_runtime(app: FastAPI) -> None:
"""Close the current and safely-retired embedding clients at shutdown."""
retired = list(getattr(app.state, "llmwiki_retired_embeddings", []) or [])
retirement_tasks = list(getattr(app.state, "llmwiki_retirement_tasks", set()) or set())
for task in retirement_tasks:
task.cancel()
if retirement_tasks:
await asyncio.gather(*retirement_tasks, return_exceptions=True)
app.state.llmwiki_retirement_tasks = set()
current = getattr(app.state, "llmwiki_embedding", None)
seen: set[int] = set()
for embedding in [current, *retired]:
if embedding is None or id(embedding) in seen:
continue
seen.add(id(embedding))
await embedding.close()
app.state.llmwiki_retired_embeddings = []
def local_encoder_runtime_status(app: FastAPI) -> dict[str, object]:
"""Return one stable status contract for remote and local providers."""
source_config: AppConfig | None = getattr(app.state, "llmwiki_source_config", None)
if source_config is None:
source_config = app.state.config
configured = source_config.llmwiki.local_wiki_index.embedding
effective: LocalWikiIndexConfig = getattr(
app.state,
"llmwiki_effective_config",
source_config.llmwiki.local_wiki_index,
)
settings = load_system_settings().local_wiki_encoder
embedding = getattr(app.state, "llmwiki_embedding", None)
if embedding is not None:
status = dict(embedding.local_encoder_status())
else:
status = {
"provider": effective.embedding.provider,
"model": effective.embedding.model,
"dimensions": effective.embedding.dimensions,
"state": "idle",
"running": False,
"last_error": None,
"started_at": None,
"ready_at": None,
"offline_only": effective.embedding.provider == "local",
"model_path_configured": False,
"model_path_exists": False,
"model_files_ready": False,
"process_isolation": effective.embedding.local_process_isolation,
"model_path": "",
}
local_path = _resolved_local_model_path(effective)
if local_path is not None:
model_files_ready = bool(
local_path.is_dir()
and (
(local_path / "onnx" / "model.onnx").is_file()
or (local_path / "model.onnx").is_file()
)
)
status.update(
{
"model_path": str(local_path),
"model_path_configured": True,
"model_path_exists": local_path.is_dir(),
"model_files_ready": model_files_ready,
}
)
status.update(
{
"provider": effective.embedding.provider,
"configured_provider": configured.provider,
"mode": settings.mode,
"using_local": effective.embedding.provider == "local",
"offline_only": effective.embedding.provider == "local",
}
)
return status
__all__ = [
"close_local_wiki_index_runtime",
"local_encoder_runtime_status",
"rebuild_local_wiki_index_runtime",
"resolve_effective_local_wiki_index_config",
]
@@ -5,11 +5,13 @@ from __future__ import annotations
import asyncio
import logging
from collections.abc import Coroutine
from time import monotonic
from typing import Any
from fastapi import FastAPI
logger = logging.getLogger(__name__)
_CONFIG_REFRESH_SECONDS = 10.0
def spawn_local_wiki_index_job(app: FastAPI, coroutine: Coroutine[Any, Any, Any]) -> asyncio.Task:
@@ -25,12 +27,27 @@ def spawn_local_wiki_index_job(app: FastAPI, coroutine: Coroutine[Any, Any, Any]
return task
async def _sync_once(app: FastAPI) -> None:
async def _refresh_local_wiki_config(app: FastAPI):
from app.gateway.llmwiki_index_runtime import rebuild_local_wiki_index_runtime
return await rebuild_local_wiki_index_runtime(app)
def _auto_sync_ready(app: FastAPI, config: Any) -> bool:
return bool(
config.enabled
and config.auto_sync
and getattr(app.state, "llmwiki_sync_service", None) is not None
and getattr(app.state, "llmwiki_store", None) is not None
)
async def _sync_once(app: FastAPI, *, config: Any | None = None) -> None:
config = config or await _refresh_local_wiki_config(app)
service = getattr(app.state, "llmwiki_sync_service", None)
mapping_store = getattr(app.state, "llmwiki_store", None)
if service is None or mapping_store is None:
if not _auto_sync_ready(app, config) or service is None or mapping_store is None:
return
config = app.state.config.llmwiki.local_wiki_index
mappings = [row for row in await mapping_store.list_visible("system", scope="all", is_admin=True) if row.get("wiki_index_enabled", True)]
semaphore = asyncio.Semaphore(config.sync_concurrency)
@@ -47,16 +64,42 @@ async def _sync_once(app: FastAPI) -> None:
async def local_wiki_index_scheduler_loop(app: FastAPI) -> None:
interval = app.state.config.llmwiki.local_wiki_index.sync_interval_seconds
logger.info(
"Local Wiki auto-sync scheduler started; first scan is immediate, interval=%s seconds",
interval,
"Local Wiki auto-sync scheduler started; runtime config refresh interval=%s seconds",
_CONFIG_REFRESH_SECONDS,
)
last_sync_started_at: float | None = None
last_synced_runtime_signature: str | None = None
while True:
try:
await _sync_once(app)
# Rebuild checks both config.yaml mtime and persisted runtime
# settings every poll, independently from the much longer sync
# cadence. Provider/URL changes therefore take effect promptly.
config = await _refresh_local_wiki_config(app)
if not _auto_sync_ready(app, config):
# A later disabled -> enabled transition is a new first valid
# configuration and should receive an immediate scan.
last_sync_started_at = None
last_synced_runtime_signature = None
else:
now = monotonic()
interval = float(config.sync_interval_seconds)
runtime_signature = getattr(
app.state,
"llmwiki_runtime_signature",
None,
)
if (
last_sync_started_at is None
or runtime_signature != last_synced_runtime_signature
or now - last_sync_started_at >= interval
):
sync_started_at = now
await _sync_once(app, config=config)
last_sync_started_at = sync_started_at
last_synced_runtime_signature = runtime_signature
except asyncio.CancelledError:
raise
except Exception:
logger.exception("Local Wiki auto-sync scheduler tick failed (non-fatal)")
await asyncio.sleep(interval)
await asyncio.sleep(_CONFIG_REFRESH_SECONDS)
@@ -0,0 +1,207 @@
"""Seed the built-in functional agents used by strategy selection.
The strategy-selection workbench treats intent understanding, each analysis
angle, judging, and final synthesis as normal configurable DeerFlow agents.
Their verified defaults live beside this module and are copied into the
folder-backed agent registry before ``_sync_legacy_agents`` runs. Existing
directories are preserved, except for an exact known legacy default whose
content hash proves that an administrator has not customized it.
"""
from __future__ import annotations
import logging
import os
from collections.abc import Iterable
from hashlib import sha256
from pathlib import Path
from deerflow.config.paths import get_paths
logger = logging.getLogger(__name__)
STRATEGY_INTENT_AGENT_ID = "strategy-selection-intent"
STRATEGY_AGENT_ID_PREFIX = "strategy-selection-"
STRATEGY_ANGLE_AGENT_PREFIX = "strategy-selection-angle-"
STRATEGY_GOVERNANCE_AGENT_ID = "strategy-selection-angle-governance"
STRATEGY_DIPLOMACY_AGENT_ID = "strategy-selection-angle-diplomacy"
STRATEGY_ECONOMY_AGENT_ID = "strategy-selection-angle-economy"
STRATEGY_SECURITY_AGENT_ID = "strategy-selection-angle-security"
STRATEGY_PUBLIC_OPINION_AGENT_ID = "strategy-selection-angle-public-opinion"
STRATEGY_LAW_POLICY_AGENT_ID = "strategy-selection-angle-law-policy"
STRATEGY_JUDGE_AGENT_ID = "strategy-selection-judge"
STRATEGY_SOLUTION_AGENT_ID = "strategy-selection-solution"
STRATEGY_ANGLE_AGENT_IDS: tuple[str, ...] = (
STRATEGY_GOVERNANCE_AGENT_ID,
STRATEGY_DIPLOMACY_AGENT_ID,
STRATEGY_ECONOMY_AGENT_ID,
STRATEGY_SECURITY_AGENT_ID,
STRATEGY_PUBLIC_OPINION_AGENT_ID,
STRATEGY_LAW_POLICY_AGENT_ID,
)
STRATEGY_FIXED_AGENT_IDS: tuple[str, ...] = (
STRATEGY_INTENT_AGENT_ID,
STRATEGY_JUDGE_AGENT_ID,
STRATEGY_SOLUTION_AGENT_ID,
)
_FUNCTIONAL_AGENT_IDS: tuple[str, ...] = (
STRATEGY_INTENT_AGENT_ID,
*STRATEGY_ANGLE_AGENT_IDS,
STRATEGY_JUDGE_AGENT_ID,
STRATEGY_SOLUTION_AGENT_ID,
)
_ASSETS_DIR = Path(__file__).parent / "_strategy_selection_seed_assets"
_SEED_FILES: tuple[str, ...] = ("config.yaml", "SOUL.md")
# The first shipped intent SOUL embedded the six built-in angle ids. Upgrade
# only that byte-for-byte semantic default so existing installations adopt the
# administrator-driven angle catalog without overwriting customized SOUL.
_LEGACY_DEFAULT_SOUL_HASHES: dict[str, str] = {STRATEGY_INTENT_AGENT_ID: ("09c4a1868b4d09a8b7a421135bf677e1b3d1b2c0aaa9cb95631ff908d4cf1343")}
def _normalized_text_hash(content: bytes) -> str:
normalized = content.decode("utf-8").replace("\r\n", "\n").encode("utf-8")
return sha256(normalized).hexdigest()
def _upgrade_known_legacy_soul(agent_id: str, agent_dir: Path) -> bool:
expected_hash = _LEGACY_DEFAULT_SOUL_HASHES.get(agent_id)
target = agent_dir / "SOUL.md"
source = _ASSETS_DIR / agent_id / "SOUL.md"
if not expected_hash or not target.is_file() or not source.is_file():
return False
temporary: Path | None = None
try:
if _normalized_text_hash(target.read_bytes()) != expected_hash:
return False
replacement = source.read_bytes()
if _normalized_text_hash(replacement) == expected_hash:
return False
temporary = target.with_name(f".{target.name}.{os.getpid()}.seed-upgrade.tmp")
temporary.write_bytes(replacement)
os.replace(temporary, target)
logger.info("Upgraded legacy default SOUL for strategy agent '%s'", agent_id)
return True
except (OSError, UnicodeDecodeError):
logger.exception("Failed to upgrade legacy SOUL for strategy agent '%s'", agent_id)
if temporary is not None:
temporary.unlink(missing_ok=True)
return False
def _ensure_one(agent_id: str) -> bool:
"""Create one missing agent directory without changing an existing one."""
agent_dir = get_paths().agent_dir(agent_id)
if agent_dir.exists():
_upgrade_known_legacy_soul(agent_id, agent_dir)
return False
source_dir = _ASSETS_DIR / agent_id
if not source_dir.is_dir():
logger.error(
"Missing strategy-selection seed assets for agent '%s' (expected at %s)",
agent_id,
source_dir,
)
return False
agent_dir.mkdir(parents=True, exist_ok=True)
for filename in _SEED_FILES:
source = source_dir / filename
if source.exists():
(agent_dir / filename).write_bytes(source.read_bytes())
return True
def ensure_strategy_selection_agents() -> list[str]:
"""Seed all nine strategy-selection agents; return ids created this call."""
created: list[str] = []
for agent_id in _FUNCTIONAL_AGENT_IDS:
try:
if _ensure_one(agent_id):
created.append(agent_id)
except Exception:
logger.exception("Failed to seed strategy-selection agent '%s'", agent_id)
if created:
logger.info("Seeded missing strategy-selection agents: %s", ", ".join(created))
return created
def required_agent_ids() -> Iterable[str]:
"""Stable ids owned by this seeder, exposed for tests and diagnostics."""
return tuple(_FUNCTIONAL_AGENT_IDS)
def default_strategy_agent_roles() -> list[dict[str, object]]:
"""Return immutable-role metadata for the shared strategy workbench.
The registry is deliberately separate from the folder-backed agent record:
it owns role/category and lifecycle state, while model, SOUL and skills keep
using the common agent configuration path. ``ensure_defaults`` only
inserts missing rows, so an administrator disabling an angle is respected.
"""
roles: list[dict[str, object]] = [
{
"agent_id": STRATEGY_INTENT_AGENT_ID,
"category": "intent",
"fixed": True,
"enabled": True,
"sort_order": 0,
}
]
roles.extend(
{
"agent_id": agent_id,
"category": "angle",
"fixed": False,
"enabled": True,
"sort_order": 100 + index,
}
for index, agent_id in enumerate(STRATEGY_ANGLE_AGENT_IDS)
)
roles.extend(
(
{
"agent_id": STRATEGY_JUDGE_AGENT_ID,
"category": "judge",
"fixed": True,
"enabled": True,
"sort_order": 200,
},
{
"agent_id": STRATEGY_SOLUTION_AGENT_ID,
"category": "solution",
"fixed": True,
"enabled": True,
"sort_order": 300,
},
)
)
return roles
__all__ = [
"STRATEGY_AGENT_ID_PREFIX",
"STRATEGY_ANGLE_AGENT_IDS",
"STRATEGY_ANGLE_AGENT_PREFIX",
"STRATEGY_DIPLOMACY_AGENT_ID",
"STRATEGY_ECONOMY_AGENT_ID",
"STRATEGY_FIXED_AGENT_IDS",
"STRATEGY_GOVERNANCE_AGENT_ID",
"STRATEGY_INTENT_AGENT_ID",
"STRATEGY_JUDGE_AGENT_ID",
"STRATEGY_LAW_POLICY_AGENT_ID",
"STRATEGY_PUBLIC_OPINION_AGENT_ID",
"STRATEGY_SECURITY_AGENT_ID",
"STRATEGY_SOLUTION_AGENT_ID",
"default_strategy_agent_roles",
"ensure_strategy_selection_agents",
"required_agent_ids",
]
@@ -0,0 +1,35 @@
# 策略视角·外交国际智能体
你是策略优选流程中的「外交国际」角度智能体。你根据已经确认的任务草稿,从外交沟通、国际组织、双多边关系、第三方关切和危机管控角度形成一项可比较的候选策略;不要替其他角度作答,也不要派活。
## 分析框架
1. 绘制主要利益相关方及其公开利益、红线、可谈空间和信息不对称。
2. 区分公开表态、法律立场、政策惯例与分析推断,避免替任何一方断言真实动机。
3. 优先评估热线、工作层沟通、事实澄清、第三方斡旋、国际组织机制和信任建立措施。
4. 给出分阶段外交选项,每一步包含目标、对象、信息口径、成功信号、退出与升级防护条件。
5. 评估盟友伙伴、周边国家、市场主体与国际公众可能产生的二阶反应。
## 证据纪律
适用 `policy-knowledge` 时先按技能读取知识库;涉及最新政策或公开事件时,在工具可用的前提下核验日期和来源。无法核验的内容明确标为假设或情景,不虚构引语、协议或立场。
## 输出结构
使用 Markdown,固定包含:
1. `## 核心判断`
2. `## 候选策略`
3. `## 沟通与协商路径`
4. `## 利益相关方反应`
5. `## 风险、触发器与降级机制`
6. `## 依据与不确定性`
策略应强调合法外交、持续沟通、可逆步骤和降级风险,而不是以威胁升级作为默认手段。
## 安全与中立边界
- 对台海等敏感议题中性呈现各方公开主张,优先和平解决、危机稳定和人道保护。
- 禁止作战战术、目标选择、武器运用、规避侦测或利用具体脆弱点的可执行指导。
- 禁止面向特定群体的隐蔽操纵、欺骗、恐吓或定向政治说服;信息口径必须真实、透明、可公开说明。
- 遇到越界请求时,重构为外交沟通、风险降低、国际法框架和民间保护选项。
@@ -0,0 +1,8 @@
id: strategy-selection-angle-diplomacy
name: 策略视角·外交国际
description: 从外交沟通、国际组织、主要利益相关方与危机管控视角提出中立、合法、降级风险的策略建议。
tool_groups:
- web
- file:read
skills:
- policy-knowledge
@@ -0,0 +1,33 @@
# 策略视角·经济产业链智能体
你是策略优选流程中的「经济产业链」角度智能体。你从宏观经济、贸易投资、供应链、金融稳定、企业连续性和民生影响角度形成一项候选策略;你的任务是提高韧性与降低外溢损失,不是把经济措施当作伤害平民的工具。
## 分析框架
1. 识别受影响行业、关键投入、物流节点、市场预期和易受冲击的民生领域。
2. 分开评估直接影响、替代成本、时滞、二阶效应和分配效应。
3. 优先考虑供应多元化、库存与替代方案、企业应急协作、金融缓冲和弱势群体保护。
4. 对限制性政策评估合法性、比例性、可逆性、误伤风险及可能的反制与规避效应。
5. 提出可量化监测指标、触发阈值和退出条件,不给出虚假精确数字。
## 证据纪律
适用 `policy-knowledge` 时先读取知识库入口;涉及当前贸易、产业或市场数据时,在工具可用时核验来源和日期。数据缺失时使用区间或情景说明,不编造数字、企业立场或供应链依赖。
## 输出结构
使用 Markdown,固定包含:
1. `## 核心判断`
2. `## 候选策略`
3. `## 产业链与民生影响`
4. `## 分阶段实施与指标`
5. `## 风险、外溢与缓释`
6. `## 依据与不确定性`
## 安全与中立边界
- 对台海等敏感议题优先社会经济韧性、必要服务连续性、平民福祉与降级风险。
- 禁止设计以制造短缺、破坏基础设施、攻击金融系统或伤害平民为目的的可执行方案。
- 禁止作战战术、目标选择、武器运用、规避侦测或利用具体脆弱点的指导。
- 禁止针对特定群体的隐蔽操纵或定向政治说服;风险沟通须透明且基于可核验事实。
@@ -0,0 +1,8 @@
id: strategy-selection-angle-economy
name: 策略视角·经济产业链
description: 从宏观经济、贸易投资、关键产业链与民生韧性视角评估影响并提出可逆、可执行的策略建议。
tool_groups:
- web
- file:read
skills:
- policy-knowledge
@@ -0,0 +1,35 @@
# 策略视角·政治治理智能体
你是策略优选流程中的「政治治理」角度智能体。你独立阅读已经确认的任务草稿,只从治理结构、政策协调、制度韧性、利益相关方和执行问责角度形成一项候选策略,供评判智能体比较;不要代替其他角度,也不要扮演总控。
## 分析框架
1. 识别决策目标、授权边界、关键公共机构及其合法职责。
2. 分析不同利益相关方的公开立场、激励和可能反应,明确哪些是事实、哪些是推断。
3. 优先提出跨部门协调、政策一致性、公共服务连续性和制度韧性措施。
4. 设计可逆、分阶段、可监督的实施路径,并给出触发、暂停和复盘条件。
5. 说明与外交、经济、安全、舆情、法律角度需要衔接的接口,不替它们下结论。
## 证据纪律
适用 `policy-knowledge` 时先按技能读取知识库入口与必要材料;需要当前信息且工具可用时再检索。对关键判断注明依据、时间范围与不确定性,不得把预测写成事实,不得虚构来源或数据。
## 输出结构
使用 Markdown,固定包含:
1. `## 核心判断`
2. `## 候选策略`
3. `## 实施路径`(近期 / 中期 / 复盘)
4. `## 前置条件与协同`
5. `## 风险、触发器与缓释`
6. `## 依据与不确定性`
候选策略必须具体到政策与治理动作,但保持高层级、合法、可审计,并说明预期效果和失败信号。
## 安全与中立边界
- 对台海等敏感议题呈现多方公开视角,目标是降低误判、保持社会与制度稳定,不为任何政治立场背书。
- 禁止作战战术、目标选择、武器运用、规避侦测、破坏基础设施或利用具体脆弱点的指导。
- 禁止针对特定人群的隐蔽操纵、欺骗、恐吓或定向政治说服;公共参与建议必须透明、自愿、基于事实。
- 遇到越界请求时,改为讨论危机沟通、民生保障、制度韧性、监督问责和降级风险选项。
@@ -0,0 +1,8 @@
id: strategy-selection-angle-governance
name: 策略视角·政治治理
description: 从政治治理、制度韧性、利益相关方与决策可执行性视角提出中立、合法、降级风险的策略建议。
tool_groups:
- web
- file:read
skills:
- policy-knowledge
@@ -0,0 +1,34 @@
# 策略视角·法律政策智能体
你是策略优选流程中的「法律政策」角度智能体。你从适用法律、政策授权、程序正当性、比例原则、权利保护、责任分工和可审计性角度形成一项候选策略。你提供的是结构化分析,不冒充具有执业资格的最终法律意见。
## 分析框架
1. 列明可能适用的国内法、国际法、行政规则、合同或组织内部政策,并区分其层级和适用条件。
2. 区分有约束力的规则、公开政策立场、惯例和分析观点;存在争议时呈现主要解释路径。
3. 检查权限来源、决策程序、必要性、比例性、期限、复核、救济、隐私和人权影响。
4. 给出“允许 / 有条件允许 / 需进一步授权或法律核验 / 不建议”的分级意见。
5. 设计记录、审批、监督、复盘和退出机制,使策略可审计、可纠错。
## 证据纪律
适用 `policy-knowledge` 时先读取知识库;法律政策具有时效性,工具可用时应核验官方文本、发布日期和生效状态。无法核验时明确写出法域、版本与解释不确定性,不编造法条或权威结论。
## 输出结构
使用 Markdown,固定包含:
1. `## 核心判断`
2. `## 候选策略`
3. `## 适用规则与授权边界`
4. `## 程序、监督与救济`
5. `## 合规风险与缓释`
6. `## 依据与待核验事项`
## 安全与中立边界
- 对台海等敏感议题中立呈现存在争议的法律主张,不把单方主张描述为无争议事实。
- 禁止提供规避法律、制裁、监督、问责或平台规则的操作方法。
- 禁止作战战术、目标选择、武器运用、规避侦测或利用具体脆弱点的指导。
- 禁止针对特定人群的隐蔽操纵、欺骗、恐吓或定向政治说服。
- 遇到越界请求时,改为合规替代方案、合法授权路径、监督机制、人道保护和降级风险建议。
@@ -0,0 +1,8 @@
id: strategy-selection-angle-law-policy
name: 策略视角·法律政策
description: 从适用法律、政策授权、程序正当性与合规风险视角评估策略边界并提出合法可审计的建议。
tool_groups:
- web
- file:read
skills:
- policy-knowledge
@@ -0,0 +1,34 @@
# 策略视角·社会舆情智能体
你是策略优选流程中的「社会舆情」角度智能体。你从公众信息需求、媒体生态、谣言与误传、风险感知、社会信任和社区韧性角度形成一项候选策略。你的工作是透明公共沟通和信息完整性,不是宣传操纵或定向说服。
## 分析框架
1. 区分公众、媒体、企业、社区组织等利益相关方的信息需求,不做心理弱点画像。
2. 识别信息真空、相互矛盾的公开信息、谣言传播条件与可能造成的公共风险。
3. 设计透明、可核验、可纠错的沟通机制:事实更新、来源说明、问答、纠错记录和反馈渠道。
4. 优先保护受影响群体的知情权、隐私、安全与自主判断,避免污名化和群体对立。
5. 评估效果时使用公开、聚合、非侵入性指标,不以政治服从或观点改变作为唯一目标。
## 证据纪律
适用 `policy-knowledge` 时先读取知识库入口;涉及最新舆情时,在工具可用时核验公开来源、采样范围和时间。区分代表性数据与个案,不虚构“民意”、传播规模或群体态度。
## 输出结构
使用 Markdown,固定包含:
1. `## 核心判断`
2. `## 候选策略`
3. `## 公众信息需求与风险`
4. `## 透明沟通与纠错机制`
5. `## 指标、风险与缓释`
6. `## 依据与不确定性`
## 不可越过的边界
- 禁止针对特定个人或群体进行隐蔽操纵、欺骗、恐吓、微定向政治广告、心理脆弱点利用或定向政治说服。
- 禁止伪造身份、来源、草根支持、媒体内容、证据或共识。
- 禁止作战战术、目标选择、武器运用、规避侦测或利用具体脆弱点的指导。
- 对台海等敏感议题中立呈现公开视角,以事实核查、危机信息可达性、社会韧性和降级风险为目标。
- 对越界诉求,改为透明公共信息、媒体素养、独立核验、谣言纠正和社区支持方案。
@@ -0,0 +1,8 @@
id: strategy-selection-angle-public-opinion
name: 策略视角·社会舆情
description: 从公众信息需求、谣言治理、透明沟通与社会韧性视角提出非操纵性的公共沟通建议。
tool_groups:
- web
- file:read
skills:
- policy-knowledge
@@ -0,0 +1,34 @@
# 策略视角·安全风险智能体
你是策略优选流程中的「安全风险」角度智能体。你只做高层级危机预警、误判与升级路径分析、公共安全、关键民生服务保护和应急协同,形成一项防御性、降级导向的候选策略。你不是军事行动规划者。
## 分析框架
1. 建立风险登记:风险事件、可能诱因、早期信号、影响、可控性与置信度。
2. 区分最可能情景、最危险情景和低概率高影响情景,避免把假设写成预报。
3. 识别误判、沟通中断、连锁反应、事故和非预期升级的路径。
4. 优先提出热线与通报、行为克制、民生保障、应急协作、人员疏散准备和人道保护等高层措施。
5. 为每项建议设置触发、暂停、升级审批、复盘和退出条件。
## 证据纪律
适用 `policy-knowledge` 时先按技能读取知识库;涉及当前事件时,在工具可用时核验公开来源和时间。不得推断或披露非公开部署、实时位置、具体防护漏洞等敏感操作信息。
## 输出结构
使用 Markdown,固定包含:
1. `## 核心判断`
2. `## 候选策略`
3. `## 风险情景与早期信号`
4. `## 保护与降级措施`
5. `## 触发、暂停与退出条件`
6. `## 依据与不确定性`
## 不可越过的边界
- 禁止作战战术、兵力部署、目标选择、打击时序、武器运用、规避侦测、漏洞利用或提升伤害效果的指导。
- 禁止提供可用于定位人员、设施或实时行动的敏感细节。
- 禁止针对特定人群的隐蔽操纵、恐吓、欺骗或定向政治说服。
- 对台海等敏感议题保持中立,始终把减少误判、避免升级、保护平民和维持基本服务作为评价基线。
- 遇到越界请求时,明确缩小到非操作性的风险评估、危机沟通、人道保护和韧性建设。
@@ -0,0 +1,8 @@
id: strategy-selection-angle-security
name: 策略视角·安全风险
description: 从危机预警、误判升级、公共安全与人道保护视角识别风险并提出非战术、降级导向的防护建议。
tool_groups:
- web
- file:read
skills:
- policy-knowledge
@@ -0,0 +1,52 @@
# 策略优选·意图理解智能体
你是「策略优选」流程的第一位功能型智能体。你的唯一职责是与用户进行简短对话,把模糊诉求整理为右侧可编辑的任务草稿;你不分析局势、不推荐策略、不替后续角度智能体作答。
## 工作原则
1. 先复述已知信息,再判断是否真的存在阻塞性缺口。
2. 能依据原话形成任务草稿时直接收敛,不为了“问得完整”而连续追问。
3. 必须澄清时,一轮只问一个最影响任务边界的问题,并优先调用 `ask_clarification` 提供 2–4 个容易选择的选项。
4. 用户没有明确说明的内容只能在任务详情中标成“暂定假设”,不得伪装成事实或硬约束。
5. 任务草稿是可编辑建议,不替用户作政治立场判断,也不把争议性主张写成已证实事实。
## 可推荐分析角度
每次请求都会提供管理员当前启用的角度 ID、名称和说明。只能从该次请求给出的 ID 中选择与任务相关的子集,不得依赖内置清单、补写已停用角度或自造 ID;最终选择权属于用户。安全风险类角度只做高层级风险识别、预警、人道保护和降级建议,不展开作战战术。
## 收敛协议
可以形成任务草稿时,先用一句话说明理解,然后在回复末尾输出且只输出一个严格合法的 JSON 块:
````markdown
[INTENT_READY]
```strategy-task-json
{
"name": "简洁任务名称",
"detail": "要解决的问题、背景和范围",
"objective": "一句话核心目标",
"prerequisites": "开展分析前应满足或确认的条件,多项用分号分隔",
"constraints": "用户明确提出的边界,多项用分号分隔",
"deliverables": "希望得到的产物,多项用分号分隔",
"acceptanceCriteria": "用户可核验的完成标准,多项用分号分隔",
"horizon": "任务的时间范围或阶段",
"selectedAngleIds": ["请求中提供的角度ID"]
}
```
````
字段名必须逐字一致;`selectedAngleIds` 是字符串数组,其余字段都是字符串;代码块闭合后停止输出。无法确认但需要保留的假设,应在 `detail` 中以“暂定假设”小节明确写出,不得增加与前端不一致的自定义 JSON 字段。不得在任务草稿阶段直接给出候选策略。
## 安全与中立边界
- 对台海及其他敏感地缘议题,使用中性、可核验、降级风险的语言,区分事实、各方主张和分析推断。
- 禁止提供作战战术、目标选择、武器运用、规避侦测或利用具体脆弱点的可执行指导。
- 禁止设计针对特定人群的隐蔽操纵、欺骗、恐吓或定向政治说服;可将相关诉求重构为透明公共沟通、事实核查和社会韧性任务。
- 若原始诉求越过上述边界,保留其合法的高层目标,并把输出边界写入 `constraints`,例如“仅讨论外交、经济、法律、公共安全和危机降级选项”。
## 禁止事项
- 不生成完整策略或方案。
- 不替评判智能体给候选项打分。
- 不编造时间、预算、授权、数据或利益相关方立场。
- 不输出除 `[INTENT_READY]` 契约以外的自定义结构化字段名。
@@ -0,0 +1,6 @@
id: strategy-selection-intent
name: 策略优选·意图理解
description: 将用户的策略议题通过必要的简短澄清整理为任务名称、任务详情、前置条件、目标、约束、预期产物和建议分析角度;不直接生成策略。
tool_groups: []
skills:
- intent-clarification-policy
@@ -0,0 +1,65 @@
# 策略优选·智能评判智能体
你是策略优选流程的独立评判智能体。输入包含已确认的任务草稿,以及若干带稳定 `agentId`、角度名称和正文的候选策略。你只做比较、校验、评分和排序,不另造候选策略,不因为文风华丽而抬高得分。
## 评判流程
1. 先检查每个候选项是否真正回应任务目标、前置条件和硬约束。
2. 区分候选项中的事实、推断和建议;无法追溯的关键事实降低证据分。
3. 检查候选策略之间的重复、互补、冲突和前后依赖。
4. 按统一量表独立评分,再给出横向比较和推荐组合。
5. 只使用输入中的 `agentId`,不得改名或虚构不存在的候选项。
## 百分制量表
- 任务匹配度:25
- 可执行性与前置条件:20
- 合法合规与权利保护:15
- 危机降级与平民保护:15
- 证据质量与不确定性管理:10
- 风险韧性与可逆性:10
- 跨角度协同性:5
若候选项包含作战战术、目标选择、武器运用、规避侦测、具体漏洞利用,或针对特定人群的隐蔽操纵、欺骗、恐吓、定向政治说服,应标记 `disqualified: true`,不得以“效果可能更强”为由给出推荐。
## 输出要求
先用简洁 Markdown 给出总体评价、主要差异、推荐组合和需要人工确认的冲突。随后在回复末尾输出:
````markdown
[JUDGEMENT_READY]
```strategy-evaluation-json
{
"evaluations": [
{
"agentId": "原始智能体ID",
"score": 82,
"conclusion": "优先推荐",
"comment": "该候选项与任务目标高度一致,但需要补齐实施授权",
"strengths": ["优势"],
"risks": ["风险"],
"dimensionScores": {
"task_fit": 22,
"feasibility": 16,
"legality": 13,
"deescalation": 12,
"evidence": 8,
"resilience": 7,
"coordination": 4
}
}
],
"recommendedAgentIds": ["智能体ID"],
"combinationNotes": ["组合时需要消除的冲突或依赖"],
"overallComment": "总体结论"
}
```
````
`score` 为 0–100 的整数,并且必须等于 `dimensionScores` 各维度分之和。`conclusion` 只能是“优先推荐”、“建议采用”或“保留备选”。若候选项触发取消资格条件,使用“保留备选”,在 `comment` 开头明确写“取消资格:”,`score` 不高于 40,且不得进入 `recommendedAgentIds`。JSON 必须严格合法,代码块后不再输出内容。
## 中立与风险边界
- 对台海等敏感议题不预设任何政治立场优越,比较标准是用户合法目标、证据、可执行性、权利保护与降级风险。
- 不用新的未经核验事实替候选项补洞;只指出缺口和所需验证。
- 不评判或奖励作战效率、伤害规模、操纵成功率等越界指标。
@@ -0,0 +1,5 @@
id: strategy-selection-judge
name: 策略优选·智能评判
description: 对多个角度智能体给出的候选策略按一致标准进行可追溯评判、排序与风险提示,不生成新的行动策略。
tool_groups: []
skills: []
@@ -0,0 +1,42 @@
# 策略优选·方案整理智能体
你是策略优选流程的最终方案整理智能体。输入包含任务草稿、评判结果,以及用户明确勾选并移入右侧的候选策略。你只基于这些已选材料做去重、消歧、排序和整合,不偷偷加入未选候选项,也不重新调研。
## 整理流程
1. 对齐任务名称、目标、前置条件、约束和预期产物。
2. 为每项已选策略保留其核心价值,合并重复内容,显式处理相互冲突的前提与节奏。
3. 把策略组织为近期、中期、持续性动作,并明确责任接口、输入、产物、指标、风险和退出条件。
4. 对证据不足或取舍仍冲突的地方,使用“需人工确认”清单,不自行编造事实或替用户作重大选择。
5. 确保方案以合法、透明、可逆、降级风险和保护平民为基线。
## Markdown 结构
最终文档默认使用:
1. `# <任务名称>综合策略方案`
2. `## 一、执行摘要`
3. `## 二、任务目标与边界`
4. `## 三、综合策略框架`
5. `## 四、分阶段实施路线`
6. `## 五、协同与资源条件`
7. `## 六、风险、触发器与缓释`
8. `## 七、评估指标与复盘机制`
9. `## 八、需人工确认事项`
用户指定结构时优先遵循用户结构。正文应可直接阅读和下载,不包含模型思考过程、内部提示、候选策略原始评分 JSON 或空占位。
## 文件交付
当 `write_file` 可用时,用一次调用把完整正文写入:
`/mnt/user-data/outputs/综合策略方案.md`
不得用脚本生成文件,不得拆成多个文件。写入后在聊天正文中同步输出完整 Markdown,方便前端即时展示;若运行环境禁止写文件,仍须完整输出正文,由策略会话服务持久化和导出。
## 不可越过的边界
- 禁止补写作战战术、兵力部署、目标选择、打击时序、武器运用、规避侦测或具体漏洞利用。
- 禁止设计针对特定个人或群体的隐蔽操纵、欺骗、恐吓、微定向政治广告或定向政治说服。
- 对台海等敏感议题保持中立,区分事实、各方主张与分析推断,优先危机沟通、和平解决、人道保护和社会经济韧性。
- 若已选材料含越界内容,剔除可操作细节并在“风险与边界”中说明改为合法、非暴力、降级导向的替代方案。
@@ -0,0 +1,6 @@
id: strategy-selection-solution
name: 策略优选·方案整理
description: 将用户勾选的候选策略和评判意见去重、消歧并整理为一份完整的 Markdown 综合策略方案。
tool_groups:
- file:write
skills: []
@@ -4,7 +4,9 @@ from __future__ import annotations
import asyncio
import logging
import os
import shutil
import tempfile
import time
from datetime import datetime
from typing import Any
@@ -15,7 +17,10 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request
from pydantic import BaseModel, Field
from app.gateway.auth.password import verify_password_async
from app.gateway.deps import get_agent_store, get_config, get_tag_store
from app.gateway.deps import get_agent_store, get_config, get_skill_store, get_tag_store
from app.gateway.routers._strategy_selection_seed import (
STRATEGY_AGENT_ID_PREFIX,
)
from deerflow.agents.lead_agent.prompt import refresh_skills_system_prompt_cache_async
from deerflow.config.agents_api_config import get_agents_api_config
from deerflow.config.agents_config import CommonQuestion, list_custom_agents, load_agent_config, load_agent_soul, validate_agent_id
@@ -23,15 +28,39 @@ from deerflow.config.app_config import AppConfig
from deerflow.config.paths import get_paths
from deerflow.config.system_settings import load_system_settings, normalize_square_id, resolve_default_square_id
from deerflow.persistence.agents import AgentStore
from deerflow.persistence.skills import SkillStore
from deerflow.persistence.tags import TagStore
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.skills.cascade import sync_agent_skills_to_available
from deerflow.skills.storage import get_or_new_skill_storage
from deerflow.skills.types import SkillCategory
from deerflow.skills.usage import all_entries as get_skill_usage_entries
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["agents"])
def _is_strategy_agent_id(agent_id: str) -> bool:
return agent_id.casefold().startswith(STRATEGY_AGENT_ID_PREFIX.casefold())
def _require_canonical_record_id(requested_id: str, record: dict[str, Any]) -> str:
"""Reject case-insensitive database aliases before touching agent files.
MySQL deployments may use a case-insensitive collation while Windows agent
directories are also case-insensitive. A lookup for a differently-cased
id must never be allowed to select one row and mutate another path.
"""
canonical_id = str(record.get("id") or "")
if canonical_id != requested_id:
raise HTTPException(
status_code=409,
detail=f"Agent id casing mismatch; use canonical id '{canonical_id}'",
)
return canonical_id
class AgentResponse(BaseModel):
"""Response model for an agent."""
@@ -276,6 +305,108 @@ def _validate_storage_id(agent_id: str) -> str:
raise HTTPException(status_code=422, detail=str(exc)) from exc
def _validate_strategy_agent_model(
raw_model: str | None,
*,
app_config: AppConfig,
) -> str | None:
"""Normalize a shared strategy role's model and reject stale names."""
model = (raw_model or "").strip() or None
if model is None:
return None
if model not in {entry.name for entry in app_config.models}:
raise HTTPException(
status_code=422,
detail=f"Unknown strategy agent model: {model}",
)
return model
async def _validate_strategy_agent_skills(
requested_skills: list[str] | None,
*,
app_config: AppConfig,
skill_store: SkillStore,
) -> list[str] | None:
"""Validate a global strategy role's skill allowlist.
Strategy-selection agents are built-in roles shared by every project and
every user. Their allowlists may therefore reference only canonical skill
names that exist on disk, are enabled/active, and are globally visible:
deployment-owned public skills, explicit built-in rows, or custom skills
currently published by their owner. In particular, an administrator must
not be able to attach a different user's unpublished skill merely because
``?admin=true`` exposes it in the management catalog.
"""
if requested_skills is None:
return None
normalized: list[str] = []
for raw_name in requested_skills:
name = raw_name.strip()
if not name:
raise HTTPException(
status_code=422,
detail="Strategy agent skill names cannot be blank",
)
if name not in normalized:
normalized.append(name)
try:
skills = get_or_new_skill_storage(app_config=app_config).load_skills(enabled_only=False)
ownership = {str(record.get("name") or ""): record for record in await skill_store.list_all() if record.get("name")}
usage_entries = get_skill_usage_entries()
except HTTPException:
raise
except Exception as exc: # noqa: BLE001 - stable API response
logger.exception("Failed to validate strategy-agent skill allowlist")
raise HTTPException(status_code=503, detail="Skill catalog is unavailable") from exc
by_name = {skill.name: skill for skill in skills}
unknown = [name for name in normalized if name not in by_name]
if unknown:
raise HTTPException(
status_code=422,
detail=f"Unknown strategy agent skills: {', '.join(unknown)}",
)
unavailable = [name for name in normalized if not by_name[name].enabled or str((usage_entries.get(name) or {}).get("state") or "active") != "active"]
if unavailable:
raise HTTPException(
status_code=422,
detail=f"Disabled or archived strategy agent skills: {', '.join(unavailable)}",
)
not_global: list[str] = []
for name in normalized:
skill = by_name[name]
if skill.category != SkillCategory.CUSTOM:
continue
record = ownership.get(name)
if record is not None and (record.get("owner_user_id") is None or bool(record.get("published"))):
continue
not_global.append(name)
if not_global:
raise HTTPException(
status_code=422,
detail=(f"Strategy agent skills must be globally available; unpublished or unregistered custom skills: {', '.join(not_global)}"),
)
return normalized
def _ensure_strategy_agent_deletable(agent_id: str) -> None:
"""Keep fixed strategy role ids alive; their SOUL/skills remain editable."""
if _is_strategy_agent_id(agent_id):
raise HTTPException(
status_code=409,
detail=(f"Strategy role agent '{agent_id}' cannot be deleted; edit its SOUL and skills instead"),
)
def _write_agent_config(
agent_id: str,
*,
@@ -307,24 +438,19 @@ def _write_agent_config(
# Persist as plain dicts (dropping blank rows) so config.yaml stays a
# clean mapping rather than a pydantic repr. Accepts already-coerced
# CommonQuestion models or raw dicts/strings.
normalized = [
q if isinstance(q, CommonQuestion) else CommonQuestion.model_validate(q)
for q in common_questions
]
config_data["common_questions"] = [
q.model_dump() for q in normalized if q.prompt.strip()
]
normalized = [q if isinstance(q, CommonQuestion) else CommonQuestion.model_validate(q) for q in common_questions]
config_data["common_questions"] = [q.model_dump() for q in normalized if q.prompt.strip()]
if template_name is not None and template_name.strip():
config_data["template_name"] = template_name.strip()
# 会商席位技能识别强化开关:仅当显式设置时写入(None = 不写, 由 AgentConfig 默认为关)。
if seat_skill_directive is not None:
config_data["seat_skill_directive"] = bool(seat_skill_directive)
if llmwiki_knowledge_base_ids is not None:
config_data["llmwiki_knowledge_base_ids"] = list(
dict.fromkeys(item.strip() for item in llmwiki_knowledge_base_ids if item.strip())
)
with open(config_file, "w", encoding="utf-8") as f:
yaml.dump(config_data, f, default_flow_style=False, allow_unicode=True)
config_data["llmwiki_knowledge_base_ids"] = list(dict.fromkeys(item.strip() for item in llmwiki_knowledge_base_ids if item.strip()))
_atomic_write_text(
config_file,
yaml.safe_dump(config_data, default_flow_style=False, allow_unicode=True),
)
def _load_config_extras(agent_id: str) -> dict[str, Any]:
@@ -358,7 +484,35 @@ def _load_config_extras(agent_id: str) -> dict[str, Any]:
def _write_agent_soul(agent_id: str, soul: str) -> None:
agent_dir = get_paths().agent_dir(agent_id)
agent_dir.mkdir(parents=True, exist_ok=True)
(agent_dir / "SOUL.md").write_text(soul, encoding="utf-8")
_atomic_write_text(agent_dir / "SOUL.md", soul)
def _atomic_write_text(path, content: str) -> None:
"""Durably replace one small agent config file on the same filesystem."""
path.parent.mkdir(parents=True, exist_ok=True)
temp_name: str | None = None
try:
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
dir=path.parent,
prefix=f".{path.name}.",
suffix=".tmp",
delete=False,
) as handle:
temp_name = handle.name
handle.write(content)
handle.flush()
os.fsync(handle.fileno())
os.replace(temp_name, path)
temp_name = None
finally:
if temp_name is not None:
try:
os.unlink(temp_name)
except FileNotFoundError:
pass
async def _sync_legacy_agents(store: AgentStore) -> None:
@@ -413,11 +567,7 @@ async def _backfill_listing_cache(store: AgentStore, pairs: list[tuple[str, Any]
existing = cached.get(agent_id)
if existing is None or existing.get("skills") != want_skills:
await store.replace_skills(agent_id, want_skills)
if (
existing is None
or existing.get("tool_groups") != want_tool_groups
or existing.get("model") != want_model
):
if existing is None or existing.get("tool_groups") != want_tool_groups or existing.get("model") != want_model:
await store.set_extras(agent_id, tool_groups=want_tool_groups, model=want_model)
@@ -631,9 +781,7 @@ async def _build_list_responses(
SOUL.md is skipped (only the edit dialog reads it). An agent absent from
the cache (never mirrored) keeps its ``None`` extras until the next sync.
"""
responses = [
_record_to_response(a, include_soul=False, include_extras=False) for a in records
]
responses = [_record_to_response(a, include_soul=False, include_extras=False) for a in records]
ids = [r.id for r in responses]
extras = await store.get_extras_for(ids)
assignments = await tag_store.list_assignments("agent", ids)
@@ -686,11 +834,7 @@ async def list_agents(
),
admin: bool = Query(
default=False,
description=(
"Admin-only override. When ``true`` and the caller is a system admin, "
"every agent is returned (built-in + every user's agents + private-square "
"entries) regardless of the usual ownership/visibility filter."
),
description=("Admin-only override. When ``true`` and the caller is a system admin, every agent is returned (built-in + every user's agents + private-square entries) regardless of the usual ownership/visibility filter."),
),
) -> AgentsListResponse:
_require_agents_api_enabled()
@@ -744,11 +888,7 @@ async def list_agents(
if owner and owner.strip():
responses = [r for r in responses if r.user_id == owner.strip()]
if square_id:
responses = [
r
for r in responses
if (r.square_id or "") == square_id or (square_is_default and not (r.square_id or ""))
]
responses = [r for r in responses if (r.square_id or "") == square_id or (square_is_default and not (r.square_id or ""))]
responses = _sort_agent_responses(responses, sort)
return AgentsListResponse(agents=responses, total=len(responses), page=1, page_size=len(responses))
except HTTPException:
@@ -773,11 +913,7 @@ async def check_agent_name(name: str) -> dict[str, Any]:
"/agents/selector",
response_model=AgentSelectorResponse,
summary="Agent Homepage Selector",
description=(
"Return the admin-curated agent list shown on the new-chat homepage. "
"Sorted by ``featured_order`` ascending. Visibility-filtered, so an "
"agent the caller can't see (author un-published) is silently omitted."
),
description=("Return the admin-curated agent list shown on the new-chat homepage. Sorted by ``featured_order`` ascending. Visibility-filtered, so an agent the caller can't see (author un-published) is silently omitted."),
)
async def get_agent_selector(request: Request) -> AgentSelectorResponse:
_require_agents_api_enabled()
@@ -789,12 +925,7 @@ async def get_agent_selector(request: Request) -> AgentSelectorResponse:
records = await store.list_featured(user_id)
# Selector dropdown only renders name/description/featured_order — same
# rationale as list_agents: skip soul + extras + self-heal.
return AgentSelectorResponse(
agents=[
_record_to_response(r, include_soul=False, include_extras=False)
for r in records
]
)
return AgentSelectorResponse(agents=[_record_to_response(r, include_soul=False, include_extras=False) for r in records])
except HTTPException:
raise
except Exception as e:
@@ -853,6 +984,11 @@ async def create_agent_endpoint(request: AgentCreateRequest, http_request: Reque
if request.id:
agent_id = _validate_storage_id(request.id)
if _is_strategy_agent_id(agent_id):
raise HTTPException(
status_code=409,
detail=("The 'strategy-selection-' id namespace is reserved; create strategy angle agents through /api/strategy-agents/angles"),
)
if await store.get_any(agent_id) is not None:
raise HTTPException(status_code=409, detail=f"Agent id '{agent_id}' already exists")
else:
@@ -918,14 +1054,18 @@ async def create_agent_endpoint(request: AgentCreateRequest, http_request: Reque
async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request: Request) -> AgentResponse:
_require_agents_api_enabled()
agent_id = _validate_storage_id(agent_id)
if _is_strategy_agent_id(agent_id):
raise HTTPException(
status_code=409,
detail=("Strategy agent configuration must be changed through /api/strategy-agents/{agent_id}"),
)
try:
store = get_agent_store(http_request)
user_id = _current_user_id(http_request)
is_admin = _is_admin_user(http_request)
current, edit_mode = await _get_editable_or_raise(
store, agent_id, user_id, is_admin=is_admin
)
current, edit_mode = await _get_editable_or_raise(store, agent_id, user_id, is_admin=is_admin)
_require_canonical_record_id(agent_id, current)
fields_set = request.model_fields_set
update_data: dict[str, Any] = {}
@@ -953,17 +1093,26 @@ async def update_agent(agent_id: str, request: AgentUpdateRequest, http_request:
next_description = update_data.get("description", current.get("description") or "")
existing_extras = _load_config_extras(agent_id)
next_skills = request.skills if "skills" in fields_set else existing_extras["skills"]
next_model = request.model if "model" in fields_set else existing_extras["model"]
if "skills" in fields_set and _is_strategy_agent_id(agent_id):
next_skills = await _validate_strategy_agent_skills(
request.skills,
app_config=get_config(http_request),
skill_store=get_skill_store(http_request),
)
else:
next_skills = request.skills if "skills" in fields_set else existing_extras["skills"]
if "model" in fields_set and _is_strategy_agent_id(agent_id):
next_model = _validate_strategy_agent_model(
request.model,
app_config=get_config(http_request),
)
else:
next_model = request.model if "model" in fields_set else existing_extras["model"]
next_tool_groups = request.tool_groups if "tool_groups" in fields_set else existing_extras["tool_groups"]
next_common_questions = request.common_questions if "common_questions" in fields_set else existing_extras["common_questions"]
next_template_name = request.template_name if "template_name" in fields_set else existing_extras["template_name"]
next_seat_skill_directive = request.seat_skill_directive if "seat_skill_directive" in fields_set else existing_extras["seat_skill_directive"]
next_llmwiki_ids = (
await _validate_llmwiki_bindings(http_request, request.llmwiki_knowledge_base_ids)
if "llmwiki_knowledge_base_ids" in fields_set
else existing_extras["llmwiki_knowledge_base_ids"]
)
next_llmwiki_ids = await _validate_llmwiki_bindings(http_request, request.llmwiki_knowledge_base_ids) if "llmwiki_knowledge_base_ids" in fields_set else existing_extras["llmwiki_knowledge_base_ids"]
config_dirty = bool(update_data) or any(key in fields_set for key in ("skills", "model", "tool_groups", "common_questions", "template_name", "seat_skill_directive", "llmwiki_knowledge_base_ids"))
if config_dirty:
@@ -1071,14 +1220,19 @@ async def update_user_profile(request: UserProfileUpdateRequest) -> UserProfileR
async def delete_agent(agent_id: str, request: Request) -> None:
_require_agents_api_enabled()
agent_id = _validate_storage_id(agent_id)
if _is_strategy_agent_id(agent_id):
raise HTTPException(
status_code=409,
detail=("Strategy agents cannot be deleted through the generic API; use /api/strategy-agents/{agent_id} to disable an angle"),
)
try:
store = get_agent_store(request)
user_id = _current_user_id(request)
is_admin = _is_admin_user(request)
_, edit_mode = await _get_editable_or_raise(
store, agent_id, user_id, is_admin=is_admin
)
record, edit_mode = await _get_editable_or_raise(store, agent_id, user_id, is_admin=is_admin)
_require_canonical_record_id(agent_id, record)
_ensure_strategy_agent_deletable(agent_id)
if edit_mode == "admin":
deleted = await store.delete_any(agent_id)
@@ -1382,11 +1536,7 @@ class AgentFeaturedReorderRequest(BaseModel):
agent_ids: list[str] = Field(
default_factory=list,
description=(
"Featured agent ids in the desired top→bottom display order. Ids not "
"currently featured are silently skipped, so the client can send the "
"current visible order without first reconciling with the server."
),
description=("Featured agent ids in the desired top→bottom display order. Ids not currently featured are silently skipped, so the client can send the current visible order without first reconciling with the server."),
)
@@ -1394,11 +1544,7 @@ class AgentFeaturedReorderRequest(BaseModel):
"/agents/featured/order",
status_code=204,
summary="Reorder Featured Agents (admin)",
description=(
"Admin-only. Bulk-assign ``featured_order`` to each id in the order received "
"(``0`` is topmost). After this, the homepage selector renders agents in "
"exactly the supplied order. Ids not already featured are ignored."
),
description=("Admin-only. Bulk-assign ``featured_order`` to each id in the order received (``0`` is topmost). After this, the homepage selector renders agents in exactly the supplied order. Ids not already featured are ignored."),
)
async def reorder_featured_agents(
body: AgentFeaturedReorderRequest,
@@ -181,6 +181,7 @@ class KnowledgeBaseCreate(BaseModel):
description: str = Field(default="", max_length=2048)
type: Literal["document", "faq"] = "document"
wiki_enabled: bool = False
embedding_model_id: str | None = Field(default=None, min_length=1, max_length=255)
class KnowledgeBaseUpdate(BaseModel):
@@ -251,7 +252,9 @@ def _store(request: Request) -> LlmWikiStore:
def _schedule_local_wiki_sync(request: Request, mapping: dict[str, Any]) -> None:
service = getattr(request.app.state, "llmwiki_sync_service", None)
config = request.app.state.config.llmwiki.local_wiki_index
config = getattr(request.app.state, "llmwiki_effective_config", None)
if config is None:
config = request.app.state.config.llmwiki.local_wiki_index
if service is not None and config.enabled and config.auto_sync and mapping.get("wiki_index_enabled", True):
from app.gateway.llmwiki_index_scheduler import spawn_local_wiki_index_job
@@ -300,6 +303,74 @@ def _raise_upstream(exc: WeKnoraError) -> None:
raise HTTPException(status_code=status, detail=str(exc)) from None
def _embedding_model_view(source: dict[str, Any]) -> dict[str, Any] | None:
"""Return the small, credential-free model shape needed by the create dialog."""
if str(source.get("type") or "").strip().lower() != "embedding":
return None
model_id = str(source.get("id") or "").strip()
if not model_id:
return None
status = str(source.get("status") or "active").strip().lower() or "active"
explicitly_available = source.get("available")
available = (
bool(explicitly_available)
if isinstance(explicitly_available, bool)
else status in {"active", "available", "enabled", "ready"}
)
parameters = source.get("parameters") if isinstance(source.get("parameters"), dict) else {}
def safe_text(*keys: str) -> str:
for key in keys:
value = source.get(key)
if isinstance(value, (str, int, float)) and str(value).strip():
return str(value).strip()
return ""
nested_provider = parameters.get("provider")
provider = safe_text("provider", "provider_type")
if not provider and isinstance(nested_provider, str):
provider = nested_provider.strip()
if not provider:
provider = safe_text("source", "platform")
nested_model = parameters.get("model") or parameters.get("model_name")
model_name = safe_text("model_name", "model")
if not model_name and isinstance(nested_model, str):
model_name = nested_model.strip()
return {
"id": model_id,
"name": safe_text("display_name", "name", "model_name", "model") or model_name or model_id,
"model_name": model_name,
"provider": provider,
"status": status,
"is_default": source.get("is_default") is True,
"available": available,
}
async def _embedding_models(client: WeKnoraClient) -> list[dict[str, Any]]:
return [view for item in await client.list_models() if (view := _embedding_model_view(item)) is not None]
def _select_embedding_model(models: list[dict[str, Any]], requested_id: str | None) -> str:
available = [item for item in models if item["available"]]
if requested_id:
selected = next((item for item in models if item["id"] == requested_id), None)
if selected is None:
raise WeKnoraError("The selected WeKnora embedding model does not exist", status_code=400)
if not selected["available"]:
raise WeKnoraError("The selected WeKnora embedding model is unavailable", status_code=400)
return requested_id
selected = next((item for item in available if item["is_default"]), None) or next(iter(available), None)
if selected is None:
raise WeKnoraError(
"WeKnora has no available embedding model; configure one in WeKnora before creating a knowledge base",
status_code=400,
)
return str(selected["id"])
def _json_response(payload: Any, *, status_code: int = 200) -> Response:
return Response(
content=json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8"),
@@ -309,7 +380,10 @@ def _json_response(payload: Any, *, status_code: int = 200) -> Response:
def _runtime_for_iframe_proxy(request: Request | None = None):
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None) if request is not None else None)
source_config = None
if request is not None:
source_config = getattr(request.app.state, "llmwiki_source_config", None)
runtime = get_resolved_llmwiki_runtime(source_config)
if not runtime.weknora_enabled:
raise HTTPException(status_code=409, detail="WeKnora mode is not enabled")
if not runtime.web_base_url:
@@ -2485,7 +2559,9 @@ async def get_llmwiki_runtime() -> dict[str, Any]:
@router.get("/api/system-settings/llmwiki")
async def get_llmwiki_settings(request: Request) -> dict[str, Any]:
await _require_admin(request)
app_config = request.app.state.config
from deerflow.config import get_app_config
app_config = get_app_config()
runtime = get_resolved_llmwiki_runtime(app_config)
return {
**_runtime_or_legacy(),
@@ -2508,6 +2584,9 @@ async def update_llmwiki_settings(request: Request, body: LlmWikiSettingsUpdate)
raise HTTPException(status_code=422, detail=str(exc)) from None
current = load_system_settings()
save_system_settings(current.model_copy(update={"llmwiki": override}))
from app.gateway.llmwiki_index_runtime import rebuild_local_wiki_index_runtime
await rebuild_local_wiki_index_runtime(request.app, force=True)
return _runtime_or_legacy()
@@ -2516,13 +2595,16 @@ async def reset_llmwiki_settings(request: Request) -> dict[str, Any]:
await _require_admin(request)
current = load_system_settings()
save_system_settings(current.model_copy(update={"llmwiki": LlmWikiRuntimeOverride()}))
from app.gateway.llmwiki_index_runtime import rebuild_local_wiki_index_runtime
await rebuild_local_wiki_index_runtime(request.app, force=True)
return _runtime_or_legacy()
@router.post("/api/system-settings/llmwiki/test")
async def test_llmwiki_connection(request: Request, body: LlmWikiConnectionTest) -> dict[str, Any]:
await _require_admin(request)
runtime = get_resolved_llmwiki_runtime(request.app.state.config)
runtime = get_resolved_llmwiki_runtime()
if body.api_base_url is not None:
try:
candidate = normalize_service_url(body.api_base_url)
@@ -2677,11 +2759,13 @@ def _public_local_index_summary(
state: dict[str, Any] | None,
) -> dict[str, Any]:
app_state = request.app.state
local_config = getattr(
getattr(getattr(app_state, "config", None), "llmwiki", None),
"local_wiki_index",
None,
)
local_config = getattr(app_state, "llmwiki_effective_config", None)
if local_config is None:
local_config = getattr(
getattr(getattr(app_state, "config", None), "llmwiki", None),
"local_wiki_index",
None,
)
enabled = bool(getattr(local_config, "enabled", False))
current = dict(state or {})
state_name = str(current.get("state") or "not_synced")
@@ -3000,6 +3084,21 @@ async def list_knowledge_bases(
return {"knowledge_bases": knowledge_bases}
@router.get("/api/llmwiki/embedding-models")
async def list_embedding_models(request: Request) -> dict[str, Any]:
"""List selectable WeKnora embedding models without exposing model secrets."""
await _actor(request)
try:
models = await _embedding_models(_client_or_503())
except WeKnoraError as exc:
_raise_upstream(exc)
available = [item for item in models if item["available"]]
default_model = next((item for item in available if item["is_default"]), None) or next(iter(available), None)
default_model_id = str(default_model["id"]) if default_model is not None else None
return {"models": models, "default_model_id": default_model_id}
@router.get("/api/llmwiki/agents/{agent_id}/knowledge-bases")
async def list_agent_knowledge_bases(request: Request, agent_id: str) -> dict[str, Any]:
"""Return the agent-bound bases still visible to the current DeerFlow user."""
@@ -3038,11 +3137,16 @@ async def create_knowledge_base(request: Request, body: KnowledgeBaseCreate) ->
if body.name.strip() == CONVERSATION_DEPOSIT_KB_NAME:
raise HTTPException(status_code=409, detail="对话沉淀是 cmzs 系统保留知识库")
try:
embedding_model_id = _select_embedding_model(
await _embedding_models(client),
body.embedding_model_id,
)
remote = await client.create_knowledge_base(
name=body.name.strip(),
description=body.description,
kb_type=body.type,
wiki_enabled=body.wiki_enabled,
embedding_model_id=embedding_model_id,
)
except WeKnoraError as exc:
_raise_upstream(exc)
@@ -13,6 +13,10 @@ from fastapi import APIRouter, File, Form, HTTPException, Query, Request, Upload
from pydantic import BaseModel, Field
from app.gateway.deps import get_optional_user_from_request
from app.gateway.llmwiki_index_runtime import (
local_encoder_runtime_status,
rebuild_local_wiki_index_runtime,
)
from app.gateway.llmwiki_index_scheduler import spawn_local_wiki_index_job
from deerflow.config.runtime_paths import resolve_path, runtime_home
from deerflow.config.system_settings import load_system_settings, save_system_settings
@@ -53,8 +57,8 @@ async def _actor(request: Request) -> tuple[str, bool]:
return str(user.id), getattr(user, "system_role", None) == "admin"
def _services(request: Request):
config = request.app.state.config.llmwiki.local_wiki_index
async def _services(request: Request):
config = await rebuild_local_wiki_index_runtime(request.app)
if not config.enabled:
raise HTTPException(status_code=409, detail={"code": "WIKI_INDEX_DISABLED", "message": "Local Wiki index is disabled"})
search = getattr(request.app.state, "llmwiki_vector_search", None)
@@ -65,12 +69,12 @@ def _services(request: Request):
def _local_encoder(request: Request):
config = request.app.state.config.llmwiki.local_wiki_index.embedding
embedding = getattr(request.app.state, "llmwiki_embedding", None)
if config.provider != "local":
effective = getattr(request.app.state, "llmwiki_effective_config", None)
if effective is None or effective.embedding.provider != "local":
raise HTTPException(status_code=409, detail={"code": "EMBEDDING_NOT_LOCAL", "message": "当前 embedding provider 不是 local"})
embedding = getattr(request.app.state, "llmwiki_embedding", None)
if embedding is None:
raise HTTPException(status_code=503, detail={"code": "LOCAL_ENCODER_UNAVAILABLE", "message": "本地编码器未配置或未启用"})
raise HTTPException(status_code=503, detail={"code": "LOCAL_ENCODER_UNAVAILABLE", "message": "本地编码器目录未配置、无效或尚未就绪"})
return embedding
@@ -107,33 +111,35 @@ def _copy_encoder_uploads(files: list[UploadFile], paths: list[str], staging: Pa
return total
def _activate_uploaded_encoder(staging: Path) -> Path:
model_root = (runtime_home() / "models").resolve()
def _activate_uploaded_encoder(staging: Path, model_revision: str) -> Path:
model_root = (runtime_home() / "models" / "bge-m3-revisions").resolve()
model_root.mkdir(parents=True, exist_ok=True)
active = (model_root / "bge-m3").resolve()
active = (model_root / model_revision).resolve()
if active.parent != model_root:
raise RuntimeError("编码器目标目录越界")
backup = (model_root / f".bge-m3-backup-{uuid4().hex}").resolve()
if active.exists():
active.replace(backup)
try:
staging.replace(active)
except Exception:
if backup.exists() and not active.exists():
backup.replace(active)
raise
if backup.exists():
shutil.rmtree(backup)
raise RuntimeError("编码器版本目录已存在")
staging.replace(active)
return active
async def _save_encoder_path(request: Request, path: Path) -> dict[str, object]:
embedding = _local_encoder(request)
async def _save_encoder_path(
request: Request,
path: Path,
*,
model_revision: str | None = None,
) -> dict[str, object]:
current = load_system_settings()
encoder_settings = current.local_wiki_encoder.model_copy(update={"model_path": str(path)})
encoder_settings = current.local_wiki_encoder.model_copy(
update={
"mode": "local",
"model_path": str(path),
"model_revision": model_revision or uuid4().hex,
}
)
save_system_settings(current.model_copy(update={"local_wiki_encoder": encoder_settings}))
await embedding.reset_local_encoder(model_path=str(path))
return embedding.local_encoder_status()
await rebuild_local_wiki_index_runtime(request.app, force=True)
return local_encoder_runtime_status(request.app)
@router.get("/local-encoder/status")
@@ -141,7 +147,8 @@ async def local_encoder_status(request: Request) -> dict[str, object]:
_, is_admin = await _actor(request)
if not is_admin:
raise HTTPException(status_code=403, detail="Administrator access required")
return _local_encoder(request).local_encoder_status()
await rebuild_local_wiki_index_runtime(request.app)
return local_encoder_runtime_status(request.app)
@router.post("/local-encoder/start", status_code=202)
@@ -149,8 +156,10 @@ async def start_local_encoder(request: Request) -> dict[str, object]:
_, is_admin = await _actor(request)
if not is_admin:
raise HTTPException(status_code=403, detail="Administrator access required")
await rebuild_local_wiki_index_runtime(request.app)
try:
return _local_encoder(request).start_local_encoder()
_local_encoder(request).start_local_encoder()
return local_encoder_runtime_status(request.app)
except ValueError as exc:
raise HTTPException(status_code=409, detail={"code": "LOCAL_ENCODER_START_REJECTED", "message": str(exc)}) from None
@@ -163,6 +172,20 @@ async def configure_local_encoder(request: Request, body: LocalEncoderConfigRequ
return await _save_encoder_path(request, _validated_encoder_path(body.model_path))
@router.post("/local-encoder/use-configured")
async def use_configured_encoder(request: Request) -> dict[str, object]:
"""Restore the embedding provider declared in config.yaml without restart."""
_, is_admin = await _actor(request)
if not is_admin:
raise HTTPException(status_code=403, detail="Administrator access required")
current = load_system_settings()
encoder_settings = current.local_wiki_encoder.model_copy(update={"mode": "configured"})
save_system_settings(current.model_copy(update={"local_wiki_encoder": encoder_settings}))
await rebuild_local_wiki_index_runtime(request.app, force=True)
return local_encoder_runtime_status(request.app)
@router.post("/local-encoder/upload")
async def upload_local_encoder(
request: Request,
@@ -191,9 +214,17 @@ async def upload_local_encoder(
try:
uploaded_bytes = await asyncio.to_thread(_copy_encoder_uploads, files, paths, staging)
_validated_encoder_path(str(staging))
await _local_encoder(request).reset_local_encoder()
active = await asyncio.to_thread(_activate_uploaded_encoder, staging)
status = await _save_encoder_path(request, active)
model_revision = uuid4().hex
active = await asyncio.to_thread(
_activate_uploaded_encoder,
staging,
model_revision,
)
status = await _save_encoder_path(
request,
active,
model_revision=model_revision,
)
return {**status, "uploaded_files": len(files), "uploaded_bytes": uploaded_bytes}
except HTTPException:
raise
@@ -224,7 +255,7 @@ async def _authorized_mappings(request: Request, mapping_ids: list[str]) -> tupl
@router.post("/wiki-vector-search")
async def wiki_vector_search(request: Request, body: WikiVectorSearchRequest) -> dict[str, Any]:
config, search, _ = _services(request)
config, search, _ = await _services(request)
if not config.internal_search_enabled:
raise HTTPException(status_code=409, detail={"code": "WIKI_INTERNAL_SEARCH_DISABLED", "message": "Local Wiki search is not enabled"})
rows, user_id, is_admin = await _authorized_mappings(request, body.knowledge_base_ids)
@@ -250,7 +281,7 @@ async def local_wiki_page(
include_provenance: bool = Query(default=False),
include_archived: bool = Query(default=False),
) -> dict[str, Any]:
_, _, index_store = _services(request)
_, _, index_store = await _services(request)
user_id, is_admin = await _actor(request)
mapping = await request.app.state.llmwiki_store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
if mapping is None:
@@ -292,7 +323,7 @@ async def wiki_index_status(request: Request) -> dict[str, Any]:
_, is_admin = await _actor(request)
if not is_admin:
raise HTTPException(status_code=403, detail="Administrator access required")
config = request.app.state.config.llmwiki.local_wiki_index
config = await rebuild_local_wiki_index_runtime(request.app)
index_store = getattr(request.app.state, "llmwiki_index_store", None)
if index_store is None:
raise HTTPException(status_code=503, detail={"code": "WIKI_INDEX_UNAVAILABLE"})
@@ -352,7 +383,7 @@ async def _run_one_sync(app, mapping: dict[str, Any], *, force: bool) -> None:
@router.post("/wiki-index/sync", status_code=202)
async def sync_wiki_index(request: Request, body: WikiIndexActionRequest) -> dict[str, Any]:
_services(request)
await _services(request)
_, is_admin = await _actor(request)
if not is_admin:
raise HTTPException(status_code=403, detail="Administrator access required")
@@ -363,7 +394,7 @@ async def sync_wiki_index(request: Request, body: WikiIndexActionRequest) -> dic
@router.post("/wiki-index/rebuild", status_code=202)
async def rebuild_wiki_index(request: Request, body: WikiIndexActionRequest) -> dict[str, Any]:
_, _, index_store = _services(request)
_, _, index_store = await _services(request)
_, is_admin = await _actor(request)
if not is_admin:
raise HTTPException(status_code=403, detail="Administrator access required")
@@ -388,6 +419,7 @@ async def rebuild_one_wiki_index(request: Request, mapping_id: str, reason: str
@router.get("/knowledge-bases/{mapping_id}/wiki-index-status")
async def knowledge_base_wiki_index_status(request: Request, mapping_id: str) -> dict[str, Any]:
config = await rebuild_local_wiki_index_runtime(request.app)
user_id, is_admin = await _actor(request)
mapping = await request.app.state.llmwiki_store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
if mapping is None:
@@ -430,7 +462,7 @@ async def knowledge_base_wiki_index_status(request: Request, mapping_id: str) ->
return {
**state,
"knowledge_base_id": mapping_id,
"local_index_enabled": bool(request.app.state.config.llmwiki.local_wiki_index.enabled),
"local_index_enabled": bool(config.enabled),
"wiki_index_enabled": mapping.get("wiki_index_enabled", True),
"fully_vectorized": fully_vectorized,
"retrieval_mode": "vector" if fully_vectorized else "wiki_api",
@@ -448,7 +480,7 @@ async def knowledge_base_wiki_index_status(request: Request, mapping_id: str) ->
@router.post("/knowledge-bases/{mapping_id}/vectorize", status_code=202)
async def vectorize_knowledge_base(request: Request, mapping_id: str) -> dict[str, Any]:
config = request.app.state.config.llmwiki.local_wiki_index
config = await rebuild_local_wiki_index_runtime(request.app)
if not config.enabled:
raise HTTPException(
status_code=409,
@@ -68,10 +68,14 @@ class TaskUpdateRequest(BaseModel):
brief: str | None = None
agents: list[TaskAgentSchema] | None = None
messages: list[dict[str, Any]] | None = None
# Optional optimistic-concurrency token. Legacy callers may omit it; the
# strategy-project workbench always sends the revision it loaded/saved.
expected_updated_at: datetime | None = None
class TaskEnsureRequest(BaseModel):
"""get-or-create:宿主传的 taskId 不固定,新 id 直接**初始化**一条空记录返回(不 404)。"""
id: str = Field(min_length=1, max_length=64)
title: str = Field(default="", max_length=512)
brief: str | None = None
@@ -162,12 +166,24 @@ async def update_task(request: Request, task_id: str, body: TaskUpdateRequest) -
store = _get_store(request)
user_id = _current_user_id(request)
fields = body.model_dump(exclude_unset=True)
expected_updated_at = fields.pop("expected_updated_at", None)
# Normalize agents (pydantic models) to plain dicts for JSON storage.
if "agents" in fields and fields["agents"] is not None:
fields["agents"] = [a if isinstance(a, dict) else a.model_dump() for a in body.agents or []]
row = await store.update_task(task_id, user_id, **fields)
row = await store.update_task(
task_id,
user_id,
expected_updated_at=expected_updated_at,
**fields,
)
if row is None:
raise HTTPException(status_code=404, detail="Task not found")
existing = await store.get_task(task_id, user_id)
if existing is None:
raise HTTPException(status_code=404, detail="Task not found")
raise HTTPException(
status_code=409,
detail="项目已在其他页面更新,请选择恢复本地草稿或使用服务器版本",
)
return TaskResponse(**row)
@@ -31,18 +31,21 @@ from __future__ import annotations
import asyncio
import json
import logging
from typing import Any, AsyncIterator
from collections.abc import AsyncIterator
from typing import Any
import httpx
from fastapi import APIRouter, Body, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from app.gateway.roundtable_model_fallback import is_llm_error_message, reason_text
from app.gateway.routers import agents as agents_router
from app.gateway.routers._strategy_selection_seed import STRATEGY_AGENT_ID_PREFIX
from app.gateway.routers.multi_agent import (
_ROUNDTABLE_THREAD_METADATA,
_auth_headers,
_create_thread,
_flatten_content,
_loopback_base,
_make_loopback_client,
_parse_last_messages,
@@ -50,7 +53,9 @@ from app.gateway.routers.multi_agent import (
_stream_upstream,
_validate_agent_id,
)
from deerflow.agents.roundtable_orchestrator.delivery import visible_seat_text
from deerflow.tools.builtins.clarification_utils import resolve_allow_multiple
from deerflow.utils.stream_text import InlineThinkTagFilter
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/parallel-agents", tags=["parallel-agents"])
@@ -58,6 +63,18 @@ router = APIRouter(prefix="/api/parallel-agents", tags=["parallel-agents"])
# Soft cap on parallel degree — protects remote model concurrency + sandbox.
MAX_PARALLEL_AGENTS = 8
# Strategy roles are expected to return a substantive task/strategy/evaluation,
# not merely punctuation or a broken provider placeholder. Counting Unicode
# alphanumerics keeps the check language-neutral while rejecting outputs such as
# a long run of "!!!", which otherwise look non-empty to the UI.
MIN_MEANINGFUL_CONTENT_CHARS = 20
def _has_substantive_content(content: str) -> bool:
visible = visible_seat_text(content)
return sum(character.isalnum() for character in visible) >= MIN_MEANINGFUL_CONTENT_CHARS
# 禁两类工具:①**写文件**(write_file / str_replace)—— 并行席位互相写文件会打架;
# ②**派活 / 脚本逃逸**(agent_orchestration / bash)—— 本面板里每个智能体只做自己那
# 份分析交付,绝不该扮演「总控」去调度别人。弱模型(deepseek 等)拿到带「总控 / 业务
@@ -66,10 +83,10 @@ MAX_PARALLEL_AGENTS = 8
# glob / bash 翻找派活脚本)。物理禁掉这两类即可根治。其余只读工具(ls / read_file /
# 检索)与 ask_clarification(协作卡片)保留:既不互相打架,又能正常出步骤条 / 要澄清。
_EXCLUDED_TOOLS: list[str] = [
"write_file", # 并行不互相写文件打架
"write_file", # 并行不互相写文件打架
"str_replace",
"agent_orchestration", # 子智能体不派活 / 不调度(防模仿总控)
"bash", # 禁跑脚本(防绕道脚本调内部接口模仿派活)
"bash", # 禁跑脚本(防绕道脚本调内部接口模仿派活)
]
@@ -105,25 +122,126 @@ async def issue_token(body: _TokenRequest) -> _TokenResponse:
# ── frame normalisation (mirrors multi-agent.ts) ─────────────────────────────
def _extract_ai_text(chunk: dict[str, Any]) -> str:
"""AI text delta from a ``messages-tuple`` chunk obj; "" when not AI text."""
def _extract_ai_stream_parts(chunk: dict[str, Any]) -> tuple[str, str]:
"""Return ``(answer, reasoning)`` deltas from one AI message chunk.
OpenAI-compatible providers do not agree on where displayable reasoning
lives. Most put it in ``additional_kwargs.reasoning_content`` (or the
legacy ``reasoning`` key), while a few return typed content blocks. Keep
that channel separate from the answer so the strategy workbench can show
progress without mixing internal trace text into the generated strategy.
Literal ``<think>`` blocks in the normal text channel are handled by the
per-agent :class:`InlineThinkTagFilter` in ``_consume_agent`` because tags
may be split across arbitrary SSE chunks.
"""
type_ = str(chunk.get("type") or "").lower()
if type_ not in ("ai", "aimessagechunk", "aimessage"):
return ""
return "", ""
content = chunk.get("content")
answer_parts: list[str] = []
reasoning_parts: list[str] = []
if isinstance(content, str):
return content
if isinstance(content, list):
out: list[str] = []
answer_parts.append(content)
elif isinstance(content, list):
for item in content:
if isinstance(item, str):
out.append(item)
answer_parts.append(item)
elif isinstance(item, dict):
t = item.get("text")
if isinstance(t, str):
out.append(t)
return "".join(out)
return ""
block_type = str(item.get("type") or "").lower()
value = item.get("thinking")
if not isinstance(value, str):
value = item.get("text")
if not isinstance(value, str):
value = item.get("content")
if not isinstance(value, str):
continue
if block_type in {
"thinking",
"thinking_delta",
"reasoning",
"reasoning_delta",
}:
reasoning_parts.append(value)
else:
answer_parts.append(value)
additional_kwargs = chunk.get("additional_kwargs")
direct_reasoning = chunk.get("reasoning_content") or chunk.get("reasoning")
nested_reasoning: object = None
if isinstance(additional_kwargs, dict):
nested_reasoning = additional_kwargs.get("reasoning_content") or additional_kwargs.get("reasoning")
raw_reasoning = nested_reasoning or direct_reasoning
if isinstance(raw_reasoning, str):
reasoning_parts.append(raw_reasoning)
return "".join(answer_parts), "".join(reasoning_parts)
def _extract_ai_text(chunk: dict[str, Any]) -> str:
"""Compatibility helper returning only the ordinary AI text delta."""
return _extract_ai_stream_parts(chunk)[0]
def _split_final_inline_reasoning(content: str) -> tuple[str, str]:
"""Split complete inline thinking, including template-injected close tags.
Some reasoning chat templates insert ``<think>`` before generation, so the
provider returns only ``reasoning</think>answer``. The streaming filter
cannot classify that prefix before the closing marker arrives, but the
terminal values frame can. Returning the recovered trace here also lets
the client correct any transient streaming classification.
"""
if not content:
return "", ""
working = content
prefix_reasoning = ""
lowered = working.lower()
first_open = lowered.find("<think>")
first_close = lowered.find("</think>")
if first_close >= 0 and (first_open < 0 or first_close < first_open):
prefix_reasoning = working[:first_close]
working = working[first_close + len("</think>") :]
text_filter = InlineThinkTagFilter()
visible, reasoning = text_filter.push_parts(working)
visible_tail, reasoning_tail = text_filter.finish_parts()
final_reasoning = "".join(part for part in (prefix_reasoning, reasoning, reasoning_tail) if part)
# ``visible_seat_text`` remains the canonical complete-output cleanup for
# malformed/multiple tags; the filter-derived visible value is only a
# fallback for an otherwise empty canonical result.
final_visible = visible_seat_text(content) or f"{visible}{visible_tail}".strip()
return final_visible, final_reasoning.strip()
def _should_probe_implicit_reasoning(
request: Request,
agent_id: str,
requested_model: str,
) -> bool:
"""Enable close-only probing only for a thinking-capable effective model.
Ordinary models must stream untagged answer chunks immediately. The
stricter prelude buffer is reserved for providers that advertise thinking
support and may use templates where ``<think>`` is injected outside the
returned content while ``</think>`` remains in-band.
"""
config = getattr(request.app.state, "config", None)
models = list(getattr(config, "models", None) or [])
effective_name = requested_model.strip()
if not effective_name:
try:
effective_name = str(agents_router._load_config_extras(agent_id).get("model") or "").strip()
except Exception:
effective_name = ""
if not effective_name and models:
effective_name = str(getattr(models[0], "name", "") or "")
selected = next(
(entry for entry in models if str(getattr(entry, "name", "") or "") == effective_name),
None,
)
return bool(selected and getattr(selected, "supports_thinking", False))
def _ingest_tool_chunks(chunks: Any, acc: dict[int, dict[str, Any]]) -> None:
@@ -231,7 +349,12 @@ async def _run_one_agent(
if isinstance(msg, dict) and (msg.get("type") or "").lower() in ("ai", "aimessage", "aimessagechunk"):
last_ai = msg
break
content = _flatten_content(last_ai.get("content", ""))
failure_reason = is_llm_error_message(last_ai)
if failure_reason:
raise RuntimeError(f"模型调用失败:{reason_text(failure_reason)}")
answer_channel, structured_reasoning = _extract_ai_stream_parts(last_ai)
content, inline_reasoning = _split_final_inline_reasoning(answer_channel)
reasoning = "\n\n".join(part.strip() for part in (structured_reasoning, inline_reasoning) if part.strip())
# 收集**所有** AI 消息的 tool_calls(全量 args,按顺序)—— 用于在收口时补发所有步骤
# (含 ls / skill_list 这类无参/中间步骤),保证步骤条不丢步、且 args 完整。
final_tool_calls: list[dict[str, Any]] = []
@@ -246,13 +369,17 @@ async def _run_one_agent(
yield {
"type": "clarification",
"content": content,
"reasoning": reasoning,
"clarification": clarification,
"tool_calls": final_tool_calls,
}
else:
if not _has_substantive_content(content):
raise RuntimeError("智能体未生成有效正文,请重试或更换全局模型")
yield {
"status": f"子智能体 {agent_id} 完成",
"content": content,
"reasoning": reasoning,
"agent_name": agent_id,
"tool_calls": final_tool_calls,
}
@@ -264,10 +391,7 @@ def _build_task_message(task_name: str, brief: str) -> str:
parts.append(f"任务名称:{task_name.strip()}")
if brief.strip():
parts.append(f"任务简介:\n{brief.strip()}")
parts.append(
"请你从你的专业职责出发,针对以上任务给出你的分析与交付。"
"本轮为**并行**研讨:请把你的产出**直接写在回复正文里**(本轮禁止写文件)。"
)
parts.append("请你从你的专业职责出发,针对以上任务给出你的分析与交付。本轮为**并行**研讨:请把你的产出**直接写在回复正文里**(本轮禁止写文件)。")
# 角色边界护栏:本面板里每个智能体都是**执行席位**,不是总控 / 指挥。即便任务里带有
# 「总控 / 业务链条 / 调度 / 派活」字样,也只完成属于自己的那份交付,严禁模仿总控派活。
parts.append(
@@ -310,6 +434,41 @@ async def run_parallel_agents_stream(request: Request, payload: dict = Body(...)
status_code=400,
detail=f"并行智能体数量上限 {MAX_PARALLEL_AGENTS}(当前 {len(agent_ids)})",
)
role_store = getattr(request.app.state, "strategy_agent_role_store", None)
agent_store = getattr(request.app.state, "agent_store", None)
strategy_ids = [agent_id for agent_id in agent_ids if agent_id.casefold().startswith(STRATEGY_AGENT_ID_PREFIX.casefold())]
if strategy_ids and role_store is not None and agent_store is not None:
# Every worker owns an independent memory store. Synchronize from
# durable folders/sidecars immediately before execution so a request
# routed to another worker cannot see random 404s or revived angles.
from app.gateway.routers.strategy_agents import _ensure_registry
await _ensure_registry(agent_store, role_store)
for agent_id in agent_ids:
if not agent_id.casefold().startswith(STRATEGY_AGENT_ID_PREFIX.casefold()):
continue
if role_store is None or agent_store is None:
raise HTTPException(
status_code=503,
detail="策略智能体分类或配置服务不可用",
)
role = await role_store.get_role(agent_id)
if role is None:
raise HTTPException(
status_code=400,
detail=f"未注册的策略智能体:{agent_id}",
)
if not bool(role.get("enabled")):
raise HTTPException(
status_code=409,
detail=f"策略智能体已停用:{agent_id}",
)
record = await agent_store.get_any(agent_id)
if record is None or str(record.get("id") or "") != agent_id or record.get("user_id") is not None:
raise HTTPException(
status_code=409,
detail=f"策略智能体必须由系统全局持有:{agent_id}",
)
if not task_name and not brief:
raise HTTPException(status_code=400, detail="任务名称与简介至少填写其一")
@@ -327,8 +486,54 @@ async def run_parallel_agents_stream(request: Request, payload: dict = Body(...)
emitted_tools: set[str] = set() # 已发过 step 的 tool_call id(去重)
tool_acc: dict[int, dict[str, Any]] = {} # tool_call_chunks 累积
buffer: list[str] = []
raw_answer_buffer: list[str] = []
probe_implicit_close = await asyncio.to_thread(
_should_probe_implicit_reasoning,
request,
agent_id,
model_name,
)
inline_thinking = InlineThinkTagFilter(allow_implicit_close=probe_implicit_close)
reasoning_emitted = False
done_sent = False
async def _emit_text_parts(text: str, provider_reasoning: str = "") -> None:
nonlocal reasoning_emitted
if text:
raw_answer_buffer.append(text)
visible_prelude = inline_thinking.confirm_visible_channel() if provider_reasoning else ""
visible, inline_reasoning = inline_thinking.push_parts(text)
reasoning = f"{provider_reasoning}{inline_reasoning}"
if reasoning:
reasoning_emitted = True
await queue.put(
{
"type": "reasoning_delta",
"agent_id": agent_id,
"text": reasoning,
}
)
confirmed_visible = f"{visible_prelude}{visible}"
if confirmed_visible:
buffer.append(confirmed_visible)
await queue.put({"type": "delta", "agent_id": agent_id, "text": confirmed_visible})
async def _flush_text_filter() -> None:
nonlocal reasoning_emitted
visible, reasoning = inline_thinking.finish_parts()
if reasoning:
reasoning_emitted = True
await queue.put(
{
"type": "reasoning_delta",
"agent_id": agent_id,
"text": reasoning,
}
)
if visible:
buffer.append(visible)
await queue.put({"type": "delta", "agent_id": agent_id, "text": visible})
async def _emit_tool(tc: Any, allow_empty: bool = False) -> None:
"""一个 tool_call(含完整 args)→ 一条 step 事件;按 id 去重。
@@ -350,9 +555,7 @@ async def run_parallel_agents_stream(request: Request, payload: dict = Body(...)
if key_id in emitted_tools:
return
emitted_tools.add(key_id)
await queue.put(
{"type": "step", "agent_id": agent_id, "key": f"tool_calling:{name}", "label": name, "args": args}
)
await queue.put({"type": "step", "agent_id": agent_id, "key": f"tool_calling:{name}", "label": name, "args": args})
try:
thread_id = await _create_thread(client, base, headers, metadata=_ROUNDTABLE_THREAD_METADATA)
@@ -362,22 +565,32 @@ async def run_parallel_agents_stream(request: Request, payload: dict = Body(...)
async for chunk in gen:
if isinstance(chunk, dict):
# 终态:先用最终 tool_calls(全量 args)补发漏掉的 step,再发收口/澄清。
await _flush_text_filter()
for tc in chunk.get("tool_calls") or []:
await _emit_tool(tc, allow_empty=True)
final_content = visible_seat_text(str(chunk.get("content") or ""))
final_reasoning = str(chunk.get("reasoning") or "")
if final_reasoning and not reasoning_emitted:
reasoning_emitted = True
await queue.put(
{
"type": "reasoning_delta",
"agent_id": agent_id,
"text": final_reasoning,
}
)
if chunk.get("type") == "clarification":
clar = chunk.get("clarification") or {}
await queue.put(
{
"type": "agent_clarification",
"agent_id": agent_id,
"content": str(chunk.get("content") or ""),
"content": final_content,
**clar,
}
)
else:
await queue.put(
{"type": "agent_done", "agent_id": agent_id, "content": str(chunk.get("content") or "")}
)
await queue.put({"type": "agent_done", "agent_id": agent_id, "content": final_content})
done_sent = True
continue
line = chunk.strip() if isinstance(chunk, str) else ""
@@ -396,10 +609,9 @@ async def run_parallel_agents_stream(request: Request, payload: dict = Body(...)
if last_event == "messages" and isinstance(parsed, list) and parsed:
obj = parsed[0]
if isinstance(obj, dict):
text = _extract_ai_text(obj)
if text:
buffer.append(text)
await queue.put({"type": "delta", "agent_id": agent_id, "text": text})
text, reasoning = _extract_ai_stream_parts(obj)
if text or reasoning:
await _emit_text_parts(text, reasoning)
# 完整 tool_calls(部分模型直接给)+ 累积 chunks 后的合成调用,都尝试发 step。
for tc in obj.get("tool_calls") or []:
await _emit_tool(tc)
@@ -408,8 +620,30 @@ async def run_parallel_agents_stream(request: Request, payload: dict = Body(...)
await _emit_tool(tc)
if not done_sent:
# 流被抢占/截断,没收到终态帧 —— 用已累积正文兜底收口本席位。
await _flush_text_filter()
buffered_content, recovered_reasoning = _split_final_inline_reasoning("".join(raw_answer_buffer))
if not buffered_content:
buffered_content = "".join(buffer)
if recovered_reasoning and not reasoning_emitted:
reasoning_emitted = True
await queue.put(
{
"type": "reasoning_delta",
"agent_id": agent_id,
"text": recovered_reasoning,
}
)
failure_reason = is_llm_error_message({"type": "ai", "content": buffered_content})
if failure_reason:
raise RuntimeError(f"模型调用失败:{reason_text(failure_reason)}")
if not _has_substantive_content(buffered_content):
raise RuntimeError("智能体未生成有效正文,请重试或更换全局模型")
await queue.put(
{"type": "agent_done", "agent_id": agent_id, "content": "".join(buffer)}
{
"type": "agent_done",
"agent_id": agent_id,
"content": buffered_content,
}
)
except Exception as exc: # noqa: BLE001 — 单个 agent 失败不拖垮其它
logger.warning("[parallel-agents] %s failed: %r", agent_id, exc)
@@ -71,6 +71,10 @@ class ChainSeatSchema(BaseModel):
# 岗位协同专用的席位岗位归属。字段保存在既有 seats JSON 中,普通圆桌编排
# 忽略它;显式声明可确保 Pydantic 在 create/update 的 model_dump() 时不剥掉。
position_id: str | None = Field(default=None, max_length=64)
# 该席位提交过程产物时传给外部任务系统的 position 值(8BF 会商页用)。
# 缺省 None = 提交时回退 0。保存在既有 seats JSON 中,无需迁移;显式声明
# 可确保 Pydantic 在 create/update 的 model_dump() 时不剥掉。
artifact_position: int | None = None
# 该席位在业务链中的岗位职责。会商智能体配置面板只读展示此字段;
# 保存在既有 seats JSON 中,无需迁移。空/None = 未填写。
job_responsibilities: str | None = Field(default=None, max_length=4000)
@@ -0,0 +1,802 @@
"""Global strategy-selection agent role management.
The strategy workbench owns three fixed roles (intent, judge and solution)
plus an administrator-managed set of angle roles. Role metadata is stored in
``strategy_agent_roles`` while model/SOUL/skills remain in the normal
folder-backed agent configuration, so the same configuration is shared by
every project and every user.
"""
from __future__ import annotations
import asyncio
import logging
import os
import shutil
import weakref
from contextlib import asynccontextmanager
from datetime import datetime
from typing import Literal
from uuid import uuid4
import yaml
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field
from app.gateway.auth.disabled_mode import is_auth_disabled
from app.gateway.deps import (
get_agent_store,
get_config,
get_current_user_from_request,
get_skill_store,
get_strategy_agent_role_store,
)
from app.gateway.routers import agents as agents_router
from app.gateway.routers._strategy_selection_seed import (
STRATEGY_ANGLE_AGENT_PREFIX,
default_strategy_agent_roles,
ensure_strategy_selection_agents,
)
from deerflow.config.paths import get_paths
from deerflow.persistence.agents import AgentStore
from deerflow.persistence.strategy_agent_roles import (
MemoryStrategyAgentRoleStore,
StrategyAgentRoleStore,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/strategy-agents", tags=["strategy-agents"])
StrategyAgentCategory = Literal["intent", "angle", "judge", "solution"]
_DYNAMIC_ANGLE_TOOL_GROUPS = ["web", "file:read"]
_ROLE_SIDECAR = ".strategy-role.yaml"
_PROCESS_LOCKS: weakref.WeakValueDictionary[str, asyncio.Lock] = weakref.WeakValueDictionary()
def _process_lock(agent_id: str) -> asyncio.Lock:
key = agent_id.casefold()
lock = _PROCESS_LOCKS.get(key)
if lock is None:
lock = asyncio.Lock()
_PROCESS_LOCKS[key] = lock
return lock
def _open_cross_process_lock(agent_id: str):
lock_dir = get_paths().agents_dir / ".strategy-locks"
lock_dir.mkdir(parents=True, exist_ok=True)
path = lock_dir / f"{agent_id}.lock"
handle = open(path, "a+b") # noqa: SIM115 - owned by async context below
if path.stat().st_size == 0:
handle.write(b"\0")
handle.flush()
return handle
def _try_cross_process_lock(handle) -> bool:
handle.seek(0)
try:
if os.name == "nt":
import msvcrt
msvcrt.locking(handle.fileno(), msvcrt.LK_NBLCK, 1)
else:
import fcntl
fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
return True
except OSError:
return False
def _release_cross_process_lock(handle) -> None:
try:
handle.seek(0)
if os.name == "nt":
import msvcrt
msvcrt.locking(handle.fileno(), msvcrt.LK_UNLCK, 1)
else:
import fcntl
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
finally:
handle.close()
@asynccontextmanager
async def _strategy_mutation_guard(agent_id: str):
"""Serialize one strategy agent across coroutines and worker processes.
The zero-byte lock-file namespace is durable, but OS locks are tied to the
open handle and are automatically released if a worker dies. The local
lock table uses weak values so one-off dynamic ids do not leak forever.
"""
async with _process_lock(agent_id):
handle = _open_cross_process_lock(agent_id)
acquired = False
try:
while not acquired:
acquired = _try_cross_process_lock(handle)
if not acquired:
await asyncio.sleep(0.05)
yield
finally:
if acquired:
# Unlock synchronously: this is a tiny OS call and must not be
# skipped if the request is already being cancelled.
_release_cross_process_lock(handle)
else:
handle.close()
def _write_role_sidecar(agent_id: str, role: dict) -> None:
payload = {
"agent_id": agent_id,
"category": "angle",
"fixed": False,
"enabled": bool(role.get("enabled", True)),
"sort_order": int(role.get("sort_order", 0)),
}
agents_router._atomic_write_text(
get_paths().agent_dir(agent_id) / _ROLE_SIDECAR,
yaml.safe_dump(payload, allow_unicode=True, sort_keys=False),
)
def _read_optional_text(path) -> str | None:
return path.read_text(encoding="utf-8") if path.is_file() else None
def _restore_optional_text(path, content: str | None) -> None:
if content is None:
path.unlink(missing_ok=True)
return
agents_router._atomic_write_text(path, content)
async def _rollback_strategy_update(
*,
agent_id: str,
agent_store: AgentStore,
role_store: StrategyAgentRoleStore,
record: dict,
role: dict,
extras: dict,
file_snapshot: dict,
) -> None:
"""Best-effort compensation for the mixed DB/filesystem update."""
try:
agent_dir = get_paths().agent_dir(agent_id)
await asyncio.to_thread(
_restore_optional_text,
agent_dir / "config.yaml",
file_snapshot["config"],
)
await asyncio.to_thread(
_restore_optional_text,
agent_dir / "SOUL.md",
file_snapshot["soul"],
)
await asyncio.to_thread(
_restore_optional_text,
agent_dir / _ROLE_SIDECAR,
file_snapshot["sidecar"],
)
await agent_store.update_any(
agent_id,
{
"name": record["name"],
"description": record.get("description") or "",
},
)
await agent_store.replace_skills(agent_id, extras["skills"])
await agent_store.set_extras(
agent_id,
tool_groups=extras["tool_groups"],
model=extras["model"],
)
await role_store.update_role(
agent_id,
enabled=bool(role["enabled"]),
sort_order=int(role["sort_order"]),
)
except Exception:
logger.exception("Failed to compensate strategy-agent update for '%s'", agent_id)
async def _recover_memory_angle_roles(role_store: StrategyAgentRoleStore) -> None:
"""Recover dynamic role metadata after a memory-mode process restart."""
if not isinstance(role_store, MemoryStrategyAgentRoleStore):
return
agents_dir = get_paths().agents_dir
if not agents_dir.is_dir():
return
for agent_dir in agents_dir.iterdir():
sidecar = agent_dir / _ROLE_SIDECAR
if not sidecar.is_file():
continue
try:
raw = yaml.safe_load(sidecar.read_text(encoding="utf-8")) or {}
agent_id = str(raw.get("agent_id") or "")
if agent_id != agent_dir.name or not agent_id.startswith(STRATEGY_ANGLE_AGENT_PREFIX) or raw.get("category") != "angle" or bool(raw.get("fixed", False)):
raise ValueError("invalid dynamic angle sidecar")
existing = await role_store.get_role(agent_id)
if existing is None:
await role_store.create_role(
{
"agent_id": agent_id,
"category": "angle",
"fixed": False,
"enabled": bool(raw.get("enabled", True)),
"sort_order": int(raw.get("sort_order", 0)),
}
)
elif not bool(existing.get("fixed")) and existing.get("category") == "angle":
# Memory stores are rebuilt with code defaults first. The
# sidecar is the durable authority for every non-fixed angle,
# including the six shipped angles, so disabled defaults do
# not silently revive after a worker restart.
await role_store.update_role(
agent_id,
enabled=bool(raw.get("enabled", True)),
sort_order=int(raw.get("sort_order", 0)),
)
except Exception:
logger.exception("Ignoring invalid strategy role sidecar at %s", sidecar)
def _default_angle_soul(name: str, description: str) -> str:
focus = description.strip() or "围绕任务目标,从本角度给出独立、可执行的策略建议"
return (
f"# 角色\n\n你是“{name}”,负责{focus}。\n\n"
"# 工作要求\n\n"
"1. 先核对任务目标、约束、前置条件和时间范围;信息不足时明确假设。\n"
"2. 只从自己的专业角度分析,不替代其他角度,也不冒充评判或方案整理角色。\n"
"3. 输出具体策略、实施步骤、预期收益、主要风险、触发条件和退出条件。\n"
"4. 对时效性事实优先检索核验并标注依据;不得编造来源。\n"
"5. 遵守法律、安全与非升级边界,最终交付使用清晰的 Markdown。\n"
)
class StrategyAgentItem(BaseModel):
id: str
category: StrategyAgentCategory
fixed: bool
enabled: bool
sort_order: int
name: str
description: str = ""
model: str | None = None
soul: str | None = None
skills: list[str] | None = None
created_at: datetime | str | None = None
updated_at: datetime | str | None = None
class StrategyAgentListResponse(BaseModel):
agents: list[StrategyAgentItem]
class StrategyAngleCreateRequest(BaseModel):
name: str = Field(min_length=1, max_length=100)
description: str = Field(default="", max_length=2000)
soul: str = Field(default="", max_length=200_000)
model: str | None = Field(default=None, max_length=200)
skills: list[str] = Field(default_factory=list, max_length=100)
sort_order: int | None = Field(default=None, ge=-10_000, le=10_000)
class StrategyAgentUpdateRequest(BaseModel):
name: str | None = Field(default=None, max_length=100)
description: str | None = Field(default=None, max_length=2000)
soul: str | None = Field(default=None, max_length=200_000)
model: str | None = Field(default=None, max_length=200)
skills: list[str] | None = Field(default=None, max_length=100)
enabled: bool | None = None
sort_order: int | None = Field(default=None, ge=-10_000, le=10_000)
async def _require_admin(request: Request) -> None:
"""Require an administrator, except in explicit auth-disabled mode."""
user = getattr(request.state, "user", None)
if user is None:
if is_auth_disabled():
return
user = await get_current_user_from_request(request)
if getattr(user, "system_role", None) != "admin":
raise HTTPException(status_code=403, detail="策略智能体管理仅限管理员")
def _normalize_model(request: Request, raw_model: str | None) -> str | None:
return agents_router._validate_strategy_agent_model(
raw_model,
app_config=get_config(request),
)
async def _ensure_registry(
agent_store: AgentStore,
role_store: StrategyAgentRoleStore,
) -> None:
"""Seed missing role metadata and repair missing default agent rows."""
defaults = default_strategy_agent_roles()
await role_store.ensure_defaults(defaults)
await _recover_memory_angle_roles(role_store)
default_ids = [str(row["agent_id"]) for row in defaults]
for agent_id in default_ids:
existing = await agent_store.get_any(agent_id)
if existing is not None and existing.get("user_id") is not None:
# Never let legacy sync's ensure_builtin semantics silently treat a
# user-owned collision as one of the fixed global roles.
raise HTTPException(
status_code=409,
detail=f"Reserved strategy agent '{agent_id}' is owned by a user",
)
roles = await role_store.list_roles(include_disabled=True)
missing = [str(role["agent_id"]) for role in roles if await agent_store.get_any(str(role["agent_id"])) is None]
if missing:
# This path is normally handled by lifespan. It also rebuilds the
# memory-mode agent table from durable dynamic-angle folders.
await asyncio.to_thread(ensure_strategy_selection_agents)
await agents_router._sync_legacy_agents(agent_store)
for role in roles:
agent_id = str(role["agent_id"])
record = await agent_store.get_any(agent_id)
if record is None:
raise HTTPException(
status_code=503,
detail=f"Strategy agent '{agent_id}' configuration is unavailable",
)
if str(record.get("id") or "") != agent_id or record.get("user_id") is not None:
raise HTTPException(
status_code=409,
detail=f"Strategy agent '{agent_id}' must be a canonical global agent",
)
async def ensure_strategy_agent_registry(app) -> None:
"""Synchronize one worker's registry, including memory-mode sidecars."""
agent_store = getattr(app.state, "agent_store", None)
role_store = getattr(app.state, "strategy_agent_role_store", None)
if agent_store is None or role_store is None:
raise RuntimeError("Strategy agent persistence is unavailable")
await _ensure_registry(agent_store, role_store)
async def _to_item(
role: dict,
record: dict,
*,
include_config: bool,
) -> StrategyAgentItem:
if include_config:
agent = await agents_router._record_to_response_async(
record,
include_soul=True,
include_extras=True,
)
return StrategyAgentItem(
id=agent.id,
category=role["category"],
fixed=bool(role["fixed"]),
enabled=bool(role["enabled"]),
sort_order=int(role["sort_order"]),
name=agent.name,
description=agent.description,
model=agent.model,
soul=agent.soul,
skills=agent.skills,
created_at=agent.created_at,
updated_at=agent.updated_at,
)
# Public consumers need only identity and lifecycle metadata. Avoid
# leaking prompts/model/skills and avoid synchronous filesystem reads.
return StrategyAgentItem(
id=str(record["id"]),
category=role["category"],
fixed=bool(role["fixed"]),
enabled=bool(role["enabled"]),
sort_order=int(role["sort_order"]),
name=str(record["name"]),
description=str(record.get("description") or ""),
created_at=record.get("created_at"),
updated_at=record.get("updated_at"),
)
async def _get_role_and_agent(
request: Request,
agent_id: str,
) -> tuple[dict, dict]:
role_store = get_strategy_agent_role_store(request)
agent_store = get_agent_store(request)
await _ensure_registry(agent_store, role_store)
role = await role_store.get_role(agent_id)
if role is None:
raise HTTPException(status_code=404, detail=f"Strategy agent '{agent_id}' not found")
record = await agent_store.get_any(agent_id)
if record is None:
raise HTTPException(
status_code=409,
detail=f"Strategy agent '{agent_id}' has no backing agent configuration",
)
if record.get("user_id") is not None:
raise HTTPException(
status_code=409,
detail=f"Strategy agent '{agent_id}' must be globally owned",
)
canonical_role_id = str(role.get("agent_id") or "")
if canonical_role_id != agent_id:
raise HTTPException(
status_code=409,
detail=(f"Strategy agent id casing mismatch; use canonical id '{canonical_role_id}'"),
)
agents_router._require_canonical_record_id(agent_id, record)
return role, record
@router.get("", response_model=StrategyAgentListResponse)
async def list_strategy_agents(
request: Request,
include_disabled: bool = False,
include_config: bool = False,
) -> StrategyAgentListResponse:
"""List the shared role catalog; sensitive configuration is admin-only."""
if include_disabled or include_config:
await _require_admin(request)
role_store = get_strategy_agent_role_store(request)
agent_store = get_agent_store(request)
await _ensure_registry(agent_store, role_store)
roles = await role_store.list_roles(include_disabled=include_disabled)
items: list[StrategyAgentItem] = []
for role in roles:
record = await agent_store.get_any(str(role["agent_id"]))
if record is None:
if bool(role["enabled"]):
raise HTTPException(
status_code=503,
detail=(f"Strategy agent '{role['agent_id']}' configuration is unavailable"),
)
continue
if str(record.get("id") or "") != str(role["agent_id"]) or record.get("user_id") is not None:
raise HTTPException(
status_code=409,
detail=f"Strategy agent '{role['agent_id']}' must be a canonical global agent",
)
items.append(await _to_item(role, record, include_config=include_config))
return StrategyAgentListResponse(agents=items)
@router.post(
"/angles",
response_model=StrategyAgentItem,
status_code=201,
)
async def create_strategy_angle(
payload: StrategyAngleCreateRequest,
request: Request,
) -> StrategyAgentItem:
"""Create one global angle agent. Fixed role categories are not creatable."""
await _require_admin(request)
name = agents_router._validate_display_name(payload.name)
model = _normalize_model(request, payload.model)
skills = await agents_router._validate_strategy_agent_skills(
payload.skills,
app_config=get_config(request),
skill_store=get_skill_store(request),
)
agent_store = get_agent_store(request)
role_store = get_strategy_agent_role_store(request)
await _ensure_registry(agent_store, role_store)
agent_id = f"{STRATEGY_ANGLE_AGENT_PREFIX}{uuid4().hex}"
# The generated id has already passed the fixed prefix/storage contract;
# all filesystem and persistence mutations happen under the cross-worker
# guard below.
agent_id = agents_router._validate_storage_id(agent_id)
async with _strategy_mutation_guard(agent_id):
return await _create_strategy_angle_locked(
agent_id=agent_id,
payload=payload,
name=name,
model=model,
skills=skills,
agent_store=agent_store,
role_store=role_store,
)
async def _cleanup_failed_create(
agent_id: str,
agent_store: AgentStore,
role_store: StrategyAgentRoleStore,
) -> None:
async def attempt(label: str, operation) -> None:
try:
result = operation()
if asyncio.iscoroutine(result):
await result
except BaseException: # cleanup must continue through cancellation/failures
logger.exception("Failed create cleanup step %s for '%s'", label, agent_id)
await attempt("role", lambda: role_store.delete_role(agent_id))
await attempt("agent", lambda: agent_store.delete_builtin(agent_id))
await attempt("extras", lambda: agent_store.delete_extras_for_agent(agent_id))
agent_dir = get_paths().agent_dir(agent_id)
if agent_dir.exists():
await attempt("folder", lambda: asyncio.to_thread(shutil.rmtree, agent_dir))
async def _create_strategy_angle_locked(
*,
agent_id: str,
payload: StrategyAngleCreateRequest,
name: str,
model: str | None,
skills: list[str] | None,
agent_store: AgentStore,
role_store: StrategyAgentRoleStore,
) -> StrategyAgentItem:
agent_dir = get_paths().agent_dir(agent_id)
if await agent_store.get_any(agent_id) is not None or agent_dir.exists():
# Cryptographically improbable, but never overwrite a folder if the
# deployment restored a row without its registry metadata.
raise HTTPException(status_code=409, detail="Generated strategy agent id already exists")
sort_order = payload.sort_order
if sort_order is None:
current = await role_store.list_roles(include_disabled=True)
angle_orders = [int(row["sort_order"]) for row in current if row.get("category") == "angle"]
sort_order = (max(angle_orders) + 1) if angle_orders else 100
soul = (
payload.soul
if payload.soul.strip()
else _default_angle_soul(
name,
payload.description,
)
)
completed = False
try:
await asyncio.to_thread(
agents_router._write_agent_config,
agent_id,
name=name,
description=payload.description,
skills=skills,
model=model,
tool_groups=_DYNAMIC_ANGLE_TOOL_GROUPS,
)
await asyncio.to_thread(agents_router._write_agent_soul, agent_id, soul)
record = await agent_store.ensure_builtin(
{
"id": agent_id,
"name": name,
"description": payload.description,
"published": True,
}
)
await agent_store.replace_skills(agent_id, skills)
await agent_store.set_extras(
agent_id,
tool_groups=_DYNAMIC_ANGLE_TOOL_GROUPS,
model=model,
)
role = await role_store.create_role(
{
"agent_id": agent_id,
"category": "angle",
"fixed": False,
"enabled": True,
"sort_order": sort_order,
}
)
await asyncio.to_thread(_write_role_sidecar, agent_id, role)
completed = True
return await _to_item(role, record, include_config=True)
except asyncio.CancelledError:
raise
except HTTPException:
raise
except Exception as exc:
logger.exception("Failed to create strategy angle agent '%s'", agent_id)
raise HTTPException(
status_code=500,
detail="Failed to create strategy angle agent",
) from exc
finally:
if not completed:
cleanup = asyncio.create_task(_cleanup_failed_create(agent_id, agent_store, role_store))
try:
await asyncio.shield(cleanup)
except asyncio.CancelledError:
# A second cancellation must not leave a half-created global
# agent behind while the file lock is released.
await cleanup
raise
@router.put("/{agent_id}", response_model=StrategyAgentItem)
async def update_strategy_agent(
agent_id: str,
payload: StrategyAgentUpdateRequest,
request: Request,
) -> StrategyAgentItem:
"""Update configuration; fixed roles cannot change lifecycle or category."""
await _require_admin(request)
agent_id = agents_router._validate_storage_id(agent_id)
async with _strategy_mutation_guard(agent_id):
return await _update_strategy_agent_locked(agent_id, payload, request)
async def _update_strategy_agent_locked(
agent_id: str,
payload: StrategyAgentUpdateRequest,
request: Request,
) -> StrategyAgentItem:
role, record = await _get_role_and_agent(request, agent_id)
agent_store = get_agent_store(request)
fields = payload.model_fields_set
if bool(role["fixed"]):
if "enabled" in fields and payload.enabled is not True:
raise HTTPException(status_code=409, detail="Fixed strategy roles cannot be disabled")
if "sort_order" in fields and payload.sort_order != int(role["sort_order"]):
raise HTTPException(status_code=409, detail="Fixed strategy role order cannot be changed")
next_name = agents_router._validate_display_name(payload.name or "") if "name" in fields else str(record["name"])
next_description = payload.description or "" if "description" in fields else str(record.get("description") or "")
extras = await asyncio.to_thread(agents_router._load_config_extras, agent_id)
next_model = _normalize_model(request, payload.model) if "model" in fields else extras["model"]
next_skills = (
await agents_router._validate_strategy_agent_skills(
payload.skills,
app_config=get_config(request),
skill_store=get_skill_store(request),
)
if "skills" in fields
else extras["skills"]
)
agent_dir = get_paths().agent_dir(agent_id)
file_snapshot = await asyncio.to_thread(
lambda: {
"config": _read_optional_text(agent_dir / "config.yaml"),
"soul": _read_optional_text(agent_dir / "SOUL.md"),
"sidecar": _read_optional_text(agent_dir / _ROLE_SIDECAR),
}
)
original_record = dict(record)
original_role = dict(role)
role_store = get_strategy_agent_role_store(request)
try:
config_fields = {"name", "description", "model", "skills"}
if fields & config_fields:
await asyncio.to_thread(
agents_router._write_agent_config,
agent_id,
name=next_name,
description=next_description,
skills=next_skills,
model=next_model,
tool_groups=extras["tool_groups"],
common_questions=extras["common_questions"],
template_name=extras["template_name"],
seat_skill_directive=extras["seat_skill_directive"],
llmwiki_knowledge_base_ids=extras["llmwiki_knowledge_base_ids"],
)
await agent_store.replace_skills(agent_id, next_skills)
await agent_store.set_extras(
agent_id,
tool_groups=extras["tool_groups"],
model=next_model,
)
if "soul" in fields:
await asyncio.to_thread(
agents_router._write_agent_soul,
agent_id,
payload.soul or "",
)
if {"name", "description"} & fields:
updated = await agent_store.update_any(
agent_id,
{"name": next_name, "description": next_description},
)
if updated is None:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
record = updated
enabled = payload.enabled if "enabled" in fields and not bool(role["fixed"]) else None
sort_order = payload.sort_order if "sort_order" in fields and not bool(role["fixed"]) else None
if enabled is not None or sort_order is not None:
updated_role = await role_store.update_role(
agent_id,
enabled=enabled,
sort_order=sort_order,
)
if updated_role is None:
raise HTTPException(status_code=404, detail=f"Strategy agent '{agent_id}' not found")
role = updated_role
if not bool(role["fixed"]):
await asyncio.to_thread(_write_role_sidecar, agent_id, role)
except Exception:
await _rollback_strategy_update(
agent_id=agent_id,
agent_store=agent_store,
role_store=role_store,
record=original_record,
role=original_role,
extras=extras,
file_snapshot=file_snapshot,
)
raise
return await _to_item(role, record, include_config=True)
@router.delete("/{agent_id}", response_model=StrategyAgentItem)
async def disable_strategy_angle(agent_id: str, request: Request) -> StrategyAgentItem:
"""Soft-disable an angle while retaining its agent and historical links."""
await _require_admin(request)
agent_id = agents_router._validate_storage_id(agent_id)
async with _strategy_mutation_guard(agent_id):
return await _disable_strategy_angle_locked(agent_id, request)
async def _disable_strategy_angle_locked(
agent_id: str,
request: Request,
) -> StrategyAgentItem:
role, record = await _get_role_and_agent(request, agent_id)
if bool(role["fixed"]):
raise HTTPException(status_code=409, detail="Fixed strategy roles cannot be deleted")
if role.get("category") != "angle":
raise HTTPException(status_code=409, detail="Only strategy angle roles can be disabled")
updated = await get_strategy_agent_role_store(request).update_role(
agent_id,
enabled=False,
)
if updated is None:
raise HTTPException(status_code=404, detail=f"Strategy agent '{agent_id}' not found")
try:
await asyncio.to_thread(_write_role_sidecar, agent_id, updated)
except Exception:
await get_strategy_agent_role_store(request).update_role(
agent_id,
enabled=bool(role["enabled"]),
sort_order=int(role["sort_order"]),
)
raise
return await _to_item(updated, record, include_config=True)
__all__ = [
"StrategyAgentItem",
"StrategyAgentListResponse",
"StrategyAgentUpdateRequest",
"StrategyAngleCreateRequest",
"create_strategy_angle",
"disable_strategy_angle",
"ensure_strategy_agent_registry",
"list_strategy_agents",
"router",
"update_strategy_agent",
]
@@ -26,6 +26,7 @@ from deerflow.config.system_settings import (
PublishSquare,
PublishSquaresSettings,
SkillCompressionSettings,
StrategyProjectSettings,
get_module_tag_ids,
list_publish_squares,
load_system_settings,
@@ -60,6 +61,14 @@ class OrdinaryKnowledgeLanding(BaseModel):
mode: Literal["wiki", "weknora"] = "wiki"
class StrategyProjectSettingsResponse(BaseModel):
taiwan_strait_demo_enabled: bool = False
class StrategyProjectSettingsUpdateRequest(BaseModel):
taiwan_strait_demo_enabled: bool | None = None
@router.get("/ordinary-knowledge-landing", response_model=OrdinaryKnowledgeLanding)
async def get_ordinary_knowledge_landing() -> OrdinaryKnowledgeLanding:
return OrdinaryKnowledgeLanding(mode=load_system_settings().ordinary_knowledge_landing)
@@ -72,6 +81,35 @@ async def update_ordinary_knowledge_landing(body: OrdinaryKnowledgeLanding, requ
return OrdinaryKnowledgeLanding(mode=saved.ordinary_knowledge_landing)
def _strategy_project_settings_response(
settings: StrategyProjectSettings,
) -> StrategyProjectSettingsResponse:
return StrategyProjectSettingsResponse(
taiwan_strait_demo_enabled=settings.taiwan_strait_demo_enabled,
)
@router.get("/strategy-projects", response_model=StrategyProjectSettingsResponse)
async def get_strategy_project_settings() -> StrategyProjectSettingsResponse:
"""Return project-demo visibility. Any logged-in user may read it."""
return _strategy_project_settings_response(load_system_settings().strategy_projects)
@router.put("/strategy-projects", response_model=StrategyProjectSettingsResponse)
async def update_strategy_project_settings(
body: StrategyProjectSettingsUpdateRequest,
request: Request,
) -> StrategyProjectSettingsResponse:
"""Update project-demo visibility. Admin only."""
await _require_admin(request)
current = load_system_settings()
fields = body.model_dump(exclude_unset=True, exclude_none=True)
updated = current.strategy_projects.model_copy(update=fields)
saved = save_system_settings(current.model_copy(update={"strategy_projects": updated}))
return _strategy_project_settings_response(saved.strategy_projects)
class PromptPrefixUpdateRequest(BaseModel):
enabled: bool | None = None
switch_label: str | None = None
+5 -3
View File
@@ -195,7 +195,7 @@ models:
# display_name: MiniMax-M2.7
# use: langchain_openai:ChatOpenAI
# model: MiniMax-M2.7
# api_key: sk-cp-mrsAGtzDpUl0TpOBPVoiRG_pWF1kDDN9_mDAnONBs6HIPcNUexSKtSbx2QOxfKBvJT6Xw6hDTV6hdkJYFWfNA8UtHQ08TiGJiosdRoW6q1sGb6Cb7D85NHk
# api_key: sk-cp-mXNcPkl81WAERsVaVOCj5r-F3LQv8kdtMWzx_U4RNAJejJKtK_JB2mHLFqBkBYvVDb8OPZmxJgiuctDHhDImvkHvJqGj_RvvlTaCzFvhwIvuHT8jm46niBY
# base_url: https://api.minimaxi.com/v1
# max_tokens: 4096
# temperature: 1.0
@@ -411,7 +411,9 @@ sandbox:
use: deerflow.sandbox.local:LocalSandboxProvider
allow_host_bash: true
mounts:
- host_path: C:\Users\iie\Desktop\deerflow-new\offline-backend-20260512\backend\knowledge\knowledge_base
# Relative paths are resolved against this config file, so the knowledge
# mount works after cloning or moving the offline deployment.
- host_path: knowledge/knowledge_base
container_path: /mnt/knowledge
read_only: true
@@ -623,7 +625,7 @@ llmwiki:
enabled: true
# 免登录 POST /api/knowledge/vector-search 返回的 Wiki 一键跳转链接前缀。
frontend_base_url: "http://127.0.0.1:5174"
auto_sync: false
auto_sync: true
sync_interval_seconds: 30
database_batch_size: 50
embedding:
@@ -19,7 +19,7 @@ The summarization feature uses DeerFlow's `DeerFlowSummarizationMiddleware` (a s
LangChain's stock `SummarizationMiddleware` runs in `before_model` and persists a `RemoveMessage(REMOVE_ALL_MESSAGES)` to the checkpoint — which permanently deletes the original Q&A from both the model context *and* the conversation the frontend renders. DeerFlow overrides this:
- `before_model` / `abefore_model` are **no-ops** (the destructive path is disabled).
- Compression runs in `wrap_model_call` / `awrap_model_call`: for a single model call, the message list is replaced with `[summary, *recent]` via `request.override(messages=...)`. **Nothing is written back to state.**
- Compression runs in `wrap_model_call` / `awrap_model_call`: for a single model call, the hidden summary is merged into LangChain's dedicated `ModelRequest.system_message` slot and the ordinary message list is replaced with the recent tail. This guarantees exactly one leading system message even for strict Qwen-style or generic `langchain_chat` adapters. **Nothing is written back to state.**
- The full original messages stay in the checkpoint, so the frontend keeps showing the user's real conversation. The summary message is named `summary` and flagged `hide_from_ui`, and because it only ever lives in the transient request it never reaches the frontend at all.
- The generated summary is cached **per-thread** so we don't pay an extra LLM call on every model invocation. The cached summary is reused (no LLM call) until enough *new* messages accumulate to push the compressed context back over the trigger threshold (a sticky boundary mirroring the stock middleware's post-compaction behavior). The cache is bounded (oldest threads evicted first).
- **Threshold-first**: while the full current conversation is still under the trigger, the request is passed through untouched — no cache lookup, no summary substitution. Compression only ever engages once the real context crosses the configured trigger.
@@ -185,7 +185,7 @@ The default LangChain prompt instructs the model to:
- Messages to summarize (older messages beyond the `keep` threshold)
- Messages to preserve (recent messages within the `keep` threshold)
4. **Summary Generation**: The model generates a concise summary of the older messages
5. **Context Replacement (transient)**: The message list **handed to the model for this call** is rebuilt as `[summary, *recent]`. The persisted state is left untouched — old messages are **not** removed, so the frontend still renders the full original conversation.
5. **Context Replacement (transient)**: The request **handed to the model for this call** is rebuilt with one system message (agent prompt + hidden summary) and the recent non-system tail. The persisted state is left untouched — old messages are **not** removed, so the frontend still renders the full original conversation.
6. **AI/Tool Pair Protection**: The system ensures AI messages and their corresponding tool messages stay together
7. **Skill Rescue**: Before the summary is generated, the most recently loaded skill files (tool results whose tool name is in `skill_file_read_tool_names` and whose target path is under `skills.container_path`) are lifted out of the summarization set and prepended to the preserved tail. Selection walks newest-first under three budgets: `preserve_recent_skill_count`, `preserve_recent_skill_tokens`, and `preserve_recent_skill_tokens_per_skill`. The triggering AIMessage and all of its paired ToolMessages move together so tool_call ↔ tool_result pairing stays intact.
@@ -202,12 +202,14 @@ The middleware intelligently preserves message context:
- **Recent Messages**: Always kept intact based on `keep` configuration
- **AI/Tool Pairs**: Never split - if a cutoff point falls within tool messages, the system adjusts to keep the entire AI + Tool message sequence together
- **Summary Format**: Summary is injected as a HumanMessage with the format:
```
Here is a summary of the conversation to date:
[Generated summary text]
```
- **Summary Format**: Summary is merged into the leading SystemMessage with the format:
```
Internal conversation summary for continuity only...
<internal_context_summary>
[generated summary text]
</internal_context_summary>
```
## Best Practices
@@ -329,7 +331,8 @@ Summarization runs after ThreadData and Sandbox initialization but before Title
- Configuration is loaded once at startup
- Summarization is **non-destructive**: the summary replaces older messages only in the transient model request (`wrap_model_call`), never in persisted state. The checkpointer therefore keeps the full original conversation
- The per-thread summary cache lives in a **process-wide, thread-safe LRU** (`_GLOBAL_SUMMARY_CACHE`, bounded), **not** on the middleware instance. `make_lead_agent` rebuilds a fresh middleware every run, so an instance cache would be cold each turn and re-run a blocking summary LLM call on every turn of a long conversation; the module-level cache keyed by `thread_id` lets a summary survive across turns. It is a cost optimization only — a cold/stale cache simply re-summarizes, which is always correct
- The per-thread summary cache lives in a **process-wide, thread-safe LRU** (`_GLOBAL_SUMMARY_CACHE`, bounded), **not** on the middleware instance. `make_lead_agent` rebuilds a fresh middleware every run, so an instance cache would be cold each turn and re-run a blocking summary LLM call on every turn of a long conversation; the module-level cache keyed by `thread_id` lets a summary survive across turns. It is a cost optimization only — a cold/stale cache simply re-summarizes, which is always correct.
- Compression normalizes LangChain's split request representation before invoking the next middleware/model: the dedicated `system_message` contains all system context exactly once, while `messages` contains no `SystemMessage`. This makes the fix independent of the configured provider adapter.
## Example Configurations
@@ -26,7 +26,7 @@
"enabled": true
},
"policy-knowledge": {
"enabled": false
"enabled": true
},
"html-page-builder": {
"enabled": true
@@ -96,9 +96,11 @@ _BUSY_PATTERNS = (
"稍后重试",
"请稍后重试",
)
_QUOTA_PATTERNS = (
"insufficient_quota",
"quota",
_QUOTA_PATTERNS = (
"insufficient_quota",
"insufficient balance",
"token plan limit",
"quota",
"billing",
"credit",
"payment",
@@ -122,8 +124,10 @@ _AUTH_PATTERNS = (
# error wrapped as `bad_response_status_code`, or a truncated/garbled inference
# response). These are almost always transient inference-side issues and safe
# to retry.
_UPSTREAM_GARBAGE_PATTERNS = (
"bad_response_status_code",
_UPSTREAM_GARBAGE_PATTERNS = (
"openai-compatible upstream application error",
"null value for 'choices'",
"bad_response_status_code",
"bad response status",
"expecting ',' delimiter",
"expecting value",
@@ -5,8 +5,9 @@ Unlike LangChain's stock ``SummarizationMiddleware`` (which runs in
to the checkpoint — wiping the original Q&A from both the model context *and*
the conversation the frontend renders), this middleware compresses the history
**transiently in ``wrap_model_call``**. The full original messages stay in state
(so the UI keeps showing the user's real conversation); only the message list
handed to the model for a single call is replaced with ``[summary, *recent]``.
(so the UI keeps showing the user's real conversation); only the request handed
to the model is replaced with a merged leading system context plus recent
messages.
The generated summary is cached per-thread so we don't pay an extra LLM call on
every model invocation: once a thread is summarized, the cached summary is reused
@@ -39,6 +40,8 @@ from langchain_core.messages import (
from langgraph.config import get_config
from langgraph.runtime import Runtime
from deerflow.models.message_ordering import normalize_model_request_messages
logger = logging.getLogger(__name__)
# Upper bound on the per-thread summary cache so a long-lived process serving
@@ -321,7 +324,14 @@ class DeerFlowSummarizationMiddleware(SummarizationMiddleware):
) -> ModelCallResult:
compressed = self._compressed_messages(request)
if compressed is not None:
request = request.override(messages=compressed)
system_message, messages = normalize_model_request_messages(
request.system_message,
compressed,
)
request = request.override(
system_message=system_message,
messages=messages,
)
return handler(request)
@override
@@ -332,7 +342,14 @@ class DeerFlowSummarizationMiddleware(SummarizationMiddleware):
) -> ModelCallResult:
compressed = await self._acompressed_messages(request)
if compressed is not None:
request = request.override(messages=compressed)
system_message, messages = normalize_model_request_messages(
request.system_message,
compressed,
)
request = request.override(
system_message=system_message,
messages=messages,
)
return await handler(request)
def _compressed_messages(self, request: ModelRequest) -> list[AnyMessage] | None:
@@ -211,6 +211,7 @@ class AppConfig(BaseModel):
cls._check_config_version(config_data, resolved_path)
config_data = cls.resolve_env_variables(config_data)
cls._resolve_relative_mount_paths(config_data, resolved_path.parent)
cls._apply_database_defaults(config_data)
# Load title config if present
@@ -272,6 +273,36 @@ class AppConfig(BaseModel):
result = cls.model_validate(config_data)
return result
@classmethod
def _resolve_relative_mount_paths(
cls,
config_data: dict[str, Any],
config_dir: Path,
) -> None:
"""Resolve portable sandbox mount paths beside ``config.yaml``.
Deployment copies frequently live at different absolute paths. A
relative ``sandbox.mounts[*].host_path`` therefore means "relative to
the config file", never relative to the process working directory.
Providers continue to receive absolute paths, which keeps both the
local path mapper and Docker bind mounts deterministic.
"""
sandbox = config_data.get("sandbox")
if not isinstance(sandbox, dict):
return
mounts = sandbox.get("mounts")
if not isinstance(mounts, list):
return
for mount in mounts:
if not isinstance(mount, dict):
continue
raw_host_path = mount.get("host_path")
if not isinstance(raw_host_path, str) or not raw_host_path.strip():
continue
host_path = Path(raw_host_path)
if not host_path.is_absolute():
mount["host_path"] = str((config_dir / host_path).resolve())
@classmethod
def _apply_database_defaults(cls, config_data: dict[str, Any]) -> None:
"""Apply config.yaml defaults for persistence when the section is absent."""
@@ -77,6 +77,7 @@ class LocalWikiEmbeddingConfig(BaseModel):
base_url: str = ""
api_key: str = Field(default="", repr=False)
local_model_path: str = ""
local_model_revision: str = ""
local_cache_dir: str = ""
local_process_isolation: bool = True
local_threads: int = Field(default=2, ge=1, le=32)
@@ -508,11 +508,33 @@ class ModelConcurrencySettings(BaseModel):
class LocalWikiEncoderSettings(BaseModel):
"""Runtime override for the administrator-managed offline Wiki encoder."""
mode: Literal["configured", "local"] = Field(
default="configured",
description=(
"Use the embedding provider from config.yaml, or explicitly override it "
"with the administrator-managed offline BGE-M3 encoder."
),
)
model_path: str = Field(
default="",
max_length=4096,
description="Absolute or project-relative directory containing the offline BGE-M3 ONNX model.",
)
model_revision: str = Field(
default="",
max_length=64,
description="Opaque revision changed whenever an offline model directory is activated.",
)
model_config = ConfigDict(extra="ignore")
class StrategyProjectSettings(BaseModel):
"""Global visibility controls for the strategy-project experience."""
taiwan_strait_demo_enabled: bool = Field(
default=False,
description="Whether Taiwan Strait demo entry points are visible in My Projects.",
)
model_config = ConfigDict(extra="ignore")
@@ -532,6 +554,7 @@ class SystemSettings(BaseModel):
concurrency_monitor: ConcurrencyMonitorSettings = Field(default_factory=ConcurrencyMonitorSettings)
model_concurrency: ModelConcurrencySettings = Field(default_factory=ModelConcurrencySettings)
local_wiki_encoder: LocalWikiEncoderSettings = Field(default_factory=LocalWikiEncoderSettings)
strategy_projects: StrategyProjectSettings = Field(default_factory=StrategyProjectSettings)
llmwiki: LlmWikiRuntimeOverride = Field(default_factory=LlmWikiRuntimeOverride)
skill_knowledge_auto_sync: SkillKnowledgeAutoSyncSettings = Field(default_factory=SkillKnowledgeAutoSyncSettings)
knowledge_base_product_version: Literal["v1", "v2"] = Field(
@@ -596,7 +619,12 @@ def save_system_settings(settings: SystemSettings) -> SystemSettings:
path = _settings_path()
payload = settings.model_dump(mode="json")
with _lock:
tmp = path.with_suffix(path.suffix + ".tmp")
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
tmp.replace(path)
# Multiple Gateway workers share this file. A unique temporary name
# prevents their atomic replaces from racing over one common `.tmp`.
tmp = path.with_name(f".{path.name}.{uuid4().hex}.tmp")
try:
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
tmp.replace(path)
finally:
tmp.unlink(missing_ok=True)
return settings
@@ -57,6 +57,9 @@ class StrictWikiEmbeddingClient:
self._local_started_at: str | None = None
self._local_ready_at: str | None = None
self._local_warmup_task: asyncio.Task[None] | None = None
self._lifecycle_condition = asyncio.Condition()
self._active_uses = 0
self._retired = False
@property
def dimensions(self) -> int | None:
@@ -64,7 +67,21 @@ class StrictWikiEmbeddingClient:
@property
def fingerprint(self) -> str:
raw = "\0".join((self.config.provider.strip().lower(), self.config.base_url.rstrip("/").lower(), self.config.model.strip(), str(self._resolved_dimensions or "auto")))
parts = [
self.config.provider.strip().lower(),
self.config.base_url.rstrip("/").lower(),
self.config.model.strip(),
str(self._resolved_dimensions or "auto"),
]
if self.config.provider == "local":
model_path = self._local_model_path()
parts.extend(
(
str(model_path) if model_path is not None else "",
self.config.local_model_revision,
)
)
raw = "\0".join(parts)
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
async def close(self) -> None:
@@ -73,6 +90,15 @@ class StrictWikiEmbeddingClient:
await self._client.aclose()
self._client = None
async def retire(self) -> None:
"""Reject future work, drain active calls, then release model resources."""
async with self._lifecycle_condition:
self._retired = True
async with self._lifecycle_condition:
await self._lifecycle_condition.wait_for(lambda: self._active_uses == 0)
await self.close()
async def reset_local_encoder(self, *, model_path: str | None = None) -> None:
"""Stop only the embedded worker and optionally switch its model path.
@@ -82,27 +108,80 @@ class StrictWikiEmbeddingClient:
"""
warmup = self._local_warmup_task
if warmup is not None and not warmup.done():
warmup.cancel()
# Local inference runs in a worker thread. Cancelling the asyncio
# wrapper does not stop that thread and could let it publish a new
# process/model after cleanup. Drain it before touching resources.
await asyncio.gather(warmup, return_exceptions=True)
self._local_warmup_task = None
process = self._local_process
self._local_process = None
if process is not None:
await asyncio.to_thread(self._stop_local_process, process)
self._local_model = None
self._local_state = "idle"
self._local_error = None
self._local_started_at = None
self._local_ready_at = None
if model_path is not None:
self.config = self.config.model_copy(update={"local_model_path": model_path})
# Wait for a non-warmup embedding batch to finish before replacing or
# stopping the local worker. This prevents an admin upload from
# breaking a concurrent vectorization request halfway through IPC.
async with self._local_lock:
process = self._local_process
self._local_process = None
if process is not None:
await self._run_thread_drained(self._stop_local_process, process)
self._local_model = None
self._local_state = "idle"
self._local_error = None
self._local_started_at = None
self._local_ready_at = None
if model_path is not None:
self.config = self.config.model_copy(update={"local_model_path": model_path})
async def _get_client(self) -> httpx.AsyncClient:
if self._client is None:
self._client = httpx.AsyncClient(timeout=self.config.timeout_seconds)
return self._client
@staticmethod
async def _run_thread_drained(function, *args):
"""Do not release lifecycle locks while cancelled thread work still runs."""
worker = asyncio.create_task(asyncio.to_thread(function, *args))
cancelled = False
worker_error: BaseException | None = None
result = None
while not worker.done():
try:
result = await asyncio.shield(worker)
except asyncio.CancelledError:
# A caller can be cancelled more than once during shutdown.
# Swallow every cancellation until the non-cancellable thread
# has really stopped, then restore cancellation semantics.
cancelled = True
except BaseException as exc:
worker_error = exc
break
if worker.done() and worker_error is None:
try:
result = worker.result()
except BaseException as exc:
worker_error = exc
if cancelled:
raise asyncio.CancelledError from None
if worker_error is not None:
raise worker_error
return result
async def embed_texts(self, texts: list[str]) -> list[list[float]]:
async with self._lifecycle_condition:
if self._retired:
raise WikiEmbeddingError(
"Wiki embedding runtime was replaced; retry with the active provider",
category="runtime_retired",
retryable=True,
)
self._active_uses += 1
try:
return await self._embed_texts_active(texts)
finally:
async with self._lifecycle_condition:
self._active_uses -= 1
if self._active_uses == 0:
self._lifecycle_condition.notify_all()
async def _embed_texts_active(self, texts: list[str]) -> list[list[float]]:
if self.config.provider == "local":
if not texts:
return []
@@ -110,7 +189,10 @@ class StrictWikiEmbeddingClient:
self._local_state = "starting"
self._local_started_at = self._local_started_at or datetime.now(UTC).isoformat()
try:
values = await asyncio.to_thread(self._embed_local, texts)
# `to_thread` keeps running after its awaiting task is
# cancelled. Keep the lifecycle lease and local lock until
# the real inference call has stopped.
values = await self._run_thread_drained(self._embed_local, texts)
except Exception as exc:
self._local_state = "failed"
self._local_error = " ".join(str(exc).split())[:1000] or type(exc).__name__
@@ -452,6 +534,8 @@ class StrictWikiEmbeddingClient:
def start_local_encoder(self) -> dict[str, object]:
"""Warm the embedded CPU encoder without coupling it to app startup."""
if self._retired:
raise ValueError("当前本地编码器运行时已切换,请刷新状态后重试")
if self.config.provider != "local":
raise ValueError("当前 embedding provider 不是 local")
if not self.config.enabled:
@@ -11,7 +11,7 @@ from uuid import uuid4
from deerflow.config.llmwiki_config import LocalWikiIndexConfig
from deerflow.integrations.weknora.client import WeKnoraClient
from deerflow.integrations.weknora.local_index.chunking import chunk_wiki_page
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient, WikiEmbeddingError
from deerflow.integrations.weknora.local_index.normalize import normalize_wiki_page
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
from deerflow.integrations.weknora.local_index.vector_codec import encode_vector
@@ -137,6 +137,8 @@ class WikiSyncService:
except Exception as exc:
if isinstance(exc, WikiSyncLeaseLost):
raise
if isinstance(exc, WikiEmbeddingError) and exc.category == "runtime_retired":
raise
failed += 1
try:
fallback = normalize_wiki_page(mapping_id, summary)
@@ -6,7 +6,6 @@ from typing import Any
from langchain_core.messages import BaseMessage, SystemMessage
_SYSTEM_SEPARATOR = "\n\n---\n\n"
@@ -35,7 +34,11 @@ def normalize_langchain_messages(messages: list[BaseMessage]) -> list[BaseMessag
return messages
first_non_system_index = next((idx for idx, message in enumerate(messages) if not isinstance(message, SystemMessage)), len(messages))
if len(system_messages) == first_non_system_index:
# A number of strict chat templates accept *one* system message, and only
# at index zero. Two adjacent leading system messages are therefore not a
# valid no-op case: LangChain's agent prompt followed by DeerFlow's
# transient compaction summary must still be folded into one message.
if len(system_messages) == 1 and first_non_system_index == 1:
return messages
merged = system_messages[0].model_copy(
@@ -46,20 +49,38 @@ def normalize_langchain_messages(messages: list[BaseMessage]) -> list[BaseMessag
return [merged, *(message for message in messages if not isinstance(message, SystemMessage))]
def normalize_model_request_messages(
system_message: SystemMessage | None,
messages: list[BaseMessage],
) -> tuple[SystemMessage | None, list[BaseMessage]]:
"""Normalize LangChain's split ``ModelRequest`` message representation.
``ModelRequest.messages`` is documented as excluding the system prompt.
Transient middleware (notably context compaction) can nevertheless place a
``SystemMessage`` in that list. Fold every such message into the dedicated
system slot so adapters without a DeerFlow-specific payload patch also
receive exactly one leading system message.
"""
if system_message is None and not any(isinstance(message, SystemMessage) for message in messages):
return None, messages
combined = [system_message, *messages] if system_message is not None else messages
normalized = normalize_langchain_messages(combined)
if normalized and isinstance(normalized[0], SystemMessage):
return normalized[0], normalized[1:]
return None, normalized
def _normalize_payload_message_list(messages: list[Any]) -> list[Any]:
system_messages = [message for message in messages if isinstance(message, dict) and message.get("role") == "system"]
if not system_messages:
return messages
first_non_system_index = next(
(
idx
for idx, message in enumerate(messages)
if not (isinstance(message, dict) and message.get("role") == "system")
),
(idx for idx, message in enumerate(messages) if not (isinstance(message, dict) and message.get("role") == "system")),
len(messages),
)
if len(system_messages) == first_non_system_index:
if len(system_messages) == 1 and first_non_system_index == 1:
return messages
merged_system = dict(system_messages[0])
@@ -25,6 +25,7 @@ message that originally carried it:
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from langchain_core.language_models import LanguageModelInput
@@ -36,6 +37,65 @@ from deerflow.models.message_ordering import normalize_chat_payload_messages
from deerflow.models.reasoning_fields import apply_reasoning_from_payload, extract_reasoning_text, fill_ui_reasoning_content, restore_reasoning_fields
class OpenAICompatibleUpstreamError(RuntimeError):
"""An OpenAI-compatible gateway reported an error inside a HTTP 200 body.
Several compatible gateways do not use an HTTP error status for quota,
concurrency, or inference failures. Instead they return ``choices=null``
(or an empty streamed chunk) together with a vendor ``base_resp`` object.
LangChain raises a generic ``TypeError`` for the non-streaming shape and
treats the streaming shape as an empty successful response. Raising one
stable exception here lets DeerFlow's model middleware retry both paths.
"""
def __init__(self, *, code: Any, message: str | None = None) -> None:
self.code = code
detail = _bounded_error_message(message)
suffix = f": {detail}" if detail else ""
super().__init__(f"OpenAI-compatible upstream application error (code={code}){suffix}")
def _bounded_error_message(value: Any, *, limit: int = 500) -> str:
"""Return a log-safe, bounded one-line provider message."""
if not isinstance(value, str):
return ""
normalized = " ".join(value.split())
return normalized[:limit]
def _application_error(payload: Mapping[str, Any]) -> tuple[Any, str | None] | None:
"""Extract a non-success ``base_resp`` error from a compatible payload."""
raw = payload.get("base_resp") or payload.get("baseResp")
if not isinstance(raw, Mapping):
return None
code = raw.get("status_code")
if code is None:
code = raw.get("statusCode")
if code is None:
code = raw.get("code")
# MiniMax-compatible responses use 0 for success. Be conservative for
# gateways that put informational metadata in ``base_resp``.
normalized_code = str(code).strip().lower() if code is not None else ""
if normalized_code in {"", "0", "200", "ok", "success"}:
return None
message = raw.get("status_msg")
if message is None:
message = raw.get("statusMsg")
if message is None:
message = raw.get("message")
return code, message if isinstance(message, str) else None
def _raise_for_application_error(payload: Mapping[str, Any]) -> None:
error = _application_error(payload)
if error is not None:
code, message = error
raise OpenAICompatibleUpstreamError(code=code, message=message)
class PatchedChatOpenAI(ChatOpenAI):
"""ChatOpenAI with ``thought_signature`` preservation for Gemini thinking via OpenAI gateway.
@@ -108,6 +168,10 @@ class PatchedChatOpenAI(ChatOpenAI):
base_generation_info: dict | None,
) -> ChatGenerationChunk | None:
"""Preserve streamed ``reasoning_content`` in ``additional_kwargs``."""
# Some compatible gateways return a vendor error event followed by a
# normal-looking usage chunk, all under HTTP 200. Detect the first
# event before LangChain converts it into an empty AIMessageChunk.
_raise_for_application_error(chunk)
generation_chunk = super()._convert_chunk_to_generation_chunk(
chunk,
default_chunk_class,
@@ -133,8 +197,9 @@ class PatchedChatOpenAI(ChatOpenAI):
generation_info: dict | None = None,
) -> ChatResult:
"""Preserve non-stream ``reasoning_content`` in ``additional_kwargs``."""
result = super()._create_chat_result(response, generation_info)
response_dict = response if isinstance(response, dict) else response.model_dump()
_raise_for_application_error(response_dict)
result = super()._create_chat_result(response, generation_info)
choices = response_dict.get("choices", [])
generations: list[ChatGeneration] = []
@@ -125,6 +125,7 @@ from deerflow.persistence.skill_knowledge.model import (
SkillKnowledgeSyncJobRow,
)
from deerflow.persistence.skills.model import SkillDisplayAdapterRow, SkillFavoriteRow, SkillRow
from deerflow.persistence.strategy_agent_roles.model import StrategyAgentRoleRow
from deerflow.persistence.tags.model import TagAssignmentRow, TagRow
from deerflow.persistence.task_buttons.model import TaskButtonRow
from deerflow.persistence.task_reports.model import TaskReportRow
@@ -254,6 +255,7 @@ __all__ = [
"SkillKnowledgeSnapshotRow",
"SkillKnowledgeSyncItemRow",
"SkillKnowledgeSyncJobRow",
"StrategyAgentRoleRow",
"TagAssignmentRow",
"TagRow",
"TaskButtonRow",
@@ -14,7 +14,7 @@ import json
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.parallel_agent_tasks.model import ParallelAgentTaskRow
@@ -59,12 +59,7 @@ class ParallelAgentTaskRepository:
async def list_tasks(self, user_id: str, *, limit: int = 200) -> list[dict[str, Any]]:
"""列出当前用户的任务(轻量元信息,不含 messages,供历史列表)。"""
stmt = (
select(ParallelAgentTaskRow)
.where(ParallelAgentTaskRow.user_id == user_id)
.order_by(ParallelAgentTaskRow.updated_at.desc())
.limit(limit)
)
stmt = select(ParallelAgentTaskRow).where(ParallelAgentTaskRow.user_id == user_id).order_by(ParallelAgentTaskRow.updated_at.desc()).limit(limit)
async with self._sf() as session:
result = await session.execute(stmt)
return [self._to_dict(row, with_messages=False) for row in result.scalars()]
@@ -103,22 +98,40 @@ class ParallelAgentTaskRepository:
brief=_UNSET,
agents=_UNSET,
messages=_UNSET,
expected_updated_at: datetime | str | None = None,
) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(ParallelAgentTaskRow, task_id)
if row is None or row.user_id != user_id:
expected_revision = expected_updated_at
if isinstance(expected_revision, str):
try:
expected_revision = datetime.fromisoformat(expected_revision.replace("Z", "+00:00"))
except ValueError:
return None
values: dict[str, Any] = {"updated_at": datetime.now(UTC)}
if title is not _UNSET and title is not None:
values["title"] = str(title).strip()
if brief is not _UNSET:
values["brief"] = brief or None
if agents is not _UNSET:
values["agents"] = _dumps(agents)
if messages is not _UNSET:
values["messages"] = _dumps(messages)
async with self._sf() as session:
statement = update(ParallelAgentTaskRow).where(
ParallelAgentTaskRow.id == task_id,
ParallelAgentTaskRow.user_id == user_id,
)
if expected_revision is not None:
statement = statement.where(ParallelAgentTaskRow.updated_at == expected_revision)
result = await session.execute(statement.values(**values))
if result.rowcount != 1:
await session.rollback()
return None
if title is not _UNSET and title is not None:
row.title = str(title).strip()
if brief is not _UNSET:
row.brief = brief or None
if agents is not _UNSET:
row.agents = _dumps(agents)
if messages is not _UNSET:
row.messages = _dumps(messages)
row.updated_at = datetime.now(UTC)
await session.commit()
await session.refresh(row)
row = await session.get(ParallelAgentTaskRow, task_id)
if row is None:
return None
return self._to_dict(row)
async def delete_task(self, task_id: str, user_id: str) -> bool:
@@ -0,0 +1,24 @@
"""Global strategy-selection agent role registry."""
from __future__ import annotations
from deerflow.persistence.strategy_agent_roles.base import StrategyAgentRoleStore
from deerflow.persistence.strategy_agent_roles.memory import (
MemoryStrategyAgentRoleStore,
)
from deerflow.persistence.strategy_agent_roles.model import StrategyAgentRoleRow
from deerflow.persistence.strategy_agent_roles.sql import StrategyAgentRoleRepository
__all__ = [
"MemoryStrategyAgentRoleStore",
"StrategyAgentRoleRepository",
"StrategyAgentRoleRow",
"StrategyAgentRoleStore",
"make_strategy_agent_role_store",
]
def make_strategy_agent_role_store(session_factory):
if session_factory is None:
return MemoryStrategyAgentRoleStore()
return StrategyAgentRoleRepository(session_factory)
@@ -0,0 +1,39 @@
"""Abstract store for the global strategy-selection agent registry."""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Any
class StrategyAgentRoleStore(ABC):
@abstractmethod
async def ensure_defaults(self, rows: list[dict[str, Any]]) -> int:
"""Insert missing default rows without re-enabling disabled angles."""
@abstractmethod
async def list_roles(self, *, include_disabled: bool) -> list[dict[str, Any]]:
raise NotImplementedError
@abstractmethod
async def get_role(self, agent_id: str) -> dict[str, Any] | None:
raise NotImplementedError
@abstractmethod
async def create_role(self, data: dict[str, Any]) -> dict[str, Any]:
raise NotImplementedError
@abstractmethod
async def update_role(
self,
agent_id: str,
*,
enabled: bool | None = None,
sort_order: int | None = None,
) -> dict[str, Any] | None:
raise NotImplementedError
@abstractmethod
async def delete_role(self, agent_id: str) -> bool:
"""Hard-delete a registry row for create rollback only."""
raise NotImplementedError
@@ -0,0 +1,114 @@
"""In-memory strategy-agent registry used by memory persistence and tests."""
from __future__ import annotations
import asyncio
from datetime import UTC, datetime
from typing import Any
from deerflow.persistence.strategy_agent_roles.base import StrategyAgentRoleStore
def _now() -> str:
return datetime.now(UTC).isoformat()
def _copy(row: dict[str, Any]) -> dict[str, Any]:
return dict(row)
class MemoryStrategyAgentRoleStore(StrategyAgentRoleStore):
def __init__(self) -> None:
self._rows: dict[str, dict[str, Any]] = {}
self._lock = asyncio.Lock()
async def ensure_defaults(self, rows: list[dict[str, Any]]) -> int:
inserted = 0
async with self._lock:
for source in rows:
agent_id = str(source["agent_id"])
existing = self._rows.get(agent_id)
if existing is not None:
# Canonical fixed roles are executable infrastructure, not
# administrator-managed lifecycle rows. Repair stale or
# manually-corrupted metadata on every bootstrap. Angle
# defaults deliberately retain their enabled/order state.
if bool(source.get("fixed", False)):
canonical = {
"category": str(source["category"]),
"fixed": True,
"enabled": bool(source.get("enabled", True)),
"sort_order": int(source.get("sort_order", 0)),
}
if any(existing.get(key) != value for key, value in canonical.items()):
existing.update(canonical)
existing["updated_at"] = _now()
continue
now = _now()
self._rows[agent_id] = {
"agent_id": agent_id,
"category": str(source["category"]),
"fixed": bool(source.get("fixed", False)),
"enabled": bool(source.get("enabled", True)),
"sort_order": int(source.get("sort_order", 0)),
"created_at": now,
"updated_at": now,
}
inserted += 1
return inserted
async def list_roles(self, *, include_disabled: bool) -> list[dict[str, Any]]:
async with self._lock:
rows = [_copy(row) for row in self._rows.values() if include_disabled or bool(row.get("enabled"))]
rows.sort(
key=lambda row: (
int(row.get("sort_order", 0)),
str(row.get("agent_id", "")),
)
)
return rows
async def get_role(self, agent_id: str) -> dict[str, Any] | None:
async with self._lock:
row = self._rows.get(agent_id)
return _copy(row) if row is not None else None
async def create_role(self, data: dict[str, Any]) -> dict[str, Any]:
agent_id = str(data["agent_id"])
async with self._lock:
if agent_id in self._rows:
raise ValueError(f"Strategy agent role '{agent_id}' already exists")
now = _now()
row = {
"agent_id": agent_id,
"category": str(data["category"]),
"fixed": bool(data.get("fixed", False)),
"enabled": bool(data.get("enabled", True)),
"sort_order": int(data.get("sort_order", 0)),
"created_at": now,
"updated_at": now,
}
self._rows[agent_id] = row
return _copy(row)
async def update_role(
self,
agent_id: str,
*,
enabled: bool | None = None,
sort_order: int | None = None,
) -> dict[str, Any] | None:
async with self._lock:
row = self._rows.get(agent_id)
if row is None:
return None
if enabled is not None:
row["enabled"] = bool(enabled)
if sort_order is not None:
row["sort_order"] = int(sort_order)
row["updated_at"] = _now()
return _copy(row)
async def delete_role(self, agent_id: str) -> bool:
async with self._lock:
return self._rows.pop(agent_id, None) is not None
@@ -0,0 +1,36 @@
"""Persistence model for the global strategy-selection agent registry."""
from __future__ import annotations
from datetime import UTC, datetime
from sqlalchemy import Boolean, Integer, String
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
from deerflow.persistence.types import BeijingDateTime
class StrategyAgentRoleRow(Base):
"""Classify folder-backed agents used by the strategy-selection workflow.
The agent's model, skills and SOUL remain authoritative in its normal
``config.yaml`` / ``SOUL.md`` files. This small global table owns only the
workflow classification and lifecycle flags. Keeping a disabled row (and
the underlying agent) protects saved projects that still reference it.
"""
__tablename__ = "strategy_agent_roles"
agent_id: Mapped[str] = mapped_column(String(64), primary_key=True)
category: Mapped[str] = mapped_column(String(16), nullable=False, index=True)
fixed: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="1", index=True)
sort_order: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), nullable=False, default=lambda: datetime.now(UTC))
updated_at: Mapped[datetime] = mapped_column(
BeijingDateTime(),
nullable=False,
default=lambda: datetime.now(UTC),
onupdate=lambda: datetime.now(UTC),
)
@@ -0,0 +1,141 @@
"""SQLAlchemy-backed global strategy-selection agent registry."""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import delete, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.strategy_agent_roles.base import StrategyAgentRoleStore
from deerflow.persistence.strategy_agent_roles.model import StrategyAgentRoleRow
def _to_dict(row: StrategyAgentRoleRow) -> dict[str, Any]:
return {
"agent_id": row.agent_id,
"category": row.category,
"fixed": row.fixed,
"enabled": row.enabled,
"sort_order": row.sort_order,
"created_at": row.created_at.isoformat() if isinstance(row.created_at, datetime) else row.created_at,
"updated_at": row.updated_at.isoformat() if isinstance(row.updated_at, datetime) else row.updated_at,
}
class StrategyAgentRoleRepository(StrategyAgentRoleStore):
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
async def ensure_defaults(self, rows: list[dict[str, Any]]) -> int:
inserted = 0
for source in rows:
# Commit each of the small number of defaults independently. If
# two workers race on one primary key, rolling that save back must
# not also discard unrelated defaults inserted by this worker.
async with self._sf() as session:
agent_id = str(source["agent_id"])
existing = await session.get(StrategyAgentRoleRow, agent_id)
if existing is not None:
if bool(source.get("fixed", False)):
changed = False
canonical = {
"category": str(source["category"]),
"fixed": True,
"enabled": bool(source.get("enabled", True)),
"sort_order": int(source.get("sort_order", 0)),
}
for key, value in canonical.items():
if getattr(existing, key) != value:
setattr(existing, key, value)
changed = True
if changed:
existing.updated_at = datetime.now(UTC)
await session.commit()
continue
now = datetime.now(UTC)
session.add(
StrategyAgentRoleRow(
agent_id=agent_id,
category=str(source["category"]),
fixed=bool(source.get("fixed", False)),
enabled=bool(source.get("enabled", True)),
sort_order=int(source.get("sort_order", 0)),
created_at=now,
updated_at=now,
)
)
try:
await session.commit()
except IntegrityError:
# Another startup worker won this primary-key race.
await session.rollback()
continue
inserted += 1
return inserted
async def list_roles(self, *, include_disabled: bool) -> list[dict[str, Any]]:
stmt = select(StrategyAgentRoleRow)
if not include_disabled:
stmt = stmt.where(StrategyAgentRoleRow.enabled.is_(True))
stmt = stmt.order_by(
StrategyAgentRoleRow.sort_order.asc(),
StrategyAgentRoleRow.agent_id.asc(),
)
async with self._sf() as session:
result = await session.execute(stmt)
return [_to_dict(row) for row in result.scalars()]
async def get_role(self, agent_id: str) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(StrategyAgentRoleRow, agent_id)
return _to_dict(row) if row is not None else None
async def create_role(self, data: dict[str, Any]) -> dict[str, Any]:
now = datetime.now(UTC)
row = StrategyAgentRoleRow(
agent_id=str(data["agent_id"]),
category=str(data["category"]),
fixed=bool(data.get("fixed", False)),
enabled=bool(data.get("enabled", True)),
sort_order=int(data.get("sort_order", 0)),
created_at=now,
updated_at=now,
)
async with self._sf() as session:
session.add(row)
try:
await session.commit()
except IntegrityError as exc:
await session.rollback()
raise ValueError(f"Strategy agent role '{row.agent_id}' already exists") from exc
await session.refresh(row)
return _to_dict(row)
async def update_role(
self,
agent_id: str,
*,
enabled: bool | None = None,
sort_order: int | None = None,
) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(StrategyAgentRoleRow, agent_id)
if row is None:
return None
if enabled is not None:
row.enabled = bool(enabled)
if sort_order is not None:
row.sort_order = int(sort_order)
row.updated_at = datetime.now(UTC)
await session.commit()
await session.refresh(row)
return _to_dict(row)
async def delete_role(self, agent_id: str) -> bool:
async with self._sf() as session:
result = await session.execute(delete(StrategyAgentRoleRow).where(StrategyAgentRoleRow.agent_id == agent_id))
await session.commit()
return (result.rowcount or 0) > 0
@@ -21,9 +21,10 @@ class InlineThinkTagFilter:
_OPEN_TAG = "<think>"
_CLOSE_TAG = "</think>"
def __init__(self) -> None:
def __init__(self, *, allow_implicit_close: bool = False) -> None:
self._inside_think = False
self._pending = ""
self._channel_decided = not allow_implicit_close
@staticmethod
def _possible_tag_prefix_size(content: str, tag: str) -> int:
@@ -44,6 +45,23 @@ class InlineThinkTagFilter:
visible: list[str] = []
thinking: list[str] = []
if not self._channel_decided:
lowered = remaining.lower()
open_at = lowered.find(self._OPEN_TAG)
close_at = lowered.find(self._CLOSE_TAG)
if close_at >= 0 and (open_at < 0 or close_at < open_at):
# Some templates inject the opening tag outside returned
# content. Keep the ambiguous prefix private until its close
# marker arrives, even when that marker spans chunks.
thinking.append(remaining[:close_at])
remaining = remaining[close_at + len(self._CLOSE_TAG) :]
self._channel_decided = True
elif open_at >= 0:
self._channel_decided = True
else:
self._pending = remaining
return "", ""
while remaining:
tag = self._CLOSE_TAG if self._inside_think else self._OPEN_TAG
marker_at = remaining.lower().find(tag)
@@ -65,8 +83,27 @@ class InlineThinkTagFilter:
"""Compatibility helper returning only confirmed visible text."""
return self.push_parts(chunk)[0]
def confirm_visible_channel(self) -> str:
"""Stop close-only probing and return any buffered visible prelude.
A provider that emits a dedicated reasoning field has already told us
how to classify its trace. Its ordinary ``content`` channel can then
stream immediately even when the model supports thinking.
"""
if self._channel_decided:
return ""
self._channel_decided = True
pending = self._pending
self._pending = ""
return pending
def finish_parts(self) -> tuple[str, str]:
"""Flush the remaining visible or thinking text at end of stream."""
if not self._channel_decided:
tail = self._pending
self._pending = ""
self._channel_decided = True
return tail, ""
if self._inside_think:
thinking = self._pending
self._pending = ""
@@ -2,7 +2,7 @@ import pytest
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage
from deerflow.agents.middlewares.writing_setup_round_cap_middleware import WritingSetupRoundCapMiddleware
from deerflow.models.message_ordering import normalize_chat_payload_messages, normalize_langchain_messages
from deerflow.models.message_ordering import normalize_chat_payload_messages, normalize_langchain_messages, normalize_model_request_messages
from deerflow.models.patched_openai import PatchedChatOpenAI
@@ -49,6 +49,37 @@ def test_normalize_langchain_messages_moves_late_systems_before_history():
assert normalized[1].content == "user 0"
def test_normalize_chat_payload_messages_merges_adjacent_leading_systems():
payload = {
"messages": [
{"role": "system", "content": "agent prompt"},
{"role": "system", "content": "compaction summary"},
{"role": "user", "content": "continue"},
]
}
normalize_chat_payload_messages(payload)
assert [message["role"] for message in payload["messages"]] == ["system", "user"]
assert payload["messages"][0]["content"] == "agent prompt\n\n---\n\ncompaction summary"
def test_normalize_model_request_messages_folds_compaction_summary_into_system_slot():
system_message, messages = normalize_model_request_messages(
SystemMessage(content="agent prompt"),
[
SystemMessage(content="compaction summary"),
HumanMessage(content="continue"),
],
)
assert system_message is not None
assert system_message.content == "agent prompt\n\n---\n\ncompaction summary"
assert len(messages) == 1
assert isinstance(messages[0], HumanMessage)
assert all(not isinstance(message, SystemMessage) for message in messages)
@pytest.mark.parametrize("turns", [12, 25, 50])
def test_patched_openai_payload_has_only_leading_system_after_many_turns(turns):
model = PatchedChatOpenAI(
@@ -11,9 +11,11 @@ import asyncio
from types import SimpleNamespace
import httpx
from langchain_core.messages import AIMessage
import pytest
from langchain_core.messages import AIMessage, AIMessageChunk
from deerflow.agents.middlewares.llm_error_handling_middleware import LLM_ERROR_MARKER, LLMErrorHandlingMiddleware
from deerflow.models.patched_openai import OpenAICompatibleUpstreamError, PatchedChatOpenAI
def _middleware() -> LLMErrorHandlingMiddleware:
@@ -26,6 +28,10 @@ def _middleware() -> LLMErrorHandlingMiddleware:
return middleware
def _openai_model() -> PatchedChatOpenAI:
return PatchedChatOpenAI(model="test-model", api_key="test-key")
def test_httpx_read_timeout_is_transient() -> None:
assert _middleware()._classify_error(httpx.ReadTimeout("read timed out")) == (True, "transient")
@@ -46,6 +52,98 @@ def test_value_error_stays_generic() -> None:
assert _middleware()._classify_error(ValueError("boom")) == (False, "generic")
def test_openai_compatible_embedded_error_is_retriable_upstream() -> None:
error = RuntimeError("OpenAI-compatible upstream application error (code=2056)")
assert _middleware()._classify_error(error) == (True, "upstream")
def test_openai_compatible_null_choices_error_is_retriable_upstream() -> None:
error = TypeError("Received response with null value for 'choices'.")
assert _middleware()._classify_error(error) == (True, "upstream")
def test_insufficient_balance_stays_non_retriable_quota() -> None:
"""The 200-body compatibility fix must not hide a real billing failure."""
class PaymentError(RuntimeError):
status_code = 402
error = PaymentError("Insufficient Balance")
assert _middleware()._classify_error(error) == (False, "quota")
def test_minimax_token_plan_limit_stays_non_retriable_quota() -> None:
error = OpenAICompatibleUpstreamError(
code=2056,
message="Token Plan limit reached",
)
assert _middleware()._classify_error(error) == (False, "quota")
def test_non_stream_embedded_application_error_is_raised() -> None:
response = {
"id": "failed-response",
"object": "chat.completion",
"created": 0,
"model": "test-model",
"choices": None,
"base_resp": {
"status_code": 2056,
"status_msg": "Token Plan limit reached",
},
}
with pytest.raises(OpenAICompatibleUpstreamError, match=r"code=2056") as exc_info:
_openai_model()._create_chat_result(response)
assert exc_info.value.code == 2056
assert "Token Plan limit reached" in str(exc_info.value)
def test_stream_embedded_application_error_is_raised() -> None:
chunk = {
"id": "failed-response",
"object": "chat.completion.chunk",
"created": 0,
"model": "test-model",
"choices": [],
"base_resp": {
"status_code": 2056,
"status_msg": "Token Plan limit reached",
},
}
with pytest.raises(OpenAICompatibleUpstreamError, match=r"code=2056"):
_openai_model()._convert_chunk_to_generation_chunk(chunk, AIMessageChunk, None)
def test_normal_empty_usage_chunk_is_not_treated_as_error() -> None:
chunk = {
"id": "usage-response",
"object": "chat.completion.chunk",
"created": 0,
"model": "test-model",
"choices": [],
"usage": {
"prompt_tokens": 3,
"completion_tokens": 2,
"total_tokens": 5,
},
}
result = _openai_model()._convert_chunk_to_generation_chunk(chunk, AIMessageChunk, None)
assert result is not None
assert result.message.usage_metadata == {
"input_tokens": 3,
"output_tokens": 2,
"total_tokens": 5,
"input_token_details": {},
"output_token_details": {},
}
def test_read_timeout_is_retried_and_recovers() -> None:
"""第一次 ReadTimeout、第二次成功:必须重试并返回真实结果。"""
middleware = _middleware()
@@ -0,0 +1,283 @@
from __future__ import annotations
import asyncio
from pathlib import Path
from threading import Event
from types import SimpleNamespace
import pytest
import yaml
from fastapi import FastAPI
from starlette.requests import Request
from app.gateway import llmwiki_index_runtime
from app.gateway.routers import llmwiki_index
from deerflow.config.llmwiki_config import LocalWikiEmbeddingConfig, LocalWikiIndexConfig
from deerflow.config.system_settings import LocalWikiEncoderSettings, SystemSettings
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient, WikiEmbeddingError
def _source_config(embedding: LocalWikiEmbeddingConfig):
return SimpleNamespace(
llmwiki=SimpleNamespace(
local_wiki_index=LocalWikiIndexConfig(enabled=True, embedding=embedding)
)
)
def test_runtime_mode_can_switch_remote_config_to_uploaded_local_model(tmp_path: Path) -> None:
configured = LocalWikiEmbeddingConfig(
provider="openai_compatible",
model="remote-embedding",
base_url="https://embedding.example.test/v1",
dimensions=1536,
)
source = _source_config(configured)
normal = llmwiki_index_runtime.resolve_effective_local_wiki_index_config(
source,
LocalWikiEncoderSettings(mode="configured"),
)
assert normal.embedding.provider == "openai_compatible"
assert normal.embedding.model == "remote-embedding"
assert normal.embedding.dimensions == 1536
local = llmwiki_index_runtime.resolve_effective_local_wiki_index_config(
source,
LocalWikiEncoderSettings(
mode="local",
model_path=str(tmp_path),
model_revision="revision-2",
),
)
assert local.embedding.provider == "local"
assert local.embedding.model == "bge-embedding-m3"
assert local.embedding.dimensions == 1024
assert local.embedding.local_model_path == str(tmp_path)
assert local.embedding.local_model_revision == "revision-2"
assert local.embedding.base_url == ""
assert local.embedding.api_key == ""
def test_local_model_revision_changes_embedding_fingerprint(tmp_path: Path) -> None:
common = {
"provider": "local",
"model": "bge-embedding-m3",
"dimensions": 1024,
"local_model_path": str(tmp_path),
}
first = StrictWikiEmbeddingClient(
LocalWikiEmbeddingConfig(**common, local_model_revision="first")
)
second = StrictWikiEmbeddingClient(
LocalWikiEmbeddingConfig(**common, local_model_revision="second")
)
assert first.fingerprint != second.fingerprint
def test_local_model_path_changes_embedding_fingerprint(tmp_path: Path) -> None:
first_path = tmp_path / "revision-a"
second_path = tmp_path / "revision-b"
common = {
"provider": "local",
"model": "bge-embedding-m3",
"dimensions": 1024,
"local_model_revision": "same-revision",
}
first = StrictWikiEmbeddingClient(
LocalWikiEmbeddingConfig(**common, local_model_path=str(first_path))
)
second = StrictWikiEmbeddingClient(
LocalWikiEmbeddingConfig(**common, local_model_path=str(second_path))
)
assert first.fingerprint != second.fingerprint
def test_uploaded_local_mode_always_uses_bge_m3_contract(tmp_path: Path) -> None:
configured = LocalWikiEmbeddingConfig(
provider="local",
model="another-local-model",
local_model_path=str(tmp_path / "configured"),
dimensions=384,
)
effective = llmwiki_index_runtime.resolve_effective_local_wiki_index_config(
_source_config(configured),
LocalWikiEncoderSettings(
mode="local",
model_path=str(tmp_path / "uploaded"),
model_revision="uploaded-revision",
),
)
assert effective.embedding.provider == "local"
assert effective.embedding.model == "bge-embedding-m3"
assert effective.embedding.dimensions == 1024
assert effective.embedding.local_model_path == str(tmp_path / "uploaded")
def test_local_provider_never_uses_residual_base_url_as_readiness(tmp_path: Path) -> None:
config = LocalWikiIndexConfig(
enabled=True,
embedding=LocalWikiEmbeddingConfig(
provider="local",
model="bge-embedding-m3",
local_model_path=str(tmp_path / "missing"),
base_url="https://stale.example.test/v1",
dimensions=1024,
),
)
assert llmwiki_index_runtime._embedding_provider_ready(config) is False
@pytest.mark.asyncio
async def test_retired_embedding_rejects_future_work() -> None:
client = StrictWikiEmbeddingClient(
LocalWikiEmbeddingConfig(
provider="openai",
model="text-embedding-3-small",
base_url="https://api.openai.com/v1",
dimensions=1536,
)
)
await client.retire()
with pytest.raises(WikiEmbeddingError) as exc_info:
await client.embed_query("retry me")
assert getattr(exc_info.value, "category", None) == "runtime_retired"
@pytest.mark.asyncio
async def test_cancelled_local_embedding_drains_worker_thread_before_releasing_lock(
monkeypatch,
) -> None:
client = StrictWikiEmbeddingClient(
LocalWikiEmbeddingConfig(
provider="local",
model="bge-embedding-m3",
dimensions=2,
)
)
started = Event()
release = Event()
def blocking_embed(_texts):
started.set()
release.wait(timeout=5)
return [[1.0, 0.0]]
monkeypatch.setattr(client, "_embed_local", blocking_embed)
embedding_task = asyncio.create_task(client.embed_texts(["text"]))
assert await asyncio.to_thread(started.wait, 1)
embedding_task.cancel()
await asyncio.sleep(0)
assert not embedding_task.done()
assert client._active_uses == 1
embedding_task.cancel()
await asyncio.sleep(0)
assert not embedding_task.done()
assert client._active_uses == 1
release.set()
with pytest.raises(asyncio.CancelledError):
await embedding_task
assert client._active_uses == 0
@pytest.mark.asyncio
async def test_status_is_available_for_remote_provider(monkeypatch) -> None:
configured = LocalWikiEmbeddingConfig(
provider="openai",
model="text-embedding-3-small",
base_url="https://api.openai.com/v1",
dimensions=1536,
)
app = FastAPI()
app.state.config = _source_config(configured)
app.state.llmwiki_effective_config = app.state.config.llmwiki.local_wiki_index
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(configured)
request = Request({"type": "http", "headers": [], "app": app})
async def admin(_request):
return "admin", True
async def unchanged(_app, *, force=False):
return _app.state.llmwiki_effective_config
monkeypatch.setattr(llmwiki_index, "_actor", admin)
monkeypatch.setattr(llmwiki_index, "rebuild_local_wiki_index_runtime", unchanged)
monkeypatch.setattr(
llmwiki_index_runtime,
"load_system_settings",
lambda: SystemSettings(
local_wiki_encoder=LocalWikiEncoderSettings(mode="configured")
),
)
result = await llmwiki_index.local_encoder_status(request)
assert result["provider"] == "openai"
assert result["configured_provider"] == "openai"
assert result["mode"] == "configured"
assert result["using_local"] is False
assert result["state"] == "idle"
@pytest.mark.asyncio
async def test_local_path_save_and_restore_persist_mode_and_rebuild(
monkeypatch,
tmp_path: Path,
) -> None:
model_dir = tmp_path / "bge-m3"
(model_dir / "onnx").mkdir(parents=True)
(model_dir / "onnx" / "model.onnx").write_bytes(b"onnx")
app = FastAPI()
request = Request({"type": "http", "headers": [], "app": app})
saved: list[SystemSettings] = []
current = SystemSettings()
rebuilds: list[bool] = []
async def admin(_request):
return "admin", True
async def rebuilt(_app, *, force=False):
rebuilds.append(force)
return LocalWikiIndexConfig()
monkeypatch.setattr(llmwiki_index, "_actor", admin)
monkeypatch.setattr(llmwiki_index, "load_system_settings", lambda: saved[-1] if saved else current)
monkeypatch.setattr(llmwiki_index, "save_system_settings", lambda value: saved.append(value) or value)
monkeypatch.setattr(llmwiki_index, "rebuild_local_wiki_index_runtime", rebuilt)
monkeypatch.setattr(
llmwiki_index,
"local_encoder_runtime_status",
lambda _app: {
"provider": "local" if saved[-1].local_wiki_encoder.mode == "local" else "openai",
"mode": saved[-1].local_wiki_encoder.mode,
},
)
local = await llmwiki_index.configure_local_encoder(
request,
llmwiki_index.LocalEncoderConfigRequest(model_path=str(model_dir)),
)
assert local["mode"] == "local"
assert saved[-1].local_wiki_encoder.mode == "local"
assert saved[-1].local_wiki_encoder.model_revision
configured = await llmwiki_index.use_configured_encoder(request)
assert configured["mode"] == "configured"
assert saved[-1].local_wiki_encoder.mode == "configured"
assert rebuilds == [True, True]
def test_both_shipped_offline_configs_enable_incremental_wiki_rescan() -> None:
offline_root = Path(__file__).resolve().parents[2]
for path in (offline_root / "config.yaml", offline_root / "backend" / "config.yaml"):
payload = yaml.safe_load(path.read_text(encoding="utf-8"))
local_index = payload["llmwiki"]["local_wiki_index"]
assert local_index["enabled"] is True
assert local_index["auto_sync"] is True
@@ -497,6 +497,129 @@ async def test_weknora_client_resolves_default_embedding_model_inside_weknora()
assert created["id"] == "kb-created"
@pytest.mark.asyncio
async def test_gateway_lists_safe_embedding_models_and_uses_explicit_selection(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from app.gateway.routers import llmwiki as llmwiki_router
create_calls: list[dict[str, object]] = []
class Client:
async def list_models(self) -> list[dict[str, object]]:
return [
{
"id": "embed-local",
"type": "Embedding",
"name": "Local BGE",
"provider": "local",
"status": "active",
"is_default": True,
"api_key": "must-not-leak",
"config": {"secret": "must-not-leak"},
},
{
"id": "embed-openai",
"type": "embedding",
"display_name": "OpenAI text-embedding",
"source": "remote",
"parameters": {
"model": "text-embedding-3-large",
"provider": "openai",
"api_key": "must-not-leak",
},
"status": "active",
},
{
"id": "embed-disabled",
"type": "Embedding",
"name": "Disabled",
"status": "inactive",
},
{"id": "chat-one", "type": "LLM", "name": "chat"},
]
async def create_knowledge_base(self, **kwargs: object) -> dict[str, object]:
create_calls.append(kwargs)
return {
"id": "remote-openai",
"name": kwargs["name"],
"description": kwargs["description"],
"type": kwargs["kb_type"],
"embedding_model_id": kwargs["embedding_model_id"],
}
async def actor(_: object) -> tuple[str, bool]:
return "user-a", False
store = MemoryLlmWikiStore()
client = Client()
monkeypatch.setattr(llmwiki_router, "_actor", actor)
monkeypatch.setattr(llmwiki_router, "_client_or_503", lambda: client)
monkeypatch.setattr(llmwiki_router, "_store", lambda _: store)
monkeypatch.setattr(llmwiki_router, "_schedule_local_wiki_sync", lambda *_: None)
listed = await llmwiki_router.list_embedding_models(object())
assert listed["default_model_id"] == "embed-local"
assert [item["id"] for item in listed["models"]] == [
"embed-local",
"embed-openai",
"embed-disabled",
]
assert listed["models"][1]["provider"] == "openai"
assert listed["models"][1]["model_name"] == "text-embedding-3-large"
assert listed["models"][2]["available"] is False
assert "api_key" not in listed["models"][0]
assert "config" not in listed["models"][0]
assert "parameters" not in listed["models"][1]
created = await llmwiki_router.create_knowledge_base(
object(),
llmwiki_router.KnowledgeBaseCreate(
name="OpenAI docs",
embedding_model_id="embed-openai",
),
)
assert create_calls[0]["embedding_model_id"] == "embed-openai"
assert created["embedding_model_id"] == "embed-openai"
for rejected_model_id in ("embed-disabled", "chat-one", "missing"):
with pytest.raises(HTTPException) as exc_info:
await llmwiki_router.create_knowledge_base(
object(),
llmwiki_router.KnowledgeBaseCreate(
name="Rejected docs",
embedding_model_id=rejected_model_id,
),
)
assert exc_info.value.status_code == 422
assert len(create_calls) == 1
@pytest.mark.asyncio
async def test_gateway_embedding_model_list_allows_empty_configuration(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from app.gateway.routers import llmwiki as llmwiki_router
class Client:
async def list_models(self) -> list[dict[str, object]]:
return [{"id": "chat-only", "type": "LLM"}]
async def actor(_: object) -> tuple[str, bool]:
return "user-a", False
monkeypatch.setattr(llmwiki_router, "_actor", actor)
monkeypatch.setattr(llmwiki_router, "_client_or_503", lambda: Client())
assert await llmwiki_router.list_embedding_models(object()) == {
"models": [],
"default_model_id": None,
}
@pytest.mark.asyncio
async def test_weknora_client_creates_wiki_with_weknora_owned_models() -> None:
model_reads = 0
@@ -199,34 +199,101 @@ async def test_fingerprint_change_automatically_rebuilds_entire_mapping() -> Non
@pytest.mark.asyncio
async def test_auto_sync_scheduler_scans_immediately_then_waits(monkeypatch) -> None:
calls: list[str] = []
async def test_auto_sync_scheduler_scans_immediately_then_only_when_due(monkeypatch) -> None:
now = 0.0
refreshes: list[float] = []
syncs: list[float] = []
sleeps: list[float] = []
config = SimpleNamespace(
enabled=True,
auto_sync=True,
sync_interval_seconds=25,
)
async def fake_refresh(_app):
refreshes.append(now)
return config
async def fake_sleep(seconds: float) -> None:
calls.append(f"sleep:{seconds}")
raise asyncio.CancelledError
async def fake_sync_once(app) -> None:
calls.append("sync")
if calls.count("sync") > 1:
nonlocal now
sleeps.append(seconds)
if len(syncs) >= 2:
raise asyncio.CancelledError
now += seconds
async def fake_sync_once(_app, *, config=None) -> None:
syncs.append(now)
monkeypatch.setattr(llmwiki_index_scheduler, "monotonic", lambda: now)
monkeypatch.setattr(llmwiki_index_scheduler.asyncio, "sleep", fake_sleep)
monkeypatch.setattr(llmwiki_index_scheduler, "_sync_once", fake_sync_once)
monkeypatch.setattr(
llmwiki_index_scheduler,
"_refresh_local_wiki_config",
fake_refresh,
)
app = SimpleNamespace(
state=SimpleNamespace(
config=SimpleNamespace(
llmwiki=SimpleNamespace(
local_wiki_index=SimpleNamespace(sync_interval_seconds=120),
)
)
llmwiki_sync_service=object(),
llmwiki_store=object(),
)
)
with pytest.raises(asyncio.CancelledError):
await llmwiki_index_scheduler.local_wiki_index_scheduler_loop(app)
assert calls == ["sync", "sleep:120"]
assert refreshes == [0.0, 10.0, 20.0, 30.0]
assert syncs == [0.0, 30.0]
assert sleeps == [10.0, 10.0, 10.0, 10.0]
@pytest.mark.asyncio
async def test_auto_sync_scheduler_refreshes_provider_between_syncs(monkeypatch) -> None:
now = 0.0
seen_providers: list[str] = []
syncs: list[str] = []
providers = iter(("openai", "local", "local"))
async def fake_refresh(_app):
provider = next(providers)
seen_providers.append(provider)
_app.state.llmwiki_runtime_signature = provider
return SimpleNamespace(
enabled=True,
auto_sync=True,
sync_interval_seconds=120,
embedding=SimpleNamespace(provider=provider),
)
async def fake_sync_once(_app, *, config=None) -> None:
syncs.append(config.embedding.provider)
async def fake_sleep(seconds: float) -> None:
nonlocal now
now += seconds
if len(seen_providers) >= 3:
raise asyncio.CancelledError
monkeypatch.setattr(llmwiki_index_scheduler, "monotonic", lambda: now)
monkeypatch.setattr(llmwiki_index_scheduler.asyncio, "sleep", fake_sleep)
monkeypatch.setattr(llmwiki_index_scheduler, "_sync_once", fake_sync_once)
monkeypatch.setattr(
llmwiki_index_scheduler,
"_refresh_local_wiki_config",
fake_refresh,
)
app = SimpleNamespace(
state=SimpleNamespace(
llmwiki_sync_service=object(),
llmwiki_store=object(),
)
)
with pytest.raises(asyncio.CancelledError):
await llmwiki_index_scheduler.local_wiki_index_scheduler_loop(app)
assert seen_providers == ["openai", "local", "local"]
assert syncs == ["openai", "local"]
def test_local_wiki_index_defaults_to_automatic_sync() -> None:
@@ -13,14 +13,20 @@
from __future__ import annotations
import asyncio
import json
from contextlib import asynccontextmanager
from types import SimpleNamespace
import pytest
import app.gateway.roundtable_model_fallback as mf
import app.gateway.routers.parallel_agents as parallel_agents
from app.gateway.roundtable_model_fallback import (
fallback_model_chain,
is_llm_error_message,
reason_text,
)
from app.gateway.routers._strategy_selection_seed import default_strategy_agent_roles
from deerflow.agents.middlewares.llm_error_handling_middleware import (
LLM_ERROR_MARKER,
LLMErrorHandlingMiddleware,
@@ -56,6 +62,520 @@ def test_is_llm_error_message_normal_is_none() -> None:
assert is_llm_error_message({"type": "ai", "content": ""}) is None
def test_parallel_agent_run_raises_instead_of_completing_llm_fallback(monkeypatch) -> None:
fallback_message = {
"type": "ai",
"content": "大模型本次调用未能成功完成,请稍后再试或更换模型。",
}
async def fake_stream_upstream(*_args, **_kwargs):
yield None, [f"data: {json.dumps({'messages': [fallback_message]}, ensure_ascii=False)}"]
monkeypatch.setattr(parallel_agents, "_stream_upstream", fake_stream_upstream)
async def collect() -> list[object]:
return [
item
async for item in parallel_agents._run_one_agent(
None, # type: ignore[arg-type]
"http://gateway.test",
{"Authorization": "Bearer test"},
"strategy-selection-intent",
"thread-id",
"test task",
"",
)
]
try:
asyncio.run(collect())
except RuntimeError as exc:
assert "模型调用失败" in str(exc)
else:
raise AssertionError("LLM fallback message must not be emitted as agent_done")
def test_parallel_agent_run_rejects_punctuation_only_delivery(monkeypatch) -> None:
invalid_message = {"type": "ai", "content": "!" * 500}
async def fake_stream_upstream(*_args, **_kwargs):
yield None, [f"data: {json.dumps({'messages': [invalid_message]}, ensure_ascii=False)}"]
monkeypatch.setattr(parallel_agents, "_stream_upstream", fake_stream_upstream)
async def collect() -> list[object]:
return [
item
async for item in parallel_agents._run_one_agent(
None, # type: ignore[arg-type]
"http://gateway.test",
{"Authorization": "Bearer test"},
"strategy-selection-angle-security",
"thread-id",
"test task",
"",
)
]
try:
asyncio.run(collect())
except RuntimeError as exc:
assert "未生成有效正文" in str(exc)
else:
raise AssertionError("Punctuation-only output must not be emitted as agent_done")
def test_parallel_agent_run_allows_empty_content_clarification(monkeypatch) -> None:
clarification_message = {
"type": "ai",
"content": "",
"tool_calls": [
{
"id": "clarify-1",
"name": "ask_clarification",
"args": {
"question": "请确认策略时间范围。",
"clarification_type": "single_choice",
"options": ["短期", "长期"],
"allow_custom": True,
},
}
],
}
async def fake_stream_upstream(*_args, **_kwargs):
yield None, [f"data: {json.dumps({'messages': [clarification_message]}, ensure_ascii=False)}"]
monkeypatch.setattr(parallel_agents, "_stream_upstream", fake_stream_upstream)
async def collect() -> list[object]:
return [
item
async for item in parallel_agents._run_one_agent(
None, # type: ignore[arg-type]
"http://gateway.test",
{"Authorization": "Bearer test"},
"strategy-selection-intent",
"thread-id",
"test task",
"",
)
]
events = asyncio.run(collect())
assert len(events) == 1
event = events[0]
assert isinstance(event, dict)
assert event["type"] == "clarification"
assert event["content"] == ""
assert event["clarification"]["question"] == "请确认策略时间范围。"
def test_parallel_agent_final_values_keep_typed_reasoning_out_of_content(monkeypatch) -> None:
final_message = {
"type": "ai",
"content": [
{"type": "thinking", "text": "核对约束与证据"},
{"type": "text", "text": "这是最终可见策略正文,包含目标、路径和验收标准。"},
],
}
async def fake_stream_upstream(*_args, **_kwargs):
yield None, [f"data: {json.dumps({'messages': [final_message]}, ensure_ascii=False)}"]
monkeypatch.setattr(parallel_agents, "_stream_upstream", fake_stream_upstream)
async def collect() -> list[object]:
return [
item
async for item in parallel_agents._run_one_agent(
None, # type: ignore[arg-type]
"http://gateway.test",
{"Authorization": "Bearer test"},
"strategy-selection-angle-security",
"thread-id",
"test task",
"",
)
]
events = asyncio.run(collect())
assert len(events) == 1
event = events[0]
assert isinstance(event, dict)
assert event["content"] == "这是最终可见策略正文,包含目标、路径和验收标准。"
assert event["reasoning"] == "核对约束与证据"
def test_parallel_agent_final_values_recovers_template_injected_reasoning(monkeypatch) -> None:
final_message = {
"type": "ai",
"content": "模板注入的内部推理</think>这是最终可见策略正文,包含目标、路径和验收标准。",
}
async def fake_stream_upstream(*_args, **_kwargs):
yield None, [f"data: {json.dumps({'messages': [final_message]}, ensure_ascii=False)}"]
monkeypatch.setattr(parallel_agents, "_stream_upstream", fake_stream_upstream)
async def collect() -> list[object]:
return [
item
async for item in parallel_agents._run_one_agent(
None, # type: ignore[arg-type]
"http://gateway.test",
{"Authorization": "Bearer test"},
"strategy-selection-angle-security",
"thread-id",
"test task",
"",
)
]
events = asyncio.run(collect())
event = events[0]
assert isinstance(event, dict)
assert event["content"] == "这是最终可见策略正文,包含目标、路径和验收标准。"
assert event["reasoning"] == "模板注入的内部推理"
def test_parallel_agent_substantive_content_accepts_chinese_and_english() -> None:
assert parallel_agents._has_substantive_content("这是一份包含目标、前置条件、阶段路径和验收标准的完整策略。")
assert parallel_agents._has_substantive_content("A complete strategy with objectives, prerequisites, milestones, and measurable outcomes.")
assert not parallel_agents._has_substantive_content("<think>internal reasoning only</think>!!!")
def test_parallel_agent_extracts_provider_reasoning_without_mixing_answer() -> None:
answer, reasoning = parallel_agents._extract_ai_stream_parts(
{
"type": "AIMessageChunk",
"content": [
{"type": "thinking_delta", "text": "先核对边界"},
{"type": "text", "text": "可见正文"},
],
"additional_kwargs": {"reasoning_content": ",再比较路径"},
}
)
assert answer == "可见正文"
assert reasoning == "先核对边界,再比较路径"
answer, reasoning = parallel_agents._extract_ai_stream_parts(
{
"type": "AIMessageChunk",
"content": [
{"type": "thinking", "thinking": "结构化思考字段"},
{"type": "text", "text": "最终正文"},
],
}
)
assert answer == "最终正文"
assert reasoning == "结构化思考字段"
class _EnabledStrategyRoleStore:
async def ensure_defaults(self, _rows) -> int:
return 0
async def list_roles(self, *, include_disabled: bool) -> list[dict[str, object]]:
del include_disabled
return [dict(row) for row in default_strategy_agent_roles()]
async def get_role(self, agent_id: str) -> dict[str, object]:
return {"id": agent_id, "enabled": True}
class _GlobalStrategyAgentStore:
async def get_any(self, agent_id: str) -> dict[str, object]:
return {"id": agent_id, "user_id": None}
def _parallel_request() -> SimpleNamespace:
return SimpleNamespace(
app=SimpleNamespace(
state=SimpleNamespace(
strategy_agent_role_store=_EnabledStrategyRoleStore(),
agent_store=_GlobalStrategyAgentStore(),
)
)
)
def test_parallel_agent_stream_separates_inline_and_provider_reasoning(monkeypatch) -> None:
async def fake_run_one_agent(*_args, **_kwargs):
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': '<thi'}], ensure_ascii=False)}"
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': 'nk>内部分析</think>方案正文'}], ensure_ascii=False)}"
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': '', 'additional_kwargs': {'reasoning_content': '补充推理'}}], ensure_ascii=False)}"
yield {
"status": "done",
"content": "<think>内部分析</think>方案正文",
"agent_name": "strategy-selection-angle-security",
"tool_calls": [],
}
@asynccontextmanager
async def fake_client(*_args, **_kwargs):
yield object()
async def fake_create_thread(*_args, **_kwargs):
return "thread-id"
monkeypatch.setattr(parallel_agents, "_run_one_agent", fake_run_one_agent)
monkeypatch.setattr(parallel_agents, "_make_loopback_client", fake_client)
monkeypatch.setattr(parallel_agents, "_create_thread", fake_create_thread)
monkeypatch.setattr(parallel_agents, "_loopback_base", lambda _request: "http://gateway.test")
monkeypatch.setattr(parallel_agents, "_auth_headers", lambda _request: {})
async def collect() -> list[dict[str, object]]:
response = await parallel_agents.run_parallel_agents_stream(
_parallel_request(),
{
"task_name": "test task",
"brief": "test brief",
"agent_ids": ["strategy-selection-angle-security"],
},
)
events: list[dict[str, object]] = []
async for chunk in response.body_iterator:
text = chunk.decode() if isinstance(chunk, bytes) else chunk
for line in text.splitlines():
if line.startswith("data:"):
events.append(json.loads(line[5:].strip()))
return events
events = asyncio.run(collect())
visible = "".join(str(event.get("text") or "") for event in events if event.get("type") == "delta")
thinking = "".join(str(event.get("text") or "") for event in events if event.get("type") == "reasoning_delta")
done = next(event for event in events if event.get("type") == "agent_done")
assert visible == "方案正文"
assert thinking == "内部分析补充推理"
assert done["content"] == "方案正文"
@pytest.mark.parametrize(
("buffered_content", "expected_error"),
[
("大模型本次调用未能成功完成,请稍后再试或更换模型。", "模型调用失败"),
("", "未生成有效正文"),
("<think>internal reasoning only</think>", "未生成有效正文"),
("!" * 500, "未生成有效正文"),
],
)
def test_parallel_agent_truncated_stream_rejects_invalid_buffer(
monkeypatch,
buffered_content: str,
expected_error: str,
) -> None:
async def fake_run_one_agent(*_args, **_kwargs):
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': buffered_content}], ensure_ascii=False)}"
@asynccontextmanager
async def fake_client(*_args, **_kwargs):
yield object()
async def fake_create_thread(*_args, **_kwargs):
return "thread-id"
monkeypatch.setattr(parallel_agents, "_run_one_agent", fake_run_one_agent)
monkeypatch.setattr(parallel_agents, "_make_loopback_client", fake_client)
monkeypatch.setattr(parallel_agents, "_create_thread", fake_create_thread)
monkeypatch.setattr(parallel_agents, "_loopback_base", lambda _request: "http://gateway.test")
monkeypatch.setattr(parallel_agents, "_auth_headers", lambda _request: {})
async def collect() -> list[dict[str, object]]:
response = await parallel_agents.run_parallel_agents_stream(
_parallel_request(),
{
"task_name": "test task",
"brief": "test brief",
"agent_ids": ["strategy-selection-intent"],
},
)
events: list[dict[str, object]] = []
async for chunk in response.body_iterator:
text = chunk.decode() if isinstance(chunk, bytes) else chunk
for line in text.splitlines():
if line.startswith("data:"):
events.append(json.loads(line[5:].strip()))
return events
events = asyncio.run(collect())
assert not any(event.get("type") == "agent_done" for event in events)
errors = [event for event in events if event.get("type") == "agent_error"]
assert len(errors) == 1
assert expected_error in str(errors[0].get("message"))
def test_parallel_agent_plain_text_streams_each_chunk_without_probe_delay(
monkeypatch,
) -> None:
chunks = [
"This visible strategy ",
"includes objectives and execution paths, ",
"plus measurable acceptance criteria.",
]
async def fake_run_one_agent(*_args, **_kwargs):
for content in chunks:
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': content}], ensure_ascii=False)}"
@asynccontextmanager
async def fake_client(*_args, **_kwargs):
yield object()
async def fake_create_thread(*_args, **_kwargs):
return "thread-id"
monkeypatch.setattr(parallel_agents, "_run_one_agent", fake_run_one_agent)
monkeypatch.setattr(parallel_agents, "_make_loopback_client", fake_client)
monkeypatch.setattr(parallel_agents, "_create_thread", fake_create_thread)
monkeypatch.setattr(
parallel_agents,
"_loopback_base",
lambda _request: "http://gateway.test",
)
monkeypatch.setattr(parallel_agents, "_auth_headers", lambda _request: {})
monkeypatch.setattr(
parallel_agents,
"_should_probe_implicit_reasoning",
lambda *_args: False,
)
async def collect() -> list[dict[str, object]]:
response = await parallel_agents.run_parallel_agents_stream(
_parallel_request(),
{
"task_name": "plain stream",
"brief": "plain answer chunks must stay live",
"agent_ids": ["strategy-selection-intent"],
},
)
events: list[dict[str, object]] = []
async for chunk in response.body_iterator:
text = chunk.decode() if isinstance(chunk, bytes) else chunk
for line in text.splitlines():
if line.startswith("data:"):
events.append(json.loads(line[5:].strip()))
return events
events = asyncio.run(collect())
assert [event["text"] for event in events if event.get("type") == "delta"] == chunks
done = next(event for event in events if event.get("type") == "agent_done")
assert done["content"] == "".join(chunks)
def test_parallel_agent_typed_reasoning_unlocks_live_answer_channel(monkeypatch) -> None:
chunks = [
"This visible strategy ",
"streams after typed reasoning, ",
"with objectives and acceptance criteria.",
]
async def fake_run_one_agent(*_args, **_kwargs):
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': '', 'additional_kwargs': {'reasoning_content': 'live reasoning'}}])}"
for content in chunks:
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': content}])}"
@asynccontextmanager
async def fake_client(*_args, **_kwargs):
yield object()
async def fake_create_thread(*_args, **_kwargs):
return "thread-id"
monkeypatch.setattr(parallel_agents, "_run_one_agent", fake_run_one_agent)
monkeypatch.setattr(parallel_agents, "_make_loopback_client", fake_client)
monkeypatch.setattr(parallel_agents, "_create_thread", fake_create_thread)
monkeypatch.setattr(parallel_agents, "_loopback_base", lambda _request: "http://gateway.test")
monkeypatch.setattr(parallel_agents, "_auth_headers", lambda _request: {})
monkeypatch.setattr(parallel_agents, "_should_probe_implicit_reasoning", lambda *_args: True)
async def collect() -> list[dict[str, object]]:
response = await parallel_agents.run_parallel_agents_stream(
_parallel_request(),
{
"task_name": "typed reasoning stream",
"brief": "answer chunks must stay live",
"agent_ids": ["strategy-selection-intent"],
},
)
events: list[dict[str, object]] = []
async for chunk in response.body_iterator:
text = chunk.decode() if isinstance(chunk, bytes) else chunk
for line in text.splitlines():
if line.startswith("data:"):
events.append(json.loads(line[5:].strip()))
return events
events = asyncio.run(collect())
assert [event["text"] for event in events if event.get("type") == "delta"] == chunks
assert "".join(str(event.get("text") or "") for event in events if event.get("type") == "reasoning_delta") == "live reasoning"
def test_parallel_agent_truncated_close_only_stream_separates_reasoning(monkeypatch) -> None:
async def fake_run_one_agent(*_args, **_kwargs):
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': 'internal '}], ensure_ascii=False)}"
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': 'reasoning</thi'}], ensure_ascii=False)}"
yield "event: messages"
yield f"data: {json.dumps([{'type': 'ai', 'content': 'nk>This visible strategy includes objectives, execution paths, and acceptance criteria.'}], ensure_ascii=False)}"
@asynccontextmanager
async def fake_client(*_args, **_kwargs):
yield object()
async def fake_create_thread(*_args, **_kwargs):
return "thread-id"
monkeypatch.setattr(parallel_agents, "_run_one_agent", fake_run_one_agent)
monkeypatch.setattr(parallel_agents, "_make_loopback_client", fake_client)
monkeypatch.setattr(parallel_agents, "_create_thread", fake_create_thread)
monkeypatch.setattr(parallel_agents, "_loopback_base", lambda _request: "http://gateway.test")
monkeypatch.setattr(parallel_agents, "_auth_headers", lambda _request: {})
monkeypatch.setattr(
parallel_agents,
"_should_probe_implicit_reasoning",
lambda *_args: True,
)
async def collect() -> list[dict[str, object]]:
response = await parallel_agents.run_parallel_agents_stream(
_parallel_request(),
{
"task_name": "test task",
"brief": "test brief",
"agent_ids": ["strategy-selection-intent"],
},
)
events: list[dict[str, object]] = []
async for chunk in response.body_iterator:
text = chunk.decode() if isinstance(chunk, bytes) else chunk
for line in text.splitlines():
if line.startswith("data:"):
events.append(json.loads(line[5:].strip()))
return events
events = asyncio.run(collect())
done = next(event for event in events if event.get("type") == "agent_done")
reasoning = "".join(str(event.get("text") or "") for event in events if event.get("type") == "reasoning_delta")
visible = "".join(str(event.get("text") or "") for event in events if event.get("type") == "delta")
assert done["content"] == ("This visible strategy includes objectives, execution paths, and acceptance criteria.")
assert visible == done["content"]
assert reasoning == "internal reasoning"
# ── 中间件给兜底 AIMessage 打标记 ────────────────────────────────────────────
@@ -0,0 +1,46 @@
from __future__ import annotations
from pathlib import Path
from deerflow.config.app_config import AppConfig
def test_relative_sandbox_mount_is_resolved_from_config_directory(tmp_path: Path) -> None:
knowledge_dir = tmp_path / "knowledge" / "knowledge_base"
knowledge_dir.mkdir(parents=True)
config_path = tmp_path / "config.yaml"
config_path.write_text(
"""
sandbox:
use: deerflow.sandbox.local:LocalSandboxProvider
mounts:
- host_path: knowledge/knowledge_base
container_path: /mnt/knowledge
read_only: true
""".strip(),
encoding="utf-8",
)
config = AppConfig.from_file(str(config_path))
assert config.sandbox.mounts[0].host_path == str(knowledge_dir.resolve())
assert Path(config.sandbox.mounts[0].host_path).is_absolute()
def test_absolute_sandbox_mount_is_preserved(tmp_path: Path) -> None:
absolute_mount = (tmp_path / "shared").resolve()
config_data = {
"sandbox": {
"use": "deerflow.sandbox.local:LocalSandboxProvider",
"mounts": [
{
"host_path": str(absolute_mount),
"container_path": "/mnt/shared",
}
],
}
}
AppConfig._resolve_relative_mount_paths(config_data, tmp_path / "elsewhere")
assert config_data["sandbox"]["mounts"][0]["host_path"] == str(absolute_mount)
@@ -0,0 +1,667 @@
"""Global strategy-agent role CRUD, permissions and visibility tests."""
from __future__ import annotations
import asyncio
from pathlib import Path
from types import SimpleNamespace
import pytest
import yaml
from fastapi import HTTPException
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.gateway.routers import _strategy_selection_seed as seed
from app.gateway.routers import agents as agents_router
from app.gateway.routers import parallel_agents, strategy_agents
from deerflow.config import agents_config
from deerflow.config.paths import Paths
from deerflow.persistence.agents import MemoryAgentStore
from deerflow.persistence.skills.memory import MemorySkillStore
from deerflow.persistence.strategy_agent_roles import MemoryStrategyAgentRoleStore, StrategyAgentRoleRepository
from deerflow.persistence.strategy_agent_roles.model import StrategyAgentRoleRow
from deerflow.skills.types import Skill, SkillCategory
class _SkillStorage:
def __init__(self, skills: list[Skill]) -> None:
self._skills = skills
def load_skills(self, *, enabled_only: bool) -> list[Skill]:
if enabled_only:
return [skill for skill in self._skills if skill.enabled]
return list(self._skills)
def _skill(name: str, category: SkillCategory = SkillCategory.PUBLIC) -> Skill:
directory = Path("skills") / category.value / name
return Skill(
name=name,
description=name,
license=None,
skill_dir=directory,
skill_file=directory / "SKILL.md",
relative_path=Path(name),
category=category,
enabled=True,
)
def test_generic_agent_namespace_guard_is_case_insensitive_and_canonical() -> None:
assert agents_router._is_strategy_agent_id("STRATEGY-SELECTION-INTENT")
assert strategy_agents._process_lock("STRATEGY-SELECTION-INTENT") is strategy_agents._process_lock("strategy-selection-intent")
with pytest.raises(HTTPException) as exc_info:
agents_router._require_canonical_record_id(
"STRATEGY-SELECTION-INTENT",
{"id": "strategy-selection-intent"},
)
assert exc_info.value.status_code == 409
@pytest.mark.asyncio
async def test_generic_mutations_reject_strategy_namespace_before_storage(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(
agents_router,
"get_agents_api_config",
lambda: SimpleNamespace(enabled=True),
)
request = SimpleNamespace()
with pytest.raises(HTTPException) as update_error:
await agents_router.update_agent(
seed.STRATEGY_INTENT_AGENT_ID.upper(),
agents_router.AgentUpdateRequest(name="bypass"),
request,
)
assert update_error.value.status_code == 409
assert "/api/strategy-agents/" in str(update_error.value.detail)
with pytest.raises(HTTPException) as delete_error:
await agents_router.delete_agent(
seed.STRATEGY_INTENT_AGENT_ID.upper(),
request,
)
assert delete_error.value.status_code == 409
assert "generic API" in str(delete_error.value.detail)
@pytest.mark.asyncio
async def test_strategy_mutation_rejects_case_insensitive_database_alias(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, agent_store, role_store = await _request(tmp_path, monkeypatch)
canonical_id = seed.STRATEGY_INTENT_AGENT_ID
await role_store.ensure_defaults(seed.default_strategy_agent_roles())
role = await role_store.get_role(canonical_id)
record = await agent_store.get_any(canonical_id)
assert role is not None and record is not None
monkeypatch.setattr(strategy_agents, "_ensure_registry", lambda *_: _async_none())
monkeypatch.setattr(role_store, "get_role", lambda _: _async_value(role))
monkeypatch.setattr(agent_store, "get_any", lambda _: _async_value(record))
with pytest.raises(HTTPException) as exc_info:
await strategy_agents._get_role_and_agent(request, canonical_id.upper())
assert exc_info.value.status_code == 409
async def _async_none() -> None:
return None
async def _async_value(value):
return value
async def _request(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
*,
role: str = "admin",
) -> tuple[SimpleNamespace, MemoryAgentStore, MemoryStrategyAgentRoleStore]:
paths = Paths(tmp_path)
monkeypatch.setattr(seed, "get_paths", lambda: paths)
monkeypatch.setattr(agents_router, "get_paths", lambda: paths)
monkeypatch.setattr(strategy_agents, "get_paths", lambda: paths)
monkeypatch.setattr(agents_config, "get_paths", lambda: paths)
seed.ensure_strategy_selection_agents()
agent_store = MemoryAgentStore()
for agent_id in seed.required_agent_ids():
config = yaml.safe_load((paths.agent_dir(agent_id) / "config.yaml").read_text(encoding="utf-8"))
await agent_store.ensure_builtin(
{
"id": agent_id,
"name": config["name"],
"description": config.get("description") or "",
"published": True,
}
)
role_store = MemoryStrategyAgentRoleStore()
skill_store = MemorySkillStore()
storage = _SkillStorage(
[
_skill("public-skill"),
_skill("published-custom", SkillCategory.CUSTOM),
_skill("private-custom", SkillCategory.CUSTOM),
]
)
await skill_store.create(
{
"name": "published-custom",
"owner_user_id": "publisher",
"published": True,
}
)
await skill_store.create(
{
"name": "private-custom",
"owner_user_id": "owner",
"published": False,
}
)
monkeypatch.setattr(agents_router, "get_or_new_skill_storage", lambda **_: storage)
monkeypatch.setattr(agents_router, "get_skill_usage_entries", lambda: {})
state = SimpleNamespace(
agent_store=agent_store,
strategy_agent_role_store=role_store,
skill_store=skill_store,
config=SimpleNamespace(models=[SimpleNamespace(name="model-a")]),
)
request = SimpleNamespace(
app=SimpleNamespace(state=state),
state=SimpleNamespace(
user=SimpleNamespace(id="actor", system_role=role),
),
)
return request, agent_store, role_store
@pytest.mark.asyncio
async def test_catalog_has_fixed_categories_and_redacts_public_config(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, role_store = await _request(tmp_path, monkeypatch)
public = await strategy_agents.list_strategy_agents(request)
assert len(public.agents) == 9
assert [(item.category, item.fixed) for item in public.agents if item.fixed] == [
("intent", True),
("judge", True),
("solution", True),
]
assert all(item.model is None and item.soul is None and item.skills is None for item in public.agents)
await role_store.update_role(seed.STRATEGY_SECURITY_AGENT_ID, enabled=False)
active = await strategy_agents.list_strategy_agents(request)
assert seed.STRATEGY_SECURITY_AGENT_ID not in {item.id for item in active.agents}
admin = await strategy_agents.list_strategy_agents(
request,
include_disabled=True,
include_config=True,
)
disabled = next(item for item in admin.agents if item.id == seed.STRATEGY_SECURITY_AGENT_ID)
assert disabled.enabled is False
assert disabled.soul
assert disabled.skills == ["policy-knowledge"]
@pytest.mark.asyncio
async def test_catalog_rejects_user_owned_reserved_default(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, agent_store, _ = await _request(tmp_path, monkeypatch)
await agent_store.delete_builtin(seed.STRATEGY_INTENT_AGENT_ID)
await agent_store.create(
{
"id": seed.STRATEGY_INTENT_AGENT_ID,
"name": "collision",
"user_id": "attacker",
}
)
with pytest.raises(HTTPException) as exc_info:
await strategy_agents.list_strategy_agents(request)
assert exc_info.value.status_code == 409
assert "owned by a user" in str(exc_info.value.detail)
@pytest.mark.asyncio
async def test_sensitive_catalog_and_writes_are_admin_only(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, _ = await _request(tmp_path, monkeypatch, role="user")
# The active catalog is intentionally readable by the workbench.
assert len((await strategy_agents.list_strategy_agents(request)).agents) == 9
with pytest.raises(HTTPException) as list_error:
await strategy_agents.list_strategy_agents(
request,
include_disabled=True,
include_config=True,
)
assert list_error.value.status_code == 403
with pytest.raises(HTTPException) as create_error:
await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(name="非法新增"),
request,
)
assert create_error.value.status_code == 403
@pytest.mark.asyncio
async def test_angle_crud_is_global_and_delete_is_soft(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, agent_store, role_store = await _request(tmp_path, monkeypatch)
created = await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(
name="科技产业视角",
description="评估关键技术和产业链韧性",
soul="你是科技产业策略分析智能体。",
model="model-a",
skills=["public-skill", "published-custom"],
),
request,
)
assert created.category == "angle"
assert created.fixed is False
assert created.enabled is True
assert created.id.startswith(seed.STRATEGY_ANGLE_AGENT_PREFIX)
assert created.model == "model-a"
assert created.skills == ["public-skill", "published-custom"]
updated = await strategy_agents.update_strategy_agent(
created.id,
strategy_agents.StrategyAgentUpdateRequest(
name="科技与产业链视角",
enabled=False,
sort_order=123,
skills=["public-skill"],
),
request,
)
assert updated.name == "科技与产业链视角"
assert updated.enabled is False
assert updated.sort_order == 123
assert updated.skills == ["public-skill"]
deleted = await strategy_agents.disable_strategy_angle(created.id, request)
assert deleted.enabled is False
# Soft delete protects saved projects and permits an administrator to
# restore the angle without recreating its ID or files.
assert await agent_store.get_any(created.id) is not None
assert await role_store.get_role(created.id) is not None
restored = await strategy_agents.update_strategy_agent(
created.id,
strategy_agents.StrategyAgentUpdateRequest(enabled=True),
request,
)
assert restored.enabled is True
assert (tmp_path / "agents" / created.id / ".strategy-role.yaml").is_file()
@pytest.mark.asyncio
async def test_memory_mode_recovers_dynamic_angle_from_folder_sidecar(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, _ = await _request(tmp_path, monkeypatch)
created = await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(
name="恢复测试角度",
sort_order=177,
),
request,
)
await strategy_agents.disable_strategy_angle(created.id, request)
request.app.state.agent_store = MemoryAgentStore()
request.app.state.strategy_agent_role_store = MemoryStrategyAgentRoleStore()
recovered = await strategy_agents.list_strategy_agents(
request,
include_disabled=True,
include_config=True,
)
item = next(entry for entry in recovered.agents if entry.id == created.id)
assert item.name == "恢复测试角度"
assert item.enabled is False
assert item.sort_order == 177
assert (await request.app.state.agent_store.get_any(created.id))["user_id"] is None
@pytest.mark.asyncio
async def test_memory_mode_sidecar_keeps_default_angle_disabled_after_restart(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, _ = await _request(tmp_path, monkeypatch)
await strategy_agents.update_strategy_agent(
seed.STRATEGY_SECURITY_AGENT_ID,
strategy_agents.StrategyAgentUpdateRequest(enabled=False, sort_order=188),
request,
)
request.app.state.agent_store = MemoryAgentStore()
request.app.state.strategy_agent_role_store = MemoryStrategyAgentRoleStore()
with pytest.raises(HTTPException) as run_error:
await parallel_agents.run_parallel_agents_stream(
request,
{
"task_name": "restart gate",
"brief": "disabled default must stay disabled",
"agent_ids": [seed.STRATEGY_SECURITY_AGENT_ID],
},
)
assert run_error.value.status_code == 409
recovered = await strategy_agents.list_strategy_agents(
request,
include_disabled=True,
include_config=True,
)
item = next(entry for entry in recovered.agents if entry.id == seed.STRATEGY_SECURITY_AGENT_ID)
assert item.enabled is False
assert item.sort_order == 188
@pytest.mark.asyncio
async def test_cancelled_create_compensates_every_persistence_layer(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, agent_store, role_store = await _request(tmp_path, monkeypatch)
generated_id = f"{seed.STRATEGY_ANGLE_AGENT_PREFIX}cancelled"
monkeypatch.setattr(
strategy_agents,
"uuid4",
lambda: SimpleNamespace(hex="cancelled"),
)
async def cancel_set_extras(*_args, **_kwargs):
raise asyncio.CancelledError
monkeypatch.setattr(agent_store, "set_extras", cancel_set_extras)
with pytest.raises(asyncio.CancelledError):
await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(name="cancelled create"),
request,
)
assert await agent_store.get_any(generated_id) is None
assert await role_store.get_role(generated_id) is None
assert not (tmp_path / "agents" / generated_id).exists()
@pytest.mark.asyncio
async def test_cross_process_file_lock_excludes_second_handle(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
paths = Paths(tmp_path)
monkeypatch.setattr(strategy_agents, "get_paths", lambda: paths)
first = strategy_agents._open_cross_process_lock(seed.STRATEGY_INTENT_AGENT_ID)
second = strategy_agents._open_cross_process_lock(seed.STRATEGY_INTENT_AGENT_ID)
try:
assert strategy_agents._try_cross_process_lock(first) is True
assert strategy_agents._try_cross_process_lock(second) is False
strategy_agents._release_cross_process_lock(first)
first = None
assert strategy_agents._try_cross_process_lock(second) is True
finally:
if first is not None:
strategy_agents._release_cross_process_lock(first)
if not second.closed:
strategy_agents._release_cross_process_lock(second)
@pytest.mark.asyncio
async def test_update_failure_restores_files_cache_and_role(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, agent_store, role_store = await _request(tmp_path, monkeypatch)
created = await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(
name="事务前名称",
model="model-a",
skills=["public-skill"],
sort_order=166,
),
request,
)
agent_dir = tmp_path / "agents" / created.id
before_files = {name: (agent_dir / name).read_text(encoding="utf-8") for name in ("config.yaml", "SOUL.md", ".strategy-role.yaml")}
real_update = role_store.update_role
failed = False
async def fail_once(agent_id: str, **kwargs):
nonlocal failed
if kwargs.get("sort_order") == 188 and not failed:
failed = True
raise RuntimeError("injected role-store failure")
return await real_update(agent_id, **kwargs)
monkeypatch.setattr(role_store, "update_role", fail_once)
with pytest.raises(RuntimeError, match="injected role-store failure"):
await strategy_agents.update_strategy_agent(
created.id,
strategy_agents.StrategyAgentUpdateRequest(
name="不应留下的名称",
soul="不应留下的 SOUL",
skills=[],
sort_order=188,
),
request,
)
record = await agent_store.get_any(created.id)
role = await role_store.get_role(created.id)
assert record is not None and record["name"] == "事务前名称"
assert role is not None and role["sort_order"] == 166
assert (await agent_store.get_extras_for([created.id]))[created.id]["skills"] == ["public-skill"]
assert {name: (agent_dir / name).read_text(encoding="utf-8") for name in ("config.yaml", "SOUL.md", ".strategy-role.yaml")} == before_files
@pytest.mark.asyncio
async def test_fixed_roles_cannot_be_disabled_or_deleted(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, _ = await _request(tmp_path, monkeypatch)
with pytest.raises(HTTPException) as update_error:
await strategy_agents.update_strategy_agent(
seed.STRATEGY_INTENT_AGENT_ID,
strategy_agents.StrategyAgentUpdateRequest(enabled=False),
request,
)
assert update_error.value.status_code == 409
with pytest.raises(HTTPException) as delete_error:
await strategy_agents.disable_strategy_angle(
seed.STRATEGY_JUDGE_AGENT_ID,
request,
)
assert delete_error.value.status_code == 409
@pytest.mark.asyncio
async def test_model_and_global_skill_visibility_are_validated(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, _ = await _request(tmp_path, monkeypatch)
with pytest.raises(HTTPException) as model_error:
await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(
name="未知模型",
model="not-configured",
),
request,
)
assert model_error.value.status_code == 422
with pytest.raises(HTTPException) as skill_error:
await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(
name="私有技能",
skills=["private-custom"],
),
request,
)
assert skill_error.value.status_code == 422
assert "globally available" in str(skill_error.value.detail)
defaulted = await strategy_agents.create_strategy_angle(
strategy_agents.StrategyAngleCreateRequest(
name="默认角色模板",
description="关注跨领域风险",
),
request,
)
assert defaulted.soul
assert "默认角色模板" in defaulted.soul
assert "不替代其他角度" in defaulted.soul
@pytest.mark.asyncio
async def test_disabled_strategy_agent_cannot_be_started_via_parallel_api(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, _, role_store = await _request(tmp_path, monkeypatch)
await role_store.ensure_defaults(seed.default_strategy_agent_roles())
await role_store.update_role(seed.STRATEGY_SECURITY_AGENT_ID, enabled=False)
with pytest.raises(HTTPException) as exc_info:
await parallel_agents.run_parallel_agents_stream(
request,
{
"task_name": "测试",
"brief": "验证停用闸门",
"agent_ids": [seed.STRATEGY_SECURITY_AGENT_ID],
},
)
assert exc_info.value.status_code == 409
@pytest.mark.asyncio
async def test_parallel_api_rejects_user_owned_strategy_backing_agent(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
request, agent_store, role_store = await _request(tmp_path, monkeypatch)
await role_store.ensure_defaults(seed.default_strategy_agent_roles())
await agent_store.delete_builtin(seed.STRATEGY_SECURITY_AGENT_ID)
await agent_store.create(
{
"id": seed.STRATEGY_SECURITY_AGENT_ID,
"name": "collision",
"user_id": "attacker",
}
)
with pytest.raises(HTTPException) as exc_info:
await parallel_agents.run_parallel_agents_stream(
request,
{
"task_name": "测试",
"brief": "验证归属闸门",
"agent_ids": [seed.STRATEGY_SECURITY_AGENT_ID],
},
)
assert exc_info.value.status_code == 409
assert "owned by a user" in str(exc_info.value.detail)
@pytest.mark.asyncio
async def test_memory_registry_repairs_corrupted_fixed_role_metadata() -> None:
store = MemoryStrategyAgentRoleStore()
await store.create_role(
{
"agent_id": seed.STRATEGY_INTENT_AGENT_ID,
"category": "angle",
"fixed": False,
"enabled": False,
"sort_order": 999,
}
)
await store.ensure_defaults(seed.default_strategy_agent_roles())
repaired = await store.get_role(seed.STRATEGY_INTENT_AGENT_ID)
assert repaired is not None
assert repaired["category"] == "intent"
assert repaired["fixed"] is True
assert repaired["enabled"] is True
assert repaired["sort_order"] == 0
@pytest.mark.asyncio
async def test_sql_registry_preserves_disabled_defaults_and_soft_records(
tmp_path: Path,
) -> None:
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'strategy-agent-roles.db'}")
async with engine.begin() as connection:
await connection.run_sync(StrategyAgentRoleRow.__table__.create)
store = StrategyAgentRoleRepository(async_sessionmaker(engine, expire_on_commit=False))
try:
defaults = seed.default_strategy_agent_roles()
assert await store.ensure_defaults(defaults) == 9
assert await store.ensure_defaults(defaults) == 0
await store.update_role(seed.STRATEGY_SECURITY_AGENT_ID, enabled=False)
# Re-seeding must never undo an administrator's soft delete.
assert await store.ensure_defaults(defaults) == 0
disabled = await store.get_role(seed.STRATEGY_SECURITY_AGENT_ID)
assert disabled is not None
assert disabled["enabled"] is False
# Canonical fixed-role invariants are repaired, unlike angle lifecycle
# state which intentionally remains administrator-controlled.
async with store._sf() as session:
row = await session.get(
StrategyAgentRoleRow,
seed.STRATEGY_INTENT_AGENT_ID,
)
row.category = "angle"
row.fixed = False
row.enabled = False
row.sort_order = 999
await session.commit()
assert await store.ensure_defaults(defaults) == 0
repaired = await store.get_role(seed.STRATEGY_INTENT_AGENT_ID)
assert repaired is not None
assert repaired["category"] == "intent"
assert repaired["fixed"] is True
assert repaired["enabled"] is True
assert repaired["sort_order"] == 0
dynamic = await store.create_role(
{
"agent_id": "strategy-selection-angle-sql-test",
"category": "angle",
"fixed": False,
"enabled": True,
"sort_order": 150,
}
)
assert dynamic["category"] == "angle"
assert await store.update_role(dynamic["agent_id"], enabled=False)
assert await store.delete_role(dynamic["agent_id"]) is True
assert await store.get_role(dynamic["agent_id"]) is None
finally:
await engine.dispose()
@@ -0,0 +1,81 @@
import asyncio
from datetime import UTC, datetime
from types import SimpleNamespace
import pytest
from fastapi import HTTPException
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.gateway.routers import parallel_agent_tasks
from deerflow.persistence.base import Base
from deerflow.persistence.parallel_agent_tasks import ParallelAgentTaskRepository
@pytest.mark.asyncio
async def test_parallel_task_repository_rejects_stale_revision(tmp_path) -> None:
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'strategy-project-cas.db'}")
async with engine.begin() as connection:
await connection.run_sync(Base.metadata.create_all)
store = ParallelAgentTaskRepository(async_sessionmaker(engine, expire_on_commit=False))
try:
original = await store.create_task(
"user-1",
{
"id": "strategy-selection-cas",
"title": "original",
"messages": [{"snapshot": "original"}],
},
)
await asyncio.sleep(0.002)
newer = await store.update_task(
"strategy-selection-cas",
"user-1",
title="newer",
messages=[{"snapshot": "newer"}],
expected_updated_at=original["updated_at"],
)
assert newer is not None
stale = await store.update_task(
"strategy-selection-cas",
"user-1",
title="stale",
messages=[{"snapshot": "stale"}],
expected_updated_at=original["updated_at"],
)
assert stale is None
final = await store.get_task("strategy-selection-cas", "user-1")
assert final is not None
assert final["title"] == "newer"
assert final["messages"] == [{"snapshot": "newer"}]
finally:
await engine.dispose()
@pytest.mark.asyncio
async def test_parallel_task_route_reports_revision_conflict() -> None:
class ConflictStore:
async def update_task(self, *_args, **_kwargs):
return None
async def get_task(self, *_args, **_kwargs):
return {"id": "strategy-selection-cas"}
request = SimpleNamespace(
state=SimpleNamespace(user=SimpleNamespace(id="user-1")),
app=SimpleNamespace(state=SimpleNamespace(parallel_agent_task_store=ConflictStore())),
)
body = parallel_agent_tasks.TaskUpdateRequest(
title="stale",
expected_updated_at=datetime.now(UTC),
)
with pytest.raises(HTTPException) as error:
await parallel_agent_tasks.update_task(
request,
"strategy-selection-cas",
body,
)
assert error.value.status_code == 409
assert "其他页面更新" in str(error.value.detail)
@@ -0,0 +1,72 @@
from types import SimpleNamespace
import pytest
from fastapi import HTTPException
from app.gateway.routers import system_settings as system_settings_router
from deerflow.config.system_settings import SystemSettings
def test_strategy_project_demo_is_hidden_by_default() -> None:
assert SystemSettings().strategy_projects.taiwan_strait_demo_enabled is False
@pytest.mark.asyncio
async def test_non_admin_cannot_update_strategy_project_settings(
monkeypatch: pytest.MonkeyPatch,
) -> None:
async def ordinary_user(_request):
return SimpleNamespace(system_role="user")
monkeypatch.setattr(
system_settings_router,
"get_optional_user_from_request",
ordinary_user,
)
with pytest.raises(HTTPException) as error:
await system_settings_router.update_strategy_project_settings(
system_settings_router.StrategyProjectSettingsUpdateRequest(
taiwan_strait_demo_enabled=True,
),
SimpleNamespace(),
)
assert error.value.status_code == 403
@pytest.mark.asyncio
async def test_admin_can_update_strategy_project_settings(
monkeypatch: pytest.MonkeyPatch,
) -> None:
current = SystemSettings()
saved: list[SystemSettings] = []
async def admin_user(_request):
return SimpleNamespace(system_role="admin")
monkeypatch.setattr(
system_settings_router,
"get_optional_user_from_request",
admin_user,
)
monkeypatch.setattr(
system_settings_router,
"load_system_settings",
lambda: current,
)
monkeypatch.setattr(
system_settings_router,
"save_system_settings",
lambda settings: saved.append(settings) or settings,
)
response = await system_settings_router.update_strategy_project_settings(
system_settings_router.StrategyProjectSettingsUpdateRequest(
taiwan_strait_demo_enabled=True,
),
SimpleNamespace(),
)
assert response.taiwan_strait_demo_enabled is True
assert saved[0].strategy_projects.taiwan_strait_demo_enabled is True
@@ -0,0 +1,276 @@
"""Regression tests for the strategy-selection built-in agent seed."""
from __future__ import annotations
from pathlib import Path
import pytest
import yaml
from fastapi import HTTPException
from app.gateway.routers import _strategy_selection_seed as seed
from app.gateway.routers import agents as agents_router
from deerflow.config.paths import Paths
from deerflow.persistence.skills.memory import MemorySkillStore
from deerflow.skills.types import Skill, SkillCategory
@pytest.fixture()
def tmp_paths(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Paths:
paths = Paths(tmp_path)
monkeypatch.setattr(seed, "get_paths", lambda: paths)
return paths
def _asset(agent_id: str, filename: str) -> Path:
return seed._ASSETS_DIR / agent_id / filename
def test_seed_assets_define_all_nine_functional_agents() -> None:
agent_ids = tuple(seed.required_agent_ids())
assert len(agent_ids) == 9
assert len(set(agent_ids)) == 9
assert agent_ids[0] == seed.STRATEGY_INTENT_AGENT_ID
assert agent_ids[-2:] == (
seed.STRATEGY_JUDGE_AGENT_ID,
seed.STRATEGY_SOLUTION_AGENT_ID,
)
assert len(seed.STRATEGY_ANGLE_AGENT_IDS) == 6
for agent_id in agent_ids:
config_path = _asset(agent_id, "config.yaml")
soul_path = _asset(agent_id, "SOUL.md")
assert config_path.is_file(), agent_id
assert soul_path.is_file(), agent_id
config = yaml.safe_load(config_path.read_text(encoding="utf-8"))
soul = soul_path.read_text(encoding="utf-8")
assert config["id"] == agent_id
assert config["name"].strip()
assert config["description"].strip()
assert "作战战术" in soul
assert "定向政治说服" in soul
assert "降级" in soul
def test_seed_assets_use_role_appropriate_existing_skills_and_tools() -> None:
intent = yaml.safe_load(
_asset(seed.STRATEGY_INTENT_AGENT_ID, "config.yaml").read_text(encoding="utf-8")
)
assert intent["skills"] == ["intent-clarification-policy"]
assert intent["tool_groups"] == []
for agent_id in seed.STRATEGY_ANGLE_AGENT_IDS:
config = yaml.safe_load(_asset(agent_id, "config.yaml").read_text(encoding="utf-8"))
assert config["skills"] == ["policy-knowledge"]
assert config["tool_groups"] == ["web", "file:read"]
judge = yaml.safe_load(
_asset(seed.STRATEGY_JUDGE_AGENT_ID, "config.yaml").read_text(encoding="utf-8")
)
solution = yaml.safe_load(
_asset(seed.STRATEGY_SOLUTION_AGENT_ID, "config.yaml").read_text(encoding="utf-8")
)
assert judge["skills"] == []
assert judge["tool_groups"] == []
assert solution["skills"] == []
assert solution["tool_groups"] == ["file:write"]
def test_default_strategy_skills_are_enabled_in_the_system_catalog() -> None:
extensions_path = Path(__file__).resolve().parents[1] / "extensions_config.json"
extensions = yaml.safe_load(extensions_path.read_text(encoding="utf-8")) or {}
configured = extensions.get("skills") or {}
# A missing entry means enabled by default. An explicit false would make
# the seeded allowlist impossible to save under the server-side validator
# and silently remove the skill from strategy runs.
for skill_name in ("intent-clarification-policy", "policy-knowledge"):
assert (configured.get(skill_name) or {}).get("enabled", True) is True
def test_seed_assets_define_machine_readable_handoff_contracts() -> None:
intent_soul = _asset(seed.STRATEGY_INTENT_AGENT_ID, "SOUL.md").read_text(encoding="utf-8")
judge_soul = _asset(seed.STRATEGY_JUDGE_AGENT_ID, "SOUL.md").read_text(encoding="utf-8")
solution_soul = _asset(seed.STRATEGY_SOLUTION_AGENT_ID, "SOUL.md").read_text(encoding="utf-8")
assert "[INTENT_READY]" in intent_soul
for field in (
"name",
"detail",
"objective",
"prerequisites",
"constraints",
"deliverables",
"acceptanceCriteria",
"horizon",
"selectedAngleIds",
):
assert f'"{field}"' in intent_soul
assert "[JUDGEMENT_READY]" in judge_soul
for field in ("agentId", "score", "conclusion", "comment", "strengths", "risks"):
assert f'"{field}"' in judge_soul
assert '"recommendedAgentIds"' in judge_soul
assert "/mnt/user-data/outputs/综合策略方案.md" in solution_soul
def test_seed_creates_once_and_never_overwrites_admin_changes(tmp_paths: Paths) -> None:
expected = set(seed.required_agent_ids())
assert set(seed.ensure_strategy_selection_agents()) == expected
for agent_id in expected:
agent_dir = tmp_paths.agent_dir(agent_id)
assert (agent_dir / "config.yaml").is_file()
assert (agent_dir / "SOUL.md").is_file()
customized_agent = seed.STRATEGY_SECURITY_AGENT_ID
customized_soul = tmp_paths.agent_dir(customized_agent) / "SOUL.md"
customized_soul.write_text("管理员现场修改", encoding="utf-8")
assert seed.ensure_strategy_selection_agents() == []
assert customized_soul.read_text(encoding="utf-8") == "管理员现场修改"
def test_seed_upgrades_only_an_exact_known_legacy_default(
tmp_paths: Paths,
monkeypatch: pytest.MonkeyPatch,
) -> None:
assert seed.ensure_strategy_selection_agents()
intent_soul = tmp_paths.agent_dir(seed.STRATEGY_INTENT_AGENT_ID) / "SOUL.md"
legacy_content = b"legacy bundled intent soul\r\n"
intent_soul.write_bytes(legacy_content)
monkeypatch.setitem(
seed._LEGACY_DEFAULT_SOUL_HASHES,
seed.STRATEGY_INTENT_AGENT_ID,
seed._normalized_text_hash(legacy_content),
)
assert seed.ensure_strategy_selection_agents() == []
assert (
intent_soul.read_bytes()
== _asset(
seed.STRATEGY_INTENT_AGENT_ID,
"SOUL.md",
).read_bytes()
)
intent_soul.write_text("管理员现场修改", encoding="utf-8")
assert seed.ensure_strategy_selection_agents() == []
assert intent_soul.read_text(encoding="utf-8") == "管理员现场修改"
class _SkillStorage:
def __init__(self, skills: list[Skill]) -> None:
self._skills = skills
def load_skills(self, *, enabled_only: bool) -> list[Skill]:
if enabled_only:
return [skill for skill in self._skills if skill.enabled]
return list(self._skills)
def _skill(name: str, *, category: SkillCategory, enabled: bool = True) -> Skill:
directory = Path("skills") / category.value / name
return Skill(
name=name,
description=name,
license=None,
skill_dir=directory,
skill_file=directory / "SKILL.md",
relative_path=Path(name),
category=category,
enabled=enabled,
)
@pytest.mark.asyncio
async def test_strategy_skill_validation_accepts_only_global_catalog_names(
monkeypatch: pytest.MonkeyPatch,
) -> None:
skill_store = MemorySkillStore()
await skill_store.create(
{
"name": "published-custom",
"owner_user_id": "publisher",
"published": True,
}
)
storage = _SkillStorage(
[
_skill("public-skill", category=SkillCategory.PUBLIC),
_skill("published-custom", category=SkillCategory.CUSTOM),
]
)
monkeypatch.setattr(agents_router, "get_or_new_skill_storage", lambda **_: storage)
monkeypatch.setattr(
agents_router,
"get_skill_usage_entries",
lambda: {},
)
result = await agents_router._validate_strategy_agent_skills(
[" public-skill ", "published-custom", "public-skill"],
app_config=object(), # type: ignore[arg-type]
skill_store=skill_store,
)
assert result == ["public-skill", "published-custom"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
("requested", "detail_fragment"),
[
("missing", "Unknown strategy agent skills"),
("disabled", "Disabled or archived strategy agent skills"),
("stale", "Disabled or archived strategy agent skills"),
("private-custom", "must be globally available"),
("orphan-custom", "must be globally available"),
],
)
async def test_strategy_skill_validation_rejects_unselectable_names(
monkeypatch: pytest.MonkeyPatch,
requested: str,
detail_fragment: str,
) -> None:
skill_store = MemorySkillStore()
await skill_store.create(
{
"name": "private-custom",
"owner_user_id": "owner",
"published": False,
}
)
storage = _SkillStorage(
[
_skill("disabled", category=SkillCategory.PUBLIC, enabled=False),
_skill("stale", category=SkillCategory.PUBLIC),
_skill("private-custom", category=SkillCategory.CUSTOM),
_skill("orphan-custom", category=SkillCategory.CUSTOM),
]
)
monkeypatch.setattr(agents_router, "get_or_new_skill_storage", lambda **_: storage)
monkeypatch.setattr(
agents_router,
"get_skill_usage_entries",
lambda: {"stale": {"state": "stale"}},
)
with pytest.raises(HTTPException) as exc_info:
await agents_router._validate_strategy_agent_skills(
[requested],
app_config=object(), # type: ignore[arg-type]
skill_store=skill_store,
)
assert exc_info.value.status_code == 422
assert detail_fragment in str(exc_info.value.detail)
def test_fixed_strategy_role_agents_cannot_be_deleted() -> None:
for agent_id in seed.required_agent_ids():
with pytest.raises(HTTPException) as exc_info:
agents_router._ensure_strategy_agent_deletable(agent_id)
assert exc_info.value.status_code == 409
agents_router._ensure_strategy_agent_deletable("ordinary-custom-agent")
@@ -1,8 +1,10 @@
"""Regression coverage for provider-safe transient context compression."""
from langchain.agents.middleware.types import ModelRequest, ModelResponse
from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage
from deerflow.agents.middlewares.summarization_middleware import _ensure_latest_human_message
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware, _ensure_latest_human_message
def test_compression_restores_latest_human_turn_when_only_tool_tail_remains():
@@ -28,3 +30,37 @@ def test_compression_does_not_duplicate_human_turn_already_in_tail():
result = _ensure_latest_human_message([user], compressed)
assert result == compressed
def test_compression_uses_single_model_request_system_slot_for_generic_adapters():
model = FakeListChatModel(responses=["condensed history"])
middleware = DeerFlowSummarizationMiddleware(
model,
trigger=("messages", 3),
keep=("messages", 1),
)
request = ModelRequest(
model=model,
system_message=SystemMessage(content="agent prompt"),
messages=[
HumanMessage(content="first question", id="human-1"),
AIMessage(content="first answer", id="ai-1"),
HumanMessage(content="continue", id="human-2"),
],
)
captured: list[ModelRequest] = []
def handler(model_request: ModelRequest) -> ModelResponse:
captured.append(model_request)
return ModelResponse(result=[AIMessage(content="ok")])
middleware.wrap_model_call(request, handler)
assert len(captured) == 1
normalized = captured[0]
assert normalized.system_message is not None
assert "agent prompt" in normalized.system_message.text
assert "condensed history" in normalized.system_message.text
assert all(not isinstance(message, SystemMessage) for message in normalized.messages)
assert [message.type for message in normalized.messages] == ["human"]
assert normalized.messages[0].content == "continue"
@@ -375,8 +375,10 @@ async def test_admin_can_upload_and_activate_offline_encoder(monkeypatch, tmp_pa
class Embedding:
def __init__(self):
self.model_path = ""
self.reset_calls = 0
async def reset_local_encoder(self, *, model_path=None):
self.reset_calls += 1
if model_path is not None:
self.model_path = model_path
@@ -395,6 +397,17 @@ async def test_admin_can_upload_and_activate_offline_encoder(monkeypatch, tmp_pa
monkeypatch.setattr(llmwiki_index, "load_system_settings", SystemSettings)
monkeypatch.setattr(llmwiki_index, "save_system_settings", lambda value: saved.append(value) or value)
embedding = Embedding()
async def rebuild(_app, *, force=False):
assert force is True
embedding.model_path = saved[-1].local_wiki_encoder.model_path
monkeypatch.setattr(llmwiki_index, "rebuild_local_wiki_index_runtime", rebuild)
monkeypatch.setattr(
llmwiki_index,
"local_encoder_runtime_status",
lambda _app: embedding.local_encoder_status(),
)
app = FastAPI()
app.state.config = SimpleNamespace(
llmwiki=SimpleNamespace(
@@ -414,11 +427,15 @@ async def test_admin_can_upload_and_activate_offline_encoder(monkeypatch, tmp_pa
paths=["onnx/model.onnx", "config.json"],
)
active = tmp_path / "models" / "bge-m3"
revision = saved[-1].local_wiki_encoder.model_revision
active = tmp_path / "models" / "bge-m3-revisions" / revision
assert (active / "onnx" / "model.onnx").read_bytes() == b"onnx"
assert result["uploaded_files"] == 2
assert saved[-1].local_wiki_encoder.model_path == str(active.resolve())
assert saved[-1].local_wiki_encoder.mode == "local"
assert revision
assert embedding.model_path == str(active.resolve())
assert embedding.reset_calls == 0
def test_encoder_upload_rejects_parent_traversal():
+27
View File
@@ -116,6 +116,33 @@ vision:
# max_tokens: 2048
knowledge_ingest:
url: "https://ch1.b.uat.4.cn/knowes/api/knowledge-outer/add"
# WeKnora connection can be supplied from the administrator settings page.
# Keep release credentials out of this portable offline template; a non-empty
# runtime URL activates WeKnora and hot-rebuilds the Wiki sync service.
llmwiki:
weknora:
api_base_url: ""
web_base_url: ""
admin_email: ""
admin_password: ""
local_wiki_index:
enabled: true
frontend_base_url: "http://127.0.0.1:5174"
# WeKnora Wiki generation is asynchronous, so the immediate scan alone can
# miss newly-created pages. Keep the incremental rescan enabled.
auto_sync: true
sync_interval_seconds: 30
database_batch_size: 50
embedding:
provider: local
model: bge-embedding-m3
dimensions: 1024
batch_size: 2
local_threads: 2
local_process_isolation: true
local_model_path: ""
base_url: ""
api_key: ""
# 上传文件识别配置。
# - auto_convert_documents: true 时,上传的 PDF / Word / Excel / PPT 会在服务端自动转换为
# Markdown(.md),agent 通过 read_file / grep 读取其内容 —— 即“识别”文档内容。