sync update from LAN repo (source master @ c21a2a4a)

This commit is contained in:
gozeros committed 2026-11-10 14:03:50 +08:00
1 parent ea9dcff9c0
commit 66a443e915
140 files changed
+13777 -1040

No files matched your search

+27 -2
View File
@@ -1,5 +1,27 @@
# CLAUDE.md
## Model-independent skill invocation attribution (2026-11-07)
Never derive skill-use UI or counters from model narration or the optional tool
`description`. The canonical signals are a successful `skill_view`, a successful
`read_file` under `/mnt/skills/{public,custom}/<skill>/...`, and an executed
`bash` command containing such a skill path. `sandbox/tools.py` resolves names
through the active skill view so a directory/frontmatter-name mismatch remains
correct; both public and custom skills count.
`skills/usage.py::bump_use(..., event_key=run_id)` de-duplicates all file/script
signals to one use per skill per user run. It updates `.usage.json` and schedules
an absolute, monotonic update of the `skills` MySQL catalog projection on the
engine home loop. Keep this bridge inside `deerflow.*`; the harness must not
import `app.*`. The skill-card counter is intentionally independent of the
heavier full-tool audit switch. `skill_metrics_enabled` defaults on and mounts
`ToolMetricsMiddleware(skill_only=True)` while `tool_metrics_enabled` is off;
non-skill calls pass through without DB work, but attributed skill calls still
feed audit pages and leaderboards. Frontend labels are recovered by
`core/tools/skill-invocation.ts` from explicit skill-tool arguments or skill
paths. Regression coverage lives in `test_skill_usage_signals.py`,
`test_skill_view_fast_path.py`, and `skill-invocation.test.ts`.
## Cold agent construction (2026-10-30)
`runtime/runs/worker.py` dispatches opted-in synchronous factories through
@@ -549,7 +571,10 @@ handoff replaces the foreground lease with the durable job occupancy. There is
no per-user pool. Admission is serialized by a database gate row, and the cap
is enforced at acquire, init, every model-producing run, and background-job
start. Full pools return HTTP 409; an enabled gate with missing storage or a
missing session id fails closed. The switch applies on the next admission
missing session id fails closed. Foreground leases last ``lease_minutes``
(default 5, range 1–60) and are renewed by the browser heartbeat; changing the
setting applies on the next admission or heartbeat and never extends
``max_hold_minutes``. The switch applies on the next admission
without a restart. Position and eight-department roundtables are not counted.
Roundtable backend jobs use a database unique active-dedupe key plus a leased
@@ -837,7 +862,7 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
| **Task Deep-link Threads** (`/api/task-deeplink-threads`) | Dedicated shared history index for the `3qfx` / `rwfx` / `xdfx` single-agent task workspaces. `GET /?task_id=&go_path=&limit=` performs an indexed `(task_id, go_path)` lookup in `task_deeplink_threads` and joins `threads_meta` by `thread_id` for the current title/status/timestamps; LangGraph checkpoints remain the source of truth for messages. Thread metadata carrying `taskRecordId + taskGoPath` is dual-written atomically by both SQL and memory thread stores and deleted with the thread. `3qfx/rwfx` use the task id as `taskRecordId`; `xdfx` uses the original action id while retaining the resolved real task id in `metadata.taskId`. `thread_type=task_deeplink` keeps these rows out of general recent chats; explicit `taskId` searches still opt out of that exclusion for compatibility. Alembic `20261019_01` is schema-only and intentionally does **not** backfill legacy task threads. Frontend hook: `core/threads/use-task-deeplink-threads.ts`. Tests: `packages/harness/tests/test_task_deeplink_threads.py`, `packages/harness/tests/test_thread_meta_task_lookup.py`. |
| **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
| **Threads search** (`POST /threads/search`, LangGraph-compat) | Conversation listing. **System threads are excluded by default**: rows whose `metadata.system` is truthy (scheduler runs, roundtable workers) or whose `thread_type` is a known system type (`task-analysis` seat runs included; older seat rows are also matched by `task_analysis_seat`) never crowd the user's list; a caller whose `metadata` filter explicitly references `thread_type`/`system` (e.g. studio notebooks) opts out of the exclusion. New `query` field = case-insensitive title (`display_name`) substring search, used by the chats page's server-side pagination. `POST /threads/count` takes the same request body (limit/offset ignored) and returns `{total}` under identical filtering/exclusion semantics ? powers the chats page's ??/??? display without fetching rows (`ThreadMetaStore.count`). `ThreadMetaStore.search` gained `exclude_system`/`query` params; JSON-blob filters batch-scan until the requested `limit`/`offset` window is full (so pagination over a heavily-filtered set never under-fills), with a `thread_id` tie-breaker on the `updated_at DESC` ordering for stable OFFSET paging. Tests: `tests/test_thread_meta_search.py` |
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`, `/api/strategy-project-artifacts/{id}/download`) | `GET /{path}` streams thread artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `GET /api/threads/{id}/artifact-preview?path=` returns bounded text, using streaming statistics plus deterministic head/middle/tail samples above the database threshold. `GET /api/artifact-library` combines every current-user normal thread's authoritative `outputs/` files with completed strategy-selection syntheses projected from the owner-scoped `parallel_agent_tasks.messages` snapshot. Strategy items use `source_type=strategy`, carry `strategy_project_id` / `strategy_name` / `strategy_stale`, and explicitly distinguish `strategy_source_kind=strategy_selection` from an `analysis_project_mode=strategy` thread (`analysis_project`); virtual items also expose `download_url`, so clients never infer routing from sentinel thread/path values. Search/source filters cover both. `workflowStale=true` keeps the last completed historical synthesis visible and downloadable. The dedicated strategy download route revalidates current-user ownership and snapshot completion and serves Markdown without pretending the virtual item is a thread artifact. Uploads and image/audio/video media remain excluded. Library metadata is persisted in `artifact_library_items`; list/search/filter/count/pagination read only this user-scoped SQL index. A generation-fenced `artifact_library_sync_states` lease schedules low-concurrency filesystem/task reconciliation in the background on first/stale reads or `refresh=true`, keeps the last successful snapshot visible during refresh/failure, and reports `sync_status` / `syncing` / `indexed_at` / `sync_error` to the client. Files themselves remain authoritative on disk and are never copied into the database. |
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`, `/api/strategy-project-artifacts/{id}/download`) | `GET /{path}` streams thread artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. HTML/XHTML additionally accepts `offline_bundle=true`: it returns `{stem}-离线包.zip` with `index.html`, the referenced trusted Gateway runtime/image dependencies rewritten under `assets/`, a manifest/README and relevant third-party licenses; public view-share files support the same query. The builder never fetches arbitrary remote URLs, and records remaining Leaflet tile/user-supplied external URLs in the manifest. `GET /api/threads/{id}/artifact-preview?path=` returns bounded text, using streaming statistics plus deterministic head/middle/tail samples above the database threshold. `GET /api/artifact-library` combines every current-user normal thread's authoritative `outputs/` files with completed strategy-selection syntheses projected from the owner-scoped `parallel_agent_tasks.messages` snapshot. Strategy items use `source_type=strategy`, carry `strategy_project_id` / `strategy_name` / `strategy_stale`, and explicitly distinguish `strategy_source_kind=strategy_selection` from an `analysis_project_mode=strategy` thread (`analysis_project`); virtual items also expose `download_url`, so clients never infer routing from sentinel thread/path values. Search/source filters cover both. `workflowStale=true` keeps the last completed historical synthesis visible and downloadable. The dedicated strategy download route revalidates current-user ownership and snapshot completion and serves Markdown without pretending the virtual item is a thread artifact. Uploads and image/audio/video media remain excluded. Library metadata is persisted in `artifact_library_items`; list/search/filter/count/pagination read only this user-scoped SQL index. A generation-fenced `artifact_library_sync_states` lease schedules low-concurrency filesystem/task reconciliation in the background on first/stale reads or `refresh=true`, keeps the last successful snapshot visible during refresh/failure, and reports `sync_status` / `syncing` / `indexed_at` / `sync_error` to the client. Files themselves remain authoritative on disk and are never copied into the database. |
| **Word export** (`POST /api/writing/export/docx`) | Shared Markdown → DOCX generator used by conversation, artifact, scheduled-task, Deep Research, and AI-writing download surfaces. `app/gateway/word_export.py` builds OOXML with no Python conversion dependency: A4 portrait, 3.7/3.5/2.8/2.6 cm margins, real multilevel heading numbering (`一、` / `(一)` / `1.`), two-character first-line indent on heading levels 1–3, formal body/table styles, page numbers beginning at 1 with odd pages left-aligned and even pages right-aligned, and embedded `app/gateway/assets/fonts/方正小标宋简体.ttf`. Before numbering it removes complete manual heading prefixes (including `2.1` / `2.1.1`) and drops Markdown horizontal rules from Word output. The ordinary-Q&A Lead Agent can emit the matching Markdown heading forms and avoid unordered bullet lists when the admin enables `prompt_prefix.ordinary_qa_markdown_format_enabled` in `system_settings.json`; the switch defaults off, while writing setup, writing mode, notebook, scheduled runs, and SOUL-driven custom agents keep their independent prompts. The font loader validates its internal family, SHA-256, and OpenType embedding permission (`OS/2.fsType`) before export. Tests: `tests/test_word_export.py`, `tests/test_es_query_routing_prompt.py`. |
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. The compact writer validates generated Markdown against that frozen selection: `fixed` requires every non-placeholder heading with the same level and order, while `adaptive` retries only when no core template heading survives. That correction shares the single existing retry budget (never more than two writer calls), and its deterministic fallback is outline-shaped as well. When `custom_outline` is empty (the workbench default — the user did not pick a report structure), chat collection first calls `POST /sessions/{id}/intent-outline` after the collector turn ends: the chosen model plans a Markdown skeleton from the topic plus collected materials, the outline is frozen into the session snapshot and shown in the「开始撰写」form for editing. If that step is skipped or fails, `basic`/`quick` still call `generate_intent_outline` at write time, emit `plan_created` (`source: intent`), then write through the same shared writer; generation failure is a recoverable `intent_outline_fallback` warning and does not abort the report. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
| **Gateway auth** | Owner-scoped Deep Research routes await the platform's asynchronous `get_current_user` resolver before they read or write persistence, ensuring each session and job operation receives a concrete user ID. Deep Research artifacts are always resolved from a whitelisted filename via `/mnt/user-data/outputs/<name>` before mapping to a host path, preserving the sandbox guard on native Windows and Linux hosts. |
+27 -7
View File
@@ -1,5 +1,18 @@
# DeerFlow Backend
## 技能调用识别与统计(2026-11-07)
技能调用的展示和“调用次数”不再依赖模型主动说明。后端以真实工具信号为准:
成功的 `skill_view`、对 `/mnt/skills/{public,custom}/<name>/...` 的文件读取,
以及 `bash` 直接执行该目录下的技能脚本,都会识别为对应技能调用。一次问答中同一技能即使先读
`SKILL.md`、再读参考文件并执行多个脚本,也只累计一次;不同问答分别累计。
计数会同时更新 `.usage.json` 和技能列表使用的数据库投影,因此内置技能与自定义技能都能在
页面刷新后看到最新调用次数。对话步骤也直接从 `skill_view` 参数或真实技能路径显示
“调用技能「名称」”;弱模型未填写工具描述、未在正文里自述技能名时仍可识别。
完整工具审计仍由 `runtime_performance.tool_metrics_enabled` 独立控制;默认开启的
`skill_metrics_enabled` 只把可归因到具体技能的低频调用写入统计表,供技能排行和调用统计使用。
## 首次问答初始化响应性(2026-10-30)
普通问答、开放问答接口和工作流智能体节点的模型 SDK 加载、客户端初始化及
@@ -1715,7 +1728,9 @@ Multi-agent roundtable step 2 has a separate room cap. Administrators change
it under Settings → 会商并发 (`roundtable_room_limit`, default on, 3 rooms)
without restarting. The count is the number of live roundtable executions
across every account (foreground leases/live LangGraph runs plus executing
background jobs), not the current login. Merely opening a tab or viewing chat
background jobs), not the current login. A foreground lease stays occupied for
``lease_minutes`` without a renewal (default 5 minutes, range 1–60) and is
refreshed by the page heartbeat. Merely opening a tab or viewing chat
history does not count, but two tabs that both run the same saved draft count
as two rooms. All seats and Step 3 work in one execution share one room;
foreground-to-background handoff keeps one room. A full pool keeps the user on
@@ -1815,11 +1830,12 @@ Beta(8,2) 先验抑制小样本极值;零样本返回 `score: null` 和
实际加载的 `config.yaml`(`DEER_FLOW_CONFIG_PATH` 指定文件优先)支持以下独立开关。
`skill_view_reuse_enabled` 和 `skill_view_cache_enabled` 在配置里不写时默认开启;
要恢复每次工具调用都全量刷新技能视图,把对应项写成 `false` 后重启。
模型计时与线程池开关默认关闭,工具统计写入也默认暂停:
模型计时与线程池开关默认关闭,普通工具审计默认暂停,低频技能统计默认保留:
```yaml
runtime_performance:
tool_metrics_enabled: false
skill_metrics_enabled: true
skill_view_reuse_enabled: true
skill_view_cache_enabled: true
model_timing_nonblocking_enabled: true
@@ -1827,13 +1843,17 @@ runtime_performance:
asyncio_thread_pool_size: 100
```
- `tool_metrics_enabled`:工具/技能调用统计写入开关,**缺省为 false(暂停)**,旧部署配置
没有此项时也暂停。主智能体和子智能体不再装载工具统计中间件,不创建统计写库任务、
不执行技能统计合并查询,从源头减少与登录共用业务数据库连接池的竞争。
- `tool_metrics_enabled`:普通工具审计写入开关,**缺省为 false(暂停)**,旧部署配置
没有此项时也暂停。主智能体和子智能体不会为普通工具创建统计写库任务,
从源头减少与登录共用业务数据库连接池的竞争。
正常工具调用、错误处理、问答消息/checkpoint 和模型调用统计继续运行;已有
`tool_call_metrics` 记录与查询/导出接口保留。暂停期间工具/技能审计明细及排行榜的
工具/技能调用计数不再新增,且不缓存或事后补录。恢复时设为 `true` 并重启后台。
`tool_call_metrics` 记录与查询/导出接口保留。暂停期间普通工具调用计数不再新增,
且不缓存或事后补录。恢复时设为 `true` 并重启后台。
部署新代码并重启后才对所有运行实例生效;这不消除历史统计查询或其他数据库操作的开销。
- `skill_metrics_enabled`:只记录从 `skill_view` 参数或
`/mnt/skills/{public,custom}/<name>/...` 工具路径确定出的技能调用,缺省为 `true`。
非技能工具直接透传,不执行技能合并查询或写库;因此普通工具审计暂停时,技能排行与
技能调用统计仍会更新。若连这部分低频写入也要暂停,显式设为 `false` 并重启。
- `skill_view_reuse_enabled`:一次沙箱工具调用共用一份已验证技能视图,路径检查、
本地/AIO 沙箱初始化、命令转换和输出脱敏不再反复刷新。下一次工具调用重新验证,
不跨问答永久沿用启停状态。
@@ -186,6 +186,15 @@ def current_user_id(request: Request) -> str:
return get_effective_user_id()
def _lease_seconds(settings: Any) -> int:
minutes = getattr(settings, "lease_minutes", 5)
try:
minutes = int(minutes)
except (TypeError, ValueError):
minutes = 5
return max(1, min(60, minutes)) * 60
def _store(request: Request):
return getattr(request.app.state, "roundtable_room_store", None)
@@ -232,6 +241,7 @@ async def admit_roundtable_room(
enabled=settings.enabled,
max_rooms=settings.max_rooms,
max_hold_minutes=getattr(settings, "max_hold_minutes", 30),
lease_seconds=_lease_seconds(settings),
extra_keys=extra_keys,
)
@@ -265,6 +275,7 @@ async def renew_roundtable_room(
enabled=settings.enabled,
max_rooms=settings.max_rooms,
max_hold_minutes=getattr(settings, "max_hold_minutes", 30),
lease_seconds=_lease_seconds(settings),
extra_keys=extra_keys,
)
@@ -425,6 +436,7 @@ async def enforce_roundtable_run(
enabled=settings.enabled,
max_rooms=settings.max_rooms,
max_hold_minutes=getattr(settings, "max_hold_minutes", 30),
lease_seconds=_lease_seconds(settings),
extra_keys=await backend_occupancy_keys(request),
)
if not decision.get("ok", False):
@@ -30,7 +30,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request
from fastapi.responses import StreamingResponse
from langchain_core.messages import HumanMessage, SystemMessage
from pydantic import BaseModel, Field
from sqlalchemy import and_, case, desc, func, not_, or_, select, text
from sqlalchemy import and_, case, desc, func, literal, not_, or_, select, text
from app.gateway.admin_query_window import (
ADMIN_QUERY_WINDOW_DAYS,
@@ -279,6 +279,41 @@ class AdminLeaderboardDimension(BaseModel):
last_seen_at: str = ""
class AdminLeaderboardDetailItem(BaseModel):
"""One row in the expanded skill, tool, or agent analytics drawer."""
name: str
count: int = 0
user_count: int = 0
total_tokens: int = 0
avg_duration_ms: int = 0
class AdminLeaderboardDetailSummary(BaseModel):
count: int = 0
item_count: int = 0
user_count: int = 0
total_tokens: int = 0
avg_duration_ms: int = 0
class AdminLeaderboardDetailResponse(BaseModel):
"""Fresh, drill-down analytics for a leaderboard dimension.
Unlike the dashboard's compact daily snapshots, this response is queried
directly. It lets an administrator inspect the entire matching dimension
(up to the explicit limit) and does not lose long-tail rows because they
were outside a daily top-N snapshot.
"""
kind: str
range_days: int
since: str
until: str
summary: AdminLeaderboardDetailSummary
items: list[AdminLeaderboardDetailItem] = Field(default_factory=list)
class AdminLeaderboardTrendPoint(BaseModel):
date: str
runs: int = 0
@@ -961,6 +996,125 @@ async def _query_run_dimensions(
]
async def _query_leaderboard_dimension_details(
session,
since: datetime,
until: datetime,
*,
kind: str,
limit: int,
exclude_user_ids: set[str],
include_scheduled: bool,
include_admins: bool,
include_failed: bool,
) -> tuple[
AdminLeaderboardDetailSummary,
list[AdminLeaderboardDetailItem],
]:
"""Load complete, fresh detail statistics for one leaderboard dimension.
The main leaderboard deliberately reads compact daily snapshots so it can
remain quick under load. The drawer is an intentional admin drill-down,
therefore it runs a bounded aggregate against the source metric tables and
returns exact unique-user counts for the selected interval.
"""
if kind in {"skill", "tool"}:
name_col = ToolCallMetricRow.skill_name if kind == "skill" else ToolCallMetricRow.tool_name
base_filters = [
_tool_time_filter(since, until),
name_col.is_not(None),
name_col != "",
]
def apply_scope(stmt):
return _apply_tool_scope(
stmt,
exclude_user_ids=exclude_user_ids,
include_admins=include_admins,
include_scheduled=include_scheduled,
)
count_col = func.count(ToolCallMetricRow.id)
user_count_col = func.count(func.distinct(ToolCallMetricRow.user_id))
total_tokens_col = literal(0)
avg_duration_col = func.coalesce(func.avg(ToolCallMetricRow.duration_ms), 0)
elif kind == "agent":
name_col = RunRow.assistant_id
base_filters = [
_run_time_filter(since, until),
name_col.is_not(None),
name_col != "",
]
def apply_scope(stmt):
return _apply_run_scope(
stmt,
user_column=RunRow.user_id,
run_id_column=RunRow.run_id,
exclude_user_ids=exclude_user_ids,
include_admins=include_admins,
include_failed=include_failed,
include_scheduled=include_scheduled,
)
count_col = func.count(RunRow.run_id)
user_count_col = func.count(func.distinct(RunRow.user_id))
total_tokens_col = func.coalesce(func.sum(RunRow.total_tokens), 0)
# Run rows have no reliable wall-clock duration. Keep this explicit at
# zero instead of inferring elapsed time from an asynchronously updated
# journal timestamp.
avg_duration_col = literal(0)
else:
raise ValueError(f"Unknown leaderboard detail kind: {kind}")
item_stmt = (
select(
name_col.label("name"),
count_col.label("count"),
user_count_col.label("user_count"),
total_tokens_col.label("total_tokens"),
avg_duration_col.label("avg_duration_ms"),
)
.where(*base_filters)
.group_by(name_col)
.order_by(count_col.desc(), name_col.asc())
.limit(limit)
)
item_stmt = apply_scope(item_stmt)
summary_stmt = select(
count_col.label("count"),
func.count(func.distinct(name_col)).label("item_count"),
user_count_col.label("user_count"),
total_tokens_col.label("total_tokens"),
avg_duration_col.label("avg_duration_ms"),
).where(*base_filters)
summary_stmt = apply_scope(summary_stmt)
item_rows = (await _execute_leaderboard_query(session, f"{kind}_detail.items", item_stmt)).all()
summary_row = (await _execute_leaderboard_query(session, f"{kind}_detail.summary", summary_stmt)).one()
summary = AdminLeaderboardDetailSummary(
count=int(summary_row[0] or 0),
item_count=int(summary_row[1] or 0),
user_count=int(summary_row[2] or 0),
total_tokens=int(summary_row[3] or 0),
avg_duration_ms=int(summary_row[4] or 0),
)
items = [
AdminLeaderboardDetailItem(
name=row[0] or "",
count=int(row[1] or 0),
user_count=int(row[2] or 0),
total_tokens=int(row[3] or 0),
avg_duration_ms=int(row[4] or 0),
)
for row in item_rows
]
return summary, items
async def _query_model_rankings(
session,
since: datetime,
@@ -1876,6 +2030,59 @@ async def get_user_leaderboard(
)
@router.get("/leaderboard/details", response_model=AdminLeaderboardDetailResponse)
async def get_user_leaderboard_details(
request: Request,
kind: str = Query(description="One of: skill, tool, agent", pattern="^(skill|tool|agent)$"),
days: int = Query(default=30, ge=1, le=ADMIN_QUERY_WINDOW_DAYS),
since: str | None = Query(default=None, description="Inclusive range start, ISO datetime"),
until: str | None = Query(default=None, description="Inclusive range end, ISO datetime"),
limit: int = Query(default=500, ge=1, le=1000),
) -> AdminLeaderboardDetailResponse:
"""Return fresh drill-down statistics for one leaderboard card.
This endpoint intentionally reads the source metric tables instead of the
asynchronous daily leaderboard snapshots. A drawer represents an explicit
admin investigation, where exact unique-user counts, all matching entries,
and complete detail rows are more useful than a compact cache.
"""
await _require_admin(request)
session_factory = get_session_factory()
if session_factory is None:
raise HTTPException(status_code=503, detail="Analytics requires database persistence")
settings = load_system_settings().leaderboard
since_dt, until_dt, range_days = resolve_admin_query_window(
days=days,
since=since,
until=until,
)
excluded, include_scheduled, include_admins, include_failed, _settings_hash = _resolve_leaderboard_snapshot_scope(settings)
async with session_factory() as session:
await _set_analytics_read_timeout(session)
summary, items = await _query_leaderboard_dimension_details(
session,
since_dt,
until_dt,
kind=kind,
limit=limit,
exclude_user_ids=excluded,
include_scheduled=include_scheduled,
include_admins=include_admins,
include_failed=include_failed,
)
return AdminLeaderboardDetailResponse(
kind=kind,
range_days=range_days,
since=_iso(since_dt),
until=_iso(until_dt),
summary=summary,
items=items,
)
@router.get("/leaderboard/snapshots", response_model=AdminLeaderboardSnapshotStatusResponse)
async def get_leaderboard_snapshot_status(
request: Request,
@@ -1,14 +1,16 @@
"""Application chrome name and logo.
"""Application chrome name, logo, and optional red-theme artwork.
Stored separately from ``generated_page_branding`` and the
``generated-page-branding/`` directory. An empty name or a missing logo leaves
the built-in title and static theme logos in place.
``generated-page-branding/`` directory. Missing custom images leave the
built-in theme artwork in place.
"""
from __future__ import annotations
import asyncio
from typing import Annotated
from uuid import uuid4
from xml.etree import ElementTree
from fastapi import APIRouter, File, Form, HTTPException, Request, UploadFile
from fastapi.responses import FileResponse
@@ -18,6 +20,8 @@ from app.gateway.routers.system_settings import _require_admin
from deerflow.config.runtime_paths import runtime_home
from deerflow.config.system_settings import (
AppIdentitySettings,
app_identity_light_red_header_background_path,
app_identity_light_red_nav_active_path,
app_identity_logo_path,
load_system_settings,
save_system_settings,
@@ -26,25 +30,48 @@ from deerflow.config.system_settings import (
router = APIRouter(prefix="/api/system-settings", tags=["system-settings"])
public_router = APIRouter(prefix="/api/public/app-identity", tags=["app-identity"])
_MAX_LOGO_BYTES = 10 * 1024 * 1024
_MAX_IMAGE_BYTES = 10 * 1024 * 1024
_PUBLIC_LOGO_PATH = "/api/public/app-identity/logo"
_PUBLIC_LIGHT_RED_HEADER_PATH = "/api/public/app-identity/light-red-header-background"
_PUBLIC_LIGHT_RED_ACTIVE_PATH = "/api/public/app-identity/light-red-nav-active"
class AppIdentityResponse(BaseModel):
system_name: str = ""
header_title_font_mode: str = "adaptive"
header_title_font_size: int = 30
logo_url: str | None = None
logo_version: str | None = None
has_logo: bool = False
light_red_header_background_url: str | None = None
light_red_header_background_version: str | None = None
has_light_red_header_background: bool = False
light_red_nav_active_url: str | None = None
light_red_nav_active_version: str | None = None
has_light_red_nav_active: bool = False
def _detect_logo(data: bytes) -> tuple[str, str]:
def _detect_image(
data: bytes,
label: str,
*,
allow_svg: bool = False,
) -> tuple[str, str]:
if data.startswith(b"\x89PNG\r\n\x1a\n"):
return ".png", "image/png"
if data.startswith(b"\xff\xd8\xff"):
return ".jpg", "image/jpeg"
if len(data) >= 12 and data[:4] == b"RIFF" and data[8:12] == b"WEBP":
return ".webp", "image/webp"
raise HTTPException(status_code=422, detail="Logo 仅支持 PNG、JPG 或 WebP 图片")
if allow_svg:
try:
root = ElementTree.fromstring(data)
except ElementTree.ParseError:
root = None
if root is not None and root.tag.rsplit("}", 1)[-1].lower() == "svg":
return ".svg", "image/svg+xml"
supported = "PNG、JPG、WebP 或 SVG" if allow_svg else "PNG、JPG 或 WebP"
raise HTTPException(status_code=422, detail=f"{label}仅支持 {supported} 图片")
def _normalize_system_name(value: str | None) -> str:
@@ -54,21 +81,61 @@ def _normalize_system_name(value: str | None) -> str:
return normalized
def _normalize_header_title_font_mode(value: str) -> str:
normalized = str(value).strip().lower()
if normalized not in {"adaptive", "fixed"}:
raise HTTPException(status_code=422, detail="系统名称字号模式仅支持自适应或固定大小")
return normalized
def _normalize_header_title_font_size(value: int) -> int:
normalized = int(value)
if normalized < 16 or normalized > 40:
raise HTTPException(status_code=422, detail="系统名称固定字号必须在 16px 到 40px 之间")
return normalized
def _response(settings: AppIdentitySettings) -> AppIdentityResponse:
path = app_identity_logo_path(settings)
version = settings.logo_version if path is not None else None
header_background_path = app_identity_light_red_header_background_path(settings)
header_background_version = (
settings.light_red_header_background_version
if header_background_path is not None
else None
)
nav_active_path = app_identity_light_red_nav_active_path(settings)
nav_active_version = (
settings.light_red_nav_active_version if nav_active_path is not None else None
)
return AppIdentityResponse(
system_name=settings.system_name,
header_title_font_mode=settings.header_title_font_mode,
header_title_font_size=settings.header_title_font_size,
logo_url=f"{_PUBLIC_LOGO_PATH}?v={version}" if version else None,
logo_version=version,
has_logo=path is not None,
light_red_header_background_url=(
f"{_PUBLIC_LIGHT_RED_HEADER_PATH}?v={header_background_version}"
if header_background_version
else None
),
light_red_header_background_version=header_background_version,
has_light_red_header_background=header_background_path is not None,
light_red_nav_active_url=(
f"{_PUBLIC_LIGHT_RED_ACTIVE_PATH}?v={nav_active_version}"
if nav_active_version
else None
),
light_red_nav_active_version=nav_active_version,
has_light_red_nav_active=nav_active_path is not None,
)
def _write_logo(data: bytes, suffix: str) -> str:
def _write_asset(data: bytes, suffix: str, prefix: str) -> str:
directory = runtime_home() / "app-identity"
directory.mkdir(parents=True, exist_ok=True)
filename = f"logo-{uuid4().hex}{suffix}"
filename = f"{prefix}-{uuid4().hex}{suffix}"
destination = directory / filename
temporary = directory / f".{filename}.tmp"
try:
@@ -79,9 +146,40 @@ def _write_logo(data: bytes, suffix: str) -> str:
return filename
async def _read_uploaded_image(
upload: UploadFile,
label: str,
*,
allow_svg: bool = False,
max_bytes: int | None = _MAX_IMAGE_BYTES,
) -> tuple[bytes, str, str]:
data = await upload.read(max_bytes + 1) if max_bytes is not None else await upload.read()
await upload.close()
if not data:
raise HTTPException(status_code=422, detail=f"{label}文件不能为空")
if max_bytes is not None and len(data) > max_bytes:
raise HTTPException(status_code=413, detail=f"{label}不能超过 10MB")
suffix, media_type = _detect_image(data, label, allow_svg=allow_svg)
return data, suffix, media_type
def _public_asset_headers(media_type: str | None) -> dict[str, str]:
headers = {"Cache-Control": "public, max-age=31536000, immutable"}
if media_type == "image/svg+xml":
headers.update(
{
"Content-Security-Policy": (
"sandbox; default-src 'none'; style-src 'unsafe-inline'; img-src data:"
),
"X-Content-Type-Options": "nosniff",
}
)
return headers
@router.get("/app-identity", response_model=AppIdentityResponse)
async def get_app_identity_settings() -> AppIdentityResponse:
"""Return the application name and uploaded logo. Any logged-in user may read it."""
"""Return the application identity settings readable by all logged-in users."""
return _response(load_system_settings().app_identity)
@@ -89,68 +187,160 @@ async def get_app_identity_settings() -> AppIdentityResponse:
@router.put("/app-identity", response_model=AppIdentityResponse)
async def update_app_identity_settings(
request: Request,
system_name: str = Form(default=""),
remove_logo: bool = Form(default=False),
logo: UploadFile | None = File(default=None),
system_name: Annotated[str, Form()] = "",
remove_logo: Annotated[bool, Form()] = False,
logo: Annotated[UploadFile | None, File()] = None,
header_title_font_mode: Annotated[str | None, Form()] = None,
header_title_font_size: Annotated[int | None, Form()] = None,
light_red_header_background: Annotated[UploadFile | None, File()] = None,
remove_light_red_header_background: Annotated[bool, Form()] = False,
light_red_nav_active: Annotated[UploadFile | None, File()] = None,
remove_light_red_nav_active: Annotated[bool, Form()] = False,
) -> AppIdentityResponse:
"""Save the application chrome name and optional logo. Admin only."""
"""Save the application chrome identity and optional red-theme artwork."""
await _require_admin(request)
if logo is not None and remove_logo:
raise HTTPException(status_code=422, detail="上传 Logo 时不能同时恢复静态 Logo")
if light_red_header_background is not None and remove_light_red_header_background:
raise HTTPException(status_code=422, detail="上传导航栏背景图时不能同时恢复默认图片")
if light_red_nav_active is not None and remove_light_red_nav_active:
raise HTTPException(status_code=422, detail="上传菜单选中样式图时不能同时恢复默认图片")
current = load_system_settings()
previous_logo = app_identity_logo_path(current.app_identity)
previous_header_background = app_identity_light_red_header_background_path(
current.app_identity
)
previous_nav_active = app_identity_light_red_nav_active_path(current.app_identity)
update: dict[str, object] = {"system_name": _normalize_system_name(system_name)}
new_logo_path = None
if logo is not None:
data = await logo.read(_MAX_LOGO_BYTES + 1)
await logo.close()
if not data:
raise HTTPException(status_code=422, detail="Logo 文件不能为空")
if len(data) > _MAX_LOGO_BYTES:
raise HTTPException(status_code=413, detail="Logo 文件不能超过 10MB")
suffix, media_type = _detect_logo(data)
filename = await asyncio.to_thread(_write_logo, data, suffix)
new_logo_path = runtime_home() / "app-identity" / filename
update.update(
{
"logo_filename": filename,
"logo_media_type": media_type,
"logo_version": uuid4().hex,
}
if header_title_font_mode is not None:
update["header_title_font_mode"] = _normalize_header_title_font_mode(header_title_font_mode)
if header_title_font_size is not None:
update["header_title_font_size"] = _normalize_header_title_font_size(header_title_font_size)
logo_asset = (
await _read_uploaded_image(logo, "Logo") if logo is not None else None
)
header_background_asset = (
await _read_uploaded_image(
light_red_header_background,
"导航栏背景图",
allow_svg=True,
max_bytes=None,
)
elif remove_logo:
update.update(
{
"logo_filename": None,
"logo_media_type": None,
"logo_version": None,
}
if light_red_header_background is not None
else None
)
nav_active_asset = (
await _read_uploaded_image(
light_red_nav_active,
"菜单选中样式图",
allow_svg=True,
max_bytes=None,
)
if light_red_nav_active is not None
else None
)
updated = current.app_identity.model_copy(update=update)
staged_paths = []
try:
if logo_asset is not None:
data, suffix, media_type = logo_asset
filename = await asyncio.to_thread(_write_asset, data, suffix, "logo")
staged_paths.append(runtime_home() / "app-identity" / filename)
update.update(
{
"logo_filename": filename,
"logo_media_type": media_type,
"logo_version": uuid4().hex,
}
)
elif remove_logo:
update.update(
{
"logo_filename": None,
"logo_media_type": None,
"logo_version": None,
}
)
if header_background_asset is not None:
data, suffix, media_type = header_background_asset
filename = await asyncio.to_thread(
_write_asset, data, suffix, "light-red-header"
)
staged_paths.append(runtime_home() / "app-identity" / filename)
update.update(
{
"light_red_header_background_filename": filename,
"light_red_header_background_media_type": media_type,
"light_red_header_background_version": uuid4().hex,
}
)
elif remove_light_red_header_background:
update.update(
{
"light_red_header_background_filename": None,
"light_red_header_background_media_type": None,
"light_red_header_background_version": None,
}
)
if nav_active_asset is not None:
data, suffix, media_type = nav_active_asset
filename = await asyncio.to_thread(
_write_asset, data, suffix, "light-red-active"
)
staged_paths.append(runtime_home() / "app-identity" / filename)
update.update(
{
"light_red_nav_active_filename": filename,
"light_red_nav_active_media_type": media_type,
"light_red_nav_active_version": uuid4().hex,
}
)
elif remove_light_red_nav_active:
update.update(
{
"light_red_nav_active_filename": None,
"light_red_nav_active_media_type": None,
"light_red_nav_active_version": None,
}
)
updated = current.app_identity.model_copy(update=update)
saved = save_system_settings(current.model_copy(update={"app_identity": updated}))
except Exception:
if new_logo_path is not None:
await asyncio.to_thread(new_logo_path.unlink, missing_ok=True)
for staged_path in staged_paths:
await asyncio.to_thread(staged_path.unlink, missing_ok=True)
raise
if previous_logo is not None and previous_logo != app_identity_logo_path(saved.app_identity):
await asyncio.to_thread(previous_logo.unlink, missing_ok=True)
previous_paths = (previous_logo, previous_header_background, previous_nav_active)
saved_paths = {
app_identity_logo_path(saved.app_identity),
app_identity_light_red_header_background_path(saved.app_identity),
app_identity_light_red_nav_active_path(saved.app_identity),
}
for previous_path in previous_paths:
if previous_path is not None and previous_path not in saved_paths:
await asyncio.to_thread(previous_path.unlink, missing_ok=True)
return _response(saved.app_identity)
@router.delete("/app-identity", response_model=AppIdentityResponse)
async def delete_app_identity_settings(request: Request) -> AppIdentityResponse:
"""Clear the application name and uploaded logo. Admin only."""
"""Clear the application identity and uploaded theme artwork. Admin only."""
await _require_admin(request)
current = load_system_settings()
previous_logo = app_identity_logo_path(current.app_identity)
previous_paths = (
app_identity_logo_path(current.app_identity),
app_identity_light_red_header_background_path(current.app_identity),
app_identity_light_red_nav_active_path(current.app_identity),
)
saved = save_system_settings(current.model_copy(update={"app_identity": AppIdentitySettings()}))
if previous_logo is not None:
await asyncio.to_thread(previous_logo.unlink, missing_ok=True)
for previous_path in previous_paths:
if previous_path is not None:
await asyncio.to_thread(previous_path.unlink, missing_ok=True)
return _response(saved.app_identity)
@@ -166,5 +356,35 @@ async def get_app_identity_logo() -> FileResponse:
path,
media_type=settings.logo_media_type or "image/png",
content_disposition_type="inline",
headers={"Cache-Control": "public, max-age=31536000, immutable"},
headers=_public_asset_headers(settings.logo_media_type),
)
@public_router.get("/light-red-header-background")
async def get_light_red_header_background() -> FileResponse:
settings = load_system_settings().app_identity
path = app_identity_light_red_header_background_path(settings)
if path is None:
raise HTTPException(status_code=404, detail="未上传红色导航栏背景图")
return FileResponse(
path,
media_type=settings.light_red_header_background_media_type or "image/png",
content_disposition_type="inline",
headers=_public_asset_headers(
settings.light_red_header_background_media_type
),
)
@public_router.get("/light-red-nav-active")
async def get_light_red_nav_active() -> FileResponse:
settings = load_system_settings().app_identity
path = app_identity_light_red_nav_active_path(settings)
if path is None:
raise HTTPException(status_code=404, detail="未上传红色菜单选中样式图")
return FileResponse(
path,
media_type=settings.light_red_nav_active_media_type or "image/png",
content_disposition_type="inline",
headers=_public_asset_headers(settings.light_red_nav_active_media_type),
)
@@ -1,7 +1,9 @@
import asyncio
import logging
import mimetypes
import os
import re
import tempfile
import zipfile
from datetime import UTC, datetime
from pathlib import Path
@@ -11,12 +13,16 @@ from urllib.parse import quote
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import FileResponse, PlainTextResponse, Response
from pydantic import BaseModel, Field
from starlette.background import BackgroundTask
from app.gateway.authz import require_permission
from app.gateway.operation_timing import OperationTiming
from app.gateway.path_utils import aresolve_thread_virtual_path
from app.gateway.utils import sanitize_log_param
from deerflow.config.paths import get_paths
from deerflow.config.system_settings import generated_page_logo_path, load_system_settings
from deerflow.html_runtime.addresses import html_map_tile_url, html_runtime_base_url
from deerflow.html_runtime.offline_bundle import build_html_offline_bundle
from deerflow.runtime.thread_paths import aresolve_path_user_id
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.utils.file_io import file_io
@@ -213,6 +219,44 @@ def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | Non
return headers
def _remove_temporary_bundle(path: str) -> None:
try:
Path(path).unlink(missing_ok=True)
except OSError:
logger.warning("Failed to remove temporary HTML offline bundle: %s", path, exc_info=True)
async def build_html_offline_bundle_response(actual_path: Path) -> FileResponse:
"""Build and stream a portable ZIP for one generated HTML artifact."""
descriptor, temporary_name = tempfile.mkstemp(prefix="deerflow-html-offline-", suffix=".zip")
os.close(descriptor)
temporary_path = Path(temporary_name)
try:
branding = load_system_settings().generated_page_branding
logo_path = generated_page_logo_path(branding)
await file_io.run_until_complete(
build_html_offline_bundle,
actual_path,
temporary_path,
runtime_base_url=html_runtime_base_url(),
map_tile_url=html_map_tile_url(),
generated_logo_path=logo_path,
)
except BaseException:
temporary_path.unlink(missing_ok=True)
raise
download_name = f"{actual_path.stem}-离线包.zip"
return FileResponse(
temporary_path,
filename=download_name,
media_type="application/zip",
headers={"Cache-Control": "private, no-store"},
background=BackgroundTask(_remove_temporary_bundle, str(temporary_path)),
)
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
"""Check if file is text by examining content for null bytes."""
try:
@@ -988,10 +1032,20 @@ async def preview_artifact(
@router.get(
"/threads/{thread_id}/artifacts/{path:path}",
summary="Get Artifact File",
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
description=(
"Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, "
"while active web content is always downloaded. HTML files can be exported with their DeerFlow "
"runtime dependencies as a portable ZIP."
),
)
@require_permission("threads", "read", owner_check=True)
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
async def get_artifact(
thread_id: str,
path: str,
request: Request,
download: bool = False,
offline_bundle: bool = False,
) -> Response:
"""Get an artifact file by its path.
The endpoint automatically detects file types and returns appropriate content types.
@@ -1018,6 +1072,8 @@ async def get_artifact(thread_id: str, path: str, request: Request, download: bo
download (bool): If true, forces attachment download for file types that are
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
is always downloaded regardless of this flag.
offline_bundle (bool): If true for an HTML/XHTML file, returns a ZIP containing
``index.html`` plus the packaged DeerFlow JS/CSS/image runtime dependencies.
Example:
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
@@ -1075,6 +1131,11 @@ async def get_artifact(thread_id: str, path: str, request: Request, download: bo
mime_type, _ = mimetypes.guess_type(actual_path)
if offline_bundle:
if actual_path.suffix.lower() not in {".htm", ".html", ".xhtml"}:
raise HTTPException(status_code=400, detail="Offline bundles are available for HTML files only")
return await build_html_offline_bundle_response(actual_path)
if download:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
@@ -5,11 +5,17 @@
读对所有登录用户开放(定时任务表单下拉也要用),写(创建/编辑/上传覆盖/删除/重置)
仅管理员——与敏感词管理同一鉴权口径。旧端点 ``GET /api/scheduled-tasks/html-templates``
保持不变(内部同样读存储),本路由是它的管理超集。
结构化模板(kind="structured",如 no.1)额外暴露默认数据的读写接口:
``GET /{id}/data`` 读、``PUT /{id}/data`` 整体替换(管理页「编辑数据」的持久化入口,
替换前经转换契约校验)。``GET /{id}/content`` 默认返回组合后的完整文档(预览用),
``?raw=1`` 返回骨架原文(编辑用)。
"""
from __future__ import annotations
import asyncio
import re
from typing import Any
from fastapi import APIRouter, HTTPException, Request
@@ -17,7 +23,7 @@ from pydantic import BaseModel, Field
from app.gateway.catalog_cache import cached_catalog, catalog_key, invalidate_catalog
from app.gateway.deps import get_optional_user_from_request
from deerflow.runtime.scheduler import template_store
from deerflow.runtime.scheduler import template_fill, template_store
router = APIRouter(prefix="/api/html-templates", tags=["html-templates"])
@@ -48,6 +54,33 @@ class TemplateUpdateRequest(BaseModel):
html: str | None = Field(default=None, description="新的模板正文(上传覆盖走这里)")
class TemplateDataUpdateRequest(BaseModel):
data: dict[str, Any] = Field(..., description="新的默认数据(整体替换 data.json,须通过契约校验)")
manifest: dict[str, Any] | None = Field(
default=None,
description="可选:新的 MANIFEST(整体替换 manifest.json,系统名称/导航页面/交互规则)",
)
class FieldVisibilityUpdateRequest(BaseModel):
hidden: list[str] = Field(default_factory=list, description="要隐藏的字段显示名列表(其余字段全部显示)")
extract_locked: list[str] | None = Field(
default=None,
description="可选:不接受抽取更新的字段名单(None=不动;给出则整体替换,空表=全部接受)",
)
class CustomModuleListUpdateRequest(BaseModel):
modules: list[dict[str, Any]] = Field(..., description="自定义模块全量列表(整体替换)")
class CustomModuleGenerateRequest(BaseModel):
page: str = Field(..., description="挂载的页面编号(manifest.页面 的 编号)")
title: str = Field(default="", description="模块标题(留空由模型起名)")
description: str = Field(..., min_length=4, description="用自然语言描述想新增的展示内容")
model_name: str | None = Field(default=None, description="可选:指定生成模型")
@router.get("")
async def list_html_templates(request: Request) -> dict:
"""模板清单(含 builtin/convertible/size/时间),供管理页与定时任务表单共用。"""
@@ -69,15 +102,230 @@ async def get_html_template(template_id: str, request: Request) -> dict[str, Any
@router.get("/{template_id}/content")
async def get_html_template_content(template_id: str, request: Request) -> dict:
async def get_html_template_content(
template_id: str, request: Request, raw: bool = False
) -> dict:
"""模板内容。默认返回可预览的完整文档(结构化模板 = 默认数据组合产物);
``?raw=1`` 返回正文原文(结构化模板 = 带 __PAGE_* 标记的骨架,编辑用)。"""
async def load() -> dict:
try:
html = await asyncio.to_thread(template_store.read_template_html, template_id)
except template_store.TemplateStoreError:
raise _not_found(template_id)
if raw:
try:
html = await asyncio.to_thread(template_store.read_template_html, template_id)
except template_store.TemplateStoreError:
raise _not_found(template_id)
else:
try:
html = await asyncio.to_thread(template_fill.read_template_text, template_id)
except template_fill.TemplateError:
raise _not_found(template_id)
return {"template_id": template_id, "html": html}
return await cached_catalog(request, "html_templates", catalog_key("content", template_id), load)
suffix = "raw" if raw else "doc"
return await cached_catalog(request, "html_templates", catalog_key("content", template_id, suffix), load)
@router.get("/{template_id}/data")
async def get_html_template_data(template_id: str, request: Request) -> dict[str, Any]:
"""结构化模板的默认数据(data.json) + MANIFEST + 各模块中文名(管理页「编辑数据」用)。"""
async def load() -> dict[str, Any]:
try:
data, manifest, modules = await asyncio.gather(
asyncio.to_thread(template_store.read_template_data, template_id),
asyncio.to_thread(template_store.read_template_manifest, template_id),
asyncio.to_thread(template_fill.structured_module_labels, template_id),
)
except (template_store.TemplateStoreError, template_fill.TemplateError) as exc:
status = 404 if "不存在" in str(exc) else 400
raise HTTPException(status_code=status, detail=str(exc))
return {
"template_id": template_id,
"data": data,
"manifest": manifest,
"modules": modules,
}
return await cached_catalog(request, "html_templates", catalog_key("data", template_id), load)
@router.put("/{template_id}/data")
async def update_html_template_data(
template_id: str, body: TemplateDataUpdateRequest, request: Request
) -> dict[str, Any]:
"""整体替换结构化模板的默认数据/MANIFEST 并持久化(仅管理员;先经契约校验)。"""
await _require_admin(request)
try:
meta = template_store.write_template_data(
template_id, body.data, manifest=body.manifest
)
except template_store.TemplateStoreError as exc:
status = 404 if "不存在" in str(exc) else 400
raise HTTPException(status_code=status, detail=str(exc))
await invalidate_catalog(request, "html_templates")
return meta
# ── 字段显示 / 自定义模块 ────────────────────────────────────────────────────
@router.get("/{template_id}/fields")
async def get_html_template_fields(template_id: str, request: Request) -> dict[str, Any]:
"""字段目录(fields.json,含中文说明)× 当前可见性 + 页面清单 + 自定义模块列表。"""
async def load() -> dict[str, Any]:
try:
catalog, manifest = await asyncio.gather(
asyncio.to_thread(template_store.read_template_fields, template_id),
asyncio.to_thread(template_store.read_template_manifest, template_id),
)
except template_store.TemplateStoreError as exc:
status = 404 if "不存在" in str(exc) else 400
raise HTTPException(status_code=status, detail=str(exc))
hidden = manifest.get("字段显示")
hidden = hidden if isinstance(hidden, dict) else {}
locks = manifest.get("字段抽取关闭")
locks = locks if isinstance(locks, dict) else {}
fields = [
{
"label": str(f.get("label") or ""),
"pages": [str(p) for p in (f.get("pages") or [])],
"description": str(f.get("description") or ""),
"visible": hidden.get(str(f.get("label"))) is not False,
# 是否接受定时任务抽取数据更新(默认 True);布局分区无独立数据键,恒为 True 且前端禁用开关。
"extract": locks.get(str(f.get("label"))) is not True,
"has_key": bool(f.get("keys")),
}
for f in catalog
]
pages = [
{"key": str(p.get("编号") or ""), "label": str(p.get("名称") or p.get("编号") or "")}
for p in manifest.get("页面") or []
if isinstance(p, dict)
]
custom = manifest.get("自定义模块")
return {
"template_id": template_id,
"fields": fields,
"pages": pages,
"custom_modules": custom if isinstance(custom, list) else [],
}
return await cached_catalog(request, "html_templates", catalog_key("fields", template_id), load)
@router.put("/{template_id}/fields")
async def update_html_template_fields(
template_id: str, body: FieldVisibilityUpdateRequest, request: Request
) -> dict[str, Any]:
"""整体替换字段显示/隐藏配置(可选一并替换抽取锁定名单;仅管理员)。"""
await _require_admin(request)
try:
meta = template_store.write_template_field_visibility(template_id, body.hidden)
if body.extract_locked is not None:
template_store.write_template_field_locks(template_id, body.extract_locked)
except template_store.TemplateStoreError as exc:
status = 404 if "不存在" in str(exc) else 400
raise HTTPException(status_code=status, detail=str(exc))
await invalidate_catalog(request, "html_templates")
return meta
@router.put("/{template_id}/custom-modules")
async def update_html_template_custom_modules(
template_id: str, body: CustomModuleListUpdateRequest, request: Request
) -> dict[str, Any]:
"""自定义模块列表整体替换(入库前逐条消毒;仅管理员)。"""
await _require_admin(request)
try:
meta = template_store.write_template_custom_modules(template_id, body.modules)
except template_store.TemplateStoreError as exc:
status = 404 if "不存在" in str(exc) else 400
raise HTTPException(status_code=status, detail=str(exc))
await invalidate_catalog(request, "html_templates")
return meta
_CUSTOM_MODULE_PROMPT = """你是资深前端设计师,为知识数据展示系统(红白主题)绘制一个自定义展示模块的 HTML 片段。
主题与可直接使用的样式(不要自造 CSS,尽量复用):
- 颜色: 背景 #f5f5f6、卡片白 #fff、主红 #c8102e、深红 #8f1024、浅红底 #fff1f3、正文 #262626、次要 #737373
- 布局类: panel(白卡容器) panel-title(红色竖条标题) grid2(两列) metrics(三列指标组)
- 组件类: metric metric-label metric-value metric-unit metric-note(指标卡) | chip chip-red chip-green chip-orange(标签)
- 组件类: text-list text-item(条目列表) | info-grid info-item(信息格) | bar-row bar(横条统计) | entry-grid entry-card(入口卡)
- 也可以用少量 inline style 微调(颜色只允许上述色板)
硬性约束:
1. 只输出一个最外层 <div> 包裹的 HTML 片段,不要 <html>/<head>/<body>/<script>/<style>/<iframe>/<form>
2. 禁止任何 on* 事件属性、javascript: 链接、外部资源引用(图片/字体/CDN)
3. 内容用简体中文;基于下面的描述合理组织版式;没有的数据写「待补充」,不要编造精确数字
4. **文字极简(最重要)**:整个模块总文字量 ≤120 字;任何一条文字 ≤30 字;严禁出现大段连续正文或 <p> 长段落
5. **呈现方式**:数字/统计一律用 metric 指标卡(大数字+短标签+一句 ≤20 字注释);分类/状态用 chip 标签;多条信息用 text-item 短条目(每条一行短句);条目间用并列结构,不要写成文章
6. 只输出片段本身,不要任何解释文字或 markdown 代码块
挂载页面:{page}
模块标题:{title}
模块描述:{description}"""
_THINK_BLOCK_RE = re.compile(r"<think>.*?</think>", re.IGNORECASE | re.DOTALL)
_THINK_TAG_RE = re.compile(r"</?think>", re.IGNORECASE)
_FENCE_HTML_RE = re.compile(r"```(?:html)?\s*(.*?)```", re.DOTALL | re.IGNORECASE)
def _extract_fragment(raw: str) -> str:
text = str(raw or "").strip()
text = _THINK_BLOCK_RE.sub("", text)
text = _THINK_TAG_RE.sub("", text).strip()
fence = _FENCE_HTML_RE.search(text)
if fence:
text = fence.group(1).strip()
body = re.search(r"<body[^>]*>(.*?)</body>", text, re.IGNORECASE | re.DOTALL)
if body:
text = body.group(1).strip()
# 思考文字混在片段前(无标签包裹)时,从第一个 "<div" 起截取。
div_at = text.find("<div")
if div_at > 0:
text = text[div_at:].strip()
return text
@router.post("/{template_id}/custom-modules/generate")
async def generate_html_template_custom_module(
template_id: str, body: CustomModuleGenerateRequest, request: Request
) -> dict[str, Any]:
"""按用户的自然语言描述调用模型生成自定义模块 HTML 片段(已消毒,未入库)。"""
await _require_admin(request)
try:
# 模板必须存在且页面编号合法,防止生成挂在不存在的页面上。
manifest = await asyncio.to_thread(template_store.read_template_manifest, template_id)
except template_store.TemplateStoreError as exc:
status = 404 if "不存在" in str(exc) else 400
raise HTTPException(status_code=status, detail=str(exc))
valid_pages = {
str(p.get("编号") or "") for p in manifest.get("页面") or [] if isinstance(p, dict)
}
if body.page not in valid_pages:
raise HTTPException(status_code=400, detail=f"未知页面编号:{body.page}")
from langchain_core.messages import HumanMessage
from deerflow.models import create_chat_model
prompt = _CUSTOM_MODULE_PROMPT.format(
page=body.page, title=body.title or "(由你起一个简短标题)", description=body.description
)
model = create_chat_model(name=body.model_name or None, thinking_enabled=False)
response = await model.ainvoke([HumanMessage(content=prompt)])
content = getattr(response, "content", response)
if isinstance(content, list):
content = "\n".join(
part.get("text", "") if isinstance(part, dict) else str(part) for part in content
)
fragment = template_store.sanitize_html_fragment(_extract_fragment(str(content)))
if "<div" not in fragment:
raise HTTPException(status_code=502, detail="模型未返回有效的 HTML 片段,请调整描述后重试")
return {"template_id": template_id, "page": body.page, "html": fragment}
@router.post("")
@@ -681,6 +681,9 @@ def _embed_allowed_weknora_ids(ctx: WeKnoraEmbedContext) -> tuple[str, ...]:
def _embed_context_payload(ctx: WeKnoraEmbedContext) -> dict[str, Any]:
# Keep this payload bounded: it is signed directly into the browser cookie.
# The visible knowledge-base list is hydrated only for the embedded page or
# its list API call, after the cookie has been verified.
return {
"mapping_id": ctx.mapping_id,
"weknora_id": ctx.weknora_id,
@@ -688,11 +691,79 @@ def _embed_context_payload(ctx: WeKnoraEmbedContext) -> dict[str, Any]:
"is_admin": ctx.is_admin,
"can_write": ctx.can_write,
"is_conversation_deposit": ctx.is_conversation_deposit,
"allowed_weknora_ids": list(_embed_allowed_weknora_ids(ctx)),
"exp": ctx.exp,
}
async def _with_visible_embed_knowledge_bases(
request: Request,
ctx: WeKnoraEmbedContext,
) -> WeKnoraEmbedContext:
"""Hydrate the iframe-only picker allowlist without putting it in a cookie."""
visible_rows = await _store(request).list_visible(
ctx.user_id,
scope="all",
# Mirror the regular picker. An administrator must not receive another
# user's private bases merely because the iframe uses an admin token.
is_admin=False,
)
return replace(
ctx,
allowed_weknora_ids=tuple(
dict.fromkeys(
value
for value in (
ctx.weknora_id,
*(str(item.get("weknora_id") or "") for item in visible_rows),
)
if value
)
),
)
def _embed_directory_time(value: Any) -> str | None:
if isinstance(value, datetime):
return value.isoformat()
if isinstance(value, str) and value.strip():
return value
return None
async def _local_embed_knowledge_base_directory(
request: Request,
ctx: WeKnoraEmbedContext,
) -> Response:
"""Serve the iframe picker from DeerFlow mappings, never WeKnora's full directory."""
rows = await _store(request).list_visible(
ctx.user_id,
scope="all",
# Match the normal picker: the WeKnora service-admin token must not
# make an administrator's private libraries discoverable to others.
is_admin=False,
)
items = [
{
"id": str(row.get("weknora_id") or ""),
"name": str(row.get("name") or ""),
"description": str(row.get("description") or ""),
"type": str(row.get("kb_type") or "document"),
"status": "active",
"created_at": _embed_directory_time(row.get("created_at")),
"updated_at": _embed_directory_time(row.get("updated_at")),
}
for row in rows
if str(row.get("weknora_id") or "")
]
response = _json_response(
{"success": True, "data": items, "total": len(items)},
)
response.headers["Cache-Control"] = "no-store"
return response
def _weknora_id_from_platform_path(path: str) -> str | None:
prefix = "/platform/knowledge-bases/"
if not path.startswith(prefix):
@@ -708,15 +779,13 @@ async def _switch_embed_context(
ctx: WeKnoraEmbedContext,
weknora_id: str,
) -> WeKnoraEmbedContext:
"""Resolve a permitted native-UI switch to the exact cmzs mapping.
"""Resolve a native-UI switch to the exact cmzs mapping.
The embedded WeKnora UI uses a service-admin token, so every target remains
constrained to an ID signed into this short-lived session and is checked
again against the cmzs visibility rules before its page can be proxied.
The embedded WeKnora UI uses a service-admin token, so the target must be
checked against the current cmzs visibility rules on every switch. This is
deliberately dynamic rather than a signed ID list in the browser cookie.
"""
if weknora_id not in _embed_allowed_weknora_ids(ctx):
raise HTTPException(status_code=403, detail="Knowledge base is outside the embedded session scope")
store = _store(request)
mapped = await store.get_by_weknora_id(weknora_id)
if mapped is None:
@@ -2807,13 +2876,6 @@ async def _prepare_weknora_embed_session(
) -> tuple[str, dict[str, Any]]:
_runtime_for_iframe_proxy(request)
row, user_id, is_admin = await _authorized_mapping(request, mapping_id, write=False)
visible_rows = await _store(request).list_visible(
user_id,
scope="all",
# The iframe must mirror the knowledge-base picker, not grant an
# administrator an unlisted user's private knowledge bases.
is_admin=False,
)
writable = None
if not _is_conversation_deposit_mapping(row):
writable = await _store(request).get_authorized(mapping_id, user_id, write=True, is_admin=is_admin)
@@ -2825,7 +2887,6 @@ async def _prepare_weknora_embed_session(
is_admin=is_admin,
can_write=writable is not None,
is_conversation_deposit=_is_conversation_deposit_mapping(row),
allowed_weknora_ids=tuple(dict.fromkeys(value for value in (weknora_id, *(str(item.get("weknora_id") or "") for item in visible_rows)) if value)),
exp=int(time.time()) + _WEKNORA_EMBED_TTL_SECONDS,
)
return weknora_id, _embed_context_payload(ctx)
@@ -2885,6 +2946,7 @@ async def proxy_weknora_platform_page(request: Request, proxied_path: str = "")
requested_weknora_id = _weknora_id_from_platform_path(requested)
if requested_weknora_id is not None:
selected_ctx = await _switch_embed_context(request, ctx, requested_weknora_id)
selected_ctx = await _with_visible_embed_knowledge_bases(request, selected_ctx)
response = await _proxy_weknora_request(
request,
target_base_url=_runtime_for_iframe_proxy(request).web_base_url,
@@ -2909,6 +2971,7 @@ async def proxy_weknora_platform_page(request: Request, proxied_path: str = "")
public_target = _with_public_prefix(_request_public_prefix(request), allowed_path)
return RedirectResponse(f"{public_target}?{target_query}", status_code=302)
runtime = _runtime_for_iframe_proxy(request)
ctx = await _with_visible_embed_knowledge_bases(request, ctx)
return await _proxy_weknora_request(
request,
target_base_url=runtime.web_base_url,
@@ -2958,6 +3021,8 @@ async def proxy_weknora_api(request: Request, proxied_path: str) -> Response:
if request.method.upper() == "OPTIONS":
return Response(status_code=204)
ctx = await _read_embed_context(request)
if request.method.upper() in {"GET", "HEAD"} and proxied_path.rstrip("/") == "knowledge-bases":
return await _local_embed_knowledge_base_directory(request, ctx)
runtime = _runtime_for_iframe_proxy(request)
return await _proxy_weknora_request(
request,
@@ -2982,6 +3047,8 @@ async def proxy_weknora_embed_api(request: Request, proxied_path: str) -> Respon
if request.method.upper() == "OPTIONS":
return Response(status_code=204)
ctx = await _read_embed_context(request)
if request.method.upper() in {"GET", "HEAD"} and proxied_path.rstrip("/") == "knowledge-bases":
return await _local_embed_knowledge_base_directory(request, ctx)
runtime = _runtime_for_iframe_proxy(request)
return await _proxy_weknora_request(
request,
@@ -12,7 +12,7 @@ import time
import zipfile
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from typing import Any, Literal
from fastapi import APIRouter, Depends, File, HTTPException, Query, Request, Response, UploadFile
from fastapi.responses import FileResponse
@@ -35,6 +35,7 @@ from app.gateway.skill_catalog_cache import (
load_or_warm_skill_catalog_snapshot,
load_skill_catalog_projection,
refresh_skill_catalog_projection,
set_skill_catalog_enabled,
)
from deerflow.agents.lead_agent.prompt import refresh_skills_system_prompt_cache_async
from deerflow.config.app_config import AppConfig
@@ -369,6 +370,24 @@ class SkillRollbackRequest(BaseModel):
history_index: int = Field(default=-1, description="History entry index to restore from, defaulting to the latest change.")
class SkillVersionEntryResponse(BaseModel):
version: int
sha256: str
created_at: str
created_by: str | None = None
action: str
summary: str | None = None
based_on_version: int | None = None
active: bool = False
class SkillVersionsResponse(BaseModel):
enabled: bool
skill_name: str
active_version: int | None = None
versions: list[SkillVersionEntryResponse] = Field(default_factory=list)
def _skill_to_response(
skill: Skill,
ownership: dict[str, Any] | None = None,
@@ -1305,6 +1324,10 @@ async def install_skill_upload(
request: Request,
file: UploadFile = File(..., description="The .zip or .skill archive to install"),
overwrite: bool = False,
install_mode: Literal["iterate", "overwrite"] | None = Query(
default=None,
description="Same-name custom upload mode when versioning is enabled: iterate or overwrite.",
),
target_category: SkillCategory = SkillCategory.CUSTOM,
config: AppConfig = Depends(get_config),
skill_store: SkillStore = Depends(get_skill_store),
@@ -1339,9 +1362,31 @@ async def install_skill_upload(
target_category=target_category,
conflict_kind=validation.get("conflict_kind"),
)
if (overwrite and not can_install_as_is and not can_overwrite) or (not overwrite and not can_install_as_is):
requested_overwrite = overwrite or install_mode in {"iterate", "overwrite"}
if (requested_overwrite and not can_install_as_is and not can_overwrite) or (
not requested_overwrite and not can_install_as_is
):
raise HTTPException(status_code=409, detail=conflict_message)
if install_mode == "iterate":
from deerflow.skills.versioning import record_custom_skill_version, skill_versioning_enabled
if not skill_versioning_enabled():
raise HTTPException(status_code=409, detail="技能迭代功能未开启")
if target_category != SkillCategory.CUSTOM or not _owned_by_me or not can_overwrite:
raise HTTPException(status_code=409, detail="只有本人已有的同名自定义技能可以迭代")
overwrite = True
await asyncio.to_thread(
record_custom_skill_version,
preview_name,
storage=storage,
action="baseline",
author=user_id,
summary="同名上传迭代前的活动版本",
)
elif install_mode == "overwrite":
overwrite = True
result = await storage.ainstall_skill_from_archive(tmp_path, overwrite=overwrite, require_skill_suffix=False, target_category=target_category)
storage.invalidate_skills_cache()
try:
from deerflow.skills.usage import mark_uploaded
mark_uploaded(result["skill_name"])
@@ -1360,8 +1405,49 @@ async def install_skill_upload(
await skill_store.ensure_legacy(
{"name": result["skill_name"], "owner_user_id": user_id, "published": False}
)
try:
from deerflow.skills.versioning import (
record_custom_skill_version,
reset_custom_skill_versions,
skill_versioning_enabled,
)
if skill_versioning_enabled():
if result.get("overwritten") and install_mode == "iterate":
await asyncio.to_thread(
record_custom_skill_version,
result["skill_name"],
storage=storage,
action="upload_iteration",
author=user_id,
summary="上传同名技能并迭代为新版本",
)
elif result.get("overwritten"):
await asyncio.to_thread(
reset_custom_skill_versions,
result["skill_name"],
storage=storage,
author=user_id,
summary="上传同名技能并直接覆盖",
)
else:
await asyncio.to_thread(
record_custom_skill_version,
result["skill_name"],
storage=storage,
action="upload",
author=user_id,
summary="上传技能初始版本",
)
except Exception:
logger.exception("Failed to persist custom skill version after upload: %s", result["skill_name"])
# The package is already installed. Do not report the whole
# upload as failed (which would invite an unsafe duplicate
# retry); opening the version list lazily snapshots the current
# package and repairs the manifest.
await _refresh_catalog_snapshot(config)
await refresh_skills_system_prompt_cache_async()
await _sync_skill_view(result["skill_name"])
return SkillInstallResponse(**result)
except FileNotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
@@ -1603,6 +1689,7 @@ async def install_skill(
target_category=target_category,
require_skill_suffix=False,
)
storage.invalidate_skills_cache()
try:
from deerflow.skills.usage import mark_uploaded
mark_uploaded(result.skill_name)
@@ -1616,8 +1703,16 @@ async def install_skill(
await skill_store.ensure_legacy(
{"name": result.skill_name, "owner_user_id": user_id, "published": False}
)
await _record_custom_version(
result.skill_name,
storage=storage,
action="artifact_install",
author=user_id,
summary="从会话产物安装技能初始版本",
)
await _refresh_catalog_snapshot(config)
await refresh_skills_system_prompt_cache_async()
await _sync_skill_view(result.skill_name)
return result
except FileNotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
@@ -1730,6 +1825,14 @@ async def duplicate_skill(
mark_uploaded(new_name)
except Exception:
pass
storage.invalidate_skills_cache()
await _record_custom_version(
new_name,
storage=storage,
action="duplicate",
author=user_id,
summary=f"从技能 {source_name} 复制创建",
)
await _refresh_catalog_snapshot(config)
await refresh_skills_system_prompt_cache_async()
return SkillInstallResponse(
@@ -1784,6 +1887,13 @@ async def reconcile_custom_skills(
mark_uploaded(skill.name)
except Exception:
pass
await _record_custom_version(
skill.name,
storage=storage,
action="reconcile",
author=user_id,
summary="接管已有自定义技能并建立初始版本",
)
adopted.append(skill.name)
if adopted:
@@ -1869,8 +1979,16 @@ async def update_custom_skill(skill_name: str, request: CustomSkillUpdateRequest
scan = await scan_skill_content(request.content, executable=False, location=f"{skill_name}/{SKILL_MD_FILE}", app_config=config)
if scan.decision == "block":
raise HTTPException(status_code=400, detail=f"Security scan blocked the edit: {scan.reason}")
await _record_custom_version(
skill_name,
storage=storage,
action="baseline",
author=None,
summary="编辑前的活动版本",
)
prev_content = storage.read_custom_skill(skill_name)
storage.write_custom_skill(skill_name, SKILL_MD_FILE, request.content)
storage.invalidate_skills_cache()
await _refresh_catalog_snapshot(config)
storage.append_history(
skill_name,
@@ -1889,7 +2007,15 @@ async def update_custom_skill(skill_name: str, request: CustomSkillUpdateRequest
bump_patch(skill_name)
except Exception:
pass
await _record_custom_version(
skill_name,
storage=storage,
action="human_edit",
author=None,
summary="编辑 SKILL.md",
)
await refresh_skills_system_prompt_cache_async()
await _sync_skill_view(skill_name)
return await get_custom_skill(skill_name, config)
except HTTPException:
raise
@@ -1941,6 +2067,12 @@ async def delete_custom_skill(
await skill_store.delete_admin(skill_name)
else:
await skill_store.delete(skill_name, user_id)
try:
from deerflow.skills.versioning import delete_custom_skill_versions
await asyncio.to_thread(delete_custom_skill_versions, skill_name)
except Exception:
logger.warning("Failed to delete version history for %s", skill_name, exc_info=True)
await _refresh_catalog_snapshot(config)
try:
@@ -2279,6 +2411,108 @@ async def get_custom_skill_history(skill_name: str, config: AppConfig = Depends(
raise HTTPException(status_code=500, detail=f"Failed to read history: {str(e)}")
@router.get(
"/skills/custom/{skill_name}/versions",
response_model=SkillVersionsResponse,
summary="List Custom Skill Versions",
)
async def get_custom_skill_versions(
skill_name: str,
request: Request,
config: AppConfig = Depends(get_config),
skill_store: SkillStore = Depends(get_skill_store),
) -> SkillVersionsResponse:
"""List complete-package snapshots for one owned/admin-managed custom skill."""
try:
skill_name = SkillStorage.validate_skill_name(skill_name.replace("\r\n", "").replace("\n", ""))
await _get_editable_skill_or_raise(
skill_store,
skill_name,
_current_user_id(request),
is_admin=_is_admin_user(request),
)
storage = get_or_new_skill_storage(app_config=config)
if not storage.custom_skill_exists(skill_name):
raise HTTPException(status_code=404, detail=f"Custom skill '{skill_name}' not found")
from deerflow.skills.versioning import list_custom_skill_versions
payload = await asyncio.to_thread(
list_custom_skill_versions,
skill_name,
storage=storage,
author=_current_user_id(request),
)
return SkillVersionsResponse.model_validate(payload)
except HTTPException:
raise
except (FileNotFoundError, ValueError) as e:
raise HTTPException(status_code=404, detail=str(e))
except Exception as e:
logger.error("Failed to list versions for %s: %s", skill_name, e, exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to list skill versions: {e}")
@router.post(
"/skills/custom/{skill_name}/versions/{version}/activate",
response_model=SkillVersionsResponse,
summary="Activate Custom Skill Version",
)
async def activate_custom_skill_version_endpoint(
skill_name: str,
version: int,
request: Request,
config: AppConfig = Depends(get_config),
skill_store: SkillStore = Depends(get_skill_store),
) -> SkillVersionsResponse:
"""Switch the sole model-visible package without touching usage counters."""
try:
skill_name = SkillStorage.validate_skill_name(skill_name.replace("\r\n", "").replace("\n", ""))
await _get_editable_skill_or_raise(
skill_store,
skill_name,
_current_user_id(request),
is_admin=_is_admin_user(request),
)
storage = get_or_new_skill_storage(app_config=config)
if not storage.custom_skill_exists(skill_name):
raise HTTPException(status_code=404, detail=f"Custom skill '{skill_name}' not found")
from deerflow.skills.versioning import (
activate_custom_skill_version,
list_custom_skill_versions,
)
await asyncio.to_thread(
activate_custom_skill_version,
skill_name,
version,
storage=storage,
)
storage.invalidate_skills_cache()
await _refresh_catalog_snapshot(config)
await refresh_skills_system_prompt_cache_async()
await _sync_skill_view(skill_name)
payload = await asyncio.to_thread(
list_custom_skill_versions,
skill_name,
storage=storage,
author=_current_user_id(request),
)
return SkillVersionsResponse.model_validate(payload)
except HTTPException:
raise
except RuntimeError as e:
raise HTTPException(status_code=409, detail=str(e))
except FileNotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
logger.error("Failed to activate %s version %s: %s", skill_name, version, e, exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to activate skill version: {e}")
@router.post("/skills/custom/{skill_name}/rollback", response_model=CustomSkillContentResponse, summary="Rollback Custom Skill")
async def rollback_custom_skill(skill_name: str, request: SkillRollbackRequest, config: AppConfig = Depends(get_config)) -> CustomSkillContentResponse:
try:
@@ -2309,10 +2543,26 @@ async def rollback_custom_skill(skill_name: str, request: SkillRollbackRequest,
if scan.decision == "block":
storage.append_history(skill_name, history_entry)
raise HTTPException(status_code=400, detail=f"Rollback blocked by security scanner: {scan.reason}")
await _record_custom_version(
skill_name,
storage=storage,
action="baseline",
author=None,
summary="旧历史回滚前的活动版本",
)
storage.write_custom_skill(skill_name, SKILL_MD_FILE, target_content)
storage.invalidate_skills_cache()
await _refresh_catalog_snapshot(config)
storage.append_history(skill_name, history_entry)
await _record_custom_version(
skill_name,
storage=storage,
action="legacy_rollback",
author=None,
summary="从旧编辑历史恢复 SKILL.md",
)
await refresh_skills_system_prompt_cache_async()
await _sync_skill_view(skill_name)
return await get_custom_skill(skill_name, config)
except HTTPException:
raise
@@ -2775,6 +3025,28 @@ async def _sync_skill_view(name: str) -> None:
logger.warning("Failed to sync skill %s into the sandbox view", name, exc_info=True)
async def _record_custom_version(
name: str,
*,
storage: SkillStorage,
action: str,
author: str | None,
summary: str,
) -> None:
"""Capture one complete custom-skill package when versioning is enabled."""
from deerflow.skills.versioning import record_custom_skill_version
await asyncio.to_thread(
record_custom_skill_version,
name,
storage=storage,
action=action,
author=author,
summary=summary,
)
@router.put(
"/skills/{skill_name}/files/content",
response_model=SkillFileSaveResponse,
@@ -2826,7 +3098,18 @@ async def save_skill_file(
# SKILL.md edits must keep their frontmatter valid (name must match).
storage.validate_skill_markdown_content(skill.name, body.content)
if skill.category == SkillCategory.CUSTOM:
await _record_custom_version(
skill.name,
storage=storage,
action="baseline",
author=_current_user_id(request),
summary="文件编辑前的活动版本",
)
_write_skill_text_file_atomically(target, body.content)
if skill.category == SkillCategory.CUSTOM:
storage.invalidate_skills_cache()
try:
if skill.category == SkillCategory.CUSTOM:
@@ -2852,6 +3135,15 @@ async def save_skill_file(
pass
await refresh_skills_system_prompt_cache_async()
if skill.category == SkillCategory.CUSTOM:
await _record_custom_version(
skill.name,
storage=storage,
action="human_edit_file",
author=_current_user_id(request),
summary=f"编辑文件 {relative_posix}",
)
await _sync_skill_view(skill.name)
stat = target.stat()
@@ -2911,10 +3203,19 @@ async def delete_skill_file(
prev_content = target.read_text(encoding="utf-8")
except UnicodeDecodeError:
pass
if skill.category == SkillCategory.CUSTOM:
await _record_custom_version(
skill.name,
storage=get_or_new_skill_storage(app_config=config),
action="baseline",
author=_current_user_id(request),
summary="文件删除前的活动版本",
)
target.unlink()
if skill.category == SkillCategory.CUSTOM:
storage = get_or_new_skill_storage(app_config=config)
storage.invalidate_skills_cache()
try:
storage.append_history(
skill.name,
@@ -2928,6 +3229,13 @@ async def delete_skill_file(
)
except Exception as history_err: # noqa: BLE001
logger.warning("Failed to append deletion history for %s/%s: %s", skill.name, relative_posix, history_err)
await _record_custom_version(
skill.name,
storage=storage,
action="human_delete_file",
author=_current_user_id(request),
summary=f"删除文件 {relative_posix}",
)
else:
logger.info("Administrator deleted built-in skill file %s/%s", skill.name, relative_posix)
await _sync_skill_view(skill.name)
@@ -3010,6 +3318,11 @@ async def update_skill(
logger.info(f"Skills configuration updated and saved to: {config_path}")
await refresh_skills_system_prompt_cache_async()
await _sync_skill_view(skill_name)
# The list endpoint reads skills.enabled from the catalog projection.
# Write that one row before the client refetches. A missing row still
# needs the full snapshot so a newly discovered skill can appear.
if not await set_skill_catalog_enabled(skill_name, request.enabled):
await _refresh_catalog_snapshot(config)
skills = get_or_new_skill_storage(app_config=config).load_skills(enabled_only=False)
updated_skill = next((s for s in skills if s.name == skill_name), None)
@@ -16,15 +16,20 @@ from app.gateway.deps import get_current_user_from_request, get_optional_user_fr
from deerflow.config import get_app_config
from deerflow.config.system_settings import (
BUILTIN_THEME_IDS,
AIWritingPageDescription,
AppearanceRouteParam,
AuthenticationSettings,
CallMetricsSettings,
CompactShortcutButton,
ConversationContextSettings,
CustomThemePreset,
GeneralQaPageDescriptions,
HtmlMapSettings,
LeaderboardSettings,
ModelConcurrencySettings,
ModelPromptSuffix,
PageDescriptionsSettings,
PageHeaderDescription,
PluginDomainAgentMapping,
PluginManagementSettings,
PromptPrefixSettings,
@@ -33,6 +38,7 @@ from deerflow.config.system_settings import (
RetryKeepPartialSettings,
RoundtableRoomLimitSettings,
SkillCompressionSettings,
SkillVersioningSettings,
StrategyProjectSettings,
WordHeadingNumberingStyle,
WriteFileCompatibilitySettings,
@@ -208,6 +214,91 @@ class StrategyProjectSettingsUpdateRequest(BaseModel):
taiwan_strait_demo_enabled: bool | None = None
class PageDescriptionsResponse(BaseModel):
general_qa: GeneralQaPageDescriptions = Field(default_factory=GeneralQaPageDescriptions)
knowledge_base: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
ai_writing: AIWritingPageDescription = Field(default_factory=AIWritingPageDescription)
strategy_projects: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
deep_research: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
gongwen: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
light_app_management: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
light_app_list: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
agents: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
business_chains: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
skills: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
knowledge_management: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
class PageDescriptionsUpdateRequest(BaseModel):
general_qa: GeneralQaPageDescriptions | None = None
knowledge_base: PageHeaderDescription | None = None
ai_writing: AIWritingPageDescription | None = None
strategy_projects: PageHeaderDescription | None = None
deep_research: PageHeaderDescription | None = None
gongwen: PageHeaderDescription | None = None
light_app_management: PageHeaderDescription | None = None
light_app_list: PageHeaderDescription | None = None
agents: PageHeaderDescription | None = None
business_chains: PageHeaderDescription | None = None
skills: PageHeaderDescription | None = None
knowledge_management: PageHeaderDescription | None = None
def _page_descriptions_response(
settings: PageDescriptionsSettings,
) -> PageDescriptionsResponse:
return PageDescriptionsResponse(
general_qa=settings.general_qa,
knowledge_base=settings.knowledge_base,
ai_writing=settings.ai_writing,
strategy_projects=settings.strategy_projects,
deep_research=settings.deep_research,
gongwen=settings.gongwen,
light_app_management=settings.light_app_management,
light_app_list=settings.light_app_list,
agents=settings.agents,
business_chains=settings.business_chains,
skills=settings.skills,
knowledge_management=settings.knowledge_management,
)
@router.get("/page-descriptions", response_model=PageDescriptionsResponse)
async def get_page_descriptions() -> PageDescriptionsResponse:
return _page_descriptions_response(load_system_settings().page_descriptions)
@router.put("/page-descriptions", response_model=PageDescriptionsResponse)
async def update_page_descriptions(
body: PageDescriptionsUpdateRequest,
request: Request,
) -> PageDescriptionsResponse:
await _require_admin(request)
current = load_system_settings()
updated = PageDescriptionsSettings(
general_qa=body.general_qa or current.page_descriptions.general_qa,
knowledge_base=body.knowledge_base or current.page_descriptions.knowledge_base,
ai_writing=body.ai_writing or current.page_descriptions.ai_writing,
strategy_projects=body.strategy_projects or current.page_descriptions.strategy_projects,
deep_research=body.deep_research or current.page_descriptions.deep_research,
gongwen=body.gongwen or current.page_descriptions.gongwen,
light_app_management=(
body.light_app_management or current.page_descriptions.light_app_management
),
light_app_list=body.light_app_list or current.page_descriptions.light_app_list,
agents=body.agents or current.page_descriptions.agents,
business_chains=body.business_chains or current.page_descriptions.business_chains,
skills=body.skills or current.page_descriptions.skills,
knowledge_management=(
body.knowledge_management or current.page_descriptions.knowledge_management
),
)
saved = save_system_settings(
current.model_copy(update={"page_descriptions": updated}),
)
return _page_descriptions_response(saved.page_descriptions)
@router.get("/ordinary-knowledge-landing", response_model=OrdinaryKnowledgeLanding)
async def get_ordinary_knowledge_landing() -> OrdinaryKnowledgeLanding:
return OrdinaryKnowledgeLanding(mode=load_system_settings().ordinary_knowledge_landing)
@@ -491,6 +582,14 @@ class SkillCompressionUpdateRequest(BaseModel):
keep_index_in_prompt: bool | None = None
class SkillVersioningResponse(BaseModel):
enabled: bool = Field(..., description="Whether custom-skill version history is active")
class SkillVersioningUpdateRequest(BaseModel):
enabled: bool
class ModelConcurrencyResponse(BaseModel):
enabled: bool = Field(..., description="Whether per-model concurrency limiting is active")
default_limit: int = Field(..., description="Fallback max concurrent runs for models without an explicit override")
@@ -532,6 +631,17 @@ class WriteFileCompatibilityUpdateRequest(BaseModel):
enabled: bool
class CallMetricsResponse(BaseModel):
enabled: bool = Field(
default=True,
description="Whether new tool and skill calls are recorded for management statistics",
)
class CallMetricsUpdateRequest(BaseModel):
enabled: bool
class RetryKeepPartialResponse(BaseModel):
enabled: bool = Field(
default=True,
@@ -579,6 +689,7 @@ class RoundtableRoomLimitResponse(BaseModel):
enabled: bool = True
max_rooms: int = 3
max_hold_minutes: int = 30
lease_minutes: int = 5
max_cycles: int = 8
write_file_compatibility_enabled: bool = False
@@ -587,6 +698,7 @@ class RoundtableRoomLimitUpdateRequest(BaseModel):
enabled: bool | None = None
max_rooms: int | None = Field(default=None, ge=1, le=20)
max_hold_minutes: int | None = Field(default=None, ge=1, le=1440)
lease_minutes: int | None = Field(default=None, ge=1, le=60)
max_cycles: int | None = Field(default=None, ge=2, le=100)
write_file_compatibility_enabled: bool | None = None
@@ -667,6 +779,32 @@ async def update_write_file_compatibility_setting(
)
@router.get("/call-metrics", response_model=CallMetricsResponse)
async def get_call_metrics_setting(request: Request) -> CallMetricsResponse:
"""Return the administrator-controlled tool/skill metrics switch."""
await _require_admin(request)
return CallMetricsResponse(enabled=load_system_settings().call_metrics.enabled)
@router.put("/call-metrics", response_model=CallMetricsResponse)
async def update_call_metrics_setting(
body: CallMetricsUpdateRequest,
request: Request,
) -> CallMetricsResponse:
"""Enable or pause new tool/skill metric writes without a restart."""
await _require_admin(request)
current = load_system_settings()
saved = save_system_settings(
current.model_copy(
update={"call_metrics": CallMetricsSettings(enabled=body.enabled)}
)
)
logger.info("Tool and skill call metrics switch changed: enabled=%s", saved.call_metrics.enabled)
return CallMetricsResponse(enabled=saved.call_metrics.enabled)
@router.get("/retry-keep-partial", response_model=RetryKeepPartialResponse)
async def get_retry_keep_partial_setting() -> RetryKeepPartialResponse:
"""Return the retry-keeps-partial display switch (readable by chat clients)."""
@@ -734,6 +872,7 @@ def _roundtable_room_limit_response(settings: RoundtableRoomLimitSettings) -> Ro
enabled=settings.enabled,
max_rooms=settings.max_rooms,
max_hold_minutes=settings.max_hold_minutes,
lease_minutes=settings.lease_minutes,
max_cycles=settings.max_cycles,
write_file_compatibility_enabled=settings.write_file_compatibility_enabled,
)
@@ -760,6 +899,8 @@ async def update_roundtable_room_limit_setting(
updates["max_rooms"] = body.max_rooms
if body.max_hold_minutes is not None:
updates["max_hold_minutes"] = body.max_hold_minutes
if body.lease_minutes is not None:
updates["lease_minutes"] = body.lease_minutes
if body.max_cycles is not None:
updates["max_cycles"] = body.max_cycles
if body.write_file_compatibility_enabled is not None:
@@ -767,10 +908,11 @@ async def update_roundtable_room_limit_setting(
updated = current.roundtable_room_limit.model_copy(update=updates)
saved = save_system_settings(current.model_copy(update={"roundtable_room_limit": updated}))
logger.info(
"Roundtable runtime limits changed: enabled=%s max_rooms=%s max_hold_minutes=%s max_cycles=%s write_file_compatibility=%s",
"Roundtable runtime limits changed: enabled=%s max_rooms=%s max_hold_minutes=%s lease_minutes=%s max_cycles=%s write_file_compatibility=%s",
saved.roundtable_room_limit.enabled,
saved.roundtable_room_limit.max_rooms,
saved.roundtable_room_limit.max_hold_minutes,
saved.roundtable_room_limit.lease_minutes,
saved.roundtable_room_limit.max_cycles,
saved.roundtable_room_limit.write_file_compatibility_enabled,
)
@@ -917,6 +1059,42 @@ async def update_skill_compression(body: SkillCompressionUpdateRequest, request:
return _skill_compression_to_response(saved.skill_compression)
@router.get("/skill-versioning", response_model=SkillVersioningResponse)
async def get_skill_versioning() -> SkillVersioningResponse:
"""Return the custom-skill versioning gate for skill detail clients."""
return SkillVersioningResponse(enabled=load_system_settings().skill_versioning.enabled)
@router.put("/skill-versioning", response_model=SkillVersioningResponse)
async def update_skill_versioning(
body: SkillVersioningUpdateRequest,
request: Request,
) -> SkillVersioningResponse:
"""Enable/disable custom-skill version capture and switching. Admin only."""
await _require_admin(request)
current = load_system_settings()
previous = current.skill_versioning.enabled
updated = SkillVersioningSettings(enabled=body.enabled)
saved = save_system_settings(current.model_copy(update={"skill_versioning": updated}))
if body.enabled and not previous:
try:
from deerflow.skills.versioning import reconcile_custom_skill_versions
actor = await get_optional_user_from_request(request)
actor_id = str(getattr(actor, "id", "") or "") or None
await asyncio.to_thread(reconcile_custom_skill_versions, author=actor_id)
except Exception:
logger.exception("Failed to capture custom-skill baselines after enabling versioning")
# The switch remains enabled. Individual skills are bootstrapped
# lazily when their version list is first opened or next changed.
logger.info("Custom skill versioning switch changed: enabled=%s", saved.skill_versioning.enabled)
return SkillVersioningResponse(enabled=saved.skill_versioning.enabled)
@router.get("/model-concurrency", response_model=ModelConcurrencyResponse)
async def get_model_concurrency(request: Request) -> ModelConcurrencyResponse:
"""Return the per-model concurrency-limit configuration. Admin only."""
@@ -19,6 +19,7 @@ from fastapi.responses import FileResponse
from pydantic import BaseModel, Field
from app.gateway.deps import get_checkpointer, get_thread_share_store, get_thread_store
from app.gateway.routers.artifacts import build_html_offline_bundle_response
from app.gateway.thread_copy import copy_thread_state
from deerflow.config.paths import get_paths
from deerflow.persistence.thread_meta.base import ThreadMetaStore
@@ -316,6 +317,7 @@ def _public_share_disposition(mime_type: str) -> str:
async def get_public_share_file(
share_code: str,
path: str,
offline_bundle: bool = False,
share_store: ThreadShareStore = Depends(get_thread_share_store),
) -> FileResponse:
"""Serve a file referenced by a view-shared conversation.
@@ -342,6 +344,11 @@ async def get_public_share_file(
if not actual_path.exists() or not actual_path.is_file():
raise HTTPException(status_code=404, detail="File not found")
if offline_bundle:
if actual_path.suffix.lower() not in {".htm", ".html", ".xhtml"}:
raise HTTPException(status_code=400, detail="Offline bundles are available for HTML files only")
return await build_html_offline_bundle_response(actual_path)
mime_type = mimetypes.guess_type(actual_path.name)[0] or "application/octet-stream"
return FileResponse(
actual_path,
@@ -252,6 +252,29 @@ async def refresh_skill_catalog_projection(
return len(catalog)
async def set_skill_catalog_enabled(name: str, enabled: bool) -> bool:
"""Persist one skill's online flag in the catalog row the list API reads.
The extensions file is the runtime source of truth. The skills page reads
``skills.enabled`` instead, so a toggle that only rewrites the file looks
unchanged after the list refreshes. Updating this single row avoids a full
catalog rebuild, which rewrites every skill and can roll the flag back if
an unrelated row fails.
"""
session_factory = get_session_factory()
if session_factory is None or not name:
return False
async with session_factory() as session:
result = await session.execute(
update(SkillRow)
.where(SkillRow.name == name)
.values(enabled=bool(enabled), updated_at=datetime.now(UTC))
)
await session.commit()
return int(result.rowcount or 0) > 0
def _build_catalog_payload(skills: list[Any], usage_entries: dict[str, Any]) -> list[dict[str, Any]]:
payload: list[dict[str, Any]] = []
defaults = {
+7 -3
View File
@@ -2,9 +2,10 @@
config_version: 8
log_level: info
# 高并发运行时优化。技能视图两项不写也默认开启,要关掉才写成 false。修改后需重启后台。
# 高并发运行时优化。工具/技能统计与技能视图默认开启;若确有写库压力才改成 false。修改后需重启后台。
runtime_performance:
tool_metrics_enabled: false # 暂停工具/技能统计写库;不影响问答与工具执行。恢复统计时设 true 并重启
tool_metrics_enabled: true # 记录普通工具调用,供综合管理的工具排行/调用统计使用
skill_metrics_enabled: true # 记录可归因到具体技能的调用,供综合管理的技能排行/调用统计使用
skill_view_reuse_enabled: true # 默认开启;false 才关闭。单次工具调用复用一次技能视图
skill_view_cache_enabled: true # 默认开启;false 才关闭。校验版本后复用未变化的技能视图
model_timing_nonblocking_enabled: true # 模型计时日志独立队列,不争抢工具线程池
@@ -661,7 +662,10 @@ ai_writing:
# admin_password: "$WEKNORA_ADMIN_PASSWORD"
llmwiki:
weknora:
api_base_url: "http://119.254.86.251:8088/deerflow-api"
# The restored WeKnora deployment exposes its API at the site root.
# The historical /deerflow-api prefix is a frontend-only route there and
# turns API mutations (POST/OPTIONS) into nginx 405 responses.
api_base_url: "http://119.254.86.251:8088"
web_base_url: "http://119.254.86.251:8088"
admin_email: "lqq@qq.com"
admin_password: "1qaz@WSX"
@@ -217,9 +217,9 @@ or_(
---
---
## 十、已知限制
## 十、实现状态与已知限制
1. **`use_count` 永不增长**:`bump_use()` 已定义但全后端无调用点。"使用一个技能"的语义模糊(技能是 prompt 知识,不是工具),尚未接线。`view_count`(`skill_view` 触发)与 `patch_count`(回滚触发)是正常工作的。
1. **`use_count` 按问答轮次统计**:成功 `skill_view`、读取 `/mnt/skills/{public,custom}/<name>/...` 文件或通过 `bash` 执行该目录脚本,都视为真实调用;同一 run 内同一技能只加一次,不依赖模型自述。计数同步写入 `.usage.json` 与技能列表数据库投影。
2. **`.usage.json` 全局单文件**:不按用户分文件;因技能名全局唯一,一个技能只有一份 state/计数,这是设计取舍。
3. **enabled 开关**:`extensions_config.json` 的技能 `enabled` 是全局的(非 per-user),前端开关已移除,归档功能覆盖了"对 Agent 隐藏技能"的需求。
4. **历史物理归档**:早期 curator 曾把技能目录物理移到 `custom/.archive/`,这些遗留目录不会出现在新的归档抽屉里(列表 API 扫不到);新流程已统一为软归档。
@@ -42,6 +42,15 @@
},
"intent-clarification-policy-copy": {
"enabled": true
},
"multi-seat-delivery-gating": {
"enabled": true
},
"stage-handoff-chat-delivery": {
"enabled": true
},
"leaflet-tile-zoom-guard": {
"enabled": true
}
}
}
@@ -264,17 +264,17 @@ def _get_enabled_skills():
def _get_enabled_skills_for_config(app_config: AppConfig | None = None) -> list[Skill]:
"""Return enabled skills using the caller's config source.
Resolve storage from the caller's config and reuse the parsed catalogue only
when its root matches. A cold cache or a different root falls back to that
storage, keeping request-scoped skill directories isolated.
Resolve storage from the caller's config and reuse the parsed catalogue only
when its root matches. A cold cache or a different root falls back to that
storage, keeping request-scoped skill directories isolated.
"""
if app_config is None:
return _get_enabled_skills()
storage = get_or_new_skill_storage(app_config=app_config)
warm = get_warm_enabled_skills(storage)
if warm is not None:
return warm
return _apply_always_on(_drop_archived_skills(list(storage.load_skills(enabled_only=True))))
storage = get_or_new_skill_storage(app_config=app_config)
warm = get_warm_enabled_skills(storage)
if warm is not None:
return warm
return _apply_always_on(_drop_archived_skills(list(storage.load_skills(enabled_only=True))))
def _skill_mutability_label(category: SkillCategory | str) -> str:
@@ -617,44 +617,44 @@ task(description="Oracle Cloud analysis", prompt="...", subagent_type="general-p
</subagent_system>"""
_ORDINARY_QA_HEADING_RULES = {
"hybrid_decimal": """- 一级章节使用 `## 一、标题`,二级章节使用 `### 1.1 标题`,三级章节使用 `#### 1.1.1 标题`。
- 二级章节按 `1.1`、`1.2`、`2.1` 的形式编号,三级及更深层级继续使用完整多级阿拉伯数字。
- 确需更深层级时,四级章节使用 `##### 1.1.1.1 标题`,五级章节使用 `###### 1.1.1.1.1 标题`。""",
"official_chinese": """- 一级章节使用 `## 一、标题`,二级章节使用 `### (一) 标题`,三级章节使用 `#### 1. 标题`。
- 确需更深层级时,四级章节使用 `##### (1) 标题`,五级章节使用 `###### 1) 标题`。""",
"decimal": """- 一级章节使用 `## 1 标题`,二级章节使用 `### 1.1 标题`,三级章节使用 `#### 1.1.1 标题`。
- 确需更深层级时,继续使用 `##### 1.1.1.1 标题`、`###### 1.1.1.1.1 标题`。""",
"chapter_section": """- 一级章节使用 `## 第一章 标题`,二级章节使用 `### 第一节 标题`,三级章节使用 `#### 一、标题`。
- 确需更深层级时,四级章节使用 `##### (一) 标题`,五级章节使用 `###### 1. 标题`。""",
"none": """- 各级章节只写标题文字,不手写编号:`## 一级标题`、`### 二级标题`、`#### 三级标题`。
- 确需更深层级时使用 `##### 四级标题`、`###### 五级标题`,仍不添加编号。""",
}
def _word_heading_numbering_style() -> str:
"""Return the configured Word heading-numbering preset with a safe default."""
try:
return load_system_settings().prompt_prefix.word_heading_numbering_style
except Exception:
logger.debug("Failed to read the Word heading-numbering style; using default", exc_info=True)
return "hybrid_decimal"
def _build_ordinary_qa_markdown_format_section() -> str:
rules = _ORDINARY_QA_HEADING_RULES.get(
_word_heading_numbering_style(),
_ORDINARY_QA_HEADING_RULES["hybrid_decimal"],
)
return f"""
<ordinary_qa_markdown_format>
当普通问答需要输出带章节结构的较长内容时,必须采用以下 Markdown 规范,以便下载为 Word 后得到正确的自动编号和公文标题格式:
- 文档主标题使用 `# 主标题`;主标题只写标题文字,不添加章节编号。
{rules}
- 默认禁止使用以 `-`、`*`、`+` 或 `•` 开头的无序圆点列表;并列内容优先改写为完整自然段,确需逐项表达时使用有序列表。只有用户明确要求无序列表时才使用。
- 简短问答不必为了套用格式而强行添加标题;本规范只约束实际使用的标题和列表。
</ordinary_qa_markdown_format>
"""
_ORDINARY_QA_HEADING_RULES = {
"hybrid_decimal": """- 一级章节使用 `## 一、标题`,二级章节使用 `### 1.1 标题`,三级章节使用 `#### 1.1.1 标题`。
- 二级章节按 `1.1`、`1.2`、`2.1` 的形式编号,三级及更深层级继续使用完整多级阿拉伯数字。
- 确需更深层级时,四级章节使用 `##### 1.1.1.1 标题`,五级章节使用 `###### 1.1.1.1.1 标题`。""",
"official_chinese": """- 一级章节使用 `## 一、标题`,二级章节使用 `### (一) 标题`,三级章节使用 `#### 1. 标题`。
- 确需更深层级时,四级章节使用 `##### (1) 标题`,五级章节使用 `###### 1) 标题`。""",
"decimal": """- 一级章节使用 `## 1 标题`,二级章节使用 `### 1.1 标题`,三级章节使用 `#### 1.1.1 标题`。
- 确需更深层级时,继续使用 `##### 1.1.1.1 标题`、`###### 1.1.1.1.1 标题`。""",
"chapter_section": """- 一级章节使用 `## 第一章 标题`,二级章节使用 `### 第一节 标题`,三级章节使用 `#### 一、标题`。
- 确需更深层级时,四级章节使用 `##### (一) 标题`,五级章节使用 `###### 1. 标题`。""",
"none": """- 各级章节只写标题文字,不手写编号:`## 一级标题`、`### 二级标题`、`#### 三级标题`。
- 确需更深层级时使用 `##### 四级标题`、`###### 五级标题`,仍不添加编号。""",
}
def _word_heading_numbering_style() -> str:
"""Return the configured Word heading-numbering preset with a safe default."""
try:
return load_system_settings().prompt_prefix.word_heading_numbering_style
except Exception:
logger.debug("Failed to read the Word heading-numbering style; using default", exc_info=True)
return "hybrid_decimal"
def _build_ordinary_qa_markdown_format_section() -> str:
rules = _ORDINARY_QA_HEADING_RULES.get(
_word_heading_numbering_style(),
_ORDINARY_QA_HEADING_RULES["hybrid_decimal"],
)
return f"""
<ordinary_qa_markdown_format>
当普通问答需要输出带章节结构的较长内容时,必须采用以下 Markdown 规范,以便下载为 Word 后得到正确的自动编号和公文标题格式:
- 文档主标题使用 `# 主标题`;主标题只写标题文字,不添加章节编号。
{rules}
- 默认禁止使用以 `-`、`*`、`+` 或 `•` 开头的无序圆点列表;并列内容优先改写为完整自然段,确需逐项表达时使用有序列表。只有用户明确要求无序列表时才使用。
- 简短问答不必为了套用格式而强行添加标题;本规范只约束实际使用的标题和列表。
</ordinary_qa_markdown_format>
"""
_ORDINARY_QA_VISUALIZATION_SECTION = """
@@ -1261,13 +1261,13 @@ RAG/reference display mode is enabled.
return prompt
def _get_memory_context(
agent_name: str | None = None,
*,
app_config: AppConfig | None = None,
injection_enabled: bool = True,
include_legacy_knowledge_routing: bool = True,
) -> str:
def _get_memory_context(
agent_name: str | None = None,
*,
app_config: AppConfig | None = None,
injection_enabled: bool = True,
include_legacy_knowledge_routing: bool = True,
) -> str:
"""组装注入系统提示词的记忆上下文 (Memory V2)。
通过 :class:`MemoryManager` 收集各 provider 的记忆块。P1 期只有 builtin
@@ -1278,10 +1278,10 @@ def _get_memory_context(
agent_name: agent 标识;``None`` 时按 "default" 处理。
app_config: 显式应用配置 (本函数实际按 per-user 覆盖解析生效配置)。
injection_enabled: 运行时开关;``False`` 时跳过注入直接返回空串。
include_legacy_knowledge_routing: 是否注入 ``/mnt/knowledge/routing.md``。
智能普通知识库检索已有数据库授权范围和专用检索工具时必须关闭,
避免旧文件路由把“这个库”错误引向沙箱目录;其它记忆 provider
仍按原配置保留。
include_legacy_knowledge_routing: 是否注入 ``/mnt/knowledge/routing.md``。
智能普通知识库检索已有数据库授权范围和专用检索工具时必须关闭,
避免旧文件路由把“这个库”错误引向沙箱目录;其它记忆 provider
仍按原配置保留。
Returns:
包在 ``<memory>`` 标签里的记忆上下文;未启用或为空时返回空串。
@@ -1306,11 +1306,11 @@ def _get_memory_context(
)
# 只取静态块 (builtin 的 frozen snapshot + Hindsight 的模式说明)。
# Hindsight 的动态召回需要当前 query,走 MemoryMiddleware.before_model。
context_content = manager.build_system_prompt(
excluded_provider_names=(
None if include_legacy_knowledge_routing else {"knowledge"}
)
)
context_content = manager.build_system_prompt(
excluded_provider_names=(
None if include_legacy_knowledge_routing else {"knowledge"}
)
)
if not context_content.strip():
return ""
@@ -1878,7 +1878,7 @@ def apply_prompt_template(
is_bootstrap: bool = False,
app_config: AppConfig | None = None,
memory_injection_enabled: bool = True,
include_legacy_knowledge_routing: bool = True,
include_legacy_knowledge_routing: bool = True,
rag_mode_enabled: bool = False,
is_notebook_mode: bool = False,
notebook_source_ids: list[int] | None = None,
@@ -1887,12 +1887,12 @@ def apply_prompt_template(
storage_agent_id = agent_id if agent_id is not None else agent_name
# Get memory context
memory_context = _get_memory_context(
storage_agent_id,
app_config=app_config,
injection_enabled=memory_injection_enabled,
include_legacy_knowledge_routing=include_legacy_knowledge_routing,
)
memory_context = _get_memory_context(
storage_agent_id,
app_config=app_config,
injection_enabled=memory_injection_enabled,
include_legacy_knowledge_routing=include_legacy_knowledge_routing,
)
# Include subagent section only if enabled (from runtime parameter)
n = max_concurrent_subagents
@@ -2004,8 +2004,8 @@ When a search, knowledge, skill, or MCP tool returns a result list:
if rag_mode_enabled
else ""
)
ordinary_qa_format_section = (
_build_ordinary_qa_markdown_format_section()
ordinary_qa_format_section = (
_build_ordinary_qa_markdown_format_section()
if _ordinary_qa_markdown_format_enabled()
and not is_bootstrap
and not writing_mode
@@ -136,13 +136,18 @@ def _build_runtime_middlewares(
# wrap_tool_call chain: its handler is the raw tool, so it observes the
# unconverted exception (before ToolErrorHandlingMiddleware turns it into
# an error ToolMessage) and the raw return value.
# Omit the recorder entirely while paused: no record allocation, skill-merge
# lookup, or fire-and-forget DB task is created by any tool invocation.
# Keep tool execution/error handling and LLM metrics independent of this flag.
if app_config.runtime_performance.tool_metrics_enabled:
from deerflow.agents.middlewares.tool_metrics_middleware import ToolMetricsMiddleware
middlewares.append(ToolMetricsMiddleware())
# Full auditing can stay paused under high concurrency while the low-volume
# skill-only recorder remains active. In skill-only mode non-skill calls are
# passed through without a merge query or fire-and-forget DB write.
performance = app_config.runtime_performance
if performance.tool_metrics_enabled or performance.skill_metrics_enabled:
from deerflow.agents.middlewares.tool_metrics_middleware import ToolMetricsMiddleware
middlewares.append(
ToolMetricsMiddleware(
skill_only=not performance.tool_metrics_enabled,
)
)
return middlewares
@@ -179,20 +184,20 @@ def build_subagent_runtime_middlewares(
model_name = app_config.models[0].name
model_config = app_config.get_model_config(model_name) if model_name else None
if model_config is not None and model_config.supports_vision:
from deerflow.agents.middlewares.view_image_middleware import ViewImageMiddleware
middlewares.append(ViewImageMiddleware())
# Must remain the innermost model-call wrapper: subagent-specific context,
# vision payloads, system prompts and tool schemas all need to be included
# in the final input-size decision.
from deerflow.agents.middlewares.context_window_guard_middleware import ContextWindowGuardMiddleware
middlewares.append(
ContextWindowGuardMiddleware(
max_input_tokens=model_config.max_input_tokens if model_config is not None else None,
)
)
return middlewares
if model_config is not None and model_config.supports_vision:
from deerflow.agents.middlewares.view_image_middleware import ViewImageMiddleware
middlewares.append(ViewImageMiddleware())
# Must remain the innermost model-call wrapper: subagent-specific context,
# vision payloads, system prompts and tool schemas all need to be included
# in the final input-size decision.
from deerflow.agents.middlewares.context_window_guard_middleware import ContextWindowGuardMiddleware
middlewares.append(
ContextWindowGuardMiddleware(
max_input_tokens=model_config.max_input_tokens if model_config is not None else None,
)
)
return middlewares
@@ -1,6 +1,7 @@
"""Middleware that records per-tool-call metrics for the admin audit page.
Every tool / skill invocation ends with one row in ``tool_call_metrics``:
While the administrator-controlled call-metrics switch is enabled, every tool
/ skill invocation ends with one row in ``tool_call_metrics``:
- Successes record the wall-clock duration.
- Failures record a coarse ``error_category`` plus a one-line message.
@@ -40,6 +41,7 @@ from langgraph.errors import GraphBubbleUp
from langgraph.prebuilt.tool_node import ToolCallRequest
from langgraph.types import Command
from deerflow.config.system_settings import get_call_metrics_settings
from deerflow.persistence.engine import fire_and_forget_db_write, get_session_factory
from deerflow.persistence.tool_metrics import ToolMetricRecord, make_tool_metrics_store
from deerflow.persistence.tool_metrics.base import ToolMetricsStore
@@ -188,11 +190,18 @@ def _evaluate_result(result: Any, tool_name: str | None) -> tuple[str, str | Non
class ToolMetricsMiddleware(AgentMiddleware[AgentState]):
"""Wrap each tool call to time it and persist a metric row."""
"""Time tool calls, optionally persisting only attributable skill calls."""
def __init__(self, *, store: ToolMetricsStore | None = None, **kwargs: Any) -> None:
def __init__(
self,
*,
store: ToolMetricsStore | None = None,
skill_only: bool = False,
**kwargs: Any,
) -> None:
super().__init__(**kwargs)
self._store_override = store
self._skill_only = skill_only
# -- store / context ----------------------------------------------------
@@ -205,6 +214,12 @@ class ToolMetricsMiddleware(AgentMiddleware[AgentState]):
return None
return make_tool_metrics_store(sf)
@staticmethod
def _metrics_enabled() -> bool:
"""Read the administrator switch at call time so no restart is needed."""
return bool(get_call_metrics_settings().enabled)
@staticmethod
def _runtime_lookup(request: ToolCallRequest, keys: tuple[str, ...]) -> str | None:
runtime = getattr(request, "runtime", None)
@@ -280,7 +295,11 @@ class ToolMetricsMiddleware(AgentMiddleware[AgentState]):
handler: Callable[[ToolCallRequest], ToolMessage | Command],
) -> ToolMessage | Command:
record = self._build_base_record(request)
if is_internal_tool_name(record.tool_name):
if (
not self._metrics_enabled()
or is_internal_tool_name(record.tool_name)
or (self._skill_only and not record.skill_name)
):
return handler(request)
started = time.monotonic()
try:
@@ -303,7 +322,11 @@ class ToolMetricsMiddleware(AgentMiddleware[AgentState]):
handler: Callable[[ToolCallRequest], Awaitable[ToolMessage | Command]],
) -> ToolMessage | Command:
record = self._build_base_record(request)
if is_internal_tool_name(record.tool_name):
if (
not self._metrics_enabled()
or is_internal_tool_name(record.tool_name)
or (self._skill_only and not record.skill_name)
):
return await handler(request)
started = time.monotonic()
try:
@@ -482,13 +482,14 @@ def _load_and_cache_app_config(config_path: str | None = None) -> AppConfig:
)
performance = _app_config.runtime_performance
logger.info(
"RUNTIME_PERFORMANCE skill_view_reuse_enabled=%s skill_view_cache_enabled=%s model_timing_nonblocking_enabled=%s asyncio_thread_pool_enabled=%s asyncio_thread_pool_size=%d tool_metrics_enabled=%s",
"RUNTIME_PERFORMANCE skill_view_reuse_enabled=%s skill_view_cache_enabled=%s model_timing_nonblocking_enabled=%s asyncio_thread_pool_enabled=%s asyncio_thread_pool_size=%d tool_metrics_enabled=%s skill_metrics_enabled=%s",
performance.skill_view_reuse_enabled,
performance.skill_view_cache_enabled,
performance.model_timing_nonblocking_enabled,
performance.asyncio_thread_pool_enabled,
performance.asyncio_thread_pool_size,
performance.tool_metrics_enabled,
performance.skill_metrics_enabled,
)
return _app_config
@@ -2,9 +2,11 @@
Skill-view reuse and cache are on when those keys are absent. Set them to
false in config.yaml to restore a full refresh on every tool call. Model
timing and the Gateway thread pool stay off unless configured. If application
configuration cannot be loaded, every switch stays off. Tool-call metric writes
are paused by default; explicitly enable them to resume audit statistics.
timing and the Gateway thread pool stay off unless configured. Tool and skill
metrics are on by default so the management leaderboard has a complete source
of truth. Deployments that need to shed this best-effort audit write load can
explicitly pause either metric type in ``config.yaml``. If configuration
cannot be loaded, the fallback keeps every switch off.
"""
from pydantic import BaseModel, ConfigDict, Field
@@ -15,7 +17,14 @@ class RuntimePerformanceConfig(BaseModel):
skill_view_reuse_enabled: bool = Field(default=True, description="Reuse one verified skill view within each sandbox tool invocation. Set false to refresh on every call.")
skill_view_cache_enabled: bool = Field(default=True, description="Reuse unchanged skill views after validating source, target and availability versions. Set false to rebuild every call.")
tool_metrics_enabled: bool = Field(default=False, description="Persist tool/skill audit statistics to tool_call_metrics. Disabled by default to avoid competing with interactive database traffic; historical reads remain available.")
tool_metrics_enabled: bool = Field(
default=True,
description="Persist tool/skill audit statistics to tool_call_metrics. Enabled by default so management leaderboard tool rankings receive data; set false only to pause ordinary tool auditing.",
)
skill_metrics_enabled: bool = Field(
default=True,
description="Persist calls that can be attributed to a concrete skill, keeping skill statistics available even if ordinary tool auditing is paused.",
)
model_timing_nonblocking_enabled: bool = Field(default=False, description="Dispatch model timing logs through a dedicated bounded queue, outside the shared tool executor")
asyncio_thread_pool_enabled: bool = Field(default=False, description="Override the Gateway event loop's default executor at startup; disabled preserves Python defaults")
asyncio_thread_pool_size: int = Field(default=100, ge=1, le=1024, strict=True, description="Maximum Gateway executor threads when enabled, unless overridden by the documented environment variables")
@@ -24,7 +33,12 @@ class RuntimePerformanceConfig(BaseModel):
# Field defaults turn the skill-view fast path on. This fallback is only for
# callers that cannot read application configuration, and it keeps every
# switch off.
_DEFAULT = RuntimePerformanceConfig(skill_view_reuse_enabled=False, skill_view_cache_enabled=False)
_DEFAULT = RuntimePerformanceConfig(
skill_view_reuse_enabled=False,
skill_view_cache_enabled=False,
tool_metrics_enabled=False,
skill_metrics_enabled=False,
)
def get_runtime_performance_config() -> RuntimePerformanceConfig:
@@ -21,7 +21,7 @@ from pathlib import Path
from typing import Literal
from uuid import uuid4
from pydantic import BaseModel, ConfigDict, Field, field_validator
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from deerflow.config.llmwiki_config import LlmWikiRuntimeOverride
from deerflow.config.runtime_paths import runtime_home
@@ -474,6 +474,20 @@ class SkillCompressionSettings(BaseModel):
model_config = ConfigDict(extra="ignore")
class SkillVersioningSettings(BaseModel):
"""Deployment-wide gate for custom-skill package version history.
The default is intentionally disabled so existing deployments retain the
legacy in-place editing behaviour until an administrator opts in.
"""
enabled: bool = Field(
default=False,
description="Whether custom-skill package versioning and switching are enabled",
)
model_config = ConfigDict(extra="ignore")
class AgentModuleSettings(BaseModel):
"""Admin-configured「按类型聚合」module → selected agent-tag ids.
@@ -668,6 +682,24 @@ class WriteFileCompatibilitySettings(BaseModel):
model_config = ConfigDict(extra="ignore")
class CallMetricsSettings(BaseModel):
"""Administrator-controlled recording for tool and skill call statistics.
Both rankings use the same ``tool_call_metrics`` audit stream. Keeping a
single switch prevents a partially enabled configuration where the
management page can show one ranking but not the other.
"""
enabled: bool = Field(
default=True,
description=(
"Whether new tool and skill calls are persisted for management "
"statistics. Existing records are retained when disabled."
),
)
model_config = ConfigDict(extra="ignore")
class RetryKeepPartialSettings(BaseModel):
"""Keep already-streamed partial answers visible across silent LLM retries.
@@ -717,6 +749,15 @@ class RoundtableRoomLimitSettings(BaseModel):
"but never extend this deadline."
),
)
lease_minutes: int = Field(
default=5,
ge=1,
le=60,
description=(
"How long a foreground step-2 lease stays occupied without a renewal. "
"The browser heartbeat renews it; this does not extend max_hold_minutes."
),
)
max_cycles: int = Field(
default=8,
ge=2,
@@ -769,7 +810,40 @@ class HtmlMapSettings(BaseModel):
model_config = ConfigDict(extra="ignore")
_GENERATED_PAGE_LOGO_FILE_RE = re.compile(r"^logo-[a-f0-9]{32}\.(?:png|jpg|webp)$")
class GeneratedPageBrandingSettings(BaseModel):
"""Optional generated-page logo settings retained for offline bundles."""
chrome_enabled: bool = True
header_layout: Literal["left", "center"] = "center"
system_name: str | None = Field(default=None, max_length=80)
footer_text: str | None = Field(default=None, max_length=160)
logo_filename: str | None = None
logo_mode: Literal["custom", "default", "none"] = "custom"
logo_media_type: Literal["image/png", "image/jpeg", "image/webp"] | None = None
logo_version: str | None = Field(default=None, max_length=64)
model_config = ConfigDict(extra="ignore")
@field_validator("logo_filename")
@classmethod
def _validate_logo_filename(cls, value: str | None) -> str | None:
if value is None:
return None
normalized = str(value).strip().lower()
if not _GENERATED_PAGE_LOGO_FILE_RE.fullmatch(normalized):
raise ValueError("Invalid generated-page logo filename")
return normalized
_APP_IDENTITY_LOGO_FILE_RE = re.compile(r"^logo-[a-f0-9]{32}\.(?:png|jpg|webp)$")
_APP_IDENTITY_LIGHT_RED_HEADER_FILE_RE = re.compile(
r"^light-red-header-[a-f0-9]{32}\.(?:png|jpg|webp|svg)$"
)
_APP_IDENTITY_LIGHT_RED_ACTIVE_FILE_RE = re.compile(
r"^light-red-active-[a-f0-9]{32}\.(?:png|jpg|webp|svg)$"
)
class AppIdentitySettings(BaseModel):
@@ -780,9 +854,21 @@ class AppIdentitySettings(BaseModel):
"""
system_name: str = Field(default="", max_length=80)
header_title_font_mode: Literal["adaptive", "fixed"] = "adaptive"
header_title_font_size: int = Field(default=30, ge=16, le=40)
logo_filename: str | None = None
logo_media_type: Literal["image/png", "image/jpeg", "image/webp"] | None = None
logo_version: str | None = None
light_red_header_background_filename: str | None = None
light_red_header_background_media_type: Literal[
"image/png", "image/jpeg", "image/webp", "image/svg+xml"
] | None = None
light_red_header_background_version: str | None = None
light_red_nav_active_filename: str | None = None
light_red_nav_active_media_type: Literal[
"image/png", "image/jpeg", "image/webp", "image/svg+xml"
] | None = None
light_red_nav_active_version: str | None = None
model_config = ConfigDict(extra="ignore")
@field_validator("system_name")
@@ -801,6 +887,26 @@ class AppIdentitySettings(BaseModel):
raise ValueError("Invalid application logo filename")
return normalized
@field_validator("light_red_header_background_filename")
@classmethod
def _validate_light_red_header_filename(cls, value: str | None) -> str | None:
if value is None:
return None
normalized = str(value).strip().lower()
if not _APP_IDENTITY_LIGHT_RED_HEADER_FILE_RE.fullmatch(normalized):
raise ValueError("Invalid light-red header background filename")
return normalized
@field_validator("light_red_nav_active_filename")
@classmethod
def _validate_light_red_active_filename(cls, value: str | None) -> str | None:
if value is None:
return None
normalized = str(value).strip().lower()
if not _APP_IDENTITY_LIGHT_RED_ACTIVE_FILE_RE.fullmatch(normalized):
raise ValueError("Invalid light-red active navigation filename")
return normalized
def app_identity_logo_path(
settings: AppIdentitySettings | None = None,
@@ -814,6 +920,42 @@ def app_identity_logo_path(
return path if path.is_file() else None
def app_identity_light_red_header_background_path(
settings: AppIdentitySettings | None = None,
) -> Path | None:
"""Resolve the uploaded light-red header background inside runtime storage."""
current = settings or load_system_settings().app_identity
if not current.light_red_header_background_filename:
return None
path = runtime_home() / "app-identity" / current.light_red_header_background_filename
return path if path.is_file() else None
def app_identity_light_red_nav_active_path(
settings: AppIdentitySettings | None = None,
) -> Path | None:
"""Resolve the uploaded light-red active-navigation image inside runtime storage."""
current = settings or load_system_settings().app_identity
if not current.light_red_nav_active_filename:
return None
path = runtime_home() / "app-identity" / current.light_red_nav_active_filename
return path if path.is_file() else None
def generated_page_logo_path(
settings: GeneratedPageBrandingSettings | None = None,
) -> Path | None:
"""Resolve the custom generated-page logo without accepting arbitrary paths."""
current = settings or load_system_settings().generated_page_branding
if current.logo_mode != "custom" or not current.logo_filename:
return None
path = runtime_home() / "generated-page-branding" / current.logo_filename
return path if path.is_file() else None
class HtmlRuntimeSettings(BaseModel):
"""Addresses written into newly generated HTML pages.
@@ -844,6 +986,108 @@ class StrategyProjectSettings(BaseModel):
model_config = ConfigDict(extra="ignore")
class GeneralQaPageDescription(BaseModel):
"""Administrator-provided copy for the ordinary Q&A landing page."""
welcome_title: str = Field(default="", max_length=80)
welcome_description: str = Field(default="", max_length=500)
input_placeholder: str = Field(default="", max_length=120)
model_config = ConfigDict(extra="ignore")
@field_validator("welcome_title", "welcome_description", "input_placeholder")
@classmethod
def _normalize_copy(cls, value: str | None) -> str:
return str(value or "").strip()
class GeneralQaPageDescriptions(BaseModel):
"""Separate ordinary-Q&A copy for the full and compact layouts."""
full: GeneralQaPageDescription = Field(default_factory=GeneralQaPageDescription)
compact: GeneralQaPageDescription = Field(default_factory=GeneralQaPageDescription)
model_config = ConfigDict(extra="ignore")
@model_validator(mode="before")
@classmethod
def _migrate_legacy_single_description(cls, value: object) -> object:
if isinstance(value, GeneralQaPageDescription):
return {"full": value, "compact": value}
if not isinstance(value, dict):
return value
if "full" in value or "compact" in value:
return value
legacy = {
key: value.get(key, "")
for key in ("welcome_title", "welcome_description", "input_placeholder")
}
return {"full": legacy, "compact": legacy}
class PageHeaderDescription(BaseModel):
"""Configurable title and explanatory copy for a product landing page."""
title: str = Field(default="", max_length=80)
description: str = Field(default="", max_length=500)
model_config = ConfigDict(extra="ignore")
@field_validator("title", "description")
@classmethod
def _normalize_copy(cls, value: str | None) -> str:
return str(value or "").strip()
class AIWritingGuideCard(BaseModel):
"""Editable text for one of the fixed AI-writing welcome cards."""
title: str = Field(default="", max_length=80)
description: str = Field(default="", max_length=240)
model_config = ConfigDict(extra="ignore")
@field_validator("title", "description")
@classmethod
def _normalize_copy(cls, value: str | None) -> str:
return str(value or "").strip()
def _empty_ai_writing_guide_cards() -> list[AIWritingGuideCard]:
return [AIWritingGuideCard() for _ in range(3)]
class AIWritingPageDescription(PageHeaderDescription):
"""Configurable copy for the classic AI-writing welcome page."""
guide_cards: list[AIWritingGuideCard] = Field(
default_factory=_empty_ai_writing_guide_cards,
max_length=3,
)
@model_validator(mode="before")
@classmethod
def _keep_three_guide_cards(cls, value: object) -> object:
if not isinstance(value, dict) or not isinstance(value.get("guide_cards"), list):
return value
cards = value["guide_cards"][:3]
return {**value, "guide_cards": [*cards, *({} for _ in range(3 - len(cards)))]}
class PageDescriptionsSettings(BaseModel):
"""Page-scoped, administrator-managed display copy."""
general_qa: GeneralQaPageDescriptions = Field(default_factory=GeneralQaPageDescriptions)
knowledge_base: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
ai_writing: AIWritingPageDescription = Field(default_factory=AIWritingPageDescription)
strategy_projects: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
deep_research: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
gongwen: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
light_app_management: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
light_app_list: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
agents: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
business_chains: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
skills: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
knowledge_management: PageHeaderDescription = Field(default_factory=PageHeaderDescription)
model_config = ConfigDict(extra="ignore")
class ModelPromptSuffix(BaseModel):
"""Administrator-owned instructions appended to one configured model's system prompt."""
@@ -862,9 +1106,14 @@ class SystemSettings(BaseModel):
private_square: PrivateSquareSettings = Field(default_factory=PrivateSquareSettings)
publish_squares: PublishSquaresSettings = Field(default_factory=PublishSquaresSettings)
appearance: AppearanceSettings = Field(default_factory=AppearanceSettings)
generated_page_branding: GeneratedPageBrandingSettings = Field(
default_factory=GeneratedPageBrandingSettings
)
page_descriptions: PageDescriptionsSettings = Field(default_factory=PageDescriptionsSettings)
citation_display: CitationDisplaySettings = Field(default_factory=CitationDisplaySettings)
leaderboard: LeaderboardSettings = Field(default_factory=LeaderboardSettings)
skill_compression: SkillCompressionSettings = Field(default_factory=SkillCompressionSettings)
skill_versioning: SkillVersioningSettings = Field(default_factory=SkillVersioningSettings)
plugin_management: PluginManagementSettings = Field(default_factory=PluginManagementSettings)
agent_modules: AgentModuleSettings = Field(default_factory=AgentModuleSettings)
wx_common_agents: WxCommonAgentsSettings = Field(default_factory=WxCommonAgentsSettings)
@@ -875,6 +1124,7 @@ class SystemSettings(BaseModel):
write_file_compatibility: WriteFileCompatibilitySettings = Field(
default_factory=WriteFileCompatibilitySettings
)
call_metrics: CallMetricsSettings = Field(default_factory=CallMetricsSettings)
retry_keep_partial: RetryKeepPartialSettings = Field(
default_factory=RetryKeepPartialSettings
)
@@ -1007,6 +1257,19 @@ def get_write_file_compatibility_settings() -> WriteFileCompatibilitySettings:
return WriteFileCompatibilitySettings()
def get_call_metrics_settings() -> CallMetricsSettings:
"""Return the live call-metrics switch, defaulting to recording enabled."""
try:
return load_system_settings().call_metrics
except Exception:
logger.debug(
"Failed to read call metrics setting; keeping metrics enabled",
exc_info=True,
)
return CallMetricsSettings()
def get_roundtable_room_limit_settings() -> RoundtableRoomLimitSettings:
"""Return the admin-controlled roundtable room cap, defaulting to 3 enabled rooms."""
try:
@@ -0,0 +1,321 @@
"""Build portable ZIP bundles for generated HTML artifacts.
Generated pages intentionally reference the Gateway's vendored browser runtime
instead of public CDNs. A plain ``.html`` download therefore stops working
after it is moved away from the DeerFlow deployment. This module rewrites
those trusted Gateway URLs to relative paths and copies the corresponding
packaged assets into a ZIP. It never downloads arbitrary third-party URLs.
"""
from __future__ import annotations
import hashlib
import json
import re
import zipfile
from dataclasses import dataclass
from importlib.resources import files
from pathlib import Path, PurePosixPath
from deerflow.html_runtime.public_images import resolve_html_image_path, verify_html_image_token
_STATIC_ASSETS = {
"alpine.min.js",
"bootstrap.bundle.min.js",
"bootstrap.min.css",
"brand-logo-dark.png",
"echarts.min.js",
"g6.min.js",
"leaflet.css",
"leaflet.js",
"topbar-dark.svg",
"topbar-light.svg",
}
_GEO_MAP_ASSETS = {
"china-provinces.geojson",
"japan-prefectures.geojson",
"taiwan-counties.geojson",
"us-states.geojson",
}
_COGNITIVE_ASSET_SUFFIXES = {
".css",
".js",
".json",
".jpeg",
".jpg",
".mjs",
".png",
".svg",
".woff2",
}
_LICENSE_BY_ASSET = {
"alpine.min.js": "LICENSE.alpine.txt",
"bootstrap.bundle.min.js": "LICENSE.bootstrap.txt",
"bootstrap.min.css": "LICENSE.bootstrap.txt",
"echarts.min.js": "LICENSE.echarts.txt",
"g6.min.js": "LICENSE.g6.txt",
"leaflet.css": "LICENSE.leaflet.txt",
"leaflet.js": "LICENSE.leaflet.txt",
}
_RUNTIME_ARCHIVE_ROOT = PurePosixPath("assets/html-runtime")
_HTML_IMAGES_ARCHIVE_ROOT = PurePosixPath("assets/html-images")
_STANDARD_RUNTIME_BASE_RE = (
r"(?:https?://[^/\"'\s<>()]+)?"
r"(?:/[A-Za-z0-9._~!$&()*+,;=:@%+-]+)*/api/public/html-runtime"
)
_HTML_IMAGE_URL_RE = re.compile(
r"(?P<url>(?:https?://[^/\"'\s<>()]+)?"
r"(?:/[A-Za-z0-9._~!$&()*+,;=:@%+-]+)*/api/public/html-images/"
r"(?P<token>[^\"'\s<>()?#]+)(?:\?[^\"'\s<>()#]*)?)",
re.IGNORECASE,
)
_EXTERNAL_URL_RE = re.compile(r"https?://[^\s\"'<>]+", re.IGNORECASE)
@dataclass(frozen=True)
class OfflineBundleResult:
"""Summary of one completed offline bundle."""
bundled_assets: tuple[str, ...]
external_urls: tuple[str, ...]
def _runtime_base_pattern(runtime_base_url: str) -> str:
bases = [_STANDARD_RUNTIME_BASE_RE]
configured = str(runtime_base_url or "").strip().rstrip("/")
if configured:
bases.insert(0, re.escape(configured))
return "(?:" + "|".join(bases) + ")"
def _safe_runtime_source(runtime_dir: Path, relative_path: str) -> Path | None:
"""Resolve only assets that the public HTML-runtime router exposes."""
normalized = relative_path.strip().lstrip("/").replace("\\", "/")
pure_path = PurePosixPath(normalized)
if not normalized or pure_path.is_absolute() or ".." in pure_path.parts:
return None
allowed = False
if len(pure_path.parts) == 1 and normalized in _STATIC_ASSETS:
allowed = True
elif len(pure_path.parts) == 2 and pure_path.parts[0] == "maps" and pure_path.name in _GEO_MAP_ASSETS:
allowed = True
elif pure_path.parts and pure_path.parts[0] == "cognitive-situation-map":
allowed = pure_path.suffix.lower() in _COGNITIVE_ASSET_SUFFIXES
if not allowed:
return None
root = runtime_dir.resolve()
source = (root / Path(*pure_path.parts)).resolve()
try:
source.relative_to(root)
except ValueError:
return None
return source if source.is_file() else None
def _add_file(
archive_files: dict[str, Path],
archive_path: PurePosixPath,
source: Path,
) -> None:
archive_files[archive_path.as_posix()] = source
def _add_cognitive_runtime(archive_files: dict[str, Path], runtime_dir: Path) -> None:
source_root = runtime_dir / "cognitive-situation-map"
if not source_root.is_dir():
return
for source in source_root.rglob("*"):
if not source.is_file():
continue
is_license = source.name.lower().startswith("license")
if source.suffix.lower() not in _COGNITIVE_ASSET_SUFFIXES and not is_license:
continue
relative = source.relative_to(runtime_dir)
_add_file(archive_files, _RUNTIME_ARCHIVE_ROOT / PurePosixPath(relative.as_posix()), source)
def _generated_logo_destination(generated_logo_path: Path | None) -> PurePosixPath:
suffix = generated_logo_path.suffix.lower() if generated_logo_path is not None else ".png"
if suffix not in {".jpg", ".jpeg", ".png", ".webp"}:
suffix = ".png"
return _RUNTIME_ARCHIVE_ROOT / f"generated-page-logo{suffix}"
def _runtime_script(
runtime_dir: Path,
*,
map_tile_url: str,
include_geo_maps: bool,
) -> str:
config: dict[str, object] = {"geoMaps": {}, "mapTileUrl": map_tile_url}
if include_geo_maps:
config["geoMaps"] = {
Path(filename).stem: (_RUNTIME_ARCHIVE_ROOT / "maps" / filename).as_posix()
for filename in sorted(_GEO_MAP_ASSETS)
}
prefix = "window.DEERFLOW_HTML_RUNTIME_CONFIG=" + json.dumps(
config,
ensure_ascii=False,
separators=(",", ":"),
) + ";\n"
return prefix + (runtime_dir / "deerflow-runtime.js").read_text(encoding="utf-8")
def _readme(original_name: str, external_urls: list[str]) -> str:
external_note = (
"未打包的外部地址已记录在 manifest.json 的 external_urls 中;例如在线地图瓦片、"
"用户主动引用的第三方图片或接口仍需要网络。"
if external_urls
else "本页没有检测到仍需联网的 http(s) 地址。"
)
return (
"DeerFlow 网页离线包\n"
"====================\n\n"
f"原始文件:{original_name}\n"
"入口文件:index.html\n"
"依赖目录:assets/\n\n"
"部署方式:请保持目录结构不变,将解压后的全部文件一起放到任意静态网站目录。\n"
"若页面含 GeoJSON 地图等 fetch 请求,不建议直接双击 file:// 打开;可使用 Nginx、"
"Apache、IIS 或其他静态 HTTP 服务访问 index.html。\n\n"
f"{external_note}\n"
)
def build_html_offline_bundle(
html_path: Path,
destination: Path,
*,
runtime_base_url: str,
map_tile_url: str,
generated_logo_path: Path | None = None,
runtime_dir: Path | None = None,
) -> OfflineBundleResult:
"""Write a portable HTML ZIP to *destination* and return its manifest summary."""
html_path = Path(html_path)
destination = Path(destination)
runtime_dir = runtime_dir or Path(str(files("deerflow.html_runtime.assets")))
runtime_dir = Path(runtime_dir)
html = html_path.read_text(encoding="utf-8", errors="replace")
archive_files: dict[str, Path] = {}
generated_files: dict[str, str] = {}
bundled_assets: set[str] = set()
licenses: set[str] = set()
include_geo_maps = bool(re.search(r"data-df-geo\s*=", html, re.IGNORECASE))
uses_leaflet_map = bool(re.search(r"data-df-map\s*=", html, re.IGNORECASE))
base_pattern = _runtime_base_pattern(runtime_base_url)
cognitive_base_re = re.compile(
rf"(?P<url>{base_pattern}/cognitive-situation-map)(?=$|[\"'\s<>()/])",
re.IGNORECASE,
)
def replace_cognitive_base(match: re.Match[str]) -> str:
_add_cognitive_runtime(archive_files, runtime_dir)
bundled_assets.add((_RUNTIME_ARCHIVE_ROOT / "cognitive-situation-map").as_posix() + "/")
return (_RUNTIME_ARCHIVE_ROOT / "cognitive-situation-map").as_posix()
html = cognitive_base_re.sub(replace_cognitive_base, html)
runtime_url_re = re.compile(
rf"(?P<url>{base_pattern}/(?P<asset>[A-Za-z0-9._/-]+)"
r"(?:\?[^\"'\s<>()#]*)?)",
re.IGNORECASE,
)
def replace_runtime_url(match: re.Match[str]) -> str:
asset = match.group("asset").strip("/")
if asset == "generated-page-logo":
source = generated_logo_path if generated_logo_path and generated_logo_path.is_file() else runtime_dir / "brand-logo-dark.png"
if not source.is_file():
return match.group("url")
archive_path = _generated_logo_destination(source)
_add_file(archive_files, archive_path, source)
bundled_assets.add(archive_path.as_posix())
return archive_path.as_posix()
if asset == "deerflow-runtime.js":
source = runtime_dir / asset
if not source.is_file():
return match.group("url")
archive_path = _RUNTIME_ARCHIVE_ROOT / asset
generated_files[archive_path.as_posix()] = _runtime_script(
runtime_dir,
map_tile_url=map_tile_url,
include_geo_maps=include_geo_maps,
)
bundled_assets.add(archive_path.as_posix())
if include_geo_maps:
for filename in sorted(_GEO_MAP_ASSETS):
map_source = _safe_runtime_source(runtime_dir, f"maps/{filename}")
if map_source is not None:
map_archive_path = _RUNTIME_ARCHIVE_ROOT / "maps" / filename
_add_file(archive_files, map_archive_path, map_source)
bundled_assets.add(map_archive_path.as_posix())
return archive_path.as_posix()
source = _safe_runtime_source(runtime_dir, asset)
if source is None:
return match.group("url")
archive_path = _RUNTIME_ARCHIVE_ROOT / PurePosixPath(asset)
_add_file(archive_files, archive_path, source)
bundled_assets.add(archive_path.as_posix())
license_name = _LICENSE_BY_ASSET.get(asset)
if license_name and (runtime_dir / license_name).is_file():
licenses.add(license_name)
return archive_path.as_posix()
html = runtime_url_re.sub(replace_runtime_url, html)
def replace_html_image(match: re.Match[str]) -> str:
token = match.group("token")
try:
_thread_id, _kind, filename = verify_html_image_token(token)
source = resolve_html_image_path(_thread_id, _kind, filename)
except (FileNotFoundError, ValueError):
return match.group("url")
digest = hashlib.sha256(token.encode("utf-8")).hexdigest()[:12]
safe_name = Path(filename).name
archive_path = _HTML_IMAGES_ARCHIVE_ROOT / f"{digest}-{safe_name}"
_add_file(archive_files, archive_path, source)
bundled_assets.add(archive_path.as_posix())
return archive_path.as_posix()
html = _HTML_IMAGE_URL_RE.sub(replace_html_image, html)
external_url_set = {
match.group(0).rstrip("),.;")
for match in _EXTERNAL_URL_RE.finditer(html)
}
if uses_leaflet_map and str(map_tile_url).lower().startswith(("http://", "https://")):
external_url_set.add(str(map_tile_url))
external_urls = sorted(external_url_set)
manifest = {
"format": "deerflow-html-offline-bundle-v1",
"entry": "index.html",
"source_name": html_path.name,
"bundled_assets": sorted(bundled_assets),
"external_urls": external_urls,
}
destination.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(destination, "w", compression=zipfile.ZIP_DEFLATED, compresslevel=6) as archive:
archive.writestr("index.html", html.encode("utf-8"))
archive.writestr(
"manifest.json",
json.dumps(manifest, ensure_ascii=False, indent=2).encode("utf-8"),
)
archive.writestr("README.txt", _readme(html_path.name, external_urls).encode("utf-8"))
for archive_path, source in sorted(archive_files.items()):
archive.write(source, archive_path)
for archive_path, content in sorted(generated_files.items()):
archive.writestr(archive_path, content.encode("utf-8"))
for license_name in sorted(licenses):
archive.write(runtime_dir / license_name, f"THIRD_PARTY_LICENSES/{license_name}")
return OfflineBundleResult(
bundled_assets=tuple(sorted(bundled_assets)),
external_urls=tuple(external_urls),
)
@@ -36,7 +36,9 @@ from deerflow.persistence.roundtable_jobs.model import RoundtableJobRow
from deerflow.persistence.roundtable_jobs.sql import compute_active_dedupe_key
from deerflow.persistence.roundtable_rooms.model import RoundtableRoomLeaseRow
LEASE_SECONDS = 60
DEFAULT_LEASE_SECONDS = 5 * 60
MIN_LEASE_SECONDS = 60
MAX_LEASE_SECONDS = 60 * 60
JOB_LIVE_SECONDS = 15 * 60
OCCUPYING_JOB_STATUSES = ("queued", "running", "awaiting_input", "cancel_requested")
# A background job keeps running while Step 3 generates its artifacts, but the
@@ -80,6 +82,15 @@ def _job_is_live(
return updated is not None and updated >= live_after
def normalize_lease_seconds(value: int | None) -> int:
"""Clamp a foreground room lease to the supported 1–60 minute window."""
try:
seconds = int(value) if value is not None else DEFAULT_LEASE_SECONDS
except (TypeError, ValueError):
seconds = DEFAULT_LEASE_SECONDS
return max(MIN_LEASE_SECONDS, min(MAX_LEASE_SECONDS, seconds))
def _hard_deadline_reached(
created_at: datetime | None,
now: datetime,
@@ -104,6 +115,7 @@ class RoundtableRoomRepository:
enabled: bool,
max_rooms: int,
max_hold_minutes: int = 30,
lease_seconds: int = DEFAULT_LEASE_SECONDS,
extra_keys: set[str] | Mapping[str, str | None] | None = None,
) -> dict[str, Any]:
"""Atomically reserve/renew a room, or reject when the pool is full."""
@@ -119,6 +131,7 @@ class RoundtableRoomRepository:
enabled=enabled,
max_rooms=max_rooms,
max_hold_minutes=max_hold_minutes,
lease_seconds=lease_seconds,
extra_keys=extra_keys,
)
@@ -133,6 +146,7 @@ class RoundtableRoomRepository:
enabled: bool,
max_rooms: int,
max_hold_minutes: int = 30,
lease_seconds: int = DEFAULT_LEASE_SECONDS,
extra_keys: set[str] | Mapping[str, str | None] | None = None,
) -> dict[str, Any]:
"""Atomically admit a pre-resolved room key.
@@ -201,6 +215,7 @@ class RoundtableRoomRepository:
task_id=task_id,
holder_user_id=holder_user_id,
now=now,
lease_seconds=lease_seconds,
)
await session.commit()
return self._decision(
@@ -272,6 +287,7 @@ class RoundtableRoomRepository:
enabled: bool,
max_rooms: int,
max_hold_minutes: int = 30,
lease_seconds: int = DEFAULT_LEASE_SECONDS,
extra_keys: set[str] | Mapping[str, str | None] | None = None,
) -> dict[str, Any]:
"""Renew one live foreground lease without ever creating it.
@@ -298,6 +314,7 @@ class RoundtableRoomRepository:
enabled=enabled,
max_rooms=max_rooms,
max_hold_minutes=max_hold_minutes,
lease_seconds=lease_seconds,
extra_keys=extra_keys,
)
@@ -309,6 +326,7 @@ class RoundtableRoomRepository:
enabled: bool,
max_rooms: int,
max_hold_minutes: int = 30,
lease_seconds: int = DEFAULT_LEASE_SECONDS,
extra_keys: set[str] | Mapping[str, str | None] | None = None,
) -> dict[str, Any]:
"""Renew a resolved persisted key without creating a missing lease."""
@@ -354,7 +372,7 @@ class RoundtableRoomRepository:
message="会商连接已断开,请重新恢复研讨。" if enabled else "",
)
row.lease_until = now + timedelta(seconds=LEASE_SECONDS)
row.lease_until = now + timedelta(seconds=normalize_lease_seconds(lease_seconds))
row.updated_at = now
await session.commit()
return self._decision(
@@ -538,9 +556,10 @@ class RoundtableRoomRepository:
task_id: str | None,
holder_user_id: str | None,
now: datetime,
lease_seconds: int = DEFAULT_LEASE_SECONDS,
) -> None:
row = await session.get(RoundtableRoomLeaseRow, room_key)
lease_until = now + timedelta(seconds=LEASE_SECONDS)
lease_until = now + timedelta(seconds=normalize_lease_seconds(lease_seconds))
if row is None:
session.add(
RoundtableRoomLeaseRow(
@@ -23,7 +23,7 @@ from deerflow.config.agents_config import AGENT_ID_PATTERN, load_agent_config, l
from deerflow.config.paths import get_paths
from deerflow.persistence.scheduled_tasks import ScheduledTaskStore
from deerflow.runtime.scheduler import html_page as html_page_utils
from deerflow.runtime.scheduler import template_fill
from deerflow.runtime.scheduler import template_fill, template_store
from deerflow.runtime.scheduler.element import ElementDeliveryClient
from deerflow.runtime.serialization import serialize_channel_values
from deerflow.runtime.user_context import reset_current_user, set_current_user
@@ -1636,6 +1636,10 @@ class ScheduledTaskService:
# 不影响 Markdown 生成等其它正常操作。缺省(未存该键的老任务)视为开启:
# 模板有真实数据来源后,默认走「智能体检索 → 转换填充」,显式 False 才关闭。
extract = cfg.get("template_extract") is not False
# 「抽取信息写回模板数据」:开启后转换成功的抽取结果除填充本次产物外,还会
# 写回模板的 data.json(默认数据持久化——管理页预览与后续任务即时沿用新数据)。
# 只写数据,MANIFEST(系统名称/导航页面等)绝不动;缺省(老任务)视为关闭。
writeback = cfg.get("template_data_writeback") is True
also_markdown = _task_requires_markdown(task)
prompt = str(task.get("prompt") or "").strip()
description = str(task.get("description") or "").strip()
@@ -1702,8 +1706,21 @@ class ScheduledTaskService:
page_title, page_subtitle = template_fill.pop_page_titles(converted)
# 子页面没抽到有效内容的,回退模板原有数据,避免子页面空白。
data_obj = template_fill.merge_with_reference(tpl_id, converted)
# 字段管理里「接受抽取更新」被关闭(锁定)的字段,恢复模板现有值,
# 抽取结果不覆盖——对本次产物与(开启时的)数据写回同时生效。
data_obj = template_store.restore_locked_fields(tpl_id, data_obj)
content = f"已检索数据并套用模板「{spec.name}」(空子页面回退原数据,含全部子页面)"
used_default = False
if writeback:
try:
# 抽取结果写回模板默认数据(data.json)——仅数据,MANIFEST/系统
# 名称不动;管理页预览与后续任务即时基于更新后的默认数据。
template_store.write_template_data(tpl_id, data_obj)
content += ";抽取数据已写回模板默认数据"
except Exception:
logger.warning(
"模板数据回写失败(不影响本次产物) run=%s tpl=%s", run_id, tpl_id, exc_info=True
)
except TemplateFillError:
logger.warning("模板参考转换失败,回退默认数据 run=%s tpl=%s", run_id, tpl_id, exc_info=True)
data_obj = template_fill.reference_data(tpl_id)
@@ -14,6 +14,12 @@ HTML 模板需要的 ``DATA`` 结构后,本模块负责把数据**填充**进模
行内无换行),所以用 ``^const VAR=.*;$`` (MULTILINE) 精确命中该行整体替换,绝不
误伤模板里别处出现的同名子串。替换值用函数回调写入,避开 ``re.sub`` 把 JSON 里的
反斜杠当反向引用的坑。
* **结构化模板(kind="structured")**: no.1 起引入 page-generator 范式——目录含
``template.html``(骨架,带 :data:`STRUCTURED_MARKERS` 注入标记)/``runtime.js``
(渲染逻辑 + 页面内 JSON 编辑器)/``data.json``(默认数据,兼作转换契约)/可选
``manifest.json``。填充走 :func:`compose_structured` 标记组合,产物仍是自包含
单文件 HTML;``reference_data`` 读 ``data.json``,与单文件模板的契约/校验/合并
语义完全一致。
* **harness 层、零 app.* 依赖**:可被调度器(harness)与 Gateway 路由(app)共同 import。
公开入口:``list_templates`` / ``get_template`` / ``reference_data`` /
@@ -107,6 +113,14 @@ def extract_embedded_json(text: str | None) -> Any | None:
# 模板资源根目录(随 git 发布,与本模块同级 ``templates/`` 子目录)。
_TEMPLATES_ROOT = Path(__file__).parent / "templates"
# 结构化模板骨架里的三个注入标记(compose_structured 逐个替换)。
DATA_MARKER = "__PAGE_DATA__"
MANIFEST_MARKER = "__PAGE_MANIFEST__"
RUNTIME_MARKER = "__PAGE_RUNTIME__"
STRUCTURED_MARKERS: tuple[str, ...] = (DATA_MARKER, MANIFEST_MARKER, RUNTIME_MARKER)
# 结构化模板目录里的组成文件(manifest.json 可选,其余必备)。
STRUCTURED_PARTS: tuple[str, ...] = ("template.html", "runtime.js", "data.json", "manifest.json")
@dataclass(frozen=True)
class TemplateSpec:
@@ -115,9 +129,20 @@ class TemplateSpec:
``file`` 相对 :data:`_TEMPLATES_ROOT`;``data_var``/``manifest_var`` 是模板里
待替换的 JS 常量名(``manifest_var`` 为空表示该模板没有 MANIFEST 行)。
``kind`` 有两种:
* ``"single"``(默认): 单文件模板,数据以 ``const DATA={…};`` 单行内嵌,填充 =
行级替换;
* ``"structured"``: 结构化模板(page-generator 范式),目录含
``template.html``(骨架,带 :data:`STRUCTURED_MARKERS` 三个注入标记)/
``runtime.js``(渲染 + 页面内 JSON 编辑器)/``data.json``(默认数据,即转换
契约)/可选 ``manifest.json``,填充 = 标记组合,产物仍是自包含单文件 HTML。
``legacy_file`` 指向单文件时代的随包正文,供 template_store 自动升级时做
「未被手改」的字节级比对。
``convertible=False`` 表示该模板暂只「直接展示已有数据」——执行时原样输出模板
HTML,不做 JSON 转换/校验/填充(无 ``const DATA=`` 注入行的模板用它,后续接入
转换后改回 True)。
转换后再改回 True)。
"""
id: str
@@ -127,6 +152,12 @@ class TemplateSpec:
data_var: str = "DATA"
manifest_var: str = ""
convertible: bool = True
kind: str = "single"
legacy_file: str = ""
@property
def structured(self) -> bool:
return self.kind == "structured"
# 已注册模板(随包 seed 源 + 存储缺失时的兜底注册表)。新增模板时在此追加一条
@@ -136,9 +167,14 @@ TEMPLATES: tuple[TemplateSpec, ...] = (
id="no1",
name="no.1",
description="",
file="no1/taiwan_littoral_ui_redwhite_v2.html",
file="no1/template.html",
data_var="DATA",
manifest_var="MANIFEST",
# 结构化模板(page-generator 范式):骨架 + runtime + 默认数据 + MANIFEST,
# 填充走 compose_structured 标记组合;legacy_file 是单文件时代的随包正文,
# 供 template_store 把「未被手改」的旧存储副本自动升级到结构化版。
kind="structured",
legacy_file="no1/taiwan_littoral_ui_redwhite_v2.html",
),
TemplateSpec(
id="no2",
@@ -401,6 +437,7 @@ def get_template(template_id: str) -> TemplateSpec | None:
data_var="DATA",
manifest_var=str(meta.get("manifest_var") or ""),
convertible=bool(meta.get("convertible")),
kind=str(meta.get("kind") or "single"),
)
return _TEMPLATES_BY_ID.get(template_id)
@@ -435,8 +472,122 @@ def _template_path(spec: TemplateSpec) -> Path:
return _TEMPLATES_ROOT / spec.file
# ── 结构化模板(骨架 + runtime + 数据分文件,组合成单文件产物) ────────────────
def _structured_part(template_id: str, name: str) -> Path | None:
"""结构化模板某个组成文件的路径:存储副本优先,随包文件兜底;都没有返回 None。
``manifest.json`` 可选;其余文件缺失属打包损坏,由 :func:`_structured_text` 报错。
"""
from deerflow.runtime.scheduler import template_store
store_file = template_store.store_root() / template_id / name
if store_file.is_file():
return store_file
fallback = _TEMPLATES_ROOT / template_id / name
return fallback if fallback.is_file() else None
def _structured_text(template_id: str, name: str) -> str:
path = _structured_part(template_id, name)
if path is None:
raise TemplateError(f"结构化模板文件缺失:{template_id}/{name}")
return path.read_text(encoding="utf-8")
def _read_default_manifest(template_id: str) -> dict[str, Any]:
"""结构化模板的默认 MANIFEST;没有 manifest.json 时返回空对象。"""
path = _structured_part(template_id, "manifest.json")
if path is None:
return {}
try:
obj = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
raise TemplateError(f"解析 {template_id}/manifest.json 失败:{exc}") from exc
return obj if isinstance(obj, dict) else {}
def structured_module_labels(template_id: str) -> list[dict[str, str]]:
"""结构化模板各数据模块(DATA 顶层键)的中文显示名,管理页「编辑数据」用。
标签来源按优先级:MANIFEST.页面 的「编号」精确命中 → 「数据源/架构数据」去掉
``data/`` 前缀和 ``.json`` 后缀后命中(如 organization_charts) → 编号按 ``_``
拆分后命中任一段(如 events/projects 命中 events_projects)。都没有则回退键名。
"""
ref = reference_data(template_id)
manifest = _read_default_manifest(template_id)
labels: dict[str, str] = {}
split_index: dict[str, str] = {}
pages = manifest.get("页面")
if isinstance(pages, list):
for page in pages:
if not isinstance(page, dict):
continue
name = str(page.get("名称") or "").strip()
if not name:
continue
candidates: list[str] = []
for field in ("编号", "数据源", "架构数据"):
raw = str(page.get(field) or "").strip()
if not raw:
continue
key = raw.removeprefix("data/").removesuffix(".json")
if key:
candidates.append(key)
for key in candidates:
labels[key] = name
first = candidates[0] if candidates else ""
if first and "_" in first:
for part in first.split("_"):
if part:
split_index[part] = name
out: list[dict[str, str]] = []
for key in ref:
out.append({"key": key, "label": labels.get(key) or split_index.get(key) or key})
return out
def compose_structured(
*,
skeleton: str,
runtime: str,
data: Any,
manifest: dict[str, Any] | None = None,
) -> str:
"""把数据/运行时按标记组合进骨架,产出自包含单文件 HTML。
与 page-generator 技能的 render 脚本同一套安全转义:数据 JSON 里的 ``</`` →
``<\\/``(JSON 字符串里 ``\\/`` 合法),runtime 里的 ``</script`` → ``<\\/script``,
防止内容提前闭合 script 标签。组合后断言无残留标记。
"""
serialized_data = json.dumps(data, ensure_ascii=False).replace("</", "<\\/")
out = skeleton.replace(DATA_MARKER, serialized_data)
if MANIFEST_MARKER in out:
serialized_manifest = (
json.dumps(manifest, ensure_ascii=False).replace("</", "<\\/")
if manifest is not None
else "{}"
)
out = out.replace(MANIFEST_MARKER, serialized_manifest)
out = out.replace(RUNTIME_MARKER, runtime.replace("</script", "<\\/script"))
residual = [m for m in STRUCTURED_MARKERS if m in out]
if residual:
raise TemplateError(f"结构化模板组合后仍残留标记:{'、'.join(residual)}")
return out
def read_template_text(template_id: str) -> str:
spec = _require(template_id)
if spec.structured:
# 结构化模板对外始终呈现「组合后的完整文档」(预览/只展示/参考展示共用);
# 骨架原文(编辑用)由 template_store.read_template_html 直接读文件。
return compose_structured(
skeleton=_structured_text(template_id, "template.html"),
runtime=_structured_text(template_id, "runtime.js"),
data=reference_data(template_id),
manifest=_read_default_manifest(template_id),
)
path = _template_path(spec)
if not path.is_file():
raise TemplateError(f"模板文件缺失:{path}")
@@ -464,8 +615,22 @@ def _extract_const_object(text: str, var: str) -> Any:
@lru_cache(maxsize=16)
def reference_data(template_id: str) -> dict[str, Any]:
"""模板内嵌的参考 ``DATA`` 对象(目标格式范例 + 校验基准),按模板缓存。"""
"""模板内嵌的参考 ``DATA`` 对象(目标格式范例 + 校验基准),按模板缓存。
单文件模板解析 ``const DATA=`` 行;结构化模板直接读 ``data.json``(两者语义一致:
顶层键即子页面,validate_data/merge_with_reference/骨架 prompt 全部通用)。
"""
spec = _require(template_id)
if spec.structured:
try:
obj = json.loads(_structured_text(template_id, "data.json"))
except OSError as exc:
raise TemplateError(f"读取 {template_id}/data.json 失败:{exc}") from exc
except ValueError as exc:
raise TemplateError(f"解析 {template_id}/data.json 失败:{exc}") from exc
if not isinstance(obj, dict):
raise TemplateError(f"模板 {template_id} 的 data.json 不是对象")
return obj
text = read_template_text(template_id)
obj = _extract_const_object(text, spec.data_var)
if not isinstance(obj, dict):
@@ -774,28 +939,18 @@ def pop_page_titles(obj: dict[str, Any]) -> tuple[str | None, str | None]:
return title, subtitle
def fill_template(
template_id: str,
data_obj: Any,
*,
manifest_overrides: dict[str, Any] | None = None,
update_date: date | None = None,
title: str | None = None,
subtitle: str | None = None,
) -> str:
"""把 ``data_obj`` 填进模板的 ``const DATA=…`` 行,产出完整 HTML 文档。
def _apply_page_titles(
text: str,
title: str | None,
subtitle: str | None,
manifest_overrides: dict[str, Any] | None,
) -> tuple[str, dict[str, Any] | None]:
"""替换页头静态标题/副标题;给出 title 时同步返回含 ``系统.名称`` 的 overrides。
可选地更新 MANIFEST 的「系统.更新时间」(``update_date``,默认不动) 以及任意
``manifest_overrides``(深合并到解析后的 MANIFEST,如改系统名称)。
``title``/``subtitle`` 来自转换结果里的「页面标题/页面副标题」:模板页头
(<title>、第一个 <h1>、其后紧跟的副标题 <p>)是写死的静态 HTML,不随 DATA
变化——给出新标题时替换这三处,并把 MANIFEST 的 系统.名称 一并同步;
不给则保持模板原文。
模板页头(``<title>``、第一个 ``<h1>``、其后紧跟的副标题 ``<p>``)是写死的静态
HTML,不随 DATA 变化——给出新标题时替换这两处,并把 MANIFEST 的 系统.名称 一并
同步;不给则保持模板原文。单文件/结构化两条填充路径共用。
"""
spec = _require(template_id)
text = read_template_text(template_id)
if title:
overrides = dict(manifest_overrides or {})
system = overrides.get("系统")
@@ -815,6 +970,70 @@ def fill_template(
count=1,
flags=re.DOTALL,
)
return text, manifest_overrides
def _fill_structured(
template_id: str,
data_obj: Any,
*,
manifest_overrides: dict[str, Any] | None,
update_date: date | None,
title: str | None,
subtitle: str | None,
) -> str:
"""结构化模板填充:标记组合出完整单文件 HTML(数据内嵌 JSON 元素 + runtime 内联)。"""
skeleton, manifest_overrides = _apply_page_titles(
_structured_text(template_id, "template.html"), title, subtitle, manifest_overrides
)
manifest = _read_default_manifest(template_id)
if update_date is not None:
system = manifest.get("系统")
if isinstance(system, dict):
system["更新时间"] = update_date.isoformat()
if manifest_overrides:
_deep_merge(manifest, manifest_overrides)
return compose_structured(
skeleton=skeleton,
runtime=_structured_text(template_id, "runtime.js"),
data=data_obj,
manifest=manifest,
)
def fill_template(
template_id: str,
data_obj: Any,
*,
manifest_overrides: dict[str, Any] | None = None,
update_date: date | None = None,
title: str | None = None,
subtitle: str | None = None,
) -> str:
"""把 ``data_obj`` 填进模板,产出完整 HTML 文档。
单文件模板:行级替换 ``const DATA=…``;结构化模板::func:`_fill_structured`
标记组合。可选地更新 MANIFEST 的「系统.更新时间」(``update_date``,默认不动)
以及任意 ``manifest_overrides``(深合并到解析后的 MANIFEST,如改系统名称)。
``title``/``subtitle`` 来自转换结果里的「页面标题/页面副标题」:模板页头
(<title>、第一个 <h1>、其后紧跟的副标题 <p>)是写死的静态 HTML,不随 DATA
变化——给出新标题时替换这三处,并把 MANIFEST 的 系统.名称 一并同步;
不给则保持模板原文。
"""
spec = _require(template_id)
if spec.structured:
return _fill_structured(
template_id,
data_obj,
manifest_overrides=manifest_overrides,
update_date=update_date,
title=title,
subtitle=subtitle,
)
text = read_template_text(template_id)
text, manifest_overrides = _apply_page_titles(text, title, subtitle, manifest_overrides)
data_re = _const_line_re(spec.data_var)
if data_re.search(text) is None:
@@ -5,17 +5,27 @@
升级为**文件型存储**,模板管理页与定时任务表单共用:
{base_dir}/html-templates/<id>/
├── template.html # 模板正文
└── meta.json # {id, name, description, builtin, convertible, manifest_var, created_at, updated_at}
├── template.html # 模板正文(单文件模板 = 完整文档;结构化模板 = 骨架)
├── runtime.js # 结构化模板:渲染逻辑 + 页面内 JSON 编辑器(可选件)
├── data.json # 结构化模板:默认数据,兼作转换契约(可选件)
├── manifest.json # 结构化模板:系统/页面/交互规则(可选件)
└── meta.json # {id, name, description, builtin, kind, convertible, manifest_var, created_at, updated_at}
设计要点
--------
* **seed 不覆盖**:{base_dir}/html-templates/<id>/ 目录缺失时才从随包模板复制(与
``_template_builder_seed.py`` 同语义),线上/管理页的修改绝不会被启动 seed 回写。
* **convertible 自动探测**:写入(创建/改正文/重置)时扫描正文是否含可解析的内联
``const DATA={…};`` 行(当前填充引擎唯一支持的注入点)——有则该模板可「JSON 转换
填充」,无则「只展示」。上传带注入行的新模板即可零代码接入转换管线;探测同时认
``const MANIFEST=`` 行并记入 meta,``fill_template`` 的更新时间/标题替换可生效。
**例外——结构化自动升级**:注册表条目升级为 ``kind="structured"`` 后,若存储里的
旧单文件副本与随包 ``legacy_file`` 字节一致(即从未被手改),启动时自动重建为
结构化版;改过的保留旧版,由管理页「重置」手动升级。
* **convertible 自动探测**(单文件模板):写入(创建/改正文/重置)时扫描正文是否含
可解析的内联 ``const DATA={…};`` 行——有则该模板可「JSON 转换填充」,无则「只展示」。
结构化模板的 convertible/manifest_var 以注册表/``data.json`` 为准,不靠正文探测。
* **结构化模板的数据持久化**:``read_template_data`` / ``write_template_data`` 支撑
管理页「编辑数据」——整体替换 ``data.json``,替换前经
:func:`template_fill.validate_data` 契约校验(顶层键不可增删),改契约先改模板。
编辑结构化正文仅允许改骨架,且必须保留 :data:`template_fill.STRUCTURED_MARKERS`
三个注入标记。
* **内置模板可改不可删**:删除会把引用它的定时任务全部弄挂,内置模板(builtin)
只允许编辑/上传覆盖/重置回随包原版,自建模板可删。
* harness 层、零 app.* 依赖:调度器与 Gateway 路由共同 import;所有写操作都会
@@ -114,42 +124,84 @@ def _validate_html(html: str) -> str:
def ensure_seeded() -> bool:
"""把随包内置模板 seed 进存储;目录已存在(尊重管理页修改)则跳过。
返回 True 表示本次至少复制了一个模板。任何单模板失败只影响该模板,不抛出。
结构化模板多一步**自动升级**:注册表条目已是 ``kind="structured"`` 而存储 meta
还没有 kind(单文件时代的旧副本)时,若旧副本正文与随包 ``legacy_file`` 字节一致
(从未手改)则重建为结构化版;改过的保留,由管理页「重置」手动升级。
返回 True 表示本次至少新建/升级了一个模板。任何单模板失败只影响该模板,不抛出。
"""
created = False
changed = False
for spec in template_fill.TEMPLATES:
try:
tpl_dir = store_root() / spec.id
if tpl_dir.exists():
if spec.structured:
changed |= _maybe_upgrade_structured(spec)
continue
tpl_dir.mkdir(parents=True, exist_ok=True)
src = template_fill._TEMPLATES_ROOT / spec.file
html = src.read_text(encoding="utf-8")
(tpl_dir / "template.html").write_text(html, encoding="utf-8")
now = _now_iso()
(tpl_dir / "meta.json").write_text(
json.dumps(
{
"id": spec.id,
"name": spec.name,
"description": spec.description,
"builtin": True,
"convertible": spec.convertible,
"manifest_var": spec.manifest_var,
"created_at": now,
"updated_at": now,
},
ensure_ascii=False,
indent=1,
),
encoding="utf-8",
)
created = True
_seed_spec_dir(spec, tpl_dir)
changed = True
except Exception:
continue
if created:
if changed:
template_fill.invalidate_cache()
return created
return changed
def _seed_spec_dir(spec: Any, tpl_dir: Path) -> None:
"""按注册表条目把一个内置模板 seed 进存储目录(调用方保证目录不存在)。"""
tpl_dir.mkdir(parents=True, exist_ok=True)
src_root = template_fill._TEMPLATES_ROOT / spec.id
if spec.structured:
# 必备件缺失属打包损坏:抛错由 ensure_seeded 按单模板吞掉。
for name in ("template.html", "runtime.js", "data.json"):
(tpl_dir / name).write_text(
(src_root / name).read_text(encoding="utf-8"), encoding="utf-8"
)
for name in ("manifest.json", "fields.json"):
optional = src_root / name
if optional.is_file():
(tpl_dir / name).write_text(
optional.read_text(encoding="utf-8"), encoding="utf-8"
)
else:
src = template_fill._TEMPLATES_ROOT / spec.file
(tpl_dir / "template.html").write_text(
src.read_text(encoding="utf-8"), encoding="utf-8"
)
_write_meta(
spec.id,
{
"id": spec.id,
"name": spec.name,
"description": spec.description,
"builtin": True,
"kind": spec.kind,
"convertible": spec.convertible,
"manifest_var": spec.manifest_var,
},
keep_created=False,
)
def _maybe_upgrade_structured(spec: Any) -> bool:
"""旧单文件存储副本 → 结构化版(仅当正文与随包 legacy 文件字节一致,即从未手改)。"""
tpl_dir = store_root() / spec.id
if not tpl_dir.is_dir() or not spec.legacy_file:
return False
if _load_meta(tpl_dir).get("kind"):
return False # 已是结构化
legacy = template_fill._TEMPLATES_ROOT / spec.legacy_file
current = tpl_dir / "template.html"
if not legacy.is_file() or not current.is_file():
return False
try:
if legacy.read_bytes() != current.read_bytes():
return False # 线上手改过 → 保留旧版,「重置」手动升级
except OSError:
return False
shutil.rmtree(tpl_dir)
_seed_spec_dir(spec, tpl_dir)
return True
# ── 读取 ─────────────────────────────────────────────────────────────────────
@@ -169,8 +221,9 @@ def _load_meta(tpl_dir: Path) -> dict[str, Any]:
def list_templates() -> list[dict[str, Any]]:
"""存储内全部模板(内置按注册表顺序在前,自建按创建时间在后)。
返回条目:id/name/description/builtin/convertible/manifest_var/size/created_at/
updated_at;正文缺失的目录跳过(半成品不进下拉)。
返回条目:id/name/description/builtin/kind/convertible/manifest_var/size/created_at/
updated_at;正文缺失的目录跳过(半成品不进下拉)。结构化模板的 size 汇总
template.html+runtime.js+data.json+manifest.json。
"""
ensure_seeded()
root = store_root()
@@ -184,15 +237,24 @@ def list_templates() -> list[dict[str, Any]]:
if not html_file.is_file():
continue
meta = _load_meta(tpl_dir)
kind = str(meta.get("kind") or "single")
size = html_file.stat().st_size
if kind == "structured":
size += sum(
(tpl_dir / name).stat().st_size
for name in ("runtime.js", "data.json", "manifest.json")
if (tpl_dir / name).is_file()
)
entries.append(
{
"id": tpl_dir.name,
"name": str(meta.get("name") or tpl_dir.name),
"description": meta.get("description") or "",
"builtin": bool(meta.get("builtin")),
"kind": kind,
"convertible": bool(meta.get("convertible")),
"manifest_var": str(meta.get("manifest_var") or ""),
"size": html_file.stat().st_size,
"size": size,
"created_at": meta.get("created_at"),
"updated_at": meta.get("updated_at"),
}
@@ -215,12 +277,286 @@ def get_template_meta(template_id: str) -> dict[str, Any] | None:
def read_template_html(template_id: str) -> str:
"""模板正文**原文**(结构化模板 = 带 __PAGE_* 标记的骨架,非完整可预览文档)。
需要结构化模板的完整文档(预览/只展示)请用
:func:`template_fill.read_template_text`,它会把默认数据与 runtime 组合进去。
"""
path = store_template_file(template_id)
if not path.is_file():
raise TemplateStoreError(f"模板不存在:{template_id}")
return path.read_text(encoding="utf-8")
# ── 结构化模板数据(data.json)的读写(管理页「编辑数据」持久化) ────────────────
def _require_structured(template_id: str) -> dict[str, Any]:
entry = get_template_meta(template_id)
if entry is None:
raise TemplateStoreError(f"模板不存在:{template_id}")
if str(entry.get("kind") or "single") != "structured":
raise TemplateStoreError(f"模板不是结构化模板,不支持编辑数据:{template_id}")
return entry
def read_template_data(template_id: str) -> dict[str, Any]:
"""结构化模板的默认数据(data.json);存储缺文件时按引擎同一回退读随包。"""
_require_structured(template_id)
path = store_root() / template_id / "data.json"
if not path.is_file():
return template_fill.reference_data(template_id)
try:
obj = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
raise TemplateStoreError(f"读取 data.json 失败:{exc}") from exc
if not isinstance(obj, dict):
raise TemplateStoreError("data.json 不是 JSON 对象")
return obj
def read_template_manifest(template_id: str) -> dict[str, Any]:
"""结构化模板的 MANIFEST(manifest.json,系统名称/导航页面/交互规则);缺失返回空对象。"""
_require_structured(template_id)
path = store_root() / template_id / "manifest.json"
if not path.is_file():
return {}
try:
obj = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
raise TemplateStoreError(f"读取 manifest.json 失败:{exc}") from exc
if not isinstance(obj, dict):
raise TemplateStoreError("manifest.json 不是 JSON 对象")
return obj
def _dump_json_checked(obj: Any, label: str) -> str:
payload = json.dumps(obj, ensure_ascii=False)
encoded = payload.encode("utf-8")
if len(encoded) > MAX_HTML_BYTES:
raise TemplateStoreError(
f"{label}过大({len(encoded)} 字节,上限 {MAX_HTML_BYTES} 字节)"
)
return payload
def write_template_data(
template_id: str,
data_obj: Any,
*,
manifest: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""整体替换结构化模板的默认数据(可选一并替换 MANIFEST)并清缓存;返回更新后的 meta。
data.json 替换前经 :func:`template_fill.validate_data` 契约校验(顶层键不可增删);
``manifest`` 给出时整体替换 manifest.json(系统名称/导航页面/交互规则的持久化入口)。
"""
_require_structured(template_id)
if not isinstance(data_obj, dict):
raise TemplateStoreError("数据必须是 JSON 对象")
errors = template_fill.validate_data(template_id, data_obj)
if errors:
raise TemplateStoreError(
"数据未通过模板契约校验(顶层键须与现有契约一致):" + ";".join(errors)
)
payload = _dump_json_checked(data_obj, "数据")
(store_root() / template_id / "data.json").write_text(payload, encoding="utf-8")
if manifest is not None:
if not isinstance(manifest, dict):
raise TemplateStoreError("MANIFEST 必须是 JSON 对象")
(store_root() / template_id / "manifest.json").write_text(
_dump_json_checked(manifest, "MANIFEST"), encoding="utf-8"
)
_write_meta(template_id, _load_meta(store_root() / template_id), keep_created=True)
template_fill.invalidate_cache()
return get_template_meta(template_id) # type: ignore[return-value]
# ── 字段显示 / 自定义模块(manifest 增量配置,字段目录读 fields.json) ─────────
def read_template_fields(template_id: str) -> list[dict[str, Any]]:
"""结构化模板的字段目录(fields.json:label/pages/description);没有目录返回空表。"""
_require_structured(template_id)
for base in (store_root() / template_id, template_fill._TEMPLATES_ROOT / template_id):
path = base / "fields.json"
if path.is_file():
try:
obj = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
raise TemplateStoreError(f"读取 fields.json 失败:{exc}") from exc
fields = obj.get("fields") if isinstance(obj, dict) else None
if isinstance(fields, list):
return [f for f in fields if isinstance(f, dict) and f.get("label")]
return []
return []
def _write_manifest_section(
template_id: str, key: str, value: Any, *, validate=None
) -> dict[str, Any]:
"""更新 manifest.json 的单个顶层键(其余原样保留)并清引擎缓存;返回更新后的 meta。
``validate(value)`` 可选,抛 TemplateStoreError 时拒绝写入。
"""
_require_structured(template_id)
manifest = read_template_manifest(template_id)
if validate is not None:
validate(value)
manifest[key] = value
(store_root() / template_id / "manifest.json").write_text(
_dump_json_checked(manifest, "MANIFEST"), encoding="utf-8"
)
_write_meta(template_id, _load_meta(store_root() / template_id), keep_created=True)
template_fill.invalidate_cache()
return get_template_meta(template_id) # type: ignore[return-value]
def write_template_field_visibility(
template_id: str, hidden_labels: list[str]
) -> dict[str, Any]:
"""字段显示/隐藏配置:manifest 的「字段显示」记 {label: false}(未记录=显示)。
标签须来自字段目录(fields.json),防止把「系统设置」里的同名键写坏。
"""
catalog = {f["label"] for f in read_template_fields(template_id)}
unknown = [x for x in hidden_labels if x not in catalog]
if unknown:
raise TemplateStoreError("未知字段:" + "、".join(unknown[:5]))
hidden = {label: False for label in hidden_labels}
return _write_manifest_section(template_id, "字段显示", hidden)
def write_template_field_locks(template_id: str, locked_labels: list[str]) -> dict[str, Any]:
"""字段「接受抽取更新」锁定名单:manifest「字段抽取关闭」记 {label: True}。
锁定后,定时任务抽取/写回不改该字段对应的数据键(保持模板现有数据);
名单为空 = 全部接受抽取更新(默认)。
"""
catalog = {f["label"] for f in read_template_fields(template_id)}
unknown = [x for x in locked_labels if x not in catalog]
if unknown:
raise TemplateStoreError("未知字段:" + "、".join(unknown[:5]))
locks = {label: True for label in locked_labels}
return _write_manifest_section(template_id, "字段抽取关闭", locks)
def locked_field_keys(template_id: str) -> set[str]:
"""被锁定(不接受抽取更新)字段对应的数据键集合;无锁定时为空集。"""
manifest = read_template_manifest(template_id)
locks = manifest.get("字段抽取关闭")
locks = locks if isinstance(locks, dict) else {}
if not any(v is True for v in locks.values()):
return set()
keys: set[str] = set()
for f in read_template_fields(template_id):
if locks.get(f["label"]) is True:
keys.update(str(k) for k in (f.get("keys") or []) if k)
return keys
def _item_identity(item: Any) -> str | None:
"""列表条目的身份键(id/编号/name/名称 取第一个非空),锁定恢复时按它匹配新旧条目。"""
if not isinstance(item, dict):
return None
for key in ("id", "编号", "name", "名称"):
value = item.get(key)
if isinstance(value, str) and value.strip():
return f"{key}:{value.strip()}"
if isinstance(value, (int, float)) and not isinstance(value, bool):
return f"{key}:{value}"
return None
def restore_locked_fields(template_id: str, data_obj: dict[str, Any]) -> dict[str, Any]:
"""把锁定字段的数据键恢复为模板现有值——抽取结果不覆盖,供调度器在
``merge_with_reference`` 之后、填充/写回之前调用。
列表分支按条目身份匹配新旧条目,匹配上才恢复(新条目保持抽取值);
对象分支(overview)逐键恢复。无锁定时原样返回。
"""
keys = locked_field_keys(template_id)
if not keys:
return data_obj
reference = template_fill.reference_data(template_id)
out = dict(data_obj)
for branch, new_val in data_obj.items():
old_val = reference.get(branch)
if isinstance(old_val, list) and isinstance(new_val, list):
old_index: dict[str, dict[str, Any]] = {}
for old_item in old_val:
if isinstance(old_item, dict):
ident = _item_identity(old_item)
if ident:
old_index[ident] = old_item
restored: list[Any] = []
for new_item in new_val:
if isinstance(new_item, dict):
ident = _item_identity(new_item)
old_item = old_index.get(ident) if ident else None
if old_item is not None:
merged = dict(new_item)
for key in keys:
if key in old_item:
merged[key] = old_item[key]
restored.append(merged)
continue
restored.append(new_item)
out[branch] = restored
elif isinstance(old_val, dict) and isinstance(new_val, dict):
merged = dict(new_val)
for key in keys:
if key in old_val:
merged[key] = old_val[key]
out[branch] = merged
return out
def write_template_custom_modules(
template_id: str, modules: list[dict[str, Any]]
) -> dict[str, Any]:
"""自定义模块列表(挂在指定页面底部的 AI 生成 HTML 片段)整体替换。
每项 {编号, 页面, 标题, html};html 经 :func:`sanitize_html_fragment` 消毒
(去 script/事件属性/javascript: 链接)后才允许入库。
"""
def _validate(value: Any) -> None:
if not isinstance(value, list):
raise TemplateStoreError("自定义模块必须是数组")
for i, item in enumerate(value):
if not isinstance(item, dict):
raise TemplateStoreError(f"自定义模块[{i}]必须是对象")
page = str(item.get("页面") or "").strip()
if not page:
raise TemplateStoreError(f"自定义模块[{i}]缺少「页面」")
html = str(item.get("html") or "").strip()
if not html:
raise TemplateStoreError(f"自定义模块[{i}]缺少内容 html")
item["html"] = sanitize_html_fragment(html)
return _write_manifest_section(template_id, "自定义模块", modules, validate=_validate)
# 自定义模块 HTML 片段消毒:去 <script>/<style 事件>/on* 事件属性/javascript: 链接。
_SCRIPT_TAG_RE = re.compile(r"<\s*/?\s*script\b[^>]*>", re.IGNORECASE)
_SCRIPT_BLOCK_RE = re.compile(r"<\s*script\b.*?<\s*/\s*script\s*>", re.IGNORECASE | re.DOTALL)
_ON_ATTR_RE = re.compile(r"\son[a-zA-Z]+\s*=\s*(?:\"[^\"]*\"|'[^']*'|[^\s>]+)", re.IGNORECASE)
_JS_URL_RE = re.compile(
r"(href|src|action)\s*=\s*(\"[^\"]*javascript:[^\"]*\"|'[^']*javascript:[^']*'|javascript:[^\s>]*)",
re.IGNORECASE,
)
def sanitize_html_fragment(html: str) -> str:
"""自定义模块片段入库前的安全清洗(页面经 innerHTML 渲染,须无脚本可执行)。"""
out = _SCRIPT_BLOCK_RE.sub("", str(html or ""))
out = _SCRIPT_TAG_RE.sub("", out)
out = _ON_ATTR_RE.sub("", out)
out = _JS_URL_RE.sub("", out)
return out.strip()
# ── 写入 ─────────────────────────────────────────────────────────────────────
@@ -275,13 +611,27 @@ def update_template(
description: str | None = None,
html: str | None = None,
) -> dict[str, Any]:
"""编辑模板(名称/描述/正文任选;改正文时重新探测 convertible 并清引擎缓存)。"""
"""编辑模板(名称/描述/正文任选;改正文时重新探测 convertible 并清引擎缓存)。
结构化模板的正文编辑仅针对**骨架**,保存前校验三个注入标记齐全(删标记 = 断了
组合管线);其 convertible/manifest_var 不靠正文探测,保持原值。改默认数据请用
:func:`write_template_data`。
"""
entry = get_template_meta(template_id)
if entry is None:
raise TemplateStoreError(f"模板不存在:{template_id}")
structured = str(entry.get("kind") or "single") == "structured"
if html is not None:
html = _validate_html(html)
if structured:
missing = [m for m in template_fill.STRUCTURED_MARKERS if m not in html]
if missing:
raise TemplateStoreError(
"结构化模板骨架缺少注入标记:"
+ "、".join(missing)
+ "(编辑正文不可删除标记;改数据请用「编辑数据」)"
)
store_template_file(template_id).write_text(html, encoding="utf-8")
meta = _load_meta(store_root() / template_id)
@@ -292,7 +642,7 @@ def update_template(
meta["name"] = name
if description is not None:
meta["description"] = description.strip()
if html is not None:
if html is not None and not structured:
convertible, manifest_var = _detect_specs(html)
meta["convertible"] = convertible
meta["manifest_var"] = manifest_var
@@ -313,14 +663,31 @@ def delete_template(template_id: str) -> None:
def reset_template(template_id: str) -> dict[str, Any]:
"""把内置模板的正文重置回随包原版(meta 名称/描述保留,convertible 重新探测)。"""
"""把内置模板重置回随包原版(meta 名称/描述保留,convertible 重新按注册表)。
结构化模板重置全部组成文件(template.html/runtime.js/data.json/manifest.json)。
"""
seed = next((s for s in template_fill.TEMPLATES if s.id == template_id), None)
entry = get_template_meta(template_id)
if seed is None or entry is None:
raise TemplateStoreError(f"仅内置模板支持重置:{template_id}")
html = (template_fill._TEMPLATES_ROOT / seed.file).read_text(encoding="utf-8")
store_template_file(template_id).write_text(html, encoding="utf-8")
if seed.structured:
src_root = template_fill._TEMPLATES_ROOT / template_id
for name in ("template.html", "runtime.js", "data.json"):
(store_root() / template_id / name).write_text(
(src_root / name).read_text(encoding="utf-8"), encoding="utf-8"
)
for name in ("manifest.json", "fields.json"):
optional = src_root / name
if optional.is_file():
(store_root() / template_id / name).write_text(
optional.read_text(encoding="utf-8"), encoding="utf-8"
)
else:
html = (template_fill._TEMPLATES_ROOT / seed.file).read_text(encoding="utf-8")
store_template_file(template_id).write_text(html, encoding="utf-8")
meta = _load_meta(store_root() / template_id)
meta["kind"] = seed.kind
meta["convertible"] = seed.convertible
meta["manifest_var"] = seed.manifest_var
_write_meta(template_id, meta, keep_created=True)
@@ -0,0 +1 @@
{"overview": {"核心指标": [{"名称": "综合区域", "数值": 9, "单位": "区", "说明": "覆盖本岛沿岸与主要离岛"}, {"名称": "组织与单位", "数值": 280, "单位": "个", "说明": "已去除组织与单位重复"}, {"名称": "单位深度档案", "数值": 179, "单位": "份", "说明": "已合并到对应组织节点"}, {"名称": "装备条目", "数值": 99, "单位": "项", "说明": "区分规划、采购、测试和列装"}, {"名称": "关键节点", "数值": 141, "单位": "个", "说明": "公共功能与韧性视角"}, {"名称": "演习与项目", "数值": 12, "单位": "项", "说明": "事件时间线与建设项目"}], "组织体系分布": [{"名称": "组织治理体系", "数值": 101}, {"名称": "海巡", "数值": 43}, {"名称": "全民动员", "数值": 29}, {"名称": "国防与联合体系", "数值": 28}, {"名称": "海军", "数值": 21}, {"名称": "陆军", "数值": 18}, {"名称": "空军", "数值": 16}, {"名称": "公共支援", "数值": 16}, {"名称": "海军陆战队", "数值": 8}], "装备类别分布": [{"名称": "无人系统", "数值": 12}, {"名称": "防空", "数值": 11}, {"名称": "海军平台", "数值": 10}, {"名称": "海巡平台", "数值": 9}, {"名称": "航空", "数值": 8}, {"名称": "地面火力", "数值": 7}, {"名称": "装甲", "数值": 5}, {"名称": "雷达与感知", "数值": 5}, {"名称": "反装甲", "数值": 4}, {"名称": "反舰", "数值": 3}, {"名称": "潜舰", "数值": 3}, {"名称": "反水雷", "数值": 3}, {"名称": "舰载防空", "数值": 3}, {"名称": "电子战", "数值": 3}, {"名称": "指挥控制", "数值": 3}, {"名称": "工程保障", "数值": 2}, {"名称": "布雷", "数值": 1}, {"名称": "综合支援", "数值": 1}, {"名称": "反无人机", "数值": 1}, {"名称": "无人水面平台", "数值": 1}, {"名称": "通信", "数值": 1}, {"名称": "机动运输", "数值": 1}, {"名称": "防护", "数值": 1}, {"名称": "医疗保障", "数值": 1}], "节点类别分布": [{"名称": "港口与海运", "数值": 26}, {"名称": "供水", "数值": 22}, {"名称": "能源与燃料", "数值": 18}, {"名称": "机场与航空", "数值": 17}, {"名称": "产业、物流与维修", "数值": 15}, {"名称": "陆路交通", "数值": 13}, {"名称": "医疗与救援", "数值": 11}, {"名称": "通信与数位韧性", "数值": 8}, {"名称": "应变指挥与警报", "数值": 6}, {"名称": "离岛持续保障", "数值": 5}], "业务入口": [{"页面": "regions", "名称": "沿岸区域", "数量": 9, "说明": "九区、县市与海岸分段"}, {"页面": "org_units", "名称": "组织与单位", "数量": 280, "说明": "组织架构图与单位深度档案"}, {"页面": "equipment", "名称": "武器装备", "数量": 99, "说明": "类别、体系、状态和公开作用"}, {"页面": "key_nodes", "名称": "关键节点", "数量": 141, "说明": "公共功能、依赖与替代"}, {"页面": "events_projects", "名称": "演习与项目", "数量": 12, "说明": "事件时间线和建设项目"}], "数据状态": {"知识库版本": "2.0", "页面版本": "2.0", "更新时间": "2026-06-23", "校验状态": "通过"}}, "regions": [{"region_id": "R01", "name": "北部沿岸", "scope": "基隆市、新北市沿岸及台北都会支撑区", "coastal_counties": ["基隆市", "新北市"], "support_areas": ["台北市", "桃园北部"], "coastal_morphology": ["野柳—东北角岬湾岩岸", "基隆港湾与人工港岸", "淡水河口沙岸与河口湿地", "八里—林口台地前缘海岸"], "segment_ids": ["S-R01-01", "S-R01-02", "S-R01-03", "S-R01-04", "S-R01-05"], "source_ids": ["SRC-WRA-TAIPEI-COAST", "SRC-TIPC-PORTS-2026", "SRC-CGA-ORG-2024", "SRC-CAA-AIRPORTS-2024"], "last_verified": "2026-06-23", "safety": "aggregate_only", "编号": "R01", "名称": "北部沿岸", "沿海县市": ["基隆市", "新北市"], "支撑区域": ["台北市", "桃园北部"], "海岸形态": ["野柳—东北角岬湾岩岸", "基隆港湾与人工港岸", "淡水河口沙岸与河口湿地", "八里—林口台地前缘海岸"], "县市子库": [{"county_id": "C-KEELUNG", "name": "基隆市", "region_id": "R01", "coastal_belt": ["基隆港城、外木山及东北角接口"], "dominant_functions": ["港口城市", "国际商港", "都会支撑"], "priority_risks": ["港城交通耦合", "强浪台风", "危险品与港区事故"], "last_verified": "2026-06-23", "safety": "aggregate_only"}, {"county_id": "C-NEWTAIPEI", "name": "新北市", "region_id": "R01", "coastal_belt": ["八里、林口、淡水、三芝、石门、金山、万里、瑞芳、贡寮"], "dominant_functions": ["台北港", "淡水河口", "北海岸观光渔业"], "priority_risks": ["海岸类型差异大", "跨都会交通", "东北角山区通道"], "last_verified": "2026-06-23", "safety": "aggregate_only"}], "海岸分段": [{"segment_id": "S-R01-01", "region_id": "R01", "name": "基隆港城段", Line truncated
@@ -0,0 +1,557 @@
{
"说明": "字段显示目录:控制页面详情字段/分区的显示与隐藏 + 是否接受定时任务抽取数据更新(keys=该字段对应的数据键;无 keys 为布局分区)。可见性存于 manifest「字段显示」,抽取锁定存于「字段抽取关闭」。",
"fields": [
{
"label": "上游依赖",
"pages": [
"关键节点"
],
"description": "该节点正常运行所依赖的上游设施",
"keys": [
"依赖类别"
]
},
{
"label": "主管或运营主体",
"pages": [
"关键节点"
],
"description": "节点的主管部门或运营单位",
"keys": [
"主管运营"
]
},
{
"label": "体系",
"pages": [
"组织与单位"
],
"description": "单位所属的组织治理体系分类",
"keys": [
"体系"
]
},
{
"label": "使用体系",
"pages": [
"武器装备"
],
"description": "装备配属使用的组织体系",
"keys": [
"使用体系中文"
]
},
{
"label": "公共功能",
"pages": [
"关键节点"
],
"description": "节点承担的公共功能",
"keys": [
"公共功能"
]
},
{
"label": "公开作用",
"pages": [
"武器装备"
],
"description": "装备公开承认的作用与用途",
"keys": [
"公开作用"
]
},
{
"label": "公开状态",
"pages": [
"武器装备"
],
"description": "装备的公开状态(在役、测试、规划等)",
"keys": [
"状态中文"
]
},
{
"label": "公开职掌与任务",
"pages": [
"组织与单位"
],
"description": "单位公开的职责与承担的任务",
"keys": [
"公开任务"
]
},
{
"label": "军令或任务上级",
"pages": [
"组织与单位"
],
"description": "其军令或任务线条上的上级单位",
"keys": [
"军令或任务上级"
]
},
{
"label": "分析限制",
"pages": [
"演习与项目"
],
"description": "对单次演习/项目的分析局限说明",
"keys": []
},
{
"label": "别名",
"pages": [
"武器装备"
],
"description": "装备的其它常用名称或编号",
"keys": [
"aliases"
]
},
{
"label": "区域",
"pages": [
"演习与项目"
],
"description": "与该条目相关的地理区域",
"keys": [
"区域中文"
]
},
{
"label": "区域、装备与演训关联",
"pages": [
"组织与单位"
],
"description": "与该单位相关的区域、装备及演习事件",
"keys": []
},
{
"label": "区域关联",
"pages": [
"组织与单位"
],
"description": "单位关联的沿岸区域",
"keys": [
"区域关联"
]
},
{
"label": "区域基本信息",
"pages": [
"沿岸区域"
],
"description": "区域的名称、编号与总体概况",
"keys": []
},
{
"label": "历史沿革与说明",
"pages": [
"组织与单位"
],
"description": "单位的历史变迁与补充说明",
"keys": [
"历史沿革"
]
},
{
"label": "县市子库",
"pages": [
"沿岸区域"
],
"description": "该区域包含的沿海县市清单",
"keys": [
"coastal_counties"
]
},
{
"label": "县市数量",
"pages": [
"沿岸区域"
],
"description": "该区域下辖沿海县市的具体数量",
"keys": [
"县市数量"
]
},
{
"label": "可信度",
"pages": [
"关键节点",
"武器装备",
"组织与单位"
],
"description": "该条信息的公开来源可信程度评价",
"keys": [
"可信度中文"
]
},
{
"label": "备注",
"pages": [
"武器装备"
],
"description": "补充备注信息",
"keys": [
"notes"
]
},
{
"label": "安全边界",
"pages": [
"组织与单位"
],
"description": "该类信息仅用于公共研究、不涉及敏感细节的边界说明",
"keys": []
},
{
"label": "当前状态",
"pages": [
"组织与单位"
],
"description": "单位当前的状态(运作、撤编等)",
"keys": [
"当前状态"
]
},
{
"label": "所属区域",
"pages": [
"关键节点"
],
"description": "节点所在的行政区域",
"keys": [
"区域中文"
]
},
{
"label": "支撑区域",
"pages": [
"沿岸区域"
],
"description": "该区域提供支撑保障的区域列表",
"keys": [
"支撑区域"
]
},
{
"label": "日期",
"pages": [
"演习与项目"
],
"description": "事件或项目发生的日期",
"keys": [
"日期显示"
]
},
{
"label": "替代与备援",
"pages": [
"关键节点"
],
"description": "节点故障时的替代与备援方案",
"keys": [
"替代备援"
]
},
{
"label": "来源",
"pages": [
"关键节点",
"武器装备",
"演习与项目"
],
"description": "资料出处(公开报道、官方文件等)",
"keys": [
"来源信息"
]
},
{
"label": "来源与核实",
"pages": [
"组织与单位"
],
"description": "资料出处以及核实情况的说明",
"keys": []
},
{
"label": "来源数量",
"pages": [
"组织与单位"
],
"description": "该条信息引用的公开来源数量",
"keys": [
"来源"
]
},
{
"label": "架构层级",
"pages": [
"组织与单位"
],
"description": "单位在组织架构中的层级",
"keys": [
"架构层级"
]
},
{
"label": "核实日期",
"pages": [
"武器装备",
"沿岸区域",
"组织与单位"
],
"description": "信息最近一次核实的时间",
"keys": [
"核实日期"
]
},
{
"label": "沿海县市",
"pages": [
"沿岸区域"
],
"description": "区域内的沿海县市名称列表",
"keys": [
"沿海县市"
]
},
{
"label": "海岸分段数量",
"pages": [
"沿岸区域"
],
"description": "海岸线被划分成研究分段的数目",
"keys": [
"分段数量"
]
},
{
"label": "海岸形态",
"pages": [
"沿岸区域"
],
"description": "海岸的地理形态分类(如沙滩、礁岸等)",
"keys": [
"海岸形态"
]
},
{
"label": "海岸研究分段",
"pages": [
"沿岸区域"
],
"description": "按海岸形态划分的研究分段明细",
"keys": [
"segment_ids"
]
},
{
"label": "演训与事件",
"pages": [
"组织与单位"
],
"description": "与该单位相关的演习与事件",
"keys": [
"演训与事件"
]
},
{
"label": "状态",
"pages": [
"演习与项目"
],
"description": "当前状态(如进行中、已完成等)",
"keys": [
"状态中文"
]
},
{
"label": "生命周期解释",
"pages": [
"武器装备"
],
"description": "对装备所处生命周期阶段的解释说明",
"keys": []
},
{
"label": "直接下级机构或单位",
"pages": [
"组织与单位"
],
"description": "直接下辖的下级机构或单位列表",
"keys": [
"下级机构"
]
},
{
"label": "研究边界",
"pages": [
"关键节点",
"沿岸区域"
],
"description": "本页面内容的研究范围与不涉及事项说明",
"keys": []
},
{
"label": "空间范围",
"pages": [
"沿岸区域"
],
"description": "该区域覆盖的地理空间范围描述",
"keys": [
"scope"
]
},
{
"label": "组织与单位概况",
"pages": [
"组织与单位"
],
"description": "单位的基本属性概览",
"keys": []
},
{
"label": "组织关系",
"pages": [
"组织与单位"
],
"description": "单位的上级、下级与协作关系",
"keys": []
},
{
"label": "组织或单位类型",
"pages": [
"组织与单位"
],
"description": "条目的类型(机构、单位、节点档案等)",
"keys": [
"单位类型"
]
},
{
"label": "节点层级",
"pages": [
"关键节点"
],
"description": "节点在基础设施体系中的层级",
"keys": [
"层级中文"
]
},
{
"label": "节点概况",
"pages": [
"关键节点"
],
"description": "关键节点的基本属性概览",
"keys": []
},
{
"label": "节点类别",
"pages": [
"关键节点"
],
"description": "节点的类别(港口、能源、通信等)",
"keys": [
"类别中文"
]
},
{
"label": "行政上级",
"pages": [
"组织与单位"
],
"description": "行政隶属关系上的上级单位",
"keys": [
"行政上级"
]
},
{
"label": "装备概况",
"pages": [
"武器装备"
],
"description": "装备的基本属性概览",
"keys": []
},
{
"label": "装备类别",
"pages": [
"武器装备",
"组织与单位"
],
"description": "相关装备的类别划分",
"keys": [
"类别中文"
]
},
{
"label": "记录类别",
"pages": [
"组织与单位"
],
"description": "条目在库中的记录分类",
"keys": [
"记录类别"
]
},
{
"label": "资料来源",
"pages": [
"沿岸区域"
],
"description": "区域资料的公开来源",
"keys": []
},
{
"label": "资料缺口",
"pages": [
"组织与单位"
],
"description": "尚缺失、待补充的资料说明",
"keys": [
"资料缺口"
]
},
{
"label": "资料说明",
"pages": [
"关键节点"
],
"description": "资料的补充说明文字",
"keys": [
"note"
]
},
{
"label": "资料边界",
"pages": [
"武器装备"
],
"description": "装备资料仅基于公开信息的范围说明",
"keys": []
},
{
"label": "运行状态",
"pages": [
"关键节点"
],
"description": "节点当前的运行状态",
"keys": [
"状态中文"
]
},
{
"label": "适用范围",
"pages": [
"组织与单位"
],
"description": "该内容适用的范围",
"keys": [
"适用范围"
]
}
]
}
@@ -0,0 +1 @@
{"系统": {"名称": "台湾滨海防卫知识数据展示系统", "版本": "2.0", "知识库版本": "2.0", "更新时间": "2026-06-23", "主题": {"模式": "红白", "主色": "#c8102e", "深红": "#8f1024", "浅红": "#fff1f3", "背景": "#f7f7f8", "卡片": "#ffffff", "文字": "#262626"}}, "页面": [{"编号": "overview", "名称": "总览", "数据源": "data/overview.json"}, {"编号": "regions", "名称": "沿岸区域", "数据源": "data/regions.json"}, {"编号": "org_units", "名称": "组织与单位", "数据源": "data/org_units.json", "架构数据": "data/organization_charts.json"}, {"编号": "equipment", "名称": "武器装备", "数据源": "data/equipment.json"}, {"编号": "key_nodes", "名称": "关键节点", "数据源": "data/key_nodes.json"}, {"编号": "events_projects", "名称": "演习与项目", "数据源": "data/events_projects.json"}], "交互规则": {"默认展示": "卡片与图形化面板", "列表限制": "不超过六个核心字段", "快速预览": "右侧抽屉", "复杂档案": "二级详情页", "长数组": "分组卡片或弹框", "查询": "各页面独立关键词、分类筛选和组合条件查询", "禁止展示": ["精确坐标", "实时部署", "兵力库存", "通信拓扑", "行动路线", "脆弱点排序"]}}
@@ -0,0 +1,336 @@
/* no.1 页面运行时(结构化模板)。
* 由 template_fill.compose_structured 内联进骨架的运行时注入点。
* 数据从骨架里的两个 JSON 元素读取:
* DATA ← <script type="application/json" id="page-data">
* MANIFEST ← <script type="application/json" id="page-manifest">
* 渲染逻辑与单文件版一致(全局函数 + inline onclick),末尾附页面 JSON 编辑器。 */
const DATA=JSON.parse(document.getElementById("page-data").textContent);
const MANIFEST=JSON.parse(document.getElementById("page-manifest").textContent);
const QUERY_SCHEMA={"版本": "2.0", "说明": "前端以中文条件构造器生成内部查询对象,不设置独立全库检索页面。", "查询域": {"沿岸区域": {"数据源": "regions", "字段": [{"字段": "名称", "显示名": "区域名称", "操作": ["包含", "等于"]}, {"字段": "沿海县市", "显示名": "沿海县市", "操作": ["包含"]}, {"字段": "海岸形态", "显示名": "海岸形态", "操作": ["包含"]}]}, "组织与单位": {"数据源": "org_units", "字段": [{"字段": "名称", "显示名": "名称", "操作": ["包含", "等于"]}, {"字段": "记录类别", "显示名": "记录类别", "操作": ["等于"]}, {"字段": "体系", "显示名": "体系", "操作": ["等于", "属于"]}, {"字段": "架构层级", "显示名": "架构层级", "操作": ["等于"]}, {"字段": "区域关联", "显示名": "区域关联", "操作": ["包含"]}, {"字段": "当前状态", "显示名": "当前状态", "操作": ["等于"]}]}, "武器装备": {"数据源": "equipment", "字段": [{"字段": "名称", "显示名": "装备名称", "操作": ["包含"]}, {"字段": "类别中文", "显示名": "装备类别", "操作": ["等于"]}, {"字段": "使用体系中文", "显示名": "使用体系", "操作": ["包含"]}, {"字段": "状态中文", "显示名": "公开状态", "操作": ["等于"]}]}, "关键节点": {"数据源": "key_nodes", "字段": [{"字段": "名称", "显示名": "节点名称", "操作": ["包含"]}, {"字段": "类别中文", "显示名": "节点类别", "操作": ["等于"]}, {"字段": "区域中文", "显示名": "所属区域", "操作": ["包含"]}, {"字段": "状态中文", "显示名": "运行状态", "操作": ["等于"]}]}, "演习与项目": {"数据源": "events_projects", "字段": [{"字段": "名称", "显示名": "名称", "操作": ["包含"]}, {"字段": "类型中文", "显示名": "事件类型", "操作": ["等于"]}, {"字段": "类别中文", "显示名": "项目类别", "操作": ["等于"]}, {"字段": "状态中文", "显示名": "状态", "操作": ["等于"]}]}}, "内部查询对象": {"数据域": "页面数据源编号", "关键词": "可选", "条件": [{"字段": "中文字段名", "操作": "包含或等于", "值": "查询值"}], "分页": {"页码": 1, "每页": 18}}};
const state={tab:"overview",mode:"chart",chart:"national_defense",q:"",filters:{},conditions:[],page:1,pageSize:18};
function e(v){return String(v??"").replace(/[&<>"']/g,m=>({"&":"&amp;","<":"&lt;",">":"&gt;",'"':"&quot;","'":"&#39;"}[m]))}
function a(v){return Array.isArray(v)?v:(v===undefined||v===null||v===""?[]:[v])}
function text(v){return Array.isArray(v)?v.join(" "):typeof v==="object"&&v?JSON.stringify(v):String(v??"")}
function chips(v,red=false,max=8){const x=a(v);return x.slice(0,max).map(y=>`<span class="chip ${red?"chip-red":""}">${e(y)}</span>`).join("")+(x.length>max?`<span class="chip">另${x.length-max}项</span>`:"")}
function statusChip(v){const green=/运行|列装|官方|确认|通过/.test(v||"");const orange=/规划|测试|建设|扩建|核验|有限/.test(v||"");return `<span class="chip ${green?"chip-green":orange?"chip-orange":"chip-red"}">${e(v||"未标注")}</span>`}
function unique(data,field){const s=new Set();data.forEach(x=>a(x[field]).forEach(v=>v&&s.add(v)));return [...s].sort()}
function setTab(tab){state.tab=tab;state.q="";state.filters={};state.conditions=[];state.page=1;location.hash="#/"+tab;render()}
function nav(){
document.getElementById("tabs").innerHTML=MANIFEST.页面.map(x=>`<button class="tab ${state.tab===x.编号?"active":""}" onclick="setTab('${x.编号}')">${x.名称}</button>`).join("")
}
function hero(title,desc,actions=""){return `<div class="hero"><div><h2>${title}</h2><p>${desc}</p><div style="margin-top:14px">${actions}</div></div><div class="hero-mark"></div></div>`}
function renderCustomModules(){
var list=a(MANIFEST.自定义模块).filter(function(m){return m&&m.页面===state.tab&&m.html});
if(!list.length)return;
var host=document.getElementById("customModules");
if(!host){host=document.createElement("div");host.id="customModules";document.getElementById("page").appendChild(host)}
host.innerHTML=list.map(function(m){return '<section class="detail-section" style="margin-top:16px"><h3>'+e(m.标题||"自定义模块")+'</h3>'+m.html+'</section>'}).join("")
}
function render(){nav();document.getElementById("page").style.display="block";document.getElementById("detailPage").classList.remove("show");const fn=window["render_"+state.tab]||render_overview;fn();renderCustomModules()}
function render_overview(){
const o=DATA.overview;
const metrics=o.核心指标.map(x=>`<div class="metric"><div class="metric-label">${x.名称}</div><div class="metric-value">${x.数值}<span class="metric-unit">${x.单位}</span></div><div class="metric-note">${x.说明}</div></div>`).join("");
const max=Math.max(...o.组织体系分布.map(x=>x.数值));
const bars=o.组织体系分布.slice(0,10).map(x=>`<div class="bar-row"><span>${e(x.名称)}</span><div class="bar"><i style="width:${x.数值/max*100}%"></i></div><b>${x.数值}</b></div>`).join("");
const entries=o.业务入口.map(x=>`<div class="entry-card" onclick="setTab('${x.页面}')"><b>${x.名称}</b><span class="entry-num">${x.数量}</span><p>${x.说明}</p></div>`).join("");
document.getElementById("page").innerHTML=hero("知识库业务总览","以区域、组织与单位、装备、公共关键节点、演习和建设项目为主线组织知识数据。")+
`<div class="metrics">${metrics}</div><div class="grid2"><div class="panel"><div class="panel-title"><span>组织体系分布</span></div>${bars}</div><div class="panel"><div class="panel-title"><span>业务入口</span></div><div class="entry-grid">${entries}</div></div></div>`;
}
function toolbar(domain,data,filterFields){
const selects=filterFields.map(f=>`<select data-filter="${f}"><option value="">${f}:全部</option>${unique(data,f).slice(0,120).map(v=>`<option value="${e(v)}" ${state.filters[f]===String(v)?"selected":""}>${e(v)}</option>`).join("")}</select>`).join("");
return `<div class="toolbar"><input id="keyword" value="${e(state.q)}" placeholder="输入关键词查询当前页面">${selects}<button class="btn btn-soft" onclick="openQueryBuilder('${domain}')">组合查询</button><button class="btn" onclick="clearQuery()">清空</button><span class="toolbar-count" id="count"></span></div>`;
}
function bindToolbar(domain,renderFn){
const q=document.getElementById("keyword");if(q)q.oninput=()=>{state.q=q.value;state.page=1;renderFn()};
document.querySelectorAll("[data-filter]").forEach(s=>s.onchange=()=>{state.filters[s.dataset.filter]=s.value;state.page=1;renderFn()});
}
function clearQuery(){state.q="";state.filters={};state.conditions=[];state.page=1;render()}
function matches(item,fields){
const q=state.q.trim().toLowerCase();
if(q&&!fields.some(f=>text(item[f]).toLowerCase().includes(q)))return false;
for(const [f,v] of Object.entries(state.filters)){if(v&&!a(item[f]).some(x=>String(x)===String(v)))return false}
for(const c of state.conditions){
const value=item[c.field],all=a(value);
if(c.op==="包含"&&!all.some(x=>String(x).includes(c.value)))return false;
if(c.op==="等于"&&!all.some(x=>String(x)===c.value))return false;
if(c.op==="属于"&&!all.some(x=>c.value.split(/[、,,]/).map(y=>y.trim()).includes(String(x))))return false;
}
return true
}
function paginate(items){const start=(state.page-1)*state.pageSize;return items.slice(start,start+state.pageSize)}
function pagination(total,rerender){const pages=Math.max(1,Math.ceil(total/state.pageSize));return `<div class="pagination"><button class="btn" ${state.page<=1?"disabled":""} onclick="state.page--;${rerender}">上一页</button><span class="page-no">第 ${state.page} / ${pages} 页</span><button class="btn" ${state.page>=pages?"disabled":""} onclick="state.page++;${rerender}">下一页</button></div>`}
function render_regions(){
const cards=DATA.regions.map(r=>`<article class="region-card" onclick="openDetail('regions','${r.编号}')"><h3>${r.编号} · ${r.名称}</h3><p>${e(r.档案摘要)}</p><div>${chips(r.海岸形态,true,3)}</div><div class="region-stats"><div><b>${r.县市数量}</b><br><small>县市子库</small></div><div><b>${r.分段数量}</b><br><small>海岸分段</small></div></div></article>`).join("");
document.getElementById("page").innerHTML=hero("沿岸区域","采用九个综合研究区,下钻到沿海县市和海岸研究分段。")+`<div class="region-grid">${cards}</div>`;
}
function render_org_units(){
const chartActive=state.mode==="chart",cardsActive=state.mode==="cards";
document.getElementById("page").innerHTML=hero("组织与单位","组织架构与单位档案使用同一套280个唯一节点。点击架构节点或机构卡片可进入完整档案。")+
`<div class="segmented"><button class="seg-btn ${chartActive?"active":""}" onclick="state.mode='chart';render_org_units()">组织架构图</button><button class="seg-btn ${cardsActive?"active":""}" onclick="state.mode='cards';render_org_units()">机构与单位卡片</button></div><div id="orgBody"></div>`;
if(chartActive)renderOrgChart();else renderOrgCards(true);
}
function renderOrgChart(){
const views=DATA.organization_charts;
const current=views.find(x=>x.id===state.chart)||views[0];
const buttons=views.map(x=>`<button class="chart-view ${x.id===current.id?"active":""}" onclick="state.chart='${x.id}';renderOrgChart()">${x.名称}</button>`).join("");
const tree=`<div class="org-chart"><ul><li>${chartNode(current.tree)}</li></ul></div>`;
const roots=DATA.org_units.filter(x=>!x.行政上级编号).length;
document.getElementById("orgBody").innerHTML=`<div class="chart-toolbar">${buttons}</div><div class="panel" style="margin-bottom:14px"><div class="panel-title"><span>${current.名称}</span></div><p style="color:#777;font-size:13px">${current.说明}</p></div><div class="summary-strip"><div class="summary-card"><b>${DATA.org_units.length}</b><span>唯一组织节点</span></div><div class="summary-card"><b>${DATA.org_units.filter(x=>x.记录类别==="单位").length}</b><span>含深度单位档案</span></div><div class="summary-card"><b>${DATA.org_units.filter(x=>x.记录类别==="机构").length}</b><span>制度与组织节点</span></div><div class="summary-card"><b>${roots}</b><span>顶层或独立根节点</span></div></div><div class="org-chart-wrap">${tree}</div>`;
}
function chartNode(n){
if(!n)return "";
const children=(n.children||[]).length?`<ul>${n.children.map(c=>`<li>${chartNode(c)}</li>`).join("")}</ul>`:"";
return `<div class="org-node" onclick="openDetail('org_units','${n.id}')"><b>${e(n.名称)}</b><small>${e(n.类型)}</small><div class="node-kind">${e(n.记录类别)} · ${e(n.状态)}</div></div>${children}`;
}
function renderOrgCards(initial=false){
const data=DATA.org_units;
if(initial){
document.getElementById("orgBody").innerHTML=toolbar("组织与单位",data,["记录类别","体系","架构层级","当前状态"])+`<div id="orgCards"></div>`;
bindToolbar("组织与单位",()=>renderOrgCards(false));
}
const filtered=data.filter(x=>matches(x,["名称","档案摘要","体系","架构层级","公开任务","区域关联","装备类别"]));
const cards=paginate(filtered).map(x=>`<article class="data-card" onclick="openDetail('org_units','${x.id}')"><div class="card-top"><div><h3>${e(x.名称)}</h3><div class="card-sub">${e(x.体系)} · ${e(x.架构层级)}</div></div><span class="card-type">${e(x.记录类别)}</span></div><p>${e(x.档案摘要)}</p><div class="card-roles">${a(x.公开任务).slice(0,2).map(y=>`<div class="role-line">${e(y)}</div>`).join("")}</div><div class="card-footer"><span>${e(x.行政上级)}</span><span>${e(x.当前状态)}</span></div></article>`).join("");
document.getElementById("orgCards").innerHTML=`<div class="card-grid">${cards||'<div class="empty">没有匹配结果</div>'}</div>${pagination(filtered.length,"renderOrgCards(false)")}`;
const count=document.getElementById("count");if(count)count.textContent=`共 ${filtered.length} 个节点`;
}
function categorySummary(data,field,onClick){
const counts={};data.forEach(x=>{const k=x[field]||"未分类";counts[k]=(counts[k]||0)+1});
return `<div class="category-grid">${Object.entries(counts).sort((a,b)=>b[1]-a[1]).slice(0,10).map(([k,v])=>`<div class="category-card" onclick="${onClick}('${e(k)}')"><b>${v}</b><span>${e(k)}</span></div>`).join("")}</div>`
}
function render_equipment(initial=true){
const data=DATA.equipment;
document.getElementById("page").innerHTML=hero("武器装备","以装备卡片展示类别、使用体系、公开作用和生命周期状态,不把规划、采购、测试和列装混为一谈。")+
categorySummary(data,"类别中文","filterEquipmentCategory")+toolbar("武器装备",data,["类别中文","状态中文","可信度中文"])+`<div id="equipmentCards"></div>`;
bindToolbar("武器装备",()=>renderEquipmentCards());
renderEquipmentCards();
}
function filterEquipmentCategory(v){state.filters["类别中文"]=v;state.page=1;render_equipment()}
function renderEquipmentCards(){
const data=DATA.equipment.filter(x=>matches(x,["名称","别名","类别中文","使用体系中文","公开作用","状态中文"]));
const html=paginate(data).map(x=>`<article class="data-card" onclick="openDetail('equipment','${x.id}')"><div class="card-top"><div><h3>${e(x.名称)}</h3><div class="card-sub">${e(x.类别中文)} · ${a(x.使用体系中文).join("、")}</div></div>${statusChip(x.状态中文)}</div><p>${e(x.档案摘要)}</p><div class="card-roles">${a(x.公开作用).slice(0,3).map(y=>`<div class="role-line">${e(y)}</div>`).join("")}</div><div class="card-footer"><span>可信度:${e(x.可信度中文)}</span><span>${e(x.verified_at||"未标注")}</span></div></article>`).join("");
document.getElementById("equipmentCards").innerHTML=`<div class="card-grid">${html||'<div class="empty">没有匹配结果</div>'}</div>${pagination(data.length,"renderEquipmentCards()")}`;
const c=document.getElementById("count");if(c)c.textContent=`共 ${data.length} 项装备`;
}
function render_key_nodes(){
const data=DATA.key_nodes;
document.getElementById("page").innerHTML=hero("关键节点","从公共功能和韧性角度展示港口、机场、交通、能源、通信、供水、医疗和产业节点。")+
categorySummary(data,"类别中文","filterNodeCategory")+toolbar("关键节点",data,["类别中文","层级中文","状态中文","可信度中文"])+`<div id="nodeCards"></div>`;
bindToolbar("关键节点",()=>renderNodeCards());renderNodeCards();
}
function filterNodeCategory(v){state.filters["类别中文"]=v;state.page=1;render_key_nodes()}
function renderNodeCards(){
const data=DATA.key_nodes.filter(x=>matches(x,["名称","类别中文","区域中文","主管运营","公共功能","状态中文"]));
const html=paginate(data).map(x=>`<article class="data-card" onclick="openDetail('key_nodes','${x.id}')"><div class="card-top"><div><h3>${e(x.名称)}</h3><div class="card-sub">${e(x.类别中文)} · ${e(x.层级中文)}</div></div>${statusChip(x.状态中文)}</div><p>${e(x.档案摘要)}</p><div>${chips(x.区域中文,true,3)}</div><div class="card-roles">${a(x.公共功能).slice(0,2).map(y=>`<div class="role-line">${e(y)}</div>`).join("")}</div><div class="card-footer"><span>${e(x.主管运营)}</span><span>可信度:${e(x.可信度中文)}</span></div></article>`).join("");
document.getElementById("nodeCards").innerHTML=`<div class="card-grid">${html||'<div class="empty">没有匹配结果</div>'}</div>${pagination(data.length,"renderNodeCards()")}`;
const c=document.getElementById("count");if(c)c.textContent=`共 ${data.length} 个节点`;
}
function render_events_projects(){
const events=DATA.events.map(x=>`<article class="time-card" onclick="openDetail('events','${x.id}')"><div class="time-date">${e(x.日期显示)}</div><h3 style="margin:0 0 7px">${e(x.名称)}</h3><div>${chips([x.类型中文,...a(x.区域中文)],true,4)}</div><p>${e(x.档案摘要)}</p></article>`).join("");
const projects=DATA.projects.map(x=>`<article class="data-card" onclick="openDetail('projects','${x.id}')"><div class="card-top"><div><h3>${e(x.名称)}</h3><div class="card-sub">${e(x.类别中文)}</div></div>${statusChip(x.状态中文)}</div><p>${e(x.档案摘要)}</p><div class="card-footer"><span>建设项目</span><span>${e(x.状态中文)}</span></div></article>`).join("");
document.getElementById("page").innerHTML=hero("演习与项目","演习采用时间线,建设项目采用卡片,分别表达日期型事件和生命周期型项目。")+
`<div class="grid2"><section class="panel"><div class="panel-title"><span>演习与重要事件</span></div><div class="timeline">${events}</div></section><section class="panel"><div class="panel-title"><span>现代化与建设项目</span></div><div class="card-grid" style="grid-template-columns:1fr">${projects}</div></section></div>`;
}
function findItem(domain,id){
if(domain==="regions")return DATA.regions.find(x=>x.编号===id);
if(domain==="org_units")return DATA.org_units.find(x=>x.id===id);
if(domain==="equipment")return DATA.equipment.find(x=>x.id===id);
if(domain==="key_nodes")return DATA.key_nodes.find(x=>x.id===id);
if(domain==="events")return DATA.events.find(x=>x.id===id);
if(domain==="projects")return DATA.projects.find(x=>x.id===id);
}
function openDetail(domain,id){
const x=findItem(domain,id);if(!x)return;
document.getElementById("page").style.display="none";
const d=document.getElementById("detailPage");d.classList.add("show");
if(domain==="org_units")d.innerHTML=detailOrg(x);
else if(domain==="regions")d.innerHTML=detailRegion(x);
else if(domain==="equipment")d.innerHTML=detailEquipment(x);
else if(domain==="key_nodes")d.innerHTML=detailNode(x);
else d.innerHTML=detailEventProject(x,domain);
location.hash=`#/detail/${domain}/${encodeURIComponent(id)}`;
window.scrollTo({top:0});
}
function back(){document.getElementById("detailPage").classList.remove("show");document.getElementById("page").style.display="block";location.hash="#/"+state.tab}
function F(label){var cfg=MANIFEST.字段显示;return !(cfg&&cfg[label]===false)}
function info(label,value){if(!F(label))return "";return `<div class="info-item"><label>${e(label)}</label><div>${value===undefined||value===null||value===""?'<span style="color:#999">未公开或未标注</span>':value}</div></div>`}
function listBlock(values){const v=a(values);return v.length?`<div class="text-list">${v.map(x=>`<div class="text-item">${e(x)}</div>`).join("")}</div>`:`<div class="empty" style="padding:20px">暂无独立条目</div>`}
function section(title,body){if(!F(title))return "";return `<section class="detail-section"><h3>${title}</h3>${body}</section>`}
function detailHeader(title,summary,tags){return `<span class="back-link" onclick="back()" style="color:var(--red);cursor:pointer;display:inline-block;margin-bottom:12px">← 返回上一页</span><div class="detail-hero"><h2>${e(title)}</h2><p>${e(summary)}</p><div class="detail-tags">${chips(tags,false,10)}</div></div>`}
function detailOrg(x){
const core=section("组织与单位概况",`<div class="info-grid">${info("记录类别",e(x.记录类别))}${info("体系",e(x.体系))}${info("架构层级",e(x.架构层级))}${info("组织或单位类型",e(x.单位类型))}${info("当前状态",statusChip(x.当前状态))}${info("可信度",e(x.可信度))}${info("行政上级",e(x.行政上级))}${info("军令或任务上级",e(x.军令或任务上级))}</div>`);
const tasks=section("公开职掌与任务",listBlock(x.公开任务));
const assoc=section("区域、装备与演训关联",`<div class="info-grid">${info("区域关联",chips(x.区域关联,true,12))}${info("装备类别",chips(x.装备类别,true,12))}${info("演训与事件",chips(x.演训与事件,true,12))}${info("适用范围",e(x.适用范围))}</div>`);
const childrenSec=section("直接下级机构或单位",x.下级机构.length?`<div class="mini-card-grid">${x.下级机构.map((n,i)=>`<div class="mini-card" ${x.下级机构编号[i]?`onclick="openDetail('org_units','${x.下级机构编号[i]}')" style="cursor:pointer"`:""}><b>${e(n)}</b><span style="font-size:11px;color:#888">点击查看档案</span></div>`).join("")}</div>`:`<div class="empty" style="padding:20px">未列出直接下级</div>`);
const relSec=section("组织关系",x.关系.length?`<div class="mini-card-grid">${x.关系.slice(0,16).map(r=>`<div class="mini-card"><b>${e(r.关系)}</b><div style="font-size:12px;line-height:1.65">${e(r.主体)} → ${e(r.客体)}</div><span style="font-size:11px;color:#888">${e(r.可信度)}</span></div>`).join("")}</div>`:`<div class="empty" style="padding:20px">暂无独立关系条目</div>`);
const history=section("历史沿革与说明",`<div class="text-item">${e(x.历史沿革)}</div>${x.说明?`<div class="text-item">${e(x.说明)}</div>`:""}`);
const gaps=section("资料缺口",`<div class="warning-box">${a(x.资料缺口).map(y=>`• ${e(y)}`).join("<br>")}</div>`);
const sources=section("来源与核实",`<div class="info-grid">${info("核实日期",e(x.核实日期))}${info("来源数量",String(a(x.来源).length))}</div><div style="margin-top:10px">${a(x.来源信息||x.来源).map(s=>`<div class="source-line">${e(s)}</div>`).join("")||'<div class="source-line">尚未列出独立来源编号</div>'}</div>`);
const safe=section("安全边界",`<div class="safe-box">${e(x.安全边界)}</div>`);
return detailHeader(x.名称,x.档案摘要,[x.记录类别,x.体系,x.当前状态])+`<div class="detail-grid"><div>${core}${tasks}${assoc}${childrenSec}${relSec}</div><aside>${history}${sources}${gaps}${safe}</aside></div>`;
}
function detailRegion(x){
const countyCards=x.县市子库.map(c=>`<div class="mini-card"><b>${e(c.name||c.county_name||c.名称||"县市子库")}</b><div style="font-size:12px;color:#777">${e(c.scope||c.沿岸范围||"沿岸县市档案")}</div></div>`).join("");
const segCards=x.海岸分段.map(s=>`<div class="mini-card"><b>${e(s.name||s.名称)}</b><div>${chips([s.coastal_type||s.海岸类型,s.所属区域],true,3)}</div><p style="font-size:11px;color:#777">${e(s.scope||s.空间范围||"")}</p></div>`).join("");
return detailHeader(x.名称,x.档案摘要,[x.编号,...a(x.海岸形态)])+
`<div class="detail-grid"><div>${section("区域基本信息",`<div class="info-grid">${info("空间范围",e(x.scope))}${info("县市数量",String(x.县市数量))}${info("海岸分段数量",String(x.分段数量))}${info("核实日期",e(x.last_verified||"未标注"))}${info("沿海县市",chips(x.沿海县市,true,15))}${info("支撑区域",chips(x.支撑区域,true,15))}</div>`)}${section("海岸形态",listBlock(x.海岸形态))}${section("海岸研究分段",`<div class="mini-card-grid">${segCards}</div>`)}</div><aside>${section("县市子库",`<div class="mini-card-grid" style="grid-template-columns:1fr">${countyCards}</div>`)}${section("资料来源",`<div>${a(x.来源信息||x.source_ids).map(s=>`<div class="source-line">${e(s)}</div>`).join("")||'<div class="source-line">待进一步补充</div>'}</div>`)}${section("研究边界",`<div class="safe-box">区域分段用于公开空间、治理和韧性研究,不等于军事防区、实际部署边界或行动分区。</div>`)}</aside></div>`;
}
function detailEquipment(x){
return detailHeader(x.名称,x.档案摘要,[x.类别中文,x.状态中文,...a(x.使用体系中文)])+
`<div class="detail-grid"><div>${section("装备概况",`<div class="info-grid">${info("装备类别",e(x.类别中文))}${info("使用体系",chips(x.使用体系中文,true,10))}${info("公开状态",statusChip(x.状态中文))}${info("可信度",e(x.可信度中文))}${info("别名",chips(x.aliases,true,10))}${info("核实日期",e(x.verified_at||"未标注"))}</div>`)}${section("公开作用",listBlock(x.公开作用))}${section("生命周期解释",`<div class="text-list"><div class="text-item">规划、预算、签约、交付、接装、训练、测评和形成能力属于不同阶段。</div><div class="text-item">本条目的当前状态为:${e(x.状态中文)}。</div><div class="text-item">装备价值需与感知、指挥、平台、保障和训练共同评估。</div></div>`)}</div><aside>${section("来源",a(x.来源信息||x.sources).map(s=>`<div class="source-line">${e(s)}</div>`).join("")||'<div class="source-line">待补</div>')}${section("备注",`<div class="text-item">${e(x.notes||"无额外公开说明。")}</div>`)}${section("资料边界",`<div class="safe-box">不记录或推断库存、精确部署、妥善率、射界、航线、战法和单位实际配赋。</div>`)}</aside></div>`;
}
function detailNode(x){
return detailHeader(x.名称,x.档案摘要,[x.类别中文,x.层级中文,x.状态中文])+
`<div class="detail-grid"><div>${section("节点概况",`<div class="info-grid">${info("节点类别",e(x.类别中文))}${info("节点层级",e(x.层级中文))}${info("所属区域",chips(x.区域中文,true,10))}${info("运行状态",statusChip(x.状态中文))}${info("主管或运营主体",e(x.主管运营))}${info("可信度",e(x.可信度中文))}</div>`)}${section("公共功能",listBlock(x.公共功能))}${section("上游依赖",listBlock(x.依赖类别))}${section("替代与备援",listBlock(x.替代备援))}</div><aside>${section("来源",a(x.来源信息||x.sources).map(s=>`<div class="source-line">${e(s)}</div>`).join("")||'<div class="source-line">待补</div>')}${section("资料说明",`<div class="text-item">${e(x.note||"无额外公开说明。")}</div>`)}${section("研究边界",`<div class="safe-box">节点用于公共功能、治理和韧性研究,不展示精确坐标、内部拓扑、实时容量、库存、脆弱点或攻击价值。</div>`)}</aside></div>`;
}
function detailEventProject(x,domain){
const isEvent=domain==="events";
return detailHeader(x.名称,x.档案摘要,[isEvent?x.类型中文:x.类别中文,x.状态中文])+
`<div class="detail-grid"><div>${section(isEvent?"事件信息":"项目信息",`<div class="info-grid">${info(isEvent?"事件类型":"项目类别",e(isEvent?x.类型中文:x.类别中文))}${info("状态",statusChip(x.状态中文))}${info("日期",e(x.日期显示||"不适用"))}${info("区域",chips(x.区域中文,true,12))}</div>`)}${section(isEvent?"主题与验证重点":"项目评估链条",isEvent?listBlock(x.topics):`<div class="text-list"><div class="text-item">规划与需求确认</div><div class="text-item">预算与采购</div><div class="text-item">生产、交付与接装</div><div class="text-item">训练、测评、保障与形成能力</div></div>`)}</div><aside>${section("来源",`<div class="source-line">${e(a(x.来源信息).join(";")||x.source||"待补")}</div>`)}${section("分析限制",`<div class="safe-box">${isEvent?"单次演习不能直接证明永久编制、完整战备或真实作战效果。":"项目存在不等于已经交付、接装或形成能力。"}</div>`)}</aside></div>`;
}
function openQueryBuilder(domainName){
const conf=QUERY_SCHEMA.查询域[domainName];if(!conf)return;
const opts=conf.字段.map(x=>`<option value="${e(x.字段)}">${e(x.显示名)}</option>`).join("");
const rows=(state.conditions.length?state.conditions:[{field:conf.字段[0].字段,op:"包含",value:""}]).map((c,i)=>queryRow(opts,c,i)).join("");
openModal(`<h2 style="margin-top:0">组合条件查询</h2><p style="color:#777;font-size:13px">使用中文字段构造条件,查询只作用于当前业务页面。</p><div id="queryRows">${rows}</div><button class="btn" onclick="addQueryRow('${domainName}')">增加条件</button> <button class="btn btn-red" onclick="applyQuery('${domainName}')">应用查询</button>`);
}
function queryRow(opts,c,i){return `<div class="query-row" data-query-row><select class="q-field">${opts.replace(`value="${c.field}"`,`value="${c.field}" selected`)}</select><select class="q-op"><option ${c.op==="包含"?"selected":""}>包含</option><option ${c.op==="等于"?"selected":""}>等于</option><option ${c.op==="属于"?"selected":""}>属于</option></select><input class="q-value" value="${e(c.value)}" placeholder="输入查询值"><button class="btn" onclick="this.parentElement.remove()">×</button></div>`}
function addQueryRow(domainName){const conf=QUERY_SCHEMA.查询域[domainName];const opts=conf.字段.map(x=>`<option value="${e(x.字段)}">${e(x.显示名)}</option>`).join("");document.getElementById("queryRows").insertAdjacentHTML("beforeend",queryRow(opts,{field:conf.字段[0].字段,op:"包含",value:""},99))}
function applyQuery(){state.conditions=[...document.querySelectorAll("[data-query-row]")].map(r=>({field:r.querySelector(".q-field").value,op:r.querySelector(".q-op").value,value:r.querySelector(".q-value").value})).filter(x=>x.value);state.page=1;closeModal();render()}
function openModal(html){document.getElementById("modal").innerHTML=html;document.getElementById("modalOverlay").classList.add("show")}
function closeModal(){document.getElementById("modalOverlay").classList.remove("show")}
function closeDrawer(){document.getElementById("drawerOverlay").classList.remove("show")}
document.getElementById("modalOverlay").onclick=x=>{if(x.target.id==="modalOverlay")closeModal()};
document.getElementById("drawerOverlay").onclick=x=>{if(x.target.id==="drawerOverlay")closeDrawer()};
/* ── 页面 JSON 编辑器 ──────────────────────────────────────────────
* 模块 = DATA 顶层键(Tab 标签取 MANIFEST.页面 的中文名)+「系统设置」编辑 MANIFEST。
* 应用 → 原地替换 DATA/MANIFEST → 重置交互状态 → render() 整体重渲染;
* 修改按数据指纹存 localStorage(沙箱 iframe 禁存储时 try/catch 静默降级,仅会话内有效),
* 不回写服务器——服务端持久化走模板管理页「编辑数据」。 */
(function(){
"use strict";
function clone(v){return JSON.parse(JSON.stringify(v))}
function draftsToData(d){var o={};Object.keys(d||{}).forEach(function(k){if(k!=="__manifest__")o[k]=d[k]});return o}
function draftsFrom(d,m){var o={};Object.keys(d||{}).forEach(function(k){o[k]=d[k]});o.__manifest__=m;return o}
function replaceInPlace(target,src){Object.keys(target).forEach(function(k){delete target[k]});Object.keys(src).forEach(function(k){target[k]=src[k]})}
var initialData=clone(DATA),initialManifest=clone(MANIFEST);
var initialModules=Object.keys(DATA).concat(["__manifest__"]);
// 本地缓存键按初始数据指纹生成:模板/产物内容变了,旧缓存自然失效。
var fingerprint=(function(){var h=2166136261,s=JSON.stringify(initialData)+"|"+JSON.stringify(initialManifest);for(var i=0;i<s.length;i++){h^=s.charCodeAt(i);h=Math.imul(h,16777619)}return (h>>>0).toString(36)})();
var storageKey="structured-page-data-v1:"+fingerprint;
function readLS(k){try{return window.localStorage?window.localStorage.getItem(k):null}catch(x){return null}}
function writeLS(k,v){try{if(window.localStorage)window.localStorage.setItem(k,v);return true}catch(x){return false}}
function removeLS(k){try{if(window.localStorage)window.localStorage.removeItem(k)}catch(x){}}
// 启动恢复:有合法缓存则应用(本段在首帧 render() 之前执行)。
(function(){
var cached=readLS(storageKey);if(!cached)return;
try{
var parsed=JSON.parse(cached);
if(parsed&&typeof parsed==="object"&&parsed.data&&parsed.manifest&&initialModules.every(function(k){
return k==="__manifest__"||typeof parsed.data[k]==="object"&&parsed.data[k]&&!Array.isArray(parsed.data[k]);
})){
replaceInPlace(DATA,parsed.data);replaceInPlace(MANIFEST,parsed.manifest)
}
}catch(x){/* 缓存损坏则忽略,沿用模板默认数据 */}
})();
var labels={};a(MANIFEST.页面).forEach(function(p){if(p&&p.编号)labels[p.编号]=p.名称||p.编号});
var modules=Object.keys(DATA).concat(["__manifest__"]);
var active=modules[0]||"__manifest__";
var drafts=draftsFrom(clone(DATA),clone(MANIFEST));
function moduleTitle(key){return key==="__manifest__"?"系统设置(MANIFEST)":(labels[key]||key)}
function validDrafts(){return modules.every(function(k){var v=drafts[k];return v&&typeof v==="object"&&!Array.isArray(v)})}
var overlay=document.getElementById("editorOverlay");
if(!overlay)return; // 骨架缺编辑器标记(被手工裁掉)时安全退出
var tabsEl=document.getElementById("editorTabs");
var editor=document.getElementById("jsonEditor");
var statusEl=document.getElementById("jsonStatus");
var titleEl=document.getElementById("editorModuleTitle");
function setStatus(ok,msg){statusEl.textContent=msg;statusEl.className="json-status "+(ok?"valid":"invalid")}
function readEditor(){
try{
var parsed=JSON.parse(editor.value);
if(!parsed||typeof parsed!=="object"||Array.isArray(parsed))throw new Error("模块必须是 JSON 对象");
setStatus(true,"JSON 有效");return parsed
}catch(x){setStatus(false,"JSON 无效:"+x.message);return null}
}
function saveActiveDraft(){var parsed=readEditor();if(!parsed)return false;drafts[active]=parsed;return true}
function loadEditor(){titleEl.textContent=moduleTitle(active);editor.value=JSON.stringify(drafts[active],null,2);setStatus(true,"JSON 有效")}
function renderTabs(){
tabsEl.innerHTML="";
modules.forEach(function(key){
var b=document.createElement("button");b.type="button";
b.className="editor-tab"+(key===active?" active":"");
b.innerHTML="<b>"+e(moduleTitle(key))+"</b><small>"+(key==="__manifest__"?"MANIFEST":key)+"</small>";
b.onclick=function(){if(key===active||!saveActiveDraft())return;active=key;renderTabs();loadEditor()};
tabsEl.appendChild(b)
})
}
function closeEditor(){overlay.classList.remove("show")}
function applyToPage(){
if(!saveActiveDraft()||!validDrafts()){setStatus(false,"存在无效模块,请先修正");return}
replaceInPlace(DATA,draftsToData(drafts));
replaceInPlace(MANIFEST,drafts.__manifest__);
// 重置交互状态后整体重渲染(tab 回到第一个页面)。
state.tab=(MANIFEST.页面&&MANIFEST.页面[0]&&MANIFEST.页面[0].编号)||"overview";
state.mode="chart";state.chart="national_defense";state.q="";state.filters={};state.conditions=[];state.page=1;
closeEditor();render();syncHeader();
writeLS(storageKey,JSON.stringify({data:DATA,manifest:MANIFEST}))
}
function legacyCopy(text){var t=document.createElement("textarea");t.value=text;document.body.appendChild(t);t.select();try{document.execCommand("copy")}catch(x){}t.remove()}
function copyText(text,done){
if(navigator.clipboard&&navigator.clipboard.writeText){navigator.clipboard.writeText(text).then(done,function(){legacyCopy(text);done()})}
else{legacyCopy(text);done()}
}
document.getElementById("openEditor").onclick=function(){
drafts=draftsFrom(clone(DATA),clone(MANIFEST));
active=modules[0]||"__manifest__";
renderTabs();loadEditor();
overlay.classList.add("show");
setTimeout(function(){editor.focus()},40)
};
document.getElementById("closeEditor").onclick=closeEditor;
overlay.onclick=function(ev){if(ev.target===overlay)closeEditor()};
document.addEventListener("keydown",function(ev){if(ev.key==="Escape"&&overlay.classList.contains("show"))closeEditor()});
document.getElementById("applyJson").onclick=applyToPage;
document.getElementById("resetJson").onclick=function(){
drafts=draftsFrom(clone(initialData),clone(initialManifest));
active=modules[0]||"__manifest__";
removeLS(storageKey);
renderTabs();loadEditor();
setStatus(true,"已恢复默认数据,点击「应用到页面」生效")
};
document.getElementById("copyJson").onclick=function(){
if(!saveActiveDraft())return;
copyText(JSON.stringify({data:draftsToData(drafts),manifest:drafts.__manifest__},null,2),function(){})
};
document.getElementById("downloadJson").onclick=function(){
if(!saveActiveDraft())return;
var blob=new Blob([JSON.stringify({data:draftsToData(drafts),manifest:drafts.__manifest__},null,2)],{type:"application/json;charset=utf-8"});
var link=document.createElement("a");link.href=URL.createObjectURL(blob);link.download="page-data.json";link.click();
setTimeout(function(){URL.revokeObjectURL(link.href)},1000)
};
})();
function syncHeader(){
var sys=MANIFEST.系统||{};
if(sys.名称){document.title=String(sys.名称);var h1=document.querySelector(".title h1");if(h1)h1.textContent=String(sys.名称)}
var hs=document.querySelector(".header-state");
if(hs){var ver=sys.知识库版本,upd=sys.更新时间;if(ver||upd)hs.innerHTML="知识库版本 "+e(ver||"")+"<br>更新日期 "+e(upd||"")}
}
syncHeader();
const hash=location.hash.match(/^#\/([^/]+)/);if(hash&&MANIFEST.页面.some(x=>x.编号===hash[1]))state.tab=hash[1];render();
@@ -0,0 +1,98 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>台湾滨海防卫知识数据展示系统</title>
<style>
:root{
--red:#c8102e;--red2:#8f1024;--light-red:#fff1f3;--soft-red:#fde7eb;
--bg:#f5f5f6;--card:#fff;--text:#262626;--muted:#737373;--line:#e5e5e5;
--green:#168a4a;--orange:#c56a00;--shadow:0 8px 24px rgba(74,12,24,.08)
}
*{box-sizing:border-box}html{scroll-behavior:smooth}body{margin:0;background:var(--bg);color:var(--text);font-family:"PingFang SC","Microsoft YaHei",Arial,sans-serif}
button,input,select{font:inherit}.header{position:sticky;top:0;z-index:30;background:#fff;border-bottom:1px solid var(--line);box-shadow:0 3px 12px rgba(0,0,0,.04)}
.header-top{height:70px;padding:0 28px;display:flex;align-items:center;gap:24px}.logo{width:42px;height:42px;border-radius:12px;background:linear-gradient(145deg,var(--red),var(--red2));display:grid;place-items:center;color:#fff;font-weight:800}
.title{flex:1}.title h1{font-size:19px;margin:0 0 5px}.title p{margin:0;font-size:12px;color:var(--muted)}.header-state{text-align:right;font-size:12px;color:var(--muted)}
.tabs{display:flex;padding:0 24px;gap:4px;overflow:auto}.tab{border:0;background:transparent;padding:13px 18px;color:#666;cursor:pointer;border-bottom:3px solid transparent;white-space:nowrap}.tab.active{color:var(--red);font-weight:700;border-bottom-color:var(--red);background:linear-gradient(to top,var(--light-red),transparent)}
.main{padding:24px 28px 48px}.hero{background:linear-gradient(135deg,#fff,#fff5f6);border:1px solid #f2d4d9;border-radius:20px;padding:24px;display:flex;justify-content:space-between;align-items:center;gap:20px;margin-bottom:18px;box-shadow:var(--shadow)}
.hero h2{margin:0 0 8px;font-size:26px}.hero p{margin:0;color:var(--muted);line-height:1.7}.hero-mark{width:90px;height:90px;border-radius:50%;background:radial-gradient(circle,#fff 0 30%,#fbd5dc 31% 55%,var(--red) 56%);opacity:.9}
.btn{border:1px solid #d8d8d8;background:#fff;color:#333;border-radius:9px;padding:8px 13px;cursor:pointer}.btn:hover{border-color:var(--red);color:var(--red)}.btn-red{background:var(--red);border-color:var(--red);color:#fff}.btn-red:hover{background:var(--red2);color:#fff}.btn-soft{background:var(--light-red);color:var(--red);border-color:#f0bec8}
.metrics{display:grid;grid-template-columns:repeat(3,1fr);gap:14px;margin-bottom:18px}.metric{background:#fff;border:1px solid var(--line);border-radius:16px;padding:17px;box-shadow:var(--shadow);position:relative;overflow:hidden}.metric:after{content:"";position:absolute;right:-20px;top:-30px;width:90px;height:90px;border-radius:50%;background:var(--light-red)}.metric-label{font-size:13px;color:var(--muted)}.metric-value{font-size:30px;font-weight:800;color:var(--red);margin-top:8px}.metric-unit{font-size:12px;margin-left:4px}.metric-note{font-size:12px;color:#888;margin-top:6px}
.grid2{display:grid;grid-template-columns:1fr 1fr;gap:16px}.panel{background:#fff;border:1px solid var(--line);border-radius:17px;padding:18px;box-shadow:var(--shadow)}.panel-title{font-weight:800;margin-bottom:14px;display:flex;align-items:center;justify-content:space-between}.panel-title:before{content:"";width:5px;height:18px;background:var(--red);border-radius:5px;margin-right:9px}.panel-title span:first-child{display:flex;align-items:center}
.bar-row{display:grid;grid-template-columns:150px 1fr 38px;gap:10px;align-items:center;font-size:12px;margin:10px 0}.bar{height:9px;background:#f1f1f1;border-radius:20px;overflow:hidden}.bar i{display:block;height:100%;background:linear-gradient(90deg,var(--red),#ed6b7e);border-radius:20px}
.entry-grid,.card-grid{display:grid;grid-template-columns:repeat(3,1fr);gap:14px}.entry-card,.data-card{background:#fff;border:1px solid var(--line);border-radius:16px;padding:17px;box-shadow:var(--shadow);cursor:pointer;transition:.18s}.entry-card:hover,.data-card:hover{transform:translateY(-2px);border-color:#e0a0ab}.entry-card b,.data-card h3{display:block;margin:0 0 8px}.entry-num{font-size:25px;color:var(--red);font-weight:800}.entry-card p,.data-card p{font-size:12px;color:var(--muted);line-height:1.65;margin:8px 0 0}
.toolbar{background:#fff;border:1px solid var(--line);border-radius:14px;padding:12px;display:flex;gap:9px;align-items:center;flex-wrap:wrap;margin-bottom:16px;box-shadow:var(--shadow)}.toolbar input,.toolbar select{height:37px;border:1px solid #ddd;border-radius:8px;background:#fff;padding:0 10px;color:#333}.toolbar input{flex:1;min-width:250px}.toolbar-count{margin-left:auto;color:var(--muted);font-size:12px}
.chip{display:inline-flex;align-items:center;border-radius:999px;padding:4px 9px;background:#f3f3f3;color:#555;font-size:11px;margin:2px}.chip-red{background:var(--light-red);color:var(--red)}.chip-green{background:#edf9f2;color:var(--green)}.chip-orange{background:#fff6e8;color:var(--orange)}
.region-grid{display:grid;grid-template-columns:repeat(3,1fr);gap:15px}.region-card{background:#fff;border:1px solid var(--line);border-radius:18px;padding:18px;box-shadow:var(--shadow);cursor:pointer;position:relative;overflow:hidden}.region-card:before{content:"";position:absolute;left:0;top:0;bottom:0;width:5px;background:var(--red)}.region-card h3{margin:0 0 8px;font-size:18px}.region-card p{font-size:12px;color:var(--muted);line-height:1.7}.region-stats{display:flex;gap:18px;margin-top:13px;border-top:1px solid #f0f0f0;padding-top:12px}.region-stats b{color:var(--red);font-size:18px}
.segmented{display:flex;gap:5px;background:#f0f0f0;padding:4px;border-radius:11px;width:max-content;margin-bottom:15px}.seg-btn{border:0;background:transparent;padding:8px 14px;border-radius:8px;color:#666;cursor:pointer}.seg-btn.active{background:#fff;color:var(--red);font-weight:700;box-shadow:0 2px 7px rgba(0,0,0,.08)}
.chart-toolbar{display:flex;gap:8px;flex-wrap:wrap;margin-bottom:16px}.chart-view{padding:9px 14px;border:1px solid #e1c3c9;background:#fff7f8;color:#8c2335;border-radius:9px;cursor:pointer}.chart-view.active{background:var(--red);color:#fff;border-color:var(--red)}
.org-chart-wrap{overflow:auto;background:#fff;border:1px solid var(--line);border-radius:18px;padding:28px;box-shadow:var(--shadow);min-height:420px}
.org-chart{display:flex;justify-content:center;min-width:max-content}.org-chart ul{padding-top:24px;position:relative;display:flex;justify-content:center;margin:0;padding-left:0}.org-chart li{list-style:none;text-align:center;position:relative;padding:24px 8px 0}.org-chart li:before,.org-chart li:after{content:"";position:absolute;top:0;right:50%;border-top:1px solid #c7a8ae;width:50%;height:24px}.org-chart li:after{right:auto;left:50%;border-left:1px solid #c7a8ae}.org-chart li:only-child:after,.org-chart li:only-child:before{display:none}.org-chart li:only-child{padding-top:0}.org-chart li:first-child:before,.org-chart li:last-child:after{border:0}.org-chart li:last-child:before{border-right:1px solid #c7a8ae;border-radius:0 6px 0 0}.org-chart li:first-child:after{border-radius:6px 0 0 0}.org-chart ul ul:before{content:"";position:absolute;top:0;left:50%;border-left:1px solid #c7a8ae;width:0;height:24px}
.org-node{width:168px;min-height:80px;background:#fff;border:1px solid #e3c1c7;border-top:4px solid var(--red);border-radius:12px;padding:10px;display:inline-block;box-shadow:0 5px 14px rgba(96,12,28,.08);cursor:pointer}.org-node:hover{background:#fff6f7;border-color:var(--red)}.org-node b{font-size:13px;display:block;line-height:1.45}.org-node small{font-size:10px;color:var(--muted);display:block;margin-top:5px}.org-node .node-kind{font-size:10px;color:var(--red);margin-top:5px}
.summary-strip{display:grid;grid-template-columns:repeat(4,1fr);gap:12px;margin:15px 0}.summary-card{background:#fff;border:1px solid var(--line);border-radius:13px;padding:14px}.summary-card b{display:block;color:var(--red);font-size:21px}.summary-card span{font-size:12px;color:var(--muted)}
.data-card{display:flex;flex-direction:column;min-height:205px}.card-top{display:flex;justify-content:space-between;gap:10px}.card-type{font-size:11px;color:var(--red);background:var(--light-red);padding:4px 8px;border-radius:999px;height:max-content}.card-sub{font-size:12px;color:#777;margin-bottom:10px}.card-roles{display:grid;gap:6px;margin-top:8px}.role-line{font-size:12px;background:#fafafa;border-left:3px solid #e7a1ad;padding:7px 9px}.card-footer{margin-top:auto;padding-top:12px;border-top:1px solid #eee;display:flex;justify-content:space-between;color:#888;font-size:11px}
.category-grid{display:grid;grid-template-columns:repeat(5,1fr);gap:10px;margin-bottom:16px}.category-card{background:#fff;border:1px solid var(--line);border-radius:13px;padding:13px;cursor:pointer}.category-card:hover{border-color:var(--red)}.category-card b{color:var(--red);font-size:21px;display:block}.category-card span{font-size:11px;color:#666}
.timeline{position:relative;padding-left:26px}.timeline:before{content:"";position:absolute;left:7px;top:4px;bottom:4px;width:2px;background:#efc5cc}.time-card{position:relative;background:#fff;border:1px solid var(--line);border-radius:14px;padding:16px;margin-bottom:14px;box-shadow:var(--shadow);cursor:pointer}.time-card:before{content:"";position:absolute;left:-25px;top:19px;width:12px;height:12px;border:3px solid var(--red);background:#fff;border-radius:50%}.time-date{color:var(--red);font-size:12px;font-weight:700;margin-bottom:5px}
.pagination{display:flex;justify-content:center;align-items:center;gap:9px;margin-top:18px}.page-no{color:#777;font-size:12px}
.overlay{position:fixed;inset:0;background:rgba(36,8,14,.42);z-index:50;display:none}.overlay.show{display:block}.drawer{position:absolute;right:0;top:0;width:min(560px,94vw);height:100%;background:#fff;padding:24px;overflow:auto;box-shadow:-12px 0 35px rgba(0,0,0,.16)}.modal{position:absolute;left:50%;top:50%;transform:translate(-50%,-50%);width:min(840px,92vw);max-height:88vh;overflow:auto;background:#fff;border-radius:18px;padding:24px}.close{float:right}
.detail-page{display:none}.detail-page.show{display:block}.detail-hero{background:linear-gradient(135deg,var(--red),var(--red2));color:#fff;border-radius:20px;padding:25px;margin-bottom:16px;box-shadow:var(--shadow)}.detail-hero h2{margin:0 0 10px;font-size:26px}.detail-hero p{margin:0;line-height:1.7;color:#ffe8ed}.detail-tags{margin-top:13px}.detail-tags .chip{background:rgba(255,255,255,.16);color:#fff}
.detail-grid{display:grid;grid-template-columns:2fr 1fr;gap:16px}.detail-section{background:#fff;border:1px solid var(--line);border-radius:16px;padding:18px;margin-bottom:16px;box-shadow:var(--shadow)}.detail-section h3{margin:0 0 14px;font-size:16px;display:flex;align-items:center}.detail-section h3:before{content:"";width:4px;height:17px;background:var(--red);border-radius:5px;margin-right:8px}.info-grid{display:grid;grid-template-columns:repeat(2,1fr);gap:10px}.info-item{background:#fafafa;border-radius:10px;padding:11px}.info-item label{display:block;font-size:11px;color:#888;margin-bottom:5px}.info-item div{font-size:13px;line-height:1.55}.text-list{display:grid;gap:9px}.text-item{padding:10px 12px;border-left:4px solid #e5a6b1;background:#fff8f9;border-radius:7px;font-size:13px;line-height:1.6}
.mini-card-grid{display:grid;grid-template-columns:repeat(2,1fr);gap:10px}.mini-card{border:1px solid #eee;border-radius:11px;padding:12px;background:#fff}.mini-card b{display:block;margin-bottom:6px}.source-line{padding:9px 0;border-bottom:1px dashed #ddd;font-size:12px}.warning-box{background:#fff7e8;border:1px solid #f1d29e;border-radius:11px;padding:13px;color:#7a4d00;font-size:12px;line-height:1.65}.safe-box{background:#f7f7f7;border:1px solid #ddd;border-radius:11px;padding:13px;color:#555;font-size:12px;line-height:1.65}
.query-row{display:grid;grid-template-columns:1fr 120px 1fr 42px;gap:8px;margin-bottom:8px}.query-row select,.query-row input{height:37px;border:1px solid #ddd;border-radius:8px;padding:0 9px}.empty{text-align:center;padding:50px;color:#888}
@media(max-width:1150px){.entry-grid,.card-grid,.region-grid{grid-template-columns:repeat(2,1fr)}.category-grid{grid-template-columns:repeat(3,1fr)}.detail-grid{grid-template-columns:1fr}.metrics{grid-template-columns:repeat(2,1fr)}}
@media(max-width:720px){.header-top{padding:10px 14px;height:auto}.header-state{display:none}.main{padding:14px}.hero-mark{display:none}.hero h2{font-size:21px}.metrics,.grid2,.entry-grid,.card-grid,.region-grid,.summary-strip,.category-grid,.info-grid,.mini-card-grid{grid-template-columns:1fr}.toolbar input{min-width:100%}.query-row{grid-template-columns:1fr}.org-chart-wrap{padding:15px}.org-node{width:145px}}
/* ── 页面 JSON 编辑器 ── */
.editor-fab{position:fixed;right:18px;bottom:26px;z-index:40;background:var(--red);color:#fff;border:0;border-radius:50%;width:46px;height:46px;display:grid;place-items:center;cursor:pointer;box-shadow:0 6px 18px rgba(143,16,36,.32)}.editor-fab:hover{background:var(--red2);transform:rotate(30deg)}.editor-fab svg{width:22px;height:22px;display:block}
.editor-overlay{z-index:60}
.editor-modal{position:absolute;left:50%;top:50%;transform:translate(-50%,-50%);width:min(1080px,94vw);height:min(680px,88vh);background:#fff;border-radius:18px;display:flex;flex-direction:column;overflow:hidden;box-shadow:0 24px 60px rgba(36,8,14,.3)}
.editor-head{display:flex;align-items:center;gap:12px;padding:14px 18px;border-bottom:1px solid var(--line)}.editor-head h2{margin:0;font-size:16px;flex:1}.editor-head .editor-note{font-size:12px;color:var(--muted);margin-right:8px}
.editor-body{display:flex;flex:1;min-height:0}
.editor-tabs{width:210px;border-right:1px solid var(--line);overflow:auto;padding:10px 8px;display:flex;flex-direction:column;gap:4px}
.editor-tab{border:0;background:transparent;text-align:left;padding:9px 11px;border-radius:9px;cursor:pointer;color:#555}.editor-tab b{display:block;font-size:13px}.editor-tab small{font-size:11px;color:#999}.editor-tab:hover{background:var(--light-red)}.editor-tab.active{background:var(--light-red);color:var(--red)}
.editor-pane{flex:1;display:flex;flex-direction:column;min-width:0;padding:14px 16px;gap:10px}
.editor-title-row{display:flex;align-items:center;justify-content:space-between}.editor-title-row b{font-size:14px}
.json-status{font-size:12px;padding:3px 10px;border-radius:999px}.json-status.valid{background:#edf9f2;color:var(--green)}.json-status.invalid{background:var(--light-red);color:var(--red)}
.json-editor{flex:1;width:100%;resize:none;border:1px solid var(--line);border-radius:12px;padding:13px;font:13px/1.6 ui-monospace,Consolas,"Courier New",monospace;color:#333;background:#fafafa;outline:0;tab-size:2}.json-editor:focus{border-color:var(--red);box-shadow:0 0 0 3px rgba(200,16,46,.1)}
.editor-actions{display:flex;gap:8px;flex-wrap:wrap}
.editor-hint{margin:0;font-size:11px;color:#999}
@media(max-width:720px){.editor-modal{width:96vw;height:92vh}.editor-body{flex-direction:column}.editor-tabs{width:auto;flex-direction:row;overflow-x:auto;border-right:0;border-bottom:1px solid var(--line)}.editor-tab{white-space:nowrap}}
</style>
</head>
<body>
<header class="header">
<div class="header-top">
<div class="logo">防</div>
<div class="title"><h1>台湾滨海防卫知识数据展示系统</h1><p>红白业务版 · 组织架构与单位档案已合并</p></div>
<div class="header-state">知识库版本 2.0<br>更新日期 2026-06-23</div>
</div>
<nav class="tabs" id="tabs"></nav>
</header>
<main class="main">
<section id="page"></section>
<section id="detailPage" class="detail-page"></section>
</main>
<div id="drawerOverlay" class="overlay"><aside class="drawer"><button class="btn close" onclick="closeDrawer()">关闭</button><div id="drawer"></div></aside></div>
<div id="modalOverlay" class="overlay"><div class="modal"><button class="btn close" onclick="closeModal()">关闭</button><div id="modal"></div></div></div>
<button type="button" id="openEditor" class="editor-fab" aria-label="JSON 编辑器" title="页面 JSON 编辑器"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M12.22 2h-.44a2 2 0 0 0-2 2v.18a2 2 0 0 1-1 1.73l-.43.25a2 2 0 0 1-2 0l-.15-.08a2 2 0 0 0-2.73.73l-.22.38a2 2 0 0 0 .73 2.73l.15.1a2 2 0 0 1 1 1.72v.51a2 2 0 0 1-1 1.74l-.15.09a2 2 0 0 0-.73 2.73l.22.38a2 2 0 0 0 2.73.73l.15-.08a2 2 0 0 1 2 0l.43.25a2 2 0 0 1 1 1.73V20a2 2 0 0 0 2 2h.44a2 2 0 0 0 2-2v-.18a2 2 0 0 1 1-1.73l.43-.25a2 2 0 0 1 2 0l.15.08a2 2 0 0 0 2.73-.73l.22-.39a2 2 0 0 0-.73-2.73l-.15-.08a2 2 0 0 1-1-1.74v-.5a2 2 0 0 1 1-1.74l.15-.09a2 2 0 0 0 .73-2.73l-.22-.38a2 2 0 0 0-2.73-.73l-.15.08a2 2 0 0 1-2 0l-.43-.25a2 2 0 0 1-1-1.73V4a2 2 0 0 0-2-2z"/><circle cx="12" cy="12" r="3"/></svg></button>
<div id="editorOverlay" class="overlay editor-overlay">
<div class="editor-modal">
<header class="editor-head"><h2>页面 JSON 编辑器</h2><span class="editor-note">按模块编辑,应用后即时重渲染;修改保存在浏览器本地,不回写服务器</span><button type="button" class="btn" id="closeEditor">关闭</button></header>
<div class="editor-body">
<nav id="editorTabs" class="editor-tabs"></nav>
<div class="editor-pane">
<div class="editor-title-row"><b id="editorModuleTitle"></b><span id="jsonStatus" class="json-status valid">JSON 有效</span></div>
<textarea id="jsonEditor" class="json-editor" spellcheck="false" aria-label="当前模块 JSON"></textarea>
<div class="editor-actions"><button type="button" class="btn btn-red" id="applyJson">应用到页面</button><button type="button" class="btn" id="resetJson">恢复默认数据</button><button type="button" class="btn" id="copyJson">复制完整 JSON</button><button type="button" class="btn" id="downloadJson">下载 JSON</button></div>
<p class="editor-hint">模块 = 页面数据(DATA)的顶层键,另附「系统设置」编辑 MANIFEST;沙箱预览(iframe)无法使用本地缓存时,修改仅当次会话有效。</p>
</div>
</div>
</div>
</div>
<script type="application/json" id="page-data">__PAGE_DATA__</script>
<script type="application/json" id="page-manifest">__PAGE_MANIFEST__</script>
<script>__PAGE_RUNTIME__</script>
</body>
</html>
@@ -314,43 +314,41 @@ def _resolve_skills_path(path: str) -> str:
return _join_path_preserving_style(skills_host, relative)
def _custom_skill_name_from_path(path: str) -> str | None:
"""从虚拟技能路径里解析出 custom 技能名,用于"被使用"埋点。
只识别 ``{skills_container}/custom/{name}/...`` 形式 —— public 技能不参与
策展,给它们埋点只会往 ``.usage.json`` 写无用条目。读技能目录下任意文件
(SKILL.md 或 references/templates/scripts 支持文件)都算一次该技能的使用。
解析不出技能名时返回 None。
"""
skills_container = _get_skills_container_path()
if not (path == skills_container or path.startswith(f"{skills_container}/")):
return None
relative = path[len(skills_container) :].lstrip("/")
parts = [segment for segment in relative.split("/") if segment]
if len(parts) >= 2 and parts[0] == "custom":
return parts[1]
return None
def _record_skill_read(virtual_path: str) -> None:
"""技能文件被 ``read_file`` 读取时记一次 use —— 技能"被使用"的真实信号。
技能正文不进 system prompt(prompt 里只有 name/description/location),Agent
按 Progressive Loading 用 ``read_file`` 加载 SKILL.md 才算真正用到该技能。
curator 的年龄流转据此判断闲置,没有这个埋点会把每天都在用的技能误判为陈旧。
best-effort:埋点失败绝不影响 ``read_file`` 本身。
"""
def _record_skill_paths(
virtual_paths: list[str],
runtime: ToolRuntime[ContextT, ThreadState] | None,
) -> None:
"""Record public/custom skills from real tool paths, independent of prose."""
try:
name = _custom_skill_name_from_path(virtual_path)
if name is None:
return
from deerflow.skills.usage import bump_use
from deerflow.sandbox.skill_view import skill_names_for_paths
from deerflow.skills.usage import bump_use, runtime_use_event_key
bump_use(name)
event_key = runtime_use_event_key(runtime)
for name in skill_names_for_paths(
virtual_paths,
container_path=_get_skills_container_path(),
):
bump_use(name, event_key=event_key)
except Exception:
pass
def _record_skill_read(
virtual_path: str,
runtime: ToolRuntime[ContextT, ThreadState] | None = None,
) -> None:
"""Count a successful skill file read as a logical skill invocation."""
_record_skill_paths([virtual_path], runtime)
def _record_skill_command(
command: str,
runtime: ToolRuntime[ContextT, ThreadState] | None = None,
) -> None:
"""Count skills whose scripts are executed directly by ``bash``."""
_record_skill_paths(_skill_paths_in_text(command), runtime)
def _is_acp_workspace_path(path: str) -> bool:
"""Check if a path is under the ACP workspace virtual path."""
return path == _ACP_WORKSPACE_VIRTUAL_PATH or path.startswith(f"{_ACP_WORKSPACE_VIRTUAL_PATH}/")
@@ -1445,6 +1443,7 @@ def bash_tool(runtime: ToolRuntime[ContextT, ThreadState], description: str, com
command: The bash command to execute. Always use absolute paths for files and directories.
"""
try:
requested_command = command
_prepare_command_skill_capability(command)
_validate_enabled_skill_paths_in_command(command)
sandbox = ensure_sandbox_initialized(runtime)
@@ -1457,6 +1456,7 @@ def bash_tool(runtime: ToolRuntime[ContextT, ThreadState], description: str, com
command = replace_virtual_paths_in_command(command, thread_data)
command = _apply_cwd_prefix(command, thread_data)
output = sandbox.execute_command(command)
_record_skill_command(requested_command, runtime)
try:
from deerflow.config.app_config import get_app_config
@@ -1473,7 +1473,9 @@ def bash_tool(runtime: ToolRuntime[ContextT, ThreadState], description: str, com
max_chars = sandbox_cfg.bash_output_max_chars if sandbox_cfg else 20000
except Exception:
max_chars = 20000
return _truncate_bash_output(sandbox.execute_command(command), max_chars)
output = sandbox.execute_command(command)
_record_skill_command(requested_command, runtime)
return _truncate_bash_output(output, max_chars)
except SandboxError as e:
return f"Error: {e}"
except PermissionError as e:
@@ -1751,7 +1753,7 @@ def read_file_tool(
max_result_chars=max_chars,
hard_max_result_chars=hard_max,
)
_record_skill_read(requested_path)
_record_skill_read(requested_path, runtime)
return inspected["rendered"]
except OSError:
pass
@@ -1767,7 +1769,7 @@ def read_file_tool(
)
# 读成功即记一次技能使用 —— requested_path 仍是虚拟路径(本地/容器沙箱都一样),
# 非技能路径会被 _record_skill_read 自行忽略。
_record_skill_read(requested_path)
_record_skill_read(requested_path, runtime)
if not content and not truncated:
return "(empty)"
if truncated:
@@ -7,6 +7,7 @@ import logging
import os
import tempfile
import threading
from collections import OrderedDict
from datetime import UTC, datetime
from pathlib import Path
@@ -19,6 +20,14 @@ _USAGE_FILENAME = ".usage.json"
# 单进程跑,故进程内锁即可保证状态流转不丢。
_usage_lock = threading.Lock()
# One user run can first load SKILL.md and then execute several helper scripts.
# Those are implementation details of one logical skill invocation, so retain a
# small process-local LRU keyed by (run, skill) and increment the public counter
# only once. A run stays on one worker; persistence across restarts is neither
# needed nor desirable here.
_MAX_RECENT_USE_EVENTS = 8192
_recent_use_events: OrderedDict[tuple[str, str], None] = OrderedDict()
STATE_ACTIVE = "active"
STATE_STALE = "stale"
STATE_ARCHIVED = "archived"
@@ -121,8 +130,110 @@ def bump_view(name: str) -> None:
_update_entry(name, lambda e: {"view_count": e.get("view_count", 0) + 1, "last_viewed_at": _now_iso()})
def bump_use(name: str) -> None:
_update_entry(name, lambda e: {"use_count": e.get("use_count", 0) + 1, "last_used_at": _now_iso()})
async def _increment_use_projection(name: str, use_count: int, used_at: datetime) -> None:
"""Keep the MySQL skill-list projection current without importing ``app``."""
try:
from sqlalchemy import case, update
from deerflow.persistence.engine import get_session_factory
from deerflow.persistence.skills.model import SkillRow
session_factory = get_session_factory()
if session_factory is None:
return
async with session_factory() as session:
await session.execute(
update(SkillRow)
.where(SkillRow.name == name)
.values(
# Fire-and-forget writes may finish out of order. Never let
# an older absolute sidecar count overwrite a newer value.
use_count=case((SkillRow.use_count < use_count, use_count), else_=SkillRow.use_count),
last_activity_at=case(
(SkillRow.last_activity_at.is_(None), used_at),
(SkillRow.last_activity_at < used_at, used_at),
else_=SkillRow.last_activity_at,
),
updated_at=case(
(SkillRow.updated_at < used_at, used_at),
else_=SkillRow.updated_at,
),
)
)
await session.commit()
except Exception:
logger.debug("Failed to update skill usage projection for %r", name, exc_info=True)
def _schedule_use_projection_increment(name: str, use_count: int, used_at: datetime) -> None:
"""Best-effort bridge from synchronous sandbox tools to the DB home loop."""
try:
from deerflow.persistence.engine import fire_and_forget_db_write
fire_and_forget_db_write(_increment_use_projection(name, use_count, used_at))
except Exception:
logger.debug("Failed to schedule skill usage projection for %r", name, exc_info=True)
def runtime_use_event_key(runtime: Any | None) -> str | None:
"""Return the stable user-run identifier carried by ``ToolRuntime``."""
if runtime is None:
return None
context = getattr(runtime, "context", None)
if isinstance(context, dict):
for key in ("run_id", "langgraph_run_id"):
value = context.get(key)
if value is not None and str(value).strip():
return str(value).strip()
config = getattr(runtime, "config", None)
if isinstance(config, dict):
for bucket_name in ("configurable", "metadata"):
bucket = config.get(bucket_name)
if not isinstance(bucket, dict):
continue
for key in ("run_id", "langgraph_run_id"):
value = bucket.get(key)
if value is not None and str(value).strip():
return str(value).strip()
return None
def bump_use(name: str, *, event_key: str | None = None) -> bool:
"""Count one logical use, optionally de-duplicated by user run.
Returns ``True`` only when the counter changed. This makes callers and tests
able to distinguish a new invocation from a second file/script access in
the same run.
"""
normalized_name = str(name or "").strip()
if not normalized_name:
return False
normalized_event = str(event_key or "").strip() or None
dedupe_key = (normalized_event, normalized_name) if normalized_event else None
used_at = datetime.now(UTC)
new_count = 0
try:
with _usage_lock:
if dedupe_key is not None and dedupe_key in _recent_use_events:
_recent_use_events.move_to_end(dedupe_key)
return False
data = _read_all()
entry = {**_DEFAULT_ENTRY, **data.get(normalized_name, {})}
new_count = max(0, int(entry.get("use_count") or 0)) + 1
entry.update({"use_count": new_count, "last_used_at": used_at.isoformat()})
data[normalized_name] = entry
_write_all(data)
if dedupe_key is not None:
_recent_use_events[dedupe_key] = None
_recent_use_events.move_to_end(dedupe_key)
while len(_recent_use_events) > _MAX_RECENT_USE_EVENTS:
_recent_use_events.popitem(last=False)
except Exception:
logger.debug("Failed to bump usage entry for %r", normalized_name, exc_info=True)
return False
_schedule_use_projection_increment(normalized_name, new_count, used_at)
return True
def bump_patch(name: str) -> None:
@@ -0,0 +1,397 @@
"""Filesystem-backed version history for custom skills.
Only the active package lives below ``skills/custom``. Historical snapshots
are kept under the writable DeerFlow runtime directory, outside the canonical
skills tree and therefore outside every model-facing discovery/mount path.
The feature is deliberately gated by ``system_settings.skill_versioning``.
When the switch is off, callers keep the legacy write behaviour and this
module performs no snapshot writes.
"""
from __future__ import annotations
import hashlib
import json
import os
import shutil
import tempfile
import threading
from collections.abc import Iterator
from contextlib import contextmanager
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from uuid import uuid4
from deerflow.config.paths import get_paths
from deerflow.config.system_settings import load_system_settings
from deerflow.skills.storage import SkillStorage, get_or_new_skill_storage
from deerflow.skills.types import SKILL_MD_FILE, SkillCategory
_MANIFEST_FILE = "manifest.json"
_SCHEMA_VERSION = 1
_LOCKS_GUARD = threading.Lock()
_THREAD_LOCKS: dict[str, threading.RLock] = {}
def skill_versioning_enabled() -> bool:
"""Return the deployment-wide custom-skill versioning switch."""
try:
return bool(load_system_settings().skill_versioning.enabled)
except Exception:
return False
def _thread_lock(name: str) -> threading.RLock:
with _LOCKS_GUARD:
return _THREAD_LOCKS.setdefault(name, threading.RLock())
def _versions_root() -> Path:
return get_paths().base_dir / "skill-versions"
def _skill_root(name: str) -> Path:
return _versions_root() / SkillStorage.validate_skill_name(name)
def _manifest_path(name: str) -> Path:
return _skill_root(name) / _MANIFEST_FILE
def _version_dir(name: str, version: int) -> Path:
return _skill_root(name) / f"v{version:06d}"
@contextmanager
def _cross_process_lock(name: str) -> Iterator[None]:
"""Serialize one skill's snapshot/switch operations across workers."""
root = _skill_root(name)
root.mkdir(parents=True, exist_ok=True)
lock_path = root / ".lock"
with _thread_lock(name):
with lock_path.open("a+b") as handle:
if os.name == "nt": # pragma: no cover - exercised on Windows hosts
import msvcrt
if lock_path.stat().st_size == 0:
handle.write(b"0")
handle.flush()
handle.seek(0)
msvcrt.locking(handle.fileno(), msvcrt.LK_LOCK, 1)
else: # pragma: no cover - exercised in Linux CI/deployments
import fcntl
fcntl.flock(handle.fileno(), fcntl.LOCK_EX)
try:
yield
finally:
if os.name == "nt": # pragma: no cover
handle.seek(0)
msvcrt.locking(handle.fileno(), msvcrt.LK_UNLCK, 1)
else: # pragma: no cover
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
def _empty_manifest(name: str) -> dict[str, Any]:
return {
"schema_version": _SCHEMA_VERSION,
"skill_name": name,
"active_version": None,
"versions": [],
}
def _load_manifest(name: str) -> dict[str, Any]:
path = _manifest_path(name)
if not path.is_file():
return _empty_manifest(name)
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise ValueError(f"Skill version manifest for '{name}' is unreadable") from exc
if not isinstance(payload, dict) or payload.get("skill_name") != name:
raise ValueError(f"Skill version manifest for '{name}' is invalid")
versions = payload.get("versions")
if not isinstance(versions, list):
raise ValueError(f"Skill version manifest for '{name}' has invalid versions")
return payload
def _save_manifest(name: str, manifest: dict[str, Any]) -> None:
path = _manifest_path(name)
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(prefix=".manifest-", suffix=".tmp", dir=path.parent)
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump(manifest, handle, ensure_ascii=False, indent=2, sort_keys=True)
handle.write("\n")
os.replace(tmp_name, path)
except Exception:
try:
os.unlink(tmp_name)
except OSError:
pass
raise
def _resolve_custom_dir(storage: SkillStorage, name: str) -> Path:
for skill in storage.load_skills(enabled_only=False):
if skill.name == name and skill.category == SkillCategory.CUSTOM:
return skill.skill_dir.resolve()
raise FileNotFoundError(f"Custom skill '{name}' not found")
def _tree_hash(root: Path) -> str:
digest = hashlib.sha256()
for path in sorted(root.rglob("*"), key=lambda item: item.relative_to(root).as_posix()):
relative = path.relative_to(root).as_posix()
if any(part in {"__pycache__", ".git"} for part in path.relative_to(root).parts):
continue
if path.is_symlink():
digest.update(b"L\0")
digest.update(relative.encode("utf-8"))
digest.update(b"\0")
digest.update(os.readlink(path).encode("utf-8", errors="surrogateescape"))
continue
if not path.is_file():
continue
digest.update(b"F\0")
digest.update(relative.encode("utf-8"))
digest.update(b"\0")
with path.open("rb") as handle:
while chunk := handle.read(1024 * 1024):
digest.update(chunk)
return digest.hexdigest()
def _copy_package(source: Path, destination: Path) -> None:
shutil.copytree(
source,
destination,
symlinks=True,
ignore=shutil.ignore_patterns("__pycache__", ".git"),
)
def record_custom_skill_version(
name: str,
*,
storage: SkillStorage | None = None,
action: str,
author: str | None = None,
summary: str | None = None,
) -> dict[str, Any] | None:
"""Snapshot the current custom package when it differs from active.
Returns the active/new version metadata. No-op (and no filesystem writes)
while the deployment switch is disabled.
"""
if not skill_versioning_enabled():
return None
name = SkillStorage.validate_skill_name(name)
storage = storage or get_or_new_skill_storage()
source = _resolve_custom_dir(storage, name)
current_hash = _tree_hash(source)
with _cross_process_lock(name):
manifest = _load_manifest(name)
active = next(
(item for item in manifest["versions"] if int(item.get("version") or 0) == int(manifest.get("active_version") or 0)),
None,
)
if active is not None and active.get("sha256") == current_hash:
return {**active, "active": True}
next_version = max([int(item.get("version") or 0) for item in manifest["versions"]] or [0]) + 1
destination = _version_dir(name, next_version)
staging = destination.parent / f".{destination.name}.staging-{uuid4().hex}"
_copy_package(source, staging)
try:
os.replace(staging, destination)
except Exception:
shutil.rmtree(staging, ignore_errors=True)
raise
now = datetime.now(UTC).isoformat()
entry = {
"version": next_version,
"sha256": current_hash,
"created_at": now,
"created_by": author,
"action": str(action or "update"),
"summary": str(summary or "").strip() or None,
"based_on_version": manifest.get("active_version"),
}
manifest["versions"].append(entry)
manifest["active_version"] = next_version
manifest["updated_at"] = now
_save_manifest(name, manifest)
return {**entry, "active": True}
def list_custom_skill_versions(
name: str,
*,
storage: SkillStorage | None = None,
author: str | None = None,
) -> dict[str, Any]:
"""Return newest-first metadata, lazily bootstrapping the current package."""
if not skill_versioning_enabled():
return {"enabled": False, "skill_name": name, "active_version": None, "versions": []}
storage = storage or get_or_new_skill_storage()
record_custom_skill_version(
name,
storage=storage,
action="baseline",
author=author,
summary="启用技能迭代时保存的当前版本",
)
with _cross_process_lock(name):
manifest = _load_manifest(name)
active_version = manifest.get("active_version")
versions = [{**item, "active": int(item.get("version") or 0) == int(active_version or 0)} for item in reversed(manifest["versions"])]
return {
"enabled": True,
"skill_name": name,
"active_version": active_version,
"versions": versions,
}
def activate_custom_skill_version(
name: str,
version: int,
*,
storage: SkillStorage | None = None,
) -> dict[str, Any]:
"""Materialize one historical snapshot as the sole model-visible package."""
if not skill_versioning_enabled():
raise RuntimeError("技能迭代功能未开启")
name = SkillStorage.validate_skill_name(name)
storage = storage or get_or_new_skill_storage()
target = _resolve_custom_dir(storage, name)
with _cross_process_lock(name):
manifest = _load_manifest(name)
entry = next(
(item for item in manifest["versions"] if int(item.get("version") or 0) == int(version)),
None,
)
if entry is None:
raise FileNotFoundError(f"Skill '{name}' version {version} not found")
source = _version_dir(name, int(version))
if not source.is_dir() or _tree_hash(source) != entry.get("sha256"):
raise ValueError(f"Skill '{name}' version {version} snapshot is missing or corrupted")
storage.validate_skill_markdown_content(
name,
(source / SKILL_MD_FILE).read_text(encoding="utf-8"),
)
if int(manifest.get("active_version") or 0) == int(version) and _tree_hash(target) == entry.get("sha256"):
return {**entry, "active": True}
parent = target.parent
staging = parent / f".{name}.version-stage-{uuid4().hex}"
backup = parent / f".{name}.version-backup-{uuid4().hex}"
_copy_package(source, staging)
moved_old = False
try:
os.replace(target, backup)
moved_old = True
os.replace(staging, target)
manifest["active_version"] = int(version)
manifest["updated_at"] = datetime.now(UTC).isoformat()
_save_manifest(name, manifest)
except Exception:
shutil.rmtree(staging, ignore_errors=True)
if moved_old:
shutil.rmtree(target, ignore_errors=True)
os.replace(backup, target)
raise
finally:
if backup.exists() and target.exists():
shutil.rmtree(backup, ignore_errors=True)
return {**entry, "active": True}
def delete_custom_skill_versions(name: str) -> None:
"""Remove stored snapshots after the custom skill itself is deleted."""
name = SkillStorage.validate_skill_name(name)
root = _skill_root(name)
if root.exists():
shutil.rmtree(root)
def reset_custom_skill_versions(
name: str,
*,
storage: SkillStorage | None = None,
author: str | None = None,
summary: str | None = None,
) -> dict[str, Any] | None:
"""Discard the previous chain and make the current package the new v1.
This is the explicit "direct overwrite" semantic offered beside
"iterate as a new version" for same-name uploads.
"""
if not skill_versioning_enabled():
return None
name = SkillStorage.validate_skill_name(name)
root = _skill_root(name)
with _cross_process_lock(name):
for child in list(root.iterdir()):
if child.name == ".lock":
continue
if child.is_dir():
shutil.rmtree(child)
else:
child.unlink(missing_ok=True)
return record_custom_skill_version(
name,
storage=storage,
action="overwrite",
author=author,
summary=summary or "直接覆盖后重置版本链",
)
def reconcile_custom_skill_versions(*, storage: SkillStorage | None = None, author: str | None = None) -> int:
"""Capture each current custom skill after the master switch is enabled."""
if not skill_versioning_enabled():
return 0
storage = storage or get_or_new_skill_storage()
captured = 0
for skill in storage.load_skills(enabled_only=False):
if skill.category != SkillCategory.CUSTOM:
continue
before = _load_manifest(skill.name).get("active_version") if _manifest_path(skill.name).exists() else None
result = record_custom_skill_version(
skill.name,
storage=storage,
action="feature_enabled",
author=author,
summary="技能迭代开关开启后捕获当前内容",
)
if result is not None and result.get("version") != before:
captured += 1
return captured
__all__ = [
"activate_custom_skill_version",
"delete_custom_skill_versions",
"list_custom_skill_versions",
"record_custom_skill_version",
"reconcile_custom_skill_versions",
"reset_custom_skill_versions",
"skill_versioning_enabled",
]
@@ -273,7 +273,16 @@ def skill_view_tool(name: str, runtime: ToolRuntime) -> str:
Args:
name: The skill name (hyphen-case, e.g. 'my-skill').
"""
return skill_view_impl(name, _resolve_active_agent_id(runtime), _resolve_extra_skills(runtime))
result = skill_view_impl(name, _resolve_active_agent_id(runtime), _resolve_extra_skills(runtime))
head = result.lstrip()[:200].lower()
if not ((head.startswith("skill '") and "not found" in head) or head.startswith("error reading skill")):
try:
from deerflow.skills.usage import bump_use, runtime_use_event_key
bump_use(name, event_key=runtime_use_event_key(runtime))
except Exception:
pass
return result
def _resolve_skills_container_base_path() -> str:
@@ -161,6 +161,26 @@ async def _publish_managed_skill(name: str, *, removed: bool = False) -> None:
await _sync_managed_skill_view(name)
async def _record_managed_skill_version(
storage: SkillStorage,
name: str,
*,
action: str,
author: str,
summary: str,
) -> None:
from deerflow.skills.versioning import record_custom_skill_version
await _to_thread(
record_custom_skill_version,
name,
storage=storage,
action=action,
author=author,
summary=summary,
)
async def _bump_patch_best_effort(name: str) -> None:
"""记一次技能维护活动(patch / edit)。
@@ -363,6 +383,14 @@ async def _skill_manage_impl(
# async stack — contextvars don't propagate into ``to_thread``.
if skill_store is not None:
await _ensure_skill_db_registered(skill_store, name, user_id)
skill_storage.invalidate_skills_cache()
await _record_managed_skill_version(
skill_storage,
name,
action="agent_create",
author=user_id,
summary="大模型创建技能",
)
await _publish_managed_skill(name)
return f"Created custom skill '{name}'."
@@ -378,6 +406,13 @@ async def _skill_manage_impl(
scan = await _scan_or_raise(content, executable=False, location=f"{name}/{SKILL_MD_FILE}")
skill_file = skill_storage.get_custom_skill_file(name)
prev_content = await _to_thread(skill_file.read_text, encoding="utf-8")
await _record_managed_skill_version(
skill_storage,
name,
action="baseline",
author=user_id,
summary="大模型编辑前的活动版本",
)
await _to_thread(skill_storage.write_custom_skill, name, SKILL_MD_FILE, content)
await _to_thread(
skill_storage.append_history,
@@ -385,6 +420,13 @@ async def _skill_manage_impl(
_history_record(action="edit", file_path=SKILL_MD_FILE, prev_content=prev_content, new_content=content, thread_id=thread_id, scanner=scan),
)
await _bump_patch_best_effort(name)
await _record_managed_skill_version(
skill_storage,
name,
action="agent_edit",
author=user_id,
summary="大模型编辑 SKILL.md",
)
await _publish_managed_skill(name)
return f"Updated custom skill '{name}'."
@@ -406,6 +448,13 @@ async def _skill_manage_impl(
new_content = prev_content.replace(find, replace, replacement_count)
await _to_thread(skill_storage.validate_skill_markdown_content, name, new_content)
scan = await _scan_or_raise(new_content, executable=False, location=f"{name}/{SKILL_MD_FILE}")
await _record_managed_skill_version(
skill_storage,
name,
action="baseline",
author=user_id,
summary="大模型修补前的活动版本",
)
await _to_thread(skill_storage.write_custom_skill, name, SKILL_MD_FILE, new_content)
await _to_thread(
skill_storage.append_history,
@@ -413,6 +462,13 @@ async def _skill_manage_impl(
_history_record(action="patch", file_path=SKILL_MD_FILE, prev_content=prev_content, new_content=new_content, thread_id=thread_id, scanner=scan),
)
await _bump_patch_best_effort(name)
await _record_managed_skill_version(
skill_storage,
name,
action="agent_patch",
author=user_id,
summary="大模型修补 SKILL.md",
)
await _publish_managed_skill(name)
return f"Patched custom skill '{name}' ({replacement_count} replacement(s) applied, {occurrences} match(es) found)."
@@ -453,6 +509,12 @@ async def _skill_manage_impl(
await skill_store.delete(name, user_id)
except Exception:
logger.debug("Failed to delete DB row for %r", name, exc_info=True)
try:
from deerflow.skills.versioning import delete_custom_skill_versions
await _to_thread(delete_custom_skill_versions, name)
except Exception:
logger.debug("Failed to delete version history for %r", name, exc_info=True)
agent_store = _default_agent_store()
if agent_store is not None:
try:
@@ -482,12 +544,26 @@ async def _skill_manage_impl(
prev_content = await _to_thread(target.read_text, encoding="utf-8") if exists else None
executable = "scripts/" in path or path.startswith("scripts/")
scan = await _scan_or_raise(content, executable=executable, location=f"{name}/{path}")
await _record_managed_skill_version(
skill_storage,
name,
action="baseline",
author=user_id,
summary="大模型写文件前的活动版本",
)
await _to_thread(skill_storage.write_custom_skill, name, path, content)
await _to_thread(
skill_storage.append_history,
name,
_history_record(action="write_file", file_path=path, prev_content=prev_content, new_content=content, thread_id=thread_id, scanner=scan),
)
await _record_managed_skill_version(
skill_storage,
name,
action="agent_write_file",
author=user_id,
summary=f"大模型写入文件 {path}",
)
await _sync_managed_skill_view(name)
return f"Wrote '{path}' for custom skill '{name}'."
@@ -502,12 +578,26 @@ async def _skill_manage_impl(
if not await _to_thread(target.exists):
raise FileNotFoundError(f"Supporting file '{path}' not found for skill '{name}'.")
prev_content = await _to_thread(target.read_text, encoding="utf-8")
await _record_managed_skill_version(
skill_storage,
name,
action="baseline",
author=user_id,
summary="大模型删文件前的活动版本",
)
await _to_thread(target.unlink)
await _to_thread(
skill_storage.append_history,
name,
_history_record(action="remove_file", file_path=path, prev_content=prev_content, new_content=None, thread_id=thread_id, scanner={"decision": "allow", "reason": "Deletion requested."}),
)
await _record_managed_skill_version(
skill_storage,
name,
action="agent_remove_file",
author=user_id,
summary=f"大模型删除文件 {path}",
)
await _sync_managed_skill_view(name)
return f"Removed '{path}' from custom skill '{name}'."
@@ -5,26 +5,28 @@ import pytest
from deerflow.config.app_config import AppConfig
_SKILL_VIEW_KEYS = ("skill_view_reuse_enabled", "skill_view_cache_enabled")
_OPT_IN_KEYS = ("model_timing_nonblocking_enabled", "asyncio_thread_pool_enabled", "tool_metrics_enabled")
_DEFAULT_ON_KEYS = (*_SKILL_VIEW_KEYS, "tool_metrics_enabled", "skill_metrics_enabled")
_OPT_IN_KEYS = ("model_timing_nonblocking_enabled", "asyncio_thread_pool_enabled")
def test_missing_runtime_performance_enables_skill_view_only():
def test_missing_runtime_performance_enables_skill_view_and_skill_metrics():
config = AppConfig(sandbox={"use": "deerflow.sandbox.local:LocalSandboxProvider"})
assert config.runtime_performance.model_dump() == {
"skill_view_reuse_enabled": True,
"skill_view_cache_enabled": True,
"tool_metrics_enabled": False,
"tool_metrics_enabled": True,
"skill_metrics_enabled": True,
"model_timing_nonblocking_enabled": False,
"asyncio_thread_pool_enabled": False,
"asyncio_thread_pool_size": 100,
}
@pytest.mark.parametrize("key", [*_SKILL_VIEW_KEYS, *_OPT_IN_KEYS])
@pytest.mark.parametrize("key", [*_DEFAULT_ON_KEYS, *_OPT_IN_KEYS])
def test_runtime_performance_switches_are_independent(tmp_path: Path, key: str):
# Skill-view switches default on, so independence is checked by turning
# one of them off. The opt-in switches default off and are turned on.
enabled = key not in _SKILL_VIEW_KEYS
# Skill-view and audit metrics default on, so independence is checked by
# turning one off. Opt-in switches default off and are turned on.
enabled = key not in _DEFAULT_ON_KEYS
config_path = tmp_path / "config.yaml"
config_path.write_text(
f"sandbox:\n use: deerflow.sandbox.local:LocalSandboxProvider\nruntime_performance:\n {key}: {'true' if enabled else 'false'}\n",
@@ -32,7 +34,7 @@ def test_runtime_performance_switches_are_independent(tmp_path: Path, key: str):
)
flags = AppConfig.from_file(str(config_path)).runtime_performance.model_dump()
assert flags[key] is enabled
for other in _SKILL_VIEW_KEYS:
for other in _DEFAULT_ON_KEYS:
if other != key:
assert flags[other] is True
for other in _OPT_IN_KEYS:
@@ -0,0 +1,95 @@
"""Regression coverage for model-independent skill usage attribution."""
from __future__ import annotations
from pathlib import Path
from types import SimpleNamespace
import pytest
from deerflow.sandbox import tools
from deerflow.skills import usage
from deerflow.tools.builtins import skill_tools
@pytest.fixture(autouse=True)
def _clear_recent_use_events():
usage._recent_use_events.clear()
yield
usage._recent_use_events.clear()
def test_bump_use_counts_a_skill_once_per_run_and_syncs_the_catalog(tmp_path: Path, monkeypatch):
usage_path = tmp_path / ".usage.json"
projected: list[tuple[str, int, object]] = []
monkeypatch.setattr(usage, "_usage_path", lambda: usage_path)
monkeypatch.setattr(
usage,
"_schedule_use_projection_increment",
lambda name, use_count, used_at: projected.append((name, use_count, used_at)),
)
assert usage.bump_use("page-generator", event_key="run-1") is True
assert usage.bump_use("page-generator", event_key="run-1") is False
assert usage.bump_use("page-generator", event_key="run-2") is True
entry = usage.get_entry("page-generator")
assert entry["use_count"] == 2
assert entry["last_used_at"]
assert [(name, count) for name, count, _used_at in projected] == [
("page-generator", 1),
("page-generator", 2),
]
def test_skill_file_and_script_paths_record_public_and_custom_skills(monkeypatch):
calls: list[tuple[str, str | None]] = []
monkeypatch.setattr(
usage,
"bump_use",
lambda name, *, event_key=None: calls.append((name, event_key)),
)
runtime = SimpleNamespace(context={"run_id": "run-weak-model"}, config={})
tools._record_skill_read("/mnt/skills/public/report-maker/SKILL.md", runtime)
tools._record_skill_command(
"python /mnt/skills/custom/data-cleaner/scripts/run.py --fast",
runtime,
)
assert calls == [
("report-maker", "run-weak-model"),
("data-cleaner", "run-weak-model"),
]
def test_skill_view_counts_as_use_without_model_narration(monkeypatch):
calls: list[tuple[str, str | None]] = []
runtime = SimpleNamespace(context={"run_id": "run-skill-view"}, config={})
monkeypatch.setattr(skill_tools, "skill_view_impl", lambda *_args, **_kwargs: "skill instructions")
monkeypatch.setattr(
usage,
"bump_use",
lambda name, *, event_key=None: calls.append((name, event_key)),
)
result = skill_tools.skill_view_tool.func(name="report-maker", runtime=runtime)
assert result == "skill instructions"
assert calls == [("report-maker", "run-skill-view")]
def test_failed_skill_view_does_not_count_as_use(monkeypatch):
calls: list[str] = []
runtime = SimpleNamespace(context={"run_id": "run-missing"}, config={})
monkeypatch.setattr(
skill_tools,
"skill_view_impl",
lambda *_args, **_kwargs: "Skill 'missing' not found.",
)
monkeypatch.setattr(usage, "bump_use", lambda name, **_kwargs: calls.append(name))
result = skill_tools.skill_view_tool.func(name="missing", runtime=runtime)
assert result == "Skill 'missing' not found."
assert calls == []
@@ -288,7 +288,7 @@ def _tool_sandbox(tmp_path: Path, view_root: Path, monkeypatch: pytest.MonkeyPat
monkeypatch.setattr(tools, "is_local_sandbox", lambda _runtime: True)
monkeypatch.setattr(tools, "is_host_bash_allowed", lambda: True)
monkeypatch.setattr(tools, "get_thread_data", lambda _runtime: {"workspace_path": str(tmp_path / "workspace"), "uploads_path": str(tmp_path / "uploads"), "outputs_path": str(tmp_path / "outputs")})
monkeypatch.setattr(tools, "_record_skill_read", lambda _path: None)
monkeypatch.setattr(tools, "_record_skill_read", lambda *_args: None)
monkeypatch.setattr(tools, "_get_skills_container_path", lambda: "/mnt/skills")
(tmp_path / "workspace").mkdir(exist_ok=True)
assert view_root.is_dir()
@@ -326,6 +326,30 @@ def test_bash_runs_the_updated_skill_script(cache_setup, monkeypatch, tmp_path):
assert (view.root / "public" / "online" / "scripts" / "run.py").read_text(encoding="utf-8") == "print('two')\n"
def test_bash_records_a_direct_skill_script_invocation(cache_setup, monkeypatch, tmp_path):
root, _config, storage = cache_setup
script = root / "public" / "online" / "scripts"
script.mkdir()
(script / "run.py").write_text("print('used')\n", encoding="utf-8")
view = skill_view.refresh_skill_sandbox_view(storage=storage)
_tool_sandbox(tmp_path, view.root, monkeypatch)
recorded: list[tuple[str, object]] = []
monkeypatch.setattr(
tools,
"_record_skill_command",
lambda command, runtime: recorded.append((command, runtime)),
)
result = tools.bash_tool.func(
runtime=None,
description="test",
command="python /mnt/skills/public/online/scripts/run.py",
)
assert result == "ok"
assert recorded == [("python /mnt/skills/public/online/scripts/run.py", None)]
def test_bash_rejects_an_offline_skill(cache_setup, monkeypatch, tmp_path):
root, config, storage = cache_setup
_skill(root, "offline")
@@ -50,3 +50,21 @@ def test_canonical_write_file_is_still_recorded(monkeypatch) -> None:
middleware.wrap_tool_call(_request("write_file"), lambda _: _result("write_file"))
flush.assert_called_once()
def test_call_metrics_switch_skips_new_metric_rows(monkeypatch) -> None:
middleware = ToolMetricsMiddleware()
flush = Mock()
monkeypatch.setattr(middleware, "_flush", flush)
monkeypatch.setattr(
"deerflow.agents.middlewares.tool_metrics_middleware.get_call_metrics_settings",
lambda: SimpleNamespace(enabled=False),
)
result = middleware.wrap_tool_call(
_request("write_file"),
lambda _: _result("write_file"),
)
assert result.name == "write_file"
flush.assert_not_called()
@@ -20,8 +20,10 @@ from deerflow.config.app_config import AppConfig
BUILDERS = [build_lead_runtime_middlewares, build_subagent_runtime_middlewares]
def _config(enabled=None):
def _config(enabled=None, *, skill_enabled=None):
options = {} if enabled is None else {"tool_metrics_enabled": enabled}
if skill_enabled is not None:
options["skill_metrics_enabled"] = skill_enabled
return AppConfig(
sandbox={"use": "deerflow.sandbox.local:LocalSandboxProvider"},
runtime_performance=options,
@@ -29,18 +31,29 @@ def _config(enabled=None):
@pytest.mark.parametrize("builder", BUILDERS)
@pytest.mark.parametrize("enabled", [None, False])
def test_paused_runtime_never_constructs_a_recorder(builder, enabled, monkeypatch):
def test_explicitly_paused_runtime_never_constructs_a_recorder(builder, monkeypatch):
constructor = Mock(side_effect=AssertionError("paused runtime constructed a DB recorder"))
monkeypatch.setattr(metrics, "ToolMetricsMiddleware", constructor)
middlewares = builder(app_config=_config(enabled))
middlewares = builder(app_config=_config(False, skill_enabled=False))
constructor.assert_not_called()
assert any(isinstance(m, ToolErrorHandlingMiddleware) for m in middlewares)
assert any(isinstance(m, LlmMetricsMiddleware) for m in middlewares)
@pytest.mark.parametrize("builder", BUILDERS)
def test_default_runtime_keeps_a_full_tool_and_skill_recorder(builder):
recorders = [
middleware
for middleware in builder(app_config=_config())
if isinstance(middleware, metrics.ToolMetricsMiddleware)
]
assert len(recorders) == 1
assert recorders[0]._skill_only is False
def _tool_wrappers(builder, enabled):
# Exercise the actual error/audit wrappers selected by both runtime builders.
return [m for m in builder(app_config=_config(enabled)) if isinstance(m, (ToolErrorHandlingMiddleware, metrics.ToolMetricsMiddleware))]
@@ -77,6 +90,30 @@ def test_sync_tool_execution_and_error_conversion_survive_pause(builder, enabled
assert flushed.call_args.args[0].status == result.status
@pytest.mark.parametrize("builder", BUILDERS)
def test_skill_only_mode_records_a_path_based_skill_call(builder, monkeypatch):
flushed = Mock()
monkeypatch.setattr(metrics.ToolMetricsMiddleware, "_flush", lambda _, record: flushed(record))
request = SimpleNamespace(
tool_call={
"id": "call-skill",
"name": "bash",
"args": {"command": "python /mnt/skills/public/report-maker/scripts/run.py"},
}
)
expected = ToolMessage(content="ok", tool_call_id="call-skill", name="bash")
def handler(_request):
return expected
for middleware in reversed(_tool_wrappers(builder, False)):
handler = partial(middleware.wrap_tool_call, handler=handler)
assert handler(request) is expected
assert flushed.call_count == 1
assert flushed.call_args.args[0].skill_name == "report-maker"
@pytest.mark.asyncio
@pytest.mark.parametrize("builder", BUILDERS)
@pytest.mark.parametrize("enabled", [False, True])
@@ -1,9 +1,16 @@
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from sqlalchemy import select
from sqlalchemy.dialects import mysql
from app.gateway.routers.admin_users import _apply_run_scope, _apply_tool_scope
from app.gateway.routers.admin_users import (
_apply_run_scope,
_apply_tool_scope,
_query_leaderboard_dimension_details,
)
from deerflow.persistence.run.model import RunRow
from deerflow.persistence.tool_metrics.model import ToolCallMetricRow
@@ -41,3 +48,83 @@ def test_tool_aggregation_does_not_join_the_full_runs_table() -> None:
assert sql.count("NOT (EXISTS") == 2
assert "LEFT OUTER JOIN" not in sql
assert " NOT IN " not in sql
class _DetailResult:
def __init__(self, *, rows=None, row=None):
self._rows = rows or []
self._row = row
def all(self):
return self._rows
def one(self):
return self._row
class _DetailSession:
def __init__(self):
self.statements = []
self.results = [
_DetailResult(rows=[("web-search", 3, 2, 0, 125.5)]),
_DetailResult(row=(3, 1, 2, 0, 125.5)),
]
async def execute(self, statement):
self.statements.append(statement)
return self.results.pop(0)
@pytest.mark.asyncio
async def test_skill_detail_uses_metric_table_and_returns_drawer_aggregates() -> None:
session = _DetailSession()
summary, items = await _query_leaderboard_dimension_details(
session,
datetime(2026, 11, 1, tzinfo=UTC),
datetime(2026, 11, 7, tzinfo=UTC),
kind="skill",
limit=500,
exclude_user_ids=set(),
include_scheduled=False,
include_admins=True,
include_failed=True,
)
assert summary.count == 3
assert summary.item_count == 1
assert items[0].name == "web-search"
assert items[0].avg_duration_ms == 125
sql = "\n".join(_mysql_sql(statement) for statement in session.statements)
assert "TOOL_CALL_METRICS" in sql
assert "LEFT OUTER JOIN" not in sql
assert " NOT IN " not in sql
@pytest.mark.asyncio
async def test_agent_detail_returns_token_aggregates_from_runs() -> None:
session = _DetailSession()
session.results = [
_DetailResult(rows=[("lead_agent", 4, 2, 960, 0)]),
_DetailResult(row=(4, 1, 2, 960, 0)),
]
summary, items = await _query_leaderboard_dimension_details(
session,
datetime(2026, 11, 1, tzinfo=UTC),
datetime(2026, 11, 7, tzinfo=UTC),
kind="agent",
limit=500,
exclude_user_ids=set(),
include_scheduled=False,
include_admins=True,
include_failed=True,
)
assert summary.total_tokens == 960
assert items[0].avg_duration_ms == 0
sql = "\n".join(_mysql_sql(statement) for statement in session.statements)
assert "FROM RUNS" in sql
assert "SCHEDULED_TASK_RUNS" in sql
@@ -405,6 +405,23 @@ async def test_skill_catalog_projection_uses_compact_rows_without_long_fields(
assert rows[0]["description"] == "compact metadata"
assert "detail" not in rows[0]
assert "detail" not in {column.key for column in skill_catalog_cache._SKILL_LIST_COLUMNS}
offline = SimpleNamespace(
name="catalog-offline",
description="stays untouched",
license=None,
category="custom",
enabled=True,
always_on=False,
skill_file=skill_file,
)
assert await skill_catalog_cache.refresh_skill_catalog_projection(skills=[skill, offline]) == 2
assert await skill_catalog_cache.set_skill_catalog_enabled("catalog-demo", False) is True
assert await skill_catalog_cache.set_skill_catalog_enabled("missing-skill", False) is False
rows = await skill_catalog_cache.load_skill_catalog_projection(user_id="user-1", admin=True)
by_name = {row["name"]: row for row in rows or []}
assert by_name["catalog-demo"]["enabled"] is False
assert by_name["catalog-offline"]["enabled"] is True
await engine.dispose()
@@ -12,6 +12,7 @@ PNG = bytes.fromhex(
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4"
"890000000a49444154789c63000100000500010d0a2db40000000049454e44ae426082"
)
SVG = b'<svg xmlns="http://www.w3.org/2000/svg" width="1920" height="64"><rect width="1920" height="64" fill="#8f1d14"/></svg>'
def _store(monkeypatch: pytest.MonkeyPatch, tmp_path):
@@ -41,6 +42,10 @@ def _store(monkeypatch: pytest.MonkeyPatch, tmp_path):
def test_new_settings_leave_app_identity_empty() -> None:
assert SystemSettings().app_identity == AppIdentitySettings()
assert SystemSettings().app_identity.header_title_font_mode == "adaptive"
assert SystemSettings().app_identity.header_title_font_size == 30
assert SystemSettings().app_identity.light_red_header_background_filename is None
assert SystemSettings().app_identity.light_red_nav_active_filename is None
@pytest.mark.asyncio
@@ -84,6 +89,58 @@ async def test_admin_saves_name_without_replacing_the_static_logo(
assert current["settings"].app_identity.logo_filename is None
@pytest.mark.asyncio
async def test_admin_saves_header_title_font_settings(
monkeypatch: pytest.MonkeyPatch,
tmp_path,
) -> None:
current = _store(monkeypatch, tmp_path)
response = await app_identity_router.update_app_identity_settings(
SimpleNamespace(),
system_name="研判中心",
remove_logo=False,
logo=None,
header_title_font_mode="fixed",
header_title_font_size=34,
)
assert response.header_title_font_mode == "fixed"
assert response.header_title_font_size == 34
assert current["settings"].app_identity.header_title_font_mode == "fixed"
assert current["settings"].app_identity.header_title_font_size == 34
renamed = await app_identity_router.update_app_identity_settings(
SimpleNamespace(),
system_name="新研判中心",
remove_logo=False,
logo=None,
)
assert renamed.header_title_font_mode == "fixed"
assert renamed.header_title_font_size == 34
@pytest.mark.asyncio
async def test_header_title_fixed_font_size_is_bounded(
monkeypatch: pytest.MonkeyPatch,
tmp_path,
) -> None:
_store(monkeypatch, tmp_path)
with pytest.raises(HTTPException) as error:
await app_identity_router.update_app_identity_settings(
SimpleNamespace(),
system_name="研判中心",
remove_logo=False,
logo=None,
header_title_font_mode="fixed",
header_title_font_size=41,
)
assert error.value.status_code == 422
@pytest.mark.asyncio
async def test_uploaded_logo_is_served_separately_from_generated_pages(
monkeypatch: pytest.MonkeyPatch,
@@ -126,6 +183,91 @@ async def test_invalid_logo_is_rejected(monkeypatch: pytest.MonkeyPatch, tmp_pat
assert error.value.status_code == 422
@pytest.mark.asyncio
async def test_red_navigation_images_can_be_uploaded_and_served(
monkeypatch: pytest.MonkeyPatch,
tmp_path,
) -> None:
current = _store(monkeypatch, tmp_path)
saved = await app_identity_router.update_app_identity_settings(
SimpleNamespace(),
system_name="研判中心",
remove_logo=False,
logo=None,
light_red_header_background=UploadFile(
file=BytesIO(SVG),
filename="header.svg",
),
light_red_nav_active=UploadFile(
file=BytesIO(PNG),
filename="active.png",
),
)
assert saved.has_light_red_header_background is True
assert saved.has_light_red_nav_active is True
assert saved.light_red_header_background_url is not None
assert saved.light_red_header_background_url.startswith(
"/api/public/app-identity/light-red-header-background?v="
)
assert saved.light_red_nav_active_url is not None
assert saved.light_red_nav_active_url.startswith(
"/api/public/app-identity/light-red-nav-active?v="
)
identity = current["settings"].app_identity
header_file = tmp_path / "app-identity" / identity.light_red_header_background_filename
active_file = tmp_path / "app-identity" / identity.light_red_nav_active_filename
assert header_file.is_file()
assert header_file.suffix == ".svg"
assert active_file.is_file()
served_header = await app_identity_router.get_light_red_header_background()
served_active = await app_identity_router.get_light_red_nav_active()
assert served_header.media_type == "image/svg+xml"
assert served_header.headers["content-security-policy"].startswith("sandbox;")
assert served_active.media_type == "image/png"
@pytest.mark.asyncio
async def test_restoring_red_navigation_images_uses_built_in_defaults(
monkeypatch: pytest.MonkeyPatch,
tmp_path,
) -> None:
current = _store(monkeypatch, tmp_path)
await app_identity_router.update_app_identity_settings(
SimpleNamespace(),
system_name="研判中心",
remove_logo=False,
logo=None,
light_red_header_background=UploadFile(
file=BytesIO(PNG),
filename="header.png",
),
light_red_nav_active=UploadFile(
file=BytesIO(PNG),
filename="active.png",
),
)
identity = current["settings"].app_identity
header_file = tmp_path / "app-identity" / identity.light_red_header_background_filename
active_file = tmp_path / "app-identity" / identity.light_red_nav_active_filename
restored = await app_identity_router.update_app_identity_settings(
SimpleNamespace(),
system_name="研判中心",
remove_logo=False,
logo=None,
remove_light_red_header_background=True,
remove_light_red_nav_active=True,
)
assert restored.has_light_red_header_background is False
assert restored.has_light_red_nav_active is False
assert header_file.exists() is False
assert active_file.exists() is False
@pytest.mark.asyncio
async def test_clearing_app_identity_removes_only_its_logo(
monkeypatch: pytest.MonkeyPatch,
@@ -144,6 +286,8 @@ async def test_clearing_app_identity_removes_only_its_logo(
cleared = await app_identity_router.delete_app_identity_settings(SimpleNamespace())
assert cleared.system_name == ""
assert cleared.header_title_font_mode == "adaptive"
assert cleared.header_title_font_size == 30
assert cleared.has_logo is False
assert logo_file.exists() is False
with pytest.raises(HTTPException) as error:
@@ -1,11 +1,15 @@
from __future__ import annotations
import json
import zipfile
from pathlib import Path
from types import SimpleNamespace
import pytest
from app.gateway.routers import artifacts
from deerflow.html_runtime import offline_bundle
from deerflow.html_runtime.offline_bundle import build_html_offline_bundle
from deerflow.uploads.policy import default_upload_policy, set_runtime_upload_policy
@@ -91,3 +95,165 @@ async def test_artifact_preview_rejects_binary_content(tmp_path: Path, monkeypat
max_chars=1024,
)
assert raised.value.status_code == 415
@pytest.mark.asyncio
async def test_html_artifact_offline_bundle_uses_zip_export(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
source = tmp_path / "page.html"
source.write_text("<!doctype html><title>portable</title>", encoding="utf-8")
calls: list[Path] = []
async def resolve(_thread_id: str, _path: str) -> Path:
return source
async def bundle_response(path: Path):
calls.append(path)
return artifacts.Response(content=b"zip-content", media_type="application/zip")
monkeypatch.setattr(artifacts, "aresolve_thread_virtual_path", resolve)
monkeypatch.setattr(artifacts, "build_html_offline_bundle_response", bundle_response)
response = await artifacts.get_artifact.__wrapped__(
thread_id="thread-1",
path="mnt/user-data/outputs/page.html",
request=SimpleNamespace(),
download=True,
offline_bundle=True,
)
assert response.media_type == "application/zip"
assert calls == [source]
def _write_offline_fixture(path: Path, content: str | bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
if isinstance(content, bytes):
path.write_bytes(content)
else:
path.write_text(content, encoding="utf-8")
def _offline_runtime_fixture(root: Path) -> Path:
runtime = root / "runtime"
_write_offline_fixture(runtime / "echarts.min.js", "window.echarts={};")
_write_offline_fixture(runtime / "deerflow-runtime.js", "window.__runtimeLoaded=true;")
_write_offline_fixture(runtime / "brand-logo-dark.png", b"fallback-logo")
_write_offline_fixture(runtime / "LICENSE.echarts.txt", "ECharts license")
for name in (
"china-provinces.geojson",
"japan-prefectures.geojson",
"taiwan-counties.geojson",
"us-states.geojson",
):
_write_offline_fixture(runtime / "maps" / name, '{"type":"FeatureCollection","features":[]}')
return runtime
def test_html_offline_bundle_rewrites_and_packages_gateway_runtime(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
runtime = _offline_runtime_fixture(tmp_path)
logo = tmp_path / "branding" / "logo.webp"
_write_offline_fixture(logo, b"configured-logo")
uploaded_image = tmp_path / "uploads" / "background.png"
_write_offline_fixture(uploaded_image, b"uploaded-image")
monkeypatch.setattr(
offline_bundle,
"verify_html_image_token",
lambda _token: ("thread-1", "uploads", "background.png"),
)
monkeypatch.setattr(
offline_bundle,
"resolve_html_image_path",
lambda _thread_id, _kind, _filename: uploaded_image,
)
source = tmp_path / "态势报告.html"
source.write_text(
"""<!doctype html><html><head>
<script src="http://gateway.local/prefix/api/public/html-runtime/echarts.min.js"></script>
<script src="http://gateway.local/prefix/api/public/html-runtime/deerflow-runtime.js"></script>
<style>.brand{background:url('http://gateway.local/prefix/api/public/html-runtime/generated-page-logo?v=2')}</style>
</head><body><script type="application/json" data-df-geo="map">{"map":"china-provinces"}</script>
<script type="application/json" data-df-map="tile-map">{"markers":[]}</script>
<img src="http://gateway.local/prefix/api/public/html-images/signed.token">
<a href="https://example.com/source">来源</a></body></html>""",
encoding="utf-8",
)
destination = tmp_path / "bundle.zip"
result = build_html_offline_bundle(
source,
destination,
runtime_base_url="http://gateway.local/prefix/api/public/html-runtime",
map_tile_url="https://tiles.example/{z}/{x}/{y}.png",
generated_logo_path=logo,
runtime_dir=runtime,
)
assert destination.is_file()
assert "assets/html-runtime/echarts.min.js" in result.bundled_assets
assert "https://example.com/source" in result.external_urls
assert "https://tiles.example/{z}/{x}/{y}.png" in result.external_urls
with zipfile.ZipFile(destination) as archive:
names = set(archive.namelist())
assert "index.html" in names
assert "README.txt" in names
assert "manifest.json" in names
assert "assets/html-runtime/echarts.min.js" in names
assert "assets/html-runtime/deerflow-runtime.js" in names
assert "assets/html-runtime/generated-page-logo.webp" in names
assert "assets/html-runtime/maps/china-provinces.geojson" in names
assert any(name.startswith("assets/html-images/") for name in names)
assert "THIRD_PARTY_LICENSES/LICENSE.echarts.txt" in names
html = archive.read("index.html").decode("utf-8")
assert "gateway.local" not in html
assert 'src="assets/html-runtime/echarts.min.js"' in html
assert "assets/html-runtime/generated-page-logo.webp" in html
assert "/api/public/html-images/" not in html
runtime_script = archive.read("assets/html-runtime/deerflow-runtime.js").decode("utf-8")
assert "DEERFLOW_HTML_RUNTIME_CONFIG" in runtime_script
assert "assets/html-runtime/maps/china-provinces.geojson" in runtime_script
assert "window.__runtimeLoaded=true" in runtime_script
manifest = json.loads(archive.read("manifest.json"))
assert manifest["entry"] == "index.html"
assert manifest["source_name"] == "态势报告.html"
assert manifest["external_urls"] == [
"https://example.com/source",
"https://tiles.example/{z}/{x}/{y}.png",
]
def test_html_offline_bundle_copies_cognitive_runtime_tree(tmp_path: Path) -> None:
runtime = _offline_runtime_fixture(tmp_path)
_write_offline_fixture(runtime / "cognitive-situation-map" / "app.js", "window.cognitive=true;")
_write_offline_fixture(runtime / "cognitive-situation-map" / "styles.css", "body{color:white}")
_write_offline_fixture(runtime / "cognitive-situation-map" / "assets" / "maps" / "taiwan.png", b"map")
source = tmp_path / "认知态势图.html"
source.write_text(
"""<script>window.__DF_SKILL_BASE__="http://gw/api/public/html-runtime/cognitive-situation-map";</script>
<link rel="stylesheet" href="http://gw/api/public/html-runtime/cognitive-situation-map/styles.css">
<script src="http://gw/api/public/html-runtime/cognitive-situation-map/app.js"></script>""",
encoding="utf-8",
)
destination = tmp_path / "cognitive.zip"
build_html_offline_bundle(
source,
destination,
runtime_base_url="http://gw/api/public/html-runtime",
map_tile_url="https://tiles.example/{z}/{x}/{y}.png",
runtime_dir=runtime,
)
with zipfile.ZipFile(destination) as archive:
html = archive.read("index.html").decode("utf-8")
assert "http://gw" not in html
assert "assets/html-runtime/cognitive-situation-map" in html
assert "assets/html-runtime/cognitive-situation-map/app.js" in archive.namelist()
assert "assets/html-runtime/cognitive-situation-map/assets/maps/taiwan.png" in archive.namelist()
@@ -139,8 +139,12 @@ async def test_html_edit_invalidates_metadata_content_and_list(cached_request, m
request.state.user.system_role = "admin"
state = {"html": "old"}
monkeypatch.setattr(html_templates.template_store, "read_template_html", lambda _id: state["html"])
# content 端点默认走引擎的「完整文档」读取(结构化模板会组合默认数据)。
monkeypatch.setattr(html_templates.template_fill, "read_template_text", lambda _id: state["html"])
monkeypatch.setattr(html_templates.template_store, "list_templates", lambda: [{"name": state["html"]}])
monkeypatch.setattr(html_templates.template_store, "get_template_meta", lambda _id: {"name": state["html"]})
monkeypatch.setattr(
html_templates.template_store, "get_template_meta", lambda _id: {"id": _id, "name": state["html"]}
)
def update(_id, **kwargs):
state["html"] = kwargs["html"]
@@ -355,10 +355,7 @@ async def test_weknora_session_endpoint_sets_cookie_and_returns_platform_path(mo
return {"id": "local-kb", "weknora_id": "remote-kb"}
async def list_visible(self, *_: object, **__: object) -> list[dict[str, str]]:
return [
{"id": "local-kb", "weknora_id": "remote-kb"},
{"id": "public-kb", "weknora_id": "other-public-kb"},
]
raise AssertionError("creating an iframe session must not load every visible knowledge base")
monkeypatch.setattr(llmwiki_router, "_runtime_for_iframe_proxy", lambda _: object())
monkeypatch.setattr(llmwiki_router, "_authorized_mapping", authorized_mapping)
@@ -387,7 +384,39 @@ async def test_weknora_session_endpoint_sets_cookie_and_returns_platform_path(mo
assert ctx.mapping_id == "local-kb"
assert ctx.weknora_id == "remote-kb"
assert ctx.can_write is True
assert ctx.allowed_weknora_ids == ("remote-kb", "other-public-kb")
assert ctx.allowed_weknora_ids == ()
assert len(raw) < 1024
@pytest.mark.asyncio
async def test_weknora_embed_picker_hydrates_visible_bases_server_side(monkeypatch: pytest.MonkeyPatch) -> None:
from app.gateway.routers import llmwiki as llmwiki_router
calls: list[dict[str, object]] = []
class Store:
async def list_visible(self, user_id: str, *, scope: str, is_admin: bool) -> list[dict[str, str]]:
calls.append({"user_id": user_id, "scope": scope, "is_admin": is_admin})
return [
{"id": "local-kb", "weknora_id": "remote-kb"},
{"id": "public-kb", "weknora_id": "other-public-kb"},
]
monkeypatch.setattr(llmwiki_router, "_store", lambda _: Store())
ctx = llmwiki_router.WeKnoraEmbedContext(
mapping_id="local-kb",
weknora_id="remote-kb",
user_id="user-a",
is_admin=False,
can_write=True,
exp=4_102_444_800,
)
hydrated = await llmwiki_router._with_visible_embed_knowledge_bases(_request("/platform/knowledge-bases/remote-kb"), ctx)
assert calls == [{"user_id": "user-a", "scope": "all", "is_admin": False}]
assert hydrated.allowed_weknora_ids == ("remote-kb", "other-public-kb")
assert "allowed_weknora_ids" not in llmwiki_router._embed_context_payload(hydrated)
@pytest.mark.asyncio
@@ -2079,6 +2108,9 @@ async def test_prefixed_weknora_proxy_serves_page_static_assets_and_api_without_
async def get_authorized(self, *_: object, **__: object) -> dict[str, str]:
return {"id": "local-kb", "weknora_id": "remote-kb"}
async def list_visible(self, *_: object, **__: object) -> list[dict[str, str]]:
return [{"id": "local-kb", "weknora_id": "remote-kb"}]
runtime = SimpleNamespace(
web_base_url="http://weknora-web.test",
api_base_url="http://weknora-api.test",
@@ -2207,9 +2239,28 @@ async def test_prefixed_weknora_proxy_serves_page_static_assets_and_api_without_
api = await client.get("/deerflow/api/llmwiki/weknora-embed/api/v1/auth/me")
assert api.status_code == 200
directory = await client.get("/deerflow/api/llmwiki/weknora-embed/api/v1/knowledge-bases")
assert directory.status_code == 200
assert directory.json() == {
"success": True,
"data": [
{
"id": "remote-kb",
"name": "",
"description": "",
"type": "document",
"status": "active",
"created_at": None,
"updated_at": None,
}
],
"total": 1,
}
api_request = next(request for request in upstream_requests if request[1].startswith(runtime.api_base_url))
assert api_request[2]["Authorization"] == "Bearer admin-token"
assert api_request[2]["X-Tenant-ID"] == "tenant-1"
assert not any(request[1].endswith("/api/v1/knowledge-bases") for request in upstream_requests)
def test_weknora_iframe_hides_current_outer_sidebar_shell() -> None:
@@ -2334,7 +2385,6 @@ async def test_weknora_iframe_switch_rechecks_selected_mapping_permissions(
is_admin=False,
can_write=True,
exp=4_102_444_800,
allowed_weknora_ids=("remote-kb", "other-visible-kb"),
)
selected = await llmwiki_router._switch_embed_context(
@@ -2382,12 +2432,16 @@ async def test_weknora_platform_proxy_reissues_cookie_for_native_kb_switch(
seen["target"] = weknora_id
return selected
async def hydrate_visible_bases(_: Request, ctx: object) -> object:
return ctx
async def proxy_request(*_: object, **kwargs: object) -> Response:
seen.update(kwargs)
return Response("ok")
monkeypatch.setattr(llmwiki_router, "_read_embed_context", read_context)
monkeypatch.setattr(llmwiki_router, "_switch_embed_context", switch_context)
monkeypatch.setattr(llmwiki_router, "_with_visible_embed_knowledge_bases", hydrate_visible_bases)
monkeypatch.setattr(
llmwiki_router,
"_runtime_for_iframe_proxy",
@@ -40,7 +40,7 @@ def test_load_returns_defaults_when_missing(tmp_runtime_home: Path) -> None:
assert settings.conversation_context.recent_turn_window_enabled is False
assert settings.conversation_context.max_conversation_rounds == 20
assert settings.write_file_compatibility.enabled is False
assert settings.conversation_context.recent_turn_window_enabled is False
assert settings.call_metrics.enabled is True
def test_save_then_load_roundtrips(tmp_runtime_home: Path) -> None:
@@ -325,6 +325,28 @@ async def test_admin_can_toggle_write_file_compatibility_without_restart(
assert ss.get_write_file_compatibility_settings().enabled is False
@pytest.mark.asyncio
async def test_admin_can_toggle_call_metrics_without_restart(
tmp_runtime_home: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
_patch_admin(monkeypatch)
disabled = await ss_router.update_call_metrics_setting(
ss_router.CallMetricsUpdateRequest(enabled=False),
SimpleNamespace(),
)
assert disabled.enabled is False
assert ss.get_call_metrics_settings().enabled is False
enabled = await ss_router.update_call_metrics_setting(
ss_router.CallMetricsUpdateRequest(enabled=True),
SimpleNamespace(),
)
assert enabled.enabled is True
assert ss.get_call_metrics_settings().enabled is True
def test_prepend_prefix_tags_first_human_without_touching_content() -> None:
graph_input = {"messages": [HumanMessage(content="评价一下中美贸易")]}
out = _prepend_prompt_prefix(graph_input, "你需要站在中方角度")
@@ -69,6 +69,7 @@ async def _admit(repo: RoundtableRoomRepository, draft_id: str, user_id: str, **
room_session_id=kwargs.get("room_session_id"),
enabled=kwargs.get("enabled", True),
max_rooms=kwargs.get("max_rooms", 3),
lease_seconds=kwargs.get("lease_seconds", 5 * 60),
extra_keys=kwargs.get("extra_keys"),
)
@@ -257,6 +258,44 @@ async def test_delayed_heartbeat_cannot_resurrect_a_released_room(rooms) -> None
assert next_room["ok"] is True
@pytest.mark.asyncio
async def test_foreground_lease_defaults_to_five_minutes_and_honors_override(rooms) -> None:
repo, factory = rooms
before = datetime.now(UTC)
admitted = await _admit(
repo,
"draft-lease",
"user-a",
room_session_id="lease-session",
)
assert admitted["ok"] is True
key = compute_room_session_key("lease-session")
assert key is not None
async with factory() as session:
row = await session.get(RoundtableRoomLeaseRow, key)
assert row is not None and row.lease_until is not None
until = row.lease_until if row.lease_until.tzinfo else row.lease_until.replace(tzinfo=UTC)
remaining = until - before
assert timedelta(minutes=4, seconds=50) <= remaining <= timedelta(minutes=5, seconds=5)
renewed = await repo.renew(
draft_id="draft-lease",
user_id="user-a",
task_id=None,
room_session_id="lease-session",
enabled=True,
max_rooms=3,
lease_seconds=120,
)
assert renewed["ok"] is True
async with factory() as session:
row = await session.get(RoundtableRoomLeaseRow, key)
assert row is not None and row.lease_until is not None
until = row.lease_until if row.lease_until.tzinfo else row.lease_until.replace(tzinfo=UTC)
renewed_for = until - datetime.now(UTC)
assert timedelta(seconds=90) <= renewed_for <= timedelta(seconds=125)
@pytest.mark.asyncio
async def test_stale_run_for_leased_draft_does_not_consume_a_second_room(rooms) -> None:
"""Run-manager fallback must not double-count the authoritative lease."""
@@ -744,6 +783,7 @@ async def test_admin_can_change_roundtable_room_limit_without_restart(
assert ss.get_roundtable_room_limit_settings().enabled is True
assert ss.get_roundtable_room_limit_settings().max_rooms == 3
assert ss.get_roundtable_room_limit_settings().max_hold_minutes == 30
assert ss.get_roundtable_room_limit_settings().lease_minutes == 5
assert ss.get_roundtable_room_limit_settings().max_cycles == 8
assert ss.get_roundtable_room_limit_settings().write_file_compatibility_enabled is False
@@ -752,6 +792,7 @@ async def test_admin_can_change_roundtable_room_limit_without_restart(
enabled=False,
max_rooms=5,
max_hold_minutes=45,
lease_minutes=8,
max_cycles=12,
),
SimpleNamespace(),
@@ -759,9 +800,11 @@ async def test_admin_can_change_roundtable_room_limit_without_restart(
assert saved.enabled is False
assert saved.max_rooms == 5
assert saved.max_hold_minutes == 45
assert saved.lease_minutes == 8
assert saved.max_cycles == 12
assert ss.load_system_settings().roundtable_room_limit.max_rooms == 5
assert ss.load_system_settings().roundtable_room_limit.max_hold_minutes == 45
assert ss.load_system_settings().roundtable_room_limit.lease_minutes == 8
assert ss.load_system_settings().roundtable_room_limit.max_cycles == 12
assert saved.write_file_compatibility_enabled is False
@@ -0,0 +1,190 @@
from __future__ import annotations
import io
from pathlib import Path
from types import SimpleNamespace
import pytest
from fastapi import UploadFile
from deerflow.skills.storage import LocalSkillStorage
from deerflow.skills.types import SkillCategory
def _skill_markdown(version: str) -> str:
return f"""---
name: demo-skill
description: Demo custom skill
---
# Demo {version}
"""
@pytest.fixture
def versioned_skill(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
from deerflow.skills import versioning
skills_root = tmp_path / "skills"
skill_dir = skills_root / "custom" / "demo-skill"
skill_dir.mkdir(parents=True)
(skill_dir / "SKILL.md").write_text(_skill_markdown("v1"), encoding="utf-8")
(skill_dir / "assets").mkdir()
(skill_dir / "assets" / "payload.txt").write_text("asset-v1", encoding="utf-8")
storage = LocalSkillStorage(host_path=str(skills_root))
enabled = SimpleNamespace(enabled=True)
monkeypatch.setattr(
versioning,
"load_system_settings",
lambda: SimpleNamespace(skill_versioning=enabled),
)
monkeypatch.setattr(
versioning,
"get_paths",
lambda: SimpleNamespace(base_dir=tmp_path / "runtime"),
)
return versioning, storage, skill_dir, enabled, tmp_path
def test_versioning_is_noop_while_switch_is_disabled(versioned_skill) -> None:
versioning, storage, _skill_dir, enabled, tmp_path = versioned_skill
enabled.enabled = False
assert versioning.record_custom_skill_version("demo-skill", storage=storage, action="upload") is None
assert not (tmp_path / "runtime" / "skill-versions").exists()
def test_complete_package_versions_switch_without_resetting_usage(versioned_skill, monkeypatch: pytest.MonkeyPatch) -> None:
versioning, storage, skill_dir, _enabled, tmp_path = versioned_skill
import deerflow.skills.storage as storage_package
from deerflow.skills import usage
monkeypatch.setattr(storage_package, "get_or_new_skill_storage", lambda: storage)
monkeypatch.setattr(usage, "_schedule_use_projection_increment", lambda *args: None)
first = versioning.record_custom_skill_version("demo-skill", storage=storage, action="upload")
assert first and first["version"] == 1
assert usage.bump_use("demo-skill") is True
assert usage.bump_use("demo-skill") is True
(skill_dir / "SKILL.md").write_text(_skill_markdown("v2"), encoding="utf-8")
(skill_dir / "assets" / "payload.txt").write_text("asset-v2", encoding="utf-8")
second = versioning.record_custom_skill_version("demo-skill", storage=storage, action="upload_iteration")
assert second and second["version"] == 2
listed = versioning.list_custom_skill_versions("demo-skill", storage=storage)
assert listed["active_version"] == 2
assert [item["version"] for item in listed["versions"]] == [2, 1]
versioning.activate_custom_skill_version("demo-skill", 1, storage=storage)
assert "Demo v1" in (skill_dir / "SKILL.md").read_text(encoding="utf-8")
assert (skill_dir / "assets" / "payload.txt").read_text(encoding="utf-8") == "asset-v1"
assert usage.get_entry("demo-skill")["use_count"] == 2
snapshot_root = tmp_path / "runtime" / "skill-versions" / "demo-skill"
assert snapshot_root.is_dir()
assert not snapshot_root.is_relative_to(skills_root := storage.get_skills_root_path())
assert [skill.name for skill in storage.load_skills(enabled_only=False)] == ["demo-skill"]
assert skills_root == tmp_path / "skills"
def test_direct_overwrite_resets_version_chain(versioned_skill) -> None:
versioning, storage, skill_dir, _enabled, _tmp_path = versioned_skill
versioning.record_custom_skill_version("demo-skill", storage=storage, action="upload")
(skill_dir / "SKILL.md").write_text(_skill_markdown("v2"), encoding="utf-8")
versioning.record_custom_skill_version("demo-skill", storage=storage, action="upload_iteration")
(skill_dir / "SKILL.md").write_text(_skill_markdown("replacement"), encoding="utf-8")
reset = versioning.reset_custom_skill_versions("demo-skill", storage=storage, summary="direct overwrite")
assert reset and reset["version"] == 1
listed = versioning.list_custom_skill_versions("demo-skill", storage=storage)
assert listed["active_version"] == 1
assert [item["version"] for item in listed["versions"]] == [1]
assert "replacement" in (skill_dir / "SKILL.md").read_text(encoding="utf-8")
@pytest.mark.asyncio
@pytest.mark.parametrize(
("install_mode", "expected_action"),
[("iterate", "upload_iteration"), ("overwrite", "reset")],
)
async def test_same_name_upload_supports_iteration_or_direct_overwrite(
monkeypatch: pytest.MonkeyPatch,
install_mode: str,
expected_action: str,
) -> None:
from app.gateway.routers import skills as skills_router
from deerflow.skills import usage, versioning
class FakeStorage:
def invalidate_skills_cache(self):
return None
def validate_skill_archive(self, *_args, **_kwargs):
return {
"skill_name": "demo-skill",
"conflict_kind": "custom",
}
async def ainstall_skill_from_archive(self, *_args, **kwargs):
assert kwargs["overwrite"] is True
assert kwargs["target_category"] == SkillCategory.CUSTOM
return {
"success": True,
"skill_name": "demo-skill",
"message": "installed",
"overwritten": True,
"category": "custom",
}
class FakeSkillStore:
async def ensure_legacy(self, payload):
assert payload["name"] == "demo-skill"
return payload
storage = FakeStorage()
calls: list[str] = []
async def preview_target(**_kwargs):
return False, True, True, False, False, "user-1", "owner", "conflict"
async def no_op(*_args, **_kwargs):
return None
monkeypatch.setattr(skills_router, "get_or_new_skill_storage", lambda **_kwargs: storage)
monkeypatch.setattr(skills_router, "_preview_install_target", preview_target)
monkeypatch.setattr(skills_router, "_current_user_id", lambda _request: "user-1")
monkeypatch.setattr(skills_router, "_is_admin_user", lambda _request: False)
monkeypatch.setattr(skills_router, "_refresh_catalog_snapshot", no_op)
monkeypatch.setattr(skills_router, "refresh_skills_system_prompt_cache_async", no_op)
monkeypatch.setattr(skills_router, "_sync_skill_view", no_op)
monkeypatch.setattr(usage, "mark_uploaded", lambda _name: None)
monkeypatch.setattr(versioning, "skill_versioning_enabled", lambda: True)
monkeypatch.setattr(
versioning,
"record_custom_skill_version",
lambda _name, **kwargs: calls.append(kwargs["action"]),
)
monkeypatch.setattr(
versioning,
"reset_custom_skill_versions",
lambda _name, **_kwargs: calls.append("reset"),
)
result = await skills_router.install_skill_upload(
request=SimpleNamespace(),
file=UploadFile(filename="demo.skill", file=io.BytesIO(b"PK\x03\x04archive")),
overwrite=False,
install_mode=install_mode,
target_category=SkillCategory.CUSTOM,
config=SimpleNamespace(),
skill_store=FakeSkillStore(),
)
assert result.overwritten is True
assert expected_action in calls
if install_mode == "iterate":
assert calls == ["baseline", "upload_iteration"]
else:
assert calls == ["reset"]
@@ -221,11 +221,22 @@ async def test_update_public_skill_is_admin_only_and_preserves_raw_config(
monkeypatch.setattr(skills_router, "get_or_new_skill_storage", lambda **_kwargs: storage)
refresh_calls: list[bool] = []
catalog_refresh_calls: list[object] = []
enabled_row_updates: list[tuple[str, bool]] = []
async def _refresh() -> None:
refresh_calls.append(True)
async def _refresh_catalog(config) -> None:
catalog_refresh_calls.append(config)
async def _set_enabled(name: str, enabled: bool) -> bool:
enabled_row_updates.append((name, enabled))
return True
monkeypatch.setattr(skills_router, "refresh_skills_system_prompt_cache_async", _refresh)
monkeypatch.setattr(skills_router, "refresh_skill_catalog_projection", _refresh_catalog)
monkeypatch.setattr(skills_router, "set_skill_catalog_enabled", _set_enabled)
replace_calls: list[tuple[object, object]] = []
real_replace = os.replace
@@ -246,6 +257,7 @@ async def test_update_public_skill_is_admin_only_and_preserves_raw_config(
assert forbidden.value.status_code == 403
assert config_path.read_bytes() == before
assert refresh_calls == []
assert enabled_row_updates == []
response = await skills_router.update_skill(
"demo-skill",
@@ -264,6 +276,8 @@ async def test_update_public_skill_is_admin_only_and_preserves_raw_config(
assert persisted["mcpServers"]["preserved"]["command"] == "$MCP_COMMAND"
assert get_extensions_config().skills["demo-skill"].enabled is False
assert refresh_calls == [True]
assert enabled_row_updates == [("demo-skill", False)]
assert catalog_refresh_calls == []
config_replaces = [item for item in replace_calls if os.fspath(item[1]) == os.fspath(config_path)]
assert len(config_replaces) == 1
@@ -282,6 +296,8 @@ async def test_update_public_skill_is_admin_only_and_preserves_raw_config(
assert persisted["skills"]["demo-skill"]["enabled"] is True
assert persisted["concurrentUnknownField"] == {"survives": True}
assert refresh_calls == [True, True]
assert enabled_row_updates == [("demo-skill", False), ("demo-skill", True)]
assert catalog_refresh_calls == []
config_replaces = [item for item in replace_calls if os.fspath(item[1]) == os.fspath(config_path)]
assert len(config_replaces) == 2
@@ -4,7 +4,15 @@ import pytest
from fastapi import HTTPException
from app.gateway.routers import system_settings as system_settings_router
from deerflow.config.system_settings import SystemSettings
from deerflow.config.system_settings import (
AIWritingGuideCard,
AIWritingPageDescription,
GeneralQaPageDescription,
GeneralQaPageDescriptions,
PageDescriptionsSettings,
PageHeaderDescription,
SystemSettings,
)
def test_strategy_project_demo_is_hidden_by_default() -> None:
@@ -15,6 +23,10 @@ def test_assistant_knowledge_is_hidden_by_default() -> None:
assert SystemSettings().knowledge_base_product_version == "v1"
def test_custom_skill_versioning_is_disabled_by_default() -> None:
assert SystemSettings().skill_versioning.enabled is False
@pytest.mark.asyncio
async def test_non_admin_cannot_update_strategy_project_settings(
monkeypatch: pytest.MonkeyPatch,
@@ -194,6 +206,263 @@ def test_retry_keep_partial_defaults_to_enabled() -> None:
assert SystemSettings().retry_keep_partial.enabled is True
def test_page_descriptions_default_to_product_copy() -> None:
descriptions = SystemSettings().page_descriptions
assert descriptions.general_qa.full == GeneralQaPageDescription()
assert descriptions.general_qa.compact == GeneralQaPageDescription()
assert descriptions.knowledge_base == PageHeaderDescription()
assert descriptions.ai_writing == AIWritingPageDescription()
assert len(descriptions.ai_writing.guide_cards) == 3
assert descriptions.strategy_projects == PageHeaderDescription()
assert descriptions.deep_research == PageHeaderDescription()
assert descriptions.gongwen == PageHeaderDescription()
assert descriptions.light_app_management == PageHeaderDescription()
assert descriptions.light_app_list == PageHeaderDescription()
assert descriptions.agents == PageHeaderDescription()
assert descriptions.business_chains == PageHeaderDescription()
assert descriptions.skills == PageHeaderDescription()
assert descriptions.knowledge_management == PageHeaderDescription()
def test_legacy_general_qa_description_is_shared_by_both_layouts() -> None:
descriptions = PageDescriptionsSettings.model_validate(
{
"general_qa": {
"welcome_title": " 原有问答欢迎语 ",
"welcome_description": " 原有说明 ",
"input_placeholder": " 原有提示 ",
}
}
)
expected = GeneralQaPageDescription(
welcome_title="原有问答欢迎语",
welcome_description="原有说明",
input_placeholder="原有提示",
)
assert descriptions.general_qa.full == expected
assert descriptions.general_qa.compact == expected
legacy_request = system_settings_router.PageDescriptionsUpdateRequest(
general_qa=GeneralQaPageDescription(welcome_title="旧版请求"),
)
assert legacy_request.general_qa == GeneralQaPageDescriptions(
full=GeneralQaPageDescription(welcome_title="旧版请求"),
compact=GeneralQaPageDescription(welcome_title="旧版请求"),
)
def test_ai_writing_guide_cards_are_padded_to_three_items() -> None:
descriptions = PageDescriptionsSettings.model_validate(
{
"ai_writing": {
"guide_cards": [
{"title": " 自定义卡片 ", "description": " 自定义说明 "},
]
}
}
)
assert descriptions.ai_writing.guide_cards == [
AIWritingGuideCard(title="自定义卡片", description="自定义说明"),
AIWritingGuideCard(),
AIWritingGuideCard(),
]
@pytest.mark.asyncio
async def test_non_admin_cannot_update_page_descriptions(
monkeypatch: pytest.MonkeyPatch,
) -> None:
async def ordinary_user(_request):
return SimpleNamespace(system_role="user")
monkeypatch.setattr(
system_settings_router,
"get_optional_user_from_request",
ordinary_user,
)
with pytest.raises(HTTPException) as error:
await system_settings_router.update_page_descriptions(
system_settings_router.PageDescriptionsUpdateRequest(
general_qa=GeneralQaPageDescriptions(
full=GeneralQaPageDescription(welcome_title="不可保存"),
),
knowledge_base=PageHeaderDescription(),
ai_writing=AIWritingPageDescription(),
),
SimpleNamespace(),
)
assert error.value.status_code == 403
@pytest.mark.asyncio
async def test_admin_can_update_page_descriptions(
monkeypatch: pytest.MonkeyPatch,
) -> None:
current = SystemSettings()
saved: list[SystemSettings] = []
async def admin_user(_request):
return SimpleNamespace(system_role="admin")
monkeypatch.setattr(
system_settings_router,
"get_optional_user_from_request",
admin_user,
)
monkeypatch.setattr(system_settings_router, "load_system_settings", lambda: current)
monkeypatch.setattr(
system_settings_router,
"save_system_settings",
lambda settings: saved.append(settings) or settings,
)
response = await system_settings_router.update_page_descriptions(
system_settings_router.PageDescriptionsUpdateRequest(
general_qa=GeneralQaPageDescriptions(
full=GeneralQaPageDescription(
welcome_title=" 欢迎使用工程开发智能问答助手 ",
welcome_description=" 请描述您希望完成的工程开发任务。 ",
input_placeholder=" 输入问题或任务描述 ",
),
compact=GeneralQaPageDescription(
welcome_title=" 简洁问答欢迎语 ",
),
),
knowledge_base=PageHeaderDescription(
title=" 资料库 ",
description=" 在这里整理和检索团队资料。 ",
),
ai_writing=AIWritingPageDescription(
title=" 写作工作台 ",
description=" 将想法逐步发展为文章。 ",
guide_cards=[
AIWritingGuideCard(
title=" 明确写作意图 ",
description=" 用一句话说明想写什么。 ",
),
AIWritingGuideCard(
title=" 对齐写作参数 ",
description=" 选择体裁与篇幅。 ",
),
AIWritingGuideCard(
title=" 生成并优化文章 ",
description=" 查看并完善草稿。 ",
),
],
),
strategy_projects=PageHeaderDescription(
title=" 项目工作台 ",
description=" 集中管理各类项目成果。 ",
),
deep_research=PageHeaderDescription(
title=" 开始研究 ",
description=" 收集资料并生成研究报告。 ",
),
gongwen=PageHeaderDescription(
title=" 公文工作台 ",
description=" 对话完成公文起草和校审。 ",
),
light_app_management=PageHeaderDescription(
title=" 轻应用管理 ",
description=" 登记和维护轻应用。 ",
),
light_app_list=PageHeaderDescription(
title=" 轻应用列表 ",
description=" 快速打开已登记的轻应用。 ",
),
agents=PageHeaderDescription(
title=" 智能体工作台 ",
description=" 管理可用智能体。 ",
),
business_chains=PageHeaderDescription(
title=" 业务链目录 ",
description=" 管理可复用的智能体编排。 ",
),
skills=PageHeaderDescription(
title=" 技能工作台 ",
description=" 浏览和维护技能。 ",
),
knowledge_management=PageHeaderDescription(
title=" 知识服务 ",
description=" 管理知识库与访问范围。 ",
),
),
SimpleNamespace(),
)
assert response.general_qa == GeneralQaPageDescriptions(
full=GeneralQaPageDescription(
welcome_title="欢迎使用工程开发智能问答助手",
welcome_description="请描述您希望完成的工程开发任务。",
input_placeholder="输入问题或任务描述",
),
compact=GeneralQaPageDescription(welcome_title="简洁问答欢迎语"),
)
assert response.knowledge_base == PageHeaderDescription(
title="资料库",
description="在这里整理和检索团队资料。",
)
assert response.ai_writing == AIWritingPageDescription(
title="写作工作台",
description="将想法逐步发展为文章。",
guide_cards=[
AIWritingGuideCard(
title="明确写作意图",
description="用一句话说明想写什么。",
),
AIWritingGuideCard(
title="对齐写作参数",
description="选择体裁与篇幅。",
),
AIWritingGuideCard(
title="生成并优化文章",
description="查看并完善草稿。",
),
],
)
assert response.strategy_projects == PageHeaderDescription(
title="项目工作台",
description="集中管理各类项目成果。",
)
assert response.deep_research == PageHeaderDescription(
title="开始研究",
description="收集资料并生成研究报告。",
)
assert response.gongwen == PageHeaderDescription(
title="公文工作台",
description="对话完成公文起草和校审。",
)
assert response.light_app_management == PageHeaderDescription(
title="轻应用管理",
description="登记和维护轻应用。",
)
assert response.light_app_list == PageHeaderDescription(
title="轻应用列表",
description="快速打开已登记的轻应用。",
)
assert response.agents == PageHeaderDescription(
title="智能体工作台",
description="管理可用智能体。",
)
assert response.business_chains == PageHeaderDescription(
title="业务链目录",
description="管理可复用的智能体编排。",
)
assert response.skills == PageHeaderDescription(
title="技能工作台",
description="浏览和维护技能。",
)
assert response.knowledge_management == PageHeaderDescription(
title="知识服务",
description="管理知识库与访问范围。",
)
assert saved[0].page_descriptions.general_qa == response.general_qa
@pytest.mark.asyncio
async def test_retry_keep_partial_get_reflects_saved_settings(
monkeypatch: pytest.MonkeyPatch,
@@ -266,3 +535,83 @@ async def test_admin_can_update_retry_keep_partial(
assert response.enabled is False
assert saved[0].retry_keep_partial.enabled is False
@pytest.mark.asyncio
async def test_skill_versioning_get_reflects_saved_setting(
monkeypatch: pytest.MonkeyPatch,
) -> None:
current = SystemSettings().model_copy(
update={
"skill_versioning": system_settings_router.SkillVersioningSettings(
enabled=True
)
}
)
monkeypatch.setattr(system_settings_router, "load_system_settings", lambda: current)
response = await system_settings_router.get_skill_versioning()
assert response.enabled is True
@pytest.mark.asyncio
async def test_non_admin_cannot_update_skill_versioning(
monkeypatch: pytest.MonkeyPatch,
) -> None:
async def ordinary_user(_request):
return SimpleNamespace(system_role="user")
monkeypatch.setattr(
system_settings_router,
"get_optional_user_from_request",
ordinary_user,
)
with pytest.raises(HTTPException) as error:
await system_settings_router.update_skill_versioning(
system_settings_router.SkillVersioningUpdateRequest(enabled=True),
SimpleNamespace(),
)
assert error.value.status_code == 403
@pytest.mark.asyncio
async def test_admin_can_enable_skill_versioning_and_capture_baselines(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from deerflow.skills import versioning
current = SystemSettings()
saved: list[SystemSettings] = []
reconciled: list[str | None] = []
async def admin_user(_request):
return SimpleNamespace(system_role="admin", id="admin-1")
monkeypatch.setattr(
system_settings_router,
"get_optional_user_from_request",
admin_user,
)
monkeypatch.setattr(system_settings_router, "load_system_settings", lambda: current)
monkeypatch.setattr(
system_settings_router,
"save_system_settings",
lambda settings: saved.append(settings) or settings,
)
monkeypatch.setattr(
versioning,
"reconcile_custom_skill_versions",
lambda *, author=None: reconciled.append(author) or 0,
)
response = await system_settings_router.update_skill_versioning(
system_settings_router.SkillVersioningUpdateRequest(enabled=True),
SimpleNamespace(),
)
assert response.enabled is True
assert saved[0].skill_versioning.enabled is True
assert reconciled == ["admin-1"]
@@ -0,0 +1,597 @@
"""结构化 HTML 模板(kind=structured,先拿 no.1 落地)的引擎/存储/路由测试。
覆盖:
* seed/meta:结构化 no1 四件套正确入库,单文件模板(如 no2)不受影响;
* 契约语义与单文件时代等价:``reference_data`` 读 data.json 与旧 ``const DATA=`` 行
解析结果一致,``validate_data`` / ``merge_with_reference`` 行为不变;
* 组合:``read_template_text`` / ``fill_template`` 产出无残留标记的自包含文档,
数据注入可被页面 ``JSON.parse`` 还原,标题/副标题/更新时间生效;
* data.json 持久化(管理页「编辑数据」):整体替换 + 契约校验(顶层键不可增删);
* 骨架编辑标记校验、结构化重置;
* 旧单文件存储副本的自动升级:未手改 → 升级;手改过 → 保留旧版;
* 路由:content 的 doc/raw 双形态、data 的 GET/PUT 与权限。
"""
from __future__ import annotations
import json
import re
from types import SimpleNamespace
from typing import Any
import pytest
from fastapi import HTTPException
from test_api_read_cache import FakeRedis, _config
from app.gateway import api_read_cache
from app.gateway.routers import html_templates
from deerflow.runtime.scheduler import template_fill, template_store
# no1 是当前唯一的结构化模板。
TPL = "no1"
LEGACY_FILE = template_fill._TEMPLATES_ROOT / TPL / "taiwan_littoral_ui_redwhite_v2.html"
@pytest.fixture
def tmp_store(monkeypatch, tmp_path):
"""隔离的运行时存储:全部用例不触碰真实 .deer-flow 目录。"""
root = tmp_path / "html-templates"
monkeypatch.setattr(template_store, "store_root", lambda: root)
template_fill.invalidate_cache()
template_store.ensure_seeded()
yield root
template_fill.invalidate_cache()
@pytest.fixture
def api_request(monkeypatch, tmp_store):
"""可直接调路由函数的最小 request(admin 身份,目录缓存走 FakeRedis)。"""
redis = FakeRedis()
monkeypatch.setattr("deerflow.persistence.engine.get_session_factory", lambda: None)
monkeypatch.setattr(api_read_cache, "_api_read_cache_settings_cache", None)
monkeypatch.setattr(api_read_cache, "_client", lambda *a, **k: (redis, "test:catalog"))
request = SimpleNamespace(
app=SimpleNamespace(state=SimpleNamespace(config=_config())),
state=SimpleNamespace(user=SimpleNamespace(id="admin-1", system_role="admin")),
)
async def current_user(req):
return req.state.user
monkeypatch.setattr(html_templates, "get_optional_user_from_request", current_user)
return request
def _page_data_element(html: str) -> dict:
match = re.search(
r'<script type="application/json" id="page-data">(.*?)</script>', html, re.DOTALL
)
assert match is not None, "组合产物缺少 page-data JSON 元素"
return json.loads(match.group(1))
# ── seed / meta ──────────────────────────────────────────────────────────────
def test_structured_seed_and_single_untouched(tmp_store):
meta = template_store.get_template_meta(TPL)
assert meta is not None
assert meta["kind"] == "structured"
assert meta["convertible"] is True
assert meta["manifest_var"] == "MANIFEST"
assert sorted(p.name for p in (tmp_store / TPL).iterdir()) == [
"data.json",
"fields.json",
"manifest.json",
"meta.json",
"runtime.js",
"template.html",
]
# 单文件模板照旧(kind=single,正文 = 完整文档,无结构化部件)。
no2 = template_store.get_template_meta("no2")
assert no2 is not None and no2["kind"] == "single"
assert not (tmp_store / "no2" / "runtime.js").exists()
def test_reference_data_matches_legacy_const_line(tmp_store):
"""data.json 的契约语义必须与单文件时代的 const DATA= 行完全一致。"""
legacy = template_fill._extract_const_object(LEGACY_FILE.read_text(encoding="utf-8"), "DATA")
ref = template_fill.reference_data(TPL)
assert ref == legacy
assert template_fill.validate_data(TPL, legacy) == []
assert set(template_fill.top_level_keys(TPL)) == set(legacy.keys())
# ── 组合 / 填充 ─────────────────────────────────────────────────────────────
def test_read_template_text_composes_default_document(tmp_store):
html = template_fill.read_template_text(TPL)
for marker in template_fill.STRUCTURED_MARKERS:
assert marker not in html
assert 'id="page-data"' in html and 'id="page-manifest"' in html
assert "const DATA=JSON.parse" in html
# 页面内 JSON 编辑器随 runtime 内联。
assert 'id="editorOverlay"' in html and 'id="applyJson"' in html
assert _page_data_element(html)["overview"] == template_fill.reference_data(TPL)["overview"]
def test_fill_template_structured_injects_data_titles_and_date(tmp_store):
ref = template_fill.reference_data(TPL)
modified = json.loads(json.dumps(ref))
modified["overview"] = {
"核心指标": [{"名称": "测试指标", "数值": 7, "单位": "个", "说明": "单元测试"}]
}
import datetime as dt
html = template_fill.fill_template(
TPL,
modified,
update_date=dt.date(2026, 11, 4),
title="新标题",
subtitle="新副标题",
)
for marker in template_fill.STRUCTURED_MARKERS:
assert marker not in html
assert "<title>新标题</title>" in html
assert "<h1>新标题</h1>" in html
assert "新副标题" in html
# MANIFEST 系统.更新时间/名称 同步。
assert "2026-11-04" in html
assert '"名称": "新标题"' in html
# 注入的数据可被页面 JSON.parse 原样还原。
assert _page_data_element(html)["overview"]["核心指标"][0]["名称"] == "测试指标"
def test_merge_and_validate_semantics_unchanged(tmp_store):
# 顶层键缺失 → 硬校验报错(与单文件时代一致)。
errors = template_fill.validate_data(TPL, {"overview": {}})
assert any("缺少顶层字段" in e for e in errors)
# 空内容分支回退模板默认数据。
merged = template_fill.merge_with_reference(TPL, {"overview": {}, "regions": []})
assert merged == template_fill.reference_data(TPL)
# 有内容的分支保留转换结果。
merged2 = template_fill.merge_with_reference(
TPL, {"overview": {"核心指标": [{"名称": "x", "数值": 1, "单位": "", "说明": ""}]}}
)
assert merged2["overview"]["核心指标"][0]["名称"] == "x"
assert merged2["regions"] == template_fill.reference_data(TPL)["regions"]
# ── data.json 持久化 / 骨架编辑 / 重置 ──────────────────────────────────────
def test_write_read_template_data_roundtrip(tmp_store):
ref = template_fill.reference_data(TPL)
modified = json.loads(json.dumps(ref))
modified["overview"]["核心指标"][0]["名称"] = "持久化指标"
meta = template_store.write_template_data(TPL, modified)
assert meta["kind"] == "structured"
assert template_store.read_template_data(TPL)["overview"]["核心指标"][0]["名称"] == "持久化指标"
# 引擎缓存已失效:reference_data / 组合产物立即反映新数据。
assert template_fill.reference_data(TPL)["overview"]["核心指标"][0]["名称"] == "持久化指标"
assert "持久化指标" in template_fill.read_template_text(TPL)
def test_write_template_manifest_persists_and_composes(tmp_store):
"""MANIFEST 持久化(系统名称/导航页面/交互规则):写回 manifest.json 并进入组合产物。"""
ref = template_fill.reference_data(TPL)
manifest = template_store.read_template_manifest(TPL)
assert manifest.get("系统", {}).get("名称") == "台湾滨海防卫知识数据展示系统"
manifest["系统"]["名称"] = "改名后的系统"
manifest["页面"][0]["名称"] = "首页"
template_store.write_template_data(TPL, ref, manifest=manifest)
assert template_store.read_template_manifest(TPL)["系统"]["名称"] == "改名后的系统"
# 组合产物(MANIFEST 内嵌 JSON)反映新值;导航按钮由页面 JS 依据 MANIFEST 渲染。
composed = template_fill.read_template_text(TPL)
assert "改名后的系统" in composed
assert '"名称": "首页"' in composed
# 非对象 MANIFEST 拒绝。
with pytest.raises(template_store.TemplateStoreError, match="MANIFEST"):
template_store.write_template_data(TPL, ref, manifest=["nope"])
# 重置回随包原版。
template_store.reset_template(TPL)
assert template_store.read_template_manifest(TPL)["系统"]["名称"] == "台湾滨海防卫知识数据展示系统"
def test_write_template_data_rejects_contract_changes(tmp_store):
with pytest.raises(template_store.TemplateStoreError, match="契约"):
template_store.write_template_data(TPL, {"overview": {}})
with pytest.raises(template_store.TemplateStoreError):
template_store.write_template_data(TPL, ["not", "an", "object"])
# 单文件模板不支持数据编辑。
with pytest.raises(template_store.TemplateStoreError, match="结构化"):
template_store.write_template_data("no2", {"x": 1})
def test_update_template_skeleton_marker_guard(tmp_store):
with pytest.raises(template_store.TemplateStoreError, match="注入标记"):
template_store.update_template(TPL, html="<!doctype html><p>标记丢了</p>")
# 保留标记的骨架编辑可以通过,且不影响 kind/convertible。
skeleton = template_store.read_template_html(TPL)
edited = skeleton.replace("JSON 编辑器", "JSON 编辑器·编辑", 1)
meta = template_store.update_template(TPL, html=edited)
assert meta["kind"] == "structured" and meta["convertible"] is True
assert template_fill.read_template_text(TPL).count("JSON 编辑器·编辑") == 1
def test_reset_structured_restores_all_parts(tmp_store):
template_store.write_template_data(
TPL, _with_first_metric_name("持久化指标")
)
template_store.update_template(TPL, html=template_store.read_template_html(TPL) + "\n<!-- edited -->")
template_store.reset_template(TPL)
assert template_store.read_template_data(TPL)["overview"]["核心指标"][0]["名称"] != "持久化指标"
assert "<!-- edited -->" not in template_store.read_template_html(TPL)
assert template_fill.read_template_text(TPL) # 组合仍可用
def _with_first_metric_name(name: str) -> dict:
ref = template_fill.reference_data(TPL)
modified = json.loads(json.dumps(ref))
modified["overview"]["核心指标"][0]["名称"] = name
return modified
# ── 旧单文件存储副本的自动升级 ─────────────────────────────────────────────
def _make_legacy_copy(store_root, *, modified: bool) -> None:
import shutil
tpl_dir = store_root / TPL
if tpl_dir.exists():
shutil.rmtree(tpl_dir)
tpl_dir.mkdir(parents=True)
text = LEGACY_FILE.read_text(encoding="utf-8")
if modified:
text += "\n<!-- hand edit -->"
(tpl_dir / "template.html").write_text(text, encoding="utf-8")
(tpl_dir / "meta.json").write_text(
json.dumps({"id": TPL, "name": "no.1", "builtin": True, "convertible": True}),
encoding="utf-8",
)
def test_auto_upgrade_untouched_legacy_copy(tmp_store):
_make_legacy_copy(tmp_store, modified=False)
assert template_store.ensure_seeded() is True
meta = template_store.get_template_meta(TPL)
assert meta["kind"] == "structured"
assert (tmp_store / TPL / "runtime.js").is_file()
# 升级后的模板可正常组合。
assert "__PAGE_" not in template_fill.read_template_text(TPL)
def test_auto_upgrade_skips_hand_modified_copy(tmp_store):
_make_legacy_copy(tmp_store, modified=True)
template_store.ensure_seeded()
meta = template_store.get_template_meta(TPL)
assert meta["kind"] == "single"
assert "<!-- hand edit -->" in (tmp_store / TPL / "template.html").read_text(encoding="utf-8")
# 管理页「重置」可手动升级到结构化版。
template_store.reset_template(TPL)
assert template_store.get_template_meta(TPL)["kind"] == "structured"
assert (tmp_store / TPL / "runtime.js").is_file()
# ── 路由 ────────────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_router_content_doc_and_raw(tmp_store, api_request):
doc = await html_templates.get_html_template_content(TPL, api_request)
assert "__PAGE_DATA__" not in doc["html"]
assert 'id="page-data"' in doc["html"]
raw = await html_templates.get_html_template_content(TPL, api_request, raw=True)
assert "__PAGE_DATA__" in raw["html"] and "__PAGE_RUNTIME__" in raw["html"]
@pytest.mark.asyncio
async def test_router_data_get_put_and_guards(tmp_store, api_request):
got = await html_templates.get_html_template_data(TPL, api_request)
assert "overview" in got["data"]
# MANIFEST 随 GET 返回(管理页「系统设置」模块)。
assert got["manifest"]["系统"]["名称"] == "台湾滨海防卫知识数据展示系统"
# 模块中文名(按 MANIFEST.页面 解析,管理页「编辑数据」的模块导航用)。
labels = {m["key"]: m["label"] for m in got["modules"]}
assert labels["overview"] == "总览"
assert labels["regions"] == "沿岸区域"
assert labels["organization_charts"] == "组织与单位"
assert labels["events"] == "演习与项目"
assert set(labels) == set(got["data"].keys())
payload = got["data"]
payload["overview"]["核心指标"][0]["名称"] = "路由持久化"
manifest = json.loads(json.dumps(got["manifest"]))
manifest["系统"]["名称"] = "路由改名系统"
meta = await html_templates.update_html_template_data(
TPL,
html_templates.TemplateDataUpdateRequest(data=payload, manifest=manifest),
api_request,
)
assert meta["kind"] == "structured"
assert template_store.read_template_data(TPL)["overview"]["核心指标"][0]["名称"] == "路由持久化"
assert template_store.read_template_manifest(TPL)["系统"]["名称"] == "路由改名系统"
# 仅传 data(未动 MANIFEST)时不回写 manifest.json。
await html_templates.update_html_template_data(
TPL, html_templates.TemplateDataUpdateRequest(data=payload), api_request
)
assert template_store.read_template_manifest(TPL)["系统"]["名称"] == "路由改名系统"
# 契约被破坏(删顶层键) → 400。
broken = {k: v for k, v in payload.items() if k != "regions"}
with pytest.raises(HTTPException) as exc:
await html_templates.update_html_template_data(
TPL, html_templates.TemplateDataUpdateRequest(data=broken), api_request
)
assert exc.value.status_code == 400
# 单文件模板 → 400。
with pytest.raises(HTTPException) as exc2:
await html_templates.get_html_template_data("no2", api_request)
assert exc2.value.status_code == 400
# 非管理员不可写。
api_request.state.user = SimpleNamespace(id="u2", system_role="user")
with pytest.raises(HTTPException) as exc3:
await html_templates.update_html_template_data(
TPL, html_templates.TemplateDataUpdateRequest(data=payload), api_request
)
assert exc3.value.status_code == 403
@pytest.mark.asyncio
async def test_scheduler_fill_path_yields_selfcontained_html(tmp_store):
"""调度器模板填充路径的最终产物自包含:单文件、无标记、数据可解析。"""
data_obj = template_fill.merge_with_reference(TPL, {"overview": _with_first_metric_name("调度指标")["overview"]})
html = template_fill.fill_template(TPL, data_obj, update_date=__import__("datetime").date(2026, 1, 1))
assert html.startswith("<!doctype html>") and "__PAGE_" not in html
assert "调度指标" in html
assert _page_data_element(html)["overview"]["核心指标"][0]["名称"] == "调度指标"
# ── 字段显示 / 自定义模块 ───────────────────────────────────────────────────
def test_field_catalog_and_visibility_roundtrip(tmp_store):
catalog = template_store.read_template_fields(TPL)
labels = {f["label"] for f in catalog}
assert "节点类别" in labels and "来源" in labels
assert all(f.get("description") for f in catalog)
# 隐藏两个字段 → manifest 记 {label: false},其余全部显示。
template_store.write_template_field_visibility(TPL, ["节点类别", "来源"])
manifest = template_store.read_template_manifest(TPL)
assert manifest["字段显示"] == {"节点类别": False, "来源": False}
# 组合产物带隐藏配置,runtime 的 F() 据此不渲染。
composed = template_fill.read_template_text(TPL)
assert '"字段显示"' in composed
# 目录外字段拒绝。
with pytest.raises(template_store.TemplateStoreError, match="未知字段"):
template_store.write_template_field_visibility(TPL, ["不存在的字段"])
# 置空 = 全部显示。
template_store.write_template_field_visibility(TPL, [])
assert template_store.read_template_manifest(TPL)["字段显示"] == {}
def test_custom_modules_write_and_sanitize(tmp_store):
raw = (
'<div class="panel"><h3 onclick="evil()">x</h3><script>alert(1)</script>'
'<a href="javascript:alert(2)">链接</a><p>正常内容</p></div>'
)
template_store.write_template_custom_modules(
TPL, [{"编号": "c1", "页面": "overview", "标题": "测试模块", "html": raw}]
)
saved = template_store.read_template_manifest(TPL)["自定义模块"]
assert len(saved) == 1
html = saved[0]["html"]
assert "正常内容" in html
assert "<script" not in html and "onclick" not in html and "javascript:" not in html
# 组合产物包含模块内容与渲染函数。
composed = template_fill.read_template_text(TPL)
assert "正常内容" in composed and "renderCustomModules" in composed
# 结构校验:缺页面/缺 html 拒绝。
with pytest.raises(template_store.TemplateStoreError, match="页面"):
template_store.write_template_custom_modules(TPL, [{"html": "<div>x</div>"}])
with pytest.raises(template_store.TemplateStoreError, match="html"):
template_store.write_template_custom_modules(TPL, [{"页面": "overview"}])
def test_field_extract_locks_and_restore(tmp_store):
"""字段抽取锁定:锁定字段的键在抽取结果合并后恢复模板现有值(按条目身份匹配)。"""
ref = template_fill.reference_data(TPL)
# 锁定「记录类别」(org_units 条目的 记录类别 键)与「日期」(events 条目的 日期显示 键)。
template_store.write_template_field_locks(TPL, ["记录类别", "日期"])
keys = template_store.locked_field_keys(TPL)
assert "记录类别" in keys and "日期显示" in keys
# 模拟抽取结果:改了第一个单位的记录类别、事件日期,并保留身份键(id/名称)。
extracted = json.loads(json.dumps(ref))
unit = extracted["org_units"][0]
old_cat = ref["org_units"][0]["记录类别"]
unit["记录类别"] = "被抽取改过的类别"
event = extracted["events"][0]
event["日期显示"] = "2026-01-01"
event["名称"] = ref["events"][0].get("名称") or event.get("名称") or event.get("name") or "同身份"
restored = template_store.restore_locked_fields(TPL, extracted)
# 锁定键恢复旧值。
assert restored["org_units"][0]["记录类别"] == old_cat
assert restored["events"][0]["日期显示"] == ref["events"][0]["日期显示"]
# 未锁定分支不动。
assert restored["overview"] == extracted["overview"]
# 目录外字段拒绝。
with pytest.raises(template_store.TemplateStoreError, match="未知字段"):
template_store.write_template_field_locks(TPL, ["不存在字段"])
# 清空 = 全部接受抽取。
template_store.write_template_field_locks(TPL, [])
assert template_store.restore_locked_fields(TPL, extracted) == extracted
@pytest.mark.asyncio
async def test_router_fields_extract_flags(tmp_store, api_request):
got = await html_templates.get_html_template_fields(TPL, api_request)
by_label = {f["label"]: f for f in got["fields"]}
assert by_label["记录类别"]["extract"] is True
assert by_label["记录类别"]["has_key"] is True
assert by_label["组织与单位概况"]["has_key"] is False # 布局分区
meta = await html_templates.update_html_template_fields(
TPL,
html_templates.FieldVisibilityUpdateRequest(hidden=[], extract_locked=["记录类别"]),
api_request,
)
assert meta["kind"] == "structured"
got2 = await html_templates.get_html_template_fields(TPL, api_request)
by2 = {f["label"]: f for f in got2["fields"]}
assert by2["记录类别"]["extract"] is False and by2["日期"]["extract"] is True
@pytest.mark.asyncio
async def test_router_fields_and_custom_modules(tmp_store, api_request):
got = await html_templates.get_html_template_fields(TPL, api_request)
assert any(f["label"] == "节点类别" and f["visible"] for f in got["fields"])
assert [p["key"] for p in got["pages"]][0] == "overview"
assert got["custom_modules"] == []
meta = await html_templates.update_html_template_fields(
TPL, html_templates.FieldVisibilityUpdateRequest(hidden=["节点类别"]), api_request
)
assert meta["kind"] == "structured"
got2 = await html_templates.get_html_template_fields(TPL, api_request)
by_label = {f["label"]: f["visible"] for f in got2["fields"]}
assert by_label["节点类别"] is False and by_label["来源"] is True
await html_templates.update_html_template_custom_modules(
TPL,
html_templates.CustomModuleListUpdateRequest(
modules=[{"页面": "overview", "标题": "t", "html": "<div>hi</div>"}]
),
api_request,
)
got3 = await html_templates.get_html_template_fields(TPL, api_request)
assert got3["custom_modules"][0]["html"] == "<div>hi</div>"
# 非管理员不可写。
api_request.state.user = SimpleNamespace(id="u2", system_role="user")
with pytest.raises(HTTPException) as exc:
await html_templates.update_html_template_fields(
TPL, html_templates.FieldVisibilityUpdateRequest(hidden=[]), api_request
)
assert exc.value.status_code == 403
# ── 定时任务「抽取信息写回模板数据」开关(html_config.template_data_writeback) ──
def _writeback_task(writeback: bool | None) -> dict[str, Any]:
ctx: dict[str, Any] = {"require_html": True, "html_config": {}}
if writeback is not None:
ctx["html_config"]["template_data_writeback"] = writeback
return {
"task_id": "t1",
"user_id": "u1",
"name": "回写联调",
"prompt": "更新总览的核心指标名称为「回写指标」",
"description": "",
"timezone": "UTC",
"enabled": True,
"execution_context": ctx,
}
def _build_writeback_service(monkeypatch, *, converted: dict | None):
"""构造 mock 掉检索/转换/归档的 service,返回 (service, write_mock)。"""
from datetime import UTC
from datetime import datetime as dt
from unittest.mock import AsyncMock, MagicMock
from deerflow.persistence.scheduled_tasks import MemoryScheduledTaskStore
from deerflow.runtime.scheduler import service as scheduler_service
from deerflow.runtime.scheduler.service import ScheduledTaskService
service = ScheduledTaskService(app=None, store=MemoryScheduledTaskStore(), poll_interval_seconds=60.0)
ref = template_fill.reference_data(TPL)
source_obj = json.loads(json.dumps(ref))
source_obj["overview"] = {"核心指标": [{"名称": "回写指标", "数值": 3, "单位": "个", "说明": ""}]}
monkeypatch.setattr(
service, "_gather_html_data", AsyncMock(return_value=("一些源数据", []))
)
if converted is None:
async def _fail(*a, **k):
raise scheduler_service.TemplateFillError("boom")
monkeypatch.setattr(service, "_convert_to_template_data", AsyncMock(side_effect=_fail))
else:
# 成功分支返回「抽取了新指标」的转换结果(converted 仅作成功/失败哨兵)。
monkeypatch.setattr(service, "_convert_to_template_data", AsyncMock(return_value=source_obj))
monkeypatch.setattr(service, "_finalize_template_html", AsyncMock(return_value={"ok": True}))
monkeypatch.setattr(service, "_archive_html_index", lambda *_a, **_k: None)
# write_template_data 是同步函数,用 MagicMock 而非 AsyncMock。
write_mock = MagicMock()
monkeypatch.setattr(template_store, "write_template_data", write_mock)
scheduled_for = dt(2026, 11, 4, 12, 0, tzinfo=UTC)
return service, write_mock, scheduled_for, source_obj
@pytest.mark.asyncio
@pytest.mark.parametrize("writeback,converted,expect_write", [
(True, "converted", True),
(False, "converted", False),
(None, "converted", False), # 老任务缺省关闭
(True, None, False), # 转换失败(回退默认数据)不写回
])
async def test_html_template_reference_writeback_switch(tmp_store, monkeypatch, writeback, converted, expect_write):
service, write_mock, scheduled_for, _source = _build_writeback_service(
monkeypatch, converted=converted
)
await service._run_html_template_reference(
_writeback_task(writeback),
tpl_id=TPL,
runtime_thread_id="rt",
scheduled_for=scheduled_for,
run_id="r1",
)
if expect_write:
assert write_mock.call_count == 1
# 只写 data.json:调用参数不含 manifest kwarg(系统名称等 MANIFEST 不动)。
args, kwargs = write_mock.call_args
assert args[0] == TPL and "manifest" not in kwargs
assert args[1]["overview"]["核心指标"][0]["名称"] == "回写指标"
else:
assert write_mock.call_count == 0
@pytest.mark.asyncio
async def test_writeback_respects_field_locks(tmp_store, monkeypatch):
"""锁定字段不接受抽取更新:写回 data.json 的数据里,锁定键保留模板旧值。"""
template_store.write_template_field_locks(TPL, ["日期"])
service, write_mock, scheduled_for, source_obj = _build_writeback_service(
monkeypatch, converted=source_marker()
)
await service._run_html_template_reference(
_writeback_task(True),
tpl_id=TPL,
runtime_thread_id="rt",
scheduled_for=scheduled_for,
run_id="r-lock",
)
assert write_mock.call_count == 1
args, _kwargs = write_mock.call_args
old_dates = [e.get("日期显示") for e in template_fill.reference_data(TPL)["events"]]
new_dates = [e.get("日期显示") for e in args[1]["events"]]
assert new_dates == old_dates # 锁定键恢复旧值
def source_marker():
ref = template_fill.reference_data(TPL)
obj = json.loads(json.dumps(ref))
for e in obj["events"]:
e["日期显示"] = "2099-12-31"
return obj
+3 -1
View File
@@ -1,8 +1,10 @@
config_version: 8
log_level: info
# 高并发运行时优化。技能视图两项不写也默认开启,要关掉才写成 false。修改后需重启后台。
# 高并发运行时优化。工具/技能统计与技能视图默认开启;若确有写库压力才改成 false。修改后需重启后台。
runtime_performance:
tool_metrics_enabled: true # 记录普通工具调用,供综合管理的工具排行/调用统计使用
skill_metrics_enabled: true # 记录可归因到具体技能的调用,供综合管理的技能排行/调用统计使用
skill_view_reuse_enabled: true # 默认开启;false 才关闭。单次工具调用复用一次技能视图
skill_view_cache_enabled: true # 默认开启;false 才关闭。校验版本后复用未变化的技能视图
model_timing_nonblocking_enabled: true # 模型计时日志独立队列,不争抢工具线程池