Merge branch 'master' of http://192.168.0.200/knowledge-base-bj/deerflow-server into dhk-dev
# Conflicts: # frontend-web/src/open-canvas/components/ai-writing/AIWritingForm.tsx # frontend-web/src/open-canvas/components/ai-writing/AIWritingPanel.tsx # offline-backend-20260512/backend/CLAUDE.md # offline-backend-20260512/backend/app/gateway/services.py # offline-backend-20260512/backend/tests/test_ai_writing_knowledge_base.py
This commit is contained in:
commit
221e947488
138 files changed
+65056
-2405
No files matched your search
@@ -281,6 +281,8 @@ Seeding mirrors the roundtable pattern: verified `config.yaml` + `SOUL.md` ship
|
||||
|
||||
**Skill-driven retrieval (检索类型).** `material_source` has three values: `general` (通用检索, default), `knowledge_base` (知识库检索), and `notebook` (我的空间); legacy `skill` is normalized to `general` by the researcher node. **`knowledge_base` 直连内网 ES 接口(= 旧版「内网直连检索」)** — it bypasses skill orchestration entirely and calls `IntranetSearchProvider` over `ai_writing.intranet_search_url` (concurrent over the generated keywords; `intranet_knowledge_base`/`intranet_search_timeout` apply). **An empty/unconfigured `intranet_search_url` does NOT disable it** — the empty string is passed straight to `IntranetSearchProvider`, which falls back to its built-in default endpoint (`intranet_stub._DEFAULT_ENDPOINT`), exactly like the legacy `get_search_provider(search_provider=intranet)` path. So 内网直连检索 works out of the box without configuring an address (configure `intranet_search_url` only to point at a different endpoint). This is the "弱模型识别不了技能" escape hatch — the old no-skill retrieval path restored as an explicit option (weak models like deepseek-chat often fail to invoke configured skills). Tests: `tests/test_ai_writing_knowledge_base.py`. General retrieval is orchestrated from the **skills configured on the `ai-writing-researcher` agent** (`config.yaml → skills`, editable from the agent management page): the built-in `knowledge-base-search` skill (知识库检索) runs the native keyword fan-out web/intranet search (the old default path), while any other configured skill searches with a `description`-targeted query; skills run in configured order with early-stop at `max_materials`, and an empty/unresolvable skills config falls back to the built-in knowledge search so retrieval always works. The skill itself is seeded from `_ai_writing_seed_assets/skills/knowledge-base-search/SKILL.md` into `skills/public/` (never overwrites), and a pre-existing researcher `config.yaml` lacking the `skills` key is patched in place with the default (explicit `skills: []` is respected). **PAUSE-1 input append flow**: when the user types natural language into the material-confirm input (`re_search`/legacy `enable_skill_search` + `user_query`), the node matches the most relevant configured skill via `SkillPicker(candidates=…)` (skipped when only one skill is configured), searches with the user's words, and **appends** the new materials to the existing package (URL-dedup, original order preserved, no LLM re-rank/truncation, 100-item hard cap). **Skill-failure fault tolerance**: a configured skill that times out / raises / has all its keyword searches fail (vs. legitimately returning 0 results) no longer just silently moves on — the researcher node emits a `skill_search_step` of type `skill_error` (carrying the error `reason`), and if the configured skills collected **no** new materials at all it runs a `fallback` pass over other available skills (`_fallback_candidate_skills`: the built-in `knowledge-base-search` native path first, then other enabled catalog skills not yet tried) instead of erroring out. The frontend `SkillStepsCard` renders `skill_error` (rose) and `fallback` (amber) so the timeline shows "技能报错 → 改用其他技能搜集". Tests: `tests/test_ai_writing_material_source.py`.
|
||||
|
||||
The intranet client uses `trust_env=False` and defaults `intranet_verify_ssl=false`, so internal HTTPS endpoints with self-signed/internal-CA certificates do not fail on certificate verification; set `ai_writing.intranet_verify_ssl: true` only for environments that require strict certificate checks.
|
||||
|
||||
**Skill-call timeout** is hardcoded **300s (5 min)** — `researcher.py` `_SKILL_SEARCH_TIMEOUT_SECONDS` (per-skill wrapper) and `_skill_agent.py` `_REAL_AGENT_TIMEOUT_SECONDS` (whole real-agent skill run). The intranet ES fallback (`IntranetSearchProvider`) gets a longer **600s (10 min)** default (`intranet_stub.py`; overridable via `ai_writing.intranet_search_timeout`) — that interface is the slow one, so our own fallback timeout is generous to avoid being blamed for "检不出".
|
||||
|
||||
**Parallel intranet fallback (技能 0 条 → 直接用并行预跑的内网结果).** When `ai_writing.intranet_search_url` is configured, the researcher node launches the intranet ES search (`IntranetSearchProvider` over the generated keywords) as a **concurrent `asyncio.Task` the moment the skill loop starts** (not sequentially after all skills fail). If the configured + fallback skills collect **nothing**, the node awaits that already-running task and uses its results (`_collect`) — no extra wait. If the skills **did** get materials, the task is cancelled (`_cancel_intranet_task`, suppresses CancelledError) so the intranet call isn't wasted. URL empty → no task, behavior unchanged. Tests: `tests/test_ai_writing_intranet_parallel.py`.
|
||||
|
||||
@@ -138,6 +138,10 @@ AI writing supports a sample-imitation mode: the frontend can upload or paste a
|
||||
|
||||
Custom agents are indexed in the `agents` database table. The `id` column is the stable runtime identifier and filesystem directory name (`.deer-flow/agents/{id}/SOUL.md`); `name` is display-only, may be Chinese, and may be duplicated. Rows with `user_id = NULL` are built-in agents, user rows are private unless `published = true`, and users can only update or delete agents they created.
|
||||
|
||||
### AI Writing Intranet Retrieval
|
||||
|
||||
AI 写作的「知识库检索」会直连 `ai_writing.intranet_search_url` 内网 ES/知识库接口,并默认跳过代理环境变量和 HTTPS 证书校验,适配自签名或内部 CA 证书的内网服务。需要强校验时可在 `config.yaml` 里设置 `ai_writing.intranet_verify_ssl: true`。
|
||||
|
||||
### IM Channels
|
||||
|
||||
The IM bridge supports Feishu, Slack, and Telegram. Slack and Telegram still use the final `runs.wait()` response path, while Feishu now streams through `runs.stream(["messages-tuple", "values"])` and updates a single in-thread card in place.
|
||||
|
||||
@@ -54,6 +54,7 @@ class ActiveRunItem(BaseModel):
|
||||
user_name: str = ""
|
||||
agent_id: str = ""
|
||||
agent_name: str = ""
|
||||
model_name: str = ""
|
||||
thread_title: str = ""
|
||||
kind: str = "对话"
|
||||
multitask_strategy: str = "reject"
|
||||
@@ -245,6 +246,7 @@ async def list_active_runs(request: Request) -> ActiveRunsResponse:
|
||||
user_name=user_emails.get(user_id) or user_id or "未知用户",
|
||||
agent_id=assistant_id,
|
||||
agent_name=agent_name,
|
||||
model_name=str((r.metadata or {}).get("model_name") or ""),
|
||||
thread_title=(meta_row.display_name if meta_row else "") or "",
|
||||
kind=_kind_for(metadata),
|
||||
multitask_strategy=r.multitask_strategy,
|
||||
|
||||
@@ -4,9 +4,11 @@ import logging
|
||||
import mimetypes
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import zipfile
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
@@ -220,6 +222,7 @@ class SkillInstallRequest(BaseModel):
|
||||
|
||||
thread_id: str = Field(..., description="The thread ID where the .skill file is located")
|
||||
path: str = Field(..., description="Virtual path to the .skill file (e.g., mnt/user-data/outputs/my-skill.skill)")
|
||||
skill_name: str | None = Field(default=None, description="Optional custom install name. When set, rewrites the archive SKILL.md frontmatter name before install.")
|
||||
target_category: SkillCategory = Field(default=SkillCategory.CUSTOM, description="Destination category: 'public' (admin only) or 'custom'")
|
||||
|
||||
|
||||
@@ -233,6 +236,32 @@ class SkillInstallResponse(BaseModel):
|
||||
category: str = Field(default="custom", description="Destination category: 'public' (admin) or 'custom'")
|
||||
|
||||
|
||||
class SkillInstallPreviewRequest(BaseModel):
|
||||
"""Request model for preflighting a .skill artifact install."""
|
||||
|
||||
thread_id: str = Field(..., description="The thread ID where the .skill file is located")
|
||||
path: str = Field(..., description="Virtual path to the .skill file")
|
||||
target_category: SkillCategory = Field(default=SkillCategory.CUSTOM, description="Destination category: 'public' (admin only) or 'custom'")
|
||||
|
||||
|
||||
class SkillInstallPreviewResponse(BaseModel):
|
||||
"""Response model for skill install preflight."""
|
||||
|
||||
ok: bool = Field(..., description="Whether the archive could be parsed")
|
||||
skill_name: str = Field(..., description="Skill name parsed from the archive frontmatter")
|
||||
description: str = Field(default="", description="Parsed description from the archive frontmatter")
|
||||
license: str | None = Field(default=None, description="Parsed license from the archive frontmatter")
|
||||
allowed_tools: list[str] = Field(default_factory=list, description="Allowed tools declared in the archive")
|
||||
can_install_as_is: bool = Field(default=True, description="Whether the current user may install the archive without renaming")
|
||||
owned_by_me: bool = Field(default=False, description="True when the existing conflicting skill is already owned by the caller")
|
||||
owned_by_other: bool = Field(default=False, description="True when another user already owns the conflicting custom skill")
|
||||
builtin_conflict: bool = Field(default=False, description="True when the parsed name collides with a built-in/public skill")
|
||||
owner_username: str | None = Field(default=None, description="Friendly owner username for owned_by_other conflicts")
|
||||
owner_user_id: str | None = Field(default=None, description="Owner user id for custom conflicts")
|
||||
message: str = Field(default="", description="Human-readable install guidance for the UI")
|
||||
warnings: list[str] = Field(default_factory=list, description="Non-fatal warnings from validation")
|
||||
|
||||
|
||||
class SkillUploadValidateResponse(BaseModel):
|
||||
"""Response model for the pre-install validation endpoint."""
|
||||
|
||||
@@ -1040,6 +1069,162 @@ async def install_skill_upload(
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
async def _preview_install_target(
|
||||
*,
|
||||
request: Request,
|
||||
skill_store: SkillStore,
|
||||
skill_name: str,
|
||||
target_category: SkillCategory,
|
||||
) -> tuple[bool, bool, bool, bool, str | None, str | None, str]:
|
||||
"""Return installability / ownership info for a parsed skill name."""
|
||||
|
||||
user_id = _current_user_id(request)
|
||||
is_admin = _is_admin_user(request)
|
||||
existing = await skill_store.get_any(skill_name)
|
||||
owner_user_id = existing.get("owner_user_id") if existing else None
|
||||
owner_username: str | None = None
|
||||
if owner_user_id:
|
||||
try:
|
||||
attached = await _attach_owner_usernames(
|
||||
[SkillResponse(name=skill_name, description="", license=None, category=SkillCategory.CUSTOM, owner_user_id=owner_user_id)]
|
||||
)
|
||||
owner_username = attached[0].owner_username if attached else None
|
||||
except Exception:
|
||||
owner_username = None
|
||||
|
||||
builtin_conflict = bool(existing and owner_user_id is None)
|
||||
owned_by_me = bool(existing and owner_user_id == user_id)
|
||||
owned_by_other = bool(existing and owner_user_id not in (None, user_id))
|
||||
|
||||
if target_category == SkillCategory.PUBLIC and not is_admin:
|
||||
return False, owned_by_me, owned_by_other, builtin_conflict, owner_user_id, owner_username, "Only admin users can install skills to the public directory"
|
||||
if builtin_conflict:
|
||||
return False, owned_by_me, owned_by_other, True, owner_user_id, owner_username, f"Skill '{skill_name}' already exists as a built-in/public skill. Please install it under a different name."
|
||||
if owned_by_other:
|
||||
who = owner_username or "another user"
|
||||
return False, owned_by_me, True, builtin_conflict, owner_user_id, owner_username, f"Skill '{skill_name}' has already been installed by {who}. Please choose a different name."
|
||||
if owned_by_me:
|
||||
return False, True, False, builtin_conflict, owner_user_id, owner_username, f"You have already installed skill '{skill_name}'. You can keep using it or install this copy under a different name."
|
||||
return True, False, False, builtin_conflict, owner_user_id, owner_username, f"Skill '{skill_name}' can be installed."
|
||||
|
||||
|
||||
async def _build_renamed_skill_archive(
|
||||
archive_path: str | Path,
|
||||
*,
|
||||
new_name: str,
|
||||
require_skill_suffix: bool,
|
||||
) -> Path:
|
||||
"""Create a temporary archive whose SKILL.md frontmatter name is rewritten."""
|
||||
|
||||
from deerflow.skills.installer import resolve_skill_dir_from_archive, safe_extract_skill_archive
|
||||
|
||||
SkillStorage.validate_skill_name(new_name)
|
||||
tmp_root = Path(tempfile.mkdtemp(prefix="skill-install-rename-"))
|
||||
try:
|
||||
path = Path(archive_path)
|
||||
if require_skill_suffix and path.suffix != ".skill":
|
||||
raise ValueError("File must have .skill extension")
|
||||
with zipfile.ZipFile(path, "r") as zf:
|
||||
safe_extract_skill_archive(zf, tmp_root)
|
||||
skill_dir = resolve_skill_dir_from_archive(tmp_root)
|
||||
skill_md = skill_dir / SKILL_MD_FILE
|
||||
skill_md.write_text(
|
||||
_rewrite_skill_frontmatter_name(skill_md.read_text(encoding="utf-8"), new_name),
|
||||
encoding="utf-8",
|
||||
)
|
||||
renamed_archive = tmp_root / f"{new_name}.skill"
|
||||
with zipfile.ZipFile(renamed_archive, "w", compression=zipfile.ZIP_DEFLATED) as zf:
|
||||
for path in sorted(tmp_root.rglob("*")):
|
||||
if not path.is_file() or path == renamed_archive:
|
||||
continue
|
||||
zf.write(path, path.relative_to(tmp_root))
|
||||
return renamed_archive
|
||||
except Exception:
|
||||
shutil.rmtree(tmp_root, ignore_errors=True)
|
||||
raise
|
||||
|
||||
|
||||
async def _install_archive_with_optional_name(
|
||||
*,
|
||||
storage: SkillStorage,
|
||||
archive_path: str | Path,
|
||||
requested_name: str | None,
|
||||
target_category: SkillCategory,
|
||||
overwrite: bool = False,
|
||||
require_skill_suffix: bool = True,
|
||||
) -> SkillInstallResponse:
|
||||
normalized_requested = SkillStorage.validate_skill_name(requested_name) if requested_name else None
|
||||
temp_root: Path | None = None
|
||||
install_source = Path(archive_path)
|
||||
if normalized_requested:
|
||||
renamed_archive = await _build_renamed_skill_archive(
|
||||
archive_path,
|
||||
new_name=normalized_requested,
|
||||
require_skill_suffix=require_skill_suffix,
|
||||
)
|
||||
temp_root = renamed_archive.parent
|
||||
install_source = renamed_archive
|
||||
try:
|
||||
result = await storage.ainstall_skill_from_archive(
|
||||
install_source,
|
||||
overwrite=overwrite,
|
||||
require_skill_suffix=require_skill_suffix,
|
||||
target_category=target_category,
|
||||
)
|
||||
return SkillInstallResponse(**result)
|
||||
finally:
|
||||
if temp_root is not None:
|
||||
shutil.rmtree(temp_root, ignore_errors=True)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/skills/install-preview",
|
||||
response_model=SkillInstallPreviewResponse,
|
||||
summary="Preview Skill Install",
|
||||
description="Parse a .skill artifact from a thread and return ownership/conflict info before install.",
|
||||
)
|
||||
async def preview_skill_install(
|
||||
body: SkillInstallPreviewRequest,
|
||||
request: Request,
|
||||
config: AppConfig = Depends(get_config),
|
||||
skill_store: SkillStore = Depends(get_skill_store),
|
||||
) -> SkillInstallPreviewResponse:
|
||||
try:
|
||||
skill_file_path = await aresolve_thread_virtual_path(body.thread_id, body.path)
|
||||
validation = get_or_new_skill_storage(app_config=config).validate_skill_archive(skill_file_path, require_skill_suffix=False)
|
||||
skill_name = str(validation.get("skill_name") or "")
|
||||
can_install_as_is, owned_by_me, owned_by_other, builtin_conflict, owner_user_id, owner_username, message = await _preview_install_target(
|
||||
request=request,
|
||||
skill_store=skill_store,
|
||||
skill_name=skill_name,
|
||||
target_category=body.target_category,
|
||||
)
|
||||
return SkillInstallPreviewResponse(
|
||||
ok=True,
|
||||
skill_name=skill_name,
|
||||
description=str(validation.get("description") or ""),
|
||||
license=validation.get("license"),
|
||||
allowed_tools=[str(item) for item in (validation.get("allowed_tools") or [])],
|
||||
can_install_as_is=can_install_as_is,
|
||||
owned_by_me=owned_by_me,
|
||||
owned_by_other=owned_by_other,
|
||||
builtin_conflict=builtin_conflict,
|
||||
owner_username=owner_username,
|
||||
owner_user_id=owner_user_id,
|
||||
message=message,
|
||||
warnings=[str(item) for item in (validation.get("warnings") or [])],
|
||||
)
|
||||
except FileNotFoundError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error("Failed to preview skill install: %s", e, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Failed to preview skill install: {e}")
|
||||
|
||||
|
||||
@router.post(
|
||||
"/skills/install",
|
||||
response_model=SkillInstallResponse,
|
||||
@@ -1056,25 +1241,46 @@ async def install_skill(
|
||||
user_id = _current_user_id(request)
|
||||
is_admin = _is_admin_user(request)
|
||||
target_category = body.target_category
|
||||
requested_name = SkillStorage.validate_skill_name(body.skill_name) if body.skill_name else None
|
||||
if target_category == SkillCategory.PUBLIC and not is_admin:
|
||||
raise HTTPException(status_code=403, detail="Only admin users can install skills to the public directory")
|
||||
skill_file_path = await aresolve_thread_virtual_path(body.thread_id, body.path)
|
||||
result = await get_or_new_skill_storage(app_config=config).ainstall_skill_from_archive(skill_file_path, target_category=target_category)
|
||||
storage = get_or_new_skill_storage(app_config=config)
|
||||
validation = storage.validate_skill_archive(skill_file_path, require_skill_suffix=False)
|
||||
preview_name = requested_name or str(validation.get("skill_name") or "")
|
||||
can_install_as_is, owned_by_me, owned_by_other, builtin_conflict, _owner_user_id, _owner_username, message = await _preview_install_target(
|
||||
request=request,
|
||||
skill_store=skill_store,
|
||||
skill_name=preview_name,
|
||||
target_category=target_category,
|
||||
)
|
||||
if not can_install_as_is:
|
||||
if requested_name and requested_name != validation.get("skill_name") and not owned_by_other and not builtin_conflict:
|
||||
pass
|
||||
else:
|
||||
raise HTTPException(status_code=409, detail=message)
|
||||
result = await _install_archive_with_optional_name(
|
||||
storage=storage,
|
||||
archive_path=skill_file_path,
|
||||
requested_name=requested_name,
|
||||
target_category=target_category,
|
||||
require_skill_suffix=False,
|
||||
)
|
||||
try:
|
||||
from deerflow.skills.usage import mark_uploaded
|
||||
mark_uploaded(result["skill_name"])
|
||||
mark_uploaded(result.skill_name)
|
||||
except Exception:
|
||||
pass
|
||||
if target_category == SkillCategory.PUBLIC:
|
||||
await skill_store.ensure_legacy(
|
||||
{"name": result["skill_name"], "owner_user_id": None, "published": True}
|
||||
{"name": result.skill_name, "owner_user_id": None, "published": True}
|
||||
)
|
||||
else:
|
||||
await skill_store.ensure_legacy(
|
||||
{"name": result["skill_name"], "owner_user_id": user_id, "published": False}
|
||||
{"name": result.skill_name, "owner_user_id": user_id, "published": False}
|
||||
)
|
||||
await refresh_skills_system_prompt_cache_async()
|
||||
return SkillInstallResponse(**result)
|
||||
return result
|
||||
except FileNotFoundError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
except SkillAlreadyExistsError as e:
|
||||
|
||||
@@ -4,6 +4,8 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Literal
|
||||
from urllib.parse import urlparse
|
||||
from uuid import uuid4
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel, Field
|
||||
@@ -12,7 +14,12 @@ from app.gateway.auth.password import hash_password_async
|
||||
from app.gateway.deps import get_optional_user_from_request
|
||||
from deerflow.config import get_app_config
|
||||
from deerflow.config.system_settings import (
|
||||
AppearanceRouteParam,
|
||||
CompactShortcutButton,
|
||||
LeaderboardSettings,
|
||||
ModelConcurrencySettings,
|
||||
PluginDomainAgentMapping,
|
||||
PluginManagementSettings,
|
||||
PromptPrefixSettings,
|
||||
PublishSquare,
|
||||
PublishSquaresSettings,
|
||||
@@ -42,6 +49,21 @@ class PromptPrefixUpdateRequest(BaseModel):
|
||||
prompt_prefix: str | None = None
|
||||
|
||||
|
||||
class PluginDomainAgentMappingItem(BaseModel):
|
||||
id: str = Field(..., description="Stable row id")
|
||||
domain: str = Field(..., description="Normalized host matched against pageContext.url")
|
||||
agent_id: str = Field(..., description="Agent id selected for the host")
|
||||
enabled: bool = Field(default=True, description="Whether the mapping is active")
|
||||
|
||||
|
||||
class PluginManagementResponse(BaseModel):
|
||||
domain_agent_mappings: list[PluginDomainAgentMappingItem] = Field(default_factory=list)
|
||||
|
||||
|
||||
class PluginManagementUpdateRequest(BaseModel):
|
||||
domain_agent_mappings: list[PluginDomainAgentMappingItem] | None = None
|
||||
|
||||
|
||||
class PrivateSquareStatusResponse(BaseModel):
|
||||
"""Status of the private-square feature, readable by any logged-in user.
|
||||
|
||||
@@ -90,6 +112,34 @@ def _to_response(settings: PromptPrefixSettings) -> PromptPrefixResponse:
|
||||
)
|
||||
|
||||
|
||||
def _normalize_plugin_domain(value: str | None) -> str:
|
||||
raw = (value or "").strip()
|
||||
if not raw:
|
||||
return ""
|
||||
candidate = raw if "://" in raw else f"https://{raw}"
|
||||
try:
|
||||
parsed = urlparse(candidate)
|
||||
except Exception:
|
||||
return ""
|
||||
return (parsed.netloc or parsed.path or "").strip().lower()
|
||||
|
||||
|
||||
def _plugin_management_to_response(
|
||||
settings: PluginManagementSettings,
|
||||
) -> PluginManagementResponse:
|
||||
return PluginManagementResponse(
|
||||
domain_agent_mappings=[
|
||||
PluginDomainAgentMappingItem(
|
||||
id=item.id,
|
||||
domain=item.domain,
|
||||
agent_id=item.agent_id,
|
||||
enabled=item.enabled,
|
||||
)
|
||||
for item in settings.domain_agent_mappings
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
async def _require_admin(request: Request) -> None:
|
||||
user = await get_optional_user_from_request(request)
|
||||
if user is None:
|
||||
@@ -116,6 +166,50 @@ async def update_prompt_prefix(body: PromptPrefixUpdateRequest, request: Request
|
||||
return _to_response(saved.prompt_prefix)
|
||||
|
||||
|
||||
@router.get("/plugin-management", response_model=PluginManagementResponse)
|
||||
async def get_plugin_management() -> PluginManagementResponse:
|
||||
"""Return the iframe plugin domain -> agent mappings."""
|
||||
return _plugin_management_to_response(load_system_settings().plugin_management)
|
||||
|
||||
|
||||
@router.put("/plugin-management", response_model=PluginManagementResponse)
|
||||
async def update_plugin_management(
|
||||
body: PluginManagementUpdateRequest, request: Request
|
||||
) -> PluginManagementResponse:
|
||||
"""Replace the iframe plugin domain -> agent mappings. Admin only."""
|
||||
from uuid import uuid4
|
||||
|
||||
await _require_admin(request)
|
||||
|
||||
current = load_system_settings()
|
||||
mappings: list[PluginDomainAgentMapping] = []
|
||||
seen_domains: set[str] = set()
|
||||
for item in body.domain_agent_mappings or []:
|
||||
domain = _normalize_plugin_domain(item.domain)
|
||||
agent_id = (item.agent_id or "").strip()
|
||||
if not domain or not agent_id:
|
||||
continue
|
||||
if domain in seen_domains:
|
||||
continue
|
||||
seen_domains.add(domain)
|
||||
mappings.append(
|
||||
PluginDomainAgentMapping(
|
||||
id=(item.id or "").strip() or uuid4().hex,
|
||||
domain=domain,
|
||||
agent_id=agent_id,
|
||||
enabled=bool(item.enabled),
|
||||
)
|
||||
)
|
||||
|
||||
updated = current.plugin_management.model_copy(
|
||||
update={"domain_agent_mappings": mappings}
|
||||
)
|
||||
saved = save_system_settings(
|
||||
current.model_copy(update={"plugin_management": updated})
|
||||
)
|
||||
return _plugin_management_to_response(saved.plugin_management)
|
||||
|
||||
|
||||
class SkillCompressionResponse(BaseModel):
|
||||
enabled: bool = Field(..., description="Whether skill compression is active")
|
||||
search_top_n: int = Field(..., description="Max skills returned by the search_skills tool")
|
||||
@@ -128,6 +222,18 @@ class SkillCompressionUpdateRequest(BaseModel):
|
||||
keep_index_in_prompt: bool | None = None
|
||||
|
||||
|
||||
class ModelConcurrencyResponse(BaseModel):
|
||||
enabled: bool = Field(..., description="Whether per-model concurrency limiting is active")
|
||||
default_limit: int = Field(..., description="Fallback max concurrent runs for models without an explicit override")
|
||||
model_limits: dict[str, int] = Field(default_factory=dict, description="Per-model overrides: model_name -> max concurrent runs")
|
||||
|
||||
|
||||
class ModelConcurrencyUpdateRequest(BaseModel):
|
||||
enabled: bool | None = None
|
||||
default_limit: int | None = Field(default=None, ge=1, le=10_000)
|
||||
model_limits: dict[str, int] | None = None
|
||||
|
||||
|
||||
def _skill_compression_to_response(settings: SkillCompressionSettings) -> SkillCompressionResponse:
|
||||
return SkillCompressionResponse(
|
||||
enabled=settings.enabled,
|
||||
@@ -136,6 +242,26 @@ def _skill_compression_to_response(settings: SkillCompressionSettings) -> SkillC
|
||||
)
|
||||
|
||||
|
||||
def _model_concurrency_to_response(settings: ModelConcurrencySettings) -> ModelConcurrencyResponse:
|
||||
cleaned: dict[str, int] = {}
|
||||
for name, limit in (settings.model_limits or {}).items():
|
||||
model_name = str(name).strip()
|
||||
if not model_name:
|
||||
continue
|
||||
try:
|
||||
normalized_limit = int(limit)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if normalized_limit < 1:
|
||||
continue
|
||||
cleaned[model_name] = min(normalized_limit, 10_000)
|
||||
return ModelConcurrencyResponse(
|
||||
enabled=settings.enabled,
|
||||
default_limit=max(1, min(int(settings.default_limit), 10_000)),
|
||||
model_limits=cleaned,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/skill-compression", response_model=SkillCompressionResponse)
|
||||
async def get_skill_compression() -> SkillCompressionResponse:
|
||||
"""Return the skill-compression configuration (readable by any user).
|
||||
@@ -174,6 +300,47 @@ async def update_skill_compression(body: SkillCompressionUpdateRequest, request:
|
||||
return _skill_compression_to_response(saved.skill_compression)
|
||||
|
||||
|
||||
@router.get("/model-concurrency", response_model=ModelConcurrencyResponse)
|
||||
async def get_model_concurrency(request: Request) -> ModelConcurrencyResponse:
|
||||
"""Return the per-model concurrency-limit configuration. Admin only."""
|
||||
await _require_admin(request)
|
||||
return _model_concurrency_to_response(load_system_settings().model_concurrency)
|
||||
|
||||
|
||||
@router.put("/model-concurrency", response_model=ModelConcurrencyResponse)
|
||||
async def update_model_concurrency(
|
||||
body: ModelConcurrencyUpdateRequest,
|
||||
request: Request,
|
||||
) -> ModelConcurrencyResponse:
|
||||
"""Update the per-model concurrency-limit configuration. Admin only."""
|
||||
await _require_admin(request)
|
||||
|
||||
current = load_system_settings()
|
||||
fields = body.model_dump(exclude_unset=True)
|
||||
update: dict[str, object] = {}
|
||||
if "enabled" in fields and fields["enabled"] is not None:
|
||||
update["enabled"] = bool(fields["enabled"])
|
||||
if "default_limit" in fields and fields["default_limit"] is not None:
|
||||
update["default_limit"] = max(1, min(int(fields["default_limit"]), 10_000))
|
||||
if "model_limits" in fields and fields["model_limits"] is not None:
|
||||
cleaned: dict[str, int] = {}
|
||||
for name, limit in (body.model_limits or {}).items():
|
||||
model_name = str(name).strip()
|
||||
if not model_name:
|
||||
continue
|
||||
try:
|
||||
normalized_limit = int(limit)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if normalized_limit < 1:
|
||||
continue
|
||||
cleaned[model_name] = min(normalized_limit, 10_000)
|
||||
update["model_limits"] = cleaned
|
||||
settings = current.model_concurrency.model_copy(update=update)
|
||||
saved = save_system_settings(current.model_copy(update={"model_concurrency": settings}))
|
||||
return _model_concurrency_to_response(saved.model_concurrency)
|
||||
|
||||
|
||||
def _leaderboard_to_response(settings: LeaderboardSettings) -> LeaderboardSettingsResponse:
|
||||
return LeaderboardSettingsResponse(
|
||||
exclude_user_ids=list(dict.fromkeys([uid for uid in settings.exclude_user_ids if uid])),
|
||||
@@ -373,6 +540,7 @@ class AppearanceResponse(BaseModel):
|
||||
default_skill_view: Literal["card", "list"] = Field(..., description="System default skill list display mode — card or list")
|
||||
default_agent_view: Literal["card", "list"] = Field(..., description="System default agent list display mode — card or list")
|
||||
default_layout_mode: Literal["full", "compact"] = Field(..., description="System default interface layout — full (sidebar + header) or compact (hidden)")
|
||||
compact_shortcut_buttons: list[CompactShortcutButton] = Field(default_factory=list, description="Shortcut buttons rendered in top bars")
|
||||
|
||||
|
||||
class AppearanceUpdateRequest(BaseModel):
|
||||
@@ -380,6 +548,61 @@ class AppearanceUpdateRequest(BaseModel):
|
||||
default_skill_view: Literal["card", "list"] | None = Field(default=None)
|
||||
default_agent_view: Literal["card", "list"] | None = Field(default=None)
|
||||
default_layout_mode: Literal["full", "compact"] | None = Field(default=None)
|
||||
compact_shortcut_buttons: list[CompactShortcutButton] | None = Field(default=None)
|
||||
|
||||
|
||||
def _normalize_compact_shortcut_buttons(
|
||||
buttons: list[CompactShortcutButton] | None,
|
||||
) -> list[CompactShortcutButton]:
|
||||
normalized: list[CompactShortcutButton] = []
|
||||
seen_ids: set[str] = set()
|
||||
for index, item in enumerate(buttons or []):
|
||||
label = (item.label or "").strip() or "快捷入口"
|
||||
url = (item.url or "").strip()
|
||||
row_id = (item.id or "").strip() or uuid4().hex
|
||||
if row_id in seen_ids:
|
||||
row_id = uuid4().hex
|
||||
seen_ids.add(row_id)
|
||||
|
||||
route_params: list[AppearanceRouteParam] = []
|
||||
for param in item.route_params or []:
|
||||
key = (param.key or "").strip()
|
||||
if not key:
|
||||
continue
|
||||
if param.paramType == "theme":
|
||||
route_params.append(
|
||||
AppearanceRouteParam(
|
||||
key=key,
|
||||
paramType="theme",
|
||||
themeValue=(param.themeValue or "").strip()
|
||||
or "red=red;blue=blue;dark-blue=dark-blue",
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
source = (param.source or "access_token").strip() or "access_token"
|
||||
route_params.append(
|
||||
AppearanceRouteParam(
|
||||
key=key,
|
||||
paramType="login",
|
||||
source=source,
|
||||
customValue=(param.customValue or "").strip()
|
||||
if source == "other"
|
||||
else None,
|
||||
)
|
||||
)
|
||||
|
||||
normalized.append(
|
||||
CompactShortcutButton(
|
||||
id=row_id,
|
||||
label=label,
|
||||
url=url,
|
||||
route_params=route_params,
|
||||
enabled=bool(item.enabled),
|
||||
sort_order=index,
|
||||
)
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def _appearance_response(appearance) -> AppearanceResponse:
|
||||
@@ -388,6 +611,10 @@ def _appearance_response(appearance) -> AppearanceResponse:
|
||||
default_skill_view=appearance.default_skill_view,
|
||||
default_agent_view=appearance.default_agent_view,
|
||||
default_layout_mode=appearance.default_layout_mode,
|
||||
compact_shortcut_buttons=sorted(
|
||||
appearance.compact_shortcut_buttons,
|
||||
key=lambda item: item.sort_order,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -410,6 +637,10 @@ async def update_appearance_settings(
|
||||
for key in ("default_theme", "default_skill_view", "default_agent_view", "default_layout_mode"):
|
||||
if key in fields and fields[key] is not None:
|
||||
update[key] = fields[key]
|
||||
if "compact_shortcut_buttons" in fields and body.compact_shortcut_buttons is not None:
|
||||
update["compact_shortcut_buttons"] = _normalize_compact_shortcut_buttons(
|
||||
body.compact_shortcut_buttons
|
||||
)
|
||||
if update:
|
||||
updated_appearance = current.appearance.model_copy(update=update)
|
||||
saved = save_system_settings(
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+105
-12
@@ -127,6 +127,31 @@ def _fallback_knowledge_skill() -> SimpleNamespace:
|
||||
return SimpleNamespace(name=KNOWLEDGE_SEARCH_SKILL_NAME, description=_KNOWLEDGE_SKILL_FALLBACK_DESC)
|
||||
|
||||
|
||||
def _unique_limited(values: list, limit: int) -> list:
|
||||
"""Stable de-dup for short diagnostic lists."""
|
||||
return list(dict.fromkeys(values or []))[:limit]
|
||||
|
||||
|
||||
def _intranet_query_summary(rows: list[dict], limit: int = 5) -> str:
|
||||
"""Compact per-query stats for the visible skill step message."""
|
||||
parts: list[str] = []
|
||||
for row in rows[:limit]:
|
||||
query = str(row.get("query") or "")[:24] or "(空)"
|
||||
elapsed_ms = row.get("elapsed_ms")
|
||||
try:
|
||||
elapsed = f"{float(elapsed_ms) / 1000:.1f}s" if elapsed_ms is not None else ""
|
||||
except (TypeError, ValueError):
|
||||
elapsed = ""
|
||||
if row.get("error"):
|
||||
reason = str(row.get("error"))[:80]
|
||||
parts.append(f"{query}: 失败{f'/{elapsed}' if elapsed else ''}({reason})")
|
||||
else:
|
||||
parts.append(f"{query}: {int(row.get('count') or 0)} 条{f'/{elapsed}' if elapsed else ''}")
|
||||
if len(rows) > limit:
|
||||
parts.append(f"另 {len(rows) - limit} 个方向")
|
||||
return ";".join(parts)
|
||||
|
||||
|
||||
def _load_configured_skills() -> list:
|
||||
"""解析素材收集专家 agent(ai-writing-researcher)配置的技能列表。
|
||||
|
||||
@@ -370,7 +395,10 @@ async def _search_notebook(
|
||||
results: list[Material] = []
|
||||
mat_idx = 0
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=20) as client:
|
||||
# trust_env=False:笔记后端是内网/本机地址,但模型走公网,部署常设
|
||||
# HTTP_PROXY/HTTPS_PROXY;httpx 默认会把这个内网请求也走代理 → 不可达。
|
||||
# 强制忽略代理 env、直连(与内网 ES 检索同款修法)。
|
||||
async with httpx.AsyncClient(timeout=20, trust_env=False) as client:
|
||||
for kw in keywords[:_NOTEBOOK_MAX_KEYWORDS]:
|
||||
logger.info("[notebook-search] POST kw=%r body=%s", kw, {
|
||||
"query": kw,
|
||||
@@ -657,8 +685,24 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
# 内网慢检索超时可在 config.yaml 的 ai_writing.intranet_search_timeout 调大,
|
||||
# 默认 600s,且最低 300s。
|
||||
timeout=_intranet_timeout_seconds(ai_writing_config),
|
||||
verify_ssl=ai_writing_config.get("intranet_verify_ssl", False),
|
||||
)
|
||||
endpoint_label = intranet_url or getattr(provider, "endpoint", "内置默认内网端点")
|
||||
intranet_debug = {
|
||||
"mode": "knowledge_base",
|
||||
"configured": bool(intranet_url),
|
||||
"endpoint": endpoint_label,
|
||||
"knowledge_base": ai_writing_config.get("intranet_knowledge_base", "samples"),
|
||||
"timeout": _intranet_timeout_seconds(ai_writing_config),
|
||||
"verify_ssl": getattr(provider, "verify_ssl", False),
|
||||
"status": "running",
|
||||
"queries": [],
|
||||
"calls": [],
|
||||
"errors": [],
|
||||
"raw_responses": [],
|
||||
"result_count": 0,
|
||||
"used_as_fallback": False,
|
||||
}
|
||||
# 用户输入框追加(is_user_query_search)时 keywords 已是 [用户原话],全部检索;
|
||||
# 首次检索按 keyword_count 截断检索词。
|
||||
search_keywords = keywords if is_user_query_search else keywords[:keyword_count]
|
||||
@@ -670,20 +714,65 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
"message": f"使用「{_KNOWLEDGE_SKILL_DISPLAY}」直连内网接口({endpoint_label})检索 {total} 个方向:{('、'.join(search_keywords[:3]))[:60]}",
|
||||
"skill_name": KNOWLEDGE_SEARCH_SKILL_NAME,
|
||||
})
|
||||
|
||||
async def _search_one_keyword(kw: str) -> tuple[str, list[SearchResult], str, int]:
|
||||
started = time.monotonic()
|
||||
try:
|
||||
results = await provider.search(kw, max_results=per_keyword)
|
||||
return kw, list(results or []), "", int((time.monotonic() - started) * 1000)
|
||||
except Exception as e: # noqa: BLE001 — provider 理论上会吞异常,这里兜住意外冒泡
|
||||
logger.warning("[knowledge-base] keyword=%r search failed: %s", kw, e)
|
||||
return kw, [], f"{type(e).__name__}: {e}", int((time.monotonic() - started) * 1000)
|
||||
|
||||
gathered = await asyncio.gather(
|
||||
*[provider.search(kw, max_results=per_keyword) for kw in search_keywords],
|
||||
*[_search_one_keyword(kw) for kw in search_keywords],
|
||||
return_exceptions=True,
|
||||
)
|
||||
for results in gathered:
|
||||
if isinstance(results, Exception):
|
||||
logger.warning("[knowledge-base] search failed: %s", results)
|
||||
search_errors.append(f"{type(results).__name__}: {results}")
|
||||
for item in gathered:
|
||||
if isinstance(item, Exception):
|
||||
logger.warning("[knowledge-base] search task failed: %s", item)
|
||||
message = f"{type(item).__name__}: {item}"
|
||||
search_errors.append(message)
|
||||
intranet_debug["queries"].append({"query": "(unknown)", "count": 0, "error": message})
|
||||
continue
|
||||
kw, results, error, elapsed_ms = item
|
||||
row = {"query": kw, "count": len(results or []), "elapsed_ms": elapsed_ms}
|
||||
if error:
|
||||
row["error"] = error
|
||||
search_errors.append(error)
|
||||
intranet_debug["queries"].append(row)
|
||||
new_results.extend(results or [])
|
||||
search_errors.extend(getattr(provider, "errors", []))
|
||||
provider_errors = _unique_limited(getattr(provider, "errors", []) or [], 5)
|
||||
search_errors.extend(provider_errors)
|
||||
intranet_debug["errors"] = provider_errors
|
||||
intranet_debug["raw_responses"] = _unique_limited(getattr(provider, "raw_responses", []) or [], 3)
|
||||
intranet_debug["calls"] = list(getattr(provider, "calls", []) or [])[:20]
|
||||
intranet_debug["result_count"] = len(new_results)
|
||||
if search_errors and new_results:
|
||||
intranet_debug["status"] = "partial"
|
||||
elif search_errors:
|
||||
intranet_debug["status"] = "error"
|
||||
elif new_results:
|
||||
intranet_debug["status"] = "ok"
|
||||
else:
|
||||
intranet_debug["status"] = "empty"
|
||||
|
||||
query_summary = _intranet_query_summary(intranet_debug["queries"])
|
||||
if search_errors:
|
||||
uniq_errors = _unique_limited(search_errors, 3)
|
||||
_emit_skill_step({
|
||||
"type": "skill_error",
|
||||
"message": f"「{_KNOWLEDGE_SKILL_DISPLAY}」内网接口调用异常:{';'.join(uniq_errors)}",
|
||||
"skill_name": KNOWLEDGE_SEARCH_SKILL_NAME,
|
||||
"reason": ";".join(uniq_errors),
|
||||
})
|
||||
result_message = (
|
||||
f"「{_KNOWLEDGE_SKILL_DISPLAY}」(内网接口)调用完成,检索到 {len(new_results)} 条素材"
|
||||
+ (f"({query_summary})" if query_summary else "")
|
||||
)
|
||||
_emit_skill_step({
|
||||
"type": "skill_result",
|
||||
"message": f"「{_KNOWLEDGE_SKILL_DISPLAY}」(内网接口)检索到 {len(new_results)} 条素材",
|
||||
"message": result_message,
|
||||
"skill_name": KNOWLEDGE_SEARCH_SKILL_NAME,
|
||||
"matched_count": len(new_results),
|
||||
})
|
||||
@@ -758,7 +847,7 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
timeout=_SKILL_SEARCH_TIMEOUT_SECONDS,
|
||||
)
|
||||
return (results or []), ""
|
||||
except asyncio.TimeoutError:
|
||||
except TimeoutError:
|
||||
msg = f"检索超时(>{_SKILL_SEARCH_TIMEOUT_SECONDS}s)"
|
||||
logger.warning("skill[%s] search timed out after %ss", skill_name, _SKILL_SEARCH_TIMEOUT_SECONDS)
|
||||
search_errors.append(f"skill[{skill_name}]: {msg}")
|
||||
@@ -901,16 +990,18 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
intranet_url = (ai_writing_config.get("intranet_search_url") or "").strip()
|
||||
# 诊断开关:开启后即使技能成功也照样跑完内网并记录完整结果(用于验证内网接口是否真能通)。
|
||||
intranet_debug_force = bool(ai_writing_config.get("intranet_search_debug", False))
|
||||
intranet_task: "asyncio.Task | None" = None
|
||||
intranet_task: asyncio.Task | None = None
|
||||
intranet_holder: dict = {}
|
||||
intranet_debug = {
|
||||
"configured": bool(intranet_url),
|
||||
"endpoint": intranet_url or "(未配置 ai_writing.intranet_search_url)",
|
||||
"knowledge_base": ai_writing_config.get("intranet_knowledge_base", "samples"),
|
||||
"timeout": _intranet_timeout_seconds(ai_writing_config),
|
||||
"verify_ssl": bool(ai_writing_config.get("intranet_verify_ssl", False)),
|
||||
"debug_force": intranet_debug_force,
|
||||
"status": "pending" if intranet_url else "not_configured",
|
||||
"queries": [], # 每个检索词:{query, count, error?}
|
||||
"calls": [], # provider 每次真实 HTTP 调用,含简化词重试
|
||||
"errors": [], # provider 聚合的报错
|
||||
"raw_responses": [], # 解析失败时的原始响应片段
|
||||
"result_count": 0, # 内网总命中条数
|
||||
@@ -929,6 +1020,7 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
# 兜底接口超时可在 config.yaml 的 ai_writing.intranet_search_timeout 调大,
|
||||
# 默认 600s(10 分钟),且最低 300s。
|
||||
timeout=_intranet_timeout_seconds(ai_writing_config),
|
||||
verify_ssl=ai_writing_config.get("intranet_verify_ssl", False),
|
||||
)
|
||||
intranet_holder["provider"] = intranet
|
||||
fb_keywords = keywords[:keyword_count] or [intent]
|
||||
@@ -945,8 +1037,9 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
|
||||
continue
|
||||
intranet_debug["queries"].append({"query": kw, "count": len(res or [])})
|
||||
collected.extend(res or [])
|
||||
intranet_debug["errors"] = list(dict.fromkeys(getattr(intranet, "errors", [])))[:5]
|
||||
intranet_debug["raw_responses"] = list(dict.fromkeys(getattr(intranet, "raw_responses", [])))[:3]
|
||||
intranet_debug["errors"] = _unique_limited(getattr(intranet, "errors", []) or [], 5)
|
||||
intranet_debug["raw_responses"] = _unique_limited(getattr(intranet, "raw_responses", []) or [], 3)
|
||||
intranet_debug["calls"] = list(getattr(intranet, "calls", []) or [])[:20]
|
||||
intranet_debug["result_count"] = len(collected)
|
||||
intranet_debug["status"] = "ran"
|
||||
return collected
|
||||
|
||||
+4
-2
@@ -1,8 +1,8 @@
|
||||
from .base import BaseSearchProvider, SearchResult
|
||||
from .internet_search import InternetSearchProvider
|
||||
from .intranet_stub import IntranetSearchProvider, normalize_intranet_timeout
|
||||
from .demo_search import DemoSearchProvider
|
||||
from .duckduckgo_search import DuckDuckGoSearchProvider
|
||||
from .internet_search import InternetSearchProvider
|
||||
from .intranet_stub import IntranetSearchProvider, normalize_intranet_timeout, normalize_intranet_verify_ssl
|
||||
from .skill_search import SkillSearchProvider
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ def get_search_provider(ai_writing_config: dict) -> BaseSearchProvider:
|
||||
intranet_search_url: http://10.106.57.180:7863/knowledge_base/search_docs_es
|
||||
intranet_knowledge_base: samples
|
||||
intranet_search_timeout: 300 # 内网检索超时秒数,最少 300 秒
|
||||
intranet_verify_ssl: false # 内网 HTTPS 自签名证书场景默认关闭证书校验
|
||||
ddg_region: "cn-zh" # DuckDuckGo 区域,默认 cn-zh
|
||||
|
||||
internet 模式:DuckDuckGo 为主搜索(免 API Key),失败时 fallback 到 demo 演示素材。
|
||||
@@ -26,6 +27,7 @@ def get_search_provider(ai_writing_config: dict) -> BaseSearchProvider:
|
||||
base_url=cfg.get("intranet_search_url", ""),
|
||||
knowledge_base_name=cfg.get("intranet_knowledge_base", "samples"),
|
||||
timeout=normalize_intranet_timeout(cfg.get("intranet_search_timeout")),
|
||||
verify_ssl=normalize_intranet_verify_ssl(cfg.get("intranet_verify_ssl")),
|
||||
)
|
||||
if provider == "tavily":
|
||||
return _FallbackSearchProvider(InternetSearchProvider(), DemoSearchProvider())
|
||||
|
||||
+133
-19
@@ -2,7 +2,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import List
|
||||
import time
|
||||
|
||||
from .base import BaseSearchProvider, SearchResult, simplify_query
|
||||
|
||||
@@ -14,6 +14,8 @@ _DEFAULT_KB = "samples"
|
||||
# 内网检索最少等待 5 分钟;该接口经常慢返回,低于这个值会造成素材收集误判为空。
|
||||
_MIN_TIMEOUT_SECONDS = 300
|
||||
_DEFAULT_TIMEOUT_SECONDS = 600
|
||||
# 内网 HTTPS 检索服务常使用自签名/内部 CA 证书,默认跳过证书校验避免 SSL 证书链错误。
|
||||
_DEFAULT_VERIFY_SSL = False
|
||||
# 单条原始响应体最大保留长度,避免把超大响应整段灌进前端时间线
|
||||
_RAW_SNIPPET_LIMIT = 2000
|
||||
|
||||
@@ -27,6 +29,23 @@ def normalize_intranet_timeout(timeout: object, default: int = _DEFAULT_TIMEOUT_
|
||||
return max(value, _MIN_TIMEOUT_SECONDS)
|
||||
|
||||
|
||||
def normalize_intranet_verify_ssl(value: object, default: bool = _DEFAULT_VERIFY_SSL) -> bool:
|
||||
"""返回内网检索是否校验 SSL 证书,兼容 config.yaml 里的 bool / 字符串写法。"""
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if value is None:
|
||||
return default
|
||||
if isinstance(value, (int, float)):
|
||||
return bool(value)
|
||||
if isinstance(value, str):
|
||||
normalized = value.strip().lower()
|
||||
if normalized in {"1", "true", "yes", "on"}:
|
||||
return True
|
||||
if normalized in {"0", "false", "no", "off"}:
|
||||
return False
|
||||
return default
|
||||
|
||||
|
||||
def _pick(d: dict, keys: tuple) -> str:
|
||||
"""从结果项里按候选字段名取第一个非空值(两套接口字段名不同,做并集兼容)。"""
|
||||
for k in keys:
|
||||
@@ -38,6 +57,16 @@ def _pick(d: dict, keys: tuple) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
def _raw_snippet(body: str) -> str:
|
||||
"""Return a bounded raw response snippet for diagnostics."""
|
||||
body = (body or "").strip()
|
||||
if not body:
|
||||
return ""
|
||||
if len(body) > _RAW_SNIPPET_LIMIT:
|
||||
return body[:_RAW_SNIPPET_LIMIT] + "…(响应体过长,已截断)"
|
||||
return body
|
||||
|
||||
|
||||
class IntranetSearchProvider(BaseSearchProvider):
|
||||
"""内网知识库搜索接口适配器。
|
||||
|
||||
@@ -51,27 +80,29 @@ class IntranetSearchProvider(BaseSearchProvider):
|
||||
base_url: str = "",
|
||||
knowledge_base_name: str = _DEFAULT_KB,
|
||||
timeout: int = _DEFAULT_TIMEOUT_SECONDS,
|
||||
verify_ssl: bool = _DEFAULT_VERIFY_SSL,
|
||||
**_ignored,
|
||||
):
|
||||
self.endpoint = base_url or _DEFAULT_ENDPOINT
|
||||
self.knowledge_base_name = knowledge_base_name or _DEFAULT_KB
|
||||
# 内网 ES 接口很慢,兜底检索超时默认 10 分钟,且无论配置如何至少 5 分钟。
|
||||
self.timeout = normalize_intranet_timeout(timeout)
|
||||
self.verify_ssl = normalize_intranet_verify_ssl(verify_ssl)
|
||||
# 收集本次运行的检索异常,供 researcher 节点取出并提示用户(ES 波动等)
|
||||
self.errors: list[str] = []
|
||||
# 解析失败时收集的原始响应体,供前端「错误返回结果」折叠展示
|
||||
self.raw_responses: list[str] = []
|
||||
# 每一次真实 HTTP 调用的诊断明细。search() 可能会因 0 结果触发简化词重试,
|
||||
# 因此一个业务关键词可能对应多条 calls。
|
||||
self.calls: list[dict] = []
|
||||
|
||||
def _record_raw(self, body: str) -> None:
|
||||
"""收集解析失败 / 结构异常时的原始响应体,供前端折叠查看。"""
|
||||
body = (body or "").strip()
|
||||
if not body:
|
||||
return
|
||||
if len(body) > _RAW_SNIPPET_LIMIT:
|
||||
body = body[:_RAW_SNIPPET_LIMIT] + "…(响应体过长,已截断)"
|
||||
self.raw_responses.append(body)
|
||||
snippet = _raw_snippet(body)
|
||||
if snippet:
|
||||
self.raw_responses.append(snippet)
|
||||
|
||||
async def search(self, query: str, max_results: int = 5, **kwargs) -> List[SearchResult]:
|
||||
async def search(self, query: str, max_results: int = 5, **kwargs) -> list[SearchResult]:
|
||||
results = await self._search_once(query, max_results)
|
||||
# 0 结果兜底:复合检索词常检不出(如「特朗普竞选」),换成更核心的检索词重试一次
|
||||
# (「特朗普竞选」→「特朗普」)。仅在首检为空、且能简化出不同的词时才重试。
|
||||
@@ -82,7 +113,7 @@ class IntranetSearchProvider(BaseSearchProvider):
|
||||
results = await self._search_once(simpler, max_results)
|
||||
return results
|
||||
|
||||
async def _search_once(self, query: str, max_results: int = 5) -> List[SearchResult]:
|
||||
async def _search_once(self, query: str, max_results: int = 5) -> list[SearchResult]:
|
||||
import httpx
|
||||
|
||||
payload = {
|
||||
@@ -92,44 +123,116 @@ class IntranetSearchProvider(BaseSearchProvider):
|
||||
}
|
||||
# 先拿到原始响应文本,JSON 解析失败时把它原样回传前端便于定位接口返回了什么
|
||||
raw_text = ""
|
||||
started = time.monotonic()
|
||||
call: dict = {
|
||||
"query": query,
|
||||
"endpoint": self.endpoint,
|
||||
"knowledge_base_name": self.knowledge_base_name,
|
||||
"max_results": max_results,
|
||||
"status": "pending",
|
||||
"ok": False,
|
||||
"count": 0,
|
||||
"verify_ssl": self.verify_ssl,
|
||||
}
|
||||
# 排查用:每次真实调用都打一行请求明细(端点 / 知识库 / 检索词 / 超时),
|
||||
# 内网复现时直接看后端日志即可知道"到底请求了哪个地址、用什么参数"。
|
||||
logger.info(
|
||||
"[intranet-search] POST endpoint=%s kb=%s timeout=%ss verify_ssl=%s query=%r",
|
||||
self.endpoint, self.knowledge_base_name, self.timeout, self.verify_ssl, query,
|
||||
)
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
# trust_env=False:**关键**。内网 ES 是内网地址,但本部署的模型走公网,
|
||||
# 服务器通常设了 HTTP_PROXY/HTTPS_PROXY 出公网;httpx 默认 trust_env=True
|
||||
# 会把这个内网请求也套用公网代理 → 代理到不了内网 IP → 连接被重置/空回复,
|
||||
# 表现为「接口异常」。强制忽略代理 env、直连内网(与 multi_agent.py 的回环
|
||||
# 调用同款修法)。
|
||||
# verify=False:内网 HTTPS 常见自签名/内部 CA 证书;这里仅跳过本内网检索端点的证书校验。
|
||||
async with httpx.AsyncClient(timeout=self.timeout, trust_env=False, verify=self.verify_ssl) as client:
|
||||
resp = await client.post(self.endpoint, json=payload)
|
||||
call["http_status"] = getattr(resp, "status_code", None)
|
||||
resp.raise_for_status()
|
||||
raw_text = resp.text
|
||||
data = resp.json()
|
||||
except httpx.HTTPStatusError as e:
|
||||
# 非 2xx:接口有响应但状态码异常,把响应体一并暴露给前端
|
||||
logger.warning("intranet search failed for query=%r: %s", query, e)
|
||||
self.errors.append(f"{type(e).__name__}: {e}")
|
||||
self._record_raw(e.response.text if e.response is not None else "")
|
||||
body_for_log = e.response.text if e.response is not None else ""
|
||||
logger.warning(
|
||||
"[intranet-search] HTTP error endpoint=%s query=%r status=%s body=%s",
|
||||
self.endpoint, query, getattr(e.response, "status_code", None), _raw_snippet(body_for_log),
|
||||
)
|
||||
message = f"{type(e).__name__}: {e}"
|
||||
self.errors.append(message)
|
||||
call.update({
|
||||
"status": "http_error",
|
||||
"error": message,
|
||||
"http_status": getattr(e.response, "status_code", None),
|
||||
})
|
||||
snippet = _raw_snippet(body_for_log)
|
||||
if snippet:
|
||||
call["raw_response"] = snippet
|
||||
self._record_raw(body_for_log)
|
||||
return []
|
||||
except Exception as e:
|
||||
# 连接失败 / 超时 / JSON 解析失败等;JSON 解析失败时 raw_text 已是接口响应体
|
||||
logger.warning("intranet search failed for query=%r: %s", query, e)
|
||||
self.errors.append(f"{type(e).__name__}: {e}")
|
||||
# 连接失败 / 超时 / JSON 解析失败等;JSON 解析失败时 raw_text 已是接口响应体。
|
||||
# 打出异常类型,便于区分 ConnectError / ConnectTimeout / ReadTimeout /
|
||||
# RemoteProtocolError(空回复) / JSONDecodeError(返回非 JSON) 等具体根因。
|
||||
logger.warning(
|
||||
"[intranet-search] request failed endpoint=%s query=%r error=%s: %s",
|
||||
self.endpoint, query, type(e).__name__, e,
|
||||
)
|
||||
message = f"{type(e).__name__}: {e}"
|
||||
self.errors.append(message)
|
||||
call.update({"status": "error", "error": message})
|
||||
snippet = _raw_snippet(raw_text)
|
||||
if snippet:
|
||||
call["raw_response"] = snippet
|
||||
self._record_raw(raw_text)
|
||||
return []
|
||||
finally:
|
||||
call["elapsed_ms"] = int((time.monotonic() - started) * 1000)
|
||||
# 非 pending 表示已经在异常分支写好;成功分支会在 return 前覆盖并追加。
|
||||
if call.get("status") != "pending":
|
||||
self.calls.append(call)
|
||||
|
||||
# 正常情况下接口直接返回数组;兼容包了一层 data/results 的结构(含工作正常的
|
||||
# knowledge-search-v2:{"success":true,"results":[...]})。
|
||||
if isinstance(data, list):
|
||||
items = data
|
||||
elif isinstance(data, dict):
|
||||
if "success" in data:
|
||||
call["response_success"] = bool(data.get("success"))
|
||||
if "count" in data:
|
||||
call["response_count"] = data.get("count")
|
||||
# 接口显式标失败时把整体回传前端便于定位
|
||||
if data.get("success") is False:
|
||||
self.errors.append(f"接口返回 success=false:{str(data.get('error') or '')[:200]}")
|
||||
message = f"接口返回 success=false:{str(data.get('error') or '')[:200]}"
|
||||
self.errors.append(message)
|
||||
call.update({"status": "success_false", "error": message})
|
||||
snippet = _raw_snippet(raw_text)
|
||||
if snippet:
|
||||
call["raw_response"] = snippet
|
||||
self.calls.append(call)
|
||||
self._record_raw(raw_text)
|
||||
return []
|
||||
items = data.get("data") or data.get("results") or data.get("documents") or []
|
||||
else:
|
||||
# 返回结构既不是数组也不是已知 dict 包装:视为异常,把原始响应回传前端
|
||||
logger.warning("intranet search unexpected response type for query=%r: %s", query, type(data).__name__)
|
||||
self.errors.append(f"返回结构异常:期望数组,实际为 {type(data).__name__}")
|
||||
message = f"返回结构异常:期望数组,实际为 {type(data).__name__}"
|
||||
self.errors.append(message)
|
||||
call.update({"status": "bad_shape", "error": message})
|
||||
snippet = _raw_snippet(raw_text)
|
||||
if snippet:
|
||||
call["raw_response"] = snippet
|
||||
self.calls.append(call)
|
||||
self._record_raw(raw_text)
|
||||
return []
|
||||
|
||||
results: List[SearchResult] = []
|
||||
logger.info(
|
||||
"[intranet-search] OK endpoint=%s query=%r http_status=%s raw_items=%s",
|
||||
self.endpoint, query, call.get("http_status"), len(items) if isinstance(items, list) else 0,
|
||||
)
|
||||
results: list[SearchResult] = []
|
||||
for i, item in enumerate(items[:max_results]):
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
@@ -158,6 +261,17 @@ class IntranetSearchProvider(BaseSearchProvider):
|
||||
source_type="intranet",
|
||||
recUuid=_pick(merged, ("recUuid", "rec_uuid", "id")),
|
||||
))
|
||||
call.update({
|
||||
"status": "ok" if results else "empty",
|
||||
"ok": True,
|
||||
"count": len(results),
|
||||
"item_count": len(items) if isinstance(items, list) else 0,
|
||||
})
|
||||
if not results:
|
||||
snippet = _raw_snippet(raw_text)
|
||||
if snippet:
|
||||
call["raw_response"] = snippet
|
||||
self.calls.append(call)
|
||||
return results
|
||||
|
||||
async def health_check(self) -> bool:
|
||||
|
||||
@@ -40,6 +40,9 @@ class MaterialPackage(TypedDict, total=False):
|
||||
skill_search_used: bool
|
||||
# 技能检索的步骤记录(前端展示"使用了 X 技能"卡片)
|
||||
skill_search_steps: List[SkillStep]
|
||||
# 内网知识库接口诊断:端点、每个检索词的命中数/耗时/错误、原始响应片段等。
|
||||
# 用于排查 "检索很快返回 0 条" 时到底是否真正调用了内网接口。
|
||||
intranet_debug: dict
|
||||
|
||||
# ── 大纲 ──────────────────────────────────────────────────────────────────────
|
||||
# 三级结构:章节(##) → 小节(###) → 子要点(bullet -)。
|
||||
|
||||
+14
-9
@@ -12,9 +12,10 @@ from deerflow.agents.middlewares.prompt_prefix_middleware import PromptPrefixMid
|
||||
from deerflow.agents.middlewares.rag_citation_middleware import RagCitationMiddleware
|
||||
from deerflow.agents.middlewares.setup_agent_approval_middleware import SetupAgentApprovalMiddleware
|
||||
from deerflow.agents.middlewares.setup_writing_approval_middleware import SetupWritingApprovalMiddleware
|
||||
from deerflow.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
|
||||
from deerflow.agents.middlewares.summarization_middleware import BeforeSummarizationHook, DeerFlowSummarizationMiddleware
|
||||
from deerflow.agents.middlewares.title_middleware import TitleMiddleware
|
||||
from deerflow.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
|
||||
from deerflow.agents.middlewares.summarization_middleware import BeforeSummarizationHook, DeerFlowSummarizationMiddleware
|
||||
from deerflow.agents.middlewares.task_external_context_middleware import TaskExternalContextMiddleware
|
||||
from deerflow.agents.middlewares.title_middleware import TitleMiddleware
|
||||
from deerflow.agents.middlewares.todo_middleware import TodoMiddleware
|
||||
from deerflow.agents.middlewares.token_usage_middleware import TokenUsageMiddleware
|
||||
from deerflow.agents.middlewares.tool_error_handling_middleware import build_lead_runtime_middlewares
|
||||
@@ -326,12 +327,16 @@ def _build_middlewares(
|
||||
# to the global ``knowledge.rag_enabled`` config. No-op when knowledge is
|
||||
# disabled. Skipped in bootstrap/writing-setup/ephemeral like MemoryMiddleware.
|
||||
knowledge_cfg = getattr(resolved_app_config, "knowledge", None)
|
||||
if not is_bootstrap and not is_writing_setup and not is_ephemeral and knowledge_cfg is not None and knowledge_cfg.enabled:
|
||||
from deerflow.agents.middlewares.knowledge_rag_middleware import KnowledgeRagMiddleware
|
||||
|
||||
middlewares.append(KnowledgeRagMiddleware(limit=knowledge_cfg.rag_limit, default_enabled=knowledge_cfg.rag_enabled))
|
||||
|
||||
# Add ViewImageMiddleware only for the NATIVE-vision path (main model
|
||||
if not is_bootstrap and not is_writing_setup and not is_ephemeral and knowledge_cfg is not None and knowledge_cfg.enabled:
|
||||
from deerflow.agents.middlewares.knowledge_rag_middleware import KnowledgeRagMiddleware
|
||||
|
||||
middlewares.append(KnowledgeRagMiddleware(limit=knowledge_cfg.rag_limit, default_enabled=knowledge_cfg.rag_enabled))
|
||||
|
||||
# TaskExternalContextMiddleware — inject hidden per-run business context
|
||||
# (for example rwfx's completed 3q report) into the model request only.
|
||||
middlewares.append(TaskExternalContextMiddleware())
|
||||
|
||||
# Add ViewImageMiddleware only for the NATIVE-vision path (main model
|
||||
# ingests raw image data). In delegated mode the view_image tool already
|
||||
# returns a text description and stores nothing in `viewed_images`, so the
|
||||
# middleware would otherwise inject a spurious "No images have been viewed."
|
||||
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
"""Inject task-specific external context into the model request.
|
||||
|
||||
The web task workspace can pass business data through LangGraph run context
|
||||
without showing it in the chat transcript. This middleware converts that
|
||||
context into a transient message right before model invocation, so it is
|
||||
available to the agent but never persisted as user-visible conversation state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any, override
|
||||
|
||||
from langchain.agents import AgentState
|
||||
from langchain.agents.middleware import AgentMiddleware
|
||||
from langchain_core.messages import HumanMessage
|
||||
|
||||
_CONTEXT_KEY = "task_external_context"
|
||||
_TAG = "task_external_context"
|
||||
_MAX_CONTEXT_CHARS = 18000
|
||||
_TAIL_CONTEXT_CHARS = 2500
|
||||
|
||||
|
||||
def _truncate_middle(text: str, max_chars: int = _MAX_CONTEXT_CHARS) -> str:
|
||||
if len(text) <= max_chars:
|
||||
return text
|
||||
marker = "\n\n...[隐藏任务上下文过长,已截断中间部分]...\n\n"
|
||||
tail_chars = min(_TAIL_CONTEXT_CHARS, max_chars // 4)
|
||||
head_chars = max(0, max_chars - len(marker) - tail_chars)
|
||||
return f"{text[:head_chars]}{marker}{text[-tail_chars:]}"
|
||||
|
||||
|
||||
def _render_context(value: Any) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, str):
|
||||
return value.strip()
|
||||
try:
|
||||
text = json.dumps(value, ensure_ascii=False, indent=2, default=str).strip()
|
||||
except Exception:
|
||||
text = str(value).strip()
|
||||
return _truncate_middle(text)
|
||||
|
||||
|
||||
def _task_context_from_request(request: Any) -> Any:
|
||||
runtime = getattr(request, "runtime", None)
|
||||
context = getattr(runtime, "context", None)
|
||||
if isinstance(context, dict):
|
||||
return context.get(_CONTEXT_KEY)
|
||||
return None
|
||||
|
||||
|
||||
def _build_context_message(value: Any) -> HumanMessage | None:
|
||||
text = _render_context(value)
|
||||
if not text:
|
||||
return None
|
||||
content = (
|
||||
f"<{_TAG}>\n"
|
||||
"[系统提示:以下是业务系统随本次运行提供的隐藏任务上下文,不是用户的新输入。"
|
||||
"回答当前任务时应作为背景资料参考;不要主动说明这些内容来自隐藏上下文,"
|
||||
"也不要在回答中暴露本标签本身。]\n\n"
|
||||
f"{text}\n"
|
||||
f"</{_TAG}>"
|
||||
)
|
||||
return HumanMessage(
|
||||
content=content,
|
||||
additional_kwargs={"hide_from_ui": True, _CONTEXT_KEY: True},
|
||||
)
|
||||
|
||||
|
||||
class TaskExternalContextMiddleware(AgentMiddleware[AgentState]):
|
||||
"""Append hidden task context as a transient model-call message."""
|
||||
|
||||
state_schema = AgentState
|
||||
|
||||
@override
|
||||
async def awrap_model_call(self, request: Any, handler: Any) -> Any:
|
||||
msg = _build_context_message(_task_context_from_request(request))
|
||||
if msg is not None:
|
||||
request = request.override(messages=[*request.messages, msg])
|
||||
return await handler(request)
|
||||
|
||||
@override
|
||||
def wrap_model_call(self, request: Any, handler: Any) -> Any:
|
||||
msg = _build_context_message(_task_context_from_request(request))
|
||||
if msg is not None:
|
||||
request = request.override(messages=[*request.messages, msg])
|
||||
return handler(request)
|
||||
+159
-33
@@ -1,21 +1,49 @@
|
||||
"""Middleware for automatic thread title generation."""
|
||||
|
||||
import logging
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any, NotRequired, override
|
||||
|
||||
from langchain.agents import AgentState
|
||||
from langchain.agents.middleware import AgentMiddleware
|
||||
from langgraph.runtime import Runtime
|
||||
|
||||
from deerflow.config.title_config import get_title_config
|
||||
from deerflow.models import create_chat_model
|
||||
"""Middleware for automatic thread title generation."""
|
||||
|
||||
import copy
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, NotRequired, override
|
||||
|
||||
from langchain.agents import AgentState
|
||||
from langchain.agents.middleware import AgentMiddleware
|
||||
from langgraph.runtime import Runtime
|
||||
|
||||
from deerflow.config.title_config import get_title_config
|
||||
from deerflow.models import create_chat_model
|
||||
from deerflow.utils.time import now_iso
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from deerflow.config.app_config import AppConfig
|
||||
from deerflow.config.title_config import TitleConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger = logging.getLogger(__name__)
|
||||
_TITLE_DEBUG_LOGGER_NAME = "title_generation_debug"
|
||||
|
||||
|
||||
def _get_title_debug_logger() -> logging.Logger:
|
||||
debug_logger = logging.getLogger(_TITLE_DEBUG_LOGGER_NAME)
|
||||
if getattr(debug_logger, "_title_file_configured", False):
|
||||
return debug_logger
|
||||
|
||||
log_path = Path("logs/title_generation_debug.log")
|
||||
if not log_path.is_absolute():
|
||||
log_path = Path.cwd() / log_path
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
handler = logging.FileHandler(log_path, encoding="utf-8")
|
||||
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
|
||||
debug_logger.addHandler(handler)
|
||||
debug_logger.setLevel(logging.INFO)
|
||||
debug_logger.propagate = False
|
||||
setattr(debug_logger, "_title_file_configured", True)
|
||||
return debug_logger
|
||||
|
||||
|
||||
def log_title_debug(message: str, *args: Any) -> None:
|
||||
logger.warning(message, *args)
|
||||
_get_title_debug_logger().info(message, *args)
|
||||
|
||||
|
||||
class TitleMiddlewareState(AgentState):
|
||||
@@ -25,7 +53,7 @@ class TitleMiddlewareState(AgentState):
|
||||
title_provisional: NotRequired[bool]
|
||||
|
||||
|
||||
class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
"""Automatically generate a title for the thread after the first user message."""
|
||||
|
||||
state_schema = TitleMiddlewareState
|
||||
@@ -42,9 +70,9 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
return self._app_config.title
|
||||
return get_title_config()
|
||||
|
||||
def _normalize_content(self, content: object) -> str:
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
def _normalize_content(self, content: object) -> str:
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
|
||||
if isinstance(content, list):
|
||||
parts = [self._normalize_content(item) for item in content]
|
||||
@@ -58,8 +86,18 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
nested_content = content.get("content")
|
||||
if nested_content is not None:
|
||||
return self._normalize_content(nested_content)
|
||||
|
||||
return ""
|
||||
|
||||
return ""
|
||||
|
||||
def _message_type(self, message: object) -> str:
|
||||
if isinstance(message, dict):
|
||||
return str(message.get("type") or message.get("role") or "").lower()
|
||||
return str(getattr(message, "type", "") or getattr(message, "role", "") or "").lower()
|
||||
|
||||
def _message_content(self, message: object) -> object:
|
||||
if isinstance(message, dict):
|
||||
return message.get("content", "")
|
||||
return getattr(message, "content", "")
|
||||
|
||||
def _should_set_provisional_title(self, state: TitleMiddlewareState) -> bool:
|
||||
"""Check if we should set an instant, question-derived placeholder title.
|
||||
@@ -78,8 +116,8 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
|
||||
# Only on the very first turn, as soon as the user message arrives.
|
||||
messages = state.get("messages", [])
|
||||
user_messages = [m for m in messages if m.type == "human"]
|
||||
return len(user_messages) == 1
|
||||
user_messages = [m for m in messages if self._message_type(m) in {"human", "user"}]
|
||||
return len(user_messages) == 1
|
||||
|
||||
def _should_generate_title(self, state: TitleMiddlewareState) -> bool:
|
||||
"""Check if we should generate the final LLM title for this thread."""
|
||||
@@ -98,8 +136,8 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
return False
|
||||
|
||||
# Count user and assistant messages
|
||||
user_messages = [m for m in messages if m.type == "human"]
|
||||
assistant_messages = [m for m in messages if m.type == "ai"]
|
||||
user_messages = [m for m in messages if self._message_type(m) in {"human", "user"}]
|
||||
assistant_messages = [m for m in messages if self._message_type(m) in {"ai", "assistant"}]
|
||||
|
||||
# Generate title after first complete exchange
|
||||
return len(user_messages) == 1 and len(assistant_messages) >= 1
|
||||
@@ -112,8 +150,8 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
config = self._get_title_config()
|
||||
messages = state.get("messages", [])
|
||||
|
||||
user_msg_content = next((m.content for m in messages if m.type == "human"), "")
|
||||
assistant_msg_content = next((m.content for m in messages if m.type == "ai"), "")
|
||||
user_msg_content = next((self._message_content(m) for m in messages if self._message_type(m) in {"human", "user"}), "")
|
||||
assistant_msg_content = next((self._message_content(m) for m in messages if self._message_type(m) in {"ai", "assistant"}), "")
|
||||
|
||||
user_msg = self._normalize_content(user_msg_content)
|
||||
assistant_msg = self._strip_think_tags(self._normalize_content(assistant_msg_content))
|
||||
@@ -174,7 +212,7 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
return None
|
||||
|
||||
messages = state.get("messages", [])
|
||||
user_msg_content = next((m.content for m in messages if m.type == "human"), "")
|
||||
user_msg_content = next((self._message_content(m) for m in messages if self._message_type(m) in {"human", "user"}), "")
|
||||
user_msg = self._normalize_content(user_msg_content)
|
||||
if not user_msg.strip():
|
||||
return None
|
||||
@@ -261,10 +299,98 @@ class TitleMiddleware(AgentMiddleware[TitleMiddlewareState]):
|
||||
async def abefore_model(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | None:
|
||||
return self._provisional_title_result(state)
|
||||
|
||||
@override
|
||||
def after_model(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | None:
|
||||
return self._generate_title_result(state)
|
||||
|
||||
@override
|
||||
async def aafter_model(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | None:
|
||||
return await self._agenerate_title_result(state)
|
||||
@override
|
||||
def after_model(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | None:
|
||||
# Keep the provisional title in place during the main run. The gateway
|
||||
# worker finalizes it in a background task after the answer is complete.
|
||||
return None
|
||||
|
||||
@override
|
||||
async def aafter_model(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | None:
|
||||
# Keep the provisional title in place during the main async run so the
|
||||
# worker's post-run background finalizer can replace it with the real
|
||||
# LLM-generated title without blocking the input box.
|
||||
return None
|
||||
|
||||
|
||||
async def finalize_thread_title(
|
||||
*,
|
||||
checkpointer: Any,
|
||||
thread_store: Any | None,
|
||||
thread_id: str,
|
||||
source_state: dict[str, Any],
|
||||
app_config: "AppConfig | None" = None,
|
||||
) -> bool:
|
||||
"""Finalize a provisional thread title in the background.
|
||||
|
||||
``source_state`` must be the checkpoint snapshot captured when the first run
|
||||
completed. That avoids races where the user asks a second question before
|
||||
the background title task starts, which would otherwise make the current
|
||||
checkpoint fail the "first exchange only" predicate and leave the title
|
||||
provisional forever.
|
||||
"""
|
||||
|
||||
log_title_debug("Title finalize: evaluating background title generation for thread %s", thread_id)
|
||||
|
||||
middleware = TitleMiddleware(app_config=app_config)
|
||||
source_snapshot = copy.deepcopy(source_state)
|
||||
if not middleware._should_generate_title(source_snapshot):
|
||||
log_title_debug(
|
||||
"Title finalize: skipped for thread %s because source snapshot is not eligible",
|
||||
thread_id,
|
||||
)
|
||||
return False
|
||||
|
||||
log_title_debug("Title finalize: starting background title generation for thread %s", thread_id)
|
||||
result = await middleware._agenerate_title_result(source_snapshot)
|
||||
if not result:
|
||||
log_title_debug("Title finalize: no title result produced for thread %s", thread_id)
|
||||
return False
|
||||
|
||||
read_config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
|
||||
checkpoint_tuple = await checkpointer.aget_tuple(read_config)
|
||||
if checkpoint_tuple is None:
|
||||
log_title_debug("Title finalize: skipped for thread %s because checkpoint was not found", thread_id)
|
||||
return False
|
||||
|
||||
checkpoint: dict[str, Any] = dict(getattr(checkpoint_tuple, "checkpoint", {}) or {})
|
||||
metadata: dict[str, Any] = dict(getattr(checkpoint_tuple, "metadata", {}) or {})
|
||||
channel_values: dict[str, Any] = dict(checkpoint.get("channel_values", {}) or {})
|
||||
|
||||
# If another path already finalized or manually renamed the title, keep it.
|
||||
if channel_values.get("title_provisional") is not True:
|
||||
log_title_debug(
|
||||
"Title finalize: skipped for thread %s because title is no longer provisional",
|
||||
thread_id,
|
||||
)
|
||||
return False
|
||||
|
||||
channel_values.update(result)
|
||||
checkpoint["channel_values"] = channel_values
|
||||
metadata["updated_at"] = now_iso()
|
||||
|
||||
write_config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
new_versions: dict[str, Any] = {}
|
||||
get_next_version = getattr(checkpointer, "get_next_version", None)
|
||||
if callable(get_next_version):
|
||||
versions = dict(checkpoint.get("channel_versions", {}) or {})
|
||||
for key in ("title", "title_provisional"):
|
||||
next_version = get_next_version(versions.get(key), None)
|
||||
versions[key] = next_version
|
||||
new_versions[key] = next_version
|
||||
checkpoint["channel_versions"] = versions
|
||||
|
||||
await checkpointer.aput(write_config, checkpoint, metadata, new_versions)
|
||||
|
||||
title = result.get("title")
|
||||
if title and thread_store is not None:
|
||||
await thread_store.update_display_name(thread_id, title, user_id=None)
|
||||
|
||||
log_title_debug("Title finalize: completed for thread %s with title %r", thread_id, title)
|
||||
|
||||
return True
|
||||
@@ -18,6 +18,7 @@ import logging
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
from uuid import uuid4
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
@@ -38,6 +39,60 @@ class PromptPrefixSettings(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
class AppearanceRouteParam(BaseModel):
|
||||
"""Runtime query-param descriptor for top-bar shortcut buttons.
|
||||
|
||||
The frontend resolves login params from ``userInfo`` and theme params from
|
||||
the current UI theme, matching the light-app center's link builder.
|
||||
"""
|
||||
|
||||
key: str = Field(default="", description="Query parameter name appended to the shortcut URL")
|
||||
paramType: Literal["login", "theme"] = Field(default="login", description="Where the parameter value comes from")
|
||||
source: str | None = Field(default=None, description="login param source field in userInfo, or 'other'")
|
||||
customValue: str | None = Field(default=None, description="Literal value when source is 'other'")
|
||||
themeValue: str | None = Field(default=None, description="Theme-value mapping string, e.g. red=red;dark-blue=dark")
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
def _default_shortcut_route_params() -> list[AppearanceRouteParam]:
|
||||
return [
|
||||
AppearanceRouteParam(
|
||||
key="access_token",
|
||||
paramType="login",
|
||||
source="access_token",
|
||||
customValue="",
|
||||
themeValue="red=red;blue=blue;dark-blue=dark-blue",
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
class CompactShortcutButton(BaseModel):
|
||||
"""Configurable shortcut shown in top bars before the theme switcher."""
|
||||
|
||||
id: str = Field(default_factory=lambda: uuid4().hex, description="Stable row id for editing")
|
||||
label: str = Field(default="快捷入口", description="Button text shown next to the icon")
|
||||
url: str = Field(default="", description="Target URL opened in a new tab")
|
||||
route_params: list[AppearanceRouteParam] = Field(
|
||||
default_factory=_default_shortcut_route_params,
|
||||
description="Query parameters resolved at launch time and appended to url",
|
||||
)
|
||||
enabled: bool = Field(default=True, description="Whether this shortcut is rendered")
|
||||
sort_order: int = Field(default=0, description="Display order in top bars")
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
def _default_compact_shortcut_buttons() -> list[CompactShortcutButton]:
|
||||
return [
|
||||
CompactShortcutButton(
|
||||
id="default-compact-shortcut",
|
||||
label="快捷入口",
|
||||
url="",
|
||||
enabled=True,
|
||||
sort_order=0,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
class AppearanceSettings(BaseModel):
|
||||
"""System-wide appearance defaults that any *new* user inherits.
|
||||
|
||||
@@ -63,6 +118,10 @@ class AppearanceSettings(BaseModel):
|
||||
default="full",
|
||||
description="Default interface layout for users who haven't picked their own — full (sidebar + header) / compact (hidden)",
|
||||
)
|
||||
compact_shortcut_buttons: list[CompactShortcutButton] = Field(
|
||||
default_factory=_default_compact_shortcut_buttons,
|
||||
description="Shortcut buttons rendered before the theme switcher in top bars",
|
||||
)
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
@@ -206,6 +265,26 @@ class AgentModuleSettings(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
class PluginDomainAgentMapping(BaseModel):
|
||||
"""One configured host -> agent mapping for iframe chat auto-selection."""
|
||||
|
||||
id: str = Field(..., description="Stable row id for client-side editing")
|
||||
domain: str = Field(..., description="Normalized host matched against pageContext.url")
|
||||
agent_id: str = Field(..., description="Agent id selected when the host matches")
|
||||
enabled: bool = Field(default=True, description="Whether this mapping is active")
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
class PluginManagementSettings(BaseModel):
|
||||
"""Admin-configurable iframe plugin mappings."""
|
||||
|
||||
domain_agent_mappings: list[PluginDomainAgentMapping] = Field(
|
||||
default_factory=list,
|
||||
description="Ordered host -> agent mappings used by the iframe chat entry page.",
|
||||
)
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
def get_module_tag_ids(settings: AgentModuleSettings, key: str) -> list[str]:
|
||||
"""Return the configured agent-tag ids for ``key`` (empty when unset)."""
|
||||
return list(settings.modules.get(key, []))
|
||||
@@ -249,6 +328,26 @@ class ConcurrencyMonitorSettings(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
class ModelConcurrencySettings(BaseModel):
|
||||
"""Admin-configurable per-model concurrent Q&A limits."""
|
||||
|
||||
enabled: bool = Field(
|
||||
default=False,
|
||||
description="Whether per-model concurrency limiting is enabled.",
|
||||
)
|
||||
default_limit: int = Field(
|
||||
default=10,
|
||||
ge=1,
|
||||
le=10_000,
|
||||
description="Fallback max concurrent runs for models without an explicit override.",
|
||||
)
|
||||
model_limits: dict[str, int] = Field(
|
||||
default_factory=dict,
|
||||
description="Optional per-model overrides: model_name -> max concurrent runs.",
|
||||
)
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
class SystemSettings(BaseModel):
|
||||
"""Top-level system settings document."""
|
||||
|
||||
@@ -260,8 +359,10 @@ class SystemSettings(BaseModel):
|
||||
citation_display: CitationDisplaySettings = Field(default_factory=CitationDisplaySettings)
|
||||
leaderboard: LeaderboardSettings = Field(default_factory=LeaderboardSettings)
|
||||
skill_compression: SkillCompressionSettings = Field(default_factory=SkillCompressionSettings)
|
||||
plugin_management: PluginManagementSettings = Field(default_factory=PluginManagementSettings)
|
||||
agent_modules: AgentModuleSettings = Field(default_factory=AgentModuleSettings)
|
||||
concurrency_monitor: ConcurrencyMonitorSettings = Field(default_factory=ConcurrencyMonitorSettings)
|
||||
model_concurrency: ModelConcurrencySettings = Field(default_factory=ModelConcurrencySettings)
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
|
||||
@@ -305,6 +406,15 @@ def get_concurrency_monitor_settings() -> ConcurrencyMonitorSettings:
|
||||
return ConcurrencyMonitorSettings()
|
||||
|
||||
|
||||
def get_model_concurrency_settings() -> ModelConcurrencySettings:
|
||||
"""Convenience accessor for the model-concurrency block (safe defaults on error)."""
|
||||
try:
|
||||
return load_system_settings().model_concurrency
|
||||
except Exception:
|
||||
logger.debug("Failed to read model concurrency settings; using defaults", exc_info=True)
|
||||
return ModelConcurrencySettings()
|
||||
|
||||
|
||||
def save_system_settings(settings: SystemSettings) -> SystemSettings:
|
||||
"""Persist the settings document atomically, returning the saved value."""
|
||||
path = _settings_path()
|
||||
|
||||
+95
-44
@@ -6,20 +6,21 @@ from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
from sqlalchemy import and_, case, delete, func, or_, select
|
||||
from sqlalchemy import and_, case, delete, func, literal, not_, or_, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from deerflow.config.agent_name_desensitize import source_terms_for_needle
|
||||
from deerflow.persistence.agents.base import AgentStore
|
||||
from deerflow.persistence.agents.model import (
|
||||
AgentExtrasRow,
|
||||
AgentFavoriteRow,
|
||||
AgentPinRow,
|
||||
AgentRow,
|
||||
AgentSkillRow,
|
||||
)
|
||||
from deerflow.persistence.tags.model import TagAssignmentRow
|
||||
from deerflow.persistence.agents.model import (
|
||||
AgentExtrasRow,
|
||||
AgentFavoriteRow,
|
||||
AgentPinRow,
|
||||
AgentRow,
|
||||
AgentSkillRow,
|
||||
)
|
||||
from deerflow.persistence.roundtable_chains.model import RoundtableChainRow
|
||||
from deerflow.persistence.tags.model import TagAssignmentRow
|
||||
|
||||
|
||||
def _row_to_dict(row: AgentRow) -> dict[str, Any]:
|
||||
@@ -32,9 +33,33 @@ def _row_to_dict(row: AgentRow) -> dict[str, Any]:
|
||||
return data
|
||||
|
||||
|
||||
class AgentRepository(AgentStore):
|
||||
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||
self._sf = session_factory
|
||||
class AgentRepository(AgentStore):
|
||||
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||
self._sf = session_factory
|
||||
|
||||
@staticmethod
|
||||
def _business_chain_agent_exists(agent_id_col):
|
||||
return (
|
||||
select(RoundtableChainRow.id)
|
||||
.where(
|
||||
or_(
|
||||
RoundtableChainRow.seats.like(literal('%"agent_id": "') + agent_id_col + literal('"%')),
|
||||
RoundtableChainRow.seats.like(literal('%"agent_id":"') + agent_id_col + literal('"%')),
|
||||
)
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _valid_favorite_filter(cls):
|
||||
return or_(
|
||||
AgentFavoriteRow.origin == "user",
|
||||
and_(
|
||||
AgentFavoriteRow.origin == "position",
|
||||
AgentRow.user_id.is_not(None),
|
||||
not_(cls._business_chain_agent_exists(AgentFavoriteRow.agent_id)),
|
||||
),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _visible_filter(user_id: str):
|
||||
@@ -52,15 +77,23 @@ class AgentRepository(AgentStore):
|
||||
``published=false``, so this clause cannot lean on ``published``;
|
||||
- favorited — the "我的" listing keeps favorited agents even after
|
||||
the owner unpublishes them, so reads keep honoring the favorite.
|
||||
"""
|
||||
fav_subq = select(AgentFavoriteRow.agent_id).where(AgentFavoriteRow.user_id == user_id)
|
||||
return or_(
|
||||
AgentRow.user_id.is_(None),
|
||||
AgentRow.user_id == user_id,
|
||||
AgentRow.published.is_(True),
|
||||
AgentRow.is_private.is_(True),
|
||||
AgentRow.id.in_(fav_subq),
|
||||
)
|
||||
"""
|
||||
valid_favorite_exists = (
|
||||
select(AgentFavoriteRow.id)
|
||||
.where(
|
||||
AgentFavoriteRow.user_id == user_id,
|
||||
AgentFavoriteRow.agent_id == AgentRow.id,
|
||||
AgentRepository._valid_favorite_filter(),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
return or_(
|
||||
AgentRow.user_id.is_(None),
|
||||
AgentRow.user_id == user_id,
|
||||
AgentRow.published.is_(True),
|
||||
AgentRow.is_private.is_(True),
|
||||
valid_favorite_exists,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _listing_filter(user_id: str):
|
||||
@@ -268,11 +301,20 @@ class AgentRepository(AgentStore):
|
||||
or_(AgentRow.user_id.is_(None), AgentRow.published.is_(True)),
|
||||
AgentRow.is_private.is_(False),
|
||||
)
|
||||
if scope == "mine":
|
||||
# owned OR favorited. A correlated IN-subquery (not a JOIN) keeps a
|
||||
# single row per agent even when it is both owned and favorited.
|
||||
fav_subq = select(AgentFavoriteRow.agent_id).where(AgentFavoriteRow.user_id == user_id)
|
||||
return or_(AgentRow.user_id == user_id, AgentRow.id.in_(fav_subq))
|
||||
if scope == "mine":
|
||||
# owned OR valid favorites. Built-in agents and business-chain
|
||||
# seats only enter "mine" when explicitly favorited by the user;
|
||||
# the original position-granted behavior stays for ordinary agents.
|
||||
valid_favorite_exists = (
|
||||
select(AgentFavoriteRow.id)
|
||||
.where(
|
||||
AgentFavoriteRow.user_id == user_id,
|
||||
AgentFavoriteRow.agent_id == AgentRow.id,
|
||||
self._valid_favorite_filter(),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
return or_(AgentRow.user_id == user_id, valid_favorite_exists)
|
||||
if scope == "private":
|
||||
return AgentRow.is_private.is_(True)
|
||||
if scope == "builtin":
|
||||
@@ -437,18 +479,22 @@ class AgentRepository(AgentStore):
|
||||
|
||||
# --- Favorites ("收藏") ---
|
||||
|
||||
async def add_favorite(self, user_id: str, agent_id: str) -> None:
|
||||
async with self._sf() as session:
|
||||
existing = (
|
||||
await session.execute(
|
||||
select(AgentFavoriteRow.id)
|
||||
.where(AgentFavoriteRow.user_id == user_id, AgentFavoriteRow.agent_id == agent_id)
|
||||
.limit(1)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
return
|
||||
session.add(AgentFavoriteRow(id=uuid4().hex, user_id=user_id, agent_id=agent_id))
|
||||
async def add_favorite(self, user_id: str, agent_id: str) -> None:
|
||||
async with self._sf() as session:
|
||||
existing = (
|
||||
await session.execute(
|
||||
select(AgentFavoriteRow)
|
||||
.where(AgentFavoriteRow.user_id == user_id, AgentFavoriteRow.agent_id == agent_id)
|
||||
.limit(1)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
if existing.origin != "user" or existing.position_id is not None:
|
||||
existing.origin = "user"
|
||||
existing.position_id = None
|
||||
await session.commit()
|
||||
return
|
||||
session.add(AgentFavoriteRow(id=uuid4().hex, user_id=user_id, agent_id=agent_id))
|
||||
try:
|
||||
await session.commit()
|
||||
except IntegrityError:
|
||||
@@ -467,12 +513,17 @@ class AgentRepository(AgentStore):
|
||||
await session.commit()
|
||||
return (result.rowcount or 0) > 0
|
||||
|
||||
async def list_favorite_ids(self, user_id: str) -> set[str]:
|
||||
async with self._sf() as session:
|
||||
result = await session.execute(
|
||||
select(AgentFavoriteRow.agent_id).where(AgentFavoriteRow.user_id == user_id)
|
||||
)
|
||||
return {row[0] for row in result.all()}
|
||||
async def list_favorite_ids(self, user_id: str) -> set[str]:
|
||||
async with self._sf() as session:
|
||||
result = await session.execute(
|
||||
select(AgentFavoriteRow.agent_id)
|
||||
.join(AgentRow, AgentRow.id == AgentFavoriteRow.agent_id)
|
||||
.where(
|
||||
AgentFavoriteRow.user_id == user_id,
|
||||
self._valid_favorite_filter(),
|
||||
)
|
||||
)
|
||||
return {row[0] for row in result.all()}
|
||||
|
||||
async def delete_favorites_for_agent(self, agent_id: str) -> None:
|
||||
async with self._sf() as session:
|
||||
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
"""Stop position-granted favorites from collecting built-ins/chain seats.
|
||||
|
||||
Built-in agents and agents used as business-chain seats should only enter
|
||||
"mine" when a user explicitly clicks the favorite action. This migration
|
||||
removes historical automatic ``origin='position'`` rows for those agents while
|
||||
preserving user-created favorites (``origin='user'``) and ordinary
|
||||
position-granted agent favorites.
|
||||
|
||||
Revision ID: 20260701_01
|
||||
Revises: 20260625_04
|
||||
Create Date: 2026-07-01
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import inspect
|
||||
|
||||
revision = "20260701_01"
|
||||
down_revision = "20260625_04"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _has_table(table_name: str) -> bool:
|
||||
inspector = inspect(op.get_bind())
|
||||
return table_name in inspector.get_table_names()
|
||||
|
||||
|
||||
def _has_column(table_name: str, column_name: str) -> bool:
|
||||
if not _has_table(table_name):
|
||||
return False
|
||||
inspector = inspect(op.get_bind())
|
||||
return any(col["name"] == column_name for col in inspector.get_columns(table_name))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
if not (
|
||||
_has_table("agent_favorites")
|
||||
and _has_column("agent_favorites", "origin")
|
||||
and _has_column("agent_favorites", "agent_id")
|
||||
):
|
||||
return
|
||||
|
||||
favorites = sa.table(
|
||||
"agent_favorites",
|
||||
sa.column("agent_id", sa.String(64)),
|
||||
sa.column("origin", sa.String(16)),
|
||||
)
|
||||
cleanup_conditions = []
|
||||
if _has_table("agents") and _has_column("agents", "user_id"):
|
||||
agents = sa.table(
|
||||
"agents",
|
||||
sa.column("id", sa.String(64)),
|
||||
sa.column("user_id", sa.String(64)),
|
||||
)
|
||||
cleanup_conditions.append(
|
||||
favorites.c.agent_id.in_(sa.select(agents.c.id).where(agents.c.user_id.is_(None)))
|
||||
)
|
||||
if _has_table("roundtable_chains") and _has_column("roundtable_chains", "seats"):
|
||||
chains = sa.table(
|
||||
"roundtable_chains",
|
||||
sa.column("seats", sa.Text()),
|
||||
)
|
||||
cleanup_conditions.append(
|
||||
sa.select(sa.literal(1))
|
||||
.select_from(chains)
|
||||
.where(
|
||||
sa.or_(
|
||||
chains.c.seats.like(sa.literal('%"agent_id": "') + favorites.c.agent_id + sa.literal('"%')),
|
||||
chains.c.seats.like(sa.literal('%"agent_id":"') + favorites.c.agent_id + sa.literal('"%')),
|
||||
)
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
if not cleanup_conditions:
|
||||
return
|
||||
|
||||
op.get_bind().execute(
|
||||
favorites.delete().where(
|
||||
favorites.c.origin == "position",
|
||||
sa.or_(*cleanup_conditions),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Data cleanup only; removed auto-grants are not reconstructable.
|
||||
pass
|
||||
@@ -1,46 +1,57 @@
|
||||
"""LangGraph-compatible runtime — runs, streaming, and lifecycle management.
|
||||
|
||||
Re-exports the public API of :mod:`~deerflow.runtime.runs` and
|
||||
:mod:`~deerflow.runtime.stream_bridge` so that consumers can import
|
||||
directly from ``deerflow.runtime``.
|
||||
"""
|
||||
|
||||
from .checkpointer import checkpointer_context, get_checkpointer, make_checkpointer, reset_checkpointer
|
||||
from .runs import ConflictError, DisconnectMode, RunContext, RunManager, RunRecord, RunStatus, UnsupportedStrategyError, run_agent
|
||||
from .serialization import serialize, serialize_channel_values, serialize_lc_object, serialize_messages_tuple
|
||||
from .store import get_store, make_store, reset_store, store_context
|
||||
from .stream_bridge import END_SENTINEL, HEARTBEAT_SENTINEL, MemoryStreamBridge, StreamBridge, StreamEvent, make_stream_bridge
|
||||
|
||||
__all__ = [
|
||||
# checkpointer
|
||||
"checkpointer_context",
|
||||
"get_checkpointer",
|
||||
"make_checkpointer",
|
||||
"reset_checkpointer",
|
||||
# runs
|
||||
"ConflictError",
|
||||
"DisconnectMode",
|
||||
"RunContext",
|
||||
"RunManager",
|
||||
"RunRecord",
|
||||
"RunStatus",
|
||||
"UnsupportedStrategyError",
|
||||
"run_agent",
|
||||
# serialization
|
||||
"serialize",
|
||||
"serialize_channel_values",
|
||||
"serialize_lc_object",
|
||||
"serialize_messages_tuple",
|
||||
# store
|
||||
"get_store",
|
||||
"make_store",
|
||||
"reset_store",
|
||||
"store_context",
|
||||
# stream_bridge
|
||||
"END_SENTINEL",
|
||||
"HEARTBEAT_SENTINEL",
|
||||
"MemoryStreamBridge",
|
||||
"StreamBridge",
|
||||
"StreamEvent",
|
||||
"make_stream_bridge",
|
||||
]
|
||||
"""LangGraph-compatible runtime — runs, streaming, and lifecycle management.
|
||||
|
||||
Re-exports the public API of :mod:`~deerflow.runtime.runs` and
|
||||
:mod:`~deerflow.runtime.stream_bridge` so that consumers can import
|
||||
directly from ``deerflow.runtime``.
|
||||
"""
|
||||
|
||||
from .checkpointer import checkpointer_context, get_checkpointer, make_checkpointer, reset_checkpointer
|
||||
from .runs import (
|
||||
ConflictError,
|
||||
DisconnectMode,
|
||||
ModelConcurrencyLimitError,
|
||||
RunContext,
|
||||
RunManager,
|
||||
RunRecord,
|
||||
RunStatus,
|
||||
UnsupportedStrategyError,
|
||||
run_agent,
|
||||
)
|
||||
from .serialization import serialize, serialize_channel_values, serialize_lc_object, serialize_messages_tuple
|
||||
from .store import get_store, make_store, reset_store, store_context
|
||||
from .stream_bridge import END_SENTINEL, HEARTBEAT_SENTINEL, MemoryStreamBridge, StreamBridge, StreamEvent, make_stream_bridge
|
||||
|
||||
__all__ = [
|
||||
# checkpointer
|
||||
"checkpointer_context",
|
||||
"get_checkpointer",
|
||||
"make_checkpointer",
|
||||
"reset_checkpointer",
|
||||
# runs
|
||||
"ConflictError",
|
||||
"DisconnectMode",
|
||||
"ModelConcurrencyLimitError",
|
||||
"RunContext",
|
||||
"RunManager",
|
||||
"RunRecord",
|
||||
"RunStatus",
|
||||
"UnsupportedStrategyError",
|
||||
"run_agent",
|
||||
# serialization
|
||||
"serialize",
|
||||
"serialize_channel_values",
|
||||
"serialize_lc_object",
|
||||
"serialize_messages_tuple",
|
||||
# store
|
||||
"get_store",
|
||||
"make_store",
|
||||
"reset_store",
|
||||
"store_context",
|
||||
# stream_bridge
|
||||
"END_SENTINEL",
|
||||
"HEARTBEAT_SENTINEL",
|
||||
"MemoryStreamBridge",
|
||||
"StreamBridge",
|
||||
"StreamEvent",
|
||||
"make_stream_bridge",
|
||||
]
|
||||
+17
-16
@@ -1,16 +1,17 @@
|
||||
"""Run lifecycle management for LangGraph Platform API compatibility."""
|
||||
|
||||
from .manager import ConflictError, RunManager, RunRecord, UnsupportedStrategyError
|
||||
from .schemas import DisconnectMode, RunStatus
|
||||
from .worker import RunContext, run_agent
|
||||
|
||||
__all__ = [
|
||||
"ConflictError",
|
||||
"DisconnectMode",
|
||||
"RunContext",
|
||||
"RunManager",
|
||||
"RunRecord",
|
||||
"RunStatus",
|
||||
"UnsupportedStrategyError",
|
||||
"run_agent",
|
||||
]
|
||||
"""Run lifecycle management for LangGraph Platform API compatibility."""
|
||||
|
||||
from .manager import ConflictError, ModelConcurrencyLimitError, RunManager, RunRecord, UnsupportedStrategyError
|
||||
from .schemas import DisconnectMode, RunStatus
|
||||
from .worker import RunContext, run_agent
|
||||
|
||||
__all__ = [
|
||||
"ConflictError",
|
||||
"DisconnectMode",
|
||||
"ModelConcurrencyLimitError",
|
||||
"RunContext",
|
||||
"RunManager",
|
||||
"RunRecord",
|
||||
"RunStatus",
|
||||
"UnsupportedStrategyError",
|
||||
"run_agent",
|
||||
]
|
||||
+326
-310
@@ -1,310 +1,326 @@
|
||||
"""In-memory run registry with optional persistent RunStore backing."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from deerflow.utils.time import now_iso as _now_iso
|
||||
|
||||
from .schemas import DisconnectMode, RunStatus
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from deerflow.runtime.runs.store.base import RunStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunRecord:
|
||||
"""Mutable record for a single run."""
|
||||
|
||||
run_id: str
|
||||
thread_id: str
|
||||
assistant_id: str | None
|
||||
status: RunStatus
|
||||
on_disconnect: DisconnectMode
|
||||
multitask_strategy: str = "reject"
|
||||
metadata: dict = field(default_factory=dict)
|
||||
kwargs: dict = field(default_factory=dict)
|
||||
created_at: str = ""
|
||||
updated_at: str = ""
|
||||
task: asyncio.Task | None = field(default=None, repr=False)
|
||||
abort_event: asyncio.Event = field(default_factory=asyncio.Event, repr=False)
|
||||
abort_action: str = "interrupt"
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class RunManager:
|
||||
"""In-memory run registry with optional persistent RunStore backing.
|
||||
|
||||
All mutations are protected by an asyncio lock. When a ``store`` is
|
||||
provided, serializable metadata is also persisted to the store so
|
||||
that run history survives process restarts.
|
||||
"""
|
||||
|
||||
def __init__(self, store: RunStore | None = None) -> None:
|
||||
self._runs: dict[str, RunRecord] = {}
|
||||
self._lock = asyncio.Lock()
|
||||
self._store = store
|
||||
|
||||
async def _persist_to_store(self, record: RunRecord) -> None:
|
||||
"""Best-effort persist run record to backing store."""
|
||||
if self._store is None:
|
||||
return
|
||||
try:
|
||||
await self._store.put(
|
||||
record.run_id,
|
||||
thread_id=record.thread_id,
|
||||
assistant_id=record.assistant_id,
|
||||
status=record.status.value,
|
||||
multitask_strategy=record.multitask_strategy,
|
||||
metadata=record.metadata or {},
|
||||
kwargs=record.kwargs or {},
|
||||
created_at=record.created_at,
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist run %s to store", record.run_id, exc_info=True)
|
||||
|
||||
async def update_run_completion(self, run_id: str, **kwargs) -> None:
|
||||
"""Persist token usage and completion data to the backing store."""
|
||||
if self._store is not None:
|
||||
try:
|
||||
await self._store.update_run_completion(run_id, **kwargs)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist run completion for %s", run_id, exc_info=True)
|
||||
|
||||
async def create(
|
||||
self,
|
||||
thread_id: str,
|
||||
assistant_id: str | None = None,
|
||||
*,
|
||||
on_disconnect: DisconnectMode = DisconnectMode.cancel,
|
||||
metadata: dict | None = None,
|
||||
kwargs: dict | None = None,
|
||||
multitask_strategy: str = "reject",
|
||||
) -> RunRecord:
|
||||
"""Create a new pending run and register it."""
|
||||
run_id = str(uuid.uuid4())
|
||||
now = _now_iso()
|
||||
record = RunRecord(
|
||||
run_id=run_id,
|
||||
thread_id=thread_id,
|
||||
assistant_id=assistant_id,
|
||||
status=RunStatus.pending,
|
||||
on_disconnect=on_disconnect,
|
||||
multitask_strategy=multitask_strategy,
|
||||
metadata=metadata or {},
|
||||
kwargs=kwargs or {},
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
async with self._lock:
|
||||
self._runs[run_id] = record
|
||||
await self._persist_to_store(record)
|
||||
logger.info("Run created: run_id=%s thread_id=%s", run_id, thread_id)
|
||||
return record
|
||||
|
||||
def get(self, run_id: str) -> RunRecord | None:
|
||||
"""Return a run record by ID, or ``None``."""
|
||||
return self._runs.get(run_id)
|
||||
|
||||
def _record_from_store(self, data: dict) -> RunRecord:
|
||||
"""Rebuild a :class:`RunRecord` from a persisted store row.
|
||||
|
||||
The reconstructed record has no live ``task``/``abort_event`` wired to a
|
||||
running coroutine — it represents *history*, not an in-flight run.
|
||||
"""
|
||||
try:
|
||||
status = RunStatus(data.get("status") or RunStatus.success.value)
|
||||
except ValueError:
|
||||
status = RunStatus.success
|
||||
return RunRecord(
|
||||
run_id=data["run_id"],
|
||||
thread_id=data["thread_id"],
|
||||
assistant_id=data.get("assistant_id"),
|
||||
status=status,
|
||||
# Disconnect mode is irrelevant for a store-only record (no task to
|
||||
# cancel); ``continue_`` keeps sse_consumer's finally block a no-op.
|
||||
on_disconnect=DisconnectMode.continue_,
|
||||
multitask_strategy=data.get("multitask_strategy", "reject"),
|
||||
metadata=data.get("metadata") or {},
|
||||
kwargs=data.get("kwargs") or {},
|
||||
created_at=data.get("created_at") or "",
|
||||
updated_at=data.get("updated_at") or "",
|
||||
error=data.get("error"),
|
||||
)
|
||||
|
||||
async def get_persisted(self, run_id: str) -> RunRecord | None:
|
||||
"""Load a run from the backing store (process-restart fallback).
|
||||
|
||||
The in-memory registry is wiped on every Gateway restart, but the
|
||||
``RunStore`` row survives. When :meth:`get` misses, callers fall back
|
||||
here so a stale ``joinStream``/``GET run`` can report the run's final
|
||||
status instead of 404ing. Returns ``None`` when there is no store, no
|
||||
matching row, or the lookup fails (e.g. no user context).
|
||||
"""
|
||||
if self._store is None:
|
||||
return None
|
||||
try:
|
||||
data = await self._store.get(run_id)
|
||||
except Exception:
|
||||
logger.warning("Failed to load run %s from store", run_id, exc_info=True)
|
||||
return None
|
||||
if not data:
|
||||
return None
|
||||
return self._record_from_store(data)
|
||||
|
||||
async def list_by_thread(self, thread_id: str) -> list[RunRecord]:
|
||||
"""Return all runs for a given thread, newest first."""
|
||||
async with self._lock:
|
||||
# Dict insertion order matches creation order, so reversing it gives
|
||||
# us deterministic newest-first results even when timestamps tie.
|
||||
return [r for r in self._runs.values() if r.thread_id == thread_id]
|
||||
|
||||
async def set_status(self, run_id: str, status: RunStatus, *, error: str | None = None) -> None:
|
||||
"""Transition a run to a new status."""
|
||||
async with self._lock:
|
||||
record = self._runs.get(run_id)
|
||||
if record is None:
|
||||
logger.warning("set_status called for unknown run %s", run_id)
|
||||
return
|
||||
record.status = status
|
||||
record.updated_at = _now_iso()
|
||||
if error is not None:
|
||||
record.error = error
|
||||
if self._store is not None:
|
||||
try:
|
||||
await self._store.update_status(run_id, status.value, error=error)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist status update for run %s", run_id, exc_info=True)
|
||||
logger.info("Run %s -> %s", run_id, status.value)
|
||||
|
||||
async def cancel(self, run_id: str, *, action: str = "interrupt") -> bool:
|
||||
"""Request cancellation of a run.
|
||||
|
||||
Args:
|
||||
run_id: The run ID to cancel.
|
||||
action: "interrupt" keeps checkpoint, "rollback" reverts to pre-run state.
|
||||
|
||||
Sets the abort event with the action reason and cancels the asyncio task.
|
||||
Returns ``True`` if the run was in-flight and cancellation was initiated.
|
||||
"""
|
||||
async with self._lock:
|
||||
record = self._runs.get(run_id)
|
||||
if record is None:
|
||||
return False
|
||||
if record.status not in (RunStatus.pending, RunStatus.running):
|
||||
return False
|
||||
record.abort_action = action
|
||||
record.abort_event.set()
|
||||
if record.task is not None and not record.task.done():
|
||||
record.task.cancel()
|
||||
record.status = RunStatus.interrupted
|
||||
record.updated_at = _now_iso()
|
||||
logger.info("Run %s cancelled (action=%s)", run_id, action)
|
||||
return True
|
||||
|
||||
async def create_or_reject(
|
||||
self,
|
||||
thread_id: str,
|
||||
assistant_id: str | None = None,
|
||||
*,
|
||||
on_disconnect: DisconnectMode = DisconnectMode.cancel,
|
||||
metadata: dict | None = None,
|
||||
kwargs: dict | None = None,
|
||||
multitask_strategy: str = "reject",
|
||||
) -> RunRecord:
|
||||
"""Atomically check for inflight runs and create a new one.
|
||||
|
||||
For ``reject`` strategy, raises ``ConflictError`` if thread
|
||||
already has a pending/running run. For ``interrupt``/``rollback``,
|
||||
cancels inflight runs before creating.
|
||||
|
||||
This method holds the lock across both the check and the insert,
|
||||
eliminating the TOCTOU race in separate ``has_inflight`` + ``create``.
|
||||
"""
|
||||
run_id = str(uuid.uuid4())
|
||||
now = _now_iso()
|
||||
|
||||
_supported_strategies = ("reject", "interrupt", "rollback")
|
||||
|
||||
async with self._lock:
|
||||
if multitask_strategy not in _supported_strategies:
|
||||
raise UnsupportedStrategyError(f"Multitask strategy '{multitask_strategy}' is not yet supported. Supported strategies: {', '.join(_supported_strategies)}")
|
||||
|
||||
inflight = [r for r in self._runs.values() if r.thread_id == thread_id and r.status in (RunStatus.pending, RunStatus.running)]
|
||||
|
||||
if multitask_strategy == "reject" and inflight:
|
||||
raise ConflictError(f"Thread {thread_id} already has an active run")
|
||||
|
||||
if multitask_strategy in ("interrupt", "rollback") and inflight:
|
||||
for r in inflight:
|
||||
r.abort_action = multitask_strategy
|
||||
r.abort_event.set()
|
||||
if r.task is not None and not r.task.done():
|
||||
r.task.cancel()
|
||||
r.status = RunStatus.interrupted
|
||||
r.updated_at = now
|
||||
logger.info(
|
||||
"Cancelled %d inflight run(s) on thread %s (strategy=%s)",
|
||||
len(inflight),
|
||||
thread_id,
|
||||
multitask_strategy,
|
||||
)
|
||||
|
||||
record = RunRecord(
|
||||
run_id=run_id,
|
||||
thread_id=thread_id,
|
||||
assistant_id=assistant_id,
|
||||
status=RunStatus.pending,
|
||||
on_disconnect=on_disconnect,
|
||||
multitask_strategy=multitask_strategy,
|
||||
metadata=metadata or {},
|
||||
kwargs=kwargs or {},
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
self._runs[run_id] = record
|
||||
|
||||
await self._persist_to_store(record)
|
||||
logger.info("Run created: run_id=%s thread_id=%s", run_id, thread_id)
|
||||
return record
|
||||
|
||||
async def has_inflight(self, thread_id: str) -> bool:
|
||||
"""Return ``True`` if *thread_id* has a pending or running run."""
|
||||
async with self._lock:
|
||||
return any(r.thread_id == thread_id and r.status in (RunStatus.pending, RunStatus.running) for r in self._runs.values())
|
||||
|
||||
async def list_active(self) -> list[RunRecord]:
|
||||
"""Return every in-flight run (status pending/running), oldest first.
|
||||
|
||||
Powers the admin concurrency monitor: each record represents one live
|
||||
call into the model runtime (interactive chat, agent chat, roundtable
|
||||
seat, …), so ``len(list_active())`` is the current concurrent-call count
|
||||
and each record can be cancelled to relieve model pressure. Completed
|
||||
records linger in ``_runs`` until :meth:`cleanup`, so we filter by status
|
||||
rather than returning the whole registry.
|
||||
"""
|
||||
async with self._lock:
|
||||
# Dict insertion order == creation order, so this is oldest-first.
|
||||
return [r for r in self._runs.values() if r.status in (RunStatus.pending, RunStatus.running)]
|
||||
|
||||
async def cleanup(self, run_id: str, *, delay: float = 300) -> None:
|
||||
"""Remove a run record after an optional delay."""
|
||||
if delay > 0:
|
||||
await asyncio.sleep(delay)
|
||||
async with self._lock:
|
||||
self._runs.pop(run_id, None)
|
||||
logger.debug("Run record %s cleaned up", run_id)
|
||||
|
||||
|
||||
class ConflictError(Exception):
|
||||
"""Raised when multitask_strategy=reject and thread has inflight runs."""
|
||||
|
||||
|
||||
class UnsupportedStrategyError(Exception):
|
||||
"""Raised when a multitask_strategy value is not yet implemented."""
|
||||
"""In-memory run registry with optional persistent RunStore backing."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from deerflow.utils.time import now_iso as _now_iso
|
||||
|
||||
from .schemas import DisconnectMode, RunStatus
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from deerflow.runtime.runs.store.base import RunStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunRecord:
|
||||
"""Mutable record for a single run."""
|
||||
|
||||
run_id: str
|
||||
thread_id: str
|
||||
assistant_id: str | None
|
||||
status: RunStatus
|
||||
on_disconnect: DisconnectMode
|
||||
multitask_strategy: str = "reject"
|
||||
metadata: dict = field(default_factory=dict)
|
||||
kwargs: dict = field(default_factory=dict)
|
||||
created_at: str = ""
|
||||
updated_at: str = ""
|
||||
task: asyncio.Task | None = field(default=None, repr=False)
|
||||
abort_event: asyncio.Event = field(default_factory=asyncio.Event, repr=False)
|
||||
abort_action: str = "interrupt"
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class RunManager:
|
||||
"""In-memory run registry with optional persistent RunStore backing.
|
||||
|
||||
All mutations are protected by an asyncio lock. When a ``store`` is
|
||||
provided, serializable metadata is also persisted to the store so
|
||||
that run history survives process restarts.
|
||||
"""
|
||||
|
||||
def __init__(self, store: RunStore | None = None) -> None:
|
||||
self._runs: dict[str, RunRecord] = {}
|
||||
self._lock = asyncio.Lock()
|
||||
self._store = store
|
||||
|
||||
async def _persist_to_store(self, record: RunRecord) -> None:
|
||||
"""Best-effort persist run record to backing store."""
|
||||
if self._store is None:
|
||||
return
|
||||
try:
|
||||
await self._store.put(
|
||||
record.run_id,
|
||||
thread_id=record.thread_id,
|
||||
assistant_id=record.assistant_id,
|
||||
status=record.status.value,
|
||||
multitask_strategy=record.multitask_strategy,
|
||||
metadata=record.metadata or {},
|
||||
kwargs=record.kwargs or {},
|
||||
created_at=record.created_at,
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist run %s to store", record.run_id, exc_info=True)
|
||||
|
||||
async def update_run_completion(self, run_id: str, **kwargs) -> None:
|
||||
"""Persist token usage and completion data to the backing store."""
|
||||
if self._store is not None:
|
||||
try:
|
||||
await self._store.update_run_completion(run_id, **kwargs)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist run completion for %s", run_id, exc_info=True)
|
||||
|
||||
async def create(
|
||||
self,
|
||||
thread_id: str,
|
||||
assistant_id: str | None = None,
|
||||
*,
|
||||
on_disconnect: DisconnectMode = DisconnectMode.cancel,
|
||||
metadata: dict | None = None,
|
||||
kwargs: dict | None = None,
|
||||
multitask_strategy: str = "reject",
|
||||
) -> RunRecord:
|
||||
"""Create a new pending run and register it."""
|
||||
run_id = str(uuid.uuid4())
|
||||
now = _now_iso()
|
||||
record = RunRecord(
|
||||
run_id=run_id,
|
||||
thread_id=thread_id,
|
||||
assistant_id=assistant_id,
|
||||
status=RunStatus.pending,
|
||||
on_disconnect=on_disconnect,
|
||||
multitask_strategy=multitask_strategy,
|
||||
metadata=metadata or {},
|
||||
kwargs=kwargs or {},
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
async with self._lock:
|
||||
self._runs[run_id] = record
|
||||
await self._persist_to_store(record)
|
||||
logger.info("Run created: run_id=%s thread_id=%s", run_id, thread_id)
|
||||
return record
|
||||
|
||||
def get(self, run_id: str) -> RunRecord | None:
|
||||
"""Return a run record by ID, or ``None``."""
|
||||
return self._runs.get(run_id)
|
||||
|
||||
def _record_from_store(self, data: dict) -> RunRecord:
|
||||
"""Rebuild a :class:`RunRecord` from a persisted store row.
|
||||
|
||||
The reconstructed record has no live ``task``/``abort_event`` wired to a
|
||||
running coroutine — it represents *history*, not an in-flight run.
|
||||
"""
|
||||
try:
|
||||
status = RunStatus(data.get("status") or RunStatus.success.value)
|
||||
except ValueError:
|
||||
status = RunStatus.success
|
||||
return RunRecord(
|
||||
run_id=data["run_id"],
|
||||
thread_id=data["thread_id"],
|
||||
assistant_id=data.get("assistant_id"),
|
||||
status=status,
|
||||
# Disconnect mode is irrelevant for a store-only record (no task to
|
||||
# cancel); ``continue_`` keeps sse_consumer's finally block a no-op.
|
||||
on_disconnect=DisconnectMode.continue_,
|
||||
multitask_strategy=data.get("multitask_strategy", "reject"),
|
||||
metadata=data.get("metadata") or {},
|
||||
kwargs=data.get("kwargs") or {},
|
||||
created_at=data.get("created_at") or "",
|
||||
updated_at=data.get("updated_at") or "",
|
||||
error=data.get("error"),
|
||||
)
|
||||
|
||||
async def get_persisted(self, run_id: str) -> RunRecord | None:
|
||||
"""Load a run from the backing store (process-restart fallback).
|
||||
|
||||
The in-memory registry is wiped on every Gateway restart, but the
|
||||
``RunStore`` row survives. When :meth:`get` misses, callers fall back
|
||||
here so a stale ``joinStream``/``GET run`` can report the run's final
|
||||
status instead of 404ing. Returns ``None`` when there is no store, no
|
||||
matching row, or the lookup fails (e.g. no user context).
|
||||
"""
|
||||
if self._store is None:
|
||||
return None
|
||||
try:
|
||||
data = await self._store.get(run_id)
|
||||
except Exception:
|
||||
logger.warning("Failed to load run %s from store", run_id, exc_info=True)
|
||||
return None
|
||||
if not data:
|
||||
return None
|
||||
return self._record_from_store(data)
|
||||
|
||||
async def list_by_thread(self, thread_id: str) -> list[RunRecord]:
|
||||
"""Return all runs for a given thread, newest first."""
|
||||
async with self._lock:
|
||||
# Dict insertion order matches creation order, so reversing it gives
|
||||
# us deterministic newest-first results even when timestamps tie.
|
||||
return [r for r in self._runs.values() if r.thread_id == thread_id]
|
||||
|
||||
async def set_status(self, run_id: str, status: RunStatus, *, error: str | None = None) -> None:
|
||||
"""Transition a run to a new status."""
|
||||
async with self._lock:
|
||||
record = self._runs.get(run_id)
|
||||
if record is None:
|
||||
logger.warning("set_status called for unknown run %s", run_id)
|
||||
return
|
||||
record.status = status
|
||||
record.updated_at = _now_iso()
|
||||
if error is not None:
|
||||
record.error = error
|
||||
if self._store is not None:
|
||||
try:
|
||||
await self._store.update_status(run_id, status.value, error=error)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist status update for run %s", run_id, exc_info=True)
|
||||
logger.info("Run %s -> %s", run_id, status.value)
|
||||
|
||||
async def cancel(self, run_id: str, *, action: str = "interrupt") -> bool:
|
||||
"""Request cancellation of a run.
|
||||
|
||||
Args:
|
||||
run_id: The run ID to cancel.
|
||||
action: "interrupt" keeps checkpoint, "rollback" reverts to pre-run state.
|
||||
|
||||
Sets the abort event with the action reason and cancels the asyncio task.
|
||||
Returns ``True`` if the run was in-flight and cancellation was initiated.
|
||||
"""
|
||||
async with self._lock:
|
||||
record = self._runs.get(run_id)
|
||||
if record is None:
|
||||
return False
|
||||
if record.status not in (RunStatus.pending, RunStatus.running):
|
||||
return False
|
||||
record.abort_action = action
|
||||
record.abort_event.set()
|
||||
if record.task is not None and not record.task.done():
|
||||
record.task.cancel()
|
||||
record.status = RunStatus.interrupted
|
||||
record.updated_at = _now_iso()
|
||||
logger.info("Run %s cancelled (action=%s)", run_id, action)
|
||||
return True
|
||||
|
||||
async def create_or_reject(
|
||||
self,
|
||||
thread_id: str,
|
||||
assistant_id: str | None = None,
|
||||
*,
|
||||
on_disconnect: DisconnectMode = DisconnectMode.cancel,
|
||||
metadata: dict | None = None,
|
||||
kwargs: dict | None = None,
|
||||
multitask_strategy: str = "reject",
|
||||
model_concurrency_limit: tuple[str, int] | None = None,
|
||||
) -> RunRecord:
|
||||
"""Atomically check for inflight runs and create a new one.
|
||||
|
||||
For ``reject`` strategy, raises ``ConflictError`` if thread
|
||||
already has a pending/running run. For ``interrupt``/``rollback``,
|
||||
cancels inflight runs before creating.
|
||||
|
||||
This method holds the lock across both the check and the insert,
|
||||
eliminating the TOCTOU race in separate ``has_inflight`` + ``create``.
|
||||
"""
|
||||
run_id = str(uuid.uuid4())
|
||||
now = _now_iso()
|
||||
|
||||
_supported_strategies = ("reject", "interrupt", "rollback")
|
||||
|
||||
async with self._lock:
|
||||
if multitask_strategy not in _supported_strategies:
|
||||
raise UnsupportedStrategyError(f"Multitask strategy '{multitask_strategy}' is not yet supported. Supported strategies: {', '.join(_supported_strategies)}")
|
||||
|
||||
inflight = [r for r in self._runs.values() if r.thread_id == thread_id and r.status in (RunStatus.pending, RunStatus.running)]
|
||||
|
||||
if multitask_strategy == "reject" and inflight:
|
||||
raise ConflictError(f"Thread {thread_id} already has an active run")
|
||||
|
||||
if multitask_strategy in ("interrupt", "rollback") and inflight:
|
||||
for r in inflight:
|
||||
r.abort_action = multitask_strategy
|
||||
r.abort_event.set()
|
||||
if r.task is not None and not r.task.done():
|
||||
r.task.cancel()
|
||||
r.status = RunStatus.interrupted
|
||||
r.updated_at = now
|
||||
logger.info(
|
||||
"Cancelled %d inflight run(s) on thread %s (strategy=%s)",
|
||||
len(inflight),
|
||||
thread_id,
|
||||
multitask_strategy,
|
||||
)
|
||||
|
||||
if model_concurrency_limit is not None:
|
||||
model_name, limit = model_concurrency_limit
|
||||
active_for_model = sum(1 for r in self._runs.values() if r.status in (RunStatus.pending, RunStatus.running) and str((r.metadata or {}).get("model_name") or "").strip() == model_name)
|
||||
if active_for_model >= max(1, int(limit)):
|
||||
raise ModelConcurrencyLimitError(model_name, limit)
|
||||
|
||||
record = RunRecord(
|
||||
run_id=run_id,
|
||||
thread_id=thread_id,
|
||||
assistant_id=assistant_id,
|
||||
status=RunStatus.pending,
|
||||
on_disconnect=on_disconnect,
|
||||
multitask_strategy=multitask_strategy,
|
||||
metadata=metadata or {},
|
||||
kwargs=kwargs or {},
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
self._runs[run_id] = record
|
||||
|
||||
await self._persist_to_store(record)
|
||||
logger.info("Run created: run_id=%s thread_id=%s", run_id, thread_id)
|
||||
return record
|
||||
|
||||
async def has_inflight(self, thread_id: str) -> bool:
|
||||
"""Return ``True`` if *thread_id* has a pending or running run."""
|
||||
async with self._lock:
|
||||
return any(r.thread_id == thread_id and r.status in (RunStatus.pending, RunStatus.running) for r in self._runs.values())
|
||||
|
||||
async def list_active(self) -> list[RunRecord]:
|
||||
"""Return every in-flight run (status pending/running), oldest first.
|
||||
|
||||
Powers the admin concurrency monitor: each record represents one live
|
||||
call into the model runtime (interactive chat, agent chat, roundtable
|
||||
seat, …), so ``len(list_active())`` is the current concurrent-call count
|
||||
and each record can be cancelled to relieve model pressure. Completed
|
||||
records linger in ``_runs`` until :meth:`cleanup`, so we filter by status
|
||||
rather than returning the whole registry.
|
||||
"""
|
||||
async with self._lock:
|
||||
# Dict insertion order == creation order, so this is oldest-first.
|
||||
return [r for r in self._runs.values() if r.status in (RunStatus.pending, RunStatus.running)]
|
||||
|
||||
async def cleanup(self, run_id: str, *, delay: float = 300) -> None:
|
||||
"""Remove a run record after an optional delay."""
|
||||
if delay > 0:
|
||||
await asyncio.sleep(delay)
|
||||
async with self._lock:
|
||||
self._runs.pop(run_id, None)
|
||||
logger.debug("Run record %s cleaned up", run_id)
|
||||
|
||||
|
||||
class ConflictError(Exception):
|
||||
"""Raised when multitask_strategy=reject and thread has inflight runs."""
|
||||
|
||||
|
||||
class ModelConcurrencyLimitError(Exception):
|
||||
"""Raised when a model-specific in-flight run limit has been reached."""
|
||||
|
||||
def __init__(self, model_name: str, limit: int) -> None:
|
||||
self.model_name = model_name
|
||||
self.limit = limit
|
||||
super().__init__(f"Model {model_name} has reached its concurrency limit")
|
||||
|
||||
|
||||
class UnsupportedStrategyError(Exception):
|
||||
"""Raised when a multitask_strategy value is not yet implemented."""
|
||||
@@ -15,10 +15,10 @@ internal checkpoint callbacks that are not exposed in the Python public API.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import copy
|
||||
import inspect
|
||||
import logging
|
||||
import asyncio
|
||||
import copy
|
||||
import inspect
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from functools import lru_cache
|
||||
from typing import TYPE_CHECKING, Any, Literal, cast
|
||||
@@ -28,17 +28,31 @@ from langgraph.checkpoint.base import empty_checkpoint
|
||||
if TYPE_CHECKING:
|
||||
from langchain_core.messages import HumanMessage
|
||||
|
||||
from deerflow.config.app_config import AppConfig
|
||||
from deerflow.runtime.serialization import serialize
|
||||
from deerflow.runtime.stream_bridge import StreamBridge
|
||||
from deerflow.agents.middlewares.title_middleware import (
|
||||
finalize_thread_title,
|
||||
log_title_debug,
|
||||
)
|
||||
from deerflow.config.app_config import AppConfig
|
||||
from deerflow.runtime.serialization import serialize
|
||||
from deerflow.runtime.stream_bridge import StreamBridge
|
||||
|
||||
from .manager import RunManager, RunRecord
|
||||
from .schemas import RunStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Valid stream_mode values for LangGraph's graph.astream()
|
||||
_VALID_LG_MODES = {"values", "updates", "checkpoints", "tasks", "debug", "messages", "custom"}
|
||||
_VALID_LG_MODES = {"values", "updates", "checkpoints", "tasks", "debug", "messages", "custom"}
|
||||
|
||||
|
||||
def _log_background_title_finalize_failure(thread_id: str):
|
||||
def _done(task: asyncio.Task[bool]) -> None:
|
||||
try:
|
||||
task.result()
|
||||
except Exception:
|
||||
logger.debug("Failed to finalize background title for thread %s", thread_id, exc_info=True)
|
||||
|
||||
return _done
|
||||
|
||||
|
||||
def _build_runtime_context(
|
||||
@@ -400,18 +414,47 @@ async def run_agent(
|
||||
except Exception:
|
||||
logger.warning("Failed to persist run completion for %s (non-fatal)", run_id, exc_info=True)
|
||||
|
||||
# Sync title from checkpoint to threads_meta.display_name
|
||||
if checkpointer is not None and thread_store is not None:
|
||||
try:
|
||||
ckpt_config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
|
||||
ckpt_tuple = await checkpointer.aget_tuple(ckpt_config)
|
||||
if ckpt_tuple is not None:
|
||||
ckpt = getattr(ckpt_tuple, "checkpoint", {}) or {}
|
||||
title = ckpt.get("channel_values", {}).get("title")
|
||||
if title:
|
||||
await thread_store.update_display_name(thread_id, title)
|
||||
except Exception:
|
||||
logger.debug("Failed to sync title for thread %s (non-fatal)", thread_id)
|
||||
# Sync title from checkpoint to threads_meta.display_name
|
||||
if checkpointer is not None and thread_store is not None:
|
||||
try:
|
||||
ckpt_config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
|
||||
ckpt_tuple = await checkpointer.aget_tuple(ckpt_config)
|
||||
if ckpt_tuple is not None:
|
||||
ckpt = getattr(ckpt_tuple, "checkpoint", {}) or {}
|
||||
channel_values = ckpt.get("channel_values", {}) or {}
|
||||
title = channel_values.get("title")
|
||||
if title:
|
||||
await thread_store.update_display_name(thread_id, title)
|
||||
if (
|
||||
record.status == RunStatus.success
|
||||
and channel_values.get("title_provisional") is True
|
||||
):
|
||||
log_title_debug(
|
||||
"Run %s: scheduling background title generation for thread %s",
|
||||
run_id,
|
||||
thread_id,
|
||||
)
|
||||
title_task = asyncio.create_task(
|
||||
finalize_thread_title(
|
||||
checkpointer=checkpointer,
|
||||
thread_store=thread_store,
|
||||
thread_id=thread_id,
|
||||
source_state=copy.deepcopy(channel_values),
|
||||
app_config=ctx.app_config,
|
||||
),
|
||||
name=f"title-finalize:{thread_id}",
|
||||
)
|
||||
title_task.add_done_callback(_log_background_title_finalize_failure(thread_id))
|
||||
elif record.status == RunStatus.success:
|
||||
log_title_debug(
|
||||
"Run %s: background title generation not scheduled for thread %s because title_provisional=%r title=%r",
|
||||
run_id,
|
||||
thread_id,
|
||||
channel_values.get("title_provisional"),
|
||||
channel_values.get("title"),
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Failed to sync title for thread %s (non-fatal)", thread_id)
|
||||
|
||||
# Update threads_meta status based on run outcome
|
||||
if thread_store is not None:
|
||||
|
||||
+1
-1
@@ -22,10 +22,10 @@ from langgraph.checkpoint.base import empty_checkpoint
|
||||
from deerflow.config.agents_config import AGENT_ID_PATTERN, load_agent_config, load_agent_soul
|
||||
from deerflow.config.paths import get_paths
|
||||
from deerflow.persistence.scheduled_tasks import ScheduledTaskStore
|
||||
from deerflow.runtime import serialize_channel_values
|
||||
from deerflow.runtime.scheduler import html_page as html_page_utils
|
||||
from deerflow.runtime.scheduler import template_fill
|
||||
from deerflow.runtime.scheduler.element import ElementDeliveryClient
|
||||
from deerflow.runtime.serialization import serialize_channel_values
|
||||
from deerflow.runtime.user_context import reset_current_user, set_current_user
|
||||
from deerflow.utils.time import now_iso
|
||||
|
||||
|
||||
+63
@@ -267,6 +267,69 @@ TEMPLATES: tuple[TemplateSpec, ...] = (
|
||||
file="no18/no18.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no19",
|
||||
name="no.19",
|
||||
description="",
|
||||
file="no19/no.19.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no20",
|
||||
name="no.20",
|
||||
description="",
|
||||
file="no20/no.20.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no21",
|
||||
name="no.21",
|
||||
description="",
|
||||
file="no21/no.21.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no22",
|
||||
name="no.22",
|
||||
description="",
|
||||
file="no22/no.22.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no23",
|
||||
name="no.23",
|
||||
description="",
|
||||
file="no23/no.23.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no24",
|
||||
name="no.24",
|
||||
description="",
|
||||
file="no24/no.24.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no25",
|
||||
name="no.25",
|
||||
description="",
|
||||
file="no25/no.25.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no26",
|
||||
name="no.26",
|
||||
description="",
|
||||
file="no26/no.26.html",
|
||||
convertible=False,
|
||||
),
|
||||
TemplateSpec(
|
||||
id="no27",
|
||||
name="no.27",
|
||||
description="",
|
||||
file="no27/no.27.html",
|
||||
convertible=False,
|
||||
),
|
||||
)
|
||||
|
||||
_TEMPLATES_BY_ID: dict[str, TemplateSpec] = {t.id: t for t in TEMPLATES}
|
||||
|
||||
+1764
File diff suppressed because it is too large.
Load diff
+4252
File diff suppressed because it is too large.
Load diff
+3833
File diff suppressed because it is too large.
Load diff
+3484
File diff suppressed because it is too large.
Load diff
+2943
File diff suppressed because it is too large.
Load diff
+381
@@ -0,0 +1,381 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>日本涉台介入风险叙事专题研究系统</title>
|
||||
<style>
|
||||
:root {
|
||||
--bg: #8f1f25;
|
||||
--bg2: #b8323a;
|
||||
--panel: #fff7f3;
|
||||
--card: #ffffff;
|
||||
--ink: #241616;
|
||||
--muted: #76605d;
|
||||
--line: rgba(120, 29, 35, .18);
|
||||
--red: #b7202e;
|
||||
--red-dark: #7b121b;
|
||||
--gold: #f3c969;
|
||||
--soft: #fff2eb;
|
||||
--green: #18724d;
|
||||
--orange: #b35b13;
|
||||
--blue: #275b91;
|
||||
--shadow: 0 18px 50px rgba(58, 9, 15, .18);
|
||||
--radius: 18px;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "Microsoft YaHei", "PingFang SC", Arial, sans-serif;
|
||||
background: radial-gradient(circle at 20% 0%, #d94c4c 0, transparent 24%), linear-gradient(140deg, var(--bg), #641018 70%);
|
||||
color: var(--ink);
|
||||
min-height: 100vh;
|
||||
}
|
||||
.app { max-width: 1480px; margin: 0 auto; padding: 28px; }
|
||||
.hero {
|
||||
color: #fff;
|
||||
padding: 24px 26px 22px;
|
||||
border: 1px solid rgba(255,255,255,.18);
|
||||
background: linear-gradient(135deg, rgba(255,255,255,.14), rgba(255,255,255,.06));
|
||||
backdrop-filter: blur(10px);
|
||||
border-radius: 26px;
|
||||
box-shadow: var(--shadow);
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.hero::after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
width: 420px; height: 420px;
|
||||
border: 1px solid rgba(255,255,255,.12);
|
||||
border-radius: 50%;
|
||||
right: -160px; top: -220px;
|
||||
}
|
||||
.eyebrow { font-size: 13px; letter-spacing: .18em; opacity: .86; margin-bottom: 9px; }
|
||||
h1 { margin: 0; font-size: clamp(28px, 4vw, 48px); line-height: 1.15; }
|
||||
.subtitle { max-width: 920px; margin-top: 12px; color: rgba(255,255,255,.88); line-height: 1.75; }
|
||||
.hero-meta { display: flex; gap: 12px; flex-wrap: wrap; margin-top: 18px; }
|
||||
.pill { display:inline-flex; align-items:center; gap:6px; padding: 7px 11px; border-radius: 999px; background: rgba(255,255,255,.16); color: #fff; border: 1px solid rgba(255,255,255,.18); font-size: 13px; }
|
||||
|
||||
.toolbar {
|
||||
margin: 20px 0 14px;
|
||||
background: var(--panel);
|
||||
border-radius: 22px;
|
||||
padding: 14px;
|
||||
display: grid;
|
||||
grid-template-columns: 1.6fr .85fr .75fr .75fr;
|
||||
gap: 12px;
|
||||
box-shadow: var(--shadow);
|
||||
border: 1px solid rgba(255,255,255,.42);
|
||||
}
|
||||
input, select {
|
||||
width: 100%;
|
||||
border: 1px solid var(--line);
|
||||
background: #fff;
|
||||
border-radius: 14px;
|
||||
padding: 12px 13px;
|
||||
font-size: 14px;
|
||||
color: var(--ink);
|
||||
outline: none;
|
||||
}
|
||||
input:focus, select:focus { border-color: rgba(183,32,46,.56); box-shadow: 0 0 0 3px rgba(183,32,46,.1); }
|
||||
.tabs {
|
||||
display: flex; flex-wrap: wrap; gap: 9px;
|
||||
margin-bottom: 18px;
|
||||
}
|
||||
.tab-btn {
|
||||
border: 0;
|
||||
background: rgba(255,255,255,.18);
|
||||
color: #fff;
|
||||
border: 1px solid rgba(255,255,255,.2);
|
||||
padding: 10px 13px;
|
||||
border-radius: 999px;
|
||||
cursor: pointer;
|
||||
font-weight: 700;
|
||||
transition: .18s ease;
|
||||
}
|
||||
.tab-btn.active { background: #fff; color: var(--red-dark); box-shadow: 0 10px 26px rgba(0,0,0,.13); }
|
||||
.tab-btn:hover { transform: translateY(-1px); }
|
||||
|
||||
.panel {
|
||||
background: var(--panel);
|
||||
border-radius: 26px;
|
||||
padding: 20px;
|
||||
box-shadow: var(--shadow);
|
||||
border: 1px solid rgba(255,255,255,.44);
|
||||
min-height: 520px;
|
||||
}
|
||||
.tab-head { display:flex; justify-content:space-between; align-items:flex-start; gap:18px; margin-bottom:16px; }
|
||||
.tab-title { margin: 0; font-size: 25px; color: var(--red-dark); }
|
||||
.tab-desc { margin: 6px 0 0; color: var(--muted); line-height: 1.6; }
|
||||
.count-badge { flex: 0 0 auto; padding: 8px 11px; border-radius: 999px; background:#fff; color: var(--red-dark); border:1px solid var(--line); font-weight:800; }
|
||||
|
||||
.summary-grid { display:grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 14px; }
|
||||
.summary-card {
|
||||
background: var(--card);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 18px;
|
||||
padding: 18px;
|
||||
min-height: 144px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.summary-card::after { content:""; position:absolute; right:-38px; top:-42px; width: 110px; height:110px; border-radius:50%; background: rgba(183,32,46,.08); }
|
||||
.summary-card h3 { margin:0 0 8px; color:var(--red-dark); font-size:15px; }
|
||||
.summary-card .value { font-size: 25px; font-weight: 900; margin-bottom: 9px; color: #2b1114; }
|
||||
.summary-card p { margin:0; color:var(--muted); line-height: 1.62; font-size: 14px; }
|
||||
|
||||
.overview-grid { display:grid; grid-template-columns: 1.1fr .9fr; gap: 14px; margin-top: 14px; }
|
||||
.box { background:#fff; border:1px solid var(--line); border-radius:18px; padding:16px; }
|
||||
.box h3 { margin:0 0 12px; color: var(--red-dark); }
|
||||
.bar-row { display:grid; grid-template-columns: 105px 1fr 36px; align-items:center; gap:8px; margin: 9px 0; font-size:13px; color: var(--muted); }
|
||||
.bar { height: 10px; border-radius: 999px; background: #f4ded8; overflow: hidden; }
|
||||
.bar > span { display:block; height:100%; border-radius:999px; background: linear-gradient(90deg, var(--red), var(--gold)); }
|
||||
.risk-list { display:grid; gap:10px; }
|
||||
.risk-mini { border:1px solid var(--line); border-radius:14px; padding:11px; background:#fffdfb; cursor:pointer; }
|
||||
.risk-mini strong { color: var(--red-dark); }
|
||||
|
||||
.cards { display:grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 14px; }
|
||||
.card {
|
||||
background: #fff;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 18px;
|
||||
padding: 16px;
|
||||
display:flex; flex-direction:column; gap: 10px;
|
||||
min-height: 248px;
|
||||
transition: .18s ease;
|
||||
}
|
||||
.card:hover { transform: translateY(-2px); box-shadow: 0 12px 28px rgba(94, 20, 24, .12); }
|
||||
.card-top { display:flex; justify-content:space-between; align-items:flex-start; gap:12px; }
|
||||
.card-id { font-size:12px; color:#fff; background: var(--red-dark); border-radius:999px; padding:4px 8px; font-weight: 800; }
|
||||
.risk { font-size:12px; border-radius:999px; padding:5px 8px; font-weight: 800; white-space: nowrap; }
|
||||
.risk.xh { background: #ffe1de; color:#9b111c; }
|
||||
.risk.h { background: #fff0d5; color:#9a4e00; }
|
||||
.risk.m { background: #e9f2ff; color:#24527d; }
|
||||
.card h3 { margin:0; color:var(--red-dark); font-size:17px; line-height:1.35; }
|
||||
.meta { display:flex; gap:6px; flex-wrap:wrap; }
|
||||
.tag { font-size:12px; color:var(--red-dark); background: var(--soft); border:1px solid rgba(183,32,46,.15); border-radius:999px; padding:4px 8px; }
|
||||
.card p { margin:0; color:var(--muted); line-height: 1.62; font-size: 14px; }
|
||||
.card-footer { margin-top:auto; display:flex; justify-content:space-between; align-items:center; gap:12px; }
|
||||
.evidence { font-size:12px; color:#5e4d49; }
|
||||
.btn {
|
||||
background: var(--red);
|
||||
color:#fff;
|
||||
border:0;
|
||||
border-radius:12px;
|
||||
padding:9px 12px;
|
||||
cursor:pointer;
|
||||
font-weight:800;
|
||||
box-shadow: 0 8px 18px rgba(183,32,46,.22);
|
||||
}
|
||||
.btn.secondary { background:#fff; color:var(--red-dark); border:1px solid var(--line); box-shadow:none; }
|
||||
.empty { padding: 38px; text-align:center; color:var(--muted); background:#fff; border:1px dashed var(--line); border-radius:18px; }
|
||||
|
||||
.keyword-wrap { display:grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap:14px; margin-top:16px; }
|
||||
.keyword-group { background:#fff; border:1px solid var(--line); border-radius:18px; padding:15px; }
|
||||
.keyword-group h3 { margin:0 0 10px; color: var(--red-dark); }
|
||||
.kw { display:inline-block; margin:5px; padding:7px 10px; background:var(--soft); border:1px solid rgba(183,32,46,.18); color:var(--red-dark); border-radius:999px; font-size:13px; }
|
||||
|
||||
.modal-backdrop { display:none; position:fixed; inset:0; background:rgba(45, 7, 10, .62); z-index:1000; padding: 28px; overflow:auto; }
|
||||
.modal-backdrop.open { display:block; }
|
||||
.modal { max-width: 980px; margin: 0 auto; background:#fffaf7; border-radius: 24px; border:1px solid rgba(255,255,255,.45); box-shadow: 0 30px 90px rgba(0,0,0,.34); overflow:hidden; }
|
||||
.modal-head { background: linear-gradient(135deg, var(--red-dark), var(--red)); color:#fff; padding: 20px 22px; display:flex; justify-content:space-between; gap:16px; align-items:flex-start; }
|
||||
.modal-head h2 { margin:0; font-size:24px; }
|
||||
.close { background:rgba(255,255,255,.16); border:1px solid rgba(255,255,255,.25); color:#fff; border-radius:12px; font-size:22px; cursor:pointer; width:42px; height:38px; }
|
||||
.modal-body { padding: 20px 22px 24px; }
|
||||
.detail-grid { display:grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap:10px; margin-bottom:16px; }
|
||||
.detail-cell { background:#fff; border:1px solid var(--line); border-radius:14px; padding:10px; }
|
||||
.detail-cell span { display:block; font-size:12px; color:var(--muted); margin-bottom:4px; }
|
||||
.detail-cell strong { color:var(--red-dark); }
|
||||
.section { background:#fff; border:1px solid var(--line); border-radius:16px; padding:15px; margin:12px 0; }
|
||||
.section h3 { margin:0 0 8px; color:var(--red-dark); }
|
||||
.section p { margin:0; line-height:1.8; color:#3b2929; }
|
||||
|
||||
.footer-note { color:rgba(255,255,255,.78); text-align:center; padding: 16px 0 4px; font-size:13px; }
|
||||
@media (max-width: 1100px) {
|
||||
.toolbar { grid-template-columns: 1fr 1fr; }
|
||||
.summary-grid, .cards { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.overview-grid { grid-template-columns: 1fr; }
|
||||
.detail-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
}
|
||||
@media (max-width: 720px) {
|
||||
.app { padding: 14px; }
|
||||
.toolbar { grid-template-columns: 1fr; }
|
||||
.summary-grid, .cards, .keyword-wrap { grid-template-columns: 1fr; }
|
||||
.tab-head { flex-direction:column; }
|
||||
.detail-grid { grid-template-columns: 1fr; }
|
||||
.modal-backdrop { padding: 12px; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="app">
|
||||
<header class="hero">
|
||||
<div class="eyebrow">NARRATIVE RISK MONITORING DATABASE</div>
|
||||
<h1>日本涉台介入风险叙事专题研究系统</h1>
|
||||
<div class="subtitle">围绕“介入战争日本祸及自身”叙事的事件、人物、组织、制度、地方、媒体、经济与民意数据库</div>
|
||||
<div class="hero-meta">
|
||||
<span class="pill">版本:1.0</span>
|
||||
<span class="pill">更新:2026-06-30</span>
|
||||
<span class="pill">离线可用 · 内嵌数据</span>
|
||||
<span class="pill">Tab切换 · 搜索筛选 · 弹框详情</span>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<section class="toolbar">
|
||||
<input id="q" placeholder="搜索关键词:高市、先岛、存立危机、冲绳、旅游、认知战……" />
|
||||
<select id="narrativeFilter"><option value="">全部叙事方向</option></select>
|
||||
<select id="riskFilter"><option value="">全部风险等级</option><option value="极高">极高</option><option value="高">高</option><option value="中">中</option><option value="中低">中低</option><option value="低">低</option></select>
|
||||
<select id="evidenceFilter"><option value="">全部证据状态</option><option value="已核实">已核实</option><option value="持续更新">持续更新</option><option value="待采集">待采集</option></select>
|
||||
</section>
|
||||
|
||||
<nav class="tabs" id="tabs"></nav>
|
||||
<main class="panel" id="content"></main>
|
||||
<div class="footer-note">本页面为研究型结构化展示,不替代原始文件核验;高风险条目应回溯国会记录、政府文件、机构公告和媒体原文。</div>
|
||||
</div>
|
||||
|
||||
<div class="modal-backdrop" id="modalBackdrop" onclick="backdropClose(event)">
|
||||
<div class="modal" role="dialog" aria-modal="true">
|
||||
<div class="modal-head">
|
||||
<div>
|
||||
<div id="modalMeta" class="eyebrow"></div>
|
||||
<h2 id="modalTitle"></h2>
|
||||
</div>
|
||||
<button class="close" onclick="closeModal()">×</button>
|
||||
</div>
|
||||
<div class="modal-body" id="modalBody"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const DATA = {"project": {"title": "日本涉台介入风险叙事专题研究系统", "subtitle": "围绕“介入战争日本祸及自身”叙事的事件、人物、组织、制度、地方、媒体、经济与民意数据库", "version": "1.0", "language": "zh-CN", "last_updated": "2026-06-30", "description": "本数据用于展示日本方向涉台介入风险叙事中的事件、人物、组织、制度、地方、媒体、经济、民意与竞争叙事。"}, "tabs": [{"id": "overview", "name": "总览仪表盘", "dataset": null, "description": "展示专题总风险、叙事分布、热点地区、重点人物和最新事件。"}, {"id": "timeline", "name": "事件时间线", "dataset": "events_timeline", "description": "展示2014年以来日本涉台安全政策、政要表态、地方避难和中日冲突节点。"}, {"id": "actors_orgs", "name": "人物组织", "dataset": ["actors_japan", "organizations"], "description": "展示日本政府、政党、地方政治、专家、媒体和社会组织。"}, {"id": "security_laws", "name": "法律制度", "dataset": "security_laws", "description": "展示存立危机、重要影响、国民保护、日美安保等制度路径。"}, {"id": "local_frontline", "name": "地方前线", "dataset": "local_frontline", "description": "展示冲绳、先岛、石垣、宫古、与那国、九州、山口等地方风险。"}, {"id": "antiwar_networks", "name": "反战护宪网络", "dataset": "antiwar_networks", "description": "展示九条之会、日弁连、冲绳和平运动、地方反导弹组织和工会网络。"}, {"id": "conservative_backlash", "name": "保守反弹", "dataset": "conservative_backlash", "description": "展示威慑、防卫、中国认知战、台湾伙伴等竞争叙事。"}, {"id": "media_platform", "name": "媒体平台", "dataset": "media_platform_samples", "description": "展示全国媒体、地方媒体、YouTube、X、Yahoo、note等传播样本。"}, {"id": "economic_spillover", "name": "经济外溢", "dataset": "economic_spillover", "description": "展示旅游、航空、半导体、供应链、出口管制等风险。"}, {"id": "public_opinion", "name": "民意受众", "dataset": "public_opinion_youth", "description": "展示台湾有事、防卫费、修宪、青年、都市无党派和冲绳地方态度。"}, {"id": "japan_taiwan", "name": "日台关系", "dataset": "japan_taiwan_relations", "description": "展示日台实务窗口、亲台议员网络、地方交流、半导体合作和智库推演。"}, {"id": "corpus_keywords", "name": "语料词库", "dataset": "source_corpus_keywords", "description": "展示原始语料来源、日文关键词、平台监测词和证据状态。"}], "dashboard": {"summary_cards": [{"title": "核心叙事", "value": "介入战争日本祸及自身", "description": "围绕日本涉台介入可能引发战争卷入、地方牺牲、经济反制和社会撕裂的风险叙事。"}, {"title": "数据条目", "value": "164", "description": "覆盖事件、人物、组织、制度、地方、媒体、经济、民意和日台关系。"}, {"title": "高风险地区", "value": "冲绳、先岛、九州", "description": "冲绳本岛、石垣、宫古、与那国、九州和山口是地方风险最集中的区域。"}, {"title": "核心法律入口", "value": "存立危机事态", "description": "台湾有事是否触发日本集体自卫权,是舆论和国会争议核心。"}, {"title": "主要竞争叙事", "value": "威慑才能防止战争", "description": "保守派和安保界常用威慑、防卫、中国威胁等逻辑反击反介入叙事。"}, {"title": "总体风险", "value": "86 / 100", "description": "综合政治触发、地方影响、经济外溢、媒体扩散和反弹强度评估。"}], "narrative_distribution": [{"label": "地方牺牲", "count": 38}, {"label": "护宪反战", "count": 22}, {"label": "威慑防战", "count": 20}, {"label": "经济代价", "count": 18}, {"label": "中性解释", "count": 17}, {"label": "台湾伙伴", "count": 15}, {"label": "战争卷入", "count": 12}, {"label": "专业治理", "count": 12}, {"label": "中国胁迫", "count": 6}, {"label": "中国威胁", "count": 4}], "hotspots": [{"location": "日本", "count": 42}, {"location": "东京", "count": 15}, {"location": "冲绳", "count": 12}, {"location": "日本全国", "count": 9}, {"location": "宫古岛", "count": 5}, {"location": "华盛顿", "count": 4}, {"location": "与那国", "count": 4}, {"location": "石垣", "count": 3}, {"location": "九州/山口", "count": 3}, {"location": "熊本", "count": 3}, {"location": "先岛/九州/山口", "count": 2}, {"location": "中日之间", "count": 2}], "top_risks": [{"id": "E016", "title_cn": "高市早苗称台湾有事可能构成存立危机事态", "category": "首相表态", "summary": "高市早苗在国会答辩中将台湾有事与存立危机事态直接相连。", "detail": "该事件将保守派长期话语推入首相级官Line truncated
|
||||
const state = { activeTab: 'overview' };
|
||||
|
||||
const tabsEl = document.getElementById('tabs');
|
||||
const contentEl = document.getElementById('content');
|
||||
const qEl = document.getElementById('q');
|
||||
const narrativeEl = document.getElementById('narrativeFilter');
|
||||
const riskEl = document.getElementById('riskFilter');
|
||||
const evidenceEl = document.getElementById('evidenceFilter');
|
||||
|
||||
function init() {
|
||||
Object.values(DATA.narrative_labels).forEach(label => {
|
||||
const opt = document.createElement('option'); opt.value = label; opt.textContent = label; narrativeEl.appendChild(opt);
|
||||
});
|
||||
renderTabs();
|
||||
render();
|
||||
[qEl, narrativeEl, riskEl, evidenceEl].forEach(el => el.addEventListener('input', render));
|
||||
}
|
||||
|
||||
function renderTabs() {
|
||||
tabsEl.innerHTML = '';
|
||||
DATA.tabs.forEach(t => {
|
||||
const btn = document.createElement('button');
|
||||
btn.className = 'tab-btn' + (state.activeTab === t.id ? ' active' : '');
|
||||
btn.textContent = t.name;
|
||||
btn.onclick = () => { state.activeTab = t.id; qEl.value=''; renderTabs(); render(); };
|
||||
tabsEl.appendChild(btn);
|
||||
});
|
||||
}
|
||||
|
||||
function getActiveTab() { return DATA.tabs.find(t => t.id === state.activeTab); }
|
||||
function getItems(tab) {
|
||||
if (!tab || !tab.dataset) return [];
|
||||
if (Array.isArray(tab.dataset)) return tab.dataset.flatMap(k => DATA.datasets[k] || []);
|
||||
return DATA.datasets[tab.dataset] || [];
|
||||
}
|
||||
function textOf(x) {
|
||||
return [x.id, x.title_cn, x.category, x.summary, x.detail, x.location, x.narrative_label, ...(x.tags||[])].join(' ').toLowerCase();
|
||||
}
|
||||
function filtered(items) {
|
||||
const q = qEl.value.trim().toLowerCase();
|
||||
const n = narrativeEl.value;
|
||||
const r = riskEl.value;
|
||||
const e = evidenceEl.value;
|
||||
return items.filter(x => (!q || textOf(x).includes(q)) && (!n || x.narrative_label === n) && (!r || x.risk_level === r) && (!e || x.evidence_status === e));
|
||||
}
|
||||
function riskClass(level) { return level === '极高' ? 'xh' : level === '高' ? 'h' : 'm'; }
|
||||
|
||||
function render() {
|
||||
const tab = getActiveTab();
|
||||
if (state.activeTab === 'overview') return renderOverview();
|
||||
const items = filtered(getItems(tab));
|
||||
const keywordBlock = state.activeTab === 'corpus_keywords' ? renderKeywords() : '';
|
||||
contentEl.innerHTML = `
|
||||
<div class="tab-head">
|
||||
<div><h2 class="tab-title">${tab.name}</h2><p class="tab-desc">${tab.description}</p></div>
|
||||
<div class="count-badge">${items.length} 条</div>
|
||||
</div>
|
||||
${items.length ? `<div class="cards">${items.map(card).join('')}</div>` : '<div class="empty">没有符合当前筛选条件的数据。</div>'}
|
||||
${keywordBlock}
|
||||
`;
|
||||
}
|
||||
|
||||
function renderOverview() {
|
||||
const d = DATA.dashboard;
|
||||
const maxNarr = Math.max(...d.narrative_distribution.map(x => x.count));
|
||||
const maxHot = Math.max(...d.hotspots.map(x => x.count));
|
||||
contentEl.innerHTML = `
|
||||
<div class="tab-head">
|
||||
<div><h2 class="tab-title">总览仪表盘</h2><p class="tab-desc">展示专题总风险、叙事分布、热点地区、重点人物和最新事件。可通过上方搜索框对全库进行粗略检索。</p></div>
|
||||
<div class="count-badge">${Object.values(DATA.datasets).filter(Array.isArray).flat().length - DATA.datasets.keywords.length} 条</div>
|
||||
</div>
|
||||
<div class="summary-grid">${d.summary_cards.map(c => `<div class="summary-card"><h3>${c.title}</h3><div class="value">${c.value}</div><p>${c.description}</p></div>`).join('')}</div>
|
||||
<div class="overview-grid">
|
||||
<div class="box"><h3>叙事分布</h3>${d.narrative_distribution.map(x => `<div class="bar-row"><span>${x.label}</span><div class="bar"><span style="width:${Math.max(8, x.count/maxNarr*100)}%"></span></div><b>${x.count}</b></div>`).join('')}</div>
|
||||
<div class="box"><h3>热点地区</h3>${d.hotspots.slice(0,10).map(x => `<div class="bar-row"><span>${x.location}</span><div class="bar"><span style="width:${Math.max(8, x.count/maxHot*100)}%"></span></div><b>${x.count}</b></div>`).join('')}</div>
|
||||
</div>
|
||||
<div class="overview-grid">
|
||||
<div class="box"><h3>高风险条目</h3><div class="risk-list">${d.top_risks.slice(0,6).map(x => `<div class="risk-mini" onclick="openModalById('${x.id}')"><strong>${x.title_cn}</strong><br><span>${x.category} · 风险 ${x.risk_score} · ${x.narrative_label}</span></div>`).join('')}</div></div>
|
||||
<div class="box"><h3>主要风险驱动</h3><div class="meta">${d.risk_drivers.map(x => `<span class="tag">${x}</span>`).join('')}</div><div class="section"><h3>页面使用建议</h3><p>先从“事件时间线”定位触发节点,再到“法律制度”查看制度路径;涉及地方影响时进入“地方前线”和“反战护宪网络”;涉及传播扩散时进入“媒体平台”;涉及成本量化时进入“经济外溢”和“民意受众”。</p></div></div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
function card(x) {
|
||||
return `<article class="card">
|
||||
<div class="card-top"><span class="card-id">${x.id}</span><span class="risk ${riskClass(x.risk_level)}">${x.risk_level} · ${x.risk_score}</span></div>
|
||||
<h3>${x.title_cn}</h3>
|
||||
<div class="meta"><span class="tag">${x.category}</span><span class="tag">${x.narrative_label}</span><span class="tag">${x.location || '—'}</span></div>
|
||||
<p>${x.summary}</p>
|
||||
<div class="meta">${(x.tags || []).slice(0,4).map(t => `<span class="tag">${t}</span>`).join('')}</div>
|
||||
<div class="card-footer"><span class="evidence">证据:${x.evidence_status} · 可信度${x.confidence_level}</span><button class="btn" onclick="openModalById('${x.id}')">查看详情</button></div>
|
||||
</article>`;
|
||||
}
|
||||
|
||||
function renderKeywords() {
|
||||
return `<div class="keyword-wrap">${DATA.datasets.keywords.map(g => `<div class="keyword-group"><h3>${g.group}</h3>${g.terms.map(t => `<span class="kw">${t}</span>`).join('')}</div>`).join('')}</div>`;
|
||||
}
|
||||
|
||||
function allItems() { return Object.entries(DATA.datasets).filter(([k]) => k !== 'keywords').flatMap(([k,v]) => Array.isArray(v) ? v : []); }
|
||||
function openModalById(id) { const x = allItems().find(i => i.id === id); if (x) openModal(x); }
|
||||
function openModal(x) {
|
||||
document.getElementById('modalMeta').textContent = `${x.id} · ${x.category} · ${x.narrative_label}`;
|
||||
document.getElementById('modalTitle').textContent = x.title_cn;
|
||||
const metrics = x.metrics ? `<div class="section"><h3>关键指标</h3><p>${Object.entries(x.metrics).map(([k,v]) => `${k}:${v}`).join(';')}</p></div>` : '';
|
||||
document.getElementById('modalBody').innerHTML = `
|
||||
<div class="detail-grid">
|
||||
<div class="detail-cell"><span>风险等级</span><strong>${x.risk_level} · ${x.risk_score}</strong></div>
|
||||
<div class="detail-cell"><span>叙事方向</span><strong>${x.narrative_label}</strong></div>
|
||||
<div class="detail-cell"><span>时间</span><strong>${x.date || '持续'}</strong></div>
|
||||
<div class="detail-cell"><span>地点</span><strong>${x.location || '—'}</strong></div>
|
||||
<div class="detail-cell"><span>证据状态</span><strong>${x.evidence_status}</strong></div>
|
||||
<div class="detail-cell"><span>可信度</span><strong>${x.confidence_level}</strong></div>
|
||||
<div class="detail-cell"><span>类别</span><strong>${x.category}</strong></div>
|
||||
<div class="detail-cell"><span>编号</span><strong>${x.id}</strong></div>
|
||||
</div>
|
||||
<div class="section"><h3>摘要</h3><p>${x.summary}</p></div>
|
||||
<div class="section"><h3>详情解读</h3><p>${x.detail}</p></div>
|
||||
${metrics}
|
||||
<div class="section"><h3>关联标签</h3><p>${(x.tags || []).map(t => `#${t}`).join(' ')}</p></div>
|
||||
<div class="section"><h3>展示建议</h3><p>该条目可与同类事件、相关人物组织、地方节点、法律制度和媒体平台样本联动。若用于二级页面,建议补充时间线、争议双方观点、原始语料和平台传播样本。</p></div>
|
||||
`;
|
||||
document.getElementById('modalBackdrop').classList.add('open');
|
||||
}
|
||||
function closeModal() { document.getElementById('modalBackdrop').classList.remove('open'); }
|
||||
function backdropClose(e) { if (e.target.id === 'modalBackdrop') closeModal(); }
|
||||
window.addEventListener('keydown', e => { if(e.key === 'Escape') closeModal(); });
|
||||
init();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
+3623
File diff suppressed because it is too large.
Load diff
+1432
File diff suppressed because it is too large.
Load diff
+98
@@ -0,0 +1,98 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>菲律宾“不做美国棋子”叙事专题数据库 v0.2</title>
|
||||
<style>
|
||||
:root{--bg:#f7f1ed;--card:#fffdfb;--ink:#291817;--muted:#7a6764;--line:#eadbd4;--red:#a23434;--red2:#6f1f1f;--blue:#244b74;--blue2:#eaf2fb;--gold:#9f6a1a;--green:#477052;--shadow:0 18px 50px rgba(51,20,14,.09)}
|
||||
*{box-sizing:border-box} body{margin:0;background:linear-gradient(180deg,#f6eee9 0,#fbf7f3 100%);color:var(--ink);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","PingFang SC","Microsoft YaHei",Arial,sans-serif;line-height:1.55}
|
||||
header{padding:26px 30px 18px;background:linear-gradient(135deg,#8f2929,#b6473d 62%,#d8a16f);color:white;position:sticky;top:0;z-index:20;box-shadow:0 12px 30px rgba(76,17,17,.18)}
|
||||
.topline{display:flex;justify-content:space-between;align-items:flex-start;gap:18px;max-width:1500px;margin:0 auto}.title h1{font-size:28px;margin:0 0 6px}.title p{margin:0;opacity:.88}.tools{display:flex;gap:10px;align-items:center;flex-wrap:wrap}.search{min-width:280px;border:0;border-radius:14px;padding:12px 14px;font-size:14px;outline:none}.btn,label.btn{border:1px solid rgba(255,255,255,.55);background:rgba(255,255,255,.14);color:white;border-radius:13px;padding:10px 13px;cursor:pointer;font-size:14px}label.btn input{display:none}
|
||||
nav{background:#fff8f5;border-bottom:1px solid var(--line);position:sticky;top:105px;z-index:19}.tabs{display:flex;gap:8px;overflow:auto;padding:12px 28px;max-width:1500px;margin:0 auto}.tab{border:1px solid var(--line);background:#fff;border-radius:999px;padding:10px 15px;white-space:nowrap;cursor:pointer;color:var(--ink)}.tab.active{background:var(--red);color:white;border-color:var(--red)}
|
||||
.wrap{display:grid;grid-template-columns:280px minmax(0,1fr) 260px;gap:18px;max-width:1500px;margin:18px auto;padding:0 24px 40px}.panel,.card,.modal{background:var(--card);border:1px solid var(--line);box-shadow:var(--shadow);border-radius:22px}.filters{padding:16px;position:sticky;top:170px;align-self:start}.filters h3,.side h3{margin:0 0 12px}.field{margin:0 0 12px}.field label{display:block;color:var(--muted);font-size:12px;margin:0 0 5px}.field input,.field select{width:100%;border:1px solid var(--line);background:#fff;border-radius:12px;padding:10px 11px;outline:none}.field input:focus,.field select:focus{border-color:#ce9287}.reset{width:100%;border:1px solid var(--line);background:#fff;border-radius:12px;padding:10px;cursor:pointer}.side{padding:16px;position:sticky;top:170px;align-self:start}.summary-item{display:flex;align-items:center;justify-content:space-between;border:1px solid var(--line);border-radius:14px;padding:10px 12px;margin:8px 0;background:#fff}.summary-item strong{font-size:19px;color:var(--red)}.summary-item span{font-size:13px;color:var(--muted)}
|
||||
main{min-width:0}.section-title{display:flex;align-items:center;justify-content:space-between;gap:14px;margin:0 0 14px}.section-title h2{margin:0;font-size:24px}.section-title .sub{color:var(--muted);font-size:13px}.stats{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:14px;margin-bottom:16px}.stat{padding:18px;border-radius:20px;background:#fff;border:1px solid var(--line);box-shadow:var(--shadow)}.stat .num{font-size:32px;font-weight:800;color:var(--red)}.stat .lab{color:var(--muted)}.two-col{display:grid;grid-template-columns:1.15fr .85fr;gap:14px}.cards{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:14px}.card{padding:16px;cursor:pointer;transition:.18s;min-height:176px}.card:hover{transform:translateY(-2px);box-shadow:0 22px 60px rgba(51,20,14,.15);border-color:#d7b9ac}.card h3{margin:8px 0 8px;font-size:18px}.card p{margin:0 0 10px;color:#5b4744}.meta-row{display:flex;gap:7px;align-items:center;flex-wrap:wrap}.pill{display:inline-flex;align-items:center;gap:4px;border:1px solid var(--line);border-radius:999px;padding:4px 8px;font-size:12px;background:#fff;color:#5d4c49;margin:2px}.pill.red{background:#fff0ee;color:var(--red2);border-color:#ebc6bf}.pill.blue{background:var(--blue2);color:var(--blue);border-color:#cfe0ee}.pill.green{background:#eef8ef;color:var(--green);border-color:#cfe6d2}.pill.gold{background:#fff6e6;color:var(--gold);border-color:#ecd7ad}.risk-high{background:#fff0ee;color:#8b1d1d;border-color:#e8beb8}.risk-medium{background:#fff7e8;color:#8a5b08;border-color:#ead6a4}.risk-low{background:#eef9f1;color:#426f49;border-color:#cfe7d5}.small{font-size:12px;color:var(--muted)}.badge-title{font-size:12px;color:var(--muted);margin-top:12px;text-transform:uppercase;letter-spacing:.06em}.timeline{position:relative;padding-left:18px}.timeline:before{content:"";position:absolute;left:4px;top:6px;bottom:6px;width:2px;background:#e3c8c0}.titem{position:relative;margin:0 0 14px}.titem:before{content:"";position:absolute;left:-21px;top:20px;width:12px;height:12px;border-radius:50%;background:var(--red);box-shadow:0 0 0 5px #f8e8e4}.tcard{background:#fff;border:1px solid var(--line);border-radius:18px;padding:15px;cursor:pointer}.tdate{font-size:12px;color:var(--red);font-weight:700}.table-wrap{overflow:auto;background:#fff;border:1px solid var(--line);border-radius:18px}table{border-collapse:separate;border-spacing:0;width:100%;min-width:900px}th,td{border-bottom:1px solid var(--line);padding:12px 12px;text-align:left;vertical-align:top}th{background:#fff5f2;color:#6a3330;font-size:13px;position:sticky;top:0}td{font-size:14px}.viewbar{display:flex;gap:8px;justify-content:flex-end;margin:-4px 0 12px}.viewbar button, .inline-btn{border:1px solid var(--line);background:#fff;border-radius:999px;padding:8px 12px;cursor:pointer}.viewbar button.active{background:var(--red);color:#fff;border-color:var(--red)}.matrix{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:14px}.risk-col{border:1px solid var(--line);background:#fff;border-radius:20px;padding:14px}.empty{border:1px dashed #d7bdb5;border-radius:18px;padding:35px;text-align:center;color:var(--muted);background:#fff}.linkish,.rel-card{cursor:pointer}.rel-card{border:1px solid var(--line);border-radius:14px;padding:10px;background:#fff;margin:6px 0}.rel-card strong{display:block}.rel-card span{color:var(--muted);font-size:12px}.mini-grid{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:10px}.distance-grid{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:10px}.distance{background:#fff;border:1px solid var(--line);border-radius:16px;padding:12px}.distance strong{display:block;color:var(--red);font-size:22px}.distance span{font-size:12px;color:var(--muted)}
|
||||
.modal-backdrop{position:fixed;inset:0;background:rgba(38,15,12,.5);display:none;align-items:center;justify-content:center;z-index:100;padding:20px}.modal-backdrop.show{display:flex}.modal{width:min(1120px,100%);max-height:90vh;overflow:auto}.modal-head{padding:20px 24px;border-bottom:1px solid var(--line);display:flex;justify-content:space-between;gap:12px;align-items:flex-start;position:sticky;top:0;background:#fffdfb;z-index:2}.modal-title{margin:0;font-size:24px}.close{border:0;background:#f4e6e2;border-radius:12px;width:40px;height:40px;font-size:24px;cursor:pointer}.modal-body{padding:20px 24px 28px}.hero{padding:18px;border:1px solid var(--line);border-radius:20px;background:linear-gradient(135deg,#fff7f4,#fffdfb);margin-bottom:14px}.hero h3{margin:0 0 8px}.detail-grid{display:grid;grid-template-columns:1fr 1fr;gap:14px}.detail-section{border:1px solid var(--line);background:#fff;border-radius:18px;padding:15px;margin-bottom:14px}.detail-section h4{margin:0 0 10px;color:var(--red2);font-size:16px}.detail-section p{margin:6px 0}.detail-section ul{margin:8px 0 0;padding-left:20px}.obj-table{width:100%;min-width:0}.obj-table th,.obj-table td{font-size:13px;padding:8px}.full{grid-column:1/-1}.note-box{background:#fff9ef;border:1px solid #ecd5ad;border-radius:16px;padding:12px}.source-list{font-size:12px;color:var(--muted);line-height:1.6}.kpi-strip{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:10px;margin:12px 0}.kpi{border:1px solid var(--line);border-radius:16px;padding:12px;background:#fff}.kpi b{display:block;color:var(--red);font-size:20px}.kpi span{font-size:12px;color:var(--muted)}
|
||||
@media(max-width:1120px){.wrap{grid-template-columns:1fr}.filters,.side{position:relative;top:0}.cards{grid-template-columns:repeat(2,minmax(0,1fr))}.stats{grid-template-columns:repeat(2,minmax(0,1fr))}.two-col,.matrix,.detail-grid{grid-template-columns:1fr}nav{top:120px}.distance-grid{grid-template-columns:repeat(2,1fr)}}
|
||||
@media(max-width:680px){header{position:relative}.topline{display:block}.tools{margin-top:12px}.search{min-width:100%;width:100%}nav{position:relative;top:0}.wrap{padding:0 12px}.cards{grid-template-columns:1fr}.stats{grid-template-columns:1fr}.distance-grid,.mini-grid,.kpi-strip{grid-template-columns:1fr}.modal-backdrop{padding:6px}.modal-head{padding:16px}.modal-body{padding:14px}.tabs{padding:10px 12px}}
|
||||
@media print{header,nav,.filters,.side,.tools,.modal-backdrop{display:none!important}.wrap{display:block;max-width:none}.card,.panel,.stat{box-shadow:none}.cards{grid-template-columns:1fr 1fr}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header><div class="topline"><div class="title"><h1 id="projectTitle">专题数据库</h1><p id="projectSubtitle"></p></div><div class="tools"><input id="globalSearch" class="search" placeholder="全局搜索:EDCA / Cagayan / BAYAN / 台海 / WPS"><label class="btn">导入JSON<input id="jsonFile" type="file" accept="application/json,.json"></label><button class="btn" onclick="downloadJson()">导出JSON</button></div></div></header>
|
||||
<nav><div class="tabs" id="tabs"></div></nav>
|
||||
<div class="wrap"><aside class="panel filters"><h3>筛选器</h3><div class="field"><label>关键词</label><input id="filterKeyword" placeholder="在当前库内搜索"></div><div class="field"><label>年份</label><select id="filterYear"></select></div><div class="field"><label>事件类型</label><select id="filterCategory"></select></div><div class="field"><label>省份</label><select id="filterProvince"></select></div><div class="field"><label>阵营</label><select id="filterCamp"></select></div><div class="field"><label>风险等级</label><select id="filterRisk"></select></div><div class="field"><label>战略方向</label><select id="filterAxis"></select></div><button class="reset" onclick="resetFilters()">清空筛选</button><div class="source-list" style="margin-top:14px">说明:本页面为公开资料研究展示工具,复杂字段以详情弹框结构化展示,不再输出原始JSON。</div></aside><main id="main"></main><aside class="panel side"><h3>数据摘要</h3><div id="summaryPanel"></div><h3 style="margin-top:18px">高频标签</h3><div id="tagCloud"></div></aside></div>
|
||||
<div class="modal-backdrop" id="modalBackdrop" onclick="backdropClose(event)"><div class="modal"><div class="modal-head"><div><h2 id="modalTitle" class="modal-title"></h2><div id="modalMeta" class="meta-row"></div></div><button class="close" onclick="closeModal()">×</button></div><div id="modalBody" class="modal-body"></div></div></div>
|
||||
<script id="db-json" type="application/json">{"meta": {"project_title": "菲律宾“不做美国棋子”叙事专题数据库", "project_subtitle": "Philippines “Not a US Pawn” Narrative Research Dashboard", "version": "0.2", "last_updated": "2026-06-29", "language": "zh-CN", "description": "用于展示菲律宾“菲律宾不做美国棋子”叙事专题的结构化知识库,覆盖EDCA地点、事件线、人物组织、叙事结构、竞争叙事、风险核查和跨库关联查询。", "scope_note": "本数据库服务于公开资料研究、舆情观察、风险评估与专题分析,不用于隐蔽影响、虚假传播或操控性动员。", "revision_note": "v0.2增强版:弹框详情改为结构化页面展示,补充研究字段,尽量保留前期MD知识库中的分析内容。"}, "tabs": [{"id": "overview", "label": "专题总览", "type": "dashboard"}, {"id": "timeline", "label": "事件时间线", "type": "timeline"}, {"id": "edca_sites", "label": "EDCA地点库", "type": "cards_table"}, {"id": "actors", "label": "人物组织库", "type": "cards_table"}, {"id": "narratives", "label": "叙事结构库", "type": "cards"}, {"id": "counter_narratives", "label": "竞争叙事库", "type": "cards"}, {"id": "risks", "label": "风险核查库", "type": "risk_matrix"}, {"id": "search", "label": "综合查询", "type": "search"}], "dictionaries": {"event_categories": {"edca_signature": "EDCA签署/制度化", "edca_expansion": "EDCA扩展", "edca_infrastructure": "EDCA基建", "balikatan": "Balikatan军演", "china_coast_guard": "中国海警事件", "ph_official_protest": "菲律宾官方抗议", "us_ph_meeting": "美菲高层会晤", "anti_edca_protest": "反EDCA/反军演抗议", "multilateral_security": "多边安全合作", "wps_civil_action": "WPS民间行动"}, "strategic_axis": {"taiwan_bashi": "台海—巴士海峡方向", "wps_scs": "西菲律宾海—南海方向", "northern_luzon": "北吕宋前沿方向", "central_luzon": "中吕宋支撑方向", "central_logistics": "中部群岛后勤方向", "mindanao": "棉兰老方向", "hadr": "人道援助与灾害响应"}, "actor_camps": {"marcos_official": "马科斯官方阵营", "duterte_camp": "杜特尔特阵营", "left_anti_base": "左翼反基地阵营", "wps_sovereignty": "WPS主权派", "fact_check": "事实核查机构", "security_expert": "安全专家与智库", "external_actor": "外部行为体"}, "actor_types": {"official_person": "官方人物", "political_person": "政治人物", "organization": "组织", "party_bloc": "政党/党团", "institution": "机构", "expert": "专家", "think_tank": "智库", "media_factcheck": "事实核查/媒体"}, "stance_values": {"support": "支持", "oppose": "反对", "mixed": "复杂/混合", "neutral": "中立", "unknown": "未知"}, "risk_levels": {"low": "低", "medium": "中", "high": "高"}, "site_types": {"naval_base": "海军基地", "airport": "机场", "army_camp": "陆军营地", "air_base": "空军基地", "airfield": "机场/航空场"}, "field_labels": {"id": "编号", "name": "名称", "name_cn": "中文名", "name_en": "英文名", "title": "标题", "summary": "摘要", "brief": "简述", "description": "说明", "province": "省份", "city_or_area": "城市/区域", "designation_year": "纳入年份", "site_type": "地点类型", "strategic_axis": "战略方向", "risk_level": "风险等级", "local_position": "地方态度", "tags": "标签", "distances": "近似距离", "detail": "详情", "strategic_value": "战略意义", "local_political_positions": "地方政治人物表态", "local_reactions": "地方居民抗议或支持", "infrastructure_projects": "基建项目", "risk_controversies": "风险争议", "official_response": "官方回应", "related_events": "关联事件", "related_actors": "关联人物组织", "related_sites": "关联地点", "background": "背景", "opposition_position": "反对方立场", "ph_official_position": "菲律宾官方立场", "us_position": "美国立场", "china_position": "中国立场", "research_note": "研究备注", "category": "类别", "date": "日期", "year": "年份", "location": "地点", "actors": "相关行为体", "narrative_signal": "叙事信号", "counter_narrative": "竞争叙事/反驳框架", "current_role": "当前身份", "camp": "阵营", "type": "类型", "stance_on_edca": "对EDCA立场", "stance_on_wps": "对WPS立场", "influence_level": "影响等级", "narrative_function": "叙事功能", "typical_positions": "典型立场", "typical_keywords": "典型关键词", "risk_note": "风险备注", "profile": "画像", "monitoring_focus": "监测重点", "core_claims": "核心命题", "target_audiences": "目标受众", "typical_expressions": "典型表达", "strength": "叙事强项", "weakness": "叙事弱项", "counter_narratives": "竞争叙事", "main_speakers": "主要提出者", "used_against": "用于反驳", "evidence_types": "证据类型", "advantage": "优势", "limitation": "限制", "trigLine truncated
|
||||
<script>
|
||||
let DB = JSON.parse(document.getElementById('db-json').textContent);
|
||||
const $=s=>document.querySelector(s), $$=s=>Array.from(document.querySelectorAll(s));
|
||||
let state={tab:'overview',view:'cards',filters:{},global:''};
|
||||
const collMap={event:'events',edca_site:'edca_sites',actor:'actors',narrative:'narratives',counter_narrative:'counter_narratives',risk:'risks'};
|
||||
const tabsForType={event:'timeline',edca_site:'edca_sites',actor:'actors',narrative:'narratives',counter_narrative:'counter_narratives',risk:'risks'};
|
||||
function arr(x){return Array.isArray(x)?x:[]}
|
||||
function esc(s){return String(s??'').replace(/[&<>"']/g,m=>({'&':'&','<':'<','>':'>','"':'"',"'":'''}[m]))}
|
||||
function dict(group,key){return DB.dictionaries?.[group]?.[key] || key || ''}
|
||||
function label(k){return DB.dictionaries?.field_labels?.[k] || k}
|
||||
function titleOf(o){return o?.title || o?.name_cn || o?.name || o?.name_en || o?.id || ''}
|
||||
function summaryOf(o){return o?.summary || o?.brief || o?.description || o?.narrative_function || ''}
|
||||
function risk(level){return level?`<span class="pill risk-${esc(level)}">风险:${esc(dict('risk_levels',level))}</span>`:''}
|
||||
function chips(tags,cls=''){return arr(tags).map(t=>`<span class="pill ${cls}">${esc(t)}</span>`).join('')}
|
||||
function getAll(){let out=[];Object.entries(collMap).forEach(([type,key])=>arr(DB.data?.[key]).forEach(x=>out.push({...x,__type:type})));return out}
|
||||
function find(id){for(const [type,key] of Object.entries(collMap)){const x=arr(DB.data?.[key]).find(i=>i.id===id); if(x) return {...x,__type:type}}return null}
|
||||
function searchText(o){return JSON.stringify(o).toLowerCase()}
|
||||
function matchKw(o,kw){return !kw || searchText(o).includes(kw.toLowerCase())}
|
||||
function setup(){ $('#projectTitle').textContent=DB.meta.project_title; $('#projectSubtitle').textContent=(DB.meta.project_subtitle||'')+' · '+(DB.meta.revision_note||''); renderTabs(); fillFilters(); bind(); render(); }
|
||||
function renderTabs(){ $('#tabs').innerHTML=arr(DB.tabs).map(t=>`<button class="tab ${state.tab===t.id?'active':''}" data-id="${esc(t.id)}">${esc(t.label)}</button>`).join(''); $$('#tabs .tab').forEach(b=>b.onclick=()=>{state.tab=b.dataset.id;renderTabs();render();}); }
|
||||
function optAll(label){return `<option value="">${label}</option>`}
|
||||
function fillFilters(){ const years=[...new Set(arr(DB.data.events).map(e=>e.year).filter(Boolean))].sort((a,b)=>a-b); $('#filterYear').innerHTML=optAll('全部年份')+years.map(y=>`<option>${y}</option>`).join(''); $('#filterCategory').innerHTML=optAll('全部类型')+Object.entries(DB.dictionaries.event_categories||{}).map(([k,v])=>`<option value="${esc(k)}">${esc(v)}</option>`).join(''); const ps=[...new Set(arr(DB.data.edca_sites).map(s=>s.province).filter(Boolean))].sort(); $('#filterProvince').innerHTML=optAll('全部省份')+ps.map(p=>`<option>${esc(p)}</option>`).join(''); $('#filterCamp').innerHTML=optAll('全部阵营')+Object.entries(DB.dictionaries.actor_camps||{}).map(([k,v])=>`<option value="${esc(k)}">${esc(v)}</option>`).join(''); $('#filterRisk').innerHTML=optAll('全部风险')+Object.entries(DB.dictionaries.risk_levels||{}).map(([k,v])=>`<option value="${esc(k)}">${esc(v)}</option>`).join(''); $('#filterAxis').innerHTML=optAll('全部方向')+Object.entries(DB.dictionaries.strategic_axis||{}).map(([k,v])=>`<option value="${esc(k)}">${esc(v)}</option>`).join(''); }
|
||||
function bind(){ ['filterKeyword','filterYear','filterCategory','filterProvince','filterCamp','filterRisk','filterAxis'].forEach(id=>{$('#'+id).oninput=collect;$('#'+id).onchange=collect}); $('#globalSearch').oninput=e=>{state.global=e.target.value.trim(); if(state.global){state.tab='search';renderTabs()} render()}; $('#jsonFile').onchange=importJson; window.onkeydown=e=>{if(e.key==='Escape')closeModal()}; }
|
||||
function collect(){ state.filters={keyword:$('#filterKeyword').value.trim(),year:$('#filterYear').value,category:$('#filterCategory').value,province:$('#filterProvince').value,camp:$('#filterCamp').value,risk:$('#filterRisk').value,axis:$('#filterAxis').value}; render(); }
|
||||
function resetFilters(){['filterKeyword','filterYear','filterCategory','filterProvince','filterCamp','filterRisk','filterAxis','globalSearch'].forEach(id=>$('#'+id).value='');state.filters={};state.global='';render()}
|
||||
function commonFilter(items){const f=state.filters||{};return items.filter(x=>{if(f.keyword&&!matchKw(x,f.keyword))return false;if(f.risk&&x.risk_level!==f.risk)return false;if(f.year&&x.year&&String(x.year)!==String(f.year))return false;if(f.category&&x.category&&x.category!==f.category)return false;if(f.province&&x.province&&x.province!==f.province)return false;if(f.camp&&x.camp&&x.camp!==f.camp)return false;if(f.axis&&x.strategic_axis&&!arr(x.strategic_axis).includes(f.axis))return false;return true})}
|
||||
function render(){renderSummary();const t=state.tab;if(t==='overview')overview();if(t==='timeline')timeline();if(t==='edca_sites')sites();if(t==='actors')actors();if(t==='narratives')narratives();if(t==='counter_narratives')counters();if(t==='risks')risks();if(t==='search')search();}
|
||||
function renderSummary(){const d=DB.data;const rows=[['EDCA地点',arr(d.edca_sites).length],['事件',arr(d.events).length],['人物组织',arr(d.actors).length],['叙事结构',arr(d.narratives).length],['竞争叙事',arr(d.counter_narratives).length],['风险项',arr(d.risks).length],['高风险地点',arr(d.edca_sites).filter(x=>x.risk_level==='high').length],['高风险事件',arr(d.events).filter(x=>x.risk_level==='high').length]];$('#summaryPanel').innerHTML=rows.map(([k,v])=>`<div class="summary-item"><strong>${v}</strong><span>${k}</span></div>`).join(''); const tags={}; getAll().forEach(x=>arr(x.tags).forEach(t=>tags[t]=(tags[t]||0)+1)); $('#tagCloud').innerHTML=Object.entries(tags).sort((a,b)=>b[1]-a[1]).slice(0,20).map(([t,n])=>`<span class="pill">${esc(t)} · ${n}</span>`).join('');}
|
||||
function overview(){const d=DB.data;const high=arr(d.edca_sites).filter(x=>x.risk_level==='high');const eventCats={};arr(d.events).forEach(e=>eventCats[e.category]=(eventCats[e.category]||0)+1); const camps={};arr(d.actors).forEach(a=>camps[a.camp]=(camps[a.camp]||0)+1); $('#main').innerHTML=`<div class="section-title"><div><h2>专题总览</h2><div class="sub">${esc(DB.meta.description)}</div></div><span class="pill red">${esc(DB.meta.version)} · ${esc(DB.meta.last_updated)}</span></div><div class="stats"><div class="stat"><div class="num">${arr(d.edca_sites).length}</div><div class="lab">EDCA地点</div></div><div class="stat"><div class="num">${arr(d.events).length}</div><div class="lab">事件线节点</div></div><div class="stat"><div class="num">${arr(d.actors).length}</div><div class="lab">人物组织</div></div><div class="stat"><div class="num">${high.length}</div><div class="lab">高风险地点</div></div></div><div class="two-col"><div class="panel" style="padding:18px"><h3>核心问题</h3><p>本页面围绕“菲律宾不做美国棋子 / not a US pawn”这一叙事,展示其与EDCA、美菲同盟、Balikatan军演、西菲律宾海主权、杜特尔特—马科斯政治竞争、地方风险和事实核查之间的关系。</p><p class="small">${esc(DB.meta.scope_note||'')}</p>${chips(DB.indexes?.common_tags)}</div><div class="panel" style="padding:18px"><h3>事件类型分布</h3>${Object.entries(eventCats).map(([k,v])=>`<div class="summary-item"><strong>${v}</strong><span>${esc(dict('event_categories',k))}</span></div>`).join('')}</div></div><div style="height:14px"></div><div class="two-col"><div class="panel" style="padding:18px"><h3>阵营分布</h3>${Object.entries(camps).map(([k,v])=>`<div class="summary-item"><strong>${v}</strong><span>${esc(dict('actor_camps',k))}</span></div>`).join('')}</div><div class="panel" style="padding:18px"><h3>展示原则</h3><ul><li>卡片只看摘要,弹框展示完整知识结构。</li><li>长字段按模块呈现,不再显示原始JSON。</li><li>所有关联对象支持点击跳转。</li><li>风险内容同时展示核查重点和边界。</li></ul></div></div><div style="height:14px"></div><div class="section-title"><h2>高风险EDCA地点</h2><button class="inline-btn" onclick="state.tab='edca_sites';renderTabs();render()">查看全部</button></div><div class="cards">${high.map(siteCard).join('')}</div>`}
|
||||
function timeline(){let items=commonFilter(arr(DB.data.events)).sort((a,b)=>String(b.date).localeCompare(String(a.date))); if(state.filters.province){const ids=arr(DB.data.edca_sites).filter(s=>s.province===state.filters.province).map(s=>s.id);items=items.filter(e=>arr(e.related_sites).some(id=>ids.includes(id))||matchKw(e,state.filters.province));} $('#main').innerHTML=`<div class="section-title"><h2>事件时间线</h2><span class="pill blue">命中 ${items.length} 条</span></div>`+(items.length?`<div class="timeline">${items.map(e=>`<div class="titem"><div class="tcard" onclick="openModal('event','${esc(e.id)}')"><div class="tdate">${esc(e.date)} · ${esc(dict('event_categories',e.category))}</div><h3>${esc(e.title)}</h3><p>${esc(e.summary)}</p><div class="meta-row">${risk(e.risk_level)}${chips(e.tags)}</div></div></div>`).join('')}</div>`:empty())}
|
||||
function sites(){let items=commonFilter(arr(DB.data.edca_sites)); $('#main').innerHTML=`<div class="section-title"><h2>EDCA九个地点库</h2><span class="pill blue">命中 ${items.length} 条</span></div>${viewbar('sites')} ${state.view==='table'?siteTable(items):`<div class="cards">${items.map(siteCard).join('')}</div>`}`}
|
||||
function siteCard(s){return `<div class="card" onclick="openModal('edca_site','${esc(s.id)}')"><div class="meta-row"><span class="pill red">${esc(s.province)}</span>${risk(s.risk_level)}</div><h3>${esc(s.name)}</h3><p>${esc(s.brief)}</p><div class="badge-title">战略方向</div><div>${arr(s.strategic_axis).map(a=>`<span class="pill blue">${esc(dict('strategic_axis',a))}</span>`).join('')}</div><div class="badge-title">标签</div>${chips(s.tags)}</div>`}
|
||||
function siteTable(items){return `<div class="table-wrap"><table><thead><tr><th>地点</th><th>省份</th><th>年份</th><th>类型</th><th>战略方向</th><th>地方态度</th><th>风险</th><th>详情</th></tr></thead><tbody>${items.map(s=>`<tr><td><b>${esc(s.name)}</b><br><span class="small">${esc(s.city_or_area)}</span></td><td>${esc(s.province)}</td><td>${esc(s.designation_year)}</td><td>${esc(dict('site_types',s.site_type))}</td><td>${arr(s.strategic_axis).map(a=>esc(dict('strategic_axis',a))).join('<br>')}</td><td>${esc(s.local_position)}</td><td>${risk(s.risk_level)}</td><td><button class="inline-btn" onclick="openModal('edca_site','${esc(s.id)}')">打开</button></td></tr>`).join('')}</tbody></table></div>`}
|
||||
function actors(){let items=commonFilter(arr(DB.data.actors)); $('#main').innerHTML=`<div class="section-title"><h2>人物组织库</h2><span class="pill blue">命中 ${items.length} 条</span></div>${viewbar('actors')} ${state.view==='table'?actorTable(items):`<div class="cards">${items.map(actorCard).join('')}</div>`}`}
|
||||
function actorCard(a){return `<div class="card" onclick="openModal('actor','${esc(a.id)}')"><div class="meta-row"><span class="pill red">${esc(dict('actor_camps',a.camp))}</span><span class="pill">${esc(dict('actor_types',a.type))}</span></div><h3>${esc(a.name_cn||a.name_en)}</h3><p>${esc(a.brief)}</p><div class="badge-title">叙事功能</div><p>${esc(a.narrative_function||'')}</p>${chips(a.tags)}</div>`}
|
||||
function actorTable(items){return `<div class="table-wrap"><table><thead><tr><th>名称</th><th>类型</th><th>阵营</th><th>当前角色</th><th>EDCA立场</th><th>WPS立场</th><th>影响</th><th>详情</th></tr></thead><tbody>${items.map(a=>`<tr><td><b>${esc(a.name_cn||a.name_en)}</b><br><span class="small">${esc(a.name_en||'')}</span></td><td>${esc(dict('actor_types',a.type))}</td><td>${esc(dict('actor_camps',a.camp))}</td><td>${esc(a.current_role)}</td><td>${esc(dict('stance_values',a.stance_on_edca))}</td><td>${esc(dict('stance_values',a.stance_on_wps))}</td><td>${esc(a.influence_level)}</td><td><button class="inline-btn" onclick="openModal('actor','${esc(a.id)}')">打开</button></td></tr>`).join('')}</tbody></table></div>`}
|
||||
function narratives(){const items=commonFilter(arr(DB.data.narratives)); $('#main').innerHTML=`<div class="section-title"><h2>叙事结构库</h2><span class="pill blue">命中 ${items.length} 条</span></div><div class="cards">${items.map(n=>`<div class="card" onclick="openModal('narrative','${esc(n.id)}')"><div class="meta-row"><span class="pill red">${esc(n.category)}</span>${risk(n.risk_level)}</div><h3>${esc(n.title)}</h3><p>${esc(n.summary)}</p>${chips(n.tags)}</div>`).join('')}</div>`}
|
||||
function counters(){const items=commonFilter(arr(DB.data.counter_narratives)); $('#main').innerHTML=`<div class="section-title"><h2>竞争叙事库</h2><span class="pill blue">命中 ${items.length} 条</span></div><div class="cards">${items.map(c=>`<div class="card" onclick="openModal('counter_narrative','${esc(c.id)}')"><div class="meta-row"><span class="pill blue">竞争叙事</span>${risk(c.risk_level)}</div><h3>${esc(c.title)}</h3><p>${esc(c.summary)}</p>${chips(c.tags)}</div>`).join('')}</div>`}
|
||||
function risks(){const items=commonFilter(arr(DB.data.risks)); const levels=['high','medium','low']; $('#main').innerHTML=`<div class="section-title"><h2>风险与事实核查库</h2><span class="pill blue">命中 ${items.length} 条</span></div><div class="matrix">${levels.map(l=>`<div class="risk-col"><h3>${esc(dict('risk_levels',l))}风险</h3>${items.filter(r=>r.risk_level===l).map(r=>`<div class="card" style="min-height:auto;margin-bottom:10px" onclick="openModal('risk','${esc(r.id)}')"><h3>${esc(r.title)}</h3><p>${esc(r.description)}</p>${chips(r.tags)}</div>`).join('')||'<div class="empty">暂无</div>'}</div>`).join('')}</div>`}
|
||||
function search(){const kw=state.global||state.filters.keyword||'';let items=getAll().filter(x=>matchKw(x,kw));items=commonFilter(items); const names={event:'事件',edca_site:'EDCA地点',actor:'人物组织',narrative:'叙事结构',counter_narrative:'竞争叙事',risk:'风险'}; $('#main').innerHTML=`<div class="section-title"><div><h2>综合查询中心</h2><div class="sub">${kw?'关键词:'+esc(kw):'可在顶部或左侧输入关键词,跨库检索所有对象。'}</div></div><span class="pill blue">命中 ${items.length} 条</span></div>`+(items.length?`<div class="table-wrap"><table><thead><tr><th>类型</th><th>标题/名称</th><th>摘要</th><th>标签</th><th>风险</th><th>详情</th></tr></thead><tbody>${items.map(i=>`<tr><td><span class="pill red">${esc(names[i.__type])}</span></td><td><b>${esc(titleOf(i))}</b></td><td>${esc(summaryOf(i)).slice(0,260)}</td><td>${chips(i.tags)}</td><td>${risk(i.risk_level)}</td><td><button class="inline-btn" onclick="openModal('${esc(i.__type)}','${esc(i.id)}')">打开</button></td></tr>`).join('')}</tbody></table></div>`:empty('暂无查询结果。'))}
|
||||
function viewbar(){return `<div class="viewbar"><button class="${state.view==='cards'?'active':''}" onclick="state.view='cards';render()">卡片</button><button class="${state.view==='table'?'active':''}" onclick="state.view='table';render()">表格</button></div>`}
|
||||
function empty(t='没有匹配结果。'){return `<div class="empty">${esc(t)}</div>`}
|
||||
function openModal(type,id){const item=find(id); if(!item)return; $('#modalTitle').textContent=titleOf(item); $('#modalMeta').innerHTML=modalMeta(item); $('#modalBody').innerHTML=modalBody(item,type); $('#modalBackdrop').classList.add('show')}
|
||||
function closeModal(){$('#modalBackdrop').classList.remove('show')}
|
||||
function backdropClose(e){if(e.target.id==='modalBackdrop')closeModal()}
|
||||
function modalMeta(o){let bits=[]; if(o.date)bits.push(`<span class="pill">${esc(o.date)}</span>`); if(o.category)bits.push(`<span class="pill blue">${esc(dict('event_categories',o.category))}</span>`); if(o.province)bits.push(`<span class="pill red">${esc(o.province)}</span>`); if(o.camp)bits.push(`<span class="pill red">${esc(dict('actor_camps',o.camp))}</span>`); if(o.risk_level)bits.push(risk(o.risk_level)); return bits.join('')+chips(o.tags)}
|
||||
function section(title,content,cls=''){if(!content)return''; return `<section class="detail-section ${cls}"><h4>${esc(title)}</h4>${content}</section>`}
|
||||
function paragraph(v){return v?`<p>${esc(v)}</p>`:''}
|
||||
function list(v){if(!v||!arr(v).length)return''; return `<ul>${arr(v).map(x=>`<li>${renderVal(x)}</li>`).join('')}</ul>`}
|
||||
function objGrid(o){if(!o||typeof o!=='object')return''; return `<div class="mini-grid">${Object.entries(o).filter(([k,v])=>v!==undefined&&v!==null&&v!==''&&!(Array.isArray(v)&&!v.length)).map(([k,v])=>`<div class="rel-card"><span>${esc(label(k))}</span><strong>${renderVal(v)}</strong></div>`).join('')}</div>`}
|
||||
function objTable(items){if(!arr(items).length)return''; if(typeof items[0]!=='object')return list(items); const keys=[...new Set(items.flatMap(o=>Object.keys(o)))]; return `<div class="table-wrap"><table class="obj-table"><thead><tr>${keys.map(k=>`<th>${esc(label(k))}</th>`).join('')}</tr></thead><tbody>${items.map(o=>`<tr>${keys.map(k=>`<td>${renderVal(o[k])}</td>`).join('')}</tr>`).join('')}</tbody></table></div>`}
|
||||
function renderVal(v){if(v===undefined||v===null)return''; if(Array.isArray(v)){if(v.length&&typeof v[0]==='object')return objTable(v); return arr(v).map(x=>renderVal(x)).join('<br>')} if(typeof v==='object')return objGrid(v); const s=String(v); const x=find(s); if(x)return `<span class="linkish pill blue" onclick="jump('${esc(s)}')">${esc(titleOf(x))}</span>`; return esc(s)}
|
||||
function rels(ids){return arr(ids).map(id=>{const x=find(id); return x?`<div class="rel-card" onclick="jump('${esc(id)}')"><strong>${esc(titleOf(x))}</strong><span>${esc(typeName(x.__type))} · ${esc(summaryOf(x)).slice(0,88)}</span></div>`:`<div class="rel-card"><strong>${esc(id)}</strong></div>`}).join('')}
|
||||
function typeName(t){return {event:'事件',edca_site:'EDCA地点',actor:'人物组织',narrative:'叙事结构',counter_narrative:'竞争叙事',risk:'风险'}[t]||t}
|
||||
function distances(d){if(!d)return''; return `<div class="distance-grid">${Object.entries(d).map(([k,v])=>`<div class="distance"><strong>${esc(v)} km</strong><span>${esc(label(k))}</span></div>`).join('')}</div>`}
|
||||
function modalBody(o,type){
|
||||
let d=o.detail||{};
|
||||
if(type==='edca_site') return `<div class="hero"><h3>${esc(o.name)}</h3><p>${esc(o.brief)}</p><div class="kpi-strip"><div class="kpi"><b>${esc(o.designation_year)}</b><span>纳入年份</span></div><div class="kpi"><b>${esc(o.province)}</b><span>省份</span></div><div class="kpi"><b>${esc(dict('site_types',o.site_type))}</b><span>地点类型</span></div><div class="kpi"><b>${esc(dict('risk_levels',o.risk_level))}</b><span>风险等级</span></div></div></div><div class="detail-grid">${section('基本信息',objGrid({province:o.province,city_or_area:o.city_or_area,designation_year:o.designation_year,site_type:dict('site_types',o.site_type),strategic_axis:arr(o.strategic_axis).map(a=>dict('strategic_axis',a)).join(' / '),local_position:o.local_position}),'full')}${section('近似距离与战略方向',distances(o.distances),'full')}${section('战略意义',paragraph(d.strategic_value))}${section('叙事解读',list(d.narrative_interpretation))}${section('地方政治人物表态',objTable(d.local_political_positions),'full')}${section('地方居民抗议或支持',objGrid(d.local_reactions))}${section('基建项目',list(d.infrastructure_projects))}${section('风险争议',list(d.risk_controversies))}${section('官方回应',paragraph(d.official_response))}${section('竞争性解释',list(d.counter_reading))}${section('证据线索',list(d.evidence_clues))}${section('展示提示',`<div class="note-box">${esc(d.display_warning||'')}</div>`)}${section('研究缺口',list(d.research_gaps),'full')}${section('关联事件',rels(d.related_events),'full')}${section('关联人物组织',rels(d.related_actors),'full')}</div>`;
|
||||
if(type==='event') return `<div class="hero"><h3>${esc(o.title)}</h3><p>${esc(o.summary)}</p></div><div class="detail-grid">${section('时间地点',objGrid({date:o.date,year:o.year,location:o.location,category:dict('event_categories',o.category)}),'full')}${section('事件背景',paragraph(d.background))}${section('为什么重要',paragraph(d.why_it_matters||o.narrative_signal))}${section('叙事信号',paragraph(o.narrative_signal))}${section('竞争叙事/反驳框架',paragraph(o.counter_narrative))}${section('菲律宾官方立场',paragraph(d.ph_official_position))}${section('美国立场',paragraph(d.us_position))}${section('中国立场',paragraph(d.china_position))}${section('反对方立场',paragraph(d.opposition_position))}${section('叙事复杂性',paragraph(d.narrative_complication))}${section('研究备注',paragraph(d.research_note),'full')}${section('相关行为体',rels(o.actors),'full')}${section('关联地点',rels(o.related_sites),'full')}</div>`;
|
||||
if(type==='actor') return `<div class="hero"><h3>${esc(o.name_cn||o.name_en)}</h3><p>${esc(o.brief)}</p></div><div class="detail-grid">${section('基本信息',objGrid({name_en:o.name_en,type:dict('actor_types',o.type),camp:dict('actor_camps',o.camp),current_role:o.current_role,stance_on_edca:dict('stance_values',o.stance_on_edca),stance_on_wps:dict('stance_values',o.stance_on_wps),influence_level:o.influence_level}),'full')}${section('人物/组织画像',paragraph(d.profile))}${section('叙事功能',paragraph(o.narrative_function))}${section('典型立场',list(d.typical_positions))}${section('典型关键词',list(d.typical_keywords))}${section('监测重点',list(d.monitoring_focus),'full')}${section('风险备注',paragraph(d.risk_note))}${section('研究备注',paragraph(d.research_note))}${section('关联事件',rels(d.related_events),'full')}${section('关联地点',rels(d.related_sites),'full')}</div>`;
|
||||
if(type==='narrative') return `<div class="hero"><h3>${esc(o.title)}</h3><p>${esc(o.summary)}</p></div><div class="detail-grid">${section('核心命题',list(o.core_claims),'full')}${section('目标受众',list(o.target_audiences))}${section('典型表达',list(o.typical_expressions))}${section('适用场景',list(d.application_scenarios))}${section('叙事强项',paragraph(d.strength))}${section('叙事弱项',paragraph(d.weakness))}${section('分析备注',paragraph(d.analysis_note),'full')}${section('待补数据',list(d.data_to_collect))}${section('关联事件',rels(o.related_events),'full')}${section('关联人物组织',rels(o.related_actors),'full')}${section('竞争叙事',rels(d.counter_narratives),'full')}</div>`;
|
||||
if(type==='counter_narrative') return `<div class="hero"><h3>${esc(o.title)}</h3><p>${esc(o.summary)}</p></div><div class="detail-grid">${section('主要提出者',list(o.main_speakers))}${section('证据类型',list(o.evidence_types))}${section('优势',paragraph(d.advantage))}${section('限制',paragraph(d.limitation))}${section('观察场景',list(d.where_to_observe))}${section('对“美国棋子”叙事的影响',paragraph(d.impact_on_proxy_narrative),'full')}${section('用于反驳',rels(o.used_against),'full')}${section('关联事件',rels(d.related_events),'full')}</div>`;
|
||||
if(type==='risk') return `<div class="hero"><h3>${esc(o.title)}</h3><p>${esc(o.description)}</p></div><div class="detail-grid">${section('触发场景',list(o.trigger_scenarios),'full')}${section('事实核查重点',list(o.fact_check_focus),'full')}${section('风险边界与处理建议',list(o.mitigation_guidelines),'full')}${section('展示提示',`<div class="note-box">${esc(o.display_note||'')}</div>`)}${section('关联地点',rels(o.related_sites),'full')}${section('关联人物组织',rels(o.related_actors),'full')}</div>`;
|
||||
return section('详情',renderVal(o),'full')
|
||||
}
|
||||
function jump(id){const x=find(id); if(!x)return; closeModal(); state.tab=tabsForType[x.__type]||'search'; renderTabs(); render(); setTimeout(()=>openModal(x.__type,id),80)}
|
||||
function importJson(e){const file=e.target.files[0]; if(!file)return; const r=new FileReader(); r.onload=ev=>{try{DB=JSON.parse(ev.target.result);fillFilters();renderTabs();render();alert('JSON导入成功')}catch(err){alert('JSON解析失败:'+err.message)}}; r.readAsText(file,'utf-8')}
|
||||
function downloadJson(){const blob=new Blob([JSON.stringify(DB,null,2)],{type:'application/json;charset=utf-8'});const url=URL.createObjectURL(blob);const a=document.createElement('a');a.href=url;a.download='philippines_not_us_pawn_narrative_db_v2_export.json';a.click();URL.revokeObjectURL(url)}
|
||||
setup();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
+49
-7
@@ -12,7 +12,8 @@ Resolution rule (per resource type):
|
||||
* For a resource type **with** bound tags → the union of resources carrying any
|
||||
of those tags.
|
||||
* For a resource type **without** bound tags → the *default set*:
|
||||
- ``agent`` → all built-in agents (``user_id IS NULL``)
|
||||
- ``agent`` -> built-ins first, then filtered out of
|
||||
position-granted favorites
|
||||
- ``scheduled_task`` → empty (no built-in tasks)
|
||||
- ``skill`` → empty (the 广场 still shows every visible skill;
|
||||
a position only *adds* its tagged skills to the member's "我的技能")
|
||||
@@ -24,9 +25,12 @@ Resolution rule (per resource type):
|
||||
|
||||
Materialization (``sync_user``) only ever touches rows it created itself
|
||||
(``origin='position'``), so a user's own favorites / subscriptions are never
|
||||
clobbered. ``skill`` is materialized into ``skill_favorites`` exactly like
|
||||
``agent`` → ``agent_favorites``: the tagged skills are *added* to the user's
|
||||
"我的技能" while the browse-all 广场 stays complete (it is **not** restricted).
|
||||
clobbered. ``agent`` keeps the original position-granted behavior for ordinary
|
||||
tagged agents, but filters out built-ins and business-chain seats; those only
|
||||
enter "mine" after an explicit user favorite. ``skill`` is still materialized
|
||||
into ``skill_favorites`` so tagged skills are added to the user's "mine"
|
||||
collection while the browse-all square stays complete (it is **not**
|
||||
restricted).
|
||||
Only recommended-question visibility is computed live by :meth:`resolve_visible`
|
||||
and is *not* materialized. (The admin-configured prompts shown to users ARE the
|
||||
recommended questions — there is no separate prompt-template store.)
|
||||
@@ -34,6 +38,7 @@ recommended questions — there is no separate prompt-template store.)
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
@@ -43,13 +48,15 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from deerflow.persistence.agents.model import AgentFavoriteRow, AgentRow
|
||||
from deerflow.persistence.positions.base import PositionStore
|
||||
from deerflow.persistence.roundtable_chains.model import RoundtableChainRow
|
||||
from deerflow.persistence.scheduled_tasks.model import ScheduledTaskSubscriptionRow
|
||||
from deerflow.persistence.skills.model import SkillFavoriteRow
|
||||
from deerflow.persistence.tags.base import TagStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Materialized into "我的" tables by sync_user.
|
||||
# Reconciled by sync_user. ``agent`` preserves ordinary position-granted rows,
|
||||
# but filters built-ins and business-chain seats.
|
||||
_MATERIALIZED_TYPES = ("agent", "skill", "scheduled_task")
|
||||
# Resolved live at query time (visibility filters), never materialized.
|
||||
_VISIBILITY_TYPES = ("recommended_question",)
|
||||
@@ -93,8 +100,12 @@ class PositionSyncService:
|
||||
bindings = await self._positions.get_tag_bindings(pid) if pid else {}
|
||||
tag_ids = bindings.get(resource_type) or []
|
||||
if tag_ids:
|
||||
return set(await self._tags.list_targets_for_tags(resource_type, tag_ids))
|
||||
return await self._default_set(resource_type, session)
|
||||
target = set(await self._tags.list_targets_for_tags(resource_type, tag_ids))
|
||||
else:
|
||||
target = await self._default_set(resource_type, session)
|
||||
if resource_type == "agent":
|
||||
return await self._filter_materializable_agents(target, session)
|
||||
return target
|
||||
|
||||
async def _default_set(self, resource_type: str, session: AsyncSession) -> set[str]:
|
||||
if resource_type == "agent":
|
||||
@@ -103,6 +114,37 @@ class PositionSyncService:
|
||||
# scheduled_task (and any other materialized type) defaults to empty.
|
||||
return set()
|
||||
|
||||
async def _filter_materializable_agents(self, agent_ids: set[str], session: AsyncSession) -> set[str]:
|
||||
if not agent_ids:
|
||||
return set()
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(AgentRow.id).where(
|
||||
AgentRow.id.in_(agent_ids),
|
||||
AgentRow.user_id.is_not(None),
|
||||
)
|
||||
)
|
||||
).all()
|
||||
candidate_ids = {r[0] for r in rows}
|
||||
return candidate_ids - await self._business_chain_agent_ids(session)
|
||||
|
||||
async def _business_chain_agent_ids(self, session: AsyncSession) -> set[str]:
|
||||
rows = (await session.execute(select(RoundtableChainRow.seats))).scalars()
|
||||
agent_ids: set[str] = set()
|
||||
for raw in rows:
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
seats = json.loads(raw) if isinstance(raw, str) else raw
|
||||
except Exception:
|
||||
continue
|
||||
if not isinstance(seats, list):
|
||||
continue
|
||||
for seat in seats:
|
||||
if isinstance(seat, dict) and isinstance(seat.get("agent_id"), str):
|
||||
agent_ids.add(seat["agent_id"])
|
||||
return agent_ids
|
||||
|
||||
# --------------------------------------------------------- materialize
|
||||
|
||||
async def sync_user(self, user_id: str) -> None:
|
||||
|
||||
@@ -1,33 +1,131 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from app.gateway.routers.agents import (
|
||||
_backfill_listing_cache,
|
||||
_load_config_extras,
|
||||
_write_agent_config,
|
||||
)
|
||||
from app.gateway.services import build_run_config
|
||||
from deerflow.config import paths as paths_module
|
||||
from deerflow.config.agents_config import AgentConfig
|
||||
from deerflow.persistence.agents import MemoryAgentStore
|
||||
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
import yaml
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from app.gateway.routers.agents import (
|
||||
_backfill_listing_cache,
|
||||
_load_config_extras,
|
||||
_write_agent_config,
|
||||
)
|
||||
from app.gateway.services import build_run_config
|
||||
from deerflow.config import paths as paths_module
|
||||
from deerflow.config.agents_config import AgentConfig
|
||||
from deerflow.persistence.agents import AgentRepository, MemoryAgentStore
|
||||
from deerflow.persistence.agents.model import AgentFavoriteRow, AgentRow
|
||||
from deerflow.persistence.base import Base
|
||||
from deerflow.persistence.positions.memory import MemoryPositionStore
|
||||
from deerflow.persistence.roundtable_chains.model import RoundtableChainRow
|
||||
from deerflow.persistence.tags.memory import MemoryTagStore
|
||||
from deerflow.services.position_sync import PositionSyncService
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def isolated_paths(tmp_path, monkeypatch):
|
||||
"""Redirect get_paths() at a fresh temp directory for the duration of one test."""
|
||||
monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
|
||||
monkeypatch.setattr(paths_module, "_paths", None)
|
||||
yield tmp_path
|
||||
monkeypatch.setattr(paths_module, "_paths", None)
|
||||
def isolated_paths(tmp_path, monkeypatch):
|
||||
"""Redirect get_paths() at a fresh temp directory for the duration of one test."""
|
||||
monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
|
||||
monkeypatch.setattr(paths_module, "_paths", None)
|
||||
yield tmp_path
|
||||
monkeypatch.setattr(paths_module, "_paths", None)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def sql_session_factory(tmp_path):
|
||||
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'agent-tests.db'}")
|
||||
async with engine.begin() as conn:
|
||||
# Importing PositionSyncService above registers the agent / skill /
|
||||
# scheduled-task rows it materializes into.
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
sf = async_sessionmaker(engine, expire_on_commit=False)
|
||||
try:
|
||||
yield sf
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
async def _add_sql_agent(sf, *, agent_id: str, user_id: str | None) -> None:
|
||||
async with sf() as session:
|
||||
session.add(
|
||||
AgentRow(
|
||||
id=agent_id,
|
||||
name=agent_id,
|
||||
description="",
|
||||
user_id=user_id,
|
||||
published=True,
|
||||
is_private=False,
|
||||
square_id="",
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def _add_sql_favorite(sf, *, user_id: str, agent_id: str, origin: str) -> None:
|
||||
async with sf() as session:
|
||||
session.add(
|
||||
AgentFavoriteRow(
|
||||
id=uuid4().hex,
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
origin=origin,
|
||||
position_id="default" if origin == "position" else None,
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def _add_sql_chain(sf, *, chain_id: str, agent_ids: list[str]) -> None:
|
||||
async with sf() as session:
|
||||
session.add(
|
||||
RoundtableChainRow(
|
||||
id=chain_id,
|
||||
user_id="chain-owner",
|
||||
title=chain_id,
|
||||
description="",
|
||||
seats=json.dumps(
|
||||
[{"agent_id": agent_id, "name": agent_id, "description": ""} for agent_id in agent_ids],
|
||||
ensure_ascii=False,
|
||||
),
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def _sql_favorite_rows(sf, user_id: str) -> set[tuple[str, str]]:
|
||||
async with sf() as session:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(AgentFavoriteRow.agent_id, AgentFavoriteRow.origin).where(
|
||||
AgentFavoriteRow.user_id == user_id
|
||||
)
|
||||
)
|
||||
).all()
|
||||
return {(agent_id, origin) for agent_id, origin in rows}
|
||||
|
||||
|
||||
async def _default_positions() -> MemoryPositionStore:
|
||||
positions = MemoryPositionStore()
|
||||
await positions.create_position(
|
||||
{
|
||||
"id": "default",
|
||||
"name": "Default",
|
||||
"description": "",
|
||||
"is_default": True,
|
||||
}
|
||||
)
|
||||
return positions
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_store_allows_duplicate_chinese_names_and_filters_visibility() -> None:
|
||||
store = MemoryAgentStore()
|
||||
chinese_name = "\u4e2d\u6587\u52a9\u624b"
|
||||
|
||||
async def test_agent_store_allows_duplicate_chinese_names_and_filters_visibility() -> None:
|
||||
store = MemoryAgentStore()
|
||||
chinese_name = "\u4e2d\u6587\u52a9\u624b"
|
||||
|
||||
await store.ensure_builtin({"id": "builtin-agent", "name": chinese_name, "description": "built in"})
|
||||
await store.create({"id": "private-a", "name": chinese_name, "description": "private", "user_id": "user-a"})
|
||||
await store.create({"id": "published-b", "name": chinese_name, "description": "published", "user_id": "user-b", "published": True})
|
||||
@@ -35,14 +133,144 @@ async def test_agent_store_allows_duplicate_chinese_names_and_filters_visibility
|
||||
user_a_ids = {agent["id"] for agent in await store.list_visible("user-a")}
|
||||
assert user_a_ids == {"builtin-agent", "private-a", "published-b"}
|
||||
|
||||
user_c_ids = {agent["id"] for agent in await store.list_visible("user-c")}
|
||||
assert user_c_ids == {"builtin-agent", "published-b"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_store_only_owner_can_update_or_delete() -> None:
|
||||
store = MemoryAgentStore()
|
||||
|
||||
user_c_ids = {agent["id"] for agent in await store.list_visible("user-c")}
|
||||
assert user_c_ids == {"builtin-agent", "published-b"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_position_sync_default_does_not_collect_builtin_agents(sql_session_factory) -> None:
|
||||
await _add_sql_agent(sql_session_factory, agent_id="builtin-auto", user_id=None)
|
||||
await _add_sql_agent(sql_session_factory, agent_id="builtin-manual", user_id=None)
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="builtin-auto",
|
||||
origin="position",
|
||||
)
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="builtin-manual",
|
||||
origin="user",
|
||||
)
|
||||
|
||||
sync = PositionSyncService(
|
||||
sql_session_factory,
|
||||
await _default_positions(),
|
||||
MemoryTagStore(),
|
||||
)
|
||||
await sync.sync_user("user-a")
|
||||
|
||||
assert await _sql_favorite_rows(sql_session_factory, "user-a") == {
|
||||
("builtin-manual", "user")
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_position_sync_tagged_agents_are_not_auto_favorited(sql_session_factory) -> None:
|
||||
await _add_sql_agent(sql_session_factory, agent_id="builtin-tagged", user_id=None)
|
||||
await _add_sql_agent(sql_session_factory, agent_id="custom-tagged", user_id="owner")
|
||||
await _add_sql_agent(sql_session_factory, agent_id="chain-tagged", user_id="owner")
|
||||
await _add_sql_chain(sql_session_factory, chain_id="chain-1", agent_ids=["chain-tagged"])
|
||||
|
||||
positions = await _default_positions()
|
||||
tags = MemoryTagStore()
|
||||
await tags.create_tag({"id": "agent-tag", "name": "Agent tag", "tag_type": "agent"})
|
||||
await tags.set_target_tags("agent", "builtin-tagged", ["agent-tag"])
|
||||
await tags.set_target_tags("agent", "custom-tagged", ["agent-tag"])
|
||||
await tags.set_target_tags("agent", "chain-tagged", ["agent-tag"])
|
||||
await positions.set_tag_bindings("default", {"agent": ["agent-tag"]})
|
||||
|
||||
sync = PositionSyncService(sql_session_factory, positions, tags)
|
||||
await sync.sync_user("user-a")
|
||||
|
||||
assert await _sql_favorite_rows(sql_session_factory, "user-a") == {
|
||||
("custom-tagged", "position")
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mine_scope_ignores_auto_agent_favorites_but_keeps_manual_builtin(sql_session_factory) -> None:
|
||||
await _add_sql_agent(sql_session_factory, agent_id="builtin-auto", user_id=None)
|
||||
await _add_sql_agent(sql_session_factory, agent_id="builtin-manual", user_id=None)
|
||||
await _add_sql_agent(sql_session_factory, agent_id="custom-auto", user_id="owner")
|
||||
await _add_sql_agent(sql_session_factory, agent_id="chain-auto", user_id="owner")
|
||||
await _add_sql_chain(sql_session_factory, chain_id="chain-1", agent_ids=["chain-auto"])
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="builtin-auto",
|
||||
origin="position",
|
||||
)
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="builtin-manual",
|
||||
origin="user",
|
||||
)
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="custom-auto",
|
||||
origin="position",
|
||||
)
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="chain-auto",
|
||||
origin="position",
|
||||
)
|
||||
|
||||
store = AgentRepository(sql_session_factory)
|
||||
rows, total = await store.list_paginated(
|
||||
user_id="user-a",
|
||||
scope="mine",
|
||||
search=None,
|
||||
tag_ids=None,
|
||||
page=1,
|
||||
page_size=20,
|
||||
)
|
||||
|
||||
assert total == 2
|
||||
assert {row["id"] for row in rows} == {"builtin-manual", "custom-auto"}
|
||||
assert await store.list_favorite_ids("user-a") == {"builtin-manual", "custom-auto"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_manual_favorite_upgrades_existing_auto_builtin_favorite(sql_session_factory) -> None:
|
||||
await _add_sql_agent(sql_session_factory, agent_id="builtin-agent", user_id=None)
|
||||
await _add_sql_favorite(
|
||||
sql_session_factory,
|
||||
user_id="user-a",
|
||||
agent_id="builtin-agent",
|
||||
origin="position",
|
||||
)
|
||||
|
||||
store = AgentRepository(sql_session_factory)
|
||||
assert await store.list_favorite_ids("user-a") == set()
|
||||
|
||||
await store.add_favorite("user-a", "builtin-agent")
|
||||
|
||||
assert await _sql_favorite_rows(sql_session_factory, "user-a") == {
|
||||
("builtin-agent", "user")
|
||||
}
|
||||
assert await store.list_favorite_ids("user-a") == {"builtin-agent"}
|
||||
rows, total = await store.list_paginated(
|
||||
user_id="user-a",
|
||||
scope="mine",
|
||||
search=None,
|
||||
tag_ids=None,
|
||||
page=1,
|
||||
page_size=20,
|
||||
)
|
||||
assert total == 1
|
||||
assert [row["id"] for row in rows] == ["builtin-agent"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_store_only_owner_can_update_or_delete() -> None:
|
||||
store = MemoryAgentStore()
|
||||
|
||||
await store.ensure_builtin({"id": "builtin-agent", "name": "Built in"})
|
||||
await store.create({"id": "private-a", "name": "Owned", "user_id": "user-a"})
|
||||
|
||||
|
||||
@@ -14,7 +14,9 @@ from fastapi import UploadFile
|
||||
from app.gateway.routers.ai_writing import _extract_sample_text
|
||||
from deerflow.agents.ai_writing.nodes import researcher as R
|
||||
from deerflow.agents.ai_writing.nodes import sample_analyzer as S
|
||||
from deerflow.agents.ai_writing.search import get_search_provider
|
||||
from deerflow.agents.ai_writing.search.base import SearchResult
|
||||
from deerflow.agents.ai_writing.search.intranet_stub import IntranetSearchProvider
|
||||
|
||||
|
||||
class _FakeIntranet:
|
||||
@@ -23,16 +25,34 @@ class _FakeIntranet:
|
||||
def __init__(self, **kw):
|
||||
self.errors = []
|
||||
self.raw_responses = []
|
||||
self.calls = []
|
||||
self.kw = kw
|
||||
self.verify_ssl = kw.get("verify_ssl", False)
|
||||
_FakeIntranet.instances.append(self)
|
||||
|
||||
async def search(self, query, max_results=5, **_):
|
||||
self.calls.append({"query": query, "status": "ok", "ok": True, "count": 1, "elapsed_ms": 12})
|
||||
return [SearchResult(
|
||||
title=f"内网-{query}", content="内网正文", source="ES",
|
||||
published_at="", url=f"http://es/{query}", source_type="intranet",
|
||||
)]
|
||||
|
||||
|
||||
class _EmptyIntranet(_FakeIntranet):
|
||||
async def search(self, query, max_results=5, **_):
|
||||
self.calls.append({"query": query, "status": "empty", "ok": True, "count": 0, "elapsed_ms": 9})
|
||||
return []
|
||||
|
||||
|
||||
class _FailingIntranet(_FakeIntranet):
|
||||
async def search(self, query, max_results=5, **_):
|
||||
msg = f"ConnectError: cannot connect for {query}"
|
||||
self.errors.append(msg)
|
||||
self.raw_responses.append("{\"success\": false, \"error\": \"cannot connect\"}")
|
||||
self.calls.append({"query": query, "status": "error", "ok": False, "count": 0, "elapsed_ms": 3, "error": msg})
|
||||
return []
|
||||
|
||||
|
||||
class _SkillProviderShouldNotRun:
|
||||
"""knowledge_base 模式不应触碰通用 provider —— 调到就让测试爆。"""
|
||||
errors: list = []
|
||||
@@ -42,29 +62,41 @@ class _SkillProviderShouldNotRun:
|
||||
raise AssertionError("knowledge_base 模式不应调用通用 provider / 技能检索")
|
||||
|
||||
|
||||
def _patch(monkeypatch, intranet_url="http://es/api"):
|
||||
class _FakeResponse:
|
||||
status_code = 200
|
||||
text = '[{"title":"内网标题","content":"内网正文","url":"https://intranet/doc"}]'
|
||||
|
||||
def raise_for_status(self) -> None:
|
||||
return None
|
||||
|
||||
def json(self):
|
||||
return [{"title": "内网标题", "content": "内网正文", "url": "https://intranet/doc"}]
|
||||
|
||||
|
||||
def _patch(monkeypatch, intranet_url="http://es/api", intranet_cls=_FakeIntranet, intranet_verify_ssl=False):
|
||||
_FakeIntranet.instances = []
|
||||
cfg = {
|
||||
"max_materials": 20,
|
||||
"intranet_search_url": intranet_url,
|
||||
"intranet_knowledge_base": "samples",
|
||||
"intranet_verify_ssl": intranet_verify_ssl,
|
||||
}
|
||||
monkeypatch.setattr(R, "get_ai_writing_config", lambda: cfg)
|
||||
monkeypatch.setattr(R, "use_builtin_agent", lambda *_: False)
|
||||
monkeypatch.setattr(R, "safe_get_stream_writer", lambda: (lambda *a, **k: None))
|
||||
monkeypatch.setattr(R, "use_skill_search_agent", lambda: False)
|
||||
|
||||
async def fake_llm_json(*a, **k):
|
||||
async def fake_stream_llm_json(*a, **k):
|
||||
return {"keywords": ["特朗普竞选"]}
|
||||
|
||||
monkeypatch.setattr(R, "llm_json", fake_llm_json)
|
||||
monkeypatch.setattr(R, "_stream_llm_json", fake_stream_llm_json)
|
||||
# knowledge_base 模式不该加载/编排技能,但仍打桩以防意外路径
|
||||
monkeypatch.setattr(R, "_load_configured_skills",
|
||||
lambda: [SimpleNamespace(name="knowledge-search-v2", description="内网检索")])
|
||||
monkeypatch.setattr(R, "get_search_provider", lambda _cfg: _SkillProviderShouldNotRun())
|
||||
monkeypatch.setattr(
|
||||
"deerflow.agents.ai_writing.search.intranet_stub.IntranetSearchProvider",
|
||||
_FakeIntranet,
|
||||
intranet_cls,
|
||||
)
|
||||
|
||||
|
||||
@@ -89,6 +121,14 @@ async def test_knowledge_base_hits_intranet_directly(monkeypatch):
|
||||
assert _FakeIntranet.instances, "应实例化 IntranetSearchProvider 直连内网接口"
|
||||
# 走素材确认暂停点
|
||||
assert out["status"] == "awaiting_material_confirm"
|
||||
dbg = out["material_package"].get("intranet_debug")
|
||||
assert dbg["mode"] == "knowledge_base"
|
||||
assert dbg["status"] == "ok"
|
||||
assert dbg["endpoint"] == "http://es/api"
|
||||
assert dbg["result_count"] == 1
|
||||
assert dbg["queries"][0]["query"] == "特朗普竞选"
|
||||
assert dbg["queries"][0]["count"] == 1
|
||||
assert dbg["calls"][0]["status"] == "ok"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -104,6 +144,15 @@ async def test_knowledge_base_no_url_falls_back_to_default_endpoint(monkeypatch)
|
||||
assert _FakeIntranet.instances[0].kw.get("base_url") == "", "空地址应原样下传,由 provider 回退默认端点"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_knowledge_base_passes_verify_ssl_config(monkeypatch):
|
||||
_patch(monkeypatch, intranet_verify_ssl=True)
|
||||
out = await R.researcher_node(_state(), {"configurable": {}})
|
||||
|
||||
assert _FakeIntranet.instances[0].kw.get("verify_ssl") is True
|
||||
assert out["material_package"]["intranet_debug"]["verify_ssl"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_knowledge_base_does_not_orchestrate_skills(monkeypatch):
|
||||
"""knowledge_base 不编排技能:通用 provider 一旦被调用就会 raise。"""
|
||||
@@ -122,7 +171,7 @@ async def test_sample_analyzer_skips_normal_mode():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sample_analyzer_builds_style_profile(monkeypatch):
|
||||
async def fake_llm_json(*_args, **_kwargs):
|
||||
async def fake_stream_sample_analysis(*_args, **_kwargs):
|
||||
return {
|
||||
"summary": "短段落、评论式表达",
|
||||
"tone": "克制、坚定",
|
||||
@@ -136,7 +185,7 @@ async def test_sample_analyzer_builds_style_profile(monkeypatch):
|
||||
"sample_outline": ["问题提出", "原因分析"],
|
||||
}
|
||||
|
||||
monkeypatch.setattr(S, "llm_json", fake_llm_json)
|
||||
monkeypatch.setattr(S, "_stream_sample_analysis", fake_stream_sample_analysis)
|
||||
|
||||
out = await S.sample_analyzer_node({
|
||||
"writing_mode_type": "imitate",
|
||||
@@ -183,3 +232,75 @@ async def test_extract_sample_text_reads_txt_upload():
|
||||
|
||||
assert name == "sample.txt"
|
||||
assert text == "样文内容"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_knowledge_base_empty_result_is_visible(monkeypatch):
|
||||
"""接口成功但 0 条也要在返回里能看出来,而不是只显示"无素材"。"""
|
||||
_patch(monkeypatch, intranet_cls=_EmptyIntranet)
|
||||
out = await R.researcher_node(_state(), {"configurable": {}})
|
||||
pkg = out["material_package"]
|
||||
assert pkg["materials"] == []
|
||||
dbg = pkg.get("intranet_debug")
|
||||
assert dbg["status"] == "empty"
|
||||
assert dbg["result_count"] == 0
|
||||
assert dbg["queries"][0]["count"] == 0
|
||||
assert dbg["calls"][0]["status"] == "empty"
|
||||
steps = pkg["skill_search_steps"]
|
||||
assert any(step["type"] == "skill_result" and "0 条" in step["message"] for step in steps)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_knowledge_base_failure_is_visible(monkeypatch):
|
||||
"""provider 吞掉异常返回 [] 时,前端仍应看到报错步骤与 notice。"""
|
||||
_patch(monkeypatch, intranet_cls=_FailingIntranet)
|
||||
out = await R.researcher_node(_state(), {"configurable": {}})
|
||||
pkg = out["material_package"]
|
||||
dbg = pkg.get("intranet_debug")
|
||||
assert dbg["status"] == "error"
|
||||
assert dbg["errors"]
|
||||
assert dbg["raw_responses"]
|
||||
steps = pkg["skill_search_steps"]
|
||||
assert any(step["type"] == "skill_error" for step in steps)
|
||||
assert any(event["type"] == "notice" and event.get("raw") for event in out["progress_events"])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_intranet_search_disables_ssl_verification_for_internal_https(monkeypatch):
|
||||
import httpx
|
||||
|
||||
seen: dict = {}
|
||||
|
||||
class _FakeAsyncClient:
|
||||
def __init__(self, **kwargs):
|
||||
seen.update(kwargs)
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return None
|
||||
|
||||
async def post(self, endpoint, json):
|
||||
seen["endpoint"] = endpoint
|
||||
seen["payload"] = json
|
||||
return _FakeResponse()
|
||||
|
||||
monkeypatch.setattr(httpx, "AsyncClient", _FakeAsyncClient)
|
||||
|
||||
provider = IntranetSearchProvider(base_url="https://intranet.example/search", knowledge_base_name="kb", timeout=300)
|
||||
results = await provider.search("测试", max_results=1)
|
||||
|
||||
assert results
|
||||
assert seen["verify"] is False
|
||||
assert seen["trust_env"] is False
|
||||
assert seen["endpoint"] == "https://intranet.example/search"
|
||||
assert seen["payload"]["knowledge_base_name"] == "kb"
|
||||
assert provider.calls[0]["verify_ssl"] is False
|
||||
|
||||
|
||||
def test_intranet_verify_ssl_can_be_reenabled_from_config():
|
||||
provider = get_search_provider({"search_provider": "intranet", "intranet_verify_ssl": "true"})
|
||||
|
||||
assert isinstance(provider, IntranetSearchProvider)
|
||||
assert provider.verify_ssl is True
|
||||
@@ -0,0 +1,199 @@
|
||||
"""Tests for per-model concurrency limit settings and enforcement."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from app.gateway.routers import system_settings as ss_router
|
||||
from app.gateway.services import (
|
||||
_get_model_concurrency_limit,
|
||||
_resolve_effective_run_model_name,
|
||||
)
|
||||
from deerflow.config import system_settings as ss
|
||||
from deerflow.runtime.runs.manager import ModelConcurrencyLimitError, RunManager
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_runtime_home(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path:
|
||||
def fake_home() -> Path:
|
||||
return tmp_path
|
||||
|
||||
monkeypatch.setattr(ss, "runtime_home", fake_home)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _patch_admin(monkeypatch: pytest.MonkeyPatch, user: object | None = None) -> None:
|
||||
admin = user or SimpleNamespace(
|
||||
id="admin1",
|
||||
system_role="admin",
|
||||
email="admin@example.com",
|
||||
)
|
||||
|
||||
async def _optional(_request):
|
||||
return admin
|
||||
|
||||
monkeypatch.setattr(ss_router, "get_optional_user_from_request", _optional)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_model_concurrency_settings_round_trip(
|
||||
tmp_runtime_home: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
_patch_admin(monkeypatch)
|
||||
req = SimpleNamespace()
|
||||
|
||||
saved = await ss_router.update_model_concurrency(
|
||||
ss_router.ModelConcurrencyUpdateRequest(
|
||||
enabled=True,
|
||||
default_limit=8,
|
||||
model_limits={"glm5": 10, "deepseek": 4},
|
||||
),
|
||||
req,
|
||||
)
|
||||
assert saved.enabled is True
|
||||
assert saved.default_limit == 8
|
||||
assert saved.model_limits == {"glm5": 10, "deepseek": 4}
|
||||
|
||||
file_path = tmp_runtime_home / "system_settings.json"
|
||||
payload = json.loads(file_path.read_text(encoding="utf-8"))
|
||||
assert payload["model_concurrency"]["enabled"] is True
|
||||
assert payload["model_concurrency"]["default_limit"] == 8
|
||||
assert payload["model_concurrency"]["model_limits"]["glm5"] == 10
|
||||
|
||||
loaded = await ss_router.get_model_concurrency(req)
|
||||
assert loaded.enabled is True
|
||||
assert loaded.default_limit == 8
|
||||
assert loaded.model_limits["deepseek"] == 4
|
||||
|
||||
|
||||
def test_resolve_effective_run_model_name_prefers_requested_model(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
app_config = SimpleNamespace(
|
||||
models=[SimpleNamespace(name="default-model")],
|
||||
get_model_config=lambda name: {"name": name} if name in {"default-model", "glm5", "agent-model"} else None,
|
||||
)
|
||||
monkeypatch.setattr("app.gateway.services.get_app_config", lambda: app_config)
|
||||
monkeypatch.setattr(
|
||||
"app.gateway.services.load_agent_config",
|
||||
lambda _agent_id: SimpleNamespace(model="agent-model"),
|
||||
)
|
||||
|
||||
body = SimpleNamespace(
|
||||
assistant_id="custom-agent",
|
||||
context={"model_name": "glm5"},
|
||||
config=None,
|
||||
)
|
||||
assert _resolve_effective_run_model_name(body) == "glm5"
|
||||
|
||||
|
||||
def test_resolve_effective_run_model_name_uses_agent_default_when_request_missing(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
app_config = SimpleNamespace(
|
||||
models=[SimpleNamespace(name="default-model")],
|
||||
get_model_config=lambda name: {"name": name} if name in {"default-model", "agent-model"} else None,
|
||||
)
|
||||
monkeypatch.setattr("app.gateway.services.get_app_config", lambda: app_config)
|
||||
monkeypatch.setattr(
|
||||
"app.gateway.services.load_agent_config",
|
||||
lambda _agent_id: SimpleNamespace(model="agent-model"),
|
||||
)
|
||||
|
||||
body = SimpleNamespace(
|
||||
assistant_id="custom-agent",
|
||||
context={},
|
||||
config=None,
|
||||
)
|
||||
assert _resolve_effective_run_model_name(body) == "agent-model"
|
||||
|
||||
|
||||
def test_resolve_effective_run_model_name_uses_context_agent_id(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
app_config = SimpleNamespace(
|
||||
models=[SimpleNamespace(name="default-model")],
|
||||
get_model_config=lambda name: {"name": name} if name in {"default-model", "agent-model"} else None,
|
||||
)
|
||||
monkeypatch.setattr("app.gateway.services.get_app_config", lambda: app_config)
|
||||
monkeypatch.setattr(
|
||||
"app.gateway.services.load_agent_config",
|
||||
lambda agent_id: SimpleNamespace(model="agent-model") if agent_id == "custom-agent" else SimpleNamespace(model="default-model"),
|
||||
)
|
||||
|
||||
body = SimpleNamespace(
|
||||
assistant_id="lead_agent",
|
||||
context={"agent_id": "custom-agent"},
|
||||
config=None,
|
||||
)
|
||||
assert _resolve_effective_run_model_name(body) == "agent-model"
|
||||
|
||||
|
||||
def test_resolve_effective_run_model_name_context_agent_overrides_assistant_id(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
app_config = SimpleNamespace(
|
||||
models=[SimpleNamespace(name="default-model")],
|
||||
get_model_config=lambda name: {"name": name} if name in {"default-model", "context-model", "assistant-model"} else None,
|
||||
)
|
||||
monkeypatch.setattr("app.gateway.services.get_app_config", lambda: app_config)
|
||||
monkeypatch.setattr(
|
||||
"app.gateway.services.load_agent_config",
|
||||
lambda agent_id: SimpleNamespace(model="context-model" if agent_id == "context-agent" else "assistant-model"),
|
||||
)
|
||||
|
||||
body = SimpleNamespace(
|
||||
assistant_id="assistant-agent",
|
||||
context={"agent_id": "context-agent"},
|
||||
config=None,
|
||||
)
|
||||
assert _resolve_effective_run_model_name(body) == "context-model"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_manager_rejects_model_limit_atomically(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setattr(
|
||||
"app.gateway.services.load_system_settings",
|
||||
lambda: SimpleNamespace(
|
||||
model_concurrency=SimpleNamespace(
|
||||
enabled=True,
|
||||
default_limit=5,
|
||||
model_limits={"glm5": 2},
|
||||
)
|
||||
),
|
||||
)
|
||||
assert _get_model_concurrency_limit("glm5") == 2
|
||||
|
||||
run_mgr = RunManager()
|
||||
await run_mgr.create_or_reject("t1", metadata={"model_name": "glm5"})
|
||||
await run_mgr.create_or_reject("t2", metadata={"model_name": "glm5"})
|
||||
|
||||
with pytest.raises(ModelConcurrencyLimitError):
|
||||
await run_mgr.create_or_reject(
|
||||
"t3",
|
||||
metadata={"model_name": "glm5"},
|
||||
model_concurrency_limit=("glm5", 2),
|
||||
)
|
||||
|
||||
|
||||
def test_get_model_concurrency_limit_skips_when_disabled(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setattr(
|
||||
"app.gateway.services.load_system_settings",
|
||||
lambda: SimpleNamespace(
|
||||
model_concurrency=SimpleNamespace(
|
||||
enabled=False,
|
||||
default_limit=1,
|
||||
model_limits={"glm5": 1},
|
||||
)
|
||||
),
|
||||
)
|
||||
assert _get_model_concurrency_limit("glm5") is None
|
||||
@@ -0,0 +1,166 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from deerflow.agents.middlewares.title_middleware import (
|
||||
TitleMiddleware,
|
||||
finalize_thread_title,
|
||||
)
|
||||
|
||||
|
||||
class _FakeCheckpointer:
|
||||
def __init__(self, checkpoint: dict, metadata: dict | None = None):
|
||||
self._tuple = SimpleNamespace(
|
||||
checkpoint=checkpoint,
|
||||
metadata=metadata or {},
|
||||
config={"configurable": {"thread_id": "thread-1", "checkpoint_ns": ""}},
|
||||
)
|
||||
self.aput_calls: list[tuple[dict, dict, dict, dict]] = []
|
||||
self._version = 0
|
||||
|
||||
async def aget_tuple(self, _config):
|
||||
return self._tuple
|
||||
|
||||
def get_next_version(self, _current, _channel):
|
||||
self._version += 1
|
||||
return self._version
|
||||
|
||||
async def aput(self, config, checkpoint, metadata, versions):
|
||||
self.aput_calls.append((config, checkpoint, metadata, versions))
|
||||
self._tuple = SimpleNamespace(
|
||||
checkpoint=checkpoint,
|
||||
metadata=metadata,
|
||||
config={
|
||||
"configurable": {
|
||||
"thread_id": config["configurable"]["thread_id"],
|
||||
"checkpoint_ns": config["configurable"]["checkpoint_ns"],
|
||||
"checkpoint_id": "new-ckpt",
|
||||
}
|
||||
},
|
||||
)
|
||||
return self._tuple.config
|
||||
|
||||
|
||||
class _FakeThreadStore:
|
||||
def __init__(self):
|
||||
self.updated: list[tuple[str, str, str | None]] = []
|
||||
|
||||
async def update_display_name(self, thread_id: str, title: str, *, user_id=None):
|
||||
self.updated.append((thread_id, title, user_id))
|
||||
|
||||
|
||||
def _source_state() -> dict:
|
||||
return {
|
||||
"messages": [
|
||||
SimpleNamespace(type="human", content="please analyze the first topic"),
|
||||
SimpleNamespace(type="ai", content="this is the first answer"),
|
||||
],
|
||||
"title": "please analyze the first topic",
|
||||
"title_provisional": True,
|
||||
}
|
||||
|
||||
|
||||
def _dict_source_state() -> dict:
|
||||
return {
|
||||
"messages": [
|
||||
{"type": "human", "content": "please analyze the first topic"},
|
||||
{"type": "ai", "content": "this is the first answer"},
|
||||
],
|
||||
"title": "please analyze the first topic",
|
||||
"title_provisional": True,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finalize_thread_title_updates_checkpoint_and_display_name(monkeypatch) -> None:
|
||||
async def _fake_generate(self, state):
|
||||
assert state["title_provisional"] is True
|
||||
return {"title": "first topic analysis", "title_provisional": False}
|
||||
|
||||
monkeypatch.setattr(TitleMiddleware, "_agenerate_title_result", _fake_generate)
|
||||
|
||||
checkpointer = _FakeCheckpointer({"channel_values": _source_state()})
|
||||
thread_store = _FakeThreadStore()
|
||||
|
||||
updated = await finalize_thread_title(
|
||||
checkpointer=checkpointer,
|
||||
thread_store=thread_store,
|
||||
thread_id="thread-1",
|
||||
source_state=_source_state(),
|
||||
)
|
||||
|
||||
assert updated is True
|
||||
assert len(checkpointer.aput_calls) == 1
|
||||
_, written_checkpoint, _, new_versions = checkpointer.aput_calls[0]
|
||||
assert written_checkpoint["channel_values"]["title"] == "first topic analysis"
|
||||
assert written_checkpoint["channel_values"]["title_provisional"] is False
|
||||
assert set(new_versions) == {"title", "title_provisional"}
|
||||
assert thread_store.updated == [("thread-1", "first topic analysis", None)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finalize_thread_title_skips_if_current_thread_already_finalized(monkeypatch) -> None:
|
||||
async def _fake_generate(self, state):
|
||||
return {"title": "first topic analysis", "title_provisional": False}
|
||||
|
||||
monkeypatch.setattr(TitleMiddleware, "_agenerate_title_result", _fake_generate)
|
||||
|
||||
checkpointer = _FakeCheckpointer(
|
||||
{
|
||||
"channel_values": {
|
||||
**_source_state(),
|
||||
"title": "manually renamed title",
|
||||
"title_provisional": False,
|
||||
}
|
||||
}
|
||||
)
|
||||
thread_store = _FakeThreadStore()
|
||||
|
||||
updated = await finalize_thread_title(
|
||||
checkpointer=checkpointer,
|
||||
thread_store=thread_store,
|
||||
thread_id="thread-1",
|
||||
source_state=_source_state(),
|
||||
)
|
||||
|
||||
assert updated is False
|
||||
assert checkpointer.aput_calls == []
|
||||
assert thread_store.updated == []
|
||||
|
||||
|
||||
def test_sync_after_model_keeps_provisional_title_for_background_finalize() -> None:
|
||||
middleware = TitleMiddleware()
|
||||
|
||||
result = middleware.after_model(_source_state(), runtime=SimpleNamespace())
|
||||
|
||||
assert result is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_after_model_keeps_provisional_title_for_background_finalize() -> None:
|
||||
middleware = TitleMiddleware()
|
||||
|
||||
result = await middleware.aafter_model(_source_state(), runtime=SimpleNamespace())
|
||||
|
||||
assert result is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finalize_thread_title_accepts_serialized_dict_messages(monkeypatch) -> None:
|
||||
async def _fake_generate(self, state):
|
||||
return {"title": "first topic analysis", "title_provisional": False}
|
||||
|
||||
monkeypatch.setattr(TitleMiddleware, "_agenerate_title_result", _fake_generate)
|
||||
|
||||
checkpointer = _FakeCheckpointer({"channel_values": _dict_source_state()})
|
||||
|
||||
updated = await finalize_thread_title(
|
||||
checkpointer=checkpointer,
|
||||
thread_store=None,
|
||||
thread_id="thread-1",
|
||||
source_state=_dict_source_state(),
|
||||
)
|
||||
|
||||
assert updated is True
|
||||
Reference in new issue
Block a user