禁用了知识库技能，添加了知识库memory

2026-06-08 18:35:05 +08:00
1 parent 273503ddb9
commit 9e14c2e289
@@ -187,6 +187,10 @@ tools:
 sandbox:
  use: deerflow.sandbox.local:LocalSandboxProvider
  allow_host_bash: true
+  mounts:
+    - host_path: C:\Users\iie\Desktop\deerflow-new\offline-backend-20260512\backend\knowledge\knowledge_base
+      container_path: /mnt/knowledge
+      read_only: true

 skills:
  path: skills
@@ -6,7 +6,7 @@
    "code-retrieval": { "enabled": true },
    "news-briefing": { "enabled": true },
    "workspace-search": { "enabled": true },
-    "policy-knowledge": { "enabled": true },
+    "policy-knowledge": { "enabled": false },
    "html-page-builder": { "enabled": true },
    "html-charts": { "enabled": true }
  }
@@ -0,0 +1,124 @@
+# Knowledge Base Routing
+
+This file is the lightweight session-start routing memory for the policy knowledge base.
+
+It may be injected at conversation start. It does not replace the caller skill. The caller skill owns the final workflow and output format; this file only points to the smallest useful knowledge entry point.
+
+## Root
+
+Knowledge root:
+
+```text
+/mnt/knowledge
+```
+
+Repository source path:
+
+```text
+offline-backend-20260512/backend/knowledge/knowledge_base
+```
+
+## Core Principles
+
+Use the knowledge base as a reusable substrate:
+
+1. Use this file to choose the first manifest, rule, or routing file.
+2. Read only the files needed for the current task.
+3. Do not force the caller skill into a full policy-analysis workflow unless the caller task needs it.
+4. Prefer manifests before reading many files.
+5. Follow the caller skill's output structure; use knowledge files only as supporting evidence.
+
+## Scope
+
+Use this knowledge base for Taiwan elections, Cross-Strait relations, US/Japan-Taiwan relations, political rhetoric, public-opinion events, policy signals, party dynamics, influence analysis, and scenario planning.
+
+Do not use it for unrelated technical, lifestyle, economic, legal, medical, or private-person questions. Do not use it to produce political propaganda, campaign messaging, attacks, doxxing, private-communication analysis, or unverified allegations.
+
+For unclear tasks, ask a focused clarification question before reading many files.
+
+## Core Rules
+
+Read these rule files only when needed:
+
+```text
+rules/knowledge_usage.md
+rules/missing_and_placeholder.md
+rules/citation_and_evidence.md
+rules/realtime_news_usage.md
+rules/neutrality_and_perspective.md
+```
+
+When to read:
+
+- `rules/knowledge_usage.md`: before using knowledge files for a non-trivial task, or when unsure how much to read.
+- `rules/missing_and_placeholder.md`: when a file is missing, marked `[占位]`, or the knowledge base appears incomplete.
+- `rules/citation_and_evidence.md`: when the answer uses knowledge-base material as evidence.
+- `rules/realtime_news_usage.md`: when the task needs recent events, news facts, public-opinion developments, or source verification.
+- `rules/neutrality_and_perspective.md`: when the task involves political controversy, competing claims, attribution, blame, legitimacy, or sensitive issues.
+
+## Directory Index
+
+Use `_MANIFEST.md` files as directory-level indexes before reading many files.
+
+```text
+entities/_MANIFEST.md       # people, parties, institutions, media, foreign actors
+narratives/_MANIFEST.md     # political narratives and discourse frames
+events/_MANIFEST.md         # key events, timelines, recurring patterns
+primitives/_MANIFEST.md     # concepts, frameworks, methods
+playbooks/_MANIFEST.md      # analysis methods
+reference/_MANIFEST.md      # reference tables and standards
+data_pointers/_MANIFEST.md  # external/current material notes
+routing/_MANIFEST.md        # legacy detailed routing files
+```
+
+## Route By Task
+
+Start with these broad entry points, then narrow through manifests:
+
+```text
+Background or fact lookup      -> reference/_MANIFEST.md, events/_MANIFEST.md, entities/_MANIFEST.md
+Speech or rhetoric analysis    -> playbooks/_MANIFEST.md, narratives/_MANIFEST.md, reference/_MANIFEST.md
+Cross-Strait signal analysis   -> playbooks/_MANIFEST.md, narratives/_MANIFEST.md, events/_MANIFEST.md
+US/Taiwan/international signal -> playbooks/_MANIFEST.md, narratives/_MANIFEST.md, entities/foreign_actors/_MANIFEST.md
+Election or party analysis     -> playbooks/_MANIFEST.md, entities/_MANIFEST.md, narratives/domestic/_MANIFEST.md
+Narrative or influence analysis -> playbooks/_MANIFEST.md, narratives/_MANIFEST.md, primitives/_MANIFEST.md
+Prediction or scenarios        -> playbooks/_MANIFEST.md, events/_MANIFEST.md, narratives/_MANIFEST.md
+```
+
+Do not use a playbook for simple background lookup unless the user asks for analysis, interpretation, comparison, prediction, or attribution.
+
+For multi-aspect tasks, read `playbooks/multi_playbook_orchestration.md` only after confirming that one playbook is not enough.
+
+## Legacy Detailed Routing
+
+If this lightweight routing is not specific enough, use the legacy routing files:
+
+```text
+routing/by_question_type.md
+routing/by_topic.md
+routing/by_actor.md
+routing/by_event.md
+routing/cross_reference_rules.md
+```
+
+These are more detailed and may point to specific entities, narratives, events, primitives, and playbooks. Use them only when manifests are not specific enough.
+
+## Realtime Or Recent Facts
+
+The knowledge base may be stale. For recent events, news facts, public-opinion developments, policy updates, or source verification:
+
+1. Read `rules/realtime_news_usage.md`.
+2. Use `bm25_search_news` if available.
+3. Keep knowledge-base background separate from current news evidence.
+
+## Minimal Use Pattern
+
+```text
+1. Identify whether the caller task needs knowledge-base support.
+2. Read the relevant rule file if needed.
+3. Use this routing file or a `_MANIFEST.md` to pick a small set of files.
+4. Read only those files.
+5. If a playbook is useful, read it; otherwise use knowledge files as supporting material.
+6. Cite specific files when using knowledge-base material.
+7. Return control to the caller skill's workflow and output format.
+```
@@ -0,0 +1,66 @@
+# Citation And Evidence Rules
+
+Use this file when knowledge-base material supports an answer, analysis, decision, or recommendation.
+
+## Core Rule
+
+When using knowledge-base content, cite or name the specific file. Do not write only "the knowledge base says".
+
+## Evidence Types
+
+Keep these categories separate:
+
+- **Knowledge-base background**: local Markdown files under `/mnt/knowledge`.
+- **Current news evidence**: results from `bm25_search_news` or user-provided current materials.
+- **User-provided material**: documents, prompts, screenshots, URLs, or files supplied in the conversation.
+- **Model inference**: reasoning based on the above sources.
+
+## How To Refer To Knowledge Files
+
+For concise answers, a file-level reference is enough:
+
+```markdown
+依据 `entities/parties/dpp_overview.md` 和 `events/key_events/event_2024_election_three_way.md`，...
+```
+
+For higher-stakes or contested claims, be more specific:
+
+```markdown
+这一点来自 `playbooks/uncertainty_communication_protocol.md` 中关于信心度表达的规则。
+```
+
+## Do Not Over-Cite
+
+Do not turn every sentence into a citation. Cite when the file materially supports:
+
+- A factual statement.
+- A definition.
+- A timeline point.
+- A framework choice.
+- A confidence or uncertainty judgment.
+- A disputed interpretation.
+
+## Claims And Support
+
+Use this discipline:
+
+```text
+Fact      -> needs a file, news source, or user material.
+Context   -> can use knowledge-base background.
+Inference -> must be labeled as inference.
+Forecast  -> must include uncertainty.
+Gap       -> must be explicitly marked.
+```
+
+## Conflict Handling
+
+If files conflict:
+
+1. State the conflict.
+2. Prefer more specific, more recent, or higher-credibility material.
+3. If recency matters, use `bm25_search_news`.
+4. Mark residual uncertainty.
+
+## Realtime News Links
+
+For news-search results, use visible Markdown links in the final answer when available. Do not replace inline evidence with a bare source list at the end.
@@ -0,0 +1,75 @@
+# Knowledge Usage Rules
+
+Use this file when a caller skill needs material from the policy knowledge base.
+
+## Role Of The Knowledge Base
+
+The knowledge base is a substrate for other skills. It provides:
+
+- File navigation.
+- Background concepts.
+- Actor, event, narrative, and policy context.
+- Optional domain playbooks.
+- Evidence and source pointers.
+
+It does not automatically own the final answer structure. If the caller skill has its own workflow or output format, follow the caller skill and use knowledge-base files as supporting material.
+
+## Reading Discipline
+
+1. Start from `routing.md`, a relevant `_MANIFEST.md`, or a legacy routing file.
+2. Read only files that are relevant to the current task.
+3. Do not read the whole knowledge base.
+4. For large directories, read `_MANIFEST.md` before opening individual files.
+5. For long files, read the relevant section when possible instead of consuming the entire file.
+6. Prefer existing local files and local helper APIs over ad hoc guessing.
+
+## When To Read A Playbook
+
+Read a playbook when the task needs a domain-specific analysis method, such as:
+
+- Signal interpretation.
+- Scenario planning.
+- Narrative attribution.
+- Party strategy analysis.
+- Escalation risk assessment.
+- Election or voter-flow analysis.
+
+Do not force a playbook for simple background lookup or when the caller skill already provides a complete method.
+
+## When To Read Knowledge Files Without A Playbook
+
+Use knowledge files directly when the caller only needs:
+
+- A factual background note.
+- Actor context.
+- Event timeline context.
+- A narrative definition.
+- A concept or framework definition.
+- A reference table.
+
+In these cases, read the relevant `_MANIFEST.md` and concrete files, then return the information in the caller skill's format.
+
+## File Priority
+
+When unsure, use this order:
+
+```text
+routing.md
+  -> rules/*.md as needed
+  -> relevant _MANIFEST.md
+  -> relevant playbook if method is needed
+  -> concrete entities / narratives / events / primitives / reference files
+```
+
+## Separation From Current Events
+
+Knowledge files may be stale. Treat them as background and analytical context. Use `bm25_search_news` or user-provided current materials for recent facts.
+
+## Output Integration
+
+When using knowledge-base material:
+
+- Mention the specific file when it materially supports a claim.
+- Do not say only "the knowledge base says".
+- Preserve the caller skill's final format.
+- Mark uncertainty if the knowledge base does not cover a key point.
@@ -0,0 +1,47 @@
+# Missing And Placeholder Handling
+
+Use this file when a referenced knowledge file is missing, marked `[占位]`, incomplete, or not specific enough for the task.
+
+## Core Rule
+
+Do not invent missing knowledge-base content.
+
+If a file does not exist or is marked `[占位]`, treat that part of the knowledge base as unavailable.
+
+## What To Do
+
+1. Stop trying to repeatedly read the same missing file.
+2. Check the nearest `_MANIFEST.md` or related routing file for alternatives.
+3. Use adjacent existing files only when they genuinely cover the issue.
+4. Clearly state what the knowledge base does not cover.
+5. Mark uncertainty when the missing material affects the answer.
+6. If current facts are needed, use `bm25_search_news` or user-provided materials instead of guessing.
+
+## Recommended Wording
+
+```markdown
+知识库中关于【具体议题/人物/事件】的专门文件尚未建设；以下仅基于已存在的相关文件，结论置信度相应降低。
+```
+
+or:
+
+```markdown
+The knowledge base does not currently cover this point directly. I am using adjacent background files only as context, not as direct evidence.
+```
+
+## Do Not
+
+- Do not fabricate the contents of a missing file.
+- Do not cite a missing or placeholder file as evidence.
+- Do not hide the gap if it matters to the conclusion.
+- Do not treat adjacent files as direct support unless they directly address the point.
+
+## Fallback Pattern
+
+```text
+1. Identify the missing or placeholder file.
+2. Read the directory _MANIFEST.md.
+3. Read one or two adjacent files only if relevant.
+4. Mark the coverage gap.
+5. Continue with reduced confidence or ask the caller/user for more material.
+```
@@ -0,0 +1,59 @@
+# Neutrality And Perspective Rules
+
+Use this file when a task involves political controversy, competing claims, partisan conflict, attribution, blame, legitimacy, public opinion, or sensitive cross-Strait and international issues.
+
+## Core Rule
+
+Keep analysis separate from advocacy. Explain positions, incentives, evidence, and uncertainty without endorsing a political side or turning the answer into persuasion, mobilization, attack, or campaign messaging.
+
+## Neutral Language
+
+- Use descriptive terms instead of evaluative labels.
+- Attribute contested claims to the actor making them.
+- Quote loaded language only when it is evidence; do not adopt it as the analyst's voice.
+- Avoid moralizing words such as "good", "bad", "correct", "betrayal", "patriotic", or "evil" unless describing a quoted claim.
+- Do not write slogans, attack lines, campaign copy, or mobilizing calls to action.
+
+## Multiple Perspectives
+
+For contested issues, include the materially relevant perspectives needed to understand the dispute. Depending on the task, these may include:
+
+- Major parties or factions.
+- Government institutions.
+- Beijing, Taipei, Washington, Tokyo, or other foreign actors.
+- Media, civil society, voters, affected communities, or policy experts.
+
+Do not force artificial balance when evidence quality is uneven. If one side has stronger evidence, say so and explain why.
+
+## Facts, Interpretations, And Inference
+
+Keep these layers distinct:
+
+```text
+Fact           -> what is directly supported by sources.
+Position       -> what an actor says or claims.
+Interpretation -> how different actors frame the issue.
+Inference      -> the analyst's reasoning from evidence.
+Forecast       -> a forward-looking judgment with uncertainty.
+```
+
+Do not present an actor's framing as fact. Do not present inference as direct evidence.
+
+## Sensitive Claims
+
+For allegations, attribution, intent, coordination, corruption, foreign influence, or personal conduct:
+
+1. Require explicit evidence from knowledge files, current news, official records, or user-provided materials.
+2. Identify who made the claim and what evidence supports it.
+3. Mark uncertainty when evidence is incomplete, indirect, contested, or outdated.
+4. Avoid naming private persons unless the task and evidence justify it.
+
+## Minimal Output Check
+
+Before answering, check:
+
+- Are facts, positions, interpretations, and inferences separated?
+- Are contested claims attributed?
+- Are materially relevant perspectives included?
+- Is uncertainty marked where evidence is limited?
+- Is the answer analytical rather than persuasive or mobilizing?
@@ -0,0 +1,72 @@
+# Realtime News Usage
+
+Use this file when a task needs recent facts, current public-opinion developments, event timelines, policy updates, or source verification.
+
+## Core Rule
+
+The knowledge base is background context. It may be stale. Current facts require current material.
+
+If `bm25_search_news` is available, use it for:
+
+- Recent events.
+- News facts.
+- Public-opinion controversies.
+- People or institution updates.
+- Policy developments.
+- Source checking.
+- Timeline gaps.
+- Conflicting claims.
+
+## How To Search
+
+Start broad only when the issue is unclear:
+
+```text
+core subject + controversy / latest / recent progress / public opinion
+```
+
+Then narrow each follow-up search using concrete entities or clues from previous results:
+
+- Dates.
+- Institutions.
+- People.
+- Places.
+- Platform names.
+- Event names.
+- Official responses.
+- Allegations or counter-claims.
+- Specific policy names.
+
+Do not repeat broad synonym searches when the first result set already provides bridge terms.
+
+## Combining News With Knowledge Files
+
+Use this separation:
+
+```text
+Knowledge files -> background, frameworks, prior patterns, actor context.
+News results    -> current facts, dates, latest developments, source evidence.
+Inference       -> your reasoning over both.
+```
+
+Do not treat news commentary as knowledge-base truth. Treat it as evidence that still needs source-quality assessment.
+
+## When To Continue Searching
+
+Continue searching when:
+
+- The timeline has gaps.
+- Key actors are unclear.
+- Sources conflict.
+- Only one side is represented.
+- The result set is old but the user asks about current status.
+- A claim requires verification.
+
+## Output
+
+When using news results:
+
+- Link facts to visible Markdown links when URLs are available.
+- Reuse the same link for repeated references to the same article.
+- If no URL is available, name the title, source, and date.
+- Keep news evidence separate from knowledge-base file references.
@@ -637,28 +637,23 @@ def _get_memory_context(agent_name: str | None = None, *, app_config: AppConfig
        from deerflow.config.memory_config import get_effective_memory_config
        from deerflow.runtime.user_context import get_effective_user_id

-        user_id = get_effective_user_id()
-        config = get_effective_memory_config(user_id)
-
-        if not config.enabled or not config.injection.enabled:
-            return ""
-
-        manager = build_memory_manager(
-            user_id=user_id,
-            agent_id=agent_name or "default",
-            config=config,
-        )
+        user_id = get_effective_user_id()
+        config = get_effective_memory_config(user_id)
+
+        manager = build_memory_manager(
+            user_id=user_id,
+            agent_id=agent_name or "default",
+            config=config,
+            app_config=app_config,
+        )
        # 只取静态块 (builtin 的 frozen snapshot + Hindsight 的模式说明)。
        # Hindsight 的动态召回需要当前 query,走 MemoryMiddleware.before_model。
-        memory_content = manager.build_system_prompt()
-
-        if not memory_content.strip():
-            return ""
-
-        return f"""<memory>
-{memory_content}
-</memory>
-"""
+        context_content = manager.build_system_prompt()
+
+        if not context_content.strip():
+            return ""
+
+        return f"{context_content}\n"
    except Exception:
        logger.exception("Failed to load memory context")
        return ""
@@ -32,6 +32,8 @@ logger = logging.getLogger(__name__)

 #: 内建 provider 的固定名字。
 BUILTIN_PROVIDER_NAME = "builtin"
+KNOWLEDGE_PROVIDER_NAME = "knowledge"
+_INTERNAL_PROVIDER_NAMES = {BUILTIN_PROVIDER_NAME, KNOWLEDGE_PROVIDER_NAME}


 class MemoryManager:
@@ -49,11 +51,11 @@ class MemoryManager:
        内建 provider (``name == "builtin"``) 始终接受。外部 provider 至多 1 个,
        第二个会被拒绝并 log 警告。返回是否注册成功。
        """
-        is_builtin = provider.name == BUILTIN_PROVIDER_NAME
-        if not is_builtin:
+        is_internal = provider.name in _INTERNAL_PROVIDER_NAMES
+        if not is_internal:
            if self._has_external:
                existing = next(
-                    (p.name for p in self._providers if p.name != BUILTIN_PROVIDER_NAME),
+                    (p.name for p in self._providers if p.name not in _INTERNAL_PROVIDER_NAMES),
                    "unknown",
                )
                logger.warning(
@@ -220,6 +222,7 @@ def build_memory_manager(
    agent_id: str | None = None,
    thread_id: str | None = None,
    config: Any = None,
+    app_config: Any = None,
 ) -> MemoryManager:
    """按生效配置组装并初始化一个 MemoryManager。

@@ -246,7 +249,9 @@ def build_memory_manager(
    manager = MemoryManager()
    base_dir = str(get_paths().base_dir)

-    if getattr(config, "version", "v2") == "v1":
+    memory_enabled = bool(getattr(config, "enabled", True) and getattr(config.injection, "enabled", True))
+
+    if memory_enabled and getattr(config, "version", "v2") == "v1":
        # V1 模式:原版 DeerFlow memory.json + LLM 自动提取
        from deerflow.agents.memory.providers.v1_builtin import V1BuiltinProvider

@@ -260,7 +265,7 @@ def build_memory_manager(
                token_budget=v1_cfg.token_budget,
            )
        )
-    else:
+    elif memory_enabled:
        # V2 模式:USER.md / MEMORY.md + 可选 Hindsight
        from deerflow.agents.memory.providers.builtin import BuiltinFileProvider

@@ -286,11 +291,21 @@ def build_memory_manager(
            except Exception as exc:  # noqa: BLE001 - Hindsight 接入失败不能阻断记忆系统
                logger.warning("加载 Hindsight provider 失败,退回纯 builtin:%s", exc)

+    try:
+        from deerflow.agents.memory.providers.knowledge import KnowledgeProvider
+
+        knowledge = KnowledgeProvider(app_config=app_config)
+        if knowledge.is_available():
+            manager.add_provider(knowledge)
+    except Exception as exc:  # noqa: BLE001 - 知识库路由失败不能阻断对话
+        logger.warning("加载 Knowledge provider 失败:%s", exc)
+
    manager.initialize_all(
        user_id=user_id,
        agent_id=agent_id,
        thread_id=thread_id,
        base_dir=base_dir,
+        app_config=app_config,
    )
    return manager

@@ -227,7 +227,10 @@ class BuiltinFileProvider(MemoryProvider):
        保持系统提示词稳定,利于 prefix cache。
        """
        blocks = [b for b in (self._snapshot.get("user"), self._snapshot.get("memory")) if b]
-        return "\n\n".join(blocks)
+        content = "\n\n".join(blocks)
+        if not content:
+            return ""
+        return f"<memory>\n{content}\n</memory>"

    def get_tool_schemas(self) -> list[dict[str, Any]]:
        """builtin 的 ``memory`` 工具以 LangChain BaseTool 形式独立注册,
@@ -0,0 +1,93 @@
+"""Read-only provider for the mounted policy knowledge base.
+
+This provider keeps the current knowledge-base behaviour: inject the lightweight
+``routing.md`` guide and let the model decide when to read concrete files. It
+does not automatically retrieve snippets and never writes to the knowledge base.
+"""
+
+from __future__ import annotations
+
+import logging
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+from deerflow.agents.memory.provider import MemoryProvider
+
+logger = logging.getLogger(__name__)
+
+
+@lru_cache(maxsize=16)
+def _read_routing_file(host_path: str) -> str:
+    path = Path(host_path)
+    if not path.is_file():
+        return ""
+    return path.read_text(encoding="utf-8").strip()
+
+
+class KnowledgeProvider(MemoryProvider):
+    """Injects the knowledge routing guide as a read-only context provider."""
+
+    def __init__(self, *, app_config: Any | None = None) -> None:
+        self._app_config = app_config
+        self._routing_host_path: Path | None = None
+
+    @property
+    def name(self) -> str:
+        return "knowledge"
+
+    def is_available(self) -> bool:
+        return True
+
+    def initialize(
+        self,
+        *,
+        user_id: str,
+        agent_id: str | None = None,
+        thread_id: str | None = None,
+        base_dir: str,
+        **kwargs: Any,
+    ) -> None:
+        app_config = self._app_config or kwargs.get("app_config")
+        if app_config is None:
+            from deerflow.config import get_app_config
+
+            app_config = get_app_config()
+
+        self._routing_host_path = None
+        for mount in app_config.sandbox.mounts or []:
+            if mount.container_path.rstrip("/") == "/mnt/knowledge":
+                self._routing_host_path = Path(mount.host_path) / "routing.md"
+                break
+
+    def system_prompt_block(self) -> str:
+        if self._routing_host_path is None:
+            return ""
+
+        content = _read_routing_file(str(self._routing_host_path.resolve()))
+        if not content:
+            return ""
+
+        logger.debug("Injected knowledge routing memory from %s", self._routing_host_path)
+        return f"""<knowledge_routing_memory source="/mnt/knowledge/routing.md">
+{content}
+</knowledge_routing_memory>"""
+
+    def prefetch(self, query: str, *, thread_id: str = "") -> str:
+        return ""
+
+    def sync_turn(
+        self,
+        user_content: str,
+        assistant_content: str,
+        *,
+        thread_id: str = "",
+        turn_index: int | None = None,
+    ) -> None:
+        return None
+
+    def get_tool_schemas(self) -> list[dict[str, Any]]:
+        return []
+
+
+__all__ = ["KnowledgeProvider"]
@@ -84,6 +84,14 @@ class TestRegistration:
        assert manager.add_provider(_FakeProvider("mem0")) is False
        assert len(manager.providers) == 2

+    def test_knowledge_does_not_consume_external_slot(self):
+        manager = MemoryManager()
+        manager.add_provider(_FakeProvider("builtin"))
+        manager.add_provider(_FakeProvider("knowledge"))
+        assert manager.add_provider(_FakeProvider("hindsight")) is True
+        assert manager.add_provider(_FakeProvider("mem0")) is False
+        assert [p.name for p in manager.providers] == ["builtin", "knowledge", "hindsight"]
+
    def test_get_provider(self):
        manager = MemoryManager()
        builtin = _FakeProvider("builtin")