feat(knowledge): 阶段三 知识治理 + 阶段四 图谱增强

阶段三(治理):
- sensitive.py:写入前脱敏 token/密钥/手机号/身份证/邮箱(sensitive_redaction_enabled)
- 版本快照:更新前写 knowledge_note_versions;GET /notes/{id}/versions
- dedup.py:标题/摘要 Jaccard + 内容哈希查重;GET /duplicates、POST /merge(来源并入主条目、重复项归档)
- auto_ingest.py:独立后台队列/线程沉淀,不阻塞问答;中间件 after_agent 在 auto_ingest_enabled 时入队;价值判断(最小字符)+ skip_existing 去重;自动沉淀落 draft 待审

阶段四(图谱):
- LLM 抽取 entities/relations;service 存 knowledge_entities/knowledge_relations 并连边
- exporter.build_graph 支持实体节点 + note→entity/entity→entity 边;graph.html/json 增强
- GET /api/knowledge/graph 返回产品图谱数据

前端:知识库页「查重合并」对话框 + 相关 api/hooks
配置:sensitive_redaction_enabled / dedup_similarity_threshold / entity_extraction_enabled
测试:tests/test_knowledge.py 20 passed

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
AdministratorandClaude Opus 4.8 committed 2026-06-08 18:31:33 +08:00
1 parent 683cc6d76e
commit 81e4fd92ce
16 files changed
+691 -7

No files matched your search

@@ -21,6 +21,7 @@ from typing import Any, override
from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langchain_core.messages import HumanMessage
from langgraph.runtime import Runtime
logger = logging.getLogger(__name__)
@@ -132,3 +133,58 @@ class KnowledgeRagMiddleware(AgentMiddleware[AgentState]):
if msg is not None:
request = request.override(messages=[*request.messages, msg])
return handler(request)
# ---- after_agent: enqueue background auto-sedimentation -------------
def _resolve_thread_id(self, runtime: Runtime | None) -> str | None:
if runtime is not None and getattr(runtime, "context", None):
tid = runtime.context.get("thread_id")
if tid:
return str(tid)
try:
from langgraph.config import get_config
return get_config().get("configurable", {}).get("thread_id")
except Exception:
return None
@override
def after_agent(self, state: AgentState, runtime: Runtime) -> dict | None:
"""When auto-ingest is on, hand the finished turn to the background queue."""
try:
from deerflow.config import get_app_config
knowledge_cfg = getattr(get_app_config(), "knowledge", None)
if knowledge_cfg is None or not knowledge_cfg.enabled or not knowledge_cfg.auto_ingest_enabled:
return None
from deerflow.knowledge.auto_ingest import AutoIngestItem, get_auto_ingest_queue
queue = get_auto_ingest_queue()
if queue is None:
return None
messages = state.get("messages") if isinstance(state, dict) else getattr(state, "messages", None)
if not messages:
return None
from deerflow.runtime.references import build_reference_batches
from deerflow.runtime.serialization import serialize_lc_object
from deerflow.runtime.user_context import get_effective_user_id
serialized = [serialize_lc_object(m) for m in messages]
thread_id = self._resolve_thread_id(runtime)
if not thread_id:
return None
title = state.get("title") if isinstance(state, dict) else getattr(state, "title", None)
queue.enqueue(
AutoIngestItem(
thread_id=thread_id,
user_id=get_effective_user_id(),
messages=serialized,
thread_title=title if isinstance(title, str) else None,
reference_batches=build_reference_batches(serialized),
)
)
except Exception:
logger.exception("KnowledgeRagMiddleware.after_agent enqueue failed (non-fatal)")
return None