feat: improve assistant knowledge import and database compatibility

This commit is contained in:
吉鑫升 committed 2026-09-02 17:55:17 +08:00
1 parent fbd8dec48f
commit 6b724bd00d
51 files changed
+4393 -1021

No files matched your search

+2
View File
@@ -88,6 +88,8 @@ skills/
hermes-agent-main/
.gitignore
offline-backend-20260512/backend/tests/*
# Keep the openGauss URL/dialect/schema compatibility regression coverage versioned.
!offline-backend-20260512/backend/tests/test_gauss_database.py
!offline-backend-20260512/backend/tests/test_llmwiki_weknora.py
!offline-backend-20260512/backend/tests/test_health_auth_bypass.py
# Keep formal Word OOXML and embedded-font regression coverage versioned.
@@ -1,8 +1,10 @@
# 技能归纳、WeKnora 普通知识库与助手知识库:实现设计
本文档用于指导大模型或开发者实现知识库体系改造。整体包含两条线:一条是“将一个或多个技能归纳到一个或多个目标知识库”;另一条是新增一个自研的全局助手知识库,用于承接 WeKnora 已处理、已向量化后的 Wiki、实体和关系数据,也支持技能和管理员本地数据直接导入,并做统一实体对齐、Wiki 展示和版本管理。
本文档用于指导大模型或开发者实现知识库体系改造。整体包含两条线:一条是“将一个或多个技能归纳到一个 WeKnora Wiki 普通知识库”;另一条是新增自研助手知识库体系。助手知识库像普通知识库一样可以创建多个库,同时系统内置一个全局大库,用于统计全库数据、承接所有 WeKnora 导入沉淀、做全局实体对齐、Wiki 展示和导入审计。
本文档中,目标知识库既可以是 WeKnora 普通知识库,也可以是自研助手知识库。普通知识库负责向量化、检索、Wiki 和图谱;助手知识库负责全局知识梳理、实体对齐、Wiki 展示、搜索和版本管理。
本文档中,技能归纳目标只能是 WeKnora Wiki 普通知识库;助手知识库不接收技能直接归纳,也不接收任意本地文件。普通知识库负责向量化、检索、Wiki 和图谱;助手知识库负责自研存储、WeKnora 导入、实体对齐、Wiki 展示、搜索和导入审计。无论 WeKnora 数据导入到哪个助手知识库,最终都必须沉淀一份到系统全局大库。
> 最终约束:如果本文历史章节与本节冲突,以本节和“2. 当前边界”中的约束为准。
## 1. 背景与目标
@@ -14,57 +16,59 @@
本次要新增一条管理员工作流:
管理员在“技能管理”页面选择多个技能,再选择一个或多个目标知识库,点击归纳。系统将每个技能作为独立来源读取、解析、抽取和归纳,把结果写入对应知识库,并在技能列表和技能详情中展示“该技能已被哪些知识库归纳”。
管理员在“技能管理”页面选择多个技能,再选择一个目标 WeKnora Wiki 普通知识库,点击归纳。系统将每个技能作为独立来源读取、解析、抽取和归纳,把结果写入对应 WeKnora 知识库,并在技能列表和技能详情中展示“该技能已被哪些知识库归纳”。如果需要归纳到多个普通知识库,通过多次操作完成。
用户已确认的业务约束:
- 一次操作可以选择多个技能,也可以选择多个目标知识库;系统按 `技能 x 目标知识库` 生成处理矩阵。
- 一次操作可以选择多个技能,但只能选择一个目标 WeKnora Wiki 普通知识库。
- 同一个技能可以在不同时间归纳到多个知识库,所以最终关系是技能与知识库多对多。
- V1 支持归纳到多种目标:Wiki WeKnora、非 Wiki WeKnora、FAQ 知识库、系统会话存档知识库、只读知识库,以及自研助手知识库。
- 多知识库归纳通过多次操作完成;每次操作形成一批 `技能 x 单目标知识库` 任务。
- V1 只支持归纳到可写 WeKnora Wiki 普通知识库;不支持非 Wiki、FAQ、系统会话存档、只读知识库作为技能归纳目标。
- 技能归纳时只抽取数据、业务说明、实体、概念、对象、事件、关系等知识;Python、TypeScript、Shell 等代码和脚本类文件完全排除,不读取注释、不抽取函数、不上传、不进入 prompt。
- 技能里的 Markdown、业务数据文件、表格、PDF、Office、图片等要纳入处理范围。
- 需要同步三类内容:原始资料、归纳后的 Wiki、实体与关系。
- 关系来源包括 Markdown 自然语言描述、目录和文件引用、模型自动推断。
- 允许模型推断关系,同时前端需要提供逐条实体/关系预览审核能力。
- 支持手动重新归纳,也支持自动文件监听、定时后台同步和技能变更后的自动远程写入。
- 允许模型推断关系,但前端不提供逐条实体/关系预览审核。
- 支持管理员手动点击“重新归纳”;不新增自动文件监听、定时后台同步或技能变更自动远程写入。
- 解除关联时弹窗让用户选择:保留知识库内容,或删除该技能贡献的内容。
- 权限仅管理员。
- 总技能数约 200 个,需要支持批量进度、后台执行、失败重试。
- 同一目标知识库内,不同技能抽取出的同一实体需要合并。
- 重归纳时先生成新版本,成功后再替换旧版本;失败时旧版本继续可用。
- 新增知识库类型区分:普通知识库是 WeKnora,助手知识库是自研存储。
- 助手知识库只有一个全局大库,建议命名为“知识梳理”,所有用户可见。
- 助手知识库支持创建多个库;其中有且只有一个系统全局大库,建议命名为“知识梳理总库”,所有用户可见。
- 助手知识库只提供 Wiki 页面展示、搜索和数据导入;不落到 WeKnora。
- 助手知识库数据来源包括 WeKnora 导出、技能直接归纳、管理员本地文件导入。
- WeKnora 已向量化和处理完成的 Wiki、文档、实体、关系、图谱数据默认沉淀到助手知识库。
- 普通知识库需要提供导出或“导入到助手知识库”的功能,把 WeKnora 里已经处理好的数据导入助手知识库;导出数据需要包含 Wiki、文档、chunk、实体、关系、图谱。
- 即使没有配置 WeKnora,也要默认展示助手知识库的创建或初始化入口。
- 助手知识库数据来源只能是 WeKnora 导出包或从某个 WeKnora 知识库触发导入。
- WeKnora 已向量化和处理完成的 Wiki、文档、实体、关系、图谱数据默认沉淀到助手知识库;如果导入的是某个自建助手库,也必须同步沉淀到全局大库。
- 普通知识库需要提供导出或“导入到助手知识库”的功能,把 WeKnora 里已经处理好的数据导入指定助手知识库;导出数据需要包含 Wiki、文档、chunk、实体、关系、图谱。
- 即使没有配置 WeKnora 地址,也要默认展示助手知识库列表和创建入口,而不是展示空的普通知识库页面。
- 管理员可切换知识库产品版本:`v1` / `v2`,默认 `v1`。
- 助手知识库普通用户只读,管理员可导入、编辑、删除、查看版本和回滚 Wiki 页面。
## 2. 必须支持的扩展能力
## 2. 当前边界
以下能力原先按非目标处理,最新需求已确认全部要支持。实现时必须把它们作为正式目标,并加上必要的权限、状态和审计护栏。
以下能力明确不做,开发实现和验收必须按这些边界收敛:
- 支持一次选择多个目标知识库。一个归纳任务可包含多个技能和多个目标,后端按矩阵拆分 item。
- 支持非 Wiki 知识库、FAQ 知识库、系统会话存档知识库、只读知识库作为目标。不同目标使用不同写入 adapter;只读目标使用本地 sidecar 投影和待推送状态。
- 支持直接写 WeKnora 的 Neo4j 或底层图数据库,但必须通过受控 graph writer、显式配置开关、事务和审计,不允许 LLM 生成任意 Cypher 直接执行。
- 支持自动文件监听、定时后台同步、技能变更自动远程写入。自动写入仍必须走 staged 版本和失败保留旧版本机制。
- 前端提供逐条实体/关系预览审核,管理员可批准、拒绝、编辑、批量处理。
- 前端和 API 暴露远端 WeKnora ID,并可作为跨系统业务 key 使用;本地仍保留 surrogate id 以保证数据库关系稳定。
- 支持技能直接归纳到助手知识库,不必先经过 WeKnora。
- 支持管理员向助手知识库导入任意本地文件,导入前必须做文件类型、安全扫描和敏感信息阻断。
- 助手知识库 V1 支持 Wiki 页面版本管理、版本对比、发布、回滚和导入审计。
- 不支持一次选择多个目标知识库。多知识库归纳通过多次操作完成。
- 不支持非 Wiki 知识库、FAQ 知识库、系统会话存档知识库、只读知识库作为技能归纳目标。
- 不直接写 WeKnora 的 Neo4j 或底层图数据库。
- 不新增自动文件监听、定时后台同步或技能变更自动远程写入。
- 不在前端提供逐条实体/关系预览审核。
- 不把远端 WeKnora 内部 ID 暴露为前端业务主键。
- 不支持技能直接归纳到助手知识库。技能先进入 WeKnora 普通知识库,再由 WeKnora 导出沉淀到助手知识库。
- 不支持管理员向助手知识库导入任意本地文件;手动导入也必须是 WeKnora 导出包或从某个 WeKnora 知识库触发。
- 助手知识库 V1 不做页面版本管理,只保留当前合并后的 Wiki 状态和导入审计记录。
## 3. 术语
| 术语 | 含义 |
| --- | --- |
| 技能 | DeerFlow 技能目录,通常包含 `SKILL.md`、代码、配置、示例、文档、数据文件等。 |
| 目标知识库 | 管理员选择的一个或多个知识库,可以是 WeKnora 普通知识库,也可以是自研助手知识库。 |
| 目标知识库 | 管理员选择的一个 WeKnora Wiki 普通知识库。助手知识库不是技能归纳目标。 |
| 普通知识库 | WeKnora 知识库,负责向量化、Wiki、文档检索和图谱能力。 |
| 助手知识库 | DeerFlow 自研知识库,只维护一个全局大库,用于 Wiki 展示、搜索、WeKnora 导入、技能直接导入、本地文件导入和实体对齐。 |
| 知识梳理底库 | 助手知识库的默认全局库名,所有用户可见,普通用户只读。 |
| 助手知识库 | DeerFlow 自研知识库类型,支持创建多个库,用于 Wiki 展示、搜索、WeKnora 导入和实体对齐。 |
| 知识梳理总库 | 助手知识库体系中的系统全局大库,有且只有一个,自动统计和沉淀所有助手库与普通知识库导入数据,做全局实体对齐。 |
| 自建助手库 | 管理员创建的普通助手知识库,类似普通知识库列表中的一个库,可导入 WeKnora 普通知识库导出数据。 |
| 产品版本 | 知识库产品形态开关。`v1` 默认为普通知识库体验,`v2` 默认为助手知识库体验。 |
| WeKnora 导出包 | 从普通知识库导出的已处理知识集合,包含 Wiki 页面、文档、chunk、实体、关系、图谱等。 |
| 绑定 | 一个技能与一个目标知识库之间的持久关联。`skill_name + target_type + target_id` 唯一。 |
@@ -81,37 +85,35 @@
```mermaid
flowchart LR
A[管理员选择多个技能] --> B[选择一个或多个目标知识库]
B --> C[创建归纳任务矩阵]
A[管理员选择多个技能] --> B[选择一个 WeKnora Wiki 目标知识库]
B --> C[创建归纳任务]
C --> D[逐技能扫描与解析]
D --> E[生成技能快照]
E --> F[LLM 归纳 Wiki 与关系抽取]
F --> R[前端实体/关系预览审核]
R --> G[按目标知识库合并实体/关系]
G --> H[写入 WeKnora 或本地助手库]
G --> I[写入 Wiki/FAQ/会话存档/sidecar]
G --> J[写入 WeKnora GraphRAG 或 Neo4j]
F --> G[按目标 WeKnora Wiki 合并实体/关系]
G --> H[写入 WeKnora 原始资料与 Wiki]
G --> I[写入关系投影文档]
H --> K[校验远端写入]
I --> K
J --> K
K --> L[本地原子激活新版本]
L --> M[技能页展示已归纳知识库]
K --> N[WeKnora 导出已处理数据]
N --> O[沉淀到助手知识库]
O --> P[全局实体对齐]
P --> Q[知识梳理 Wiki 展示与搜索]
N --> O[导入指定助手知识库]
O --> P[同步沉淀到知识梳理总库]
P --> Q[全局实体对齐]
Q --> S[总库 Wiki 展示与搜索]
```
关键原则:
- DeerFlow 本地数据库保存权威绑定关系、版本状态、来源贡献和远端 artifact 映射。
- WeKnora 保存可检索知识内容,包括原文、Wiki 页面、FAQ、会话存档、关系投影文档和图谱。
- 对 WeKnora 图谱能力采取能力探测和分层写入:优先使用 WeKnora API;需要底层图写入时,使用受控 Neo4j graph writer。
- 不直接写 WeKnora 底层图数据库;关系以 WeKnora API 能力和关系投影文档承接。
- 重归纳使用 staged artifact,远端写入和校验成功后,本地才激活新版本。
- 删除或解除关联只影响该技能在目标知识库中的贡献,不误删其他技能共享的实体和关系。
- 助手知识库只有一个全局库,可以接收技能直接归纳、WeKnora 导出和管理员本地文件导入。
- WeKnora 的已处理结果默认进入助手知识库,由助手底库做全局实体对齐、Wiki 展示、搜索和版本管理。
- 未配置 WeKnora 时,仍展示助手知识库创建或初始化能力;与 WeKnora 相关的归纳和导入动作置灰或隐藏。
- 助手知识库可以创建多个库;系统全局大库只有一个,负责汇总全库数据和做跨库实体对齐。
- WeKnora 的已处理结果可以导入指定助手知识库;无论是否指定自建助手库,都必须同步沉淀到知识梳理总库。
- 未配置 WeKnora 地址时,默认进入助手知识库页面,展示助手知识库列表、创建入口和知识梳理总库;与 WeKnora 相关的导出导入动作置灰或隐藏。
## 5. 现有代码落点
@@ -193,6 +195,14 @@ items = selected_skills x selected_targets
每个 item 对应一个 `binding`,一个技能归纳到三个目标知识库,就生成三个 binding。父 job 负责展示整体进度,item 独立成功、失败、取消和重试。
当目标是自建助手库时,item 写入顺序是:
```text
写入目标助手库 -> 生成目标库 Wiki/实体/关系 -> 创建总库沉淀任务 -> 总库实体对齐和统计
```
总库沉淀失败不能让目标助手库写入结果消失,但 item 必须标记为 `partial_failed`,并允许单独重试“沉淀到总库”。
目标知识库需要通过 adapter 写入:
| 目标类型 | 写入策略 | Wiki | 关系/图谱 |
@@ -202,7 +212,7 @@ items = selected_skills x selected_targets
| WeKnora FAQ 知识库 | 把归纳结果转换为 FAQ 条目、来源文档和关系摘要 | 不写或只写 sidecar Wiki | 关系作为 FAQ metadata、projection 文档和图谱边 |
| 系统会话存档知识库 | 把技能归纳结果作为系统会话归档条目写入 | sidecar Wiki 展示 | 关系作为归档 metadata 和图谱边 |
| 只读 WeKnora 知识库 | 不修改远端原库;写本地 sidecar artifact,并生成待推送状态 | sidecar Wiki 展示 | sidecar graph + 待推送 Neo4j plan |
| 助手知识库 | 写入自研底库、实体对齐、生成 Wiki 页面版本 | 写自研 Wiki | 写自研实体/关系表 |
| 助手知识库 | 写入指定助手库、实体对齐、生成 Wiki 页面版本;同时异步沉淀到知识梳理总库 | 写自研 Wiki | 写自研实体/关系表;总库做跨库实体对齐 |
只读目标说明:
@@ -13,6 +13,8 @@ import {
} from "@/core/system-settings/api";
export const KNOWLEDGE_PRODUCT_KEY = ["system-settings", "knowledge-base-product-version"] as const;
export const KNOWLEDGE_NORMAL_VIEW_VALUE = "normal";
export const KNOWLEDGE_NORMAL_VIEW_SEARCH = `?view=${KNOWLEDGE_NORMAL_VIEW_VALUE}`;
export function useKnowledgeBaseProductVersion() {
return useQuery({ queryKey: KNOWLEDGE_PRODUCT_KEY, queryFn: loadKnowledgeBaseProductVersion, staleTime: 60_000 });
@@ -35,7 +37,7 @@ export function KnowledgeProductNav() {
return (
<div className="flex items-center gap-2 border-b bg-card px-6 py-2">
<Button size="sm" variant={assistant ? "ghost" : "secondary"} onClick={() => navigate("/page/workspace/knowledge")}>
<Button size="sm" variant={assistant ? "ghost" : "secondary"} onClick={() => navigate(`/page/workspace/knowledge${KNOWLEDGE_NORMAL_VIEW_SEARCH}`)}>
<BookOpen />普通知识库
</Button>
<Button size="sm" variant={assistant ? "secondary" : "ghost"} onClick={() => navigate("/page/workspace/knowledge/assistant")}>
@@ -22,6 +22,10 @@ import {
} from "@/components/ui/dropdown-menu";
import { Input } from "@/components/ui/input";
import { cn } from "@/lib/utils";
import {
type AssistantKnowledgeBase,
useAssistantKnowledgeBases,
} from "@/strategy-components/api/assistant-knowledge";
import {
type LlmWikiKnowledgeBase,
useAgentLlmWikiKnowledgeBases,
@@ -66,10 +70,26 @@ function SpaceIcon({ item }: { item: LlmWikiKnowledgeBase }) {
);
}
function AssistantSpaceIcon({ item }: { item: AssistantKnowledgeBase }) {
return (
<span
className={cn(
"flex size-7 shrink-0 items-center justify-center rounded-md",
item.is_global ? "bg-amber-500/10 text-amber-600 dark:text-amber-400" : "bg-sky-500/10 text-sky-600 dark:text-sky-400",
)}
>
<Database className="size-3.5" />
</span>
);
}
function SelectionContent({
rows,
assistantRows,
value,
assistantValue,
onChange,
onAssistantChange,
search,
onSearchChange,
loading,
@@ -79,8 +99,11 @@ function SelectionContent({
readOnly,
}: {
rows: LlmWikiKnowledgeBase[];
assistantRows: AssistantKnowledgeBase[];
value: string[];
assistantValue: string[];
onChange: (value: string[]) => void;
onAssistantChange: (value: string[]) => void;
search: string;
onSearchChange: (value: string) => void;
loading: boolean;
@@ -90,6 +113,7 @@ function SelectionContent({
readOnly: boolean;
}) {
const selected = useMemo(() => new Set(value), [value]);
const selectedAssistant = useMemo(() => new Set(assistantValue), [assistantValue]);
const filtered = useMemo(() => {
const query = search.trim().toLocaleLowerCase();
if (!query) return rows;
@@ -97,11 +121,28 @@ function SelectionContent({
`${item.name} ${item.description}`.toLocaleLowerCase().includes(query),
);
}, [rows, search]);
const filteredAssistant = useMemo(() => {
const query = search.trim().toLocaleLowerCase();
if (!query) return assistantRows;
return assistantRows.filter((item) =>
`${item.name} ${item.description ?? ""}`.toLocaleLowerCase().includes(query),
);
}, [assistantRows, search]);
const groups = useMemo(() => groupRows(filtered), [filtered]);
const toggle = (id: string) => {
if (readOnly) return;
onChange(selected.has(id) ? value.filter((item) => item !== id) : [...value, id]);
};
const toggleAssistant = (id: string) => {
if (readOnly) return;
onAssistantChange(
selectedAssistant.has(id)
? assistantValue.filter((item) => item !== id)
: [...assistantValue, id],
);
};
const selectedCount = value.length + assistantValue.length;
const hasRows = groups.length > 0 || filteredAssistant.length > 0;
return (
<>
@@ -127,7 +168,7 @@ function SelectionContent({
<div className="m-2 rounded-lg border border-destructive/30 bg-destructive/5 p-3 text-xs text-destructive">
{error instanceof Error ? error.message : "知识空间读取失败"}
</div>
) : groups.length === 0 ? (
) : !hasRows ? (
<div className="flex h-28 items-center justify-center px-4 text-center text-xs text-muted-foreground">
{search
? "没有匹配的知识空间"
@@ -136,91 +177,181 @@ function SelectionContent({
: "当前账号暂无可用知识空间"}
</div>
) : (
groups.map((group, groupIndex) => (
<div key={group.key}>
{groupIndex > 0 && (
variant === "popover"
? <DropdownMenuSeparator className="my-1.5" />
: <div className="my-1.5 border-t" />
)}
{variant === "popover" ? (
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
</DropdownMenuLabel>
) : (
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
</div>
)}
{group.rows.map((item) => {
const rowContent = (
<>
<SpaceIcon item={item} />
<div className="min-w-0 flex-1">
<div className="truncate text-xs font-medium">{item.name}</div>
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
{item.type === "faq" ? "FAQ" : "文档库"} · {item.knowledge_count ?? 0} 项知识
</div>
</div>
{item.publication_status === "published" && (
<Badge variant="outline" className="h-4 px-1 text-[9px]">公共</Badge>
)}
</>
);
return variant === "popover" ? (
<DropdownMenuCheckboxItem
key={item.id}
checked={selected.has(item.id)}
disabled={readOnly}
onCheckedChange={() => toggle(item.id)}
onSelect={(event) => event.preventDefault()}
className={cn(
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
selected.has(item.id) && "bg-primary/8 focus:bg-primary/12",
)}
>
{rowContent}
</DropdownMenuCheckboxItem>
<>
{groups.map((group, groupIndex) => (
<div key={group.key}>
{groupIndex > 0 && (
variant === "popover"
? <DropdownMenuSeparator className="my-1.5" />
: <div className="my-1.5 border-t" />
)}
{variant === "popover" ? (
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
</DropdownMenuLabel>
) : (
<button
key={item.id}
type="button"
role="checkbox"
aria-checked={selected.has(item.id)}
disabled={readOnly}
onClick={() => toggle(item.id)}
className={cn(
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
selected.has(item.id) && "bg-primary/8",
)}
>
<span className={cn(
"flex size-4 shrink-0 items-center justify-center rounded border",
selected.has(item.id) && "border-primary bg-primary text-primary-foreground",
)}>
{selected.has(item.id) && <Check className="size-3" />}
</span>
{rowContent}
</button>
);
})}
</div>
))
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
</div>
)}
{group.rows.map((item) => {
const rowContent = (
<>
<SpaceIcon item={item} />
<div className="min-w-0 flex-1">
<div className="truncate text-xs font-medium">{item.name}</div>
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
{item.type === "faq" ? "FAQ" : "文档库"} · {item.knowledge_count ?? 0} 项知识
</div>
</div>
{item.publication_status === "published" && (
<Badge variant="outline" className="h-4 px-1 text-[9px]">公共</Badge>
)}
</>
);
return variant === "popover" ? (
<DropdownMenuCheckboxItem
key={item.id}
checked={selected.has(item.id)}
disabled={readOnly}
onCheckedChange={() => toggle(item.id)}
onSelect={(event) => event.preventDefault()}
className={cn(
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
selected.has(item.id) && "bg-primary/8 focus:bg-primary/12",
)}
>
{rowContent}
</DropdownMenuCheckboxItem>
) : (
<button
key={item.id}
type="button"
role="checkbox"
aria-checked={selected.has(item.id)}
disabled={readOnly}
onClick={() => toggle(item.id)}
className={cn(
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
selected.has(item.id) && "bg-primary/8",
)}
>
<span className={cn(
"flex size-4 shrink-0 items-center justify-center rounded border",
selected.has(item.id) && "border-primary bg-primary text-primary-foreground",
)}>
{selected.has(item.id) && <Check className="size-3" />}
</span>
{rowContent}
</button>
);
})}
</div>
))}
{filteredAssistant.length > 0 && (
<div>
{(groups.length > 0) && (
variant === "popover"
? <DropdownMenuSeparator className="my-1.5" />
: <div className="my-1.5 border-t" />
)}
{variant === "popover" ? (
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
<Database className="size-3" />助手知识库
</DropdownMenuLabel>
) : (
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
<Database className="size-3" />助手知识库
</div>
)}
{filteredAssistant.map((item) => {
const rowContent = (
<>
<AssistantSpaceIcon item={item} />
<div className="min-w-0 flex-1">
<div className="truncate text-xs font-medium">{item.name}</div>
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
Wiki · {item.page_count ?? 0} 页 · {item.entity_count ?? 0} 实体
</div>
</div>
<Badge variant="outline" className="h-4 px-1 text-[9px]">
{item.is_global ? "总库" : "助手"}
</Badge>
</>
);
return variant === "popover" ? (
<DropdownMenuCheckboxItem
key={`assistant:${item.id}`}
checked={selectedAssistant.has(item.id)}
disabled={readOnly}
onCheckedChange={() => toggleAssistant(item.id)}
onSelect={(event) => event.preventDefault()}
className={cn(
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
selectedAssistant.has(item.id) && "bg-primary/8 focus:bg-primary/12",
)}
>
{rowContent}
</DropdownMenuCheckboxItem>
) : (
<button
key={`assistant:${item.id}`}
type="button"
role="checkbox"
aria-checked={selectedAssistant.has(item.id)}
disabled={readOnly}
onClick={() => toggleAssistant(item.id)}
className={cn(
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
selectedAssistant.has(item.id) && "bg-primary/8",
)}
>
<span className={cn(
"flex size-4 shrink-0 items-center justify-center rounded border",
selectedAssistant.has(item.id) && "border-primary bg-primary text-primary-foreground",
)}>
{selectedAssistant.has(item.id) && <Check className="size-3" />}
</span>
{rowContent}
</button>
);
})}
</div>
)}
</>
)}
</div>
<div className="flex items-center justify-between gap-2 border-t bg-muted/20 px-2.5 py-2">
<span className="text-[11px] text-muted-foreground">
{readOnly ? `本对话已选 ${value.length} 个` : `已选择 ${value.length} 个`}
{readOnly ? `本对话已选 ${selectedCount} 个` : `已选择 ${selectedCount} 个`}
</span>
{readOnly ? (
<span className="text-[11px] text-muted-foreground">对话开始后不可更改</span>
) : (
<div className="flex items-center gap-1">
<Button type="button" variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => onChange(filtered.map((item) => item.id))}>
<Button
type="button"
variant="ghost"
size="sm"
className="h-7 px-2 text-xs"
onClick={() => {
onChange(filtered.map((item) => item.id));
onAssistantChange(filteredAssistant.map((item) => item.id));
}}
>
全选
</Button>
<Button type="button" variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => onChange([])}>
<Button
type="button"
variant="ghost"
size="sm"
className="h-7 px-2 text-xs"
onClick={() => {
onChange([]);
onAssistantChange([]);
}}
>
清空
</Button>
</div>
@@ -232,7 +363,9 @@ function SelectionContent({
export function LlmWikiKnowledgeSelector({
value,
assistantValue = [],
onChange,
onAssistantChange = () => undefined,
source = "all",
agentId,
autoSelectAll = false,
@@ -243,7 +376,9 @@ export function LlmWikiKnowledgeSelector({
variant = "popover",
}: {
value: string[];
assistantValue?: string[];
onChange: (value: string[]) => void;
onAssistantChange?: (value: string[]) => void;
source?: SelectorSource;
agentId?: string | null;
autoSelectAll?: boolean;
@@ -257,28 +392,39 @@ export function LlmWikiKnowledgeSelector({
const [open, setOpen] = useState(false);
const [search, setSearch] = useState("");
const runtime = useLlmWikiRuntime();
const weknoraEnabled = runtime.data?.provider === "weknora";
const all = useLlmWikiKnowledgeBases(
"all",
runtime.data?.provider === "weknora",
weknoraEnabled,
);
const agent = useAgentLlmWikiKnowledgeBases(
agentId,
runtime.data?.provider === "weknora" && source === "agent" && !!agentId,
weknoraEnabled && source === "agent" && !!agentId,
);
const rows = all.data ?? [];
const assistantBases = useAssistantKnowledgeBases();
const rows = weknoraEnabled ? (all.data ?? []) : [];
const assistantRows = assistantBases.data ?? [];
const defaultIds = useMemo(() => {
if (source !== "agent") {
return rows.map((item) => item.id);
}
return (agent.data ?? []).map((item) => item.id);
}, [agent.data, rows, source]);
const loading = all.isLoading || (source === "agent" && !!agentId && agent.isLoading);
const error = all.error ?? (source === "agent" ? agent.error : null);
const loading = runtime.isLoading || (weknoraEnabled && all.isLoading) || assistantBases.isLoading || (source === "agent" && !!agentId && agent.isLoading);
const error = (weknoraEnabled ? all.error ?? (source === "agent" ? agent.error : null) : null) ?? assistantBases.error;
const availableIds = useMemo(() => new Set(rows.map((item) => item.id)), [rows]);
const assistantAvailableIds = useMemo(
() => new Set(assistantRows.map((item) => item.id)),
[assistantRows],
);
const initializationKey = `${source}:${agentId ?? "all"}:${defaultIds.join(",")}`;
const initialized = useRef<string | null>(null);
useEffect(() => {
if (!runtime.isLoading && !weknoraEnabled) {
if (value.length > 0) onChange([]);
return;
}
if (!all.data || (source === "agent" && agentId && !agent.data) || initialized.current === initializationKey) return;
initialized.current = initializationKey;
const valid = value.filter((id) => availableIds.has(id));
@@ -288,19 +434,30 @@ export function LlmWikiKnowledgeSelector({
if (next.length !== value.length || next.some((id, index) => id !== value[index])) {
onChange(next);
}
}, [agent.data, agentId, all.data, autoSelectAll, availableIds, defaultIds, initializationKey, onChange, source, value]);
}, [agent.data, agentId, all.data, autoSelectAll, availableIds, defaultIds, initializationKey, onChange, runtime.isLoading, source, value, weknoraEnabled]);
useEffect(() => {
if (!assistantBases.data) return;
const valid = assistantValue.filter((id) => assistantAvailableIds.has(id));
if (valid.length !== assistantValue.length || valid.some((id, index) => id !== assistantValue[index])) {
onAssistantChange(valid);
}
}, [assistantAvailableIds, assistantBases.data, assistantValue, onAssistantChange]);
useEffect(() => {
if (!open) setSearch("");
}, [open]);
if (runtime.isError || runtime.data?.provider !== "weknora") return null;
if ((runtime.isError || !weknoraEnabled) && !assistantBases.isLoading && assistantRows.length === 0) return null;
const content = (
<SelectionContent
rows={rows}
assistantRows={assistantRows}
value={value}
assistantValue={assistantValue}
onChange={onChange}
onAssistantChange={onAssistantChange}
search={search}
onSearchChange={setSearch}
loading={loading}
@@ -323,7 +480,7 @@ export function LlmWikiKnowledgeSelector({
新建该智能体对话时,只能从已绑定空间中选择本次检索范围。
</p>
</div>
<Badge variant="secondary" className="shrink-0">{value.length} 个</Badge>
<Badge variant="secondary" className="shrink-0">{value.length + assistantValue.length} 个</Badge>
</div>
{content}
</div>
@@ -333,10 +490,15 @@ export function LlmWikiKnowledgeSelector({
const selectedNames = value
.map((id) => rows.find((item) => item.id === id)?.name)
.filter((name): name is string => !!name);
const selectedLabel = selectedNames.length > 0
? selectedNames.join("、")
: value.length > 0
? `已选 ${value.length} 个知识空间`
const selectedAssistantNames = assistantValue
.map((id) => assistantRows.find((item) => item.id === id)?.name)
.filter((name): name is string => !!name);
const allSelectedNames = [...selectedNames, ...selectedAssistantNames];
const selectedCount = value.length + assistantValue.length;
const selectedLabel = allSelectedNames.length > 0
? allSelectedNames.join("、")
: selectedCount > 0
? `已选 ${selectedCount} 个知识空间`
: label;
return (
@@ -348,20 +510,20 @@ export function LlmWikiKnowledgeSelector({
size="sm"
className={cn(
"h-8 max-w-52 gap-1.5 rounded-full border-0 bg-transparent px-2.5 text-muted-foreground shadow-none hover:bg-muted/70 hover:text-foreground",
value.length > 0 && "bg-primary/8 text-primary hover:bg-primary/12 hover:text-primary",
selectedCount > 0 && "bg-primary/8 text-primary hover:bg-primary/12 hover:text-primary",
className,
)}
disabled={disabled || loading}
title={value.length > 0 ? selectedLabel : undefined}
title={selectedCount > 0 ? selectedLabel : undefined}
>
{loading ? <Loader2 className="size-3.5 animate-spin" /> : <Database className="size-3.5" />}
<span className="truncate text-xs font-normal">{selectedLabel}</span>
{value.length > 0 && (
{selectedCount > 0 && (
<span className="flex size-4 items-center justify-center rounded-full bg-primary text-[9px] font-semibold text-primary-foreground">
{value.length}
{selectedCount}
</span>
)}
{value.length > 0 && value.length === rows.length && <Check className="size-3" />}
{selectedCount > 0 && selectedCount === rows.length + assistantRows.length && <Check className="size-3" />}
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" sideOffset={8} className="w-80 overflow-hidden rounded-xl p-0 shadow-xl">
@@ -1,4 +1,4 @@
import { CheckCircle2, Database, Loader2, RefreshCw, Sparkles } from "lucide-react";
import { Database, Loader2, RefreshCw, Search, Sparkles } from "lucide-react";
import { useEffect, useMemo, useState } from "react";
import { toast } from "sonner";
@@ -10,18 +10,12 @@ import { Progress } from "@/components/ui/progress";
import { ScrollArea } from "@/components/ui/scroll-area";
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
import { useAllSkillsAdmin } from "@/core/skills/hooks";
import {
useAssistantKnowledgeActions,
useAssistantKnowledgeBase,
} from "@/strategy-components/api/assistant-knowledge";
import { useLlmWikiKnowledgeBases, useLlmWikiRuntime } from "@/strategy-components/api/llmwiki";
import {
type SkillKnowledgeTarget,
useSkillKnowledgeActions,
useSkillKnowledgeAutoSyncSettings,
useSkillKnowledgeBindings,
useSkillKnowledgeJobs,
useSkillKnowledgeReviews,
} from "@/strategy-components/api/skill-knowledge";
const PHASE_LABELS: Record<string, string> = {
@@ -39,53 +33,82 @@ export function SkillKnowledgePanel() {
const { skills, isLoading } = useAllSkillsAdmin(true);
const liveSkills = useMemo(() => skills.filter((skill) => (skill.state ?? "active") === "active"), [skills]);
const [selectedSkills, setSelectedSkills] = useState<string[]>([]);
const [selectedTargetKeys, setSelectedTargetKeys] = useState<string[]>([]);
const [reviewMode, setReviewMode] = useState<"required" | "auto_high_confidence" | "off">("auto_high_confidence");
const [skillQuery, setSkillQuery] = useState("");
const [selectedTargetKey, setSelectedTargetKey] = useState<string | null>(null);
const [reviewMode, setReviewMode] = useState<"auto_high_confidence" | "off">("auto_high_confidence");
const runtime = useLlmWikiRuntime();
const knowledgeBases = useLlmWikiKnowledgeBases("all", runtime.data?.provider === "weknora");
const assistantBase = useAssistantKnowledgeBase();
const assistantActions = useAssistantKnowledgeActions();
const bindings = useSkillKnowledgeBindings();
const jobs = useSkillKnowledgeJobs();
const reviews = useSkillKnowledgeReviews();
const actions = useSkillKnowledgeActions();
const autoSettings = useSkillKnowledgeAutoSyncSettings();
const [scheduleCron, setScheduleCron] = useState("0 */6 * * *");
const [debounceSeconds, setDebounceSeconds] = useState(120);
useEffect(() => {
if (!autoSettings.data) return;
setScheduleCron(autoSettings.data.schedule_cron);
setDebounceSeconds(autoSettings.data.debounce_seconds);
}, [autoSettings.data]);
const targets = useMemo<SkillKnowledgeTarget[]>(() => {
const rows: SkillKnowledgeTarget[] = (knowledgeBases.data ?? []).map((kb) => ({
target_type: "weknora",
target_mode: kb.can_write ? kb.type === "faq" ? "faq" : kb.type === "wiki" ? "wiki" : "document" : "readonly_sidecar",
target_id: kb.id,
target_name: kb.name,
remote_business_key: `weknora:kb:${kb.id}`,
readonly: !kb.can_write,
}));
if (assistantBase.data) {
rows.unshift({
target_type: "assistant",
target_mode: "assistant",
target_id: assistantBase.data.id,
target_name: assistantBase.data.name,
remote_business_key: `assistant:base:${assistantBase.data.id}`,
});
return (knowledgeBases.data ?? [])
.filter((kb) => {
const strategy = kb.indexing_strategy ?? {};
const capabilities = kb.capabilities ?? {};
const wikiEnabled =
kb.type === "wiki" ||
kb.wiki_index_enabled === true ||
strategy.wiki_enabled === true ||
capabilities.wiki_enabled === true ||
(kb.wiki_config && Object.keys(kb.wiki_config).length > 0);
return kb.can_write && kb.remote_status !== "missing" && !kb.is_conversation_deposit && kb.type !== "faq" && wikiEnabled;
})
.map((kb) => ({
target_type: "weknora",
target_mode: "wiki",
target_id: kb.id,
target_name: kb.name,
remote_business_key: `weknora:kb:${kb.id}`,
}));
}, [knowledgeBases.data]);
const filteredSkills = useMemo(() => {
const needle = skillQuery.trim().toLowerCase();
if (!needle) return liveSkills;
return liveSkills.filter((skill) => {
const tagText = (skill.tags ?? []).map((tag) => tag.name).join(" ");
const haystack = [
skill.name,
skill.name_zh,
skill.description,
skill.category,
tagText,
]
.filter(Boolean)
.join("\n")
.toLowerCase();
return haystack.includes(needle);
});
}, [liveSkills, skillQuery]);
const filteredSkillNames = useMemo(() => filteredSkills.map((skill) => skill.name), [filteredSkills]);
const allFilteredSelected = filteredSkillNames.length > 0 && filteredSkillNames.every((name) => selectedSkills.includes(name));
const toggleFilteredSkills = () => {
if (!filteredSkillNames.length) return;
setSelectedSkills((current) => {
if (allFilteredSelected) {
const filtered = new Set(filteredSkillNames);
return current.filter((name) => !filtered.has(name));
}
return [...current, ...filteredSkillNames.filter((name) => !current.includes(name))];
});
};
useEffect(() => {
if (selectedTargetKey && !targets.some((target) => `${target.target_type}:${target.target_id}` === selectedTargetKey)) {
setSelectedTargetKey(null);
}
return rows;
}, [assistantBase.data, knowledgeBases.data]);
}, [selectedTargetKey, targets]);
const toggle = (values: string[], value: string, setter: (next: string[]) => void) =>
setter(values.includes(value) ? values.filter((item) => item !== value) : [...values, value]);
const submit = async () => {
const selectedTargets = targets.filter((target) =>
selectedTargetKeys.includes(`${target.target_type}:${target.target_id}`),
selectedTargetKey === `${target.target_type}:${target.target_id}`,
);
if (!selectedSkills.length || !selectedTargets.length) {
toast.error("请至少选择一个技能和一个目标知识库");
@@ -108,46 +131,76 @@ export function SkillKnowledgePanel() {
<div className="grid size-full min-h-0 grid-cols-[minmax(320px,0.9fr)_minmax(420px,1.1fr)] gap-4 overflow-hidden p-6">
<section className="flex min-h-0 flex-col rounded-lg border bg-card">
<div className="border-b p-4">
<h2 className="font-semibold">批量归纳</h2>
<p className="mt-1 text-xs text-muted-foreground">按“技能 × 目标知识库”生成独立后台任务;代码与脚本不会被读取。</p>
<h2 className="font-semibold">技能到知识库管理</h2>
<p className="mt-1 text-xs text-muted-foreground">一次选择多个技能归纳到一个 WeKnora Wiki 知识库;代码与脚本不会被读取。</p>
</div>
<ScrollArea className="min-h-0 flex-1">
<div className="space-y-5 p-4">
<div>
<div className="mb-2 flex items-center justify-between">
<span className="text-sm font-medium">1. 选择技能({selectedSkills.length})</span>
<Button variant="ghost" size="sm" onClick={() => setSelectedSkills(selectedSkills.length === liveSkills.length ? [] : liveSkills.map((skill) => skill.name))}>全选/清空</Button>
<Button variant="ghost" size="sm" onClick={toggleFilteredSkills}>
{allFilteredSelected ? "清空当前结果" : "选择当前结果"}
</Button>
</div>
<div className="relative mb-2">
<Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
<Input
value={skillQuery}
onChange={(event) => setSkillQuery(event.target.value)}
placeholder="按技能中文名、英文名、描述、分类或标签搜索"
className="pl-8"
/>
</div>
<div className="max-h-60 space-y-1 overflow-y-auto rounded-md border p-2">
{isLoading ? <Loader2 className="m-4 size-5 animate-spin" /> : liveSkills.map((skill) => (
{isLoading ? <Loader2 className="m-4 size-5 animate-spin" /> : filteredSkills.map((skill) => (
<label key={skill.name} className="flex cursor-pointer items-center gap-2 rounded px-2 py-1.5 hover:bg-muted">
<Checkbox checked={selectedSkills.includes(skill.name)} onCheckedChange={() => toggle(selectedSkills, skill.name, setSelectedSkills)} />
<span className="truncate text-sm">{skill.name_zh || skill.name}</span>
<span className="ml-auto truncate text-xs text-muted-foreground">{skill.name}</span>
</label>
))}
{!isLoading && !filteredSkills.length && (
<p className="p-3 text-xs text-muted-foreground">没有匹配的技能,请换个关键词。</p>
)}
</div>
<p className="mt-1 text-xs text-muted-foreground">
当前显示 {filteredSkills.length}/{liveSkills.length} 个技能。
</p>
</div>
<div>
<div className="mb-2 flex items-center justify-between">
<span className="text-sm font-medium">2. 选择目标({selectedTargetKeys.length})</span>
{!assistantBase.data && (
<Button size="sm" variant="outline" disabled={assistantActions.initialize.isPending} onClick={() => assistantActions.initialize.mutate("知识梳理")}>创建知识梳理库</Button>
)}
<span className="text-sm font-medium">2. 选择目标知识库({selectedTargetKey ? 1 : 0})</span>
</div>
<div className="space-y-1 rounded-md border p-2">
{targets.map((target) => {
const key = `${target.target_type}:${target.target_id}`;
return (
<label key={key} className="flex cursor-pointer items-center gap-2 rounded px-2 py-2 hover:bg-muted">
<Checkbox checked={selectedTargetKeys.includes(key)} onCheckedChange={() => toggle(selectedTargetKeys, key, setSelectedTargetKeys)} />
<Checkbox
checked={selectedTargetKey === key}
onCheckedChange={() => setSelectedTargetKey((current) => (current === key ? null : key))}
/>
<Database className="size-4 text-muted-foreground" />
<span className="text-sm">{target.target_name}</span>
<Badge variant="outline" className="ml-auto text-[10px]">{target.target_type === "assistant" ? "助手库" : target.target_mode === "readonly_sidecar" ? "只读·本地投影" : target.target_mode}</Badge>
<Badge variant="outline" className="ml-auto text-[10px]">WeKnora Wiki</Badge>
</label>
);
})}
{!targets.length && <p className="p-3 text-xs text-muted-foreground">未配置普通知识库;可先创建全局“知识梳理”助手库。</p>}
{(runtime.isLoading || knowledgeBases.isLoading) && <Loader2 className="m-4 size-5 animate-spin" />}
{!runtime.isLoading && !knowledgeBases.isLoading && knowledgeBases.error && (
<p className="p-3 text-xs text-destructive">
读取 WeKnora 普通知识库失败:
{knowledgeBases.error instanceof Error ? knowledgeBases.error.message : String(knowledgeBases.error)}
</p>
)}
{!runtime.isLoading && !knowledgeBases.isLoading && !knowledgeBases.error && !targets.length && (
<p className="p-3 text-xs text-muted-foreground">
{runtime.data?.provider !== "weknora"
? "当前未配置 WeKnora 普通知识库地址;技能需要先归纳到普通知识库,再导入助手知识库。"
: "当前没有可写且已开启 Wiki 的普通知识库;请确认该库不是 FAQ/会话沉淀库,并已开启 Wiki。"}
</p>
)}
</div>
</div>
<div className="flex items-center gap-3">
@@ -156,14 +209,13 @@ export function SkillKnowledgePanel() {
<SelectTrigger className="w-52"><SelectValue /></SelectTrigger>
<SelectContent>
<SelectItem value="auto_high_confidence">高置信自动通过</SelectItem>
<SelectItem value="required">全部逐条审核</SelectItem>
<SelectItem value="off">不阻塞写入</SelectItem>
</SelectContent>
</Select>
</div>
<Button className="w-full" disabled={actions.createJob.isPending} onClick={() => void submit()}>
{actions.createJob.isPending ? <Loader2 className="animate-spin" /> : <Sparkles />}
开始归纳({selectedSkills.length * selectedTargetKeys.length} 项)
开始归纳({selectedSkills.length * (selectedTargetKey ? 1 : 0)} 项)
</Button>
</div>
</ScrollArea>
@@ -199,26 +251,9 @@ export function SkillKnowledgePanel() {
</div>
</ScrollArea>
</div>
{(reviews.data?.length ?? 0) > 0 && (
<div className="max-h-64 overflow-auto rounded-lg border bg-card p-4">
<div className="mb-3 flex items-center"><h2 className="font-semibold">待审核({reviews.data!.length})</h2><Button size="sm" className="ml-auto" onClick={() => actions.review.mutate({ review_ids: reviews.data!.map((row) => row.id), action: "approve" })}><CheckCircle2 />批量批准</Button></div>
{reviews.data!.map((review) => <div key={review.id} className="mb-2 flex items-center gap-2 rounded border p-2 text-xs"><Badge variant="outline">{review.object_type}</Badge><code className="truncate">{String(review.original.name || review.original.predicate || review.object_key)}</code><Button size="sm" variant="ghost" className="ml-auto" onClick={() => actions.review.mutate({ review_ids: [review.id], action: "approve" })}>批准</Button><Button size="sm" variant="ghost" onClick={() => actions.review.mutate({ review_ids: [review.id], action: "reject" })}>拒绝</Button></div>)}
</div>
)}
<div className="max-h-80 overflow-auto rounded-lg border bg-card p-3 text-xs">
<div className="mb-3 flex flex-wrap items-center gap-3 border-b pb-3">
<span className="font-medium">自动同步</span>
{autoSettings.data && <>
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.watch_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ watch_enabled: checked === true })} />文件监听</label>
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.schedule_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ schedule_enabled: checked === true })} />定时同步</label>
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.write_remote_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ write_remote_enabled: checked === true })} />自动写入目标</label>
<Input className="h-8 w-32" value={scheduleCron} onChange={(event) => setScheduleCron(event.target.value)} aria-label="自动同步 cron" />
<Input className="h-8 w-24" type="number" min={10} value={debounceSeconds} onChange={(event) => setDebounceSeconds(Number(event.target.value))} aria-label="防抖秒数" />
<Button size="sm" variant="outline" onClick={() => actions.updateAutoSyncSettings.mutate({ schedule_cron: scheduleCron, debounce_seconds: debounceSeconds }, { onSuccess: () => toast.success("自动同步设置已保存"), onError: (error) => toast.error(error.message) })}>保存</Button>
</>}
</div>
<div className="mb-2 text-muted-foreground">当前有效绑定:{bindings.data?.length ?? 0};同名同类型实体按来源贡献合并。</div>
<div className="space-y-2">{bindings.data?.map((binding) => <div key={binding.id} className="flex flex-wrap items-center gap-2 rounded border p-2"><span className="max-w-32 truncate font-medium">{binding.skill_name}</span><span className="max-w-40 truncate text-muted-foreground">→ {binding.target_name}</span><Badge variant="outline">{binding.status}</Badge><code className="max-w-40 truncate text-[10px] text-muted-foreground">{binding.remote_business_key}</code><label className="ml-auto flex items-center gap-1"><Checkbox checked={binding.auto_sync_enabled} onCheckedChange={(checked) => actions.setAutoSync.mutate({ id: binding.id, enabled: checked === true })} />自动</label><Button size="sm" variant="outline" disabled={actions.resync.isPending} onClick={() => actions.resync.mutate(binding.id)}>重归纳</Button><Button size="sm" variant="ghost" onClick={() => { if (window.confirm("解除关联并保留目标知识内容?")) actions.detach.mutate({ id: binding.id, deleteContributions: false }); }}>解绑并保留</Button><Button size="sm" variant="destructive" onClick={() => { if (window.confirm("解除关联并删除此技能贡献?共享知识仍会保留其他来源。")) actions.detach.mutate({ id: binding.id, deleteContributions: true }); }}>解绑并删除贡献</Button></div>)}</div>
<div className="space-y-2">{bindings.data?.map((binding) => <div key={binding.id} className="flex flex-wrap items-center gap-2 rounded border p-2"><span className="max-w-32 truncate font-medium">{binding.skill_name}</span><span className="max-w-40 truncate text-muted-foreground">→ {binding.target_name}</span><Badge variant="outline">{binding.status}</Badge><code className="max-w-40 truncate text-[10px] text-muted-foreground">{binding.remote_business_key}</code><Button size="sm" variant="outline" className="ml-auto" disabled={actions.resync.isPending} onClick={() => actions.resync.mutate(binding.id)}>重归纳</Button><Button size="sm" variant="ghost" onClick={() => { if (window.confirm("解除关联并保留目标知识内容?")) actions.detach.mutate({ id: binding.id, deleteContributions: false }); }}>解绑并保留</Button><Button size="sm" variant="destructive" onClick={() => { if (window.confirm("解除关联并删除此技能贡献?共享知识仍会保留其他来源。")) actions.detach.mutate({ id: binding.id, deleteContributions: true }); }}>解绑并删除贡献</Button></div>)}</div>
</div>
</section>
</div>
@@ -14,7 +14,7 @@ import {
SidebarMenuItem,
} from "@/components/ui/sidebar";
import { useI18n } from "@/core/i18n/hooks";
import { BookOpen, BotIcon, BrainIcon, FileSearch, ListChecks, MessageSquarePlus, MessagesSquare, PenLine, Plus, SparklesIcon, Telescope, Users } from "lucide-react";
import { BookOpen, BotIcon, BrainIcon, FileSearch, Library, ListChecks, MessageSquarePlus, MessagesSquare, PenLine, Plus, SparklesIcon, Telescope, Users } from "lucide-react";
import { KnowledgeRagToggle } from "./knowledge-rag-toggle";
import { SidebarCollapseButton } from "./workspace-header";
@@ -65,6 +65,13 @@ export const WORKSPACE_COMPACT_NAV_ITEMS: WorkspaceCompactNavItem[] = [
path: "/page/workspace/chats",
match: (pathname) => pathname === "/page/workspace/chats",
},
{
id: "workspace-library",
label: "资料库",
icon: Library,
path: "/page/workspace/library",
match: (pathname) => pathname === "/page/workspace/library",
},
{
id: "workspace-agents",
label: "智能体",
@@ -229,6 +229,14 @@ export function WorkspaceNavMenu() {
技能管理
</DropdownMenuItem>
)}
{isAdmin && (
<DropdownMenuItem
onClick={() => navigate("/page/workspace/admin/skills?tab=knowledge")}
>
<DatabaseZapIcon />
技能到知识库管理
</DropdownMenuItem>
)}
{isAdmin && (
<DropdownMenuItem
onClick={() => navigate("/page/workspace/admin/models")}
+24 -5
View File
@@ -1,5 +1,6 @@
import { apiFetch } from "../api";
import { getFileName } from "../utils/files";
import { apiFetch } from "../api";
import { downloadMarkdownAsDocx } from "../scheduled-tasks/markdown-docx";
import { getFileName } from "../utils/files";
import { urlOfArtifact } from "./utils";
@@ -39,7 +40,7 @@ async function errorMessageFromResponse(res: Response): Promise<string> {
return res.statusText || "Request failed";
}
export async function downloadArtifactFile({
export async function downloadArtifactFile({
filepath,
threadId,
isMock,
@@ -73,8 +74,26 @@ export async function downloadArtifactFile({
document.body.appendChild(a);
a.click();
a.remove();
window.setTimeout(() => URL.revokeObjectURL(objectUrl), 30_000);
}
window.setTimeout(() => URL.revokeObjectURL(objectUrl), 30_000);
}
export async function downloadArtifactMarkdownAsWord({
filepath,
threadId,
}: {
filepath: string;
threadId: string;
}): Promise<void> {
const url = urlOfArtifact({ filepath, threadId, download: false });
const res = await apiFetch(url);
if (!res.ok) {
throw new Error(await errorMessageFromResponse(res));
}
const fileName = getFileName(filepath);
const title = fileName.replace(/\.(md|markdown)$/i, "");
await downloadMarkdownAsDocx(await res.text(), fileName, title);
}
export async function openArtifactInNewTab({
filepath,
@@ -9,11 +9,15 @@ export type ArtifactLibraryKind =
| "archive"
| "other";
export type ArtifactLibrarySource = "all" | "general_qa" | "agent";
export interface ArtifactLibraryItem {
threadId: string;
threadTitle: string;
agentId: string | null;
agentName: string | null;
threadType: string | null;
sourceType: Exclude<ArtifactLibrarySource, "all">;
path: string;
name: string;
sizeBytes: number;
@@ -32,6 +36,7 @@ export interface ArtifactLibraryResponse {
export async function fetchArtifactLibrary({
query,
fileKind,
sourceType,
offset,
limit,
refresh = false,
@@ -39,6 +44,7 @@ export async function fetchArtifactLibrary({
}: {
query?: string;
fileKind: ArtifactLibraryKind;
sourceType: ArtifactLibrarySource;
offset: number;
limit: number;
refresh?: boolean;
@@ -46,6 +52,7 @@ export async function fetchArtifactLibrary({
}): Promise<ArtifactLibraryResponse> {
const search = new URLSearchParams({
file_kind: fileKind,
source_type: sourceType,
offset: String(offset),
limit: String(limit),
});
@@ -72,7 +79,9 @@ export async function fetchArtifactLibrary({
thread_id: string;
thread_title: string;
agent_id?: string | null;
agent_name?: string | null;
thread_type?: string | null;
source_type?: Exclude<ArtifactLibrarySource, "all">;
path: string;
name: string;
size_bytes: number;
@@ -90,7 +99,9 @@ export async function fetchArtifactLibrary({
threadId: item.thread_id,
threadTitle: item.thread_title,
agentId: item.agent_id ?? null,
agentName: item.agent_name ?? item.agent_id ?? null,
threadType: item.thread_type ?? null,
sourceType: item.source_type ?? (item.agent_id ? "agent" : "general_qa"),
path: item.path,
name: item.name,
sizeBytes: item.size_bytes,
+15 -9
View File
@@ -1293,15 +1293,21 @@ export function useThreadStream({
...context,
...extraContext,
};
const selectedLlmWikiIds = Array.isArray(
mergedContext.llmwiki_knowledge_base_ids,
)
? mergedContext.llmwiki_knowledge_base_ids
: [];
const ragModeEnabled =
localSettings.display.ragModeEnabled ||
isThreadRagModeLocked(threadId) ||
selectedLlmWikiIds.length > 0;
const selectedLlmWikiIds = Array.isArray(
mergedContext.llmwiki_knowledge_base_ids,
)
? mergedContext.llmwiki_knowledge_base_ids
: [];
const selectedAssistantKnowledgeIds = Array.isArray(
mergedContext.assistant_knowledge_base_ids,
)
? mergedContext.assistant_knowledge_base_ids
: [];
const ragModeEnabled =
localSettings.display.ragModeEnabled ||
isThreadRagModeLocked(threadId) ||
selectedLlmWikiIds.length > 0 ||
selectedAssistantKnowledgeIds.length > 0;
if (ragModeEnabled) {
lockThreadRagMode(threadId);
}
+2
View File
@@ -27,6 +27,8 @@ export interface AgentThreadContext extends Record<string, unknown> {
knowledge_rag_enabled?: boolean;
/** DeerFlow LLMWiki mapping ids selected for this conversation. */
llmwiki_knowledge_base_ids?: string[];
/** Assistant knowledge-base ids selected for this conversation. */
assistant_knowledge_base_ids?: string[];
/** Direct RAG modes handled in the frontend; these requests never enter Lead Agent. */
execution_profile?: "single_round_rag" | "multi_round_rag";
agent_id?: string;
+16 -2
View File
@@ -9,6 +9,7 @@ import {
SparklesIcon,
} from "lucide-react";
import { useEffect, useMemo, useState } from "react";
import { useSearchParams } from "react-router-dom";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
@@ -29,10 +30,22 @@ import { SkillAddressesPanel } from "./SkillAddressesPage";
const PAGE_SIZE = 10;
export default function AdminSkillsPage() {
const [searchParams, setSearchParams] = useSearchParams();
const [search, setSearch] = useState("");
const [page, setPage] = useState(1);
const { skills, isLoading, error } = useAllSkillsAdmin(true);
const knowledgeBindings = useSkillKnowledgeBindings();
const activeTab = useMemo(() => {
const tab = searchParams.get("tab");
return tab === "addresses" || tab === "knowledge" ? tab : "skills";
}, [searchParams]);
const changeTab = (value: string) => {
const next = new URLSearchParams(searchParams);
if (value === "skills") next.delete("tab");
else next.set("tab", value);
setSearchParams(next, { replace: true });
};
const storedAuth = useMemo(() => getStoredAuth(), []);
const currentUserId = storedAuth?.user_id ?? null;
@@ -60,7 +73,8 @@ export default function AdminSkillsPage() {
<WorkspaceHeader />
<WorkspaceBody>
<Tabs
defaultValue="skills"
value={activeTab}
onValueChange={changeTab}
className="flex size-full min-h-0 flex-col gap-0"
>
<div className="flex items-center justify-between gap-4 border-b px-6 pt-4">
@@ -77,7 +91,7 @@ export default function AdminSkillsPage() {
</TabsTrigger>
<TabsTrigger value="knowledge">
<DatabaseZap />
技能知识归纳
技能到知识库管理
</TabsTrigger>
</TabsList>
</div>
+71 -12
View File
@@ -92,6 +92,26 @@ function readPendingKnowledgeSelection(agentId: string | undefined): string[] |
}
}
function readPendingAssistantKnowledgeSelection(agentId: string | undefined): string[] | null {
if (typeof window === "undefined" || !agentId) return null;
try {
const raw = sessionStorage.getItem("pendingAgentMessage");
if (!raw) return null;
const parsed = JSON.parse(raw) as {
agentId?: string;
assistantKnowledgeBaseIds?: unknown;
};
if (parsed.agentId !== agentId || !Array.isArray(parsed.assistantKnowledgeBaseIds)) {
return null;
}
return parsed.assistantKnowledgeBaseIds
.map((item) => String(item).trim())
.filter(Boolean);
} catch {
return null;
}
}
function stringifyForHiddenContext(value: unknown): string {
if (typeof value === "string") return value;
try {
@@ -270,11 +290,17 @@ export default function AgentChatPage({
const pendingKnowledgeSelection = useRef(
readPendingKnowledgeSelection(agent_id),
);
const pendingAssistantKnowledgeSelection = useRef(
readPendingAssistantKnowledgeSelection(agent_id),
);
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>(
() => pendingKnowledgeSelection.current ?? [],
);
const [selectedAssistantKnowledgeBaseIds, setSelectedAssistantKnowledgeBaseIds] = useState<string[]>(
() => pendingAssistantKnowledgeSelection.current ?? [],
);
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(
() => pendingKnowledgeSelection.current !== null,
() => pendingKnowledgeSelection.current !== null || pendingAssistantKnowledgeSelection.current !== null,
);
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
const knowledgeSelectionThread = useRef<string | null>(null);
@@ -285,6 +311,10 @@ export default function AgentChatPage({
setSelectedKnowledgeBaseIds(ids);
setKnowledgeSelectionReady(true);
}, []);
const changeAssistantKnowledgeSelection = useCallback((ids: string[]) => {
setSelectedAssistantKnowledgeBaseIds(ids);
setKnowledgeSelectionReady(true);
}, []);
useEffect(() => {
activeThreadIdRef.current = threadId;
@@ -294,17 +324,23 @@ export default function AgentChatPage({
if (knowledgeSelectionAgent.current === agent_id) return;
knowledgeSelectionAgent.current = agent_id;
const pending = readPendingKnowledgeSelection(agent_id);
const pendingAssistant = readPendingAssistantKnowledgeSelection(agent_id);
pendingKnowledgeSelection.current = pending;
pendingAssistantKnowledgeSelection.current = pendingAssistant;
knowledgeSelectionThread.current = null;
setSelectedKnowledgeBaseIds(pending ?? []);
setKnowledgeSelectionReady(pending !== null);
setSelectedAssistantKnowledgeBaseIds(pendingAssistant ?? []);
setKnowledgeSelectionReady(pending !== null || pendingAssistant !== null);
}, [agent_id]);
useEffect(() => {
if (isNewThread) {
if (knowledgeSelectionThread.current !== "__new__") {
setSelectedKnowledgeBaseIds(pendingKnowledgeSelection.current ?? []);
setKnowledgeSelectionReady(pendingKnowledgeSelection.current !== null);
setSelectedAssistantKnowledgeBaseIds(pendingAssistantKnowledgeSelection.current ?? []);
setKnowledgeSelectionReady(
pendingKnowledgeSelection.current !== null || pendingAssistantKnowledgeSelection.current !== null,
);
}
knowledgeSelectionThread.current = "__new__";
setKnowledgeSelectionLoading(false);
@@ -319,18 +355,29 @@ export default function AgentChatPage({
if (knowledgeSelectionThread.current !== threadId) return;
const metadata = record.metadata as Record<string, unknown> | undefined;
const raw = metadata?.llmwiki_knowledge_base_ids;
const assistantRaw = metadata?.assistant_knowledge_base_ids;
const llmwikiIds = Array.isArray(raw)
? raw.map((item) => String(item).trim()).filter(Boolean)
: [];
const assistantIds = Array.isArray(assistantRaw)
? assistantRaw.map((item) => String(item).trim()).filter(Boolean)
: [];
if (Array.isArray(raw)) {
setSelectedKnowledgeBaseIds(
raw.map((item) => String(item).trim()).filter(Boolean),
);
setKnowledgeSelectionReady(true);
setSelectedKnowledgeBaseIds(llmwikiIds);
} else {
setSelectedKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
}
if (Array.isArray(assistantRaw)) {
setSelectedAssistantKnowledgeBaseIds(assistantIds);
} else {
setSelectedAssistantKnowledgeBaseIds([]);
}
setKnowledgeSelectionReady(Array.isArray(raw) || Array.isArray(assistantRaw));
})
.catch(() => {
if (knowledgeSelectionThread.current === threadId) {
setSelectedKnowledgeBaseIds([]);
setSelectedAssistantKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
}
})
@@ -353,10 +400,13 @@ export default function AgentChatPage({
() => ({
...(taskId ? taskThreadMetadata(taskContext) : {}),
...(knowledgeSelectionReady
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
? {
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
}
: {}),
}),
[knowledgeSelectionReady, selectedKnowledgeBaseIds, taskContext, taskId],
[knowledgeSelectionReady, selectedAssistantKnowledgeBaseIds, selectedKnowledgeBaseIds, taskContext, taskId],
);
const [thread, sendMessage] = useThreadStream({
threadId: isNewThread ? undefined : threadId,
@@ -365,7 +415,10 @@ export default function AgentChatPage({
...settings.context,
agent_id,
...(knowledgeSelectionReady
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
? {
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
}
: {}),
},
// 任务闸门:新建 thread 打 metadata.taskId 标签,供任务对话列表(1:N)按任务过滤;
@@ -713,9 +766,13 @@ export default function AgentChatPage({
source="agent"
agentId={agent_id}
value={selectedKnowledgeBaseIds}
assistantValue={selectedAssistantKnowledgeBaseIds}
onChange={changeKnowledgeSelection}
onAssistantChange={changeAssistantKnowledgeSelection}
autoSelectAll={
isNewThread && pendingKnowledgeSelection.current === null
isNewThread &&
pendingKnowledgeSelection.current === null &&
pendingAssistantKnowledgeSelection.current === null
}
disabled={isDemoMode || knowledgeSelectionLoading}
label="知识空间"
@@ -781,7 +838,9 @@ export default function AgentChatPage({
source="agent"
agentId={agent_id}
value={selectedKnowledgeBaseIds}
assistantValue={selectedAssistantKnowledgeBaseIds}
onChange={changeKnowledgeSelection}
onAssistantChange={changeAssistantKnowledgeSelection}
disabled={isDemoMode || knowledgeSelectionLoading}
readOnly
label="知识空间"
+169 -43
View File
@@ -3,12 +3,15 @@
import { useQuery } from "@tanstack/react-query";
import {
ArrowDownWideNarrow,
Bot,
ChevronLeft,
ChevronRight,
Download,
FileText,
FileX2,
Library,
Loader2,
MessagesSquare,
MessageSquareShare,
RefreshCw,
Search,
@@ -20,17 +23,28 @@ import { toast } from "sonner";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { ScrollArea } from "@/components/ui/scroll-area";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { Skeleton } from "@/components/ui/skeleton";
import {
WorkspaceBody,
WorkspaceContainer,
WorkspaceHeader,
} from "@/components/workspace/workspace-container";
import { downloadArtifactFile } from "@/core/artifacts/download";
import {
downloadArtifactFile,
downloadArtifactMarkdownAsWord,
} from "@/core/artifacts/download";
import {
fetchArtifactLibrary,
type ArtifactLibraryItem,
type ArtifactLibraryKind,
type ArtifactLibrarySource,
} from "@/core/artifacts/library";
import { useI18n } from "@/core/i18n/hooks";
import { bootstrapResumePath, pathOfThread } from "@/core/threads/utils";
@@ -53,6 +67,28 @@ const FILTERS: Array<{ value: ArtifactLibraryKind; label: string }> = [
{ value: "other", label: "其他" },
];
const SOURCE_FILTERS: Array<{
value: ArtifactLibrarySource;
label: string;
}> = [
{ value: "all", label: "全部来源" },
{ value: "general_qa", label: "通用问答" },
{ value: "agent", label: "智能体" },
];
type DownloadFormat = "original" | "word";
function isMarkdownArtifact(name: string): boolean {
return /\.(md|markdown)$/i.test(name);
}
function artifactDownloadKey(
item: ArtifactLibraryItem,
format: DownloadFormat,
): string {
return `${item.threadId}:${item.path}:${format}`;
}
function formatBytes(bytes: number): string {
if (!Number.isFinite(bytes) || bytes <= 0) return "0 B";
const units = ["B", "KB", "MB", "GB"];
@@ -83,12 +119,19 @@ function FileRow({
onDownload,
}: {
item: ArtifactLibraryItem;
downloading: boolean;
onDownload: (item: ArtifactLibraryItem) => void;
downloading: DownloadFormat | null;
onDownload: (item: ArtifactLibraryItem, format: DownloadFormat) => void;
}) {
const sourcePath = sourceConversationPath(item);
const displayName = desensitize(item.name);
const sourceTitle = desensitize(item.threadTitle || "新对话");
const sourceLabel = item.sourceType === "agent" ? "智能体" : "通用问答";
const sourceName =
item.sourceType === "agent"
? desensitize(item.agentName || item.agentId || "未知智能体")
: sourceLabel;
const SourceIcon = item.sourceType === "agent" ? Bot : MessagesSquare;
const busy = downloading !== null;
return (
<tr className="group border-b transition-colors hover:bg-muted/35">
@@ -106,7 +149,8 @@ function FileRow({
{displayName}
</span>
<span className="mt-1 block truncate text-xs text-muted-foreground lg:hidden">
{getFileExtensionDisplayName(item.name)} · 来自「{sourceTitle}」
{getFileExtensionDisplayName(item.name)} · {sourceName} · 来自「
{sourceTitle}」
</span>
<span className="mt-1 hidden text-xs text-muted-foreground lg:block">
{getFileExtensionDisplayName(item.name)} 文件
@@ -115,13 +159,24 @@ function FileRow({
</Link>
</td>
<td className="hidden min-w-0 px-3 py-3 lg:table-cell">
<Link
href={sourcePath}
className="block truncate text-sm text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
title={`打开原对话:${sourceTitle}`}
>
{sourceTitle}
</Link>
<div className="flex min-w-0 items-center gap-2.5">
<span className="flex size-8 shrink-0 items-center justify-center rounded-md bg-muted text-muted-foreground">
<SourceIcon className="size-4" />
</span>
<span className="min-w-0">
<span className="block truncate text-sm font-medium text-foreground">
{sourceName}
</span>
<Link
href={sourcePath}
className="block truncate text-xs text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
title={`打开原对话:${sourceTitle}`}
>
{item.sourceType === "agent" ? "智能体 · " : ""}
{sourceTitle}
</Link>
</span>
</div>
</td>
<td className="whitespace-nowrap px-3 py-3 text-sm text-muted-foreground">
{formatDateTime(item.modifiedAt)}
@@ -136,17 +191,36 @@ function FileRow({
size="icon"
variant="ghost"
className="size-8"
disabled={downloading}
onClick={() => onDownload(item)}
title={`下载 ${displayName}`}
aria-label={`下载 ${displayName}`}
disabled={busy}
onClick={() => onDownload(item, "original")}
title={`下载原文件:${displayName}`}
aria-label={`下载原文件:${displayName}`}
>
{downloading ? (
{downloading === "original" ? (
<Loader2 className="size-4 animate-spin" />
) : (
<Download className="size-4" />
)}
</Button>
{isMarkdownArtifact(item.name) && (
<Button
type="button"
size="sm"
variant="ghost"
className="h-8 gap-1 px-2 text-xs"
disabled={busy}
onClick={() => onDownload(item, "word")}
title={`下载为 Word:${displayName}`}
aria-label={`下载为 Word:${displayName}`}
>
{downloading === "word" ? (
<Loader2 className="size-4 animate-spin" />
) : (
<FileText className="size-4" />
)}
Word
</Button>
)}
<Button
asChild
size="icon"
@@ -189,6 +263,7 @@ export default function ArtifactLibraryPage() {
const [search, setSearch] = useState("");
const [debouncedSearch, setDebouncedSearch] = useState("");
const [fileKind, setFileKind] = useState<ArtifactLibraryKind>("all");
const [sourceType, setSourceType] = useState<ArtifactLibrarySource>("all");
const [page, setPage] = useState(1);
const [downloadingKey, setDownloadingKey] = useState<string | null>(null);
const forceRefreshRef = useRef(false);
@@ -204,17 +279,18 @@ export default function ArtifactLibraryPage() {
useEffect(() => {
setPage(1);
}, [debouncedSearch, fileKind]);
}, [debouncedSearch, fileKind, sourceType]);
const offset = (page - 1) * PAGE_SIZE;
const libraryQuery = useQuery({
queryKey: ["artifact-library", debouncedSearch, fileKind, page],
queryKey: ["artifact-library", debouncedSearch, fileKind, sourceType, page],
queryFn: async ({ signal }) => {
const refresh = forceRefreshRef.current;
try {
return await fetchArtifactLibrary({
query: debouncedSearch,
fileKind,
sourceType,
offset,
limit: PAGE_SIZE,
refresh,
@@ -238,19 +314,39 @@ export default function ArtifactLibraryPage() {
}, [page, totalPages]);
const itemKeys = useMemo(
() => new Set(items.map((item) => `${item.threadId}:${item.path}`)),
() =>
new Set(
items.flatMap((item) => [
artifactDownloadKey(item, "original"),
artifactDownloadKey(item, "word"),
]),
),
[items],
);
useEffect(() => {
if (downloadingKey && !itemKeys.has(downloadingKey)) setDownloadingKey(null);
}, [downloadingKey, itemKeys]);
const handleDownload = async (item: ArtifactLibraryItem) => {
const key = `${item.threadId}:${item.path}`;
const handleDownload = async (
item: ArtifactLibraryItem,
format: DownloadFormat,
) => {
const key = artifactDownloadKey(item, format);
if (downloadingKey) return;
setDownloadingKey(key);
try {
await downloadArtifactFile({ filepath: item.path, threadId: item.threadId });
if (format === "word") {
await downloadArtifactMarkdownAsWord({
filepath: item.path,
threadId: item.threadId,
});
toast.success("Word 文件已生成");
} else {
await downloadArtifactFile({
filepath: item.path,
threadId: item.threadId,
});
}
} catch (error) {
console.error(error);
toast.error(error instanceof Error ? error.message : "下载失败");
@@ -298,26 +394,49 @@ export default function ArtifactLibraryPage() {
<Input
type="search"
className="h-11 pl-9"
placeholder="搜索文件名或来源对话"
placeholder="搜索文件名、来源对话或智能体名称"
value={search}
onChange={(event) => setSearch(event.target.value)}
/>
</div>
<div className="flex flex-wrap items-center justify-between gap-3">
<div className="flex flex-wrap gap-1" aria-label="文件类型筛选">
{FILTERS.map((filter) => (
<Button
key={filter.value}
type="button"
<div className="flex flex-wrap items-center justify-between gap-4">
<div className="flex flex-wrap items-center gap-3">
<div className="flex flex-wrap gap-1" aria-label="文件类型筛选">
{FILTERS.map((filter) => (
<Button
key={filter.value}
type="button"
size="sm"
variant={fileKind === filter.value ? "secondary" : "ghost"}
aria-pressed={fileKind === filter.value}
onClick={() => setFileKind(filter.value)}
>
{filter.label}
</Button>
))}
</div>
<Select
value={sourceType}
onValueChange={(value) =>
setSourceType(value as ArtifactLibrarySource)
}
>
<SelectTrigger
size="sm"
variant={fileKind === filter.value ? "secondary" : "ghost"}
aria-pressed={fileKind === filter.value}
onClick={() => setFileKind(filter.value)}
className="w-32"
aria-label="筛选文件来源"
>
{filter.label}
</Button>
))}
<SelectValue />
</SelectTrigger>
<SelectContent>
{SOURCE_FILTERS.map((filter) => (
<SelectItem key={filter.value} value={filter.value}>
{filter.label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div className="flex items-center gap-2 text-xs text-muted-foreground">
<ArrowDownWideNarrow className="size-4" />
@@ -337,11 +456,11 @@ export default function ArtifactLibraryPage() {
<table className="w-full table-fixed border-collapse">
<thead>
<tr className="border-b text-left text-xs font-normal text-muted-foreground">
<th className="w-[48%] py-3 pr-4 pl-1 font-normal lg:w-[38%]">文件名</th>
<th className="hidden w-[27%] px-3 py-3 font-normal lg:table-cell">来源对话</th>
<th className="w-[34%] px-3 py-3 font-normal md:w-[22%] lg:w-[17%]">生成时间</th>
<th className="hidden w-[13%] px-3 py-3 font-normal md:table-cell">大小</th>
<th className="w-[18%] py-3 pr-1 pl-3 text-right font-normal md:w-[10%]">
<th className="w-[48%] py-3 pr-4 pl-1 font-normal lg:w-[35%]">文件名</th>
<th className="hidden w-[25%] px-3 py-3 font-normal lg:table-cell">来源</th>
<th className="w-[34%] px-3 py-3 font-normal md:w-[22%] lg:w-[16%]">生成时间</th>
<th className="hidden w-[13%] px-3 py-3 font-normal lg:w-[10%] md:table-cell">大小</th>
<th className="w-[18%] py-3 pr-1 pl-3 text-right font-normal md:w-[17%] lg:w-[14%]">
<span className="sr-only">操作</span>
</th>
</tr>
@@ -352,12 +471,19 @@ export default function ArtifactLibraryPage() {
) : (
items.map((item) => {
const key = `${item.threadId}:${item.path}`;
const activeDownload = downloadingKey?.startsWith(`${key}:`)
? downloadingKey.endsWith(":word")
? "word"
: "original"
: null;
return (
<FileRow
key={key}
item={item}
downloading={downloadingKey === key}
onDownload={(target) => void handleDownload(target)}
downloading={activeDownload}
onDownload={(target, format) =>
void handleDownload(target, format)
}
/>
);
})
@@ -388,7 +514,7 @@ export default function ArtifactLibraryPage() {
<FileX2 className="size-9 text-muted-foreground" />
<div>
<p className="text-sm font-medium">
{debouncedSearch || fileKind !== "all"
{debouncedSearch || fileKind !== "all" || sourceType !== "all"
? "没有找到匹配的文件"
: "还没有生成文件"}
</p>
+409 -35
View File
@@ -1,74 +1,448 @@
import { Database, FileUp, History, Loader2, RefreshCw, Search, ShieldCheck, Trash2 } from "lucide-react";
import { useMemo, useRef, useState } from "react";
import { BookOpen, Database, Download, Loader2, Plus, RefreshCw, Search, ShieldCheck, Trash2, Upload } from "lucide-react";
import { useEffect, useMemo, useRef, useState } from "react";
import { toast } from "sonner";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Input } from "@/components/ui/input";
import { ScrollArea } from "@/components/ui/scroll-area";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { MarkdownContent } from "@/components/workspace/messages/markdown-content";
import { Textarea } from "@/components/ui/textarea";
import { KnowledgeProductNav } from "@/components/workspace/knowledge-product-nav";
import { MarkdownContent } from "@/components/workspace/messages/markdown-content";
import { getStoredAuth } from "@/core/auth";
import {
listAssistantWikiRevisions,
type AssistantKnowledgeBase,
type AssistantWikiPage,
type AssistantWikiRevision,
useAssistantImportJobs,
useAssistantKnowledgeActions,
useAssistantKnowledgeBase,
useAssistantKnowledgeBases,
useAssistantKnowledgeSources,
useAssistantWikiPages,
} from "@/strategy-components/api/assistant-knowledge";
function baseSubtitle(base: AssistantKnowledgeBase) {
if (base.is_global) return "全库沉淀总库 · 所有 WeKnora 导入数据都会同步进入这里并做实体对齐";
return base.description || "自建助手知识库 · 可上传普通知识库导出的 Wiki 向量包";
}
function EmptyInitializeCard({
isAdmin,
onInitialize,
pending,
}: {
isAdmin: boolean;
onInitialize: () => void;
pending: boolean;
}) {
return (
<div className="flex flex-1 items-center justify-center p-8">
<div className="max-w-lg rounded-xl border bg-card p-8 text-center">
<Database className="mx-auto mb-4 size-10 text-primary" />
<h1 className="text-xl font-semibold">初始化助手知识库</h1>
<p className="mt-2 text-sm text-muted-foreground">
未配置 WeKnora 时会默认展示助手知识库。首次使用需要创建“知识梳理总库”,后续从 WeKnora 导入的数据都会沉淀到这里。
</p>
{isAdmin ? (
<Button className="mt-5" disabled={pending} onClick={onInitialize}>
{pending ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Database className="mr-2 size-4" />}
创建知识梳理总库
</Button>
) : (
<p className="mt-5 text-sm">请联系管理员初始化。</p>
)}
</div>
</div>
);
}
export default function AssistantKnowledgePage() {
const isAdmin = getStoredAuth()?.system_role === "admin";
const [query, setQuery] = useState("");
const [selectedBaseId, setSelectedBaseId] = useState<string | null>(null);
const [selected, setSelected] = useState<AssistantWikiPage | null>(null);
const [revisions, setRevisions] = useState<AssistantWikiRevision[]>([]);
const fileRef = useRef<HTMLInputElement>(null);
const base = useAssistantKnowledgeBase();
const pages = useAssistantWikiPages(query);
const sources = useAssistantKnowledgeSources();
const jobs = useAssistantImportJobs();
const [createOpen, setCreateOpen] = useState(false);
const [createDraft, setCreateDraft] = useState({ name: "", description: "" });
const packageInputRef = useRef<HTMLInputElement | null>(null);
const bases = useAssistantKnowledgeBases();
const actions = useAssistantKnowledgeActions();
useEffect(() => {
const values = bases.data ?? [];
if (!values.length) {
setSelectedBaseId(null);
return;
}
if (selectedBaseId && values.some((base) => base.id === selectedBaseId)) return;
setSelectedBaseId((values.find((base) => base.is_global) ?? values[0]).id);
}, [bases.data, selectedBaseId]);
useEffect(() => {
setSelected(null);
}, [selectedBaseId]);
const selectedBase = useMemo(
() => (bases.data ?? []).find((base) => base.id === selectedBaseId) ?? null,
[bases.data, selectedBaseId],
);
const pages = useAssistantWikiPages(query, selectedBaseId);
const sources = useAssistantKnowledgeSources(selectedBaseId);
const jobs = useAssistantImportJobs(selectedBaseId);
const activePage = selected ?? pages.data?.[0] ?? null;
const counts = useMemo(
() => ({
pages: pages.data?.length ?? selectedBase?.page_count ?? 0,
sources: sources.data?.length ?? selectedBase?.source_count ?? 0,
jobs: jobs.data?.length ?? 0,
}),
[jobs.data, pages.data, selectedBase?.page_count, selectedBase?.source_count, sources.data],
);
const counts = useMemo(() => ({ pages: pages.data?.length ?? 0, sources: sources.data?.length ?? 0, jobs: jobs.data?.length ?? 0 }), [pages.data, sources.data, jobs.data]);
const initializeGlobal = () => {
actions.initialize.mutate("知识梳理总库", {
onSuccess: (base) => {
setSelectedBaseId(base.id);
toast.success("知识梳理总库已创建");
},
onError: (error) => toast.error(error.message),
});
};
if (base.isLoading) return <div className="flex size-full flex-col"><KnowledgeProductNav /><div className="flex flex-1 items-center justify-center"><Loader2 className="animate-spin" /></div></div>;
if (!base.data) {
return <div className="flex size-full flex-col"><KnowledgeProductNav /><div className="flex flex-1 items-center justify-center p-8"><div className="max-w-lg rounded-xl border bg-card p-8 text-center"><Database className="mx-auto mb-4 size-10 text-primary" /><h1 className="text-xl font-semibold">创建“知识梳理”助手知识库</h1><p className="mt-2 text-sm text-muted-foreground">无需配置普通知识库服务。创建后所有用户可查看 Wiki 与搜索,只有管理员可导入和维护。</p>{isAdmin ? <Button className="mt-5" disabled={actions.initialize.isPending} onClick={() => actions.initialize.mutate("知识梳理")}>创建知识梳理库</Button> : <p className="mt-5 text-sm">请联系管理员初始化。</p>}</div></div></div>;
}
const uploadFiles = async (files: FileList | null) => {
if (!files?.length) return;
const createBase = async () => {
const name = createDraft.name.trim();
if (!name) {
toast.error("请输入助手知识库名称");
return;
}
try {
await actions.upload.mutateAsync(Array.from(files));
toast.success("文件已安全扫描并导入");
const base = await actions.createBase.mutateAsync({
name,
description: createDraft.description.trim(),
});
setSelectedBaseId(base.id);
setCreateDraft({ name: "", description: "" });
setCreateOpen(false);
toast.success("助手知识库已创建");
} catch (error) {
toast.error(error instanceof Error ? error.message : String(error));
} finally {
if (fileRef.current) fileRef.current.value = "";
}
};
const openHistory = async () => {
if (!activePage) return;
setRevisions(await listAssistantWikiRevisions(activePage.slug));
const uploadPackage = async (file: File | null | undefined) => {
if (!file) return;
try {
await actions.uploadWeKnoraPackage.mutateAsync({
file,
assistantBaseId: selectedBaseId,
});
toast.success("导出包已上传,后台正在异步导入");
} catch (error) {
toast.error(error instanceof Error ? error.message : String(error));
} finally {
if (packageInputRef.current) packageInputRef.current.value = "";
}
};
if (bases.isLoading) {
return (
<div className="flex size-full flex-col">
<KnowledgeProductNav />
<div className="flex flex-1 items-center justify-center">
<Loader2 className="animate-spin" />
</div>
</div>
);
}
if (!bases.data?.length) {
return (
<div className="flex size-full flex-col">
<KnowledgeProductNav />
<EmptyInitializeCard
isAdmin={isAdmin}
pending={actions.initialize.isPending}
onInitialize={initializeGlobal}
/>
</div>
);
}
return (
<div className="flex size-full min-h-0 flex-col bg-background">
<KnowledgeProductNav />
<header className="flex items-center gap-4 border-b px-6 py-4"><div><h1 className="text-xl font-semibold">{base.data.name}</h1><p className="text-xs text-muted-foreground">全局助手知识库 · 普通用户只读</p></div><Badge variant="outline"><ShieldCheck className="mr-1 size-3" />实体对齐与版本管理</Badge><div className="ml-auto flex gap-2">{isAdmin && <><input ref={fileRef} className="hidden" type="file" multiple onChange={(event) => void uploadFiles(event.target.files)} /><Button variant="outline" onClick={() => fileRef.current?.click()}><FileUp />导入本地文件</Button></>}<Button variant="outline" disabled={!activePage} onClick={() => void openHistory()}><History />页面版本</Button></div></header>
<Tabs defaultValue="wiki" className="flex min-h-0 flex-1 flex-col">
<div className="flex items-center gap-4 border-b px-6"><TabsList className="my-2"><TabsTrigger value="wiki">Wiki({counts.pages})</TabsTrigger><TabsTrigger value="sources">来源({counts.sources})</TabsTrigger><TabsTrigger value="jobs">导入记录({counts.jobs})</TabsTrigger><TabsTrigger value="revisions">版本({revisions.length})</TabsTrigger></TabsList><div className="relative ml-auto w-72"><Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" /><Input value={query} onChange={(event) => setQuery(event.target.value)} placeholder="搜索标题、实体别名和正文" className="pl-8" /></div></div>
<TabsContent value="wiki" className="mt-0 grid min-h-0 flex-1 grid-cols-[300px_1fr]"><ScrollArea className="border-r"><div className="space-y-1 p-3">{pages.data?.map((page) => <button key={page.id} type="button" onClick={() => setSelected(page)} className={`w-full rounded-md px-3 py-2 text-left ${activePage?.id === page.id ? "bg-primary/10 text-primary" : "hover:bg-muted"}`}><div className="truncate text-sm font-medium">{page.title}</div><div className="mt-1 flex items-center gap-2 text-[11px] text-muted-foreground"><span>{page.page_type}</span><span>{new Date(page.updated_at).toLocaleString()}</span></div></button>)}</div></ScrollArea><ScrollArea><article className="mx-auto max-w-5xl p-8">{activePage ? <><h1 className="mb-1 text-2xl font-semibold">{activePage.title}</h1><p className="mb-6 text-xs text-muted-foreground">{activePage.slug}</p><MarkdownContent content={activePage.content_markdown} isLoading={false} rehypePlugins={[]} /></> : <p className="text-sm text-muted-foreground">暂无 Wiki 页面。</p>}</article></ScrollArea></TabsContent>
<TabsContent value="sources" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">{sources.data?.map((source) => <div key={source.id} className="rounded-lg border bg-card p-4"><div className="flex items-center gap-2"><Database className="size-4" /><span className="font-medium">{source.source_name}</span><Badge variant="outline" className="ml-auto">{source.source_type}</Badge></div><p className="mt-3 text-xs text-muted-foreground">业务键:{source.source_key}</p>{isAdmin && <div className="mt-4 flex justify-end gap-2"><Button size="sm" variant="outline" disabled={source.source_type === "local_file" || actions.reimportSource.isPending} onClick={() => actions.reimportSource.mutate(source.id, { onSuccess: () => toast.success("已发起重新导入"), onError: (error) => toast.error(error.message) })}><RefreshCw />重新导入</Button><Button size="sm" variant="destructive" disabled={actions.deleteSource.isPending} onClick={() => { if (window.confirm(`确认删除来源“${source.source_name}”的贡献?仍被其他来源引用的知识会保留。`)) actions.deleteSource.mutate(source.id, { onSuccess: () => toast.success("来源贡献已删除"), onError: (error) => toast.error(error.message) }); }}><Trash2 />删除来源</Button></div>}</div>)}</div></TabsContent>
<TabsContent value="jobs" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="space-y-3">{jobs.data?.map((job) => <div key={job.id} className="flex items-center gap-3 rounded-lg border bg-card p-4"><Badge>{job.status}</Badge><span className="text-sm">{job.trigger}</span><span className="text-xs text-muted-foreground">{Object.entries(job.counts || {}).map(([key, value]) => `${key} ${value}`).join(" · ")}</span><span className="ml-auto text-xs">{new Date(job.created_at).toLocaleString()}</span></div>)}</div></TabsContent>
<TabsContent value="revisions" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="space-y-3">{revisions.map((revision) => <div key={revision.id} className="flex items-center gap-3 rounded-lg border bg-card p-4"><Badge variant="outline">v{revision.revision_no}</Badge><span className="text-sm">{revision.change_type}</span><span className="text-xs text-muted-foreground">{revision.summary}</span><span className="ml-auto text-xs">{new Date(revision.created_at).toLocaleString()}</span>{isAdmin && activePage && revision.id !== activePage.current_revision_id && <Button size="sm" variant="outline" onClick={() => actions.rollback.mutate({ slug: activePage.slug, revisionId: revision.id })}>回滚到此版本</Button>}</div>)}{!revisions.length && <p className="text-sm text-muted-foreground">请先在 Wiki 页选择页面并点击“页面版本”。</p>}</div></TabsContent>
</Tabs>
<div className="grid min-h-0 flex-1 grid-cols-[280px_1fr]">
<aside className="flex min-h-0 flex-col border-r bg-muted/20">
<div className="border-b p-4">
<div className="flex items-center justify-between">
<div>
<h2 className="text-sm font-semibold">助手知识库</h2>
<p className="mt-1 text-xs text-muted-foreground">总库置顶,自建库可上传 WeKnora 导出包</p>
</div>
{isAdmin && (
<Button size="icon" variant="outline" onClick={() => setCreateOpen(true)}>
<Plus className="size-4" />
</Button>
)}
</div>
</div>
<ScrollArea className="min-h-0 flex-1">
<div className="space-y-2 p-3">
{(bases.data ?? []).map((base) => (
<button
key={base.id}
type="button"
onClick={() => setSelectedBaseId(base.id)}
className={`w-full rounded-lg border px-3 py-3 text-left transition ${
selectedBaseId === base.id ? "border-primary bg-primary/10 text-primary" : "bg-card hover:bg-muted"
}`}
>
<div className="flex items-center gap-2">
<Database className="size-4 shrink-0" />
<span className="min-w-0 flex-1 truncate text-sm font-medium">{base.name}</span>
{base.is_global && <Badge variant="outline" className="text-[10px]">总库</Badge>}
</div>
<div className="mt-2 flex gap-3 text-[11px] text-muted-foreground">
<span>{base.page_count ?? 0} 页</span>
<span>{base.source_count ?? 0} 来源</span>
<span>{base.entity_count ?? 0} 实体</span>
</div>
</button>
))}
</div>
</ScrollArea>
</aside>
<main className="flex min-h-0 flex-col">
<header className="flex items-center gap-4 border-b px-6 py-4">
<div className="min-w-0">
<h1 className="truncate text-xl font-semibold">{selectedBase?.name}</h1>
<p className="mt-1 truncate text-xs text-muted-foreground">
{selectedBase ? baseSubtitle(selectedBase) : "请选择助手知识库"}
</p>
</div>
<Badge variant="outline">
<ShieldCheck className="mr-1 size-3" />
实体对齐
</Badge>
{isAdmin && (
<div className="ml-auto flex items-center gap-2">
<input
ref={packageInputRef}
type="file"
accept=".jsonl,.ndjson,application/x-ndjson,application/json"
className="hidden"
onChange={(event) => void uploadPackage(event.target.files?.[0])}
/>
<Button
variant="outline"
disabled={actions.uploadWeKnoraPackage.isPending || !selectedBaseId}
onClick={() => packageInputRef.current?.click()}
>
{actions.uploadWeKnoraPackage.isPending ? (
<Loader2 className="mr-2 size-4 animate-spin" />
) : (
<Upload className="mr-2 size-4" />
)}
上传 WeKnora 导出包
</Button>
</div>
)}
</header>
<Tabs defaultValue="wiki" className="flex min-h-0 flex-1 flex-col">
<div className="flex items-center gap-4 border-b px-6">
<TabsList className="my-2">
<TabsTrigger value="wiki">Wiki({counts.pages})</TabsTrigger>
<TabsTrigger value="sources">来源({counts.sources})</TabsTrigger>
<TabsTrigger value="jobs">导入记录({counts.jobs})</TabsTrigger>
</TabsList>
<div className="relative ml-auto w-72">
<Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
<Input
value={query}
onChange={(event) => setQuery(event.target.value)}
placeholder="搜索标题、实体别名和正文"
className="pl-8"
/>
</div>
</div>
<TabsContent value="wiki" className="mt-0 grid min-h-0 flex-1 grid-cols-[300px_1fr]">
<ScrollArea className="border-r">
<div className="space-y-1 p-3">
{pages.isLoading && <Loader2 className="m-4 size-5 animate-spin" />}
{pages.data?.map((page) => (
<button
key={page.id}
type="button"
onClick={() => setSelected(page)}
className={`w-full rounded-md px-3 py-2 text-left ${
activePage?.id === page.id ? "bg-primary/10 text-primary" : "hover:bg-muted"
}`}
>
<div className="truncate text-sm font-medium">{page.title}</div>
<div className="mt-1 flex items-center gap-2 text-[11px] text-muted-foreground">
<span>{page.page_type}</span>
<span>{new Date(page.updated_at).toLocaleString()}</span>
</div>
</button>
))}
{!pages.isLoading && !pages.data?.length && (
<div className="p-4 text-xs text-muted-foreground">
暂无 Wiki 页面。请先到普通知识库详情页下载向量化数据包,再在当前助手知识库上传。
</div>
)}
</div>
</ScrollArea>
<ScrollArea>
<article className="mx-auto max-w-5xl p-8">
{activePage ? (
<>
<h1 className="mb-1 flex items-center gap-2 text-2xl font-semibold">
<BookOpen className="size-5 text-primary" />
{activePage.title}
</h1>
<p className="mb-6 text-xs text-muted-foreground">{activePage.slug}</p>
<MarkdownContent content={activePage.content_markdown} isLoading={false} rehypePlugins={[]} />
</>
) : (
<p className="text-sm text-muted-foreground">暂无 Wiki 页面。</p>
)}
</article>
</ScrollArea>
</TabsContent>
<TabsContent value="sources" className="mt-0 min-h-0 flex-1 overflow-auto p-6">
<div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">
{sources.data?.map((source) => (
<div key={source.id} className="rounded-lg border bg-card p-4">
<div className="flex items-center gap-2">
<Database className="size-4" />
<span className="truncate font-medium">{source.source_name}</span>
<Badge variant="outline" className="ml-auto">{source.source_type}</Badge>
</div>
<p className="mt-3 text-xs text-muted-foreground">业务键:{source.source_key}</p>
{isAdmin && (
<div className="mt-4 flex justify-end gap-2">
<Button
size="sm"
variant="outline"
disabled={source.source_type !== "weknora" || actions.reimportSource.isPending}
onClick={() =>
actions.reimportSource.mutate(source.id, {
onSuccess: () => toast.success("已发起重新导入"),
onError: (error) => toast.error(error.message),
})
}
>
<RefreshCw className="mr-2 size-4" />
重新导入
</Button>
<Button
size="sm"
variant="destructive"
disabled={actions.deleteSource.isPending}
onClick={() => {
if (window.confirm(`确认删除来源“${source.source_name}”的贡献?仍被其他来源引用的知识会保留。`)) {
actions.deleteSource.mutate(source.id, {
onSuccess: () => toast.success("来源贡献已删除"),
onError: (error) => toast.error(error.message),
});
}
}}
>
<Trash2 className="mr-2 size-4" />
删除来源
</Button>
</div>
)}
</div>
))}
{!sources.data?.length && <p className="text-sm text-muted-foreground">暂无来源。</p>}
</div>
</TabsContent>
<TabsContent value="jobs" className="mt-0 min-h-0 flex-1 overflow-auto p-6">
<div className="space-y-3">
{jobs.data?.map((job) => (
<div key={job.id} className="rounded-lg border bg-card p-4">
<div className="flex items-center gap-3">
<Badge>{job.status}</Badge>
<span className="text-sm">{job.trigger}</span>
<span className="text-xs text-muted-foreground">阶段:{job.phase}</span>
<span className="ml-auto text-xs">{new Date(job.created_at).toLocaleString()}</span>
{isAdmin && job.package_download_url && (
<Button asChild size="sm" variant="outline">
<a href={job.package_download_url} target="_blank" rel="noreferrer">
<Download className="mr-2 size-4" />
下载导出包
</a>
</Button>
)}
</div>
<div className="mt-2 text-xs text-muted-foreground">
{Object.entries(job.counts || {}).map(([key, value]) => `${key} ${value}`).join(" · ") || "暂无统计"}
</div>
{job.last_error_message && (
<div className="mt-2 rounded-md bg-destructive/10 px-3 py-2 text-xs text-destructive">
{job.last_error_message}
</div>
)}
</div>
))}
{!jobs.data?.length && <p className="text-sm text-muted-foreground">暂无导入记录。</p>}
</div>
</TabsContent>
</Tabs>
</main>
</div>
<Dialog open={createOpen} onOpenChange={setCreateOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>新建助手知识库</DialogTitle>
<DialogDescription>
自建助手知识库用于承接从 WeKnora 普通知识库下载的 Wiki 向量包;导入时同一份数据也会沉淀到“知识梳理总库”。
</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div className="space-y-1.5">
<label className="text-sm font-medium">名称</label>
<Input
value={createDraft.name}
onChange={(event) => setCreateDraft((prev) => ({ ...prev, name: event.target.value }))}
placeholder="例如:政策资料库"
/>
</div>
<div className="space-y-1.5">
<label className="text-sm font-medium">描述</label>
<Textarea
value={createDraft.description}
onChange={(event) => setCreateDraft((prev) => ({ ...prev, description: event.target.value }))}
placeholder="可选,用于说明这个助手知识库的范围"
rows={3}
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setCreateOpen(false)}>
取消
</Button>
<Button disabled={actions.createBase.isPending} onClick={() => void createBase()}>
{actions.createBase.isPending ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Plus className="mr-2 size-4" />}
创建
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
+44 -9
View File
@@ -132,6 +132,7 @@ export default function ChatPage() {
// per-thread header toggle so stale localStorage can't re-enable it.
const [knowledgeRagEnabled] = useKnowledgeRagEnabled();
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>([]);
const [selectedAssistantKnowledgeBaseIds, setSelectedAssistantKnowledgeBaseIds] = useState<string[]>([]);
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(false);
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
const knowledgeSelectionThread = useRef<string | null>(null);
@@ -140,18 +141,26 @@ export default function ChatPage() {
setSelectedKnowledgeBaseIds(ids);
setKnowledgeSelectionReady(true);
}, []);
const changeAssistantKnowledgeSelection = useCallback((ids: string[]) => {
setSelectedAssistantKnowledgeBaseIds(ids);
setKnowledgeSelectionReady(true);
}, []);
const streamContext = useMemo(
() => ({
...settings.context,
knowledge_rag_enabled: knowledgeRagEnabled,
...(knowledgeSelectionReady
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
? {
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
}
: {}),
}),
[
settings.context,
knowledgeRagEnabled,
knowledgeSelectionReady,
selectedAssistantKnowledgeBaseIds,
selectedKnowledgeBaseIds,
],
);
@@ -190,6 +199,7 @@ export default function ChatPage() {
setHasStartedConversation(false);
setPromptPrefixOn(false);
setSelectedKnowledgeBaseIds([]);
setSelectedAssistantKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
setKnowledgeSelectionLoading(false);
knowledgeSelectionThread.current = "__new__";
@@ -215,18 +225,29 @@ export default function ChatPage() {
if (knowledgeSelectionThread.current !== threadId) return;
const metadata = record.metadata as Record<string, unknown> | undefined;
const raw = metadata?.llmwiki_knowledge_base_ids;
const assistantRaw = metadata?.assistant_knowledge_base_ids;
const llmwikiIds = Array.isArray(raw)
? raw.map((item) => String(item).trim()).filter(Boolean)
: [];
const assistantIds = Array.isArray(assistantRaw)
? assistantRaw.map((item) => String(item).trim()).filter(Boolean)
: [];
if (Array.isArray(raw)) {
setSelectedKnowledgeBaseIds(
raw.map((item) => String(item).trim()).filter(Boolean),
);
setKnowledgeSelectionReady(true);
setSelectedKnowledgeBaseIds(llmwikiIds);
} else {
setSelectedKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
}
if (Array.isArray(assistantRaw)) {
setSelectedAssistantKnowledgeBaseIds(assistantIds);
} else {
setSelectedAssistantKnowledgeBaseIds([]);
}
setKnowledgeSelectionReady(Array.isArray(raw) || Array.isArray(assistantRaw));
})
.catch(() => {
if (knowledgeSelectionThread.current === threadId) {
setSelectedKnowledgeBaseIds([]);
setSelectedAssistantKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
}
})
@@ -249,6 +270,7 @@ export default function ChatPage() {
if (!known.some((a) => a.id === selectedAgentId)) {
setSelectedAgentId(null);
setSelectedKnowledgeBaseIds([]);
setSelectedAssistantKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
}
}, [selectedAgentId, selectorAgents, mineAgents, squareAgents, selectorLoading]);
@@ -259,7 +281,10 @@ export default function ChatPage() {
threadId: isNewThread ? undefined : threadId,
context: streamContext,
threadMetadata: knowledgeSelectionReady
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
? {
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
}
: undefined,
isMock,
onStart: (createdThreadId) => {
@@ -434,7 +459,10 @@ export default function ChatPage() {
metadata: {
agent_id: selectedAgentId,
...(knowledgeSelectionReady
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
? {
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
}
: {}),
},
});
@@ -445,7 +473,10 @@ export default function ChatPage() {
threadId: created.thread_id,
text,
...(knowledgeSelectionReady
? { knowledgeBaseIds: selectedKnowledgeBaseIds }
? {
knowledgeBaseIds: selectedKnowledgeBaseIds,
assistantKnowledgeBaseIds: selectedAssistantKnowledgeBaseIds,
}
: {}),
}),
);
@@ -505,6 +536,7 @@ export default function ChatPage() {
hideAdminPrompt,
preferencesLoading,
selectedAgentId,
selectedAssistantKnowledgeBaseIds,
selectedKnowledgeBaseIds,
knowledgeSelectionReady,
agentHandoffPending,
@@ -608,6 +640,7 @@ export default function ChatPage() {
const handleAgentChange = useCallback((agentId: string | null) => {
setSelectedAgentId(agentId);
setSelectedKnowledgeBaseIds([]);
setSelectedAssistantKnowledgeBaseIds([]);
setKnowledgeSelectionReady(false);
saveLastAgentId(agentId);
}, []);
@@ -842,7 +875,9 @@ export default function ChatPage() {
source={selectedAgentId ? "agent" : "all"}
agentId={selectedAgentId}
value={selectedKnowledgeBaseIds}
assistantValue={selectedAssistantKnowledgeBaseIds}
onChange={changeKnowledgeSelection}
onAssistantChange={changeAssistantKnowledgeSelection}
autoSelectAll={isLandingState && !!selectedAgentId}
disabled={
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" ||
@@ -3,7 +3,11 @@ import { useState } from "react";
import { Navigate, useParams, useSearchParams } from "react-router-dom";
import { Button } from "@/components/ui/button";
import { KnowledgeProductNav, useKnowledgeBaseProductVersion } from "@/components/workspace/knowledge-product-nav";
import {
KNOWLEDGE_NORMAL_VIEW_VALUE,
KnowledgeProductNav,
useKnowledgeBaseProductVersion,
} from "@/components/workspace/knowledge-product-nav";
import {
Dialog,
DialogContent,
@@ -73,6 +77,7 @@ export default function LlmWikiKnowledgePage({
const runtime = useLlmWikiRuntime();
const productVersion = useKnowledgeBaseProductVersion();
const wikiSlug = searchParams.get("wikiSlug")?.trim() || "";
const explicitNormalView = searchParams.get("view") === KNOWLEDGE_NORMAL_VIEW_VALUE;
if (mode === "detail" && noteId && wikiSlug) {
return <Navigate to={buildWeKnoraWikiPagePath(noteId, wikiSlug)} replace />;
@@ -82,15 +87,18 @@ export default function LlmWikiKnowledgePage({
return <RuntimeLoading />;
}
if (mode === "list" && productVersion.data === "v2") {
if (mode === "list" && productVersion.data === "v2" && !explicitNormalView) {
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
}
if (runtime.isError || runtime.data?.provider !== "weknora") {
if (mode === "list" || mode === "detail") {
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
}
if (mode === "graph") {
return <KnowledgeGraphPage />;
}
return mode === "list" ? <div className="flex size-full min-h-0 flex-col"><KnowledgeProductNav /><div className="min-h-0 flex-1"><LegacyKnowledgePage /></div></div> : <LegacyKnowledgePage />;
return <LegacyKnowledgePage />;
}
if (mode === "graph") {
+7 -1
View File
@@ -450,7 +450,13 @@ export default function WorkspaceRoutes({
<Route
path="library"
element={
<WorkspaceLayout {...layoutProps} workspaceSidebarHidden>
<WorkspaceLayout
{...layoutProps}
workspaceSidebarHidden={
layoutProps.workspaceSidebarHidden ||
!layoutProps.pageSidebarHidden
}
>
<ArtifactLibraryPage />
</WorkspaceLayout>
}
@@ -8,13 +8,21 @@ const BASE = () => `${getBackendBaseURL()}/api/assistant-knowledge`;
export interface AssistantKnowledgeBase {
id: string;
name: string;
description?: string;
base_kind?: "global" | "custom";
is_global?: boolean;
status: string;
source_count?: number;
page_count?: number;
entity_count?: number;
relation_count?: number;
created_at: string;
updated_at: string;
}
export interface AssistantWikiPage {
id: string;
base_id: string;
slug: string;
title: string;
page_type: string;
@@ -37,6 +45,7 @@ export interface AssistantWikiRevision {
export interface AssistantKnowledgeSource {
id: string;
base_id: string;
source_type: "weknora" | "skill" | "local_file";
source_name: string;
source_key: string;
@@ -46,12 +55,34 @@ export interface AssistantKnowledgeSource {
export interface AssistantImportJob {
id: string;
base_id: string;
source_id: string;
mirrored_global_base_id?: string;
mirrored_global_job_id?: string;
package_download_url?: string;
trigger: string;
status: string;
phase: string;
counts: Record<string, number>;
last_error_message?: string | null;
created_at: string;
completed_at?: string | null;
}
export interface LlmWikiExportPackageJob {
id: string;
mapping_id: string;
status: string;
phase: string;
counts: Record<string, number>;
package_download_url?: string;
status_url?: string;
error?: string | null;
created_at?: string | null;
updated_at?: string | null;
completed_at?: string | null;
file_exists?: boolean;
file_size_bytes?: number;
}
async function readJson<T>(response: Response): Promise<T> {
@@ -65,12 +96,41 @@ async function readJson<T>(response: Response): Promise<T> {
return (await response.json()) as T;
}
function normalizeImportJob(job: AssistantImportJob): AssistantImportJob {
const url = normalizeBackendUrl(job.package_download_url);
return url ? { ...job, package_download_url: url } : job;
}
function normalizeBackendUrl(url: string | undefined | null): string | undefined {
if (!url || /^https?:\/\//i.test(url)) return url || undefined;
return `${getBackendBaseURL()}${url.startsWith("/") ? url : `/${url}`}`;
}
function normalizeExportPackageJob(job: LlmWikiExportPackageJob): LlmWikiExportPackageJob {
return {
...job,
package_download_url: normalizeBackendUrl(job.package_download_url),
status_url: normalizeBackendUrl(job.status_url),
};
}
export async function getAssistantKnowledgeBase() {
const data = await readJson<{ base: AssistantKnowledgeBase | null }>(await apiFetch(`${BASE()}/base`));
return data.base;
}
export async function initializeAssistantKnowledgeBase(name = "知识梳理") {
export async function getAssistantKnowledgeBaseById(baseId: string) {
const data = await readJson<{ base: AssistantKnowledgeBase }>(
await apiFetch(`${BASE()}/bases/${encodeURIComponent(baseId)}`),
);
return data.base;
}
export async function listAssistantKnowledgeBases() {
return (await readJson<{ bases: AssistantKnowledgeBase[] }>(await apiFetch(`${BASE()}/bases`))).bases;
}
export async function initializeAssistantKnowledgeBase(name = "知识梳理总库") {
const data = await readJson<{ base: AssistantKnowledgeBase }>(
await apiFetch(`${BASE()}/base/initialize`, {
method: "POST",
@@ -81,58 +141,124 @@ export async function initializeAssistantKnowledgeBase(name = "知识梳理") {
return data.base;
}
export async function listAssistantWikiPages(q = "") {
export async function createAssistantKnowledgeBase(input: { name: string; description?: string }) {
const data = await readJson<{ base: AssistantKnowledgeBase }>(
await apiFetch(`${BASE()}/bases`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(input),
}),
);
return data.base;
}
function withBaseParam(baseId?: string | null) {
const params = new URLSearchParams();
if (baseId) params.set("base_id", baseId);
return params;
}
export async function listAssistantWikiPages(q = "", baseId?: string | null) {
const params = withBaseParam(baseId);
if (q.trim()) params.set("q", q.trim());
return (await readJson<{ pages: AssistantWikiPage[] }>(await apiFetch(`${BASE()}/wiki-pages?${params}`))).pages;
}
export async function listAssistantKnowledgeSources() {
return (await readJson<{ sources: AssistantKnowledgeSource[] }>(await apiFetch(`${BASE()}/sources`))).sources;
export async function listAssistantKnowledgeSources(baseId?: string | null) {
const params = withBaseParam(baseId);
return (await readJson<{ sources: AssistantKnowledgeSource[] }>(await apiFetch(`${BASE()}/sources?${params}`))).sources;
}
export async function listAssistantImportJobs() {
return (await readJson<{ jobs: AssistantImportJob[] }>(await apiFetch(`${BASE()}/import-jobs`))).jobs;
export async function listAssistantImportJobs(baseId?: string | null) {
const params = withBaseParam(baseId);
return (
await readJson<{ jobs: AssistantImportJob[] }>(await apiFetch(`${BASE()}/import-jobs?${params}`))
).jobs.map(normalizeImportJob);
}
export async function listAssistantWikiRevisions(slug: string) {
export async function listAssistantWikiRevisions(slug: string, baseId?: string | null) {
const params = withBaseParam(baseId);
return (await readJson<{ revisions: AssistantWikiRevision[] }>(
await apiFetch(`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions`),
await apiFetch(`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions?${params}`),
)).revisions;
}
export async function rollbackAssistantWikiRevision(slug: string, revisionId: string) {
export async function rollbackAssistantWikiRevision(slug: string, revisionId: string, baseId?: string | null) {
const params = withBaseParam(baseId);
return readJson<AssistantWikiRevision>(
await apiFetch(
`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions/${encodeURIComponent(revisionId)}/rollback`,
`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions/${encodeURIComponent(revisionId)}/rollback?${params}`,
{ method: "POST" },
),
);
}
export async function uploadAssistantKnowledgeFiles(files: File[]) {
const form = new FormData();
files.forEach((file) => form.append("files", file));
return readJson<{ batch_id: string; jobs: AssistantImportJob[] }>(
await apiFetch(`${BASE()}/imports/files`, { method: "POST", body: form }),
export async function exportLlmWikiToAssistant(mappingId: string, assistantBaseId?: string | null) {
return normalizeImportJob(await readJson<AssistantImportJob>(
await apiFetch(
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-to-assistant`,
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ assistant_base_id: assistantBaseId || null }),
},
),
));
}
export async function createLlmWikiExportPackage(mappingId: string) {
return normalizeExportPackageJob(
await readJson<LlmWikiExportPackageJob>(
await apiFetch(
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package`,
{ method: "POST" },
),
),
);
}
export async function exportLlmWikiToAssistant(mappingId: string) {
return readJson<AssistantImportJob>(
await apiFetch(
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-to-assistant`,
{ method: "POST" },
export async function listLlmWikiExportPackages(mappingId: string) {
return (
await readJson<{ jobs: LlmWikiExportPackageJob[] }>(
await apiFetch(
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package`,
),
)
).jobs.map(normalizeExportPackageJob);
}
export async function getLlmWikiExportPackageStatus(mappingId: string, jobId: string) {
return normalizeExportPackageJob(
await readJson<LlmWikiExportPackageJob>(
await apiFetch(
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package/${encodeURIComponent(jobId)}`,
),
),
);
}
export async function reimportAssistantKnowledgeSource(sourceId: string) {
return readJson<AssistantImportJob>(
return normalizeImportJob(await readJson<AssistantImportJob>(
await apiFetch(`${BASE()}/sources/${encodeURIComponent(sourceId)}/reimport`, {
method: "POST",
}),
);
));
}
export function getAssistantImportPackageUrl(jobId: string) {
return `${BASE()}/import-jobs/${encodeURIComponent(jobId)}/package`;
}
export async function uploadAssistantWeKnoraPackage(input: { file: File; assistantBaseId?: string | null }) {
const body = new FormData();
body.set("file", input.file);
if (input.assistantBaseId) body.set("assistant_base_id", input.assistantBaseId);
return normalizeImportJob(await readJson<AssistantImportJob>(
await apiFetch(`${BASE()}/imports/weknora-package`, {
method: "POST",
body,
}),
));
}
export async function deleteAssistantKnowledgeSource(sourceId: string) {
@@ -144,20 +270,40 @@ export async function deleteAssistantKnowledgeSource(sourceId: string) {
const KEY = "assistant-knowledge";
export function useAssistantKnowledgeBase() {
return useQuery({ queryKey: [KEY, "base"], queryFn: getAssistantKnowledgeBase });
export function useAssistantKnowledgeBase(baseId?: string | null) {
return useQuery({
queryKey: [KEY, "base", baseId ?? "global"],
queryFn: () => (baseId ? getAssistantKnowledgeBaseById(baseId) : getAssistantKnowledgeBase()),
});
}
export function useAssistantWikiPages(query = "") {
return useQuery({ queryKey: [KEY, "pages", query], queryFn: () => listAssistantWikiPages(query) });
export function useAssistantKnowledgeBases() {
return useQuery({ queryKey: [KEY, "bases"], queryFn: listAssistantKnowledgeBases });
}
export function useAssistantKnowledgeSources() {
return useQuery({ queryKey: [KEY, "sources"], queryFn: listAssistantKnowledgeSources });
export function useAssistantWikiPages(query = "", baseId?: string | null) {
return useQuery({
queryKey: [KEY, "pages", baseId ?? "global", query],
queryFn: () => listAssistantWikiPages(query, baseId),
});
}
export function useAssistantImportJobs() {
return useQuery({ queryKey: [KEY, "jobs"], queryFn: listAssistantImportJobs });
export function useAssistantKnowledgeSources(baseId?: string | null) {
return useQuery({
queryKey: [KEY, "sources", baseId ?? "global"],
queryFn: () => listAssistantKnowledgeSources(baseId),
});
}
export function useAssistantImportJobs(baseId?: string | null) {
return useQuery({
queryKey: [KEY, "jobs", baseId ?? "global"],
queryFn: () => listAssistantImportJobs(baseId),
refetchInterval: (query) =>
(query.state.data ?? []).some((job) => ["queued", "running"].includes(job.status))
? 3000
: false,
});
}
export function useAssistantKnowledgeActions() {
@@ -165,12 +311,13 @@ export function useAssistantKnowledgeActions() {
const invalidate = () => client.invalidateQueries({ queryKey: [KEY] });
return {
initialize: useMutation({ mutationFn: initializeAssistantKnowledgeBase, onSuccess: invalidate }),
upload: useMutation({ mutationFn: uploadAssistantKnowledgeFiles, onSuccess: invalidate }),
createBase: useMutation({ mutationFn: createAssistantKnowledgeBase, onSuccess: invalidate }),
uploadWeKnoraPackage: useMutation({ mutationFn: uploadAssistantWeKnoraPackage, onSuccess: invalidate }),
reimportSource: useMutation({ mutationFn: reimportAssistantKnowledgeSource, onSuccess: invalidate }),
deleteSource: useMutation({ mutationFn: deleteAssistantKnowledgeSource, onSuccess: invalidate }),
rollback: useMutation({
mutationFn: ({ slug, revisionId }: { slug: string; revisionId: string }) =>
rollbackAssistantWikiRevision(slug, revisionId),
mutationFn: ({ slug, revisionId, baseId }: { slug: string; revisionId: string; baseId?: string | null }) =>
rollbackAssistantWikiRevision(slug, revisionId, baseId),
onSuccess: invalidate,
}),
};
@@ -40,6 +40,9 @@ export interface LlmWikiKnowledgeBase {
share_count?: number;
is_processing?: boolean;
is_temporary?: boolean;
wiki_index_enabled?: boolean;
external_search_enabled?: boolean;
external_search_updated_at?: string | null;
embedding_model_id?: string | null;
vector_store_id?: string | null;
vector_store_name?: string | null;
@@ -1,15 +1,11 @@
import { AlertCircle, Database, Edit3, FileText, Loader2, RefreshCw, Trash2 } from "lucide-react";
import { useEffect, useRef, useState } from "react";
import { AlertCircle, Download, Edit3, FileText, Loader2, RefreshCw, Trash2 } from "lucide-react";
import { useCallback, useEffect, useRef, useState } from "react";
import { Navigate, useNavigate, useParams, useSearchParams } from "react-router-dom";
import { toast } from "sonner";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Dialog, DialogContent, DialogHeader, DialogTitle } from "@/components/ui/dialog";
import { Input } from "@/components/ui/input";
import { Textarea } from "@/components/ui/textarea";
import { MarkdownCodeEditor } from "@/components/wiki/MarkdownCodeEditor";
@@ -24,9 +20,20 @@ import {
useLlmWikiWikiPage,
useLlmWikiRuntime,
} from "@/strategy-components/api/llmwiki";
import { exportLlmWikiToAssistant } from "@/strategy-components/api/assistant-knowledge";
import {
createLlmWikiExportPackage,
listLlmWikiExportPackages,
type LlmWikiExportPackageJob,
} from "@/strategy-components/api/assistant-knowledge";
const LIST_PATH = "/page/workspace/knowledge";
const RUNNING_EXPORT_STATUSES = new Set(["queued", "running"]);
const EXPORT_STATUS_LABELS: Record<string, string> = {
queued: "排队中",
running: "整理中",
completed: "已完成",
failed: "失败",
};
type DeerFlowThemePayload = {
type: "deerflow-weknora-theme";
@@ -94,6 +101,25 @@ function buildFrameUrl(framePath: string) {
return new URL(`${backendBase || backendOrigin}/${framePath.replace(/^\/+/, "")}`);
}
function formatBytes(value: number | undefined | null) {
const bytes = Number(value || 0);
if (!Number.isFinite(bytes) || bytes <= 0) return "—";
const units = ["B", "KB", "MB", "GB", "TB"];
let current = bytes;
let unitIndex = 0;
while (current >= 1024 && unitIndex < units.length - 1) {
current /= 1024;
unitIndex += 1;
}
return `${current >= 10 || unitIndex === 0 ? current.toFixed(0) : current.toFixed(1)} ${units[unitIndex]}`;
}
function formatTime(value: string | undefined | null) {
if (!value) return "—";
const time = new Date(value);
return Number.isNaN(time.getTime()) ? "—" : time.toLocaleString();
}
export function WeKnoraKnowledgeDetailPage() {
const {
knowledge_base_id: strategyKnowledgeBaseId,
@@ -106,16 +132,40 @@ export function WeKnoraKnowledgeDetailPage() {
const [frameSrc, setFrameSrc] = useState("");
const [frameVersion, setFrameVersion] = useState(0);
const [frameError, setFrameError] = useState<string | null>(null);
const [assistantImporting, setAssistantImporting] = useState(false);
const [exportPackageLoading, setExportPackageLoading] = useState(false);
const [exportJobsLoading, setExportJobsLoading] = useState(false);
const [exportJobs, setExportJobs] = useState<LlmWikiExportPackageJob[]>([]);
const isAdmin = getStoredAuth()?.system_role === "admin";
const runtime = useLlmWikiRuntime();
const details = useLlmWikiKnowledgeBase(knowledgeBaseId || null, !!knowledgeBaseId);
const refreshExportJobs = useCallback(
async (silent = false) => {
if (!knowledgeBaseId || !isAdmin) return;
if (!silent) setExportJobsLoading(true);
try {
setExportJobs(await listLlmWikiExportPackages(knowledgeBaseId));
} catch (error) {
if (!silent) toast.error(error instanceof Error ? error.message : String(error));
} finally {
if (!silent) setExportJobsLoading(false);
}
},
[isAdmin, knowledgeBaseId],
);
const detailsReady = !!details.data;
const requestedTab = searchParams.get("tab");
const detailsWikiEnabled =
details.data?.type === "wiki" ||
details.data?.wiki_index_enabled === true ||
details.data?.indexing_strategy?.wiki_enabled === true ||
details.data?.capabilities?.wiki_enabled === true ||
(details.data?.wiki_config && Object.keys(details.data.wiki_config).length > 0);
const preferredTab =
requestedTab === "documents" || requestedTab === "wiki" || requestedTab === "graph"
? requestedTab
: details.data?.indexing_strategy?.wiki_enabled
: detailsWikiEnabled
? "wiki"
: "documents";
const sourceChunkId = searchParams.get("sourceChunkId") || searchParams.get("chunkId") || "";
@@ -182,6 +232,22 @@ export function WeKnoraKnowledgeDetailPage() {
};
}, [frameSrc]);
useEffect(() => {
if (!isAdmin || !knowledgeBaseId || runtime.data?.provider !== "weknora") {
setExportJobs([]);
return;
}
void refreshExportJobs();
}, [isAdmin, knowledgeBaseId, refreshExportJobs, runtime.data?.provider]);
useEffect(() => {
if (!isAdmin || !knowledgeBaseId || !exportJobs.some((job) => RUNNING_EXPORT_STATUSES.has(job.status))) {
return;
}
const timer = window.setInterval(() => void refreshExportJobs(true), 5000);
return () => window.clearInterval(timer);
}, [exportJobs, isAdmin, knowledgeBaseId, refreshExportJobs]);
if (nativeWikiPageUrl) {
return <Navigate to={nativeWikiPageUrl} replace />;
}
@@ -235,8 +301,25 @@ export function WeKnoraKnowledgeDetailPage() {
const canWrite = details.data.can_write && !details.data.is_conversation_deposit;
const canEditCurrentWikiArticle =
canWrite && !!details.data.indexing_strategy?.wiki_enabled && !!wikiSlug;
const canManageWiki = canWrite && !!details.data.indexing_strategy?.wiki_enabled;
canWrite && !!detailsWikiEnabled && !!wikiSlug;
const canManageWiki = canWrite && !!detailsWikiEnabled;
const createVectorizedPackage = () => {
if (exportPackageLoading) return;
setExportPackageLoading(true);
void (async () => {
try {
const job = await createLlmWikiExportPackage(knowledgeBaseId);
setExportJobs((current) => [job, ...current.filter((item) => item.id !== job.id)]);
toast.success("已创建向量化数据包整理记录,完成后可在下方列表下载");
void refreshExportJobs(true);
} catch (error) {
toast.error(error instanceof Error ? error.message : String(error));
} finally {
setExportPackageLoading(false);
}
})();
};
return (
<div className="flex h-full min-h-0 flex-col bg-background">
@@ -258,21 +341,15 @@ export function WeKnoraKnowledgeDetailPage() {
)}
</div>
<div className="flex shrink-0 items-center gap-2">
{getStoredAuth()?.system_role === "admin" && (
{isAdmin && (
<Button
size="sm"
variant="outline"
disabled={assistantImporting}
onClick={() => {
setAssistantImporting(true);
void exportLlmWikiToAssistant(knowledgeBaseId)
.then(() => toast.success("已导入到助手知识库“知识梳理”"))
.catch((error: unknown) => toast.error(error instanceof Error ? error.message : String(error)))
.finally(() => setAssistantImporting(false));
}}
disabled={exportPackageLoading}
onClick={createVectorizedPackage}
>
{assistantImporting ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Database className="mr-2 size-4" />}
导入到助手知识库
{exportPackageLoading ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Download className="mr-2 size-4" />}
生成向量化数据包
</Button>
)}
<Button
@@ -286,6 +363,13 @@ export function WeKnoraKnowledgeDetailPage() {
</div>
</div>
)}
{canManageWiki && isAdmin && (
<ExportPackageJobsPanel
jobs={exportJobs}
loading={exportJobsLoading}
onRefresh={() => void refreshExportJobs()}
/>
)}
{frameSrc && (
<iframe
key={`${frameSrc}:${frameVersion}`}
@@ -313,6 +397,84 @@ export function WeKnoraKnowledgeDetailPage() {
);
}
function ExportPackageJobsPanel({
jobs,
loading,
onRefresh,
}: {
jobs: LlmWikiExportPackageJob[];
loading: boolean;
onRefresh: () => void;
}) {
return (
<div className="shrink-0 border-b bg-muted/20 px-4 py-3">
<div className="mb-2 flex items-center justify-between">
<div>
<div className="text-sm font-medium">向量化数据包记录</div>
<p className="text-xs text-muted-foreground">
每次点击都会生成一条后台整理记录;文件较大时,请等待状态变为“已完成”后再下载。
</p>
</div>
<Button size="sm" variant="ghost" onClick={onRefresh} disabled={loading}>
{loading ? <Loader2 className="mr-2 size-4 animate-spin" /> : <RefreshCw className="mr-2 size-4" />}
刷新
</Button>
</div>
<div className="max-h-44 space-y-2 overflow-y-auto pr-1">
{jobs.map((job) => {
const running = RUNNING_EXPORT_STATUSES.has(job.status);
const completed = job.status === "completed";
const canDownload = completed && !!job.package_download_url && job.file_exists !== false;
const counts = job.counts ?? {};
return (
<div key={job.id} className="flex items-center gap-3 rounded-md border bg-card px-3 py-2 text-xs">
<Badge
variant={job.status === "failed" ? "destructive" : completed ? "default" : "secondary"}
className="shrink-0"
>
{running && <Loader2 className="mr-1 inline size-3 animate-spin" />}
{EXPORT_STATUS_LABELS[job.status] || job.status}
</Badge>
<div className="min-w-0 flex-1">
<div className="flex flex-wrap items-center gap-x-3 gap-y-1">
<span className="font-medium">任务 {job.id.slice(0, 8)}</span>
<span className="text-muted-foreground">创建:{formatTime(job.created_at)}</span>
<span className="text-muted-foreground">大小:{formatBytes(job.file_size_bytes)}</span>
</div>
<div className="mt-1 flex flex-wrap gap-x-3 gap-y-1 text-muted-foreground">
<span>Wiki {counts.wiki_page ?? counts.wiki_pages ?? 0}</span>
<span>文档 {counts.document ?? counts.documents ?? 0}</span>
<span>片段 {counts.chunk ?? counts.chunks ?? 0}</span>
<span>向量 {counts.vector ?? counts.vectors ?? 0}</span>
<span>阶段:{job.phase}</span>
{job.error && <span className="text-destructive">{job.error}</span>}
</div>
</div>
{canDownload ? (
<Button asChild size="sm" variant="outline" className="shrink-0">
<a href={job.package_download_url} target="_blank" rel="noreferrer">
<Download className="mr-2 size-4" />
下载
</a>
</Button>
) : (
<Button size="sm" variant="outline" className="shrink-0" disabled>
{completed && job.file_exists === false ? "文件不存在" : "等待完成"}
</Button>
)}
</div>
);
})}
{!jobs.length && (
<div className="rounded-md border border-dashed bg-card px-3 py-6 text-center text-xs text-muted-foreground">
暂无导出记录。点击上方“生成向量化数据包”后,任务会显示在这里。
</div>
)}
</div>
</div>
);
}
function CurrentWikiArticleActions({
knowledgeBaseId,
slug,
+29 -12
View File
@@ -1623,23 +1623,40 @@ save fallback.
## Skill/assistant knowledge invariants
Skill distillation is implemented in `deerflow.skill_knowledge` and
`deerflow.persistence.skill_knowledge`; the single native assistant base is in
`deerflow.persistence.skill_knowledge`; native assistant bases are in
`deerflow.assistant_knowledge` and
`deerflow.persistence.assistant_knowledge`. Gateway composition, remote adapters,
the lease dispatcher, watcher/scheduler, and APIs remain in `app.gateway` so the
harness-to-app import firewall is preserved.
the lease dispatcher, and APIs remain in `app.gateway` so the harness-to-app
import firewall is preserved.
Never execute a skill or read/upload/summarize its code and script files.
Knowledge fingerprints exclude those files and Markdown code fences. All
resyncs must stage and validate the knowledge digest before activation; the old
snapshot remains authoritative on failure. A binding may have only one active
item (`active_dedupe_key` plus atomic lease claims). Detach/delete operates on
source contributions, not canonical rows, and shared entities/relations are
retired only when their final contribution is gone.
item (`active_dedupe_key` plus atomic lease claims). Skill distillation targets
only writable WeKnora Wiki knowledge bases in this product version; reject direct
assistant-base targets, non-Wiki/FAQ/conversation/readonly targets, and direct
Neo4j writes. Detach/delete operates on source contributions, not canonical rows,
and shared entities/relations are retired only when their final contribution is
gone.
There is exactly one global assistant base (`知识梳理`). Ordinary users have
read/search access and administrators own all mutations. WeKnora direct graph
writes require `llmwiki.weknora.graph.direct_write_enabled`, backend-only
credentials, and the fixed parameterized templates in
`deerflow.skill_knowledge.graph_writer`; never accept Cypher from an API, model,
skill, or frontend.
Assistant knowledge has one global full-library base (`知识梳理总库`) and any
number of admin-created custom bases. Ordinary users have read/search access and
administrators own all mutations. Assistant bases only accept WeKnora export
packages or imports triggered from a WeKnora mapping. Do not add direct skill
imports or arbitrary local-file imports to assistant knowledge. Importing into a
custom assistant base must also mirror the package into the global base for
cross-source entity alignment.
## openGauss / GaussDB persistence invariant
`database.backend=gauss` is an application-ORM backend, parallel to `mysql`.
It normalizes `database.gauss_url` to `opengauss+asyncpg://` and requires the
`gauss` optional dependency (`asyncpg` plus `opengauss-sqlalchemy`). Never route
a Gauss URL through SQLAlchemy's generic PostgreSQL dialect: the wire protocol
may connect, but dialect initialization and schema DDL can fail against the
Gauss server/version behavior. Keep the LangGraph checkpointer on the separately
configured SQLite backend; the PostgreSQL checkpointer schema is not part of the
Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
names as timezone-aware PostgreSQL-family engines in portable types and
dialect-specific session settings.
+2 -2
View File
@@ -13,8 +13,8 @@ FROM python:3.12-slim-bookworm AS builder
ARG NODE_MAJOR=22
ARG APT_MIRROR
ARG UV_INDEX_URL
# Optional extras to install (e.g. "postgres" for PostgreSQL support)
# Usage: docker build --build-arg UV_EXTRAS=postgres ...
# Optional extras to install (e.g. "postgres" or "gauss")
# Usage: docker build --build-arg UV_EXTRAS=gauss ...
ARG UV_EXTRAS
# Optionally override apt mirror for restricted networks (e.g. APT_MIRROR=mirrors.aliyun.com)
+49 -21
View File
@@ -957,26 +957,54 @@ sandbox-directory recreation cannot discard a completed rewrite.
## Skill knowledge distillation and assistant knowledge
The Gateway now exposes an administrator-only skill distillation pipeline at
`/api/skill-knowledge`. A matrix job can send up to 200 skills to multiple
WeKnora or global assistant-knowledge targets. The scanner never reads or
uploads code/script files, strips Markdown code fences, blocks credential-like
files and content, safely inspects ZIP members, and converts supported
Office/PDF/spreadsheet files off the event loop. Jobs use database claims and
renewable leases, stage snapshots before activation, retain the prior active
version on failure, support per-item retry and entity/relation review, and keep
per-binding contribution records for safe detach/delete.
`/api/skill-knowledge`. Each request can send up to 200 skills to one writable
WeKnora Wiki knowledge base; multi-base distillation is done by repeated
operations so the local binding remains `skill_name + target_id`. The scanner
never reads or uploads code/script files, strips Markdown code fences, blocks
credential-like files and content, safely inspects ZIP members, and converts
supported Office/PDF/spreadsheet files off the event loop. Jobs use database
claims and renewable leases, stage snapshots before activation, retain the
prior active version on failure, support per-item retry, and keep per-binding
contribution records for safe detach/delete. Direct assistant-base targets,
non-Wiki/FAQ/conversation/readonly targets, and direct Neo4j writes are rejected
for this product version.
Bindings may opt into debounced file watching and cron synchronization through
`/api/skill-knowledge/auto-sync-settings`. Only non-code knowledge digests can
mark a binding stale. WeKnora Wiki, document, FAQ, conversation-archive and
read-only sidecar targets have separate projections. Optional Neo4j direct
writes are disabled by default under `llmwiki.weknora.graph`; when enabled they
use only fixed parameterized templates and Neo4j's transactional HTTP endpoint.
`/api/assistant-knowledge` owns exactly one global DeerFlow-native base named
`知识梳理`. Reads and search are available to authenticated users; initialization,
local-file import, WeKnora export/import, source reimport/delete, and revision
rollback are admin-only. Imports align canonical entities and relations while
retaining source contributions, publish source/entity/relation Wiki revisions,
and remove canonical knowledge only after its final contribution disappears.
`/api/assistant-knowledge` manages DeerFlow-native assistant knowledge bases.
There is one global full-library base named `知识梳理总库`, plus admin-created
custom assistant bases. Reads and search are available to authenticated users;
global initialization, custom-base creation, WeKnora export/import, source
reimport, and source deletion are admin-only. Assistant bases only accept
WeKnora export packages or imports triggered from a WeKnora mapping; direct
skill import and arbitrary local-file import return 410. Importing into a custom
assistant base also mirrors the same package into the global base, where
canonical entities and relations are aligned across all imported sources.
The database schema is introduced by Alembic revision `20260901_01`.
## openGauss / GaussDB application database
The application ORM can use a PG-compatible openGauss/GaussDB database through
the official `opengauss-sqlalchemy` async dialect. Install the optional driver
set before enabling it:
```bash
uv sync --extra gauss
```
Configure the business database independently from the LangGraph checkpointer:
```yaml
database:
backend: gauss
gauss_url: $GAUSS_DATABASE_URL
checkpointer:
type: sqlite
connection_string: .deer-flow/data/deerflow.db
```
`GAUSS_DATABASE_URL` should normally be
`opengauss+asyncpg://user:password@host:26000/deerflow`. Plain
`opengauss://`, `gaussdb://`, and PostgreSQL-style URLs are accepted and
normalized to the openGauss dialect. Do not configure the LangGraph
PostgreSQL checkpointer against GaussDB: its migrations are PostgreSQL-specific.
For a production image, build with `--build-arg UV_EXTRAS=gauss`.
@@ -362,13 +362,21 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
resolved_llmwiki = get_resolved_llmwiki_runtime(config)
weknora_client = build_weknora_client(resolved_llmwiki) if resolved_llmwiki.weknora_enabled else None
if local_index_config.enabled:
embedding_config = local_index_config.embedding
embedding_configured = bool(
embedding_config.enabled
and embedding_config.base_url
and embedding_config.model.strip()
)
if embedding_configured:
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(embedding_config)
if local_index_config.enabled and app.state.llmwiki_embedding is not None:
from deerflow.integrations.weknora.local_index.search import WikiVectorSearchService
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(local_index_config.embedding)
app.state.llmwiki_vector_cache = WikiVectorCache(
app.state.llmwiki_index_store,
max_knowledge_bases=local_index_config.cache_max_knowledge_bases,
@@ -40,6 +40,7 @@ _MAX_SOURCE_CHARS = 1400
@dataclass(frozen=True)
class LlmWikiRagContext:
knowledge_base_ids: list[str]
assistant_knowledge_base_ids: list[str]
source_count: int
prompt: str
sources: list[dict[str, Any]]
@@ -50,6 +51,10 @@ def _store(request: Request) -> LlmWikiStore | None:
return getattr(request.app.state, "llmwiki_store", None)
def _assistant_store(request: Request) -> Any | None:
return getattr(request.app.state, "assistant_knowledge_store", None)
async def _actor(request: Request) -> tuple[str, bool]:
user = await get_optional_user_from_request(request)
if user is None:
@@ -76,6 +81,13 @@ def _body_selected_ids(body: Any) -> list[str]:
return []
def _body_selected_assistant_ids(body: Any) -> list[str]:
for source in (getattr(body, "context", None), getattr(body, "metadata", None)):
if isinstance(source, Mapping) and "assistant_knowledge_base_ids" in source:
return _as_id_list(source.get("assistant_knowledge_base_ids"))
return []
def _assistant_agent_id(body: Any) -> str | None:
assistant_id = validate_agent_id(getattr(body, "assistant_id", None))
if assistant_id:
@@ -106,8 +118,17 @@ def resolve_requested_llmwiki_knowledge_base_ids(body: Any) -> list[str]:
return _body_selected_ids(body)
def resolve_requested_assistant_knowledge_base_ids(body: Any) -> list[str]:
"""Return assistant knowledge bases explicitly selected for the run."""
return _body_selected_assistant_ids(body)
def llmwiki_selection_requested(body: Any) -> bool:
return bool(resolve_requested_llmwiki_knowledge_base_ids(body))
return bool(
resolve_requested_llmwiki_knowledge_base_ids(body)
or resolve_requested_assistant_knowledge_base_ids(body)
)
def _latest_human_query(graph_input: dict[str, Any]) -> str:
@@ -191,18 +212,50 @@ def _source_from_result(result: dict[str, Any], mapping: dict[str, Any], index:
}
def _source_from_assistant_result(result: dict[str, Any], index: int) -> dict[str, Any]:
title = str(result.get("title") or f"助手知识库片段 {index}")
matched_section = result.get("matched_section") if isinstance(result.get("matched_section"), Mapping) else {}
content = str(
matched_section.get("content")
or result.get("content_markdown")
or result.get("content")
or result.get("summary")
or ""
)
snippet = str(result.get("summary") or content)
raw = {
**result,
"source": "助手知识库",
"url": "/page/workspace/knowledge/assistant",
}
return {
"index": index,
"id": result.get("id") or result.get("slug") or f"assistant:{index}",
"title": title,
"type": "assistant_knowledge",
"snippet": _truncate(snippet, 360),
"content": content,
"source": "助手知识库",
"score": float(result.get("score") or 0),
"url": "/page/workspace/knowledge/assistant",
"skillName": "assistant_knowledge_search",
"mode": "citation",
"raw": raw,
}
def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
if not sources:
return "当前用户选择了 LLMWiki 知识库,但 DeerFlow 未检索到可用片段。请直接说明没有检索到可用背景资料,不要编造知识库内容或引用编号。"
return "当前用户选择了知识库,但 DeerFlow 未检索到可用片段。请直接说明没有检索到可用背景资料,不要编造知识库内容或引用编号。"
lines = [
"当前用户选择了 LLMWiki 知识库。DeerFlow 已在模型回答前自动检索到以下背景资料。",
"当前用户选择了知识库。DeerFlow 已在模型回答前自动检索到以下背景资料。",
"请优先依据这些资料回答;如果资料不足,请明确说明不足之处,不要编造。",
"引用规则:正文关键结论必须使用 [1]、[2] 这样的编号引用下方资料;只能引用存在的编号;不要在正文末尾重复输出完整参考文献列表,前端会展示引用卡片。",
"",
f"用户问题:{query}",
"",
f'<llmwiki_wiki_reference_batch source_count="{len(sources)}">',
f'<knowledge_reference_batch source_count="{len(sources)}">',
]
for source in sources:
meta = " / ".join(
@@ -213,12 +266,12 @@ def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
)
if item
)
lines.append(f"[{source['index']}] {source.get('title') or 'LLMWiki 片段'}")
lines.append(f"[{source['index']}] {source.get('title') or '知识库片段'}")
if meta:
lines.append(f"来源:{meta}")
lines.append(_truncate(str(source.get("content") or source.get("snippet") or ""), _MAX_SOURCE_CHARS))
lines.append("")
lines.append("</llmwiki_wiki_reference_batch>")
lines.append("</knowledge_reference_batch>")
return "\n".join(lines).strip()
@@ -229,52 +282,99 @@ async def build_llmwiki_rag_context(
graph_input: dict[str, Any],
) -> LlmWikiRagContext:
selected_ids = resolve_requested_llmwiki_knowledge_base_ids(body)
if not selected_ids:
return LlmWikiRagContext([], 0, "", [], "")
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None))
local_config = request.app.state.config.llmwiki.local_wiki_index
if not (local_config.enabled and local_config.internal_search_enabled) and not runtime.weknora_enabled:
return LlmWikiRagContext(selected_ids, 0, "", [], "")
store = _store(request)
if store is None:
return LlmWikiRagContext(selected_ids, 0, "", [], "")
assistant_selected_ids = resolve_requested_assistant_knowledge_base_ids(body)
if not selected_ids and not assistant_selected_ids:
return LlmWikiRagContext([], [], 0, "", [], "")
query = _latest_human_query(graph_input)
if not query:
return LlmWikiRagContext(selected_ids, 0, "", [], "")
return LlmWikiRagContext(selected_ids, assistant_selected_ids, 0, "", [], "")
user_id, is_admin = await _actor(request)
rows: list[dict[str, Any]] = []
for mapping_id in selected_ids:
row = await store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
if row is None:
raise HTTPException(status_code=404, detail=f"Knowledge base {mapping_id} not found")
rows.append(row)
search_service = getattr(request.app.state, "llmwiki_retrieval_service", None)
if search_service is None:
return LlmWikiRagContext(
selected_ids,
0,
"Wiki 检索暂不可用。请明确说明系统检索不可用,不要表述为资料不存在。",
[],
query,
)
try:
payload = await search_service.search(query, rows, top_k_pages=_MAX_SOURCES)
except WikiIndexError as exc:
logger.warning("Wiki-only pre-retrieval failed user=%s kb_count=%s code=%s", sanitize_log_param(user_id), len(rows), exc.code)
return LlmWikiRagContext(selected_ids, 0, f"Wiki 检索暂不可用({exc.code})。请明确说明系统检索状态,不要声称知识库没有资料。", [], query)
mapping_by_id = {str(row["id"]): row for row in rows}
sources: list[dict[str, Any]] = []
for result in payload.get("results") or []:
mapping = mapping_by_id.get(str(result.get("knowledge_base_id") or ""))
if mapping is None:
continue
sources.append(_source_from_result(result, mapping, len(sources) + 1))
if len(sources) >= _MAX_SOURCES:
break
return LlmWikiRagContext(selected_ids, len(sources), _build_prompt(query, sources), sources, query)
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None))
local_config = request.app.state.config.llmwiki.local_wiki_index
user_id, is_admin = await _actor(request)
if selected_ids:
llmwiki_available = (local_config.enabled and local_config.internal_search_enabled) or runtime.weknora_enabled
store = _store(request)
if not llmwiki_available or store is None:
if not assistant_selected_ids:
return LlmWikiRagContext(selected_ids, assistant_selected_ids, 0, "", [], query)
else:
rows: list[dict[str, Any]] = []
for mapping_id in selected_ids:
row = await store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
if row is None:
raise HTTPException(status_code=404, detail=f"Knowledge base {mapping_id} not found")
rows.append(row)
search_service = getattr(request.app.state, "llmwiki_retrieval_service", None)
if search_service is None:
if not assistant_selected_ids:
return LlmWikiRagContext(
selected_ids,
assistant_selected_ids,
0,
"Wiki 检索暂不可用。请明确说明系统检索不可用,不要表述为资料不存在。",
[],
query,
)
try:
payload = (
await search_service.search(query, rows, top_k_pages=_MAX_SOURCES)
if search_service is not None
else {"results": []}
)
except WikiIndexError as exc:
logger.warning(
"Wiki-only pre-retrieval failed user=%s kb_count=%s code=%s",
sanitize_log_param(user_id),
len(rows),
exc.code,
)
if not assistant_selected_ids:
return LlmWikiRagContext(
selected_ids,
assistant_selected_ids,
0,
f"Wiki 检索暂不可用({exc.code})。请明确说明系统检索状态,不要声称知识库没有资料。",
[],
query,
)
payload = {"results": []}
mapping_by_id = {str(row["id"]): row for row in rows}
for result in payload.get("results") or []:
mapping = mapping_by_id.get(str(result.get("knowledge_base_id") or ""))
if mapping is None:
continue
sources.append(_source_from_result(result, mapping, len(sources) + 1))
if len(sources) >= _MAX_SOURCES:
break
if assistant_selected_ids and len(sources) < _MAX_SOURCES:
assistant_store = _assistant_store(request)
if assistant_store is not None:
rows = await assistant_store.search_pages(
base_ids=assistant_selected_ids,
query=query,
limit=_MAX_SOURCES - len(sources),
include_chunks=True,
embedding_client=getattr(request.app.state, "llmwiki_embedding", None),
)
for row in rows:
sources.append(_source_from_assistant_result(row, len(sources) + 1))
if len(sources) >= _MAX_SOURCES:
break
return LlmWikiRagContext(
selected_ids,
assistant_selected_ids,
len(sources),
_build_prompt(query, sources),
sources,
query,
)
def attach_llmwiki_rag_to_graph_input(
@@ -294,6 +394,7 @@ def attach_llmwiki_rag_to_graph_input(
existing = str(extra.get("prompt_prefix") or "").strip()
extra["prompt_prefix"] = f"{existing}\n\n{rag.prompt}".strip() if existing else rag.prompt
extra["llmwiki_knowledge_base_ids"] = rag.knowledge_base_ids
extra["assistant_knowledge_base_ids"] = rag.assistant_knowledge_base_ids
extra["llmwiki_reference_query"] = rag.query
if rag.sources:
extra["llmwiki_reference_sources"] = rag.sources
@@ -461,7 +461,7 @@ async def _set_analytics_read_timeout(session) -> None:
- MariaDB: ``max_statement_time`` (seconds)
- MySQL <= 5.6: no per-statement timeout variable — skip it entirely
(5.6 raises "Unknown system variable" otherwise)
- PostgreSQL: ``statement_timeout`` (milliseconds)
- PostgreSQL/openGauss: ``statement_timeout`` (milliseconds)
"""
conn = await session.connection()
dialect = conn.dialect
@@ -474,7 +474,7 @@ async def _set_analytics_read_timeout(session) -> None:
await _try_set_session_value(session, f"SET SESSION max_execution_time = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
# else: MySQL <= 5.6 has no statement-timeout variable — skip silently.
await _try_set_session_value(session, f"SET SESSION innodb_lock_wait_timeout = {_LEADERBOARD_LOCK_WAIT_SECONDS}")
elif name == "postgresql":
elif name in {"postgresql", "opengauss"}:
await _try_set_session_value(session, f"SET LOCAL statement_timeout = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
@@ -1,49 +1,50 @@
import asyncio
import logging
import mimetypes
import logging
import mimetypes
import time
import zipfile
import zipfile
from collections import OrderedDict
from datetime import UTC, datetime
from pathlib import Path
from pathlib import Path
from typing import Literal
from urllib.parse import quote
from urllib.parse import quote
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import FileResponse, PlainTextResponse, Response
from pydantic import BaseModel
from app.gateway.authz import require_permission
from app.gateway.path_utils import aresolve_thread_virtual_path
from deerflow.config.paths import get_paths
from deerflow.runtime.thread_paths import aresolve_path_user_id
from fastapi.responses import FileResponse, PlainTextResponse, Response
from pydantic import BaseModel
from app.gateway.authz import require_permission
from app.gateway.path_utils import aresolve_thread_virtual_path
from deerflow.config.paths import get_paths
from deerflow.runtime.thread_paths import aresolve_path_user_id
from deerflow.runtime.user_context import get_effective_user_id
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["artifacts"])
ACTIVE_CONTENT_MIME_TYPES = {
"text/html",
"application/xhtml+xml",
"image/svg+xml",
}
EDITABLE_TEXT_SUFFIXES = {
".md",
".markdown",
".txt",
".json",
".yaml",
".yml",
".csv",
".xml",
".log",
}
_VIRTUAL_USER_DATA_PREFIX = "mnt/user-data"
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["artifacts"])
ACTIVE_CONTENT_MIME_TYPES = {
"text/html",
"application/xhtml+xml",
"image/svg+xml",
}
EDITABLE_TEXT_SUFFIXES = {
".md",
".markdown",
".txt",
".json",
".yaml",
".yml",
".csv",
".xml",
".log",
}
_VIRTUAL_USER_DATA_PREFIX = "mnt/user-data"
ArtifactLibraryKind = Literal["all", "document", "spreadsheet", "text_data", "archive", "other"]
ArtifactLibrarySource = Literal["all", "general_qa", "agent"]
_MEDIA_MIME_PREFIXES = ("image/", "audio/", "video/")
_MEDIA_SUFFIXES = {
@@ -121,35 +122,37 @@ _ARTIFACT_LIBRARY_SCAN_CONCURRENCY = 8
_ARTIFACT_LIBRARY_THREAD_BATCH_SIZE = 500
_artifact_library_cache: OrderedDict[str, tuple[float, list[dict]]] = OrderedDict()
def coerce_artifact_virtual_path(path: str) -> str:
"""Normalize an artifacts URL path to a sandbox virtual path.
Streaming write previews may request a bare filename such as
``未命名文件.md`` before the model has emitted a real
``/mnt/user-data/outputs/...`` path. Mapping a single path segment into
the thread outputs directory turns that into a normal 404 (file not written
yet) instead of HTTP 400 ``Path must start with /mnt/user-data``.
"""
stripped = path.strip().lstrip("/").replace("\\", "/")
if not stripped:
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
if stripped == _VIRTUAL_USER_DATA_PREFIX or stripped.startswith(_VIRTUAL_USER_DATA_PREFIX + "/"):
return stripped
if "/" in stripped or stripped in {".", ".."}:
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
return f"{_VIRTUAL_USER_DATA_PREFIX}/outputs/{stripped}"
class ArtifactUpdateRequest(BaseModel):
content: str
def coerce_artifact_virtual_path(path: str) -> str:
"""Normalize an artifacts URL path to a sandbox virtual path.
Streaming write previews may request a bare filename such as
``未命名文件.md`` before the model has emitted a real
``/mnt/user-data/outputs/...`` path. Mapping a single path segment into
the thread outputs directory turns that into a normal 404 (file not written
yet) instead of HTTP 400 ``Path must start with /mnt/user-data``.
"""
stripped = path.strip().lstrip("/").replace("\\", "/")
if not stripped:
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
if stripped == _VIRTUAL_USER_DATA_PREFIX or stripped.startswith(_VIRTUAL_USER_DATA_PREFIX + "/"):
return stripped
if "/" in stripped or stripped in {".", ".."}:
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
return f"{_VIRTUAL_USER_DATA_PREFIX}/outputs/{stripped}"
class ArtifactUpdateRequest(BaseModel):
content: str
class ArtifactLibraryItem(BaseModel):
thread_id: str
thread_title: str
agent_id: str | None = None
agent_name: str | None = None
thread_type: str | None = None
source_type: Literal["general_qa", "agent"]
path: str
name: str
size_bytes: int
@@ -163,85 +166,85 @@ class ArtifactLibraryResponse(BaseModel):
offset: int
limit: int
items: list[ArtifactLibraryItem]
def _build_content_disposition(disposition_type: str, filename: str) -> str:
"""Build an RFC 5987 encoded Content-Disposition header value."""
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
if extra_headers:
headers.update(extra_headers)
return headers
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
"""Check if file is text by examining content for null bytes."""
try:
with open(path, "rb") as f:
chunk = f.read(sample_size)
# Text files shouldn't contain null bytes
return b"\x00" not in chunk
except Exception:
return False
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
"""Extract a file from a .skill ZIP archive.
Args:
zip_path: Path to the .skill file (ZIP archive).
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
Returns:
The file content as bytes, or None if not found.
"""
if not zipfile.is_zipfile(zip_path):
return None
try:
with zipfile.ZipFile(zip_path, "r") as zip_ref:
# List all files in the archive
namelist = zip_ref.namelist()
# Try direct path first
if internal_path in namelist:
return zip_ref.read(internal_path)
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
for name in namelist:
if name.endswith("/" + internal_path) or name == internal_path:
return zip_ref.read(name)
# Not found
return None
except (zipfile.BadZipFile, KeyError):
return None
def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
"""Walk an outputs directory and return artifact listing entries.
Each entry has ``path`` (virtual ``mnt/user-data/outputs/...``), ``name``,
``size_bytes``, ``mime_type``, and ``modified_at`` (ISO 8601 UTC). Symlinks
that escape the outputs root are dropped to keep the listing scoped to the
thread. The result is sorted by ``modified_at`` descending (newest first).
"""
if not outputs_dir.exists() or not outputs_dir.is_dir():
return []
outputs_root = outputs_dir.resolve()
files: list[dict] = []
for entry in outputs_root.rglob("*"):
if not entry.is_file():
continue
try:
resolved = entry.resolve()
resolved.relative_to(outputs_root)
except (OSError, ValueError):
continue
def _build_content_disposition(disposition_type: str, filename: str) -> str:
"""Build an RFC 5987 encoded Content-Disposition header value."""
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
if extra_headers:
headers.update(extra_headers)
return headers
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
"""Check if file is text by examining content for null bytes."""
try:
with open(path, "rb") as f:
chunk = f.read(sample_size)
# Text files shouldn't contain null bytes
return b"\x00" not in chunk
except Exception:
return False
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
"""Extract a file from a .skill ZIP archive.
Args:
zip_path: Path to the .skill file (ZIP archive).
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
Returns:
The file content as bytes, or None if not found.
"""
if not zipfile.is_zipfile(zip_path):
return None
try:
with zipfile.ZipFile(zip_path, "r") as zip_ref:
# List all files in the archive
namelist = zip_ref.namelist()
# Try direct path first
if internal_path in namelist:
return zip_ref.read(internal_path)
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
for name in namelist:
if name.endswith("/" + internal_path) or name == internal_path:
return zip_ref.read(name)
# Not found
return None
except (zipfile.BadZipFile, KeyError):
return None
def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
"""Walk an outputs directory and return artifact listing entries.
Each entry has ``path`` (virtual ``mnt/user-data/outputs/...``), ``name``,
``size_bytes``, ``mime_type``, and ``modified_at`` (ISO 8601 UTC). Symlinks
that escape the outputs root are dropped to keep the listing scoped to the
thread. The result is sorted by ``modified_at`` descending (newest first).
"""
if not outputs_dir.exists() or not outputs_dir.is_dir():
return []
outputs_root = outputs_dir.resolve()
files: list[dict] = []
for entry in outputs_root.rglob("*"):
if not entry.is_file():
continue
try:
resolved = entry.resolve()
resolved.relative_to(outputs_root)
except (OSError, ValueError):
continue
try:
relative = resolved.relative_to(outputs_root).as_posix()
mime_type, _ = mimetypes.guess_type(resolved.name)
@@ -251,18 +254,18 @@ def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
# is being scanned. Treat that transient entry as absent instead of
# failing the complete thread/library listing.
continue
files.append(
{
"path": f"mnt/user-data/outputs/{relative}",
"name": resolved.name,
"size_bytes": stat.st_size,
"mime_type": mime_type,
files.append(
{
"path": f"mnt/user-data/outputs/{relative}",
"name": resolved.name,
"size_bytes": stat.st_size,
"mime_type": mime_type,
"modified_at": datetime.fromtimestamp(stat.st_mtime, tz=UTC).isoformat(),
}
)
files.sort(key=lambda f: f["modified_at"], reverse=True)
return files
}
)
files.sort(key=lambda f: f["modified_at"], reverse=True)
return files
def artifact_library_kind(name: str) -> str:
@@ -335,7 +338,10 @@ async def _scan_artifact_library(thread_store) -> list[dict]:
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
agent_id = metadata.get("agent_id")
agent_name = metadata.get("agent_name")
thread_type = metadata.get("thread_type")
normalized_agent_id = agent_id.strip() if isinstance(agent_id, str) else ""
normalized_agent_name = agent_name.strip() if isinstance(agent_name, str) else ""
title = str(row.get("display_name") or "新对话")
items: list[dict] = []
for file in files:
@@ -345,8 +351,10 @@ async def _scan_artifact_library(thread_store) -> list[dict]:
{
"thread_id": thread_id,
"thread_title": title,
"agent_id": agent_id if isinstance(agent_id, str) and agent_id else None,
"agent_id": normalized_agent_id or None,
"agent_name": normalized_agent_name or None,
"thread_type": thread_type if isinstance(thread_type, str) and thread_type else None,
"source_type": "agent" if normalized_agent_id else "general_qa",
**file,
"file_kind": artifact_library_kind(str(file.get("name") or "")),
}
@@ -393,6 +401,7 @@ async def list_artifact_library(
request: Request,
query: str | None = Query(default=None, max_length=200),
file_kind: ArtifactLibraryKind = Query(default="all"),
source_type: ArtifactLibrarySource = Query(default="all"),
offset: int = Query(default=0, ge=0),
limit: int = Query(default=50, ge=1, le=200),
refresh: bool = Query(default=False),
@@ -404,14 +413,35 @@ async def list_artifact_library(
explicit ``present_files`` call. Uploads and image/audio/video outputs are
intentionally outside this collection.
"""
from app.gateway.deps import get_thread_store
from app.gateway.deps import get_agent_store, get_thread_store
items = await _cached_artifact_library(get_thread_store(request), refresh=refresh)
agent_ids = list(dict.fromkeys(str(item.get("agent_id") or "") for item in items if item.get("agent_id")))
agent_names: dict[str, str] = {}
if agent_ids:
try:
agents = await get_agent_store(request).list_by_ids(
agent_ids,
str(get_effective_user_id() or "default"),
)
agent_names = {str(agent.get("id") or ""): str(agent.get("name") or "").strip() for agent in agents if agent.get("id") and agent.get("name")}
except Exception:
logger.warning("Failed to resolve artifact library agent names", exc_info=True)
items_with_sources = [
{
**item,
"agent_name": (agent_names.get(str(item.get("agent_id") or "")) or item.get("agent_name") or item.get("agent_id")),
}
for item in items
]
normalized_query = (query or "").strip().casefold()
filtered = [
item
for item in items
if (file_kind == "all" or item["file_kind"] == file_kind) and (not normalized_query or normalized_query in str(item.get("name") or "").casefold() or normalized_query in str(item.get("thread_title") or "").casefold())
for item in items_with_sources
if (file_kind == "all" or item["file_kind"] == file_kind)
and (source_type == "all" or item["source_type"] == source_type)
and (not normalized_query or normalized_query in str(item.get("name") or "").casefold() or normalized_query in str(item.get("thread_title") or "").casefold() or normalized_query in str(item.get("agent_name") or "").casefold())
]
page = filtered[offset : offset + limit]
return ArtifactLibraryResponse(
@@ -420,176 +450,176 @@ async def list_artifact_library(
limit=limit,
items=[ArtifactLibraryItem.model_validate(item) for item in page],
)
@router.get(
"/threads/{thread_id}/artifacts",
summary="List Artifact Files",
description="List all files under the thread's outputs directory (/mnt/user-data/outputs/). Returns an empty list if the thread has no outputs.",
)
@require_permission("threads", "read", owner_check=True)
async def list_artifacts(thread_id: str, request: Request) -> dict:
"""列出 thread 在 ``outputs/`` 下生成的产物文件。
圆桌 Step 3(``HighFidelityReport``)对每个**子席位** thread_id 调用本接口,
展示各智能体写入沙箱 ``mnt/user-data/outputs/`` 的文件;协调智能体 thread
通常无独立交付文件。
遍历路径::
.deer-flow/users/{user_id}/threads/{thread_id}/user-data/outputs/
返回 ``{ "thread_id", "files": [{ path, name, size_bytes, mime_type, modified_at }] }``,
目录不存在时 ``files`` 为空数组(非 404)。
"""
outputs_dir = get_paths().sandbox_outputs_dir(thread_id, user_id=await aresolve_path_user_id(thread_id))
return {"thread_id": thread_id, "files": collect_outputs_listing(outputs_dir)}
@router.get(
"/threads/{thread_id}/artifacts/{path:path}",
summary="Get Artifact File",
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
)
@require_permission("threads", "read", owner_check=True)
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
"""Get an artifact file by its path.
The endpoint automatically detects file types and returns appropriate content types.
Use the `download` query parameter to force file download for non-active content.
Args:
thread_id: The thread ID.
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
request: FastAPI request object (automatically injected).
Returns:
The file content as a FileResponse with appropriate content type:
- Active content (HTML/XHTML/SVG): Served as download attachment
- Text files: Plain text with proper MIME type
- Binary files: Inline display with download option
Raises:
HTTPException:
- 400 if path is invalid or not a file
- 403 if access denied (path traversal detected)
- 404 if file not found
Query Parameters:
download (bool): If true, forces attachment download for file types that are
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
is always downloaded regardless of this flag.
Example:
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
"""
path = coerce_artifact_virtual_path(path)
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
if ".skill/" in path:
# Split the path at ".skill/" to get the ZIP file path and internal path
skill_marker = ".skill/"
marker_pos = path.find(skill_marker)
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
actual_skill_path = await aresolve_thread_virtual_path(thread_id, skill_file_path)
if not actual_skill_path.exists():
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
if not actual_skill_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
# Extract the file from the .skill archive
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
if content is None:
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
# Determine MIME type based on the internal file
mime_type, _ = mimetypes.guess_type(internal_path)
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
cache_headers = {"Cache-Control": "private, max-age=300"}
download_name = Path(internal_path).name or actual_skill_path.stem
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
if mime_type and mime_type.startswith("text/"):
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
# Default to plain text for unknown types that look like text
try:
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
except UnicodeDecodeError:
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
actual_path = await aresolve_thread_virtual_path(thread_id, path)
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
if not actual_path.exists():
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
if not actual_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
mime_type, _ = mimetypes.guess_type(actual_path)
if download:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
# Always force download for active content types to prevent script execution
# in the application origin when users open generated artifacts.
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
if mime_type and mime_type.startswith("text/"):
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
if is_text_file_by_content(actual_path):
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
@router.patch(
"/threads/{thread_id}/artifacts/{path:path}",
summary="Update Artifact File",
description="Update a text artifact file in the current thread workspace.",
)
@require_permission("threads", "write", owner_check=True)
async def update_artifact(
thread_id: str,
path: str,
request: Request,
body: ArtifactUpdateRequest,
) -> Response:
"""Update a text artifact.
This endpoint is intentionally limited to normal text artifacts. Files inside
.skill archives and active web content remain read-only from the artifact UI.
"""
if ".skill/" in path:
raise HTTPException(status_code=400, detail="Skill archive contents are read-only.")
path = coerce_artifact_virtual_path(path)
actual_path = await aresolve_thread_virtual_path(thread_id, path)
logger.info(f"Updating artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
if not actual_path.exists():
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
if not actual_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
mime_type, _ = mimetypes.guess_type(actual_path)
suffix = actual_path.suffix.lower()
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
raise HTTPException(status_code=400, detail="Active web artifacts are read-only.")
if suffix not in EDITABLE_TEXT_SUFFIXES and not (mime_type and mime_type.startswith("text/")):
raise HTTPException(status_code=400, detail="Only text artifacts can be edited.")
actual_path.write_text(body.content, encoding="utf-8")
return PlainTextResponse(content=body.content, media_type=mime_type or "text/plain")
@router.get(
"/threads/{thread_id}/artifacts",
summary="List Artifact Files",
description="List all files under the thread's outputs directory (/mnt/user-data/outputs/). Returns an empty list if the thread has no outputs.",
)
@require_permission("threads", "read", owner_check=True)
async def list_artifacts(thread_id: str, request: Request) -> dict:
"""列出 thread 在 ``outputs/`` 下生成的产物文件。
圆桌 Step 3(``HighFidelityReport``)对每个**子席位** thread_id 调用本接口,
展示各智能体写入沙箱 ``mnt/user-data/outputs/`` 的文件;协调智能体 thread
通常无独立交付文件。
遍历路径::
.deer-flow/users/{user_id}/threads/{thread_id}/user-data/outputs/
返回 ``{ "thread_id", "files": [{ path, name, size_bytes, mime_type, modified_at }] }``,
目录不存在时 ``files`` 为空数组(非 404)。
"""
outputs_dir = get_paths().sandbox_outputs_dir(thread_id, user_id=await aresolve_path_user_id(thread_id))
return {"thread_id": thread_id, "files": collect_outputs_listing(outputs_dir)}
@router.get(
"/threads/{thread_id}/artifacts/{path:path}",
summary="Get Artifact File",
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
)
@require_permission("threads", "read", owner_check=True)
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
"""Get an artifact file by its path.
The endpoint automatically detects file types and returns appropriate content types.
Use the `download` query parameter to force file download for non-active content.
Args:
thread_id: The thread ID.
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
request: FastAPI request object (automatically injected).
Returns:
The file content as a FileResponse with appropriate content type:
- Active content (HTML/XHTML/SVG): Served as download attachment
- Text files: Plain text with proper MIME type
- Binary files: Inline display with download option
Raises:
HTTPException:
- 400 if path is invalid or not a file
- 403 if access denied (path traversal detected)
- 404 if file not found
Query Parameters:
download (bool): If true, forces attachment download for file types that are
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
is always downloaded regardless of this flag.
Example:
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
"""
path = coerce_artifact_virtual_path(path)
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
if ".skill/" in path:
# Split the path at ".skill/" to get the ZIP file path and internal path
skill_marker = ".skill/"
marker_pos = path.find(skill_marker)
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
actual_skill_path = await aresolve_thread_virtual_path(thread_id, skill_file_path)
if not actual_skill_path.exists():
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
if not actual_skill_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
# Extract the file from the .skill archive
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
if content is None:
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
# Determine MIME type based on the internal file
mime_type, _ = mimetypes.guess_type(internal_path)
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
cache_headers = {"Cache-Control": "private, max-age=300"}
download_name = Path(internal_path).name or actual_skill_path.stem
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
if mime_type and mime_type.startswith("text/"):
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
# Default to plain text for unknown types that look like text
try:
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
except UnicodeDecodeError:
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
actual_path = await aresolve_thread_virtual_path(thread_id, path)
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
if not actual_path.exists():
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
if not actual_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
mime_type, _ = mimetypes.guess_type(actual_path)
if download:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
# Always force download for active content types to prevent script execution
# in the application origin when users open generated artifacts.
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
if mime_type and mime_type.startswith("text/"):
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
if is_text_file_by_content(actual_path):
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
@router.patch(
"/threads/{thread_id}/artifacts/{path:path}",
summary="Update Artifact File",
description="Update a text artifact file in the current thread workspace.",
)
@require_permission("threads", "write", owner_check=True)
async def update_artifact(
thread_id: str,
path: str,
request: Request,
body: ArtifactUpdateRequest,
) -> Response:
"""Update a text artifact.
This endpoint is intentionally limited to normal text artifacts. Files inside
.skill archives and active web content remain read-only from the artifact UI.
"""
if ".skill/" in path:
raise HTTPException(status_code=400, detail="Skill archive contents are read-only.")
path = coerce_artifact_virtual_path(path)
actual_path = await aresolve_thread_virtual_path(thread_id, path)
logger.info(f"Updating artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
if not actual_path.exists():
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
if not actual_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
mime_type, _ = mimetypes.guess_type(actual_path)
suffix = actual_path.suffix.lower()
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
raise HTTPException(status_code=400, detail="Active web artifacts are read-only.")
if suffix not in EDITABLE_TEXT_SUFFIXES and not (mime_type and mime_type.startswith("text/")):
raise HTTPException(status_code=400, detail="Only text artifacts can be edited.")
actual_path.write_text(body.content, encoding="utf-8")
return PlainTextResponse(content=body.content, media_type=mime_type or "text/plain")
File diff suppressed because it is too large. Load diff
@@ -172,21 +172,15 @@ async def create_sync_job(request: Request, body: CreateSyncJobRequest) -> dict[
targets: list[dict[str, Any]] = []
for target in body.targets:
data = target.model_dump()
if target.target_type == "weknora":
mapping = await request.app.state.llmwiki_store.get_authorized(target.target_id, actor or "system", write=True, is_admin=True)
if mapping is None:
raise HTTPException(status_code=404, detail=f"普通知识库不存在:{target.target_id}")
data["target_name"] = target.target_name or str(mapping.get("name") or target.target_id)
data["remote_business_key"] = target.remote_business_key or f"weknora:kb:{mapping['weknora_id']}"
else:
base = await request.app.state.assistant_knowledge_store.get_base()
if base is None:
base = await request.app.state.assistant_knowledge_store.initialize(created_by=actor)
if target.target_id not in {base["id"], "global"}:
raise HTTPException(status_code=404, detail="助手知识库仅允许选择全局知识梳理库")
data["target_id"] = base["id"]
data["target_name"] = base["name"]
data["remote_business_key"] = f"assistant:base:{base['id']}"
if target.target_type != "weknora":
raise HTTPException(status_code=422, detail="不支持技能直接归纳到助手知识库,请先归纳到 WeKnora 普通知识库")
if target.target_mode != "wiki":
raise HTTPException(status_code=422, detail="技能归纳目标仅支持 WeKnora Wiki 知识库")
mapping = await request.app.state.llmwiki_store.get_authorized(target.target_id, actor or "system", write=True, is_admin=True)
if mapping is None:
raise HTTPException(status_code=404, detail=f"普通知识库不存在:{target.target_id}")
data["target_name"] = target.target_name or str(mapping.get("name") or target.target_id)
data["remote_business_key"] = target.remote_business_key or f"weknora:kb:{mapping['weknora_id']}"
targets.append(data)
job = await _store(request).create_job(
skill_names=body.skill_names,
@@ -405,6 +405,7 @@ _CONTEXT_CONFIGURABLE_KEYS: frozenset[str] = frozenset(
"reasoning_effort",
"rag_mode_enabled",
"llmwiki_knowledge_base_ids",
"assistant_knowledge_base_ids",
"writing_mode",
"writing_artifact_path",
"canvas_artifact_title",
@@ -1113,7 +1114,7 @@ async def start_run(
if prefix_settings.enabled and prefix_settings.prompt_prefix.strip():
graph_input = _prepend_prompt_prefix(graph_input, prefix_settings.prompt_prefix.strip())
llmwiki_rag = (
LlmWikiRagContext([], 0, "", [], "")
LlmWikiRagContext([], [], 0, "", [], "")
if fixed_answer is not None
else await build_llmwiki_rag_context(
request=request,
@@ -1125,12 +1126,14 @@ async def start_run(
graph_input = attach_llmwiki_rag_to_graph_input(graph_input, llmwiki_rag)
effective_rag_mode = True
run_metadata["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
run_metadata["assistant_knowledge_base_ids"] = llmwiki_rag.assistant_knowledge_base_ids
run_metadata["llmwiki_rag_source_count"] = llmwiki_rag.source_count
try:
await run_ctx.thread_store.update_metadata(
thread_id,
{
"llmwiki_knowledge_base_ids": llmwiki_rag.knowledge_base_ids,
"assistant_knowledge_base_ids": llmwiki_rag.assistant_knowledge_base_ids,
"rag_mode_enabled": True,
"rag_mode": True,
},
@@ -1150,6 +1153,8 @@ async def start_run(
body_context["rag_mode_enabled"] = True
if "llmwiki_knowledge_base_ids" not in body_context and llmwiki_rag.knowledge_base_ids:
body_context["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
if "assistant_knowledge_base_ids" not in body_context and llmwiki_rag.assistant_knowledge_base_ids:
body_context["assistant_knowledge_base_ids"] = llmwiki_rag.assistant_knowledge_base_ids
if not llmwiki_rag.knowledge_base_ids:
existing_excluded = list(body_context.get("excluded_tools") or getattr(body, "excluded_tools", None) or [])
if "llmwiki_search" not in existing_excluded:
+10 -5
View File
@@ -429,7 +429,11 @@ auth_login:
database:
backend: sqlite
sqlite_dir: .deer-flow/data
pool_size: 20 # 基础连接池(postgres/mysql 生效;sqlite 忽略)
# 高斯/OpenGauss 业务库示例(启用前先执行 uv sync --extra gauss):
# backend: gauss
# gauss_url: $GAUSS_DATABASE_URL # opengauss+asyncpg://user:pass@host:26000/deerflow
# checkpointer 请继续使用下方 sqlite;不要把 PostgreSQL 专用 checkpointer 迁移直接用于高斯。
pool_size: 20 # 基础连接池(postgres/gauss/mysql 生效;sqlite 忽略)
max_overflow: 20 # 高峰期可临时超出 pool_size 的额外连接数,真实上限 = pool_size + max_overflow
pool_timeout: 30 # 池满时等待空闲连接的秒数,超时报错
pool_recycle: 1800 # 连接回收周期(秒);防火墙/pgbouncer/MySQL wait_timeout 静默断连兜底。多 worker 部署:每 worker 上限 = pool_size + max_overflow,总数 × worker 数 < DB max_connections
@@ -564,9 +568,10 @@ llmwiki:
password: ""
database: "neo4j"
timeout_seconds: 30
# 本地 Wiki 向量化:只需把 enabled 改为 true,并填写下面 3 个 embedding 值。
# 默认只在页面点击“向量化”时执行;auto_sync=true 才会后台自动增量同步,
# 且服务启动时只启动定时器,不会立刻全量向量化。
# 本地 Wiki 向量化:用于普通知识库“下载向量化数据包”和助手知识库向量检索。
# 已有相同内容哈希 + 相同 embedding fingerprint 的 Wiki 向量会直接复用,不会重复编码。
# 如需导出时自动补建向量,把 enabled 改为 true,并填写下面 embedding 服务地址。
# auto_sync=true 才会后台自动增量同步;服务启动时只启动定时器,不会立刻全量向量化。
local_wiki_index:
enabled: false
# 免登录 POST /api/knowledge/vector-search 返回的 Wiki 一键跳转链接前缀。
@@ -574,7 +579,7 @@ llmwiki:
auto_sync: false
database_batch_size: 50
embedding:
model: bge-m3
model: bge-embedding-m3
base_url: ""
api_key: ""
@@ -0,0 +1,109 @@
"""File-backed assistant knowledge import/export packages.
The assistant knowledge base is fed by WeKnora-exported data. Keep that
boundary explicit: a WeKnora export is first materialized as a durable JSONL
package, then imported asynchronously from that package. JSONL keeps large
exports append-friendly and avoids treating the HTTP request body as the job
state.
"""
from __future__ import annotations
import json
from collections.abc import Iterable, Mapping
from pathlib import Path
from typing import Any
PACKAGE_VERSION = 1
PACKAGE_RECORD_TYPES = {
"source",
"wiki_page",
"document",
"chunk",
"vector",
"entity",
"relation",
"graph",
}
def package_path(root: Path, job_id: str) -> Path:
safe = "".join(ch for ch in str(job_id) if ch.isalnum() or ch in {"-", "_"})
return root / f"{safe}.jsonl"
def write_jsonl_record(handle: Any, record_type: str, payload: Mapping[str, Any]) -> None:
handle.write(json.dumps({"record_type": record_type, "payload": dict(payload)}, ensure_ascii=False))
handle.write("\n")
def write_package(path: Path, records: Iterable[tuple[str, Mapping[str, Any]]]) -> dict[str, int]:
path.parent.mkdir(parents=True, exist_ok=True)
counts: dict[str, int] = {key: 0 for key in PACKAGE_RECORD_TYPES}
tmp_path = path.with_suffix(path.suffix + ".tmp")
with tmp_path.open("w", encoding="utf-8", newline="\n") as handle:
write_jsonl_record(
handle,
"source",
{
"package_version": PACKAGE_VERSION,
},
)
counts["source"] += 1
for record_type, payload in records:
if record_type not in PACKAGE_RECORD_TYPES:
continue
write_jsonl_record(handle, record_type, payload)
counts[record_type] = counts.get(record_type, 0) + 1
tmp_path.replace(path)
return counts
def read_package(path: Path) -> dict[str, Any]:
package: dict[str, Any] = {
"source": {},
"wiki_pages": [],
"documents": [],
"chunks": [],
"vectors": [],
"entities": [],
"relations": [],
"graph": {"nodes": [], "edges": []},
}
with path.open("r", encoding="utf-8") as handle:
for line in handle:
line = line.strip()
if not line:
continue
record = json.loads(line)
if not isinstance(record, dict):
continue
record_type = str(record.get("record_type") or "")
payload = record.get("payload")
if not isinstance(payload, dict):
continue
if record_type == "source":
package["source"] = {**package.get("source", {}), **payload}
elif record_type == "wiki_page":
package["wiki_pages"].append(payload)
elif record_type == "document":
package["documents"].append(payload)
elif record_type == "chunk":
package["chunks"].append(payload)
elif record_type == "vector":
package["vectors"].append(payload)
elif record_type == "entity":
package["entities"].append(payload)
elif record_type == "relation":
package["relations"].append(payload)
elif record_type == "graph":
nodes = payload.get("nodes")
edges = payload.get("edges")
if isinstance(nodes, list):
package["graph"]["nodes"].extend(item for item in nodes if isinstance(item, dict))
if isinstance(edges, list):
package["graph"]["edges"].extend(item for item in edges if isinstance(item, dict))
for key, value in payload.items():
if key not in {"nodes", "edges"}:
package["graph"][key] = value
return package
@@ -11,10 +11,15 @@ blocking, making a unified file safe for both workloads. Writers
that contend for the lock wait via the default 5-second sqlite3
busy timeout rather than failing immediately.
Postgres mode: both use the same database URL but maintain independent
connection pools with different lifecycles.
MySQL mode: application ORM data uses MySQL. LangGraph's checkpointer does
Postgres mode: both use the same database URL but maintain independent
connection pools with different lifecycles.
Gauss mode: application ORM data uses the official openGauss SQLAlchemy
dialect over asyncpg. LangGraph's PostgreSQL checkpointer is intentionally not
reused for GaussDB; configure the legacy ``checkpointer:`` section separately
(normally sqlite).
MySQL mode: application ORM data uses MySQL. LangGraph's checkpointer does
not have a built-in MySQL backend in this project, so deployments that choose
MySQL for business tables should configure the legacy ``checkpointer:``
section separately (usually sqlite) to preserve conversation checkpoints.
@@ -42,24 +47,40 @@ from typing import Literal
from pydantic import BaseModel, Field
class DatabaseConfig(BaseModel):
backend: Literal["memory", "sqlite", "postgres", "mysql"] = Field(
default="memory",
description=("Storage backend for application data. 'memory' for development (no persistence across restarts), 'sqlite' for single-node deployment, 'postgres' for production multi-node deployment, 'mysql' for MySQL-backed business tables."),
class DatabaseConfig(BaseModel):
backend: Literal["memory", "sqlite", "postgres", "gauss", "mysql"] = Field(
default="memory",
description=(
"Storage backend for application data. 'memory' for development "
"(no persistence across restarts), 'sqlite' for single-node "
"deployment, 'postgres' for production multi-node deployment, "
"'gauss' for openGauss/GaussDB business tables, and 'mysql' for "
"MySQL-backed business tables."
),
)
sqlite_dir: str = Field(
default=".deer-flow/data",
description=("Directory for the SQLite database file. Both checkpointer and application data share {sqlite_dir}/deerflow.db."),
)
postgres_url: str = Field(
postgres_url: str = Field(
default="",
description=(
"PostgreSQL connection URL, shared by checkpointer and app. "
"Use $DATABASE_URL in config.yaml to reference .env. "
"Example: postgresql://user:pass@host:5432/deerflow "
"(the +asyncpg driver suffix is added automatically where needed)."
),
)
),
)
gauss_url: str = Field(
default="",
description=(
"openGauss/GaussDB connection URL for the application ORM. "
"Use $GAUSS_DATABASE_URL in config.yaml to reference .env. "
"Example: opengauss+asyncpg://user:pass@host:26000/deerflow. "
"Plain opengauss:// or PostgreSQL-style URLs are normalized to the "
"official asynchronous openGauss SQLAlchemy dialect."
),
)
mysql_url: str = Field(
default="",
description=(
@@ -75,26 +96,26 @@ class DatabaseConfig(BaseModel):
)
pool_size: int = Field(
default=5,
description="Connection pool size for the app ORM engine (postgres/mysql only).",
description="Connection pool size for the app ORM engine (postgres/gauss/mysql only).",
)
max_overflow: int = Field(
default=10,
description=(
"Extra connections allowed beyond pool_size under load (postgres/mysql only). "
"Extra connections allowed beyond pool_size under load (postgres/gauss/mysql only). "
"The real connection ceiling is pool_size + max_overflow. SQLAlchemy default is 10."
),
)
pool_timeout: int = Field(
default=30,
description=(
"Seconds a caller waits for a free pooled connection before erroring (postgres/mysql only). "
"Seconds a caller waits for a free pooled connection before erroring (postgres/gauss/mysql only). "
"SQLAlchemy default is 30."
),
)
pool_recycle: int = Field(
default=1800,
description=(
"Recycle pooled connections after this many seconds (postgres/mysql only). "
"Recycle pooled connections after this many seconds (postgres/gauss/mysql only). "
"Prevents stale-connection errors when a firewall, pgbouncer, or MySQL "
"wait_timeout silently drops idle connections. pool_pre_ping catches most "
"cases, but pool_recycle is the belt-and-suspenders safety net. "
@@ -133,12 +154,38 @@ class DatabaseConfig(BaseModel):
"""SQLAlchemy async URL for the application ORM engine."""
if self.backend == "sqlite":
return f"sqlite+aiosqlite:///{self.sqlite_path}"
if self.backend == "postgres":
url = self.postgres_url
if url.startswith("postgresql://"):
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
return url
if self.backend == "mysql":
if self.backend == "postgres":
url = self.postgres_url
if url.startswith("postgresql://"):
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
return url
if self.backend == "gauss":
url = self.gauss_url.strip()
if not url:
raise ValueError("database.gauss_url is required for backend='gauss'")
# GaussDB speaks the PostgreSQL wire protocol, but its server
# version string and DDL capabilities are not identical to
# PostgreSQL. Always select the official openGauss dialect instead
# of silently sending the URL through SQLAlchemy's generic PG
# dialect (the latter commonly connects and then fails while
# SQLAlchemy initializes/creates the schema).
for prefix in (
"opengauss://",
"gaussdb://",
"postgres://",
"postgresql://",
"postgresql+asyncpg://",
):
if url.startswith(prefix):
return "opengauss+asyncpg://" + url[len(prefix) :]
if url.startswith("opengauss+asyncpg://"):
return url
raise ValueError(
"database.gauss_url must use opengauss://, "
"opengauss+asyncpg://, or a PostgreSQL-style URL"
)
if self.backend == "mysql":
url = self.mysql_url
if not url:
raise ValueError("database.mysql_url is required for backend='mysql'")
@@ -73,7 +73,7 @@ class WeKnoraDirectGraphConfig(BaseModel):
class LocalWikiEmbeddingConfig(BaseModel):
enabled: bool = True
provider: str = "openai"
model: str = "bge-m3"
model: str = "bge-embedding-m3"
base_url: str = ""
api_key: str = Field(default="", repr=False)
dimensions: int | None = Field(
@@ -494,7 +494,7 @@ class WeKnoraClient:
await self._request(
"GET",
f"/api/v1/knowledgebase/{quote(knowledge_base_id, safe='')}/wiki/graph",
params={"mode": "overview", "limit": max(1, min(limit, 500))},
params={"mode": "overview", "limit": max(1, min(limit, 10000))},
)
)
if not isinstance(value, dict):
@@ -1,7 +1,37 @@
"""DeerFlow-local mirror and Wiki-only retrieval for WeKnora Wiki pages."""
from deerflow.integrations.weknora.local_index.retrieval import WikiRetrievalService, is_fully_vectorized
from deerflow.integrations.weknora.local_index.search import WikiIndexError, WikiVectorSearchService
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
from __future__ import annotations
__all__ = ["WikiIndexError", "WikiRetrievalService", "WikiVectorSearchService", "WikiSyncService", "is_fully_vectorized"]
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from deerflow.integrations.weknora.local_index.retrieval import (
WikiRetrievalService,
is_fully_vectorized,
)
from deerflow.integrations.weknora.local_index.search import WikiIndexError, WikiVectorSearchService
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
__all__ = [
"WikiIndexError",
"WikiRetrievalService",
"WikiVectorSearchService",
"WikiSyncService",
"is_fully_vectorized",
]
def __getattr__(name: str) -> Any:
if name in {"WikiRetrievalService", "is_fully_vectorized"}:
from deerflow.integrations.weknora.local_index import retrieval
return getattr(retrieval, name)
if name in {"WikiIndexError", "WikiVectorSearchService"}:
from deerflow.integrations.weknora.local_index import search
return getattr(search, name)
if name == "WikiSyncService":
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
return WikiSyncService
raise AttributeError(name)
@@ -1,10 +1,10 @@
"""ORM model for the single global DeerFlow assistant knowledge base."""
"""ORM model for DeerFlow-native assistant knowledge bases."""
from __future__ import annotations
from datetime import UTC, datetime
from sqlalchemy import Float, Integer, String, UniqueConstraint
from sqlalchemy import Float, Index, Integer, LargeBinary, String, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
@@ -19,7 +19,12 @@ class AssistantKnowledgeBaseRow(Base):
__tablename__ = "assistant_knowledge_base"
id: Mapped[str] = mapped_column(String(36), primary_key=True)
name: Mapped[str] = mapped_column(String(255), default="知识梳理")
status: Mapped[str] = mapped_column(String(24), default="active", unique=True)
description: Mapped[str | None] = mapped_column(String(2048), nullable=True)
# ``global`` is the full-library aggregation base. ``custom`` bases are
# user-created assistant knowledge bases fed from WeKnora export packages.
# Nullable keeps startup column reconciliation safe for existing databases.
base_kind: Mapped[str | None] = mapped_column(String(24), nullable=True, index=True)
status: Mapped[str] = mapped_column(String(64), default="active")
created_by: Mapped[str | None] = mapped_column(String(64), nullable=True)
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
updated_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now, onupdate=_now)
@@ -91,6 +96,32 @@ class AssistantKnowledgeWikiRevisionRow(Base):
published_at: Mapped[datetime | None] = mapped_column(BeijingDateTime(), nullable=True)
class AssistantKnowledgeVectorRow(Base):
__tablename__ = "assistant_knowledge_vectors"
__table_args__ = (
Index("ix_assistant_knowledge_vector_base_fingerprint", "base_id", "embedding_fingerprint"),
Index("ix_assistant_knowledge_vector_page", "page_id"),
Index("ix_assistant_knowledge_vector_source", "source_id"),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True)
base_id: Mapped[str] = mapped_column(String(36), index=True)
page_id: Mapped[str] = mapped_column(String(36), index=True)
source_id: Mapped[str] = mapped_column(String(36), index=True)
import_job_id: Mapped[str] = mapped_column(String(36), index=True)
source_vector_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
source_chunk_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
section_index: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
heading: Mapped[str | None] = mapped_column(String(1024), nullable=True)
section_content: Mapped[str] = mapped_column(PortableLongText(), nullable=False, default="")
content_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
embedding_model: Mapped[str | None] = mapped_column(String(255), nullable=True)
embedding_fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
embedding_dimensions: Mapped[int] = mapped_column(Integer, nullable=False)
vector_blob: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
class AssistantKnowledgeEntityRow(Base):
__tablename__ = "assistant_knowledge_entities"
__table_args__ = (UniqueConstraint("base_id", "entity_type", "normalized_name", name="uq_assistant_entity"),)
@@ -1,15 +1,21 @@
"""Repository for the single global assistant knowledge base."""
"""Repository for DeerFlow-native assistant knowledge bases."""
from __future__ import annotations
import base64
import binascii
import difflib
import hashlib
import logging
from datetime import UTC, datetime
from typing import Any
from uuid import uuid4
import numpy as np
from sqlalchemy import delete, func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.integrations.weknora.local_index.vector_codec import decode_vector, encode_vector, normalize_vector
from deerflow.persistence.assistant_knowledge.model import (
AssistantKnowledgeBaseRow,
AssistantKnowledgeEntityContributionRow,
@@ -18,11 +24,24 @@ from deerflow.persistence.assistant_knowledge.model import (
AssistantKnowledgeRelationContributionRow,
AssistantKnowledgeRelationRow,
AssistantKnowledgeSourceRow,
AssistantKnowledgeVectorRow,
AssistantKnowledgeWikiPageRow,
AssistantKnowledgeWikiRevisionRow,
)
from deerflow.skill_knowledge.merge import normalize_name, normalize_predicate
GLOBAL_BASE_NAME = "知识梳理总库"
logger = logging.getLogger(__name__)
VECTOR_FIELD_NAMES = (
"embedding",
"vector",
"embedding_vector",
"content_vector",
"dense_vector",
"dense",
)
def _now() -> datetime:
return datetime.now(UTC)
@@ -35,31 +54,246 @@ def _dict(row: Any) -> dict[str, Any]:
data[key] = value.isoformat()
elif key.endswith("_json"):
data[key[:-5]] = data.pop(key) or ([] if "aliases" in key or "evidence" in key else {})
if isinstance(row, AssistantKnowledgeBaseRow):
raw_status = str(data.get("status") or "")
data["status"] = "active" if raw_status.startswith("active") else raw_status
data["base_kind"] = data.get("base_kind") or "custom"
data["is_global"] = data["base_kind"] == "global"
data["description"] = data.get("description") or ""
return data
def _stable_slug_part(value: Any) -> str:
raw = str(value or "").strip()
if not raw:
raw = str(uuid4())
return hashlib.sha256(raw.encode("utf-8")).hexdigest()[:24]
def _content_hash(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
def _first_vector_value(raw: dict[str, Any]) -> Any:
for key in VECTOR_FIELD_NAMES:
value = raw.get(key)
if value is not None:
return value
vectors = raw.get("vectors")
if isinstance(vectors, dict):
for key in VECTOR_FIELD_NAMES:
value = vectors.get(key)
if value is not None:
return value
return None
def _vector_blob_from_payload(raw: dict[str, Any]) -> tuple[bytes, int] | None:
blob64 = raw.get("vector_blob_base64") or raw.get("embedding_blob_base64")
dimensions = raw.get("embedding_dimensions") or raw.get("dimensions")
if blob64 and dimensions:
try:
decoded = base64.b64decode(str(blob64), validate=True)
vector = decode_vector(decoded, int(dimensions))
return encode_vector(vector)
except (ValueError, TypeError, binascii.Error):
return None
value = _first_vector_value(raw)
if value is None:
return None
if isinstance(value, dict):
value = _first_vector_value(value)
if not isinstance(value, (list, tuple)):
return None
try:
return encode_vector(value, dimensions=int(dimensions) if dimensions else None)
except (ValueError, TypeError):
return None
class AssistantKnowledgeRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
async def get_base(self) -> dict[str, Any] | None:
stmt = select(AssistantKnowledgeBaseRow).where(AssistantKnowledgeBaseRow.status == "active").limit(1)
async with self._sf() as session:
row = (await session.execute(stmt)).scalar_one_or_none()
return _dict(row) if row else None
async def _base_with_stats(
self,
session: AsyncSession,
row: AssistantKnowledgeBaseRow,
) -> dict[str, Any]:
data = _dict(row)
data["source_count"] = int(
(
await session.execute(
select(func.count(AssistantKnowledgeSourceRow.id)).where(
AssistantKnowledgeSourceRow.base_id == row.id,
AssistantKnowledgeSourceRow.status != "deleted",
)
)
).scalar_one()
or 0
)
data["page_count"] = int(
(
await session.execute(
select(func.count(AssistantKnowledgeWikiPageRow.id)).where(
AssistantKnowledgeWikiPageRow.base_id == row.id,
AssistantKnowledgeWikiPageRow.status == "active",
)
)
).scalar_one()
or 0
)
data["entity_count"] = int(
(
await session.execute(
select(func.count(AssistantKnowledgeEntityRow.id)).where(
AssistantKnowledgeEntityRow.base_id == row.id,
AssistantKnowledgeEntityRow.status == "active",
)
)
).scalar_one()
or 0
)
data["relation_count"] = int(
(
await session.execute(
select(func.count(AssistantKnowledgeRelationRow.id)).where(
AssistantKnowledgeRelationRow.base_id == row.id,
AssistantKnowledgeRelationRow.status == "active",
)
)
).scalar_one()
or 0
)
return data
async def initialize(self, *, created_by: str | None, name: str = "知识梳理") -> dict[str, Any]:
async def get_base(self, base_id: str | None = None) -> dict[str, Any] | None:
async with self._sf() as session:
if base_id:
row = (
await session.execute(
select(AssistantKnowledgeBaseRow).where(
AssistantKnowledgeBaseRow.id == base_id,
AssistantKnowledgeBaseRow.status != "deleted",
)
)
).scalar_one_or_none()
return await self._base_with_stats(session, row) if row else None
row = (
await session.execute(
select(AssistantKnowledgeBaseRow)
.where(
AssistantKnowledgeBaseRow.status != "deleted",
AssistantKnowledgeBaseRow.base_kind == "global",
)
.order_by(AssistantKnowledgeBaseRow.created_at.asc())
.limit(1)
)
).scalar_one_or_none()
if row is None:
# Backward compatibility: earlier builds had a single active row
# and no ``base_kind``. Promote the first surviving base into the
# global full-library aggregation base.
row = (
await session.execute(
select(AssistantKnowledgeBaseRow)
.where(AssistantKnowledgeBaseRow.status != "deleted")
.order_by(AssistantKnowledgeBaseRow.created_at.asc())
.limit(1)
)
).scalar_one_or_none()
if row is not None:
row.base_kind = "global"
if not row.description:
row.description = "默认全库沉淀库,汇聚所有 WeKnora 导出的助手知识数据。"
row.updated_at = _now()
await session.flush()
await session.commit()
return await self._base_with_stats(session, row) if row else None
async def list_bases(self) -> list[dict[str, Any]]:
await self.get_base()
async with self._sf() as session:
rows = (
await session.execute(
select(AssistantKnowledgeBaseRow)
.where(AssistantKnowledgeBaseRow.status != "deleted")
.order_by(
AssistantKnowledgeBaseRow.base_kind.desc(),
AssistantKnowledgeBaseRow.updated_at.desc(),
)
)
).scalars()
values = [await self._base_with_stats(session, row) for row in rows]
global_values = [item for item in values if item.get("is_global")]
custom_values = sorted(
[item for item in values if not item.get("is_global")],
key=lambda item: item.get("updated_at") or "",
reverse=True,
)
return global_values + custom_values
async def initialize(self, *, created_by: str | None, name: str = GLOBAL_BASE_NAME) -> dict[str, Any]:
existing = await self.get_base()
if existing:
return existing
row = AssistantKnowledgeBaseRow(id=str(uuid4()), name=name.strip() or "知识梳理", status="active", created_by=created_by, created_at=_now(), updated_at=_now())
row = AssistantKnowledgeBaseRow(
id=str(uuid4()),
name=name.strip() or GLOBAL_BASE_NAME,
description="默认全库沉淀库,汇聚所有 WeKnora 导出的助手知识数据。",
base_kind="global",
status="active",
created_by=created_by,
created_at=_now(),
updated_at=_now(),
)
async with self._sf() as session:
session.add(row)
await session.flush()
data = _dict(row)
data = await self._base_with_stats(session, row)
await session.commit()
return data
async def create_base(
self,
*,
created_by: str | None,
name: str,
description: str = "",
) -> dict[str, Any]:
row_id = str(uuid4())
row = AssistantKnowledgeBaseRow(
id=row_id,
name=name.strip(),
description=description.strip(),
base_kind="custom",
# Some upgraded SQLite/MySQL databases may still carry the old
# unique(status) constraint from the single-base implementation and
# its original VARCHAR(24) length. Use a compact per-row active
# marker internally and normalize it in API responses.
status=f"active:{row_id[:17]}",
created_by=created_by,
created_at=_now(),
updated_at=_now(),
)
async with self._sf() as session:
session.add(row)
await session.flush()
data = await self._base_with_stats(session, row)
await session.commit()
return data
async def _ensure_base(self, base_id: str | None, *, created_by: str | None) -> dict[str, Any]:
if base_id == "global":
return await self.initialize(created_by=created_by)
if base_id:
base = await self.get_base(base_id)
if base is None:
raise ValueError("助手知识库不存在")
return base
return await self.initialize(created_by=created_by)
async def _upsert_source(
self,
session: AsyncSession,
@@ -181,8 +415,56 @@ class AssistantKnowledgeRepository:
skill_name: str | None = None,
mapping_id: str | None = None,
upload_batch_id: str | None = None,
base_id: str | None = None,
mirror_to_global: bool = True,
) -> dict[str, Any]:
base = await self.initialize(created_by=created_by)
base = await self._ensure_base(base_id, created_by=created_by)
result = await self._import_package_into_base(
base=base,
package=package,
source_type=source_type,
source_key=source_key,
source_name=source_name,
trigger=trigger,
created_by=created_by,
skill_name=skill_name,
mapping_id=mapping_id,
upload_batch_id=upload_batch_id,
)
if mirror_to_global and not base.get("is_global"):
global_base = await self.initialize(created_by=created_by)
mirror = await self._import_package_into_base(
base=global_base,
package=package,
source_type=source_type,
source_key=source_key,
source_name=source_name,
trigger=f"{trigger}:global_mirror"[:48],
created_by=created_by,
skill_name=skill_name,
mapping_id=mapping_id,
upload_batch_id=upload_batch_id,
)
result["mirrored_global_base_id"] = global_base["id"]
result["mirrored_global_job_id"] = mirror["id"]
return result
async def create_queued_import_job(
self,
*,
source_type: str,
source_key: str,
source_name: str,
trigger: str,
created_by: str | None,
metadata: dict[str, Any] | None = None,
skill_name: str | None = None,
mapping_id: str | None = None,
upload_batch_id: str | None = None,
base_id: str | None = None,
phase: str = "queued",
) -> dict[str, Any]:
base = await self._ensure_base(base_id, created_by=created_by)
now = _now()
async with self._sf() as session:
source = await self._upsert_source(
@@ -191,22 +473,18 @@ class AssistantKnowledgeRepository:
source_type=source_type,
source_key=source_key,
source_name=source_name,
metadata=package.get("source") or {},
metadata=metadata or {},
skill_name=skill_name,
mapping_id=mapping_id,
upload_batch_id=upload_batch_id,
)
# A re-import replaces this source's prior contributions while retaining
# canonical entities/relations that are still supported by other sources.
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source.id))
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source.id))
job = AssistantKnowledgeImportJobRow(
id=str(uuid4()),
base_id=base["id"],
source_id=source.id,
trigger=trigger,
status="running",
phase="aligning_entities",
status="queued",
phase=phase,
counts_json={},
created_by=created_by,
created_at=now,
@@ -214,6 +492,166 @@ class AssistantKnowledgeRepository:
session.add(job)
await session.flush()
source.last_import_job_id = job.id
result = _dict(job)
await session.commit()
return result
async def update_import_job(
self,
job_id: str,
*,
status: str | None = None,
phase: str | None = None,
counts: dict[str, Any] | None = None,
error: str | None = None,
completed: bool = False,
) -> dict[str, Any] | None:
async with self._sf() as session:
job = await session.get(AssistantKnowledgeImportJobRow, job_id)
if job is None:
return None
if status is not None:
job.status = status
if phase is not None:
job.phase = phase
if counts is not None:
job.counts_json = counts
if error is not None:
job.last_error_message = error[:2048]
elif status not in {None, "failed"}:
job.last_error_message = None
if completed:
job.completed_at = _now()
await session.flush()
result = _dict(job)
await session.commit()
return result
async def import_package_for_job(
self,
job_id: str,
*,
package: dict[str, Any],
created_by: str | None,
mirror_to_global: bool = True,
) -> dict[str, Any]:
async with self._sf() as session:
job = await session.get(AssistantKnowledgeImportJobRow, job_id)
if job is None:
raise ValueError("导入任务不存在")
source = await session.get(AssistantKnowledgeSourceRow, job.source_id)
base_row = await session.get(AssistantKnowledgeBaseRow, job.base_id)
if source is None or base_row is None:
raise ValueError("导入任务来源不存在")
base = _dict(base_row)
source_type = source.source_type
source_key = source.source_key
source_name = source.source_name
source_id = source.id
trigger = job.trigger
skill_name = source.skill_name
mapping_id = source.knowledge_base_mapping_id
upload_batch_id = source.upload_batch_id
result = await self._import_package_into_base(
base=base,
package=package,
source_type=source_type,
source_key=source_key,
source_name=source_name,
trigger=trigger,
created_by=created_by,
skill_name=skill_name,
mapping_id=mapping_id,
upload_batch_id=upload_batch_id,
existing_source_id=source_id,
existing_job_id=job_id,
)
if mirror_to_global and not base.get("is_global"):
global_base = await self.initialize(created_by=created_by)
mirror = await self._import_package_into_base(
base=global_base,
package=package,
source_type=source_type,
source_key=source_key,
source_name=source_name,
trigger=f"{trigger}:global_mirror"[:48],
created_by=created_by,
skill_name=skill_name,
mapping_id=mapping_id,
upload_batch_id=upload_batch_id,
)
result["mirrored_global_base_id"] = global_base["id"]
result["mirrored_global_job_id"] = mirror["id"]
return result
async def _import_package_into_base(
self,
*,
base: dict[str, Any],
package: dict[str, Any],
source_type: str,
source_key: str,
source_name: str,
trigger: str,
created_by: str | None,
skill_name: str | None = None,
mapping_id: str | None = None,
upload_batch_id: str | None = None,
existing_source_id: str | None = None,
existing_job_id: str | None = None,
) -> dict[str, Any]:
now = _now()
async with self._sf() as session:
if existing_source_id:
source = await session.get(AssistantKnowledgeSourceRow, existing_source_id)
if source is None or source.base_id != base["id"]:
raise ValueError("导入任务来源不存在")
source.source_name = source_name
source.status = "active"
source.metadata_json = package.get("source") or {}
source.updated_at = now
else:
source = await self._upsert_source(
session,
base_id=base["id"],
source_type=source_type,
source_key=source_key,
source_name=source_name,
metadata=package.get("source") or {},
skill_name=skill_name,
mapping_id=mapping_id,
upload_batch_id=upload_batch_id,
)
# A re-import replaces this source's prior contributions while retaining
# canonical entities/relations that are still supported by other sources.
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source.id))
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source.id))
await session.execute(delete(AssistantKnowledgeVectorRow).where(AssistantKnowledgeVectorRow.source_id == source.id))
if existing_job_id:
job = await session.get(AssistantKnowledgeImportJobRow, existing_job_id)
if job is None or job.base_id != base["id"]:
raise ValueError("导入任务不存在")
job.source_id = source.id
job.status = "running"
job.phase = "aligning_entities"
job.last_error_message = None
job.completed_at = None
else:
job = AssistantKnowledgeImportJobRow(
id=str(uuid4()),
base_id=base["id"],
source_id=source.id,
trigger=trigger,
status="running",
phase="aligning_entities",
counts_json={},
created_by=created_by,
created_at=now,
)
session.add(job)
await session.flush()
source.last_import_job_id = job.id
entity_map: dict[str, str] = {}
entity_rows: dict[str, tuple[AssistantKnowledgeEntityRow, dict[str, Any]]] = {}
@@ -363,6 +801,7 @@ class AssistantKnowledgeRepository:
wiki = package["wiki"]
page = wiki.get("skill_page") if isinstance(wiki, dict) else None
pages = [page] if page else []
page_ids_by_slug: dict[str, str] = {}
for raw in pages:
if not isinstance(raw, dict):
continue
@@ -404,16 +843,139 @@ class AssistantKnowledgeRepository:
)
session.add(revision)
page.title = title
page.page_type = str(raw.get("page_type") or "source")
page.content_markdown = content
page.summary = summary
page.search_text = "\n".join([title, summary, content, " ".join(raw.get("aliases") or [])])
page.current_revision_id = revision.id
page.status = "active"
page.updated_at = now
page_ids_by_slug[slug] = page.id
chunks = list(package.get("chunks") or [])
chunk_pages = 0
imported_vectors = 0
for index, raw in enumerate(chunks):
if not isinstance(raw, dict):
continue
content = str(
raw.get("content")
or raw.get("section_content")
or raw.get("text")
or raw.get("chunk_content")
or ""
).strip()
if not content:
continue
raw_chunk_id = str(raw.get("id") or raw.get("chunk_id") or raw.get("remote_id") or f"{source.id}:{index}")
document_id = str(raw.get("source_document_id") or raw.get("knowledge_id") or raw.get("document_id") or "")
title = str(
raw.get("title")
or raw.get("knowledge_title")
or raw.get("filename")
or raw.get("knowledge_filename")
or f"{source_name} 片段 {index + 1}"
)
heading = str(raw.get("heading") or raw.get("section_title") or "")
summary = content[:240]
page_content = "\n".join(
item
for item in [
f"# {title}",
"",
f"- 来源:{source_name}",
f"- 文档ID:{document_id}" if document_id else "",
f"- 片段ID:{raw_chunk_id}" if raw_chunk_id else "",
f"- 标题层级:{heading}" if heading else "",
"",
content,
]
if item != ""
)
chunk_page = await self._publish_page(
session,
base_id=base["id"],
slug=f"chunks/{source.id}/{_stable_slug_part(raw_chunk_id)}",
title=title,
page_type="chunk",
content=page_content,
summary=summary,
search_terms=[raw_chunk_id, document_id, heading],
source_id=source.id,
import_job_id=job.id,
created_by=created_by,
change_type="chunk_import",
)
page_ids_by_slug[chunk_page.slug] = chunk_page.id
vector_blob = _vector_blob_from_payload(raw)
if vector_blob is not None:
blob, dimensions = vector_blob
session.add(
AssistantKnowledgeVectorRow(
id=str(uuid4()),
base_id=base["id"],
page_id=chunk_page.id,
source_id=source.id,
import_job_id=job.id,
source_vector_id=str(raw.get("vector_id") or raw.get("embedding_id") or raw_chunk_id),
source_chunk_id=raw_chunk_id,
section_index=int(raw.get("section_index") or raw.get("chunk_index") or index),
heading=heading or None,
section_content=content,
content_hash=str(raw.get("content_hash") or _content_hash(content)),
embedding_model=str(raw.get("embedding_model") or raw.get("model") or ""),
embedding_fingerprint=str(raw.get("embedding_fingerprint") or ""),
embedding_dimensions=dimensions,
vector_blob=blob,
created_at=now,
)
)
imported_vectors += 1
chunk_pages += 1
for index, raw in enumerate(list(package.get("vectors") or [])):
if not isinstance(raw, dict):
continue
slug = str(raw.get("wiki_slug") or raw.get("page_slug") or raw.get("slug") or "").strip("/")
page_id = page_ids_by_slug.get(slug)
if not page_id:
continue
content = str(raw.get("section_content") or raw.get("content") or raw.get("text") or "").strip()
if not content:
continue
vector_blob = _vector_blob_from_payload(raw)
if vector_blob is None:
continue
blob, dimensions = vector_blob
raw_chunk_id = str(raw.get("source_chunk_id") or raw.get("chunk_id") or "")
session.add(
AssistantKnowledgeVectorRow(
id=str(uuid4()),
base_id=base["id"],
page_id=page_id,
source_id=source.id,
import_job_id=job.id,
source_vector_id=str(raw.get("id") or raw.get("vector_id") or f"{slug}:{index}"),
source_chunk_id=raw_chunk_id or None,
section_index=int(raw.get("section_index") or index),
heading=str(raw.get("heading") or "") or None,
section_content=content,
content_hash=str(raw.get("content_hash") or _content_hash(content)),
embedding_model=str(raw.get("embedding_model") or raw.get("model") or ""),
embedding_fingerprint=str(raw.get("embedding_fingerprint") or ""),
embedding_dimensions=dimensions,
vector_blob=blob,
created_at=now,
)
)
imported_vectors += 1
counts = {
"wiki_pages": len(pages) + len(entity_rows) + len(relation_rows),
"documents": len(package.get("documents") or []),
"chunks": len(package.get("chunks") or []),
"chunks": len(chunks),
"chunk_pages": chunk_pages,
"vectors": imported_vectors,
"entities": len(raw_entities),
"relations": len(raw_relations),
"graph_nodes": len((package.get("graph") or {}).get("nodes") or []),
@@ -439,8 +1001,24 @@ class AssistantKnowledgeRepository:
contribution = (await session.execute(select(AssistantKnowledgeRelationContributionRow.id).where(AssistantKnowledgeRelationContributionRow.relation_id == relation.id).limit(1))).scalar_one_or_none()
relation.status = "active" if contribution else "deleted"
async def list_pages(self, *, query: str | None = None, page_type: str | None = None, limit: int = 100) -> list[dict[str, Any]]:
stmt = select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.status == "active")
async def list_pages(
self,
*,
base_id: str | None = None,
query: str | None = None,
page_type: str | None = None,
limit: int = 100,
include_chunks: bool = False,
) -> list[dict[str, Any]]:
base = await self.get_base(base_id)
if base is None:
return []
stmt = select(AssistantKnowledgeWikiPageRow).where(
AssistantKnowledgeWikiPageRow.base_id == base["id"],
AssistantKnowledgeWikiPageRow.status == "active",
)
if not include_chunks and not page_type:
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
if query:
needle = f"%{query.strip()}%"
stmt = stmt.where(or_(AssistantKnowledgeWikiPageRow.title.ilike(needle), AssistantKnowledgeWikiPageRow.search_text.ilike(needle)))
@@ -450,15 +1028,178 @@ class AssistantKnowledgeRepository:
async with self._sf() as session:
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
async def get_page(self, slug: str) -> dict[str, Any] | None:
stmt = select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/"), AssistantKnowledgeWikiPageRow.status == "active")
async def search_pages(
self,
*,
base_ids: list[str],
query: str,
limit: int = 8,
include_chunks: bool = True,
embedding_client: Any | None = None,
) -> list[dict[str, Any]]:
resolved_base_ids: list[str] = []
for base_id in list(dict.fromkeys(base_ids)):
base = await self.get_base(base_id)
if base is not None:
resolved_base_ids.append(str(base["id"]))
if not resolved_base_ids:
return []
if embedding_client is not None and query.strip():
try:
vector_results = await self._search_pages_by_vector(
base_ids=resolved_base_ids,
query=query,
limit=limit,
include_chunks=include_chunks,
embedding_client=embedding_client,
)
if vector_results:
return vector_results
except Exception: # noqa: BLE001
logger.warning("Assistant knowledge vector search failed; falling back to keyword search", exc_info=True)
tokens = [token for token in query.strip().lower().split() if token]
needle = f"%{query.strip()}%" if query.strip() else "%"
stmt = select(AssistantKnowledgeWikiPageRow).where(
AssistantKnowledgeWikiPageRow.base_id.in_(resolved_base_ids),
AssistantKnowledgeWikiPageRow.status == "active",
)
if not include_chunks:
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
if query.strip():
stmt = stmt.where(
or_(
AssistantKnowledgeWikiPageRow.title.ilike(needle),
AssistantKnowledgeWikiPageRow.search_text.ilike(needle),
*[
AssistantKnowledgeWikiPageRow.search_text.ilike(f"%{token}%")
for token in tokens[:8]
],
)
)
stmt = stmt.order_by(AssistantKnowledgeWikiPageRow.updated_at.desc()).limit(max(limit * 8, limit))
async with self._sf() as session:
rows = [_dict(row) for row in (await session.execute(stmt)).scalars()]
def score(row: dict[str, Any]) -> float:
text = f"{row.get('title') or ''}\n{row.get('summary') or ''}\n{row.get('search_text') or row.get('content_markdown') or ''}".lower()
if not tokens:
return 0.0
value = 0.0
title = str(row.get("title") or "").lower()
for token in tokens:
if token in title:
value += 3.0
value += min(text.count(token), 8) * 0.25
if row.get("page_type") == "chunk":
value += 0.5
return value
rows.sort(key=score, reverse=True)
results = rows[:limit]
for row in results:
row["score"] = score(row)
return results
async def _search_pages_by_vector(
self,
*,
base_ids: list[str],
query: str,
limit: int,
include_chunks: bool,
embedding_client: Any,
) -> list[dict[str, Any]]:
raw_query_vector = await embedding_client.embed_query(query)
configured_dimensions = getattr(embedding_client, "dimensions", None)
query_vector = normalize_vector(raw_query_vector, dimensions=configured_dimensions)
dimensions = int(query_vector.size)
stmt = (
select(AssistantKnowledgeVectorRow, AssistantKnowledgeWikiPageRow)
.join(
AssistantKnowledgeWikiPageRow,
AssistantKnowledgeWikiPageRow.id == AssistantKnowledgeVectorRow.page_id,
)
.where(
AssistantKnowledgeVectorRow.base_id.in_(base_ids),
AssistantKnowledgeVectorRow.embedding_dimensions == dimensions,
AssistantKnowledgeWikiPageRow.status == "active",
)
.order_by(AssistantKnowledgeVectorRow.created_at.desc())
)
if not include_chunks:
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
async with self._sf() as session:
rows = list((await session.execute(stmt)).all())
if not rows:
return []
tokens = [token for token in query.strip().lower().split() if token]
ranked: list[tuple[float, AssistantKnowledgeVectorRow, AssistantKnowledgeWikiPageRow]] = []
for vector, page in rows:
try:
stored = decode_vector(vector.vector_blob, int(vector.embedding_dimensions))
except ValueError:
continue
score = float(np.dot(query_vector, stored))
if tokens:
haystack = f"{page.title}\n{page.summary}\n{vector.heading or ''}\n{vector.section_content}".lower()
score += min(sum(1 for token in tokens if token in haystack), 4) * 0.02
ranked.append((score, vector, page))
ranked.sort(key=lambda item: item[0], reverse=True)
results: list[dict[str, Any]] = []
seen_pages: set[str] = set()
for score, vector, page in ranked:
if len(results) >= limit:
break
page_data = _dict(page)
page_data["score"] = score
page_data["content_markdown"] = vector.section_content or page_data.get("content_markdown") or ""
page_data["summary"] = page_data.get("summary") or (vector.section_content[:240] if vector.section_content else "")
page_data["retrieval_mode"] = "vector"
page_data["matched_section"] = {
"id": vector.id,
"section_index": int(vector.section_index),
"heading": vector.heading,
"content": vector.section_content,
"score": score,
"source_chunk_id": vector.source_chunk_id,
"embedding_model": vector.embedding_model,
"embedding_fingerprint": vector.embedding_fingerprint,
}
dedupe_key = f"{page.id}:{vector.section_index}:{vector.content_hash}"
if page.id in seen_pages and page.page_type != "chunk":
continue
seen_pages.add(page.id)
seen_pages.add(dedupe_key)
results.append(page_data)
return results
async def get_page(self, slug: str, *, base_id: str | None = None) -> dict[str, Any] | None:
base = await self.get_base(base_id)
if base is None:
return None
stmt = select(AssistantKnowledgeWikiPageRow).where(
AssistantKnowledgeWikiPageRow.base_id == base["id"],
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
AssistantKnowledgeWikiPageRow.status == "active",
)
async with self._sf() as session:
row = (await session.execute(stmt)).scalar_one_or_none()
return _dict(row) if row else None
async def list_revisions(self, slug: str) -> list[dict[str, Any]]:
async def list_revisions(self, slug: str, *, base_id: str | None = None) -> list[dict[str, Any]]:
base = await self.get_base(base_id)
if base is None:
return []
async with self._sf() as session:
page = (await session.execute(select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/")))).scalar_one_or_none()
page = (
await session.execute(
select(AssistantKnowledgeWikiPageRow).where(
AssistantKnowledgeWikiPageRow.base_id == base["id"],
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
)
)
).scalar_one_or_none()
if page is None:
return []
stmt = select(AssistantKnowledgeWikiRevisionRow).where(AssistantKnowledgeWikiRevisionRow.page_id == page.id).order_by(AssistantKnowledgeWikiRevisionRow.revision_no.desc())
@@ -484,10 +1225,27 @@ class AssistantKnowledgeRepository:
)
return {"left": left, "right": right, "diff": diff}
async def rollback(self, *, slug: str, revision_id: str, actor: str | None) -> dict[str, Any] | None:
async def rollback(
self,
*,
slug: str,
revision_id: str,
actor: str | None,
base_id: str | None = None,
) -> dict[str, Any] | None:
base = await self.get_base(base_id)
if base is None:
return None
now = _now()
async with self._sf() as session:
page = (await session.execute(select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/")))).scalar_one_or_none()
page = (
await session.execute(
select(AssistantKnowledgeWikiPageRow).where(
AssistantKnowledgeWikiPageRow.base_id == base["id"],
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
)
)
).scalar_one_or_none()
source = await session.get(AssistantKnowledgeWikiRevisionRow, revision_id)
if page is None or source is None or source.page_id != page.id:
return None
@@ -517,9 +1275,19 @@ class AssistantKnowledgeRepository:
await session.commit()
return data
async def list_sources(self) -> list[dict[str, Any]]:
async def list_sources(self, *, base_id: str | None = None) -> list[dict[str, Any]]:
base = await self.get_base(base_id)
if base is None:
return []
async with self._sf() as session:
stmt = select(AssistantKnowledgeSourceRow).where(AssistantKnowledgeSourceRow.status != "deleted").order_by(AssistantKnowledgeSourceRow.updated_at.desc())
stmt = (
select(AssistantKnowledgeSourceRow)
.where(
AssistantKnowledgeSourceRow.base_id == base["id"],
AssistantKnowledgeSourceRow.status != "deleted",
)
.order_by(AssistantKnowledgeSourceRow.updated_at.desc())
)
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
async def get_source(self, source_id: str) -> dict[str, Any] | None:
@@ -547,9 +1315,17 @@ class AssistantKnowledgeRepository:
result["contributions"] = [_dict(value) for value in contributions]
return result
async def list_jobs(self, limit: int = 100) -> list[dict[str, Any]]:
async def list_jobs(self, limit: int = 100, *, base_id: str | None = None) -> list[dict[str, Any]]:
base = await self.get_base(base_id)
if base is None:
return []
async with self._sf() as session:
stmt = select(AssistantKnowledgeImportJobRow).order_by(AssistantKnowledgeImportJobRow.created_at.desc()).limit(limit)
stmt = (
select(AssistantKnowledgeImportJobRow)
.where(AssistantKnowledgeImportJobRow.base_id == base["id"])
.order_by(AssistantKnowledgeImportJobRow.created_at.desc())
.limit(limit)
)
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
async def get_job(self, job_id: str) -> dict[str, Any] | None:
@@ -565,6 +1341,7 @@ class AssistantKnowledgeRepository:
source.status = "deleted"
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source_id))
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source_id))
await session.execute(delete(AssistantKnowledgeVectorRow).where(AssistantKnowledgeVectorRow.source_id == source_id))
pages = (
await session.execute(
select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.current_revision_id.in_(select(AssistantKnowledgeWikiRevisionRow.id).where(AssistantKnowledgeWikiRevisionRow.source_id == source_id)))
@@ -162,7 +162,7 @@ async def _auto_create_postgres_db(url: str) -> None:
try:
async with maint_engine.connect() as conn:
await conn.execute(text(f'CREATE DATABASE "{db_name}"'))
logger.info("Auto-created PostgreSQL database: %s", db_name)
logger.info("Auto-created PostgreSQL-compatible database: %s", db_name)
finally:
await maint_engine.dispose()
@@ -216,10 +216,10 @@ def _ensure_orm_columns_sync(sync_conn) -> None:
added column are created as well. Table-level ``UniqueConstraint`` objects
that are declared on the model but missing from the table are also created
(as a ``CREATE UNIQUE INDEX`` of the same name, which is what the migrations
emit too, and works on SQLite/MySQL/PostgreSQL alike) — without this a newly
emit too, and works on SQLite/MySQL/PostgreSQL/openGauss alike) — without this a newly
declared uniqueness rule (e.g. ``active_dedupe_key``) would silently be absent
on an existing database, weakening dedupe.
Works on sqlite/mysql/postgres — the column DDL is compiled through the
Works on sqlite/mysql/postgres/gauss — the column DDL is compiled through the
active dialect, so portable types (``PortableJSON``/``PortableLongText``/
``BeijingDateTime``) resolve to the right backend type. Everything is
per-column best-effort: a failure is logged and startup continues.
@@ -341,13 +341,13 @@ async def init_engine(
"""Create the async engine and session factory, then auto-create tables.
Args:
backend: "memory", "sqlite", "postgres", or "mysql".
url: SQLAlchemy async URL (for sqlite/postgres/mysql).
backend: "memory", "sqlite", "postgres", "gauss", or "mysql".
url: SQLAlchemy async URL (for sqlite/postgres/gauss/mysql).
echo: Echo SQL to log.
pool_size: Base connection pool size (postgres/mysql).
max_overflow: Extra connections allowed beyond pool_size under load (postgres/mysql).
pool_timeout: Seconds to wait for a free pooled connection before erroring (postgres/mysql).
pool_recycle: Recycle connections after N seconds (postgres/mysql). Prevents stale
pool_size: Base connection pool size (postgres/gauss/mysql).
max_overflow: Extra connections allowed beyond pool_size under load (postgres/gauss/mysql).
pool_timeout: Seconds to wait for a free pooled connection before erroring (postgres/gauss/mysql).
pool_recycle: Recycle connections after N seconds (postgres/gauss/mysql). Prevents stale
connections dropped by firewalls / MySQL wait_timeout / pgbouncer.
sqlite_dir: Directory to create for SQLite (ensured to exist).
"""
@@ -365,11 +365,21 @@ async def init_engine(
except RuntimeError:
_engine_loop = None
if backend == "postgres":
try:
import asyncpg # noqa: F401
except ImportError:
raise ImportError("database.backend is set to 'postgres' but asyncpg is not installed.\nInstall it with:\n uv sync --extra postgres\nOr switch to backend: sqlite in config.yaml for single-node deployment.") from None
if backend in {"postgres", "gauss"}:
try:
import asyncpg # noqa: F401
except ImportError:
extra = "gauss" if backend == "gauss" else "postgres"
raise ImportError(
f"database.backend is set to {backend!r} but asyncpg is not installed.\n"
f"Install it with:\n uv sync --extra {extra}\n"
"Or switch to backend: sqlite in config.yaml for single-node deployment."
) from None
if backend == "gauss" and find_spec("opengauss_sqlalchemy") is None:
raise ImportError(
"database.backend is set to 'gauss' but the openGauss SQLAlchemy dialect "
"is not installed.\nInstall it with:\n uv sync --extra gauss"
)
if backend == "mysql":
_add_runtime_vendor_path()
driver = _mysql_driver_name(url)
@@ -409,7 +419,7 @@ async def init_engine(
cursor.execute("PRAGMA foreign_keys=ON;")
finally:
cursor.close()
elif backend == "postgres":
elif backend in {"postgres", "gauss"}:
_engine = create_async_engine(
url,
echo=echo,
@@ -454,7 +464,7 @@ async def init_engine(
async with _engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
except Exception as exc:
if backend == "postgres" and "does not exist" in str(exc):
if backend in {"postgres", "gauss"} and "does not exist" in str(exc).lower():
# Database not yet created — attempt to auto-create it, then retry.
await _auto_create_postgres_db(url)
# Rebuild engine against the now-existing database
@@ -497,7 +507,7 @@ async def init_engine(
# cannot do on its own. Failures don't abort startup — see the helper.
# await _run_alembic_upgrade(url)
logger.info("Persistence engine initialized: backend=%s", backend)
logger.info("Persistence engine initialized: backend=%s dialect=%s", backend, _engine.dialect.name)
async def init_engine_from_config(config) -> None:
@@ -3,6 +3,7 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from collections.abc import AsyncIterator
from typing import Any
@@ -35,6 +36,23 @@ class LlmWikiIndexStore(ABC):
@abstractmethod
async def load_vector_snapshot(self, mapping_id: str, fingerprint: str) -> dict[str, Any]: ...
async def iter_vector_snapshot_rows(
self,
mapping_id: str,
fingerprint: str,
*,
batch_size: int = 1000,
) -> AsyncIterator[dict[str, Any]]:
"""Iterate vector rows for streaming export.
Implementations with a database backend should override this method so
very large vector packages do not need to be materialized in memory.
"""
snapshot = await self.load_vector_snapshot(mapping_id, fingerprint)
for row in snapshot.get("rows") or []:
if isinstance(row, dict):
yield row
@abstractmethod
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]: ...
@@ -2,6 +2,7 @@
from __future__ import annotations
from collections.abc import AsyncIterator
from datetime import UTC, datetime, timedelta
from typing import Any
from uuid import uuid4
@@ -181,6 +182,41 @@ class SqlLlmWikiIndexStore(LlmWikiIndexStore):
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
return {"revision": int(state.index_revision if state else 0), "fingerprint": fingerprint, "rows": rows}
async def iter_vector_snapshot_rows(
self,
mapping_id: str,
fingerprint: str,
*,
batch_size: int = 1000,
) -> AsyncIterator[dict[str, Any]]:
batch = max(1, min(int(batch_size), 5000))
offset = 0
while True:
stmt = (
select(LlmWikiWikiVectorRow, LlmWikiWikiPageRow)
.join(LlmWikiWikiPageRow, LlmWikiWikiPageRow.id == LlmWikiWikiVectorRow.page_id)
.where(
LlmWikiWikiVectorRow.knowledge_base_mapping_id == mapping_id,
LlmWikiWikiVectorRow.embedding_fingerprint == fingerprint,
LlmWikiWikiPageRow.index_status == "ready",
LlmWikiWikiPageRow.is_remote_deleted.is_(False),
LlmWikiWikiPageRow.status.in_(["published", "draft"]),
)
.order_by(LlmWikiWikiVectorRow.page_id, LlmWikiWikiVectorRow.section_index)
.limit(batch)
.offset(offset)
)
async with self._sf() as session:
pairs = list((await session.execute(stmt)).all())
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
if not rows:
break
for row in rows:
yield row
if len(rows) < batch:
break
offset += len(rows)
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]:
async with self._sf() as session:
state = await session.get(LlmWikiWikiSyncStateRow, mapping_id)
@@ -0,0 +1,25 @@
"""Add assistant knowledge vector table.
Revision ID: 20260902_01
Revises: 20260901_01
Create Date: 2026-09-02
"""
from __future__ import annotations
from alembic import op
from deerflow.persistence.assistant_knowledge.model import AssistantKnowledgeVectorRow
revision = "20260902_01"
down_revision = "20260901_01"
branch_labels = None
depends_on = None
def upgrade() -> None:
AssistantKnowledgeVectorRow.__table__.create(op.get_bind(), checkfirst=True)
def downgrade() -> None:
AssistantKnowledgeVectorRow.__table__.drop(op.get_bind(), checkfirst=True)
@@ -43,6 +43,7 @@ from deerflow.persistence.assistant_knowledge.model import (
AssistantKnowledgeRelationContributionRow,
AssistantKnowledgeRelationRow,
AssistantKnowledgeSourceRow,
AssistantKnowledgeVectorRow,
AssistantKnowledgeWikiPageRow,
AssistantKnowledgeWikiRevisionRow,
)
@@ -3,7 +3,7 @@
from __future__ import annotations
import json
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta, timezone
from typing import Any
from sqlalchemy import JSON, DateTime, Text
@@ -14,6 +14,12 @@ from sqlalchemy.types import TypeDecorator
# fixed offset is exact and avoids depending on the `tzdata` package (which the
# stdlib `zoneinfo` needs on Windows / minimal containers).
BEIJING_TZ = timezone(timedelta(hours=8))
_POSTGRES_FAMILY_DIALECTS = frozenset({"postgresql", "opengauss"})
def _is_postgres_family(dialect) -> bool:
"""Return whether *dialect* preserves timezone-aware timestamps natively."""
return dialect.name in _POSTGRES_FAMILY_DIALECTS
class PortableJSON(TypeDecorator):
@@ -87,17 +93,17 @@ class BeijingDateTime(TypeDecorator):
def process_bind_param(self, value: Any, dialect) -> Any:
if not isinstance(value, datetime):
return value
if dialect.name == "postgresql":
if _is_postgres_family(dialect):
return value
if value.tzinfo is None:
# Naive values follow the codebase convention of being UTC.
value = value.replace(tzinfo=timezone.utc)
value = value.replace(tzinfo=UTC)
return value.astimezone(BEIJING_TZ).replace(tzinfo=None)
def process_result_value(self, value: Any, dialect) -> Any:
if not isinstance(value, datetime):
return value
if dialect.name == "postgresql":
if _is_postgres_family(dialect):
return value
if value.tzinfo is None:
# Stored as naive Beijing wall-clock; re-attach the zone.
@@ -52,8 +52,12 @@ postgres = [
"psycopg[binary]>=3.3.3",
"psycopg-pool>=3.3.0",
]
mysql = ["asyncmy>=0.2.10"]
pymupdf = ["pymupdf4llm>=0.0.17"]
mysql = ["asyncmy>=0.2.10"]
gauss = [
"asyncpg>=0.29",
"opengauss-sqlalchemy>=2.4,<3",
]
pymupdf = ["pymupdf4llm>=0.0.17"]
[build-system]
requires = ["hatchling"]
@@ -30,6 +30,7 @@ dependencies = [
[project.optional-dependencies]
postgres = ["deerflow-harness[postgres]"]
mysql = ["deerflow-harness[mysql]"]
gauss = ["deerflow-harness[gauss]"]
[dependency-groups]
dev = [
@@ -22,6 +22,18 @@ class _FakeThreadStore:
return self.rows[offset : offset + limit]
class _FakeAgentStore:
async def list_by_ids(self, agent_ids: list[str], user_id: str) -> list[dict]:
assert user_id == "owner"
agents = {
"analysis-agent": {
"id": "analysis-agent",
"name": "舆情分析智能体",
}
}
return [agents[agent_id] for agent_id in agent_ids if agent_id in agents]
@pytest.mark.asyncio
async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: Path, monkeypatch):
thread_one = tmp_path / "thread-one" / "outputs"
@@ -79,6 +91,7 @@ async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: P
assert [item["name"] for item in items] == ["证据.zip", "结论.md", "数据汇总.xlsx", "研判报告.docx"]
assert [item["file_kind"] for item in items] == ["archive", "text_data", "spreadsheet", "document"]
assert [item["source_type"] for item in items] == ["general_qa", "general_qa", "agent", "agent"]
assert items[-1]["thread_title"] == "战场环境综合研判"
assert items[-1]["agent_id"] == "analysis-agent"
assert all(item["name"] not in {"配图.png", "录音.mp3"} for item in items)
@@ -88,9 +101,12 @@ async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: P
@pytest.mark.asyncio
async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Path, monkeypatch):
output_dir = tmp_path / "thread-one" / "outputs"
agent_output_dir = tmp_path / "thread-agent" / "outputs"
output_dir.mkdir(parents=True)
agent_output_dir.mkdir(parents=True)
(output_dir / "任务报告.md").write_text("report", encoding="utf-8")
(output_dir / "任务数据.csv").write_text("a,b", encoding="utf-8")
(agent_output_dir / "智能体结论.md").write_text("agent report", encoding="utf-8")
store = _FakeThreadStore(
[
@@ -98,7 +114,12 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
"thread_id": "thread-one",
"display_name": "任务研判对话",
"metadata": {},
}
},
{
"thread_id": "thread-agent",
"display_name": "智能体研判对话",
"metadata": {"agent_id": "analysis-agent"},
},
]
)
@@ -113,6 +134,7 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
monkeypatch.setattr(artifacts, "aresolve_path_user_id", _resolve_owner)
monkeypatch.setattr(artifacts, "get_effective_user_id", lambda: "owner")
monkeypatch.setattr(deps, "get_thread_store", lambda _request: store)
monkeypatch.setattr(deps, "get_agent_store", lambda _request: _FakeAgentStore())
artifacts._artifact_library_cache.clear()
raw_endpoint = artifacts.list_artifact_library.__wrapped__
@@ -120,6 +142,7 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
request=SimpleNamespace(),
query="研判对话",
file_kind="text_data",
source_type="general_qa",
offset=0,
limit=1,
refresh=True,
@@ -134,9 +157,24 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
request=SimpleNamespace(),
query="任务数据",
file_kind="spreadsheet",
source_type="all",
offset=0,
limit=50,
refresh=False,
)
assert spreadsheet_response.total == 1
assert spreadsheet_response.items[0].name == "任务数据.csv"
agent_response = await raw_endpoint(
request=SimpleNamespace(),
query=None,
file_kind="all",
source_type="agent",
offset=0,
limit=50,
refresh=False,
)
assert agent_response.total == 1
assert agent_response.items[0].source_type == "agent"
assert agent_response.items[0].agent_name == "舆情分析智能体"
assert agent_response.items[0].name == "智能体结论.md"
@@ -11,6 +11,13 @@ from deerflow.persistence.assistant_knowledge.model import (
from deerflow.persistence.base import Base
class FakeEmbeddingClient:
dimensions = 3
async def embed_query(self, _query: str) -> list[float]:
return [1.0, 0.0, 0.0]
@pytest_asyncio.fixture
async def repository():
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
@@ -94,3 +101,93 @@ async def test_delete_source_keeps_entity_until_last_contribution_is_removed(
async with repository._sf() as session:
entity = (await session.execute(select(AssistantKnowledgeEntityRow))).scalar_one()
assert entity.status == "deleted"
@pytest.mark.asyncio
async def test_custom_base_import_is_also_mirrored_to_global_base(
repository: AssistantKnowledgeRepository,
) -> None:
global_base = await repository.initialize(created_by="admin")
custom_base = await repository.create_base(
created_by="admin",
name="专题助手库",
description="只承接某个专题的 WeKnora 导出数据",
)
package = {
"source": {"type": "weknora"},
"wiki_pages": [{"slug": "wiki/topic", "title": "专题 Wiki", "content": "王五负责统筹。"}],
"entities": [{"local_id": "e1", "type": "person", "name": "王五"}],
"relations": [],
}
job = await repository.import_package(
package=package,
source_type="weknora",
source_key="kb-topic",
source_name="专题普通库",
trigger="manual_from_weknora",
created_by="admin",
mapping_id="kb-topic",
base_id=custom_base["id"],
)
assert job["base_id"] == custom_base["id"]
assert job["mirrored_global_base_id"] == global_base["id"]
assert len(await repository.list_pages(base_id=custom_base["id"], query="王五")) == 2
assert len(await repository.list_pages(base_id=global_base["id"], query="王五")) == 2
bases = await repository.list_bases()
assert [base["id"] for base in bases[:2]] == [global_base["id"], custom_base["id"]]
@pytest.mark.asyncio
async def test_imported_vectors_are_reused_for_assistant_search(
repository: AssistantKnowledgeRepository,
) -> None:
base = await repository.initialize(created_by="admin")
job = await repository.import_package(
package={
"source": {"type": "weknora"},
"wiki_pages": [{"slug": "wiki/vector", "title": "向量页", "content": "页面正文"}],
"vectors": [
{
"wiki_slug": "wiki/vector",
"section_index": 0,
"section_content": "张三负责向量化检索。",
"embedding_model": "bge-embedding-m3",
"embedding_dimensions": 3,
"vector": [1.0, 0.0, 0.0],
},
{
"wiki_slug": "wiki/vector",
"section_index": 1,
"section_content": "无关片段。",
"embedding_model": "bge-embedding-m3",
"embedding_dimensions": 3,
"vector": [0.0, 1.0, 0.0],
},
],
},
source_type="weknora",
source_key="kb-vector",
source_name="向量普通库",
trigger="manual_package_upload",
created_by="admin",
)
assert job["counts"]["vectors"] == 2
results = await repository.search_pages(
base_ids=[base["id"]],
query="这个查询不靠关键词",
embedding_client=FakeEmbeddingClient(),
)
assert results[0]["retrieval_mode"] == "vector"
assert results[0]["matched_section"]["embedding_model"] == "bge-embedding-m3"
assert results[0]["content_markdown"] == "张三负责向量化检索。"
sources = await repository.list_sources()
assert await repository.delete_source(sources[0]["id"])
assert await repository.search_pages(
base_ids=[base["id"]],
query="这个查询不靠关键词",
embedding_client=FakeEmbeddingClient(),
) == []
@@ -0,0 +1,81 @@
"""Regression coverage for the openGauss/GaussDB persistence backend."""
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from pydantic import ValidationError
from sqlalchemy.engine import make_url
from sqlalchemy.schema import CreateIndex, CreateTable
from deerflow.config.database_config import DatabaseConfig
from deerflow.persistence.types import BeijingDateTime
@pytest.mark.parametrize(
("configured", "expected"),
[
(
"opengauss://user:pass@127.0.0.1:26000/deerflow",
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
),
(
"postgresql://user:pass@127.0.0.1:26000/deerflow?ssl=require",
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow?ssl=require",
),
(
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
),
],
)
def test_gauss_urls_always_select_the_opengauss_async_dialect(
configured: str,
expected: str,
) -> None:
config = DatabaseConfig(backend="gauss", gauss_url=configured)
assert config.app_sqlalchemy_url == expected
def test_gauss_backend_requires_a_supported_nonempty_url() -> None:
with pytest.raises(ValueError, match="gauss_url is required"):
_ = DatabaseConfig(backend="gauss").app_sqlalchemy_url
with pytest.raises(ValueError, match="must use opengauss"):
_ = DatabaseConfig(
backend="gauss",
gauss_url="mysql://user:pass@127.0.0.1/deerflow",
).app_sqlalchemy_url
with pytest.raises(ValidationError):
DatabaseConfig(backend="unknown")
def test_gauss_timestamps_keep_timezone_information() -> None:
dialect = type("GaussDialect", (), {"name": "opengauss"})()
value = datetime(2026, 9, 2, 8, 30, tzinfo=UTC)
column_type = BeijingDateTime()
assert column_type.process_bind_param(value, dialect) is value
assert column_type.process_result_value(value, dialect) is value
def test_official_async_dialect_compiles_the_complete_orm_schema() -> None:
"""Every registered table/index must compile before startup touches GaussDB."""
pytest.importorskip("opengauss_sqlalchemy")
import deerflow.persistence.models # noqa: F401 -- register ORM metadata
from deerflow.persistence.base import Base
dialect_cls = make_url("opengauss+asyncpg://user:pass@localhost/db").get_dialect(
_is_async=True
)
dialect = dialect_cls()
assert dialect.__class__.__module__.startswith("opengauss_sqlalchemy")
assert Base.metadata.sorted_tables
for table in Base.metadata.sorted_tables:
assert str(CreateTable(table).compile(dialect=dialect)).startswith("\nCREATE TABLE")
for index in table.indexes:
assert str(CreateIndex(index).compile(dialect=dialect)).startswith("CREATE")
@@ -66,6 +66,9 @@ async def test_sql_store_atomically_replaces_vectors_and_marks_missing_deleted(t
assert store.write_batch_size == 1
assert len(snapshot["rows"]) == 3
assert decode_vector(snapshot["rows"][0]["vector_blob"], 2).tolist() == pytest.approx([0.6, 0.8])
streamed = [row async for row in store.iter_vector_snapshot_rows("mapping-1", "fp", batch_size=2)]
assert [row["section_index"] for row in streamed] == [0, 1, 2]
assert streamed[0]["page"]["slug"] == "people/alice"
page_statuses = await store.list_page_index_status("mapping-1")
assert len(page_statuses) == 1
assert page_statuses[0] == {