feat: improve assistant knowledge import and database compatibility
This commit is contained in:
1 parent
fbd8dec48f
commit
6b724bd00d
51 files changed
+4393
-1021
No files matched your search
@@ -88,6 +88,8 @@ skills/
|
||||
hermes-agent-main/
|
||||
.gitignore
|
||||
offline-backend-20260512/backend/tests/*
|
||||
# Keep the openGauss URL/dialect/schema compatibility regression coverage versioned.
|
||||
!offline-backend-20260512/backend/tests/test_gauss_database.py
|
||||
!offline-backend-20260512/backend/tests/test_llmwiki_weknora.py
|
||||
!offline-backend-20260512/backend/tests/test_health_auth_bypass.py
|
||||
# Keep formal Word OOXML and embedded-font regression coverage versioned.
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
# 技能归纳、WeKnora 普通知识库与助手知识库:实现设计
|
||||
|
||||
本文档用于指导大模型或开发者实现知识库体系改造。整体包含两条线:一条是“将一个或多个技能归纳到一个或多个目标知识库”;另一条是新增一个自研的全局助手知识库,用于承接 WeKnora 已处理、已向量化后的 Wiki、实体和关系数据,也支持技能和管理员本地数据直接导入,并做统一实体对齐、Wiki 展示和版本管理。
|
||||
本文档用于指导大模型或开发者实现知识库体系改造。整体包含两条线:一条是“将一个或多个技能归纳到一个 WeKnora Wiki 普通知识库”;另一条是新增自研助手知识库体系。助手知识库像普通知识库一样可以创建多个库,同时系统内置一个全局大库,用于统计全库数据、承接所有 WeKnora 导入沉淀、做全局实体对齐、Wiki 展示和导入审计。
|
||||
|
||||
本文档中,目标知识库既可以是 WeKnora 普通知识库,也可以是自研助手知识库。普通知识库负责向量化、检索、Wiki 和图谱;助手知识库负责全局知识梳理、实体对齐、Wiki 展示、搜索和版本管理。
|
||||
本文档中,技能归纳目标只能是 WeKnora Wiki 普通知识库;助手知识库不接收技能直接归纳,也不接收任意本地文件。普通知识库负责向量化、检索、Wiki 和图谱;助手知识库负责自研存储、WeKnora 导入、实体对齐、Wiki 展示、搜索和导入审计。无论 WeKnora 数据导入到哪个助手知识库,最终都必须沉淀一份到系统全局大库。
|
||||
|
||||
> 最终约束:如果本文历史章节与本节冲突,以本节和“2. 当前边界”中的约束为准。
|
||||
|
||||
## 1. 背景与目标
|
||||
|
||||
@@ -14,57 +16,59 @@
|
||||
|
||||
本次要新增一条管理员工作流:
|
||||
|
||||
管理员在“技能管理”页面选择多个技能,再选择一个或多个目标知识库,点击归纳。系统将每个技能作为独立来源读取、解析、抽取和归纳,把结果写入对应知识库,并在技能列表和技能详情中展示“该技能已被哪些知识库归纳”。
|
||||
管理员在“技能管理”页面选择多个技能,再选择一个目标 WeKnora Wiki 普通知识库,点击归纳。系统将每个技能作为独立来源读取、解析、抽取和归纳,把结果写入对应 WeKnora 知识库,并在技能列表和技能详情中展示“该技能已被哪些知识库归纳”。如果需要归纳到多个普通知识库,通过多次操作完成。
|
||||
|
||||
用户已确认的业务约束:
|
||||
|
||||
- 一次操作可以选择多个技能,也可以选择多个目标知识库;系统按 `技能 x 目标知识库` 生成处理矩阵。
|
||||
- 一次操作可以选择多个技能,但只能选择一个目标 WeKnora Wiki 普通知识库。
|
||||
- 同一个技能可以在不同时间归纳到多个知识库,所以最终关系是技能与知识库多对多。
|
||||
- V1 支持归纳到多种目标:Wiki WeKnora、非 Wiki WeKnora、FAQ 知识库、系统会话存档知识库、只读知识库,以及自研助手知识库。
|
||||
- 多知识库归纳通过多次操作完成;每次操作形成一批 `技能 x 单目标知识库` 任务。
|
||||
- V1 只支持归纳到可写 WeKnora Wiki 普通知识库;不支持非 Wiki、FAQ、系统会话存档、只读知识库作为技能归纳目标。
|
||||
- 技能归纳时只抽取数据、业务说明、实体、概念、对象、事件、关系等知识;Python、TypeScript、Shell 等代码和脚本类文件完全排除,不读取注释、不抽取函数、不上传、不进入 prompt。
|
||||
- 技能里的 Markdown、业务数据文件、表格、PDF、Office、图片等要纳入处理范围。
|
||||
- 需要同步三类内容:原始资料、归纳后的 Wiki、实体与关系。
|
||||
- 关系来源包括 Markdown 自然语言描述、目录和文件引用、模型自动推断。
|
||||
- 允许模型推断关系,同时前端需要提供逐条实体/关系预览审核能力。
|
||||
- 支持手动重新归纳,也支持自动文件监听、定时后台同步和技能变更后的自动远程写入。
|
||||
- 允许模型推断关系,但前端不提供逐条实体/关系预览审核。
|
||||
- 支持管理员手动点击“重新归纳”;不新增自动文件监听、定时后台同步或技能变更自动远程写入。
|
||||
- 解除关联时弹窗让用户选择:保留知识库内容,或删除该技能贡献的内容。
|
||||
- 权限仅管理员。
|
||||
- 总技能数约 200 个,需要支持批量进度、后台执行、失败重试。
|
||||
- 同一目标知识库内,不同技能抽取出的同一实体需要合并。
|
||||
- 重归纳时先生成新版本,成功后再替换旧版本;失败时旧版本继续可用。
|
||||
- 新增知识库类型区分:普通知识库是 WeKnora,助手知识库是自研存储。
|
||||
- 助手知识库只有一个全局大库,建议命名为“知识梳理”,所有用户可见。
|
||||
- 助手知识库支持创建多个库;其中有且只有一个系统全局大库,建议命名为“知识梳理总库”,所有用户可见。
|
||||
- 助手知识库只提供 Wiki 页面展示、搜索和数据导入;不落到 WeKnora。
|
||||
- 助手知识库数据来源包括 WeKnora 导出、技能直接归纳、管理员本地文件导入。
|
||||
- WeKnora 已向量化和处理完成的 Wiki、文档、实体、关系、图谱数据默认沉淀到助手知识库。
|
||||
- 普通知识库需要提供导出或“导入到助手知识库”的功能,把 WeKnora 里已经处理好的数据导入助手知识库;导出数据需要包含 Wiki、文档、chunk、实体、关系、图谱。
|
||||
- 即使没有配置 WeKnora,也要默认展示助手知识库的创建或初始化入口。
|
||||
- 助手知识库数据来源只能是 WeKnora 导出包或从某个 WeKnora 知识库触发导入。
|
||||
- WeKnora 已向量化和处理完成的 Wiki、文档、实体、关系、图谱数据默认沉淀到助手知识库;如果导入的是某个自建助手库,也必须同步沉淀到全局大库。
|
||||
- 普通知识库需要提供导出或“导入到助手知识库”的功能,把 WeKnora 里已经处理好的数据导入指定助手知识库;导出数据需要包含 Wiki、文档、chunk、实体、关系、图谱。
|
||||
- 即使没有配置 WeKnora 地址,也要默认展示助手知识库列表和创建入口,而不是展示空的普通知识库页面。
|
||||
- 管理员可切换知识库产品版本:`v1` / `v2`,默认 `v1`。
|
||||
- 助手知识库普通用户只读,管理员可导入、编辑、删除、查看版本和回滚 Wiki 页面。
|
||||
|
||||
## 2. 必须支持的扩展能力
|
||||
## 2. 当前边界
|
||||
|
||||
以下能力原先按非目标处理,最新需求已确认全部要支持。实现时必须把它们作为正式目标,并加上必要的权限、状态和审计护栏。
|
||||
以下能力明确不做,开发实现和验收必须按这些边界收敛:
|
||||
|
||||
- 支持一次选择多个目标知识库。一个归纳任务可包含多个技能和多个目标,后端按矩阵拆分 item。
|
||||
- 支持非 Wiki 知识库、FAQ 知识库、系统会话存档知识库、只读知识库作为目标。不同目标使用不同写入 adapter;只读目标使用本地 sidecar 投影和待推送状态。
|
||||
- 支持直接写 WeKnora 的 Neo4j 或底层图数据库,但必须通过受控 graph writer、显式配置开关、事务和审计,不允许 LLM 生成任意 Cypher 直接执行。
|
||||
- 支持自动文件监听、定时后台同步、技能变更自动远程写入。自动写入仍必须走 staged 版本和失败保留旧版本机制。
|
||||
- 前端提供逐条实体/关系预览审核,管理员可批准、拒绝、编辑、批量处理。
|
||||
- 前端和 API 暴露远端 WeKnora ID,并可作为跨系统业务 key 使用;本地仍保留 surrogate id 以保证数据库关系稳定。
|
||||
- 支持技能直接归纳到助手知识库,不必先经过 WeKnora。
|
||||
- 支持管理员向助手知识库导入任意本地文件,导入前必须做文件类型、安全扫描和敏感信息阻断。
|
||||
- 助手知识库 V1 支持 Wiki 页面版本管理、版本对比、发布、回滚和导入审计。
|
||||
- 不支持一次选择多个目标知识库。多知识库归纳通过多次操作完成。
|
||||
- 不支持非 Wiki 知识库、FAQ 知识库、系统会话存档知识库、只读知识库作为技能归纳目标。
|
||||
- 不直接写 WeKnora 的 Neo4j 或底层图数据库。
|
||||
- 不新增自动文件监听、定时后台同步或技能变更自动远程写入。
|
||||
- 不在前端提供逐条实体/关系预览审核。
|
||||
- 不把远端 WeKnora 内部 ID 暴露为前端业务主键。
|
||||
- 不支持技能直接归纳到助手知识库。技能先进入 WeKnora 普通知识库,再由 WeKnora 导出沉淀到助手知识库。
|
||||
- 不支持管理员向助手知识库导入任意本地文件;手动导入也必须是 WeKnora 导出包或从某个 WeKnora 知识库触发。
|
||||
- 助手知识库 V1 不做页面版本管理,只保留当前合并后的 Wiki 状态和导入审计记录。
|
||||
|
||||
## 3. 术语
|
||||
|
||||
| 术语 | 含义 |
|
||||
| --- | --- |
|
||||
| 技能 | DeerFlow 技能目录,通常包含 `SKILL.md`、代码、配置、示例、文档、数据文件等。 |
|
||||
| 目标知识库 | 管理员选择的一个或多个知识库,可以是 WeKnora 普通知识库,也可以是自研助手知识库。 |
|
||||
| 目标知识库 | 管理员选择的一个 WeKnora Wiki 普通知识库。助手知识库不是技能归纳目标。 |
|
||||
| 普通知识库 | WeKnora 知识库,负责向量化、Wiki、文档检索和图谱能力。 |
|
||||
| 助手知识库 | DeerFlow 自研知识库,只维护一个全局大库,用于 Wiki 展示、搜索、WeKnora 导入、技能直接导入、本地文件导入和实体对齐。 |
|
||||
| 知识梳理底库 | 助手知识库的默认全局库名,所有用户可见,普通用户只读。 |
|
||||
| 助手知识库 | DeerFlow 自研知识库类型,支持创建多个库,用于 Wiki 展示、搜索、WeKnora 导入和实体对齐。 |
|
||||
| 知识梳理总库 | 助手知识库体系中的系统全局大库,有且只有一个,自动统计和沉淀所有助手库与普通知识库导入数据,做全局实体对齐。 |
|
||||
| 自建助手库 | 管理员创建的普通助手知识库,类似普通知识库列表中的一个库,可导入 WeKnora 普通知识库导出数据。 |
|
||||
| 产品版本 | 知识库产品形态开关。`v1` 默认为普通知识库体验,`v2` 默认为助手知识库体验。 |
|
||||
| WeKnora 导出包 | 从普通知识库导出的已处理知识集合,包含 Wiki 页面、文档、chunk、实体、关系、图谱等。 |
|
||||
| 绑定 | 一个技能与一个目标知识库之间的持久关联。`skill_name + target_type + target_id` 唯一。 |
|
||||
@@ -81,37 +85,35 @@
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[管理员选择多个技能] --> B[选择一个或多个目标知识库]
|
||||
B --> C[创建归纳任务矩阵]
|
||||
A[管理员选择多个技能] --> B[选择一个 WeKnora Wiki 目标知识库]
|
||||
B --> C[创建归纳任务]
|
||||
C --> D[逐技能扫描与解析]
|
||||
D --> E[生成技能快照]
|
||||
E --> F[LLM 归纳 Wiki 与关系抽取]
|
||||
F --> R[前端实体/关系预览审核]
|
||||
R --> G[按目标知识库合并实体/关系]
|
||||
G --> H[写入 WeKnora 或本地助手库]
|
||||
G --> I[写入 Wiki/FAQ/会话存档/sidecar]
|
||||
G --> J[写入 WeKnora GraphRAG 或 Neo4j]
|
||||
F --> G[按目标 WeKnora Wiki 合并实体/关系]
|
||||
G --> H[写入 WeKnora 原始资料与 Wiki]
|
||||
G --> I[写入关系投影文档]
|
||||
H --> K[校验远端写入]
|
||||
I --> K
|
||||
J --> K
|
||||
K --> L[本地原子激活新版本]
|
||||
L --> M[技能页展示已归纳知识库]
|
||||
K --> N[WeKnora 导出已处理数据]
|
||||
N --> O[沉淀到助手知识库]
|
||||
O --> P[全局实体对齐]
|
||||
P --> Q[知识梳理 Wiki 展示与搜索]
|
||||
N --> O[导入指定助手知识库]
|
||||
O --> P[同步沉淀到知识梳理总库]
|
||||
P --> Q[全局实体对齐]
|
||||
Q --> S[总库 Wiki 展示与搜索]
|
||||
```
|
||||
|
||||
关键原则:
|
||||
|
||||
- DeerFlow 本地数据库保存权威绑定关系、版本状态、来源贡献和远端 artifact 映射。
|
||||
- WeKnora 保存可检索知识内容,包括原文、Wiki 页面、FAQ、会话存档、关系投影文档和图谱。
|
||||
- 对 WeKnora 图谱能力采取能力探测和分层写入:优先使用 WeKnora API;需要底层图写入时,使用受控 Neo4j graph writer。
|
||||
- 不直接写 WeKnora 底层图数据库;关系以 WeKnora API 能力和关系投影文档承接。
|
||||
- 重归纳使用 staged artifact,远端写入和校验成功后,本地才激活新版本。
|
||||
- 删除或解除关联只影响该技能在目标知识库中的贡献,不误删其他技能共享的实体和关系。
|
||||
- 助手知识库只有一个全局库,可以接收技能直接归纳、WeKnora 导出和管理员本地文件导入。
|
||||
- WeKnora 的已处理结果默认进入助手知识库,由助手底库做全局实体对齐、Wiki 展示、搜索和版本管理。
|
||||
- 未配置 WeKnora 时,仍展示助手知识库创建或初始化能力;与 WeKnora 相关的归纳和导入动作置灰或隐藏。
|
||||
- 助手知识库可以创建多个库;系统全局大库只有一个,负责汇总全库数据和做跨库实体对齐。
|
||||
- WeKnora 的已处理结果可以导入指定助手知识库;无论是否指定自建助手库,都必须同步沉淀到知识梳理总库。
|
||||
- 未配置 WeKnora 地址时,默认进入助手知识库页面,展示助手知识库列表、创建入口和知识梳理总库;与 WeKnora 相关的导出导入动作置灰或隐藏。
|
||||
|
||||
## 5. 现有代码落点
|
||||
|
||||
@@ -193,6 +195,14 @@ items = selected_skills x selected_targets
|
||||
|
||||
每个 item 对应一个 `binding`,一个技能归纳到三个目标知识库,就生成三个 binding。父 job 负责展示整体进度,item 独立成功、失败、取消和重试。
|
||||
|
||||
当目标是自建助手库时,item 写入顺序是:
|
||||
|
||||
```text
|
||||
写入目标助手库 -> 生成目标库 Wiki/实体/关系 -> 创建总库沉淀任务 -> 总库实体对齐和统计
|
||||
```
|
||||
|
||||
总库沉淀失败不能让目标助手库写入结果消失,但 item 必须标记为 `partial_failed`,并允许单独重试“沉淀到总库”。
|
||||
|
||||
目标知识库需要通过 adapter 写入:
|
||||
|
||||
| 目标类型 | 写入策略 | Wiki | 关系/图谱 |
|
||||
@@ -202,7 +212,7 @@ items = selected_skills x selected_targets
|
||||
| WeKnora FAQ 知识库 | 把归纳结果转换为 FAQ 条目、来源文档和关系摘要 | 不写或只写 sidecar Wiki | 关系作为 FAQ metadata、projection 文档和图谱边 |
|
||||
| 系统会话存档知识库 | 把技能归纳结果作为系统会话归档条目写入 | sidecar Wiki 展示 | 关系作为归档 metadata 和图谱边 |
|
||||
| 只读 WeKnora 知识库 | 不修改远端原库;写本地 sidecar artifact,并生成待推送状态 | sidecar Wiki 展示 | sidecar graph + 待推送 Neo4j plan |
|
||||
| 助手知识库 | 写入自研底库、实体对齐、生成 Wiki 页面版本 | 写自研 Wiki | 写自研实体/关系表 |
|
||||
| 助手知识库 | 写入指定助手库、实体对齐、生成 Wiki 页面版本;同时异步沉淀到知识梳理总库 | 写自研 Wiki | 写自研实体/关系表;总库做跨库实体对齐 |
|
||||
|
||||
只读目标说明:
|
||||
|
||||
|
||||
@@ -13,6 +13,8 @@ import {
|
||||
} from "@/core/system-settings/api";
|
||||
|
||||
export const KNOWLEDGE_PRODUCT_KEY = ["system-settings", "knowledge-base-product-version"] as const;
|
||||
export const KNOWLEDGE_NORMAL_VIEW_VALUE = "normal";
|
||||
export const KNOWLEDGE_NORMAL_VIEW_SEARCH = `?view=${KNOWLEDGE_NORMAL_VIEW_VALUE}`;
|
||||
|
||||
export function useKnowledgeBaseProductVersion() {
|
||||
return useQuery({ queryKey: KNOWLEDGE_PRODUCT_KEY, queryFn: loadKnowledgeBaseProductVersion, staleTime: 60_000 });
|
||||
@@ -35,7 +37,7 @@ export function KnowledgeProductNav() {
|
||||
|
||||
return (
|
||||
<div className="flex items-center gap-2 border-b bg-card px-6 py-2">
|
||||
<Button size="sm" variant={assistant ? "ghost" : "secondary"} onClick={() => navigate("/page/workspace/knowledge")}>
|
||||
<Button size="sm" variant={assistant ? "ghost" : "secondary"} onClick={() => navigate(`/page/workspace/knowledge${KNOWLEDGE_NORMAL_VIEW_SEARCH}`)}>
|
||||
<BookOpen />普通知识库
|
||||
</Button>
|
||||
<Button size="sm" variant={assistant ? "secondary" : "ghost"} onClick={() => navigate("/page/workspace/knowledge/assistant")}>
|
||||
|
||||
@@ -22,6 +22,10 @@ import {
|
||||
} from "@/components/ui/dropdown-menu";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
type AssistantKnowledgeBase,
|
||||
useAssistantKnowledgeBases,
|
||||
} from "@/strategy-components/api/assistant-knowledge";
|
||||
import {
|
||||
type LlmWikiKnowledgeBase,
|
||||
useAgentLlmWikiKnowledgeBases,
|
||||
@@ -66,10 +70,26 @@ function SpaceIcon({ item }: { item: LlmWikiKnowledgeBase }) {
|
||||
);
|
||||
}
|
||||
|
||||
function AssistantSpaceIcon({ item }: { item: AssistantKnowledgeBase }) {
|
||||
return (
|
||||
<span
|
||||
className={cn(
|
||||
"flex size-7 shrink-0 items-center justify-center rounded-md",
|
||||
item.is_global ? "bg-amber-500/10 text-amber-600 dark:text-amber-400" : "bg-sky-500/10 text-sky-600 dark:text-sky-400",
|
||||
)}
|
||||
>
|
||||
<Database className="size-3.5" />
|
||||
</span>
|
||||
);
|
||||
}
|
||||
|
||||
function SelectionContent({
|
||||
rows,
|
||||
assistantRows,
|
||||
value,
|
||||
assistantValue,
|
||||
onChange,
|
||||
onAssistantChange,
|
||||
search,
|
||||
onSearchChange,
|
||||
loading,
|
||||
@@ -79,8 +99,11 @@ function SelectionContent({
|
||||
readOnly,
|
||||
}: {
|
||||
rows: LlmWikiKnowledgeBase[];
|
||||
assistantRows: AssistantKnowledgeBase[];
|
||||
value: string[];
|
||||
assistantValue: string[];
|
||||
onChange: (value: string[]) => void;
|
||||
onAssistantChange: (value: string[]) => void;
|
||||
search: string;
|
||||
onSearchChange: (value: string) => void;
|
||||
loading: boolean;
|
||||
@@ -90,6 +113,7 @@ function SelectionContent({
|
||||
readOnly: boolean;
|
||||
}) {
|
||||
const selected = useMemo(() => new Set(value), [value]);
|
||||
const selectedAssistant = useMemo(() => new Set(assistantValue), [assistantValue]);
|
||||
const filtered = useMemo(() => {
|
||||
const query = search.trim().toLocaleLowerCase();
|
||||
if (!query) return rows;
|
||||
@@ -97,11 +121,28 @@ function SelectionContent({
|
||||
`${item.name} ${item.description}`.toLocaleLowerCase().includes(query),
|
||||
);
|
||||
}, [rows, search]);
|
||||
const filteredAssistant = useMemo(() => {
|
||||
const query = search.trim().toLocaleLowerCase();
|
||||
if (!query) return assistantRows;
|
||||
return assistantRows.filter((item) =>
|
||||
`${item.name} ${item.description ?? ""}`.toLocaleLowerCase().includes(query),
|
||||
);
|
||||
}, [assistantRows, search]);
|
||||
const groups = useMemo(() => groupRows(filtered), [filtered]);
|
||||
const toggle = (id: string) => {
|
||||
if (readOnly) return;
|
||||
onChange(selected.has(id) ? value.filter((item) => item !== id) : [...value, id]);
|
||||
};
|
||||
const toggleAssistant = (id: string) => {
|
||||
if (readOnly) return;
|
||||
onAssistantChange(
|
||||
selectedAssistant.has(id)
|
||||
? assistantValue.filter((item) => item !== id)
|
||||
: [...assistantValue, id],
|
||||
);
|
||||
};
|
||||
const selectedCount = value.length + assistantValue.length;
|
||||
const hasRows = groups.length > 0 || filteredAssistant.length > 0;
|
||||
|
||||
return (
|
||||
<>
|
||||
@@ -127,7 +168,7 @@ function SelectionContent({
|
||||
<div className="m-2 rounded-lg border border-destructive/30 bg-destructive/5 p-3 text-xs text-destructive">
|
||||
{error instanceof Error ? error.message : "知识空间读取失败"}
|
||||
</div>
|
||||
) : groups.length === 0 ? (
|
||||
) : !hasRows ? (
|
||||
<div className="flex h-28 items-center justify-center px-4 text-center text-xs text-muted-foreground">
|
||||
{search
|
||||
? "没有匹配的知识空间"
|
||||
@@ -136,91 +177,181 @@ function SelectionContent({
|
||||
: "当前账号暂无可用知识空间"}
|
||||
</div>
|
||||
) : (
|
||||
groups.map((group, groupIndex) => (
|
||||
<div key={group.key}>
|
||||
{groupIndex > 0 && (
|
||||
variant === "popover"
|
||||
? <DropdownMenuSeparator className="my-1.5" />
|
||||
: <div className="my-1.5 border-t" />
|
||||
)}
|
||||
{variant === "popover" ? (
|
||||
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
||||
</DropdownMenuLabel>
|
||||
) : (
|
||||
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
||||
</div>
|
||||
)}
|
||||
{group.rows.map((item) => {
|
||||
const rowContent = (
|
||||
<>
|
||||
<SpaceIcon item={item} />
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="truncate text-xs font-medium">{item.name}</div>
|
||||
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
|
||||
{item.type === "faq" ? "FAQ" : "文档库"} · {item.knowledge_count ?? 0} 项知识
|
||||
</div>
|
||||
</div>
|
||||
{item.publication_status === "published" && (
|
||||
<Badge variant="outline" className="h-4 px-1 text-[9px]">公共</Badge>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
return variant === "popover" ? (
|
||||
<DropdownMenuCheckboxItem
|
||||
key={item.id}
|
||||
checked={selected.has(item.id)}
|
||||
disabled={readOnly}
|
||||
onCheckedChange={() => toggle(item.id)}
|
||||
onSelect={(event) => event.preventDefault()}
|
||||
className={cn(
|
||||
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
|
||||
selected.has(item.id) && "bg-primary/8 focus:bg-primary/12",
|
||||
)}
|
||||
>
|
||||
{rowContent}
|
||||
</DropdownMenuCheckboxItem>
|
||||
<>
|
||||
{groups.map((group, groupIndex) => (
|
||||
<div key={group.key}>
|
||||
{groupIndex > 0 && (
|
||||
variant === "popover"
|
||||
? <DropdownMenuSeparator className="my-1.5" />
|
||||
: <div className="my-1.5 border-t" />
|
||||
)}
|
||||
{variant === "popover" ? (
|
||||
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
||||
</DropdownMenuLabel>
|
||||
) : (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
role="checkbox"
|
||||
aria-checked={selected.has(item.id)}
|
||||
disabled={readOnly}
|
||||
onClick={() => toggle(item.id)}
|
||||
className={cn(
|
||||
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
|
||||
selected.has(item.id) && "bg-primary/8",
|
||||
)}
|
||||
>
|
||||
<span className={cn(
|
||||
"flex size-4 shrink-0 items-center justify-center rounded border",
|
||||
selected.has(item.id) && "border-primary bg-primary text-primary-foreground",
|
||||
)}>
|
||||
{selected.has(item.id) && <Check className="size-3" />}
|
||||
</span>
|
||||
{rowContent}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
))
|
||||
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
||||
</div>
|
||||
)}
|
||||
{group.rows.map((item) => {
|
||||
const rowContent = (
|
||||
<>
|
||||
<SpaceIcon item={item} />
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="truncate text-xs font-medium">{item.name}</div>
|
||||
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
|
||||
{item.type === "faq" ? "FAQ" : "文档库"} · {item.knowledge_count ?? 0} 项知识
|
||||
</div>
|
||||
</div>
|
||||
{item.publication_status === "published" && (
|
||||
<Badge variant="outline" className="h-4 px-1 text-[9px]">公共</Badge>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
return variant === "popover" ? (
|
||||
<DropdownMenuCheckboxItem
|
||||
key={item.id}
|
||||
checked={selected.has(item.id)}
|
||||
disabled={readOnly}
|
||||
onCheckedChange={() => toggle(item.id)}
|
||||
onSelect={(event) => event.preventDefault()}
|
||||
className={cn(
|
||||
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
|
||||
selected.has(item.id) && "bg-primary/8 focus:bg-primary/12",
|
||||
)}
|
||||
>
|
||||
{rowContent}
|
||||
</DropdownMenuCheckboxItem>
|
||||
) : (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
role="checkbox"
|
||||
aria-checked={selected.has(item.id)}
|
||||
disabled={readOnly}
|
||||
onClick={() => toggle(item.id)}
|
||||
className={cn(
|
||||
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
|
||||
selected.has(item.id) && "bg-primary/8",
|
||||
)}
|
||||
>
|
||||
<span className={cn(
|
||||
"flex size-4 shrink-0 items-center justify-center rounded border",
|
||||
selected.has(item.id) && "border-primary bg-primary text-primary-foreground",
|
||||
)}>
|
||||
{selected.has(item.id) && <Check className="size-3" />}
|
||||
</span>
|
||||
{rowContent}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
))}
|
||||
{filteredAssistant.length > 0 && (
|
||||
<div>
|
||||
{(groups.length > 0) && (
|
||||
variant === "popover"
|
||||
? <DropdownMenuSeparator className="my-1.5" />
|
||||
: <div className="my-1.5 border-t" />
|
||||
)}
|
||||
{variant === "popover" ? (
|
||||
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||
<Database className="size-3" />助手知识库
|
||||
</DropdownMenuLabel>
|
||||
) : (
|
||||
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||
<Database className="size-3" />助手知识库
|
||||
</div>
|
||||
)}
|
||||
{filteredAssistant.map((item) => {
|
||||
const rowContent = (
|
||||
<>
|
||||
<AssistantSpaceIcon item={item} />
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="truncate text-xs font-medium">{item.name}</div>
|
||||
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
|
||||
Wiki · {item.page_count ?? 0} 页 · {item.entity_count ?? 0} 实体
|
||||
</div>
|
||||
</div>
|
||||
<Badge variant="outline" className="h-4 px-1 text-[9px]">
|
||||
{item.is_global ? "总库" : "助手"}
|
||||
</Badge>
|
||||
</>
|
||||
);
|
||||
return variant === "popover" ? (
|
||||
<DropdownMenuCheckboxItem
|
||||
key={`assistant:${item.id}`}
|
||||
checked={selectedAssistant.has(item.id)}
|
||||
disabled={readOnly}
|
||||
onCheckedChange={() => toggleAssistant(item.id)}
|
||||
onSelect={(event) => event.preventDefault()}
|
||||
className={cn(
|
||||
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
|
||||
selectedAssistant.has(item.id) && "bg-primary/8 focus:bg-primary/12",
|
||||
)}
|
||||
>
|
||||
{rowContent}
|
||||
</DropdownMenuCheckboxItem>
|
||||
) : (
|
||||
<button
|
||||
key={`assistant:${item.id}`}
|
||||
type="button"
|
||||
role="checkbox"
|
||||
aria-checked={selectedAssistant.has(item.id)}
|
||||
disabled={readOnly}
|
||||
onClick={() => toggleAssistant(item.id)}
|
||||
className={cn(
|
||||
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
|
||||
selectedAssistant.has(item.id) && "bg-primary/8",
|
||||
)}
|
||||
>
|
||||
<span className={cn(
|
||||
"flex size-4 shrink-0 items-center justify-center rounded border",
|
||||
selectedAssistant.has(item.id) && "border-primary bg-primary text-primary-foreground",
|
||||
)}>
|
||||
{selectedAssistant.has(item.id) && <Check className="size-3" />}
|
||||
</span>
|
||||
{rowContent}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between gap-2 border-t bg-muted/20 px-2.5 py-2">
|
||||
<span className="text-[11px] text-muted-foreground">
|
||||
{readOnly ? `本对话已选 ${value.length} 个` : `已选择 ${value.length} 个`}
|
||||
{readOnly ? `本对话已选 ${selectedCount} 个` : `已选择 ${selectedCount} 个`}
|
||||
</span>
|
||||
{readOnly ? (
|
||||
<span className="text-[11px] text-muted-foreground">对话开始后不可更改</span>
|
||||
) : (
|
||||
<div className="flex items-center gap-1">
|
||||
<Button type="button" variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => onChange(filtered.map((item) => item.id))}>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="h-7 px-2 text-xs"
|
||||
onClick={() => {
|
||||
onChange(filtered.map((item) => item.id));
|
||||
onAssistantChange(filteredAssistant.map((item) => item.id));
|
||||
}}
|
||||
>
|
||||
全选
|
||||
</Button>
|
||||
<Button type="button" variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => onChange([])}>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="h-7 px-2 text-xs"
|
||||
onClick={() => {
|
||||
onChange([]);
|
||||
onAssistantChange([]);
|
||||
}}
|
||||
>
|
||||
清空
|
||||
</Button>
|
||||
</div>
|
||||
@@ -232,7 +363,9 @@ function SelectionContent({
|
||||
|
||||
export function LlmWikiKnowledgeSelector({
|
||||
value,
|
||||
assistantValue = [],
|
||||
onChange,
|
||||
onAssistantChange = () => undefined,
|
||||
source = "all",
|
||||
agentId,
|
||||
autoSelectAll = false,
|
||||
@@ -243,7 +376,9 @@ export function LlmWikiKnowledgeSelector({
|
||||
variant = "popover",
|
||||
}: {
|
||||
value: string[];
|
||||
assistantValue?: string[];
|
||||
onChange: (value: string[]) => void;
|
||||
onAssistantChange?: (value: string[]) => void;
|
||||
source?: SelectorSource;
|
||||
agentId?: string | null;
|
||||
autoSelectAll?: boolean;
|
||||
@@ -257,28 +392,39 @@ export function LlmWikiKnowledgeSelector({
|
||||
const [open, setOpen] = useState(false);
|
||||
const [search, setSearch] = useState("");
|
||||
const runtime = useLlmWikiRuntime();
|
||||
const weknoraEnabled = runtime.data?.provider === "weknora";
|
||||
const all = useLlmWikiKnowledgeBases(
|
||||
"all",
|
||||
runtime.data?.provider === "weknora",
|
||||
weknoraEnabled,
|
||||
);
|
||||
const agent = useAgentLlmWikiKnowledgeBases(
|
||||
agentId,
|
||||
runtime.data?.provider === "weknora" && source === "agent" && !!agentId,
|
||||
weknoraEnabled && source === "agent" && !!agentId,
|
||||
);
|
||||
const rows = all.data ?? [];
|
||||
const assistantBases = useAssistantKnowledgeBases();
|
||||
const rows = weknoraEnabled ? (all.data ?? []) : [];
|
||||
const assistantRows = assistantBases.data ?? [];
|
||||
const defaultIds = useMemo(() => {
|
||||
if (source !== "agent") {
|
||||
return rows.map((item) => item.id);
|
||||
}
|
||||
return (agent.data ?? []).map((item) => item.id);
|
||||
}, [agent.data, rows, source]);
|
||||
const loading = all.isLoading || (source === "agent" && !!agentId && agent.isLoading);
|
||||
const error = all.error ?? (source === "agent" ? agent.error : null);
|
||||
const loading = runtime.isLoading || (weknoraEnabled && all.isLoading) || assistantBases.isLoading || (source === "agent" && !!agentId && agent.isLoading);
|
||||
const error = (weknoraEnabled ? all.error ?? (source === "agent" ? agent.error : null) : null) ?? assistantBases.error;
|
||||
const availableIds = useMemo(() => new Set(rows.map((item) => item.id)), [rows]);
|
||||
const assistantAvailableIds = useMemo(
|
||||
() => new Set(assistantRows.map((item) => item.id)),
|
||||
[assistantRows],
|
||||
);
|
||||
const initializationKey = `${source}:${agentId ?? "all"}:${defaultIds.join(",")}`;
|
||||
const initialized = useRef<string | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (!runtime.isLoading && !weknoraEnabled) {
|
||||
if (value.length > 0) onChange([]);
|
||||
return;
|
||||
}
|
||||
if (!all.data || (source === "agent" && agentId && !agent.data) || initialized.current === initializationKey) return;
|
||||
initialized.current = initializationKey;
|
||||
const valid = value.filter((id) => availableIds.has(id));
|
||||
@@ -288,19 +434,30 @@ export function LlmWikiKnowledgeSelector({
|
||||
if (next.length !== value.length || next.some((id, index) => id !== value[index])) {
|
||||
onChange(next);
|
||||
}
|
||||
}, [agent.data, agentId, all.data, autoSelectAll, availableIds, defaultIds, initializationKey, onChange, source, value]);
|
||||
}, [agent.data, agentId, all.data, autoSelectAll, availableIds, defaultIds, initializationKey, onChange, runtime.isLoading, source, value, weknoraEnabled]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!assistantBases.data) return;
|
||||
const valid = assistantValue.filter((id) => assistantAvailableIds.has(id));
|
||||
if (valid.length !== assistantValue.length || valid.some((id, index) => id !== assistantValue[index])) {
|
||||
onAssistantChange(valid);
|
||||
}
|
||||
}, [assistantAvailableIds, assistantBases.data, assistantValue, onAssistantChange]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) setSearch("");
|
||||
}, [open]);
|
||||
|
||||
if (runtime.isError || runtime.data?.provider !== "weknora") return null;
|
||||
if ((runtime.isError || !weknoraEnabled) && !assistantBases.isLoading && assistantRows.length === 0) return null;
|
||||
|
||||
const content = (
|
||||
<SelectionContent
|
||||
rows={rows}
|
||||
assistantRows={assistantRows}
|
||||
value={value}
|
||||
assistantValue={assistantValue}
|
||||
onChange={onChange}
|
||||
onAssistantChange={onAssistantChange}
|
||||
search={search}
|
||||
onSearchChange={setSearch}
|
||||
loading={loading}
|
||||
@@ -323,7 +480,7 @@ export function LlmWikiKnowledgeSelector({
|
||||
新建该智能体对话时,只能从已绑定空间中选择本次检索范围。
|
||||
</p>
|
||||
</div>
|
||||
<Badge variant="secondary" className="shrink-0">{value.length} 个</Badge>
|
||||
<Badge variant="secondary" className="shrink-0">{value.length + assistantValue.length} 个</Badge>
|
||||
</div>
|
||||
{content}
|
||||
</div>
|
||||
@@ -333,10 +490,15 @@ export function LlmWikiKnowledgeSelector({
|
||||
const selectedNames = value
|
||||
.map((id) => rows.find((item) => item.id === id)?.name)
|
||||
.filter((name): name is string => !!name);
|
||||
const selectedLabel = selectedNames.length > 0
|
||||
? selectedNames.join("、")
|
||||
: value.length > 0
|
||||
? `已选 ${value.length} 个知识空间`
|
||||
const selectedAssistantNames = assistantValue
|
||||
.map((id) => assistantRows.find((item) => item.id === id)?.name)
|
||||
.filter((name): name is string => !!name);
|
||||
const allSelectedNames = [...selectedNames, ...selectedAssistantNames];
|
||||
const selectedCount = value.length + assistantValue.length;
|
||||
const selectedLabel = allSelectedNames.length > 0
|
||||
? allSelectedNames.join("、")
|
||||
: selectedCount > 0
|
||||
? `已选 ${selectedCount} 个知识空间`
|
||||
: label;
|
||||
|
||||
return (
|
||||
@@ -348,20 +510,20 @@ export function LlmWikiKnowledgeSelector({
|
||||
size="sm"
|
||||
className={cn(
|
||||
"h-8 max-w-52 gap-1.5 rounded-full border-0 bg-transparent px-2.5 text-muted-foreground shadow-none hover:bg-muted/70 hover:text-foreground",
|
||||
value.length > 0 && "bg-primary/8 text-primary hover:bg-primary/12 hover:text-primary",
|
||||
selectedCount > 0 && "bg-primary/8 text-primary hover:bg-primary/12 hover:text-primary",
|
||||
className,
|
||||
)}
|
||||
disabled={disabled || loading}
|
||||
title={value.length > 0 ? selectedLabel : undefined}
|
||||
title={selectedCount > 0 ? selectedLabel : undefined}
|
||||
>
|
||||
{loading ? <Loader2 className="size-3.5 animate-spin" /> : <Database className="size-3.5" />}
|
||||
<span className="truncate text-xs font-normal">{selectedLabel}</span>
|
||||
{value.length > 0 && (
|
||||
{selectedCount > 0 && (
|
||||
<span className="flex size-4 items-center justify-center rounded-full bg-primary text-[9px] font-semibold text-primary-foreground">
|
||||
{value.length}
|
||||
{selectedCount}
|
||||
</span>
|
||||
)}
|
||||
{value.length > 0 && value.length === rows.length && <Check className="size-3" />}
|
||||
{selectedCount > 0 && selectedCount === rows.length + assistantRows.length && <Check className="size-3" />}
|
||||
</Button>
|
||||
</DropdownMenuTrigger>
|
||||
<DropdownMenuContent align="start" sideOffset={8} className="w-80 overflow-hidden rounded-xl p-0 shadow-xl">
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { CheckCircle2, Database, Loader2, RefreshCw, Sparkles } from "lucide-react";
|
||||
import { Database, Loader2, RefreshCw, Search, Sparkles } from "lucide-react";
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { toast } from "sonner";
|
||||
|
||||
@@ -10,18 +10,12 @@ import { Progress } from "@/components/ui/progress";
|
||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
|
||||
import { useAllSkillsAdmin } from "@/core/skills/hooks";
|
||||
import {
|
||||
useAssistantKnowledgeActions,
|
||||
useAssistantKnowledgeBase,
|
||||
} from "@/strategy-components/api/assistant-knowledge";
|
||||
import { useLlmWikiKnowledgeBases, useLlmWikiRuntime } from "@/strategy-components/api/llmwiki";
|
||||
import {
|
||||
type SkillKnowledgeTarget,
|
||||
useSkillKnowledgeActions,
|
||||
useSkillKnowledgeAutoSyncSettings,
|
||||
useSkillKnowledgeBindings,
|
||||
useSkillKnowledgeJobs,
|
||||
useSkillKnowledgeReviews,
|
||||
} from "@/strategy-components/api/skill-knowledge";
|
||||
|
||||
const PHASE_LABELS: Record<string, string> = {
|
||||
@@ -39,53 +33,82 @@ export function SkillKnowledgePanel() {
|
||||
const { skills, isLoading } = useAllSkillsAdmin(true);
|
||||
const liveSkills = useMemo(() => skills.filter((skill) => (skill.state ?? "active") === "active"), [skills]);
|
||||
const [selectedSkills, setSelectedSkills] = useState<string[]>([]);
|
||||
const [selectedTargetKeys, setSelectedTargetKeys] = useState<string[]>([]);
|
||||
const [reviewMode, setReviewMode] = useState<"required" | "auto_high_confidence" | "off">("auto_high_confidence");
|
||||
const [skillQuery, setSkillQuery] = useState("");
|
||||
const [selectedTargetKey, setSelectedTargetKey] = useState<string | null>(null);
|
||||
const [reviewMode, setReviewMode] = useState<"auto_high_confidence" | "off">("auto_high_confidence");
|
||||
const runtime = useLlmWikiRuntime();
|
||||
const knowledgeBases = useLlmWikiKnowledgeBases("all", runtime.data?.provider === "weknora");
|
||||
const assistantBase = useAssistantKnowledgeBase();
|
||||
const assistantActions = useAssistantKnowledgeActions();
|
||||
const bindings = useSkillKnowledgeBindings();
|
||||
const jobs = useSkillKnowledgeJobs();
|
||||
const reviews = useSkillKnowledgeReviews();
|
||||
const actions = useSkillKnowledgeActions();
|
||||
const autoSettings = useSkillKnowledgeAutoSyncSettings();
|
||||
const [scheduleCron, setScheduleCron] = useState("0 */6 * * *");
|
||||
const [debounceSeconds, setDebounceSeconds] = useState(120);
|
||||
|
||||
useEffect(() => {
|
||||
if (!autoSettings.data) return;
|
||||
setScheduleCron(autoSettings.data.schedule_cron);
|
||||
setDebounceSeconds(autoSettings.data.debounce_seconds);
|
||||
}, [autoSettings.data]);
|
||||
|
||||
const targets = useMemo<SkillKnowledgeTarget[]>(() => {
|
||||
const rows: SkillKnowledgeTarget[] = (knowledgeBases.data ?? []).map((kb) => ({
|
||||
target_type: "weknora",
|
||||
target_mode: kb.can_write ? kb.type === "faq" ? "faq" : kb.type === "wiki" ? "wiki" : "document" : "readonly_sidecar",
|
||||
target_id: kb.id,
|
||||
target_name: kb.name,
|
||||
remote_business_key: `weknora:kb:${kb.id}`,
|
||||
readonly: !kb.can_write,
|
||||
}));
|
||||
if (assistantBase.data) {
|
||||
rows.unshift({
|
||||
target_type: "assistant",
|
||||
target_mode: "assistant",
|
||||
target_id: assistantBase.data.id,
|
||||
target_name: assistantBase.data.name,
|
||||
remote_business_key: `assistant:base:${assistantBase.data.id}`,
|
||||
});
|
||||
return (knowledgeBases.data ?? [])
|
||||
.filter((kb) => {
|
||||
const strategy = kb.indexing_strategy ?? {};
|
||||
const capabilities = kb.capabilities ?? {};
|
||||
const wikiEnabled =
|
||||
kb.type === "wiki" ||
|
||||
kb.wiki_index_enabled === true ||
|
||||
strategy.wiki_enabled === true ||
|
||||
capabilities.wiki_enabled === true ||
|
||||
(kb.wiki_config && Object.keys(kb.wiki_config).length > 0);
|
||||
return kb.can_write && kb.remote_status !== "missing" && !kb.is_conversation_deposit && kb.type !== "faq" && wikiEnabled;
|
||||
})
|
||||
.map((kb) => ({
|
||||
target_type: "weknora",
|
||||
target_mode: "wiki",
|
||||
target_id: kb.id,
|
||||
target_name: kb.name,
|
||||
remote_business_key: `weknora:kb:${kb.id}`,
|
||||
}));
|
||||
}, [knowledgeBases.data]);
|
||||
|
||||
const filteredSkills = useMemo(() => {
|
||||
const needle = skillQuery.trim().toLowerCase();
|
||||
if (!needle) return liveSkills;
|
||||
return liveSkills.filter((skill) => {
|
||||
const tagText = (skill.tags ?? []).map((tag) => tag.name).join(" ");
|
||||
const haystack = [
|
||||
skill.name,
|
||||
skill.name_zh,
|
||||
skill.description,
|
||||
skill.category,
|
||||
tagText,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
.toLowerCase();
|
||||
return haystack.includes(needle);
|
||||
});
|
||||
}, [liveSkills, skillQuery]);
|
||||
|
||||
const filteredSkillNames = useMemo(() => filteredSkills.map((skill) => skill.name), [filteredSkills]);
|
||||
const allFilteredSelected = filteredSkillNames.length > 0 && filteredSkillNames.every((name) => selectedSkills.includes(name));
|
||||
|
||||
const toggleFilteredSkills = () => {
|
||||
if (!filteredSkillNames.length) return;
|
||||
setSelectedSkills((current) => {
|
||||
if (allFilteredSelected) {
|
||||
const filtered = new Set(filteredSkillNames);
|
||||
return current.filter((name) => !filtered.has(name));
|
||||
}
|
||||
return [...current, ...filteredSkillNames.filter((name) => !current.includes(name))];
|
||||
});
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
if (selectedTargetKey && !targets.some((target) => `${target.target_type}:${target.target_id}` === selectedTargetKey)) {
|
||||
setSelectedTargetKey(null);
|
||||
}
|
||||
return rows;
|
||||
}, [assistantBase.data, knowledgeBases.data]);
|
||||
}, [selectedTargetKey, targets]);
|
||||
|
||||
const toggle = (values: string[], value: string, setter: (next: string[]) => void) =>
|
||||
setter(values.includes(value) ? values.filter((item) => item !== value) : [...values, value]);
|
||||
|
||||
const submit = async () => {
|
||||
const selectedTargets = targets.filter((target) =>
|
||||
selectedTargetKeys.includes(`${target.target_type}:${target.target_id}`),
|
||||
selectedTargetKey === `${target.target_type}:${target.target_id}`,
|
||||
);
|
||||
if (!selectedSkills.length || !selectedTargets.length) {
|
||||
toast.error("请至少选择一个技能和一个目标知识库");
|
||||
@@ -108,46 +131,76 @@ export function SkillKnowledgePanel() {
|
||||
<div className="grid size-full min-h-0 grid-cols-[minmax(320px,0.9fr)_minmax(420px,1.1fr)] gap-4 overflow-hidden p-6">
|
||||
<section className="flex min-h-0 flex-col rounded-lg border bg-card">
|
||||
<div className="border-b p-4">
|
||||
<h2 className="font-semibold">批量归纳</h2>
|
||||
<p className="mt-1 text-xs text-muted-foreground">按“技能 × 目标知识库”生成独立后台任务;代码与脚本不会被读取。</p>
|
||||
<h2 className="font-semibold">技能到知识库管理</h2>
|
||||
<p className="mt-1 text-xs text-muted-foreground">一次选择多个技能归纳到一个 WeKnora Wiki 知识库;代码与脚本不会被读取。</p>
|
||||
</div>
|
||||
<ScrollArea className="min-h-0 flex-1">
|
||||
<div className="space-y-5 p-4">
|
||||
<div>
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<span className="text-sm font-medium">1. 选择技能({selectedSkills.length})</span>
|
||||
<Button variant="ghost" size="sm" onClick={() => setSelectedSkills(selectedSkills.length === liveSkills.length ? [] : liveSkills.map((skill) => skill.name))}>全选/清空</Button>
|
||||
<Button variant="ghost" size="sm" onClick={toggleFilteredSkills}>
|
||||
{allFilteredSelected ? "清空当前结果" : "选择当前结果"}
|
||||
</Button>
|
||||
</div>
|
||||
<div className="relative mb-2">
|
||||
<Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
|
||||
<Input
|
||||
value={skillQuery}
|
||||
onChange={(event) => setSkillQuery(event.target.value)}
|
||||
placeholder="按技能中文名、英文名、描述、分类或标签搜索"
|
||||
className="pl-8"
|
||||
/>
|
||||
</div>
|
||||
<div className="max-h-60 space-y-1 overflow-y-auto rounded-md border p-2">
|
||||
{isLoading ? <Loader2 className="m-4 size-5 animate-spin" /> : liveSkills.map((skill) => (
|
||||
{isLoading ? <Loader2 className="m-4 size-5 animate-spin" /> : filteredSkills.map((skill) => (
|
||||
<label key={skill.name} className="flex cursor-pointer items-center gap-2 rounded px-2 py-1.5 hover:bg-muted">
|
||||
<Checkbox checked={selectedSkills.includes(skill.name)} onCheckedChange={() => toggle(selectedSkills, skill.name, setSelectedSkills)} />
|
||||
<span className="truncate text-sm">{skill.name_zh || skill.name}</span>
|
||||
<span className="ml-auto truncate text-xs text-muted-foreground">{skill.name}</span>
|
||||
</label>
|
||||
))}
|
||||
{!isLoading && !filteredSkills.length && (
|
||||
<p className="p-3 text-xs text-muted-foreground">没有匹配的技能,请换个关键词。</p>
|
||||
)}
|
||||
</div>
|
||||
<p className="mt-1 text-xs text-muted-foreground">
|
||||
当前显示 {filteredSkills.length}/{liveSkills.length} 个技能。
|
||||
</p>
|
||||
</div>
|
||||
<div>
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<span className="text-sm font-medium">2. 选择目标({selectedTargetKeys.length})</span>
|
||||
{!assistantBase.data && (
|
||||
<Button size="sm" variant="outline" disabled={assistantActions.initialize.isPending} onClick={() => assistantActions.initialize.mutate("知识梳理")}>创建知识梳理库</Button>
|
||||
)}
|
||||
<span className="text-sm font-medium">2. 选择目标知识库({selectedTargetKey ? 1 : 0})</span>
|
||||
</div>
|
||||
<div className="space-y-1 rounded-md border p-2">
|
||||
{targets.map((target) => {
|
||||
const key = `${target.target_type}:${target.target_id}`;
|
||||
return (
|
||||
<label key={key} className="flex cursor-pointer items-center gap-2 rounded px-2 py-2 hover:bg-muted">
|
||||
<Checkbox checked={selectedTargetKeys.includes(key)} onCheckedChange={() => toggle(selectedTargetKeys, key, setSelectedTargetKeys)} />
|
||||
<Checkbox
|
||||
checked={selectedTargetKey === key}
|
||||
onCheckedChange={() => setSelectedTargetKey((current) => (current === key ? null : key))}
|
||||
/>
|
||||
<Database className="size-4 text-muted-foreground" />
|
||||
<span className="text-sm">{target.target_name}</span>
|
||||
<Badge variant="outline" className="ml-auto text-[10px]">{target.target_type === "assistant" ? "助手库" : target.target_mode === "readonly_sidecar" ? "只读·本地投影" : target.target_mode}</Badge>
|
||||
<Badge variant="outline" className="ml-auto text-[10px]">WeKnora Wiki</Badge>
|
||||
</label>
|
||||
);
|
||||
})}
|
||||
{!targets.length && <p className="p-3 text-xs text-muted-foreground">未配置普通知识库;可先创建全局“知识梳理”助手库。</p>}
|
||||
{(runtime.isLoading || knowledgeBases.isLoading) && <Loader2 className="m-4 size-5 animate-spin" />}
|
||||
{!runtime.isLoading && !knowledgeBases.isLoading && knowledgeBases.error && (
|
||||
<p className="p-3 text-xs text-destructive">
|
||||
读取 WeKnora 普通知识库失败:
|
||||
{knowledgeBases.error instanceof Error ? knowledgeBases.error.message : String(knowledgeBases.error)}
|
||||
</p>
|
||||
)}
|
||||
{!runtime.isLoading && !knowledgeBases.isLoading && !knowledgeBases.error && !targets.length && (
|
||||
<p className="p-3 text-xs text-muted-foreground">
|
||||
{runtime.data?.provider !== "weknora"
|
||||
? "当前未配置 WeKnora 普通知识库地址;技能需要先归纳到普通知识库,再导入助手知识库。"
|
||||
: "当前没有可写且已开启 Wiki 的普通知识库;请确认该库不是 FAQ/会话沉淀库,并已开启 Wiki。"}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center gap-3">
|
||||
@@ -156,14 +209,13 @@ export function SkillKnowledgePanel() {
|
||||
<SelectTrigger className="w-52"><SelectValue /></SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="auto_high_confidence">高置信自动通过</SelectItem>
|
||||
<SelectItem value="required">全部逐条审核</SelectItem>
|
||||
<SelectItem value="off">不阻塞写入</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
<Button className="w-full" disabled={actions.createJob.isPending} onClick={() => void submit()}>
|
||||
{actions.createJob.isPending ? <Loader2 className="animate-spin" /> : <Sparkles />}
|
||||
开始归纳({selectedSkills.length * selectedTargetKeys.length} 项)
|
||||
开始归纳({selectedSkills.length * (selectedTargetKey ? 1 : 0)} 项)
|
||||
</Button>
|
||||
</div>
|
||||
</ScrollArea>
|
||||
@@ -199,26 +251,9 @@ export function SkillKnowledgePanel() {
|
||||
</div>
|
||||
</ScrollArea>
|
||||
</div>
|
||||
{(reviews.data?.length ?? 0) > 0 && (
|
||||
<div className="max-h-64 overflow-auto rounded-lg border bg-card p-4">
|
||||
<div className="mb-3 flex items-center"><h2 className="font-semibold">待审核({reviews.data!.length})</h2><Button size="sm" className="ml-auto" onClick={() => actions.review.mutate({ review_ids: reviews.data!.map((row) => row.id), action: "approve" })}><CheckCircle2 />批量批准</Button></div>
|
||||
{reviews.data!.map((review) => <div key={review.id} className="mb-2 flex items-center gap-2 rounded border p-2 text-xs"><Badge variant="outline">{review.object_type}</Badge><code className="truncate">{String(review.original.name || review.original.predicate || review.object_key)}</code><Button size="sm" variant="ghost" className="ml-auto" onClick={() => actions.review.mutate({ review_ids: [review.id], action: "approve" })}>批准</Button><Button size="sm" variant="ghost" onClick={() => actions.review.mutate({ review_ids: [review.id], action: "reject" })}>拒绝</Button></div>)}
|
||||
</div>
|
||||
)}
|
||||
<div className="max-h-80 overflow-auto rounded-lg border bg-card p-3 text-xs">
|
||||
<div className="mb-3 flex flex-wrap items-center gap-3 border-b pb-3">
|
||||
<span className="font-medium">自动同步</span>
|
||||
{autoSettings.data && <>
|
||||
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.watch_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ watch_enabled: checked === true })} />文件监听</label>
|
||||
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.schedule_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ schedule_enabled: checked === true })} />定时同步</label>
|
||||
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.write_remote_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ write_remote_enabled: checked === true })} />自动写入目标</label>
|
||||
<Input className="h-8 w-32" value={scheduleCron} onChange={(event) => setScheduleCron(event.target.value)} aria-label="自动同步 cron" />
|
||||
<Input className="h-8 w-24" type="number" min={10} value={debounceSeconds} onChange={(event) => setDebounceSeconds(Number(event.target.value))} aria-label="防抖秒数" />
|
||||
<Button size="sm" variant="outline" onClick={() => actions.updateAutoSyncSettings.mutate({ schedule_cron: scheduleCron, debounce_seconds: debounceSeconds }, { onSuccess: () => toast.success("自动同步设置已保存"), onError: (error) => toast.error(error.message) })}>保存</Button>
|
||||
</>}
|
||||
</div>
|
||||
<div className="mb-2 text-muted-foreground">当前有效绑定:{bindings.data?.length ?? 0};同名同类型实体按来源贡献合并。</div>
|
||||
<div className="space-y-2">{bindings.data?.map((binding) => <div key={binding.id} className="flex flex-wrap items-center gap-2 rounded border p-2"><span className="max-w-32 truncate font-medium">{binding.skill_name}</span><span className="max-w-40 truncate text-muted-foreground">→ {binding.target_name}</span><Badge variant="outline">{binding.status}</Badge><code className="max-w-40 truncate text-[10px] text-muted-foreground">{binding.remote_business_key}</code><label className="ml-auto flex items-center gap-1"><Checkbox checked={binding.auto_sync_enabled} onCheckedChange={(checked) => actions.setAutoSync.mutate({ id: binding.id, enabled: checked === true })} />自动</label><Button size="sm" variant="outline" disabled={actions.resync.isPending} onClick={() => actions.resync.mutate(binding.id)}>重归纳</Button><Button size="sm" variant="ghost" onClick={() => { if (window.confirm("解除关联并保留目标知识内容?")) actions.detach.mutate({ id: binding.id, deleteContributions: false }); }}>解绑并保留</Button><Button size="sm" variant="destructive" onClick={() => { if (window.confirm("解除关联并删除此技能贡献?共享知识仍会保留其他来源。")) actions.detach.mutate({ id: binding.id, deleteContributions: true }); }}>解绑并删除贡献</Button></div>)}</div>
|
||||
<div className="space-y-2">{bindings.data?.map((binding) => <div key={binding.id} className="flex flex-wrap items-center gap-2 rounded border p-2"><span className="max-w-32 truncate font-medium">{binding.skill_name}</span><span className="max-w-40 truncate text-muted-foreground">→ {binding.target_name}</span><Badge variant="outline">{binding.status}</Badge><code className="max-w-40 truncate text-[10px] text-muted-foreground">{binding.remote_business_key}</code><Button size="sm" variant="outline" className="ml-auto" disabled={actions.resync.isPending} onClick={() => actions.resync.mutate(binding.id)}>重归纳</Button><Button size="sm" variant="ghost" onClick={() => { if (window.confirm("解除关联并保留目标知识内容?")) actions.detach.mutate({ id: binding.id, deleteContributions: false }); }}>解绑并保留</Button><Button size="sm" variant="destructive" onClick={() => { if (window.confirm("解除关联并删除此技能贡献?共享知识仍会保留其他来源。")) actions.detach.mutate({ id: binding.id, deleteContributions: true }); }}>解绑并删除贡献</Button></div>)}</div>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
@@ -14,7 +14,7 @@ import {
|
||||
SidebarMenuItem,
|
||||
} from "@/components/ui/sidebar";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import { BookOpen, BotIcon, BrainIcon, FileSearch, ListChecks, MessageSquarePlus, MessagesSquare, PenLine, Plus, SparklesIcon, Telescope, Users } from "lucide-react";
|
||||
import { BookOpen, BotIcon, BrainIcon, FileSearch, Library, ListChecks, MessageSquarePlus, MessagesSquare, PenLine, Plus, SparklesIcon, Telescope, Users } from "lucide-react";
|
||||
|
||||
import { KnowledgeRagToggle } from "./knowledge-rag-toggle";
|
||||
import { SidebarCollapseButton } from "./workspace-header";
|
||||
@@ -65,6 +65,13 @@ export const WORKSPACE_COMPACT_NAV_ITEMS: WorkspaceCompactNavItem[] = [
|
||||
path: "/page/workspace/chats",
|
||||
match: (pathname) => pathname === "/page/workspace/chats",
|
||||
},
|
||||
{
|
||||
id: "workspace-library",
|
||||
label: "资料库",
|
||||
icon: Library,
|
||||
path: "/page/workspace/library",
|
||||
match: (pathname) => pathname === "/page/workspace/library",
|
||||
},
|
||||
{
|
||||
id: "workspace-agents",
|
||||
label: "智能体",
|
||||
|
||||
@@ -229,6 +229,14 @@ export function WorkspaceNavMenu() {
|
||||
技能管理
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{isAdmin && (
|
||||
<DropdownMenuItem
|
||||
onClick={() => navigate("/page/workspace/admin/skills?tab=knowledge")}
|
||||
>
|
||||
<DatabaseZapIcon />
|
||||
技能到知识库管理
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{isAdmin && (
|
||||
<DropdownMenuItem
|
||||
onClick={() => navigate("/page/workspace/admin/models")}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { apiFetch } from "../api";
|
||||
import { getFileName } from "../utils/files";
|
||||
import { apiFetch } from "../api";
|
||||
import { downloadMarkdownAsDocx } from "../scheduled-tasks/markdown-docx";
|
||||
import { getFileName } from "../utils/files";
|
||||
|
||||
import { urlOfArtifact } from "./utils";
|
||||
|
||||
@@ -39,7 +40,7 @@ async function errorMessageFromResponse(res: Response): Promise<string> {
|
||||
return res.statusText || "Request failed";
|
||||
}
|
||||
|
||||
export async function downloadArtifactFile({
|
||||
export async function downloadArtifactFile({
|
||||
filepath,
|
||||
threadId,
|
||||
isMock,
|
||||
@@ -73,8 +74,26 @@ export async function downloadArtifactFile({
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
window.setTimeout(() => URL.revokeObjectURL(objectUrl), 30_000);
|
||||
}
|
||||
window.setTimeout(() => URL.revokeObjectURL(objectUrl), 30_000);
|
||||
}
|
||||
|
||||
export async function downloadArtifactMarkdownAsWord({
|
||||
filepath,
|
||||
threadId,
|
||||
}: {
|
||||
filepath: string;
|
||||
threadId: string;
|
||||
}): Promise<void> {
|
||||
const url = urlOfArtifact({ filepath, threadId, download: false });
|
||||
const res = await apiFetch(url);
|
||||
if (!res.ok) {
|
||||
throw new Error(await errorMessageFromResponse(res));
|
||||
}
|
||||
|
||||
const fileName = getFileName(filepath);
|
||||
const title = fileName.replace(/\.(md|markdown)$/i, "");
|
||||
await downloadMarkdownAsDocx(await res.text(), fileName, title);
|
||||
}
|
||||
|
||||
export async function openArtifactInNewTab({
|
||||
filepath,
|
||||
|
||||
@@ -9,11 +9,15 @@ export type ArtifactLibraryKind =
|
||||
| "archive"
|
||||
| "other";
|
||||
|
||||
export type ArtifactLibrarySource = "all" | "general_qa" | "agent";
|
||||
|
||||
export interface ArtifactLibraryItem {
|
||||
threadId: string;
|
||||
threadTitle: string;
|
||||
agentId: string | null;
|
||||
agentName: string | null;
|
||||
threadType: string | null;
|
||||
sourceType: Exclude<ArtifactLibrarySource, "all">;
|
||||
path: string;
|
||||
name: string;
|
||||
sizeBytes: number;
|
||||
@@ -32,6 +36,7 @@ export interface ArtifactLibraryResponse {
|
||||
export async function fetchArtifactLibrary({
|
||||
query,
|
||||
fileKind,
|
||||
sourceType,
|
||||
offset,
|
||||
limit,
|
||||
refresh = false,
|
||||
@@ -39,6 +44,7 @@ export async function fetchArtifactLibrary({
|
||||
}: {
|
||||
query?: string;
|
||||
fileKind: ArtifactLibraryKind;
|
||||
sourceType: ArtifactLibrarySource;
|
||||
offset: number;
|
||||
limit: number;
|
||||
refresh?: boolean;
|
||||
@@ -46,6 +52,7 @@ export async function fetchArtifactLibrary({
|
||||
}): Promise<ArtifactLibraryResponse> {
|
||||
const search = new URLSearchParams({
|
||||
file_kind: fileKind,
|
||||
source_type: sourceType,
|
||||
offset: String(offset),
|
||||
limit: String(limit),
|
||||
});
|
||||
@@ -72,7 +79,9 @@ export async function fetchArtifactLibrary({
|
||||
thread_id: string;
|
||||
thread_title: string;
|
||||
agent_id?: string | null;
|
||||
agent_name?: string | null;
|
||||
thread_type?: string | null;
|
||||
source_type?: Exclude<ArtifactLibrarySource, "all">;
|
||||
path: string;
|
||||
name: string;
|
||||
size_bytes: number;
|
||||
@@ -90,7 +99,9 @@ export async function fetchArtifactLibrary({
|
||||
threadId: item.thread_id,
|
||||
threadTitle: item.thread_title,
|
||||
agentId: item.agent_id ?? null,
|
||||
agentName: item.agent_name ?? item.agent_id ?? null,
|
||||
threadType: item.thread_type ?? null,
|
||||
sourceType: item.source_type ?? (item.agent_id ? "agent" : "general_qa"),
|
||||
path: item.path,
|
||||
name: item.name,
|
||||
sizeBytes: item.size_bytes,
|
||||
|
||||
@@ -1293,15 +1293,21 @@ export function useThreadStream({
|
||||
...context,
|
||||
...extraContext,
|
||||
};
|
||||
const selectedLlmWikiIds = Array.isArray(
|
||||
mergedContext.llmwiki_knowledge_base_ids,
|
||||
)
|
||||
? mergedContext.llmwiki_knowledge_base_ids
|
||||
: [];
|
||||
const ragModeEnabled =
|
||||
localSettings.display.ragModeEnabled ||
|
||||
isThreadRagModeLocked(threadId) ||
|
||||
selectedLlmWikiIds.length > 0;
|
||||
const selectedLlmWikiIds = Array.isArray(
|
||||
mergedContext.llmwiki_knowledge_base_ids,
|
||||
)
|
||||
? mergedContext.llmwiki_knowledge_base_ids
|
||||
: [];
|
||||
const selectedAssistantKnowledgeIds = Array.isArray(
|
||||
mergedContext.assistant_knowledge_base_ids,
|
||||
)
|
||||
? mergedContext.assistant_knowledge_base_ids
|
||||
: [];
|
||||
const ragModeEnabled =
|
||||
localSettings.display.ragModeEnabled ||
|
||||
isThreadRagModeLocked(threadId) ||
|
||||
selectedLlmWikiIds.length > 0 ||
|
||||
selectedAssistantKnowledgeIds.length > 0;
|
||||
if (ragModeEnabled) {
|
||||
lockThreadRagMode(threadId);
|
||||
}
|
||||
|
||||
@@ -27,6 +27,8 @@ export interface AgentThreadContext extends Record<string, unknown> {
|
||||
knowledge_rag_enabled?: boolean;
|
||||
/** DeerFlow LLMWiki mapping ids selected for this conversation. */
|
||||
llmwiki_knowledge_base_ids?: string[];
|
||||
/** Assistant knowledge-base ids selected for this conversation. */
|
||||
assistant_knowledge_base_ids?: string[];
|
||||
/** Direct RAG modes handled in the frontend; these requests never enter Lead Agent. */
|
||||
execution_profile?: "single_round_rag" | "multi_round_rag";
|
||||
agent_id?: string;
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
SparklesIcon,
|
||||
} from "lucide-react";
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { useSearchParams } from "react-router-dom";
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
@@ -29,10 +30,22 @@ import { SkillAddressesPanel } from "./SkillAddressesPage";
|
||||
const PAGE_SIZE = 10;
|
||||
|
||||
export default function AdminSkillsPage() {
|
||||
const [searchParams, setSearchParams] = useSearchParams();
|
||||
const [search, setSearch] = useState("");
|
||||
const [page, setPage] = useState(1);
|
||||
const { skills, isLoading, error } = useAllSkillsAdmin(true);
|
||||
const knowledgeBindings = useSkillKnowledgeBindings();
|
||||
const activeTab = useMemo(() => {
|
||||
const tab = searchParams.get("tab");
|
||||
return tab === "addresses" || tab === "knowledge" ? tab : "skills";
|
||||
}, [searchParams]);
|
||||
|
||||
const changeTab = (value: string) => {
|
||||
const next = new URLSearchParams(searchParams);
|
||||
if (value === "skills") next.delete("tab");
|
||||
else next.set("tab", value);
|
||||
setSearchParams(next, { replace: true });
|
||||
};
|
||||
|
||||
const storedAuth = useMemo(() => getStoredAuth(), []);
|
||||
const currentUserId = storedAuth?.user_id ?? null;
|
||||
@@ -60,7 +73,8 @@ export default function AdminSkillsPage() {
|
||||
<WorkspaceHeader />
|
||||
<WorkspaceBody>
|
||||
<Tabs
|
||||
defaultValue="skills"
|
||||
value={activeTab}
|
||||
onValueChange={changeTab}
|
||||
className="flex size-full min-h-0 flex-col gap-0"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-4 border-b px-6 pt-4">
|
||||
@@ -77,7 +91,7 @@ export default function AdminSkillsPage() {
|
||||
</TabsTrigger>
|
||||
<TabsTrigger value="knowledge">
|
||||
<DatabaseZap />
|
||||
技能知识归纳
|
||||
技能到知识库管理
|
||||
</TabsTrigger>
|
||||
</TabsList>
|
||||
</div>
|
||||
|
||||
@@ -92,6 +92,26 @@ function readPendingKnowledgeSelection(agentId: string | undefined): string[] |
|
||||
}
|
||||
}
|
||||
|
||||
function readPendingAssistantKnowledgeSelection(agentId: string | undefined): string[] | null {
|
||||
if (typeof window === "undefined" || !agentId) return null;
|
||||
try {
|
||||
const raw = sessionStorage.getItem("pendingAgentMessage");
|
||||
if (!raw) return null;
|
||||
const parsed = JSON.parse(raw) as {
|
||||
agentId?: string;
|
||||
assistantKnowledgeBaseIds?: unknown;
|
||||
};
|
||||
if (parsed.agentId !== agentId || !Array.isArray(parsed.assistantKnowledgeBaseIds)) {
|
||||
return null;
|
||||
}
|
||||
return parsed.assistantKnowledgeBaseIds
|
||||
.map((item) => String(item).trim())
|
||||
.filter(Boolean);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function stringifyForHiddenContext(value: unknown): string {
|
||||
if (typeof value === "string") return value;
|
||||
try {
|
||||
@@ -270,11 +290,17 @@ export default function AgentChatPage({
|
||||
const pendingKnowledgeSelection = useRef(
|
||||
readPendingKnowledgeSelection(agent_id),
|
||||
);
|
||||
const pendingAssistantKnowledgeSelection = useRef(
|
||||
readPendingAssistantKnowledgeSelection(agent_id),
|
||||
);
|
||||
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>(
|
||||
() => pendingKnowledgeSelection.current ?? [],
|
||||
);
|
||||
const [selectedAssistantKnowledgeBaseIds, setSelectedAssistantKnowledgeBaseIds] = useState<string[]>(
|
||||
() => pendingAssistantKnowledgeSelection.current ?? [],
|
||||
);
|
||||
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(
|
||||
() => pendingKnowledgeSelection.current !== null,
|
||||
() => pendingKnowledgeSelection.current !== null || pendingAssistantKnowledgeSelection.current !== null,
|
||||
);
|
||||
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
|
||||
const knowledgeSelectionThread = useRef<string | null>(null);
|
||||
@@ -285,6 +311,10 @@ export default function AgentChatPage({
|
||||
setSelectedKnowledgeBaseIds(ids);
|
||||
setKnowledgeSelectionReady(true);
|
||||
}, []);
|
||||
const changeAssistantKnowledgeSelection = useCallback((ids: string[]) => {
|
||||
setSelectedAssistantKnowledgeBaseIds(ids);
|
||||
setKnowledgeSelectionReady(true);
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
activeThreadIdRef.current = threadId;
|
||||
@@ -294,17 +324,23 @@ export default function AgentChatPage({
|
||||
if (knowledgeSelectionAgent.current === agent_id) return;
|
||||
knowledgeSelectionAgent.current = agent_id;
|
||||
const pending = readPendingKnowledgeSelection(agent_id);
|
||||
const pendingAssistant = readPendingAssistantKnowledgeSelection(agent_id);
|
||||
pendingKnowledgeSelection.current = pending;
|
||||
pendingAssistantKnowledgeSelection.current = pendingAssistant;
|
||||
knowledgeSelectionThread.current = null;
|
||||
setSelectedKnowledgeBaseIds(pending ?? []);
|
||||
setKnowledgeSelectionReady(pending !== null);
|
||||
setSelectedAssistantKnowledgeBaseIds(pendingAssistant ?? []);
|
||||
setKnowledgeSelectionReady(pending !== null || pendingAssistant !== null);
|
||||
}, [agent_id]);
|
||||
|
||||
useEffect(() => {
|
||||
if (isNewThread) {
|
||||
if (knowledgeSelectionThread.current !== "__new__") {
|
||||
setSelectedKnowledgeBaseIds(pendingKnowledgeSelection.current ?? []);
|
||||
setKnowledgeSelectionReady(pendingKnowledgeSelection.current !== null);
|
||||
setSelectedAssistantKnowledgeBaseIds(pendingAssistantKnowledgeSelection.current ?? []);
|
||||
setKnowledgeSelectionReady(
|
||||
pendingKnowledgeSelection.current !== null || pendingAssistantKnowledgeSelection.current !== null,
|
||||
);
|
||||
}
|
||||
knowledgeSelectionThread.current = "__new__";
|
||||
setKnowledgeSelectionLoading(false);
|
||||
@@ -319,18 +355,29 @@ export default function AgentChatPage({
|
||||
if (knowledgeSelectionThread.current !== threadId) return;
|
||||
const metadata = record.metadata as Record<string, unknown> | undefined;
|
||||
const raw = metadata?.llmwiki_knowledge_base_ids;
|
||||
const assistantRaw = metadata?.assistant_knowledge_base_ids;
|
||||
const llmwikiIds = Array.isArray(raw)
|
||||
? raw.map((item) => String(item).trim()).filter(Boolean)
|
||||
: [];
|
||||
const assistantIds = Array.isArray(assistantRaw)
|
||||
? assistantRaw.map((item) => String(item).trim()).filter(Boolean)
|
||||
: [];
|
||||
if (Array.isArray(raw)) {
|
||||
setSelectedKnowledgeBaseIds(
|
||||
raw.map((item) => String(item).trim()).filter(Boolean),
|
||||
);
|
||||
setKnowledgeSelectionReady(true);
|
||||
setSelectedKnowledgeBaseIds(llmwikiIds);
|
||||
} else {
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
}
|
||||
if (Array.isArray(assistantRaw)) {
|
||||
setSelectedAssistantKnowledgeBaseIds(assistantIds);
|
||||
} else {
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
}
|
||||
setKnowledgeSelectionReady(Array.isArray(raw) || Array.isArray(assistantRaw));
|
||||
})
|
||||
.catch(() => {
|
||||
if (knowledgeSelectionThread.current === threadId) {
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
}
|
||||
})
|
||||
@@ -353,10 +400,13 @@ export default function AgentChatPage({
|
||||
() => ({
|
||||
...(taskId ? taskThreadMetadata(taskContext) : {}),
|
||||
...(knowledgeSelectionReady
|
||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
||||
? {
|
||||
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
[knowledgeSelectionReady, selectedKnowledgeBaseIds, taskContext, taskId],
|
||||
[knowledgeSelectionReady, selectedAssistantKnowledgeBaseIds, selectedKnowledgeBaseIds, taskContext, taskId],
|
||||
);
|
||||
const [thread, sendMessage] = useThreadStream({
|
||||
threadId: isNewThread ? undefined : threadId,
|
||||
@@ -365,7 +415,10 @@ export default function AgentChatPage({
|
||||
...settings.context,
|
||||
agent_id,
|
||||
...(knowledgeSelectionReady
|
||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
||||
? {
|
||||
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
// 任务闸门:新建 thread 打 metadata.taskId 标签,供任务对话列表(1:N)按任务过滤;
|
||||
@@ -713,9 +766,13 @@ export default function AgentChatPage({
|
||||
source="agent"
|
||||
agentId={agent_id}
|
||||
value={selectedKnowledgeBaseIds}
|
||||
assistantValue={selectedAssistantKnowledgeBaseIds}
|
||||
onChange={changeKnowledgeSelection}
|
||||
onAssistantChange={changeAssistantKnowledgeSelection}
|
||||
autoSelectAll={
|
||||
isNewThread && pendingKnowledgeSelection.current === null
|
||||
isNewThread &&
|
||||
pendingKnowledgeSelection.current === null &&
|
||||
pendingAssistantKnowledgeSelection.current === null
|
||||
}
|
||||
disabled={isDemoMode || knowledgeSelectionLoading}
|
||||
label="知识空间"
|
||||
@@ -781,7 +838,9 @@ export default function AgentChatPage({
|
||||
source="agent"
|
||||
agentId={agent_id}
|
||||
value={selectedKnowledgeBaseIds}
|
||||
assistantValue={selectedAssistantKnowledgeBaseIds}
|
||||
onChange={changeKnowledgeSelection}
|
||||
onAssistantChange={changeAssistantKnowledgeSelection}
|
||||
disabled={isDemoMode || knowledgeSelectionLoading}
|
||||
readOnly
|
||||
label="知识空间"
|
||||
|
||||
@@ -3,12 +3,15 @@
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import {
|
||||
ArrowDownWideNarrow,
|
||||
Bot,
|
||||
ChevronLeft,
|
||||
ChevronRight,
|
||||
Download,
|
||||
FileText,
|
||||
FileX2,
|
||||
Library,
|
||||
Loader2,
|
||||
MessagesSquare,
|
||||
MessageSquareShare,
|
||||
RefreshCw,
|
||||
Search,
|
||||
@@ -20,17 +23,28 @@ import { toast } from "sonner";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select";
|
||||
import { Skeleton } from "@/components/ui/skeleton";
|
||||
import {
|
||||
WorkspaceBody,
|
||||
WorkspaceContainer,
|
||||
WorkspaceHeader,
|
||||
} from "@/components/workspace/workspace-container";
|
||||
import { downloadArtifactFile } from "@/core/artifacts/download";
|
||||
import {
|
||||
downloadArtifactFile,
|
||||
downloadArtifactMarkdownAsWord,
|
||||
} from "@/core/artifacts/download";
|
||||
import {
|
||||
fetchArtifactLibrary,
|
||||
type ArtifactLibraryItem,
|
||||
type ArtifactLibraryKind,
|
||||
type ArtifactLibrarySource,
|
||||
} from "@/core/artifacts/library";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import { bootstrapResumePath, pathOfThread } from "@/core/threads/utils";
|
||||
@@ -53,6 +67,28 @@ const FILTERS: Array<{ value: ArtifactLibraryKind; label: string }> = [
|
||||
{ value: "other", label: "其他" },
|
||||
];
|
||||
|
||||
const SOURCE_FILTERS: Array<{
|
||||
value: ArtifactLibrarySource;
|
||||
label: string;
|
||||
}> = [
|
||||
{ value: "all", label: "全部来源" },
|
||||
{ value: "general_qa", label: "通用问答" },
|
||||
{ value: "agent", label: "智能体" },
|
||||
];
|
||||
|
||||
type DownloadFormat = "original" | "word";
|
||||
|
||||
function isMarkdownArtifact(name: string): boolean {
|
||||
return /\.(md|markdown)$/i.test(name);
|
||||
}
|
||||
|
||||
function artifactDownloadKey(
|
||||
item: ArtifactLibraryItem,
|
||||
format: DownloadFormat,
|
||||
): string {
|
||||
return `${item.threadId}:${item.path}:${format}`;
|
||||
}
|
||||
|
||||
function formatBytes(bytes: number): string {
|
||||
if (!Number.isFinite(bytes) || bytes <= 0) return "0 B";
|
||||
const units = ["B", "KB", "MB", "GB"];
|
||||
@@ -83,12 +119,19 @@ function FileRow({
|
||||
onDownload,
|
||||
}: {
|
||||
item: ArtifactLibraryItem;
|
||||
downloading: boolean;
|
||||
onDownload: (item: ArtifactLibraryItem) => void;
|
||||
downloading: DownloadFormat | null;
|
||||
onDownload: (item: ArtifactLibraryItem, format: DownloadFormat) => void;
|
||||
}) {
|
||||
const sourcePath = sourceConversationPath(item);
|
||||
const displayName = desensitize(item.name);
|
||||
const sourceTitle = desensitize(item.threadTitle || "新对话");
|
||||
const sourceLabel = item.sourceType === "agent" ? "智能体" : "通用问答";
|
||||
const sourceName =
|
||||
item.sourceType === "agent"
|
||||
? desensitize(item.agentName || item.agentId || "未知智能体")
|
||||
: sourceLabel;
|
||||
const SourceIcon = item.sourceType === "agent" ? Bot : MessagesSquare;
|
||||
const busy = downloading !== null;
|
||||
|
||||
return (
|
||||
<tr className="group border-b transition-colors hover:bg-muted/35">
|
||||
@@ -106,7 +149,8 @@ function FileRow({
|
||||
{displayName}
|
||||
</span>
|
||||
<span className="mt-1 block truncate text-xs text-muted-foreground lg:hidden">
|
||||
{getFileExtensionDisplayName(item.name)} · 来自「{sourceTitle}」
|
||||
{getFileExtensionDisplayName(item.name)} · {sourceName} · 来自「
|
||||
{sourceTitle}」
|
||||
</span>
|
||||
<span className="mt-1 hidden text-xs text-muted-foreground lg:block">
|
||||
{getFileExtensionDisplayName(item.name)} 文件
|
||||
@@ -115,13 +159,24 @@ function FileRow({
|
||||
</Link>
|
||||
</td>
|
||||
<td className="hidden min-w-0 px-3 py-3 lg:table-cell">
|
||||
<Link
|
||||
href={sourcePath}
|
||||
className="block truncate text-sm text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
|
||||
title={`打开原对话:${sourceTitle}`}
|
||||
>
|
||||
{sourceTitle}
|
||||
</Link>
|
||||
<div className="flex min-w-0 items-center gap-2.5">
|
||||
<span className="flex size-8 shrink-0 items-center justify-center rounded-md bg-muted text-muted-foreground">
|
||||
<SourceIcon className="size-4" />
|
||||
</span>
|
||||
<span className="min-w-0">
|
||||
<span className="block truncate text-sm font-medium text-foreground">
|
||||
{sourceName}
|
||||
</span>
|
||||
<Link
|
||||
href={sourcePath}
|
||||
className="block truncate text-xs text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
|
||||
title={`打开原对话:${sourceTitle}`}
|
||||
>
|
||||
{item.sourceType === "agent" ? "智能体 · " : ""}
|
||||
{sourceTitle}
|
||||
</Link>
|
||||
</span>
|
||||
</div>
|
||||
</td>
|
||||
<td className="whitespace-nowrap px-3 py-3 text-sm text-muted-foreground">
|
||||
{formatDateTime(item.modifiedAt)}
|
||||
@@ -136,17 +191,36 @@ function FileRow({
|
||||
size="icon"
|
||||
variant="ghost"
|
||||
className="size-8"
|
||||
disabled={downloading}
|
||||
onClick={() => onDownload(item)}
|
||||
title={`下载 ${displayName}`}
|
||||
aria-label={`下载 ${displayName}`}
|
||||
disabled={busy}
|
||||
onClick={() => onDownload(item, "original")}
|
||||
title={`下载原文件:${displayName}`}
|
||||
aria-label={`下载原文件:${displayName}`}
|
||||
>
|
||||
{downloading ? (
|
||||
{downloading === "original" ? (
|
||||
<Loader2 className="size-4 animate-spin" />
|
||||
) : (
|
||||
<Download className="size-4" />
|
||||
)}
|
||||
</Button>
|
||||
{isMarkdownArtifact(item.name) && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="ghost"
|
||||
className="h-8 gap-1 px-2 text-xs"
|
||||
disabled={busy}
|
||||
onClick={() => onDownload(item, "word")}
|
||||
title={`下载为 Word:${displayName}`}
|
||||
aria-label={`下载为 Word:${displayName}`}
|
||||
>
|
||||
{downloading === "word" ? (
|
||||
<Loader2 className="size-4 animate-spin" />
|
||||
) : (
|
||||
<FileText className="size-4" />
|
||||
)}
|
||||
Word
|
||||
</Button>
|
||||
)}
|
||||
<Button
|
||||
asChild
|
||||
size="icon"
|
||||
@@ -189,6 +263,7 @@ export default function ArtifactLibraryPage() {
|
||||
const [search, setSearch] = useState("");
|
||||
const [debouncedSearch, setDebouncedSearch] = useState("");
|
||||
const [fileKind, setFileKind] = useState<ArtifactLibraryKind>("all");
|
||||
const [sourceType, setSourceType] = useState<ArtifactLibrarySource>("all");
|
||||
const [page, setPage] = useState(1);
|
||||
const [downloadingKey, setDownloadingKey] = useState<string | null>(null);
|
||||
const forceRefreshRef = useRef(false);
|
||||
@@ -204,17 +279,18 @@ export default function ArtifactLibraryPage() {
|
||||
|
||||
useEffect(() => {
|
||||
setPage(1);
|
||||
}, [debouncedSearch, fileKind]);
|
||||
}, [debouncedSearch, fileKind, sourceType]);
|
||||
|
||||
const offset = (page - 1) * PAGE_SIZE;
|
||||
const libraryQuery = useQuery({
|
||||
queryKey: ["artifact-library", debouncedSearch, fileKind, page],
|
||||
queryKey: ["artifact-library", debouncedSearch, fileKind, sourceType, page],
|
||||
queryFn: async ({ signal }) => {
|
||||
const refresh = forceRefreshRef.current;
|
||||
try {
|
||||
return await fetchArtifactLibrary({
|
||||
query: debouncedSearch,
|
||||
fileKind,
|
||||
sourceType,
|
||||
offset,
|
||||
limit: PAGE_SIZE,
|
||||
refresh,
|
||||
@@ -238,19 +314,39 @@ export default function ArtifactLibraryPage() {
|
||||
}, [page, totalPages]);
|
||||
|
||||
const itemKeys = useMemo(
|
||||
() => new Set(items.map((item) => `${item.threadId}:${item.path}`)),
|
||||
() =>
|
||||
new Set(
|
||||
items.flatMap((item) => [
|
||||
artifactDownloadKey(item, "original"),
|
||||
artifactDownloadKey(item, "word"),
|
||||
]),
|
||||
),
|
||||
[items],
|
||||
);
|
||||
useEffect(() => {
|
||||
if (downloadingKey && !itemKeys.has(downloadingKey)) setDownloadingKey(null);
|
||||
}, [downloadingKey, itemKeys]);
|
||||
|
||||
const handleDownload = async (item: ArtifactLibraryItem) => {
|
||||
const key = `${item.threadId}:${item.path}`;
|
||||
const handleDownload = async (
|
||||
item: ArtifactLibraryItem,
|
||||
format: DownloadFormat,
|
||||
) => {
|
||||
const key = artifactDownloadKey(item, format);
|
||||
if (downloadingKey) return;
|
||||
setDownloadingKey(key);
|
||||
try {
|
||||
await downloadArtifactFile({ filepath: item.path, threadId: item.threadId });
|
||||
if (format === "word") {
|
||||
await downloadArtifactMarkdownAsWord({
|
||||
filepath: item.path,
|
||||
threadId: item.threadId,
|
||||
});
|
||||
toast.success("Word 文件已生成");
|
||||
} else {
|
||||
await downloadArtifactFile({
|
||||
filepath: item.path,
|
||||
threadId: item.threadId,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(error);
|
||||
toast.error(error instanceof Error ? error.message : "下载失败");
|
||||
@@ -298,26 +394,49 @@ export default function ArtifactLibraryPage() {
|
||||
<Input
|
||||
type="search"
|
||||
className="h-11 pl-9"
|
||||
placeholder="搜索文件名或来源对话"
|
||||
placeholder="搜索文件名、来源对话或智能体名称"
|
||||
value={search}
|
||||
onChange={(event) => setSearch(event.target.value)}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="flex flex-wrap items-center justify-between gap-3">
|
||||
<div className="flex flex-wrap gap-1" aria-label="文件类型筛选">
|
||||
{FILTERS.map((filter) => (
|
||||
<Button
|
||||
key={filter.value}
|
||||
type="button"
|
||||
<div className="flex flex-wrap items-center justify-between gap-4">
|
||||
<div className="flex flex-wrap items-center gap-3">
|
||||
<div className="flex flex-wrap gap-1" aria-label="文件类型筛选">
|
||||
{FILTERS.map((filter) => (
|
||||
<Button
|
||||
key={filter.value}
|
||||
type="button"
|
||||
size="sm"
|
||||
variant={fileKind === filter.value ? "secondary" : "ghost"}
|
||||
aria-pressed={fileKind === filter.value}
|
||||
onClick={() => setFileKind(filter.value)}
|
||||
>
|
||||
{filter.label}
|
||||
</Button>
|
||||
))}
|
||||
</div>
|
||||
<Select
|
||||
value={sourceType}
|
||||
onValueChange={(value) =>
|
||||
setSourceType(value as ArtifactLibrarySource)
|
||||
}
|
||||
>
|
||||
<SelectTrigger
|
||||
size="sm"
|
||||
variant={fileKind === filter.value ? "secondary" : "ghost"}
|
||||
aria-pressed={fileKind === filter.value}
|
||||
onClick={() => setFileKind(filter.value)}
|
||||
className="w-32"
|
||||
aria-label="筛选文件来源"
|
||||
>
|
||||
{filter.label}
|
||||
</Button>
|
||||
))}
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{SOURCE_FILTERS.map((filter) => (
|
||||
<SelectItem key={filter.value} value={filter.value}>
|
||||
{filter.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
||||
<ArrowDownWideNarrow className="size-4" />
|
||||
@@ -337,11 +456,11 @@ export default function ArtifactLibraryPage() {
|
||||
<table className="w-full table-fixed border-collapse">
|
||||
<thead>
|
||||
<tr className="border-b text-left text-xs font-normal text-muted-foreground">
|
||||
<th className="w-[48%] py-3 pr-4 pl-1 font-normal lg:w-[38%]">文件名</th>
|
||||
<th className="hidden w-[27%] px-3 py-3 font-normal lg:table-cell">来源对话</th>
|
||||
<th className="w-[34%] px-3 py-3 font-normal md:w-[22%] lg:w-[17%]">生成时间</th>
|
||||
<th className="hidden w-[13%] px-3 py-3 font-normal md:table-cell">大小</th>
|
||||
<th className="w-[18%] py-3 pr-1 pl-3 text-right font-normal md:w-[10%]">
|
||||
<th className="w-[48%] py-3 pr-4 pl-1 font-normal lg:w-[35%]">文件名</th>
|
||||
<th className="hidden w-[25%] px-3 py-3 font-normal lg:table-cell">来源</th>
|
||||
<th className="w-[34%] px-3 py-3 font-normal md:w-[22%] lg:w-[16%]">生成时间</th>
|
||||
<th className="hidden w-[13%] px-3 py-3 font-normal lg:w-[10%] md:table-cell">大小</th>
|
||||
<th className="w-[18%] py-3 pr-1 pl-3 text-right font-normal md:w-[17%] lg:w-[14%]">
|
||||
<span className="sr-only">操作</span>
|
||||
</th>
|
||||
</tr>
|
||||
@@ -352,12 +471,19 @@ export default function ArtifactLibraryPage() {
|
||||
) : (
|
||||
items.map((item) => {
|
||||
const key = `${item.threadId}:${item.path}`;
|
||||
const activeDownload = downloadingKey?.startsWith(`${key}:`)
|
||||
? downloadingKey.endsWith(":word")
|
||||
? "word"
|
||||
: "original"
|
||||
: null;
|
||||
return (
|
||||
<FileRow
|
||||
key={key}
|
||||
item={item}
|
||||
downloading={downloadingKey === key}
|
||||
onDownload={(target) => void handleDownload(target)}
|
||||
downloading={activeDownload}
|
||||
onDownload={(target, format) =>
|
||||
void handleDownload(target, format)
|
||||
}
|
||||
/>
|
||||
);
|
||||
})
|
||||
@@ -388,7 +514,7 @@ export default function ArtifactLibraryPage() {
|
||||
<FileX2 className="size-9 text-muted-foreground" />
|
||||
<div>
|
||||
<p className="text-sm font-medium">
|
||||
{debouncedSearch || fileKind !== "all"
|
||||
{debouncedSearch || fileKind !== "all" || sourceType !== "all"
|
||||
? "没有找到匹配的文件"
|
||||
: "还没有生成文件"}
|
||||
</p>
|
||||
|
||||
@@ -1,74 +1,448 @@
|
||||
import { Database, FileUp, History, Loader2, RefreshCw, Search, ShieldCheck, Trash2 } from "lucide-react";
|
||||
import { useMemo, useRef, useState } from "react";
|
||||
import { BookOpen, Database, Download, Loader2, Plus, RefreshCw, Search, ShieldCheck, Trash2, Upload } from "lucide-react";
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { toast } from "sonner";
|
||||
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
|
||||
import { MarkdownContent } from "@/components/workspace/messages/markdown-content";
|
||||
import { Textarea } from "@/components/ui/textarea";
|
||||
import { KnowledgeProductNav } from "@/components/workspace/knowledge-product-nav";
|
||||
import { MarkdownContent } from "@/components/workspace/messages/markdown-content";
|
||||
import { getStoredAuth } from "@/core/auth";
|
||||
import {
|
||||
listAssistantWikiRevisions,
|
||||
type AssistantKnowledgeBase,
|
||||
type AssistantWikiPage,
|
||||
type AssistantWikiRevision,
|
||||
useAssistantImportJobs,
|
||||
useAssistantKnowledgeActions,
|
||||
useAssistantKnowledgeBase,
|
||||
useAssistantKnowledgeBases,
|
||||
useAssistantKnowledgeSources,
|
||||
useAssistantWikiPages,
|
||||
} from "@/strategy-components/api/assistant-knowledge";
|
||||
|
||||
function baseSubtitle(base: AssistantKnowledgeBase) {
|
||||
if (base.is_global) return "全库沉淀总库 · 所有 WeKnora 导入数据都会同步进入这里并做实体对齐";
|
||||
return base.description || "自建助手知识库 · 可上传普通知识库导出的 Wiki 向量包";
|
||||
}
|
||||
|
||||
function EmptyInitializeCard({
|
||||
isAdmin,
|
||||
onInitialize,
|
||||
pending,
|
||||
}: {
|
||||
isAdmin: boolean;
|
||||
onInitialize: () => void;
|
||||
pending: boolean;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex flex-1 items-center justify-center p-8">
|
||||
<div className="max-w-lg rounded-xl border bg-card p-8 text-center">
|
||||
<Database className="mx-auto mb-4 size-10 text-primary" />
|
||||
<h1 className="text-xl font-semibold">初始化助手知识库</h1>
|
||||
<p className="mt-2 text-sm text-muted-foreground">
|
||||
未配置 WeKnora 时会默认展示助手知识库。首次使用需要创建“知识梳理总库”,后续从 WeKnora 导入的数据都会沉淀到这里。
|
||||
</p>
|
||||
{isAdmin ? (
|
||||
<Button className="mt-5" disabled={pending} onClick={onInitialize}>
|
||||
{pending ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Database className="mr-2 size-4" />}
|
||||
创建知识梳理总库
|
||||
</Button>
|
||||
) : (
|
||||
<p className="mt-5 text-sm">请联系管理员初始化。</p>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export default function AssistantKnowledgePage() {
|
||||
const isAdmin = getStoredAuth()?.system_role === "admin";
|
||||
const [query, setQuery] = useState("");
|
||||
const [selectedBaseId, setSelectedBaseId] = useState<string | null>(null);
|
||||
const [selected, setSelected] = useState<AssistantWikiPage | null>(null);
|
||||
const [revisions, setRevisions] = useState<AssistantWikiRevision[]>([]);
|
||||
const fileRef = useRef<HTMLInputElement>(null);
|
||||
const base = useAssistantKnowledgeBase();
|
||||
const pages = useAssistantWikiPages(query);
|
||||
const sources = useAssistantKnowledgeSources();
|
||||
const jobs = useAssistantImportJobs();
|
||||
const [createOpen, setCreateOpen] = useState(false);
|
||||
const [createDraft, setCreateDraft] = useState({ name: "", description: "" });
|
||||
const packageInputRef = useRef<HTMLInputElement | null>(null);
|
||||
const bases = useAssistantKnowledgeBases();
|
||||
const actions = useAssistantKnowledgeActions();
|
||||
|
||||
useEffect(() => {
|
||||
const values = bases.data ?? [];
|
||||
if (!values.length) {
|
||||
setSelectedBaseId(null);
|
||||
return;
|
||||
}
|
||||
if (selectedBaseId && values.some((base) => base.id === selectedBaseId)) return;
|
||||
setSelectedBaseId((values.find((base) => base.is_global) ?? values[0]).id);
|
||||
}, [bases.data, selectedBaseId]);
|
||||
|
||||
useEffect(() => {
|
||||
setSelected(null);
|
||||
}, [selectedBaseId]);
|
||||
|
||||
const selectedBase = useMemo(
|
||||
() => (bases.data ?? []).find((base) => base.id === selectedBaseId) ?? null,
|
||||
[bases.data, selectedBaseId],
|
||||
);
|
||||
const pages = useAssistantWikiPages(query, selectedBaseId);
|
||||
const sources = useAssistantKnowledgeSources(selectedBaseId);
|
||||
const jobs = useAssistantImportJobs(selectedBaseId);
|
||||
const activePage = selected ?? pages.data?.[0] ?? null;
|
||||
const counts = useMemo(
|
||||
() => ({
|
||||
pages: pages.data?.length ?? selectedBase?.page_count ?? 0,
|
||||
sources: sources.data?.length ?? selectedBase?.source_count ?? 0,
|
||||
jobs: jobs.data?.length ?? 0,
|
||||
}),
|
||||
[jobs.data, pages.data, selectedBase?.page_count, selectedBase?.source_count, sources.data],
|
||||
);
|
||||
|
||||
const counts = useMemo(() => ({ pages: pages.data?.length ?? 0, sources: sources.data?.length ?? 0, jobs: jobs.data?.length ?? 0 }), [pages.data, sources.data, jobs.data]);
|
||||
const initializeGlobal = () => {
|
||||
actions.initialize.mutate("知识梳理总库", {
|
||||
onSuccess: (base) => {
|
||||
setSelectedBaseId(base.id);
|
||||
toast.success("知识梳理总库已创建");
|
||||
},
|
||||
onError: (error) => toast.error(error.message),
|
||||
});
|
||||
};
|
||||
|
||||
if (base.isLoading) return <div className="flex size-full flex-col"><KnowledgeProductNav /><div className="flex flex-1 items-center justify-center"><Loader2 className="animate-spin" /></div></div>;
|
||||
if (!base.data) {
|
||||
return <div className="flex size-full flex-col"><KnowledgeProductNav /><div className="flex flex-1 items-center justify-center p-8"><div className="max-w-lg rounded-xl border bg-card p-8 text-center"><Database className="mx-auto mb-4 size-10 text-primary" /><h1 className="text-xl font-semibold">创建“知识梳理”助手知识库</h1><p className="mt-2 text-sm text-muted-foreground">无需配置普通知识库服务。创建后所有用户可查看 Wiki 与搜索,只有管理员可导入和维护。</p>{isAdmin ? <Button className="mt-5" disabled={actions.initialize.isPending} onClick={() => actions.initialize.mutate("知识梳理")}>创建知识梳理库</Button> : <p className="mt-5 text-sm">请联系管理员初始化。</p>}</div></div></div>;
|
||||
}
|
||||
|
||||
const uploadFiles = async (files: FileList | null) => {
|
||||
if (!files?.length) return;
|
||||
const createBase = async () => {
|
||||
const name = createDraft.name.trim();
|
||||
if (!name) {
|
||||
toast.error("请输入助手知识库名称");
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await actions.upload.mutateAsync(Array.from(files));
|
||||
toast.success("文件已安全扫描并导入");
|
||||
const base = await actions.createBase.mutateAsync({
|
||||
name,
|
||||
description: createDraft.description.trim(),
|
||||
});
|
||||
setSelectedBaseId(base.id);
|
||||
setCreateDraft({ name: "", description: "" });
|
||||
setCreateOpen(false);
|
||||
toast.success("助手知识库已创建");
|
||||
} catch (error) {
|
||||
toast.error(error instanceof Error ? error.message : String(error));
|
||||
} finally {
|
||||
if (fileRef.current) fileRef.current.value = "";
|
||||
}
|
||||
};
|
||||
|
||||
const openHistory = async () => {
|
||||
if (!activePage) return;
|
||||
setRevisions(await listAssistantWikiRevisions(activePage.slug));
|
||||
const uploadPackage = async (file: File | null | undefined) => {
|
||||
if (!file) return;
|
||||
try {
|
||||
await actions.uploadWeKnoraPackage.mutateAsync({
|
||||
file,
|
||||
assistantBaseId: selectedBaseId,
|
||||
});
|
||||
toast.success("导出包已上传,后台正在异步导入");
|
||||
} catch (error) {
|
||||
toast.error(error instanceof Error ? error.message : String(error));
|
||||
} finally {
|
||||
if (packageInputRef.current) packageInputRef.current.value = "";
|
||||
}
|
||||
};
|
||||
|
||||
if (bases.isLoading) {
|
||||
return (
|
||||
<div className="flex size-full flex-col">
|
||||
<KnowledgeProductNav />
|
||||
<div className="flex flex-1 items-center justify-center">
|
||||
<Loader2 className="animate-spin" />
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
if (!bases.data?.length) {
|
||||
return (
|
||||
<div className="flex size-full flex-col">
|
||||
<KnowledgeProductNav />
|
||||
<EmptyInitializeCard
|
||||
isAdmin={isAdmin}
|
||||
pending={actions.initialize.isPending}
|
||||
onInitialize={initializeGlobal}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex size-full min-h-0 flex-col bg-background">
|
||||
<KnowledgeProductNav />
|
||||
<header className="flex items-center gap-4 border-b px-6 py-4"><div><h1 className="text-xl font-semibold">{base.data.name}</h1><p className="text-xs text-muted-foreground">全局助手知识库 · 普通用户只读</p></div><Badge variant="outline"><ShieldCheck className="mr-1 size-3" />实体对齐与版本管理</Badge><div className="ml-auto flex gap-2">{isAdmin && <><input ref={fileRef} className="hidden" type="file" multiple onChange={(event) => void uploadFiles(event.target.files)} /><Button variant="outline" onClick={() => fileRef.current?.click()}><FileUp />导入本地文件</Button></>}<Button variant="outline" disabled={!activePage} onClick={() => void openHistory()}><History />页面版本</Button></div></header>
|
||||
<Tabs defaultValue="wiki" className="flex min-h-0 flex-1 flex-col">
|
||||
<div className="flex items-center gap-4 border-b px-6"><TabsList className="my-2"><TabsTrigger value="wiki">Wiki({counts.pages})</TabsTrigger><TabsTrigger value="sources">来源({counts.sources})</TabsTrigger><TabsTrigger value="jobs">导入记录({counts.jobs})</TabsTrigger><TabsTrigger value="revisions">版本({revisions.length})</TabsTrigger></TabsList><div className="relative ml-auto w-72"><Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" /><Input value={query} onChange={(event) => setQuery(event.target.value)} placeholder="搜索标题、实体别名和正文" className="pl-8" /></div></div>
|
||||
<TabsContent value="wiki" className="mt-0 grid min-h-0 flex-1 grid-cols-[300px_1fr]"><ScrollArea className="border-r"><div className="space-y-1 p-3">{pages.data?.map((page) => <button key={page.id} type="button" onClick={() => setSelected(page)} className={`w-full rounded-md px-3 py-2 text-left ${activePage?.id === page.id ? "bg-primary/10 text-primary" : "hover:bg-muted"}`}><div className="truncate text-sm font-medium">{page.title}</div><div className="mt-1 flex items-center gap-2 text-[11px] text-muted-foreground"><span>{page.page_type}</span><span>{new Date(page.updated_at).toLocaleString()}</span></div></button>)}</div></ScrollArea><ScrollArea><article className="mx-auto max-w-5xl p-8">{activePage ? <><h1 className="mb-1 text-2xl font-semibold">{activePage.title}</h1><p className="mb-6 text-xs text-muted-foreground">{activePage.slug}</p><MarkdownContent content={activePage.content_markdown} isLoading={false} rehypePlugins={[]} /></> : <p className="text-sm text-muted-foreground">暂无 Wiki 页面。</p>}</article></ScrollArea></TabsContent>
|
||||
<TabsContent value="sources" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">{sources.data?.map((source) => <div key={source.id} className="rounded-lg border bg-card p-4"><div className="flex items-center gap-2"><Database className="size-4" /><span className="font-medium">{source.source_name}</span><Badge variant="outline" className="ml-auto">{source.source_type}</Badge></div><p className="mt-3 text-xs text-muted-foreground">业务键:{source.source_key}</p>{isAdmin && <div className="mt-4 flex justify-end gap-2"><Button size="sm" variant="outline" disabled={source.source_type === "local_file" || actions.reimportSource.isPending} onClick={() => actions.reimportSource.mutate(source.id, { onSuccess: () => toast.success("已发起重新导入"), onError: (error) => toast.error(error.message) })}><RefreshCw />重新导入</Button><Button size="sm" variant="destructive" disabled={actions.deleteSource.isPending} onClick={() => { if (window.confirm(`确认删除来源“${source.source_name}”的贡献?仍被其他来源引用的知识会保留。`)) actions.deleteSource.mutate(source.id, { onSuccess: () => toast.success("来源贡献已删除"), onError: (error) => toast.error(error.message) }); }}><Trash2 />删除来源</Button></div>}</div>)}</div></TabsContent>
|
||||
<TabsContent value="jobs" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="space-y-3">{jobs.data?.map((job) => <div key={job.id} className="flex items-center gap-3 rounded-lg border bg-card p-4"><Badge>{job.status}</Badge><span className="text-sm">{job.trigger}</span><span className="text-xs text-muted-foreground">{Object.entries(job.counts || {}).map(([key, value]) => `${key} ${value}`).join(" · ")}</span><span className="ml-auto text-xs">{new Date(job.created_at).toLocaleString()}</span></div>)}</div></TabsContent>
|
||||
<TabsContent value="revisions" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="space-y-3">{revisions.map((revision) => <div key={revision.id} className="flex items-center gap-3 rounded-lg border bg-card p-4"><Badge variant="outline">v{revision.revision_no}</Badge><span className="text-sm">{revision.change_type}</span><span className="text-xs text-muted-foreground">{revision.summary}</span><span className="ml-auto text-xs">{new Date(revision.created_at).toLocaleString()}</span>{isAdmin && activePage && revision.id !== activePage.current_revision_id && <Button size="sm" variant="outline" onClick={() => actions.rollback.mutate({ slug: activePage.slug, revisionId: revision.id })}>回滚到此版本</Button>}</div>)}{!revisions.length && <p className="text-sm text-muted-foreground">请先在 Wiki 页选择页面并点击“页面版本”。</p>}</div></TabsContent>
|
||||
</Tabs>
|
||||
<div className="grid min-h-0 flex-1 grid-cols-[280px_1fr]">
|
||||
<aside className="flex min-h-0 flex-col border-r bg-muted/20">
|
||||
<div className="border-b p-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h2 className="text-sm font-semibold">助手知识库</h2>
|
||||
<p className="mt-1 text-xs text-muted-foreground">总库置顶,自建库可上传 WeKnora 导出包</p>
|
||||
</div>
|
||||
{isAdmin && (
|
||||
<Button size="icon" variant="outline" onClick={() => setCreateOpen(true)}>
|
||||
<Plus className="size-4" />
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<ScrollArea className="min-h-0 flex-1">
|
||||
<div className="space-y-2 p-3">
|
||||
{(bases.data ?? []).map((base) => (
|
||||
<button
|
||||
key={base.id}
|
||||
type="button"
|
||||
onClick={() => setSelectedBaseId(base.id)}
|
||||
className={`w-full rounded-lg border px-3 py-3 text-left transition ${
|
||||
selectedBaseId === base.id ? "border-primary bg-primary/10 text-primary" : "bg-card hover:bg-muted"
|
||||
}`}
|
||||
>
|
||||
<div className="flex items-center gap-2">
|
||||
<Database className="size-4 shrink-0" />
|
||||
<span className="min-w-0 flex-1 truncate text-sm font-medium">{base.name}</span>
|
||||
{base.is_global && <Badge variant="outline" className="text-[10px]">总库</Badge>}
|
||||
</div>
|
||||
<div className="mt-2 flex gap-3 text-[11px] text-muted-foreground">
|
||||
<span>{base.page_count ?? 0} 页</span>
|
||||
<span>{base.source_count ?? 0} 来源</span>
|
||||
<span>{base.entity_count ?? 0} 实体</span>
|
||||
</div>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</ScrollArea>
|
||||
</aside>
|
||||
|
||||
<main className="flex min-h-0 flex-col">
|
||||
<header className="flex items-center gap-4 border-b px-6 py-4">
|
||||
<div className="min-w-0">
|
||||
<h1 className="truncate text-xl font-semibold">{selectedBase?.name}</h1>
|
||||
<p className="mt-1 truncate text-xs text-muted-foreground">
|
||||
{selectedBase ? baseSubtitle(selectedBase) : "请选择助手知识库"}
|
||||
</p>
|
||||
</div>
|
||||
<Badge variant="outline">
|
||||
<ShieldCheck className="mr-1 size-3" />
|
||||
实体对齐
|
||||
</Badge>
|
||||
{isAdmin && (
|
||||
<div className="ml-auto flex items-center gap-2">
|
||||
<input
|
||||
ref={packageInputRef}
|
||||
type="file"
|
||||
accept=".jsonl,.ndjson,application/x-ndjson,application/json"
|
||||
className="hidden"
|
||||
onChange={(event) => void uploadPackage(event.target.files?.[0])}
|
||||
/>
|
||||
<Button
|
||||
variant="outline"
|
||||
disabled={actions.uploadWeKnoraPackage.isPending || !selectedBaseId}
|
||||
onClick={() => packageInputRef.current?.click()}
|
||||
>
|
||||
{actions.uploadWeKnoraPackage.isPending ? (
|
||||
<Loader2 className="mr-2 size-4 animate-spin" />
|
||||
) : (
|
||||
<Upload className="mr-2 size-4" />
|
||||
)}
|
||||
上传 WeKnora 导出包
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
</header>
|
||||
|
||||
<Tabs defaultValue="wiki" className="flex min-h-0 flex-1 flex-col">
|
||||
<div className="flex items-center gap-4 border-b px-6">
|
||||
<TabsList className="my-2">
|
||||
<TabsTrigger value="wiki">Wiki({counts.pages})</TabsTrigger>
|
||||
<TabsTrigger value="sources">来源({counts.sources})</TabsTrigger>
|
||||
<TabsTrigger value="jobs">导入记录({counts.jobs})</TabsTrigger>
|
||||
</TabsList>
|
||||
<div className="relative ml-auto w-72">
|
||||
<Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
|
||||
<Input
|
||||
value={query}
|
||||
onChange={(event) => setQuery(event.target.value)}
|
||||
placeholder="搜索标题、实体别名和正文"
|
||||
className="pl-8"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<TabsContent value="wiki" className="mt-0 grid min-h-0 flex-1 grid-cols-[300px_1fr]">
|
||||
<ScrollArea className="border-r">
|
||||
<div className="space-y-1 p-3">
|
||||
{pages.isLoading && <Loader2 className="m-4 size-5 animate-spin" />}
|
||||
{pages.data?.map((page) => (
|
||||
<button
|
||||
key={page.id}
|
||||
type="button"
|
||||
onClick={() => setSelected(page)}
|
||||
className={`w-full rounded-md px-3 py-2 text-left ${
|
||||
activePage?.id === page.id ? "bg-primary/10 text-primary" : "hover:bg-muted"
|
||||
}`}
|
||||
>
|
||||
<div className="truncate text-sm font-medium">{page.title}</div>
|
||||
<div className="mt-1 flex items-center gap-2 text-[11px] text-muted-foreground">
|
||||
<span>{page.page_type}</span>
|
||||
<span>{new Date(page.updated_at).toLocaleString()}</span>
|
||||
</div>
|
||||
</button>
|
||||
))}
|
||||
{!pages.isLoading && !pages.data?.length && (
|
||||
<div className="p-4 text-xs text-muted-foreground">
|
||||
暂无 Wiki 页面。请先到普通知识库详情页下载向量化数据包,再在当前助手知识库上传。
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</ScrollArea>
|
||||
<ScrollArea>
|
||||
<article className="mx-auto max-w-5xl p-8">
|
||||
{activePage ? (
|
||||
<>
|
||||
<h1 className="mb-1 flex items-center gap-2 text-2xl font-semibold">
|
||||
<BookOpen className="size-5 text-primary" />
|
||||
{activePage.title}
|
||||
</h1>
|
||||
<p className="mb-6 text-xs text-muted-foreground">{activePage.slug}</p>
|
||||
<MarkdownContent content={activePage.content_markdown} isLoading={false} rehypePlugins={[]} />
|
||||
</>
|
||||
) : (
|
||||
<p className="text-sm text-muted-foreground">暂无 Wiki 页面。</p>
|
||||
)}
|
||||
</article>
|
||||
</ScrollArea>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="sources" className="mt-0 min-h-0 flex-1 overflow-auto p-6">
|
||||
<div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">
|
||||
{sources.data?.map((source) => (
|
||||
<div key={source.id} className="rounded-lg border bg-card p-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<Database className="size-4" />
|
||||
<span className="truncate font-medium">{source.source_name}</span>
|
||||
<Badge variant="outline" className="ml-auto">{source.source_type}</Badge>
|
||||
</div>
|
||||
<p className="mt-3 text-xs text-muted-foreground">业务键:{source.source_key}</p>
|
||||
{isAdmin && (
|
||||
<div className="mt-4 flex justify-end gap-2">
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
disabled={source.source_type !== "weknora" || actions.reimportSource.isPending}
|
||||
onClick={() =>
|
||||
actions.reimportSource.mutate(source.id, {
|
||||
onSuccess: () => toast.success("已发起重新导入"),
|
||||
onError: (error) => toast.error(error.message),
|
||||
})
|
||||
}
|
||||
>
|
||||
<RefreshCw className="mr-2 size-4" />
|
||||
重新导入
|
||||
</Button>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="destructive"
|
||||
disabled={actions.deleteSource.isPending}
|
||||
onClick={() => {
|
||||
if (window.confirm(`确认删除来源“${source.source_name}”的贡献?仍被其他来源引用的知识会保留。`)) {
|
||||
actions.deleteSource.mutate(source.id, {
|
||||
onSuccess: () => toast.success("来源贡献已删除"),
|
||||
onError: (error) => toast.error(error.message),
|
||||
});
|
||||
}
|
||||
}}
|
||||
>
|
||||
<Trash2 className="mr-2 size-4" />
|
||||
删除来源
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
{!sources.data?.length && <p className="text-sm text-muted-foreground">暂无来源。</p>}
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="jobs" className="mt-0 min-h-0 flex-1 overflow-auto p-6">
|
||||
<div className="space-y-3">
|
||||
{jobs.data?.map((job) => (
|
||||
<div key={job.id} className="rounded-lg border bg-card p-4">
|
||||
<div className="flex items-center gap-3">
|
||||
<Badge>{job.status}</Badge>
|
||||
<span className="text-sm">{job.trigger}</span>
|
||||
<span className="text-xs text-muted-foreground">阶段:{job.phase}</span>
|
||||
<span className="ml-auto text-xs">{new Date(job.created_at).toLocaleString()}</span>
|
||||
{isAdmin && job.package_download_url && (
|
||||
<Button asChild size="sm" variant="outline">
|
||||
<a href={job.package_download_url} target="_blank" rel="noreferrer">
|
||||
<Download className="mr-2 size-4" />
|
||||
下载导出包
|
||||
</a>
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
<div className="mt-2 text-xs text-muted-foreground">
|
||||
{Object.entries(job.counts || {}).map(([key, value]) => `${key} ${value}`).join(" · ") || "暂无统计"}
|
||||
</div>
|
||||
{job.last_error_message && (
|
||||
<div className="mt-2 rounded-md bg-destructive/10 px-3 py-2 text-xs text-destructive">
|
||||
{job.last_error_message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
{!jobs.data?.length && <p className="text-sm text-muted-foreground">暂无导入记录。</p>}
|
||||
</div>
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
</main>
|
||||
</div>
|
||||
|
||||
<Dialog open={createOpen} onOpenChange={setCreateOpen}>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
<DialogTitle>新建助手知识库</DialogTitle>
|
||||
<DialogDescription>
|
||||
自建助手知识库用于承接从 WeKnora 普通知识库下载的 Wiki 向量包;导入时同一份数据也会沉淀到“知识梳理总库”。
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<div className="space-y-4">
|
||||
<div className="space-y-1.5">
|
||||
<label className="text-sm font-medium">名称</label>
|
||||
<Input
|
||||
value={createDraft.name}
|
||||
onChange={(event) => setCreateDraft((prev) => ({ ...prev, name: event.target.value }))}
|
||||
placeholder="例如:政策资料库"
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-1.5">
|
||||
<label className="text-sm font-medium">描述</label>
|
||||
<Textarea
|
||||
value={createDraft.description}
|
||||
onChange={(event) => setCreateDraft((prev) => ({ ...prev, description: event.target.value }))}
|
||||
placeholder="可选,用于说明这个助手知识库的范围"
|
||||
rows={3}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<DialogFooter>
|
||||
<Button variant="outline" onClick={() => setCreateOpen(false)}>
|
||||
取消
|
||||
</Button>
|
||||
<Button disabled={actions.createBase.isPending} onClick={() => void createBase()}>
|
||||
{actions.createBase.isPending ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Plus className="mr-2 size-4" />}
|
||||
创建
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -132,6 +132,7 @@ export default function ChatPage() {
|
||||
// per-thread header toggle so stale localStorage can't re-enable it.
|
||||
const [knowledgeRagEnabled] = useKnowledgeRagEnabled();
|
||||
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>([]);
|
||||
const [selectedAssistantKnowledgeBaseIds, setSelectedAssistantKnowledgeBaseIds] = useState<string[]>([]);
|
||||
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(false);
|
||||
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
|
||||
const knowledgeSelectionThread = useRef<string | null>(null);
|
||||
@@ -140,18 +141,26 @@ export default function ChatPage() {
|
||||
setSelectedKnowledgeBaseIds(ids);
|
||||
setKnowledgeSelectionReady(true);
|
||||
}, []);
|
||||
const changeAssistantKnowledgeSelection = useCallback((ids: string[]) => {
|
||||
setSelectedAssistantKnowledgeBaseIds(ids);
|
||||
setKnowledgeSelectionReady(true);
|
||||
}, []);
|
||||
const streamContext = useMemo(
|
||||
() => ({
|
||||
...settings.context,
|
||||
knowledge_rag_enabled: knowledgeRagEnabled,
|
||||
...(knowledgeSelectionReady
|
||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
||||
? {
|
||||
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
[
|
||||
settings.context,
|
||||
knowledgeRagEnabled,
|
||||
knowledgeSelectionReady,
|
||||
selectedAssistantKnowledgeBaseIds,
|
||||
selectedKnowledgeBaseIds,
|
||||
],
|
||||
);
|
||||
@@ -190,6 +199,7 @@ export default function ChatPage() {
|
||||
setHasStartedConversation(false);
|
||||
setPromptPrefixOn(false);
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
setKnowledgeSelectionLoading(false);
|
||||
knowledgeSelectionThread.current = "__new__";
|
||||
@@ -215,18 +225,29 @@ export default function ChatPage() {
|
||||
if (knowledgeSelectionThread.current !== threadId) return;
|
||||
const metadata = record.metadata as Record<string, unknown> | undefined;
|
||||
const raw = metadata?.llmwiki_knowledge_base_ids;
|
||||
const assistantRaw = metadata?.assistant_knowledge_base_ids;
|
||||
const llmwikiIds = Array.isArray(raw)
|
||||
? raw.map((item) => String(item).trim()).filter(Boolean)
|
||||
: [];
|
||||
const assistantIds = Array.isArray(assistantRaw)
|
||||
? assistantRaw.map((item) => String(item).trim()).filter(Boolean)
|
||||
: [];
|
||||
if (Array.isArray(raw)) {
|
||||
setSelectedKnowledgeBaseIds(
|
||||
raw.map((item) => String(item).trim()).filter(Boolean),
|
||||
);
|
||||
setKnowledgeSelectionReady(true);
|
||||
setSelectedKnowledgeBaseIds(llmwikiIds);
|
||||
} else {
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
}
|
||||
if (Array.isArray(assistantRaw)) {
|
||||
setSelectedAssistantKnowledgeBaseIds(assistantIds);
|
||||
} else {
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
}
|
||||
setKnowledgeSelectionReady(Array.isArray(raw) || Array.isArray(assistantRaw));
|
||||
})
|
||||
.catch(() => {
|
||||
if (knowledgeSelectionThread.current === threadId) {
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
}
|
||||
})
|
||||
@@ -249,6 +270,7 @@ export default function ChatPage() {
|
||||
if (!known.some((a) => a.id === selectedAgentId)) {
|
||||
setSelectedAgentId(null);
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
}
|
||||
}, [selectedAgentId, selectorAgents, mineAgents, squareAgents, selectorLoading]);
|
||||
@@ -259,7 +281,10 @@ export default function ChatPage() {
|
||||
threadId: isNewThread ? undefined : threadId,
|
||||
context: streamContext,
|
||||
threadMetadata: knowledgeSelectionReady
|
||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
||||
? {
|
||||
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||
}
|
||||
: undefined,
|
||||
isMock,
|
||||
onStart: (createdThreadId) => {
|
||||
@@ -434,7 +459,10 @@ export default function ChatPage() {
|
||||
metadata: {
|
||||
agent_id: selectedAgentId,
|
||||
...(knowledgeSelectionReady
|
||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
||||
? {
|
||||
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
});
|
||||
@@ -445,7 +473,10 @@ export default function ChatPage() {
|
||||
threadId: created.thread_id,
|
||||
text,
|
||||
...(knowledgeSelectionReady
|
||||
? { knowledgeBaseIds: selectedKnowledgeBaseIds }
|
||||
? {
|
||||
knowledgeBaseIds: selectedKnowledgeBaseIds,
|
||||
assistantKnowledgeBaseIds: selectedAssistantKnowledgeBaseIds,
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
);
|
||||
@@ -505,6 +536,7 @@ export default function ChatPage() {
|
||||
hideAdminPrompt,
|
||||
preferencesLoading,
|
||||
selectedAgentId,
|
||||
selectedAssistantKnowledgeBaseIds,
|
||||
selectedKnowledgeBaseIds,
|
||||
knowledgeSelectionReady,
|
||||
agentHandoffPending,
|
||||
@@ -608,6 +640,7 @@ export default function ChatPage() {
|
||||
const handleAgentChange = useCallback((agentId: string | null) => {
|
||||
setSelectedAgentId(agentId);
|
||||
setSelectedKnowledgeBaseIds([]);
|
||||
setSelectedAssistantKnowledgeBaseIds([]);
|
||||
setKnowledgeSelectionReady(false);
|
||||
saveLastAgentId(agentId);
|
||||
}, []);
|
||||
@@ -842,7 +875,9 @@ export default function ChatPage() {
|
||||
source={selectedAgentId ? "agent" : "all"}
|
||||
agentId={selectedAgentId}
|
||||
value={selectedKnowledgeBaseIds}
|
||||
assistantValue={selectedAssistantKnowledgeBaseIds}
|
||||
onChange={changeKnowledgeSelection}
|
||||
onAssistantChange={changeAssistantKnowledgeSelection}
|
||||
autoSelectAll={isLandingState && !!selectedAgentId}
|
||||
disabled={
|
||||
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" ||
|
||||
|
||||
@@ -3,7 +3,11 @@ import { useState } from "react";
|
||||
import { Navigate, useParams, useSearchParams } from "react-router-dom";
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { KnowledgeProductNav, useKnowledgeBaseProductVersion } from "@/components/workspace/knowledge-product-nav";
|
||||
import {
|
||||
KNOWLEDGE_NORMAL_VIEW_VALUE,
|
||||
KnowledgeProductNav,
|
||||
useKnowledgeBaseProductVersion,
|
||||
} from "@/components/workspace/knowledge-product-nav";
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
@@ -73,6 +77,7 @@ export default function LlmWikiKnowledgePage({
|
||||
const runtime = useLlmWikiRuntime();
|
||||
const productVersion = useKnowledgeBaseProductVersion();
|
||||
const wikiSlug = searchParams.get("wikiSlug")?.trim() || "";
|
||||
const explicitNormalView = searchParams.get("view") === KNOWLEDGE_NORMAL_VIEW_VALUE;
|
||||
|
||||
if (mode === "detail" && noteId && wikiSlug) {
|
||||
return <Navigate to={buildWeKnoraWikiPagePath(noteId, wikiSlug)} replace />;
|
||||
@@ -82,15 +87,18 @@ export default function LlmWikiKnowledgePage({
|
||||
return <RuntimeLoading />;
|
||||
}
|
||||
|
||||
if (mode === "list" && productVersion.data === "v2") {
|
||||
if (mode === "list" && productVersion.data === "v2" && !explicitNormalView) {
|
||||
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
|
||||
}
|
||||
|
||||
if (runtime.isError || runtime.data?.provider !== "weknora") {
|
||||
if (mode === "list" || mode === "detail") {
|
||||
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
|
||||
}
|
||||
if (mode === "graph") {
|
||||
return <KnowledgeGraphPage />;
|
||||
}
|
||||
return mode === "list" ? <div className="flex size-full min-h-0 flex-col"><KnowledgeProductNav /><div className="min-h-0 flex-1"><LegacyKnowledgePage /></div></div> : <LegacyKnowledgePage />;
|
||||
return <LegacyKnowledgePage />;
|
||||
}
|
||||
|
||||
if (mode === "graph") {
|
||||
|
||||
@@ -450,7 +450,13 @@ export default function WorkspaceRoutes({
|
||||
<Route
|
||||
path="library"
|
||||
element={
|
||||
<WorkspaceLayout {...layoutProps} workspaceSidebarHidden>
|
||||
<WorkspaceLayout
|
||||
{...layoutProps}
|
||||
workspaceSidebarHidden={
|
||||
layoutProps.workspaceSidebarHidden ||
|
||||
!layoutProps.pageSidebarHidden
|
||||
}
|
||||
>
|
||||
<ArtifactLibraryPage />
|
||||
</WorkspaceLayout>
|
||||
}
|
||||
|
||||
@@ -8,13 +8,21 @@ const BASE = () => `${getBackendBaseURL()}/api/assistant-knowledge`;
|
||||
export interface AssistantKnowledgeBase {
|
||||
id: string;
|
||||
name: string;
|
||||
description?: string;
|
||||
base_kind?: "global" | "custom";
|
||||
is_global?: boolean;
|
||||
status: string;
|
||||
source_count?: number;
|
||||
page_count?: number;
|
||||
entity_count?: number;
|
||||
relation_count?: number;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
export interface AssistantWikiPage {
|
||||
id: string;
|
||||
base_id: string;
|
||||
slug: string;
|
||||
title: string;
|
||||
page_type: string;
|
||||
@@ -37,6 +45,7 @@ export interface AssistantWikiRevision {
|
||||
|
||||
export interface AssistantKnowledgeSource {
|
||||
id: string;
|
||||
base_id: string;
|
||||
source_type: "weknora" | "skill" | "local_file";
|
||||
source_name: string;
|
||||
source_key: string;
|
||||
@@ -46,12 +55,34 @@ export interface AssistantKnowledgeSource {
|
||||
|
||||
export interface AssistantImportJob {
|
||||
id: string;
|
||||
base_id: string;
|
||||
source_id: string;
|
||||
mirrored_global_base_id?: string;
|
||||
mirrored_global_job_id?: string;
|
||||
package_download_url?: string;
|
||||
trigger: string;
|
||||
status: string;
|
||||
phase: string;
|
||||
counts: Record<string, number>;
|
||||
last_error_message?: string | null;
|
||||
created_at: string;
|
||||
completed_at?: string | null;
|
||||
}
|
||||
|
||||
export interface LlmWikiExportPackageJob {
|
||||
id: string;
|
||||
mapping_id: string;
|
||||
status: string;
|
||||
phase: string;
|
||||
counts: Record<string, number>;
|
||||
package_download_url?: string;
|
||||
status_url?: string;
|
||||
error?: string | null;
|
||||
created_at?: string | null;
|
||||
updated_at?: string | null;
|
||||
completed_at?: string | null;
|
||||
file_exists?: boolean;
|
||||
file_size_bytes?: number;
|
||||
}
|
||||
|
||||
async function readJson<T>(response: Response): Promise<T> {
|
||||
@@ -65,12 +96,41 @@ async function readJson<T>(response: Response): Promise<T> {
|
||||
return (await response.json()) as T;
|
||||
}
|
||||
|
||||
function normalizeImportJob(job: AssistantImportJob): AssistantImportJob {
|
||||
const url = normalizeBackendUrl(job.package_download_url);
|
||||
return url ? { ...job, package_download_url: url } : job;
|
||||
}
|
||||
|
||||
function normalizeBackendUrl(url: string | undefined | null): string | undefined {
|
||||
if (!url || /^https?:\/\//i.test(url)) return url || undefined;
|
||||
return `${getBackendBaseURL()}${url.startsWith("/") ? url : `/${url}`}`;
|
||||
}
|
||||
|
||||
function normalizeExportPackageJob(job: LlmWikiExportPackageJob): LlmWikiExportPackageJob {
|
||||
return {
|
||||
...job,
|
||||
package_download_url: normalizeBackendUrl(job.package_download_url),
|
||||
status_url: normalizeBackendUrl(job.status_url),
|
||||
};
|
||||
}
|
||||
|
||||
export async function getAssistantKnowledgeBase() {
|
||||
const data = await readJson<{ base: AssistantKnowledgeBase | null }>(await apiFetch(`${BASE()}/base`));
|
||||
return data.base;
|
||||
}
|
||||
|
||||
export async function initializeAssistantKnowledgeBase(name = "知识梳理") {
|
||||
export async function getAssistantKnowledgeBaseById(baseId: string) {
|
||||
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
||||
await apiFetch(`${BASE()}/bases/${encodeURIComponent(baseId)}`),
|
||||
);
|
||||
return data.base;
|
||||
}
|
||||
|
||||
export async function listAssistantKnowledgeBases() {
|
||||
return (await readJson<{ bases: AssistantKnowledgeBase[] }>(await apiFetch(`${BASE()}/bases`))).bases;
|
||||
}
|
||||
|
||||
export async function initializeAssistantKnowledgeBase(name = "知识梳理总库") {
|
||||
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
||||
await apiFetch(`${BASE()}/base/initialize`, {
|
||||
method: "POST",
|
||||
@@ -81,58 +141,124 @@ export async function initializeAssistantKnowledgeBase(name = "知识梳理") {
|
||||
return data.base;
|
||||
}
|
||||
|
||||
export async function listAssistantWikiPages(q = "") {
|
||||
export async function createAssistantKnowledgeBase(input: { name: string; description?: string }) {
|
||||
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
||||
await apiFetch(`${BASE()}/bases`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(input),
|
||||
}),
|
||||
);
|
||||
return data.base;
|
||||
}
|
||||
|
||||
function withBaseParam(baseId?: string | null) {
|
||||
const params = new URLSearchParams();
|
||||
if (baseId) params.set("base_id", baseId);
|
||||
return params;
|
||||
}
|
||||
|
||||
export async function listAssistantWikiPages(q = "", baseId?: string | null) {
|
||||
const params = withBaseParam(baseId);
|
||||
if (q.trim()) params.set("q", q.trim());
|
||||
return (await readJson<{ pages: AssistantWikiPage[] }>(await apiFetch(`${BASE()}/wiki-pages?${params}`))).pages;
|
||||
}
|
||||
|
||||
export async function listAssistantKnowledgeSources() {
|
||||
return (await readJson<{ sources: AssistantKnowledgeSource[] }>(await apiFetch(`${BASE()}/sources`))).sources;
|
||||
export async function listAssistantKnowledgeSources(baseId?: string | null) {
|
||||
const params = withBaseParam(baseId);
|
||||
return (await readJson<{ sources: AssistantKnowledgeSource[] }>(await apiFetch(`${BASE()}/sources?${params}`))).sources;
|
||||
}
|
||||
|
||||
export async function listAssistantImportJobs() {
|
||||
return (await readJson<{ jobs: AssistantImportJob[] }>(await apiFetch(`${BASE()}/import-jobs`))).jobs;
|
||||
export async function listAssistantImportJobs(baseId?: string | null) {
|
||||
const params = withBaseParam(baseId);
|
||||
return (
|
||||
await readJson<{ jobs: AssistantImportJob[] }>(await apiFetch(`${BASE()}/import-jobs?${params}`))
|
||||
).jobs.map(normalizeImportJob);
|
||||
}
|
||||
|
||||
export async function listAssistantWikiRevisions(slug: string) {
|
||||
export async function listAssistantWikiRevisions(slug: string, baseId?: string | null) {
|
||||
const params = withBaseParam(baseId);
|
||||
return (await readJson<{ revisions: AssistantWikiRevision[] }>(
|
||||
await apiFetch(`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions`),
|
||||
await apiFetch(`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions?${params}`),
|
||||
)).revisions;
|
||||
}
|
||||
|
||||
export async function rollbackAssistantWikiRevision(slug: string, revisionId: string) {
|
||||
export async function rollbackAssistantWikiRevision(slug: string, revisionId: string, baseId?: string | null) {
|
||||
const params = withBaseParam(baseId);
|
||||
return readJson<AssistantWikiRevision>(
|
||||
await apiFetch(
|
||||
`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions/${encodeURIComponent(revisionId)}/rollback`,
|
||||
`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions/${encodeURIComponent(revisionId)}/rollback?${params}`,
|
||||
{ method: "POST" },
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export async function uploadAssistantKnowledgeFiles(files: File[]) {
|
||||
const form = new FormData();
|
||||
files.forEach((file) => form.append("files", file));
|
||||
return readJson<{ batch_id: string; jobs: AssistantImportJob[] }>(
|
||||
await apiFetch(`${BASE()}/imports/files`, { method: "POST", body: form }),
|
||||
export async function exportLlmWikiToAssistant(mappingId: string, assistantBaseId?: string | null) {
|
||||
return normalizeImportJob(await readJson<AssistantImportJob>(
|
||||
await apiFetch(
|
||||
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-to-assistant`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ assistant_base_id: assistantBaseId || null }),
|
||||
},
|
||||
),
|
||||
));
|
||||
}
|
||||
|
||||
export async function createLlmWikiExportPackage(mappingId: string) {
|
||||
return normalizeExportPackageJob(
|
||||
await readJson<LlmWikiExportPackageJob>(
|
||||
await apiFetch(
|
||||
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package`,
|
||||
{ method: "POST" },
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export async function exportLlmWikiToAssistant(mappingId: string) {
|
||||
return readJson<AssistantImportJob>(
|
||||
await apiFetch(
|
||||
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-to-assistant`,
|
||||
{ method: "POST" },
|
||||
export async function listLlmWikiExportPackages(mappingId: string) {
|
||||
return (
|
||||
await readJson<{ jobs: LlmWikiExportPackageJob[] }>(
|
||||
await apiFetch(
|
||||
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package`,
|
||||
),
|
||||
)
|
||||
).jobs.map(normalizeExportPackageJob);
|
||||
}
|
||||
|
||||
export async function getLlmWikiExportPackageStatus(mappingId: string, jobId: string) {
|
||||
return normalizeExportPackageJob(
|
||||
await readJson<LlmWikiExportPackageJob>(
|
||||
await apiFetch(
|
||||
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package/${encodeURIComponent(jobId)}`,
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export async function reimportAssistantKnowledgeSource(sourceId: string) {
|
||||
return readJson<AssistantImportJob>(
|
||||
return normalizeImportJob(await readJson<AssistantImportJob>(
|
||||
await apiFetch(`${BASE()}/sources/${encodeURIComponent(sourceId)}/reimport`, {
|
||||
method: "POST",
|
||||
}),
|
||||
);
|
||||
));
|
||||
}
|
||||
|
||||
export function getAssistantImportPackageUrl(jobId: string) {
|
||||
return `${BASE()}/import-jobs/${encodeURIComponent(jobId)}/package`;
|
||||
}
|
||||
|
||||
export async function uploadAssistantWeKnoraPackage(input: { file: File; assistantBaseId?: string | null }) {
|
||||
const body = new FormData();
|
||||
body.set("file", input.file);
|
||||
if (input.assistantBaseId) body.set("assistant_base_id", input.assistantBaseId);
|
||||
return normalizeImportJob(await readJson<AssistantImportJob>(
|
||||
await apiFetch(`${BASE()}/imports/weknora-package`, {
|
||||
method: "POST",
|
||||
body,
|
||||
}),
|
||||
));
|
||||
}
|
||||
|
||||
export async function deleteAssistantKnowledgeSource(sourceId: string) {
|
||||
@@ -144,20 +270,40 @@ export async function deleteAssistantKnowledgeSource(sourceId: string) {
|
||||
|
||||
const KEY = "assistant-knowledge";
|
||||
|
||||
export function useAssistantKnowledgeBase() {
|
||||
return useQuery({ queryKey: [KEY, "base"], queryFn: getAssistantKnowledgeBase });
|
||||
export function useAssistantKnowledgeBase(baseId?: string | null) {
|
||||
return useQuery({
|
||||
queryKey: [KEY, "base", baseId ?? "global"],
|
||||
queryFn: () => (baseId ? getAssistantKnowledgeBaseById(baseId) : getAssistantKnowledgeBase()),
|
||||
});
|
||||
}
|
||||
|
||||
export function useAssistantWikiPages(query = "") {
|
||||
return useQuery({ queryKey: [KEY, "pages", query], queryFn: () => listAssistantWikiPages(query) });
|
||||
export function useAssistantKnowledgeBases() {
|
||||
return useQuery({ queryKey: [KEY, "bases"], queryFn: listAssistantKnowledgeBases });
|
||||
}
|
||||
|
||||
export function useAssistantKnowledgeSources() {
|
||||
return useQuery({ queryKey: [KEY, "sources"], queryFn: listAssistantKnowledgeSources });
|
||||
export function useAssistantWikiPages(query = "", baseId?: string | null) {
|
||||
return useQuery({
|
||||
queryKey: [KEY, "pages", baseId ?? "global", query],
|
||||
queryFn: () => listAssistantWikiPages(query, baseId),
|
||||
});
|
||||
}
|
||||
|
||||
export function useAssistantImportJobs() {
|
||||
return useQuery({ queryKey: [KEY, "jobs"], queryFn: listAssistantImportJobs });
|
||||
export function useAssistantKnowledgeSources(baseId?: string | null) {
|
||||
return useQuery({
|
||||
queryKey: [KEY, "sources", baseId ?? "global"],
|
||||
queryFn: () => listAssistantKnowledgeSources(baseId),
|
||||
});
|
||||
}
|
||||
|
||||
export function useAssistantImportJobs(baseId?: string | null) {
|
||||
return useQuery({
|
||||
queryKey: [KEY, "jobs", baseId ?? "global"],
|
||||
queryFn: () => listAssistantImportJobs(baseId),
|
||||
refetchInterval: (query) =>
|
||||
(query.state.data ?? []).some((job) => ["queued", "running"].includes(job.status))
|
||||
? 3000
|
||||
: false,
|
||||
});
|
||||
}
|
||||
|
||||
export function useAssistantKnowledgeActions() {
|
||||
@@ -165,12 +311,13 @@ export function useAssistantKnowledgeActions() {
|
||||
const invalidate = () => client.invalidateQueries({ queryKey: [KEY] });
|
||||
return {
|
||||
initialize: useMutation({ mutationFn: initializeAssistantKnowledgeBase, onSuccess: invalidate }),
|
||||
upload: useMutation({ mutationFn: uploadAssistantKnowledgeFiles, onSuccess: invalidate }),
|
||||
createBase: useMutation({ mutationFn: createAssistantKnowledgeBase, onSuccess: invalidate }),
|
||||
uploadWeKnoraPackage: useMutation({ mutationFn: uploadAssistantWeKnoraPackage, onSuccess: invalidate }),
|
||||
reimportSource: useMutation({ mutationFn: reimportAssistantKnowledgeSource, onSuccess: invalidate }),
|
||||
deleteSource: useMutation({ mutationFn: deleteAssistantKnowledgeSource, onSuccess: invalidate }),
|
||||
rollback: useMutation({
|
||||
mutationFn: ({ slug, revisionId }: { slug: string; revisionId: string }) =>
|
||||
rollbackAssistantWikiRevision(slug, revisionId),
|
||||
mutationFn: ({ slug, revisionId, baseId }: { slug: string; revisionId: string; baseId?: string | null }) =>
|
||||
rollbackAssistantWikiRevision(slug, revisionId, baseId),
|
||||
onSuccess: invalidate,
|
||||
}),
|
||||
};
|
||||
|
||||
@@ -40,6 +40,9 @@ export interface LlmWikiKnowledgeBase {
|
||||
share_count?: number;
|
||||
is_processing?: boolean;
|
||||
is_temporary?: boolean;
|
||||
wiki_index_enabled?: boolean;
|
||||
external_search_enabled?: boolean;
|
||||
external_search_updated_at?: string | null;
|
||||
embedding_model_id?: string | null;
|
||||
vector_store_id?: string | null;
|
||||
vector_store_name?: string | null;
|
||||
|
||||
+186
-24
@@ -1,15 +1,11 @@
|
||||
import { AlertCircle, Database, Edit3, FileText, Loader2, RefreshCw, Trash2 } from "lucide-react";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import { AlertCircle, Download, Edit3, FileText, Loader2, RefreshCw, Trash2 } from "lucide-react";
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { Navigate, useNavigate, useParams, useSearchParams } from "react-router-dom";
|
||||
import { toast } from "sonner";
|
||||
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import { Dialog, DialogContent, DialogHeader, DialogTitle } from "@/components/ui/dialog";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Textarea } from "@/components/ui/textarea";
|
||||
import { MarkdownCodeEditor } from "@/components/wiki/MarkdownCodeEditor";
|
||||
@@ -24,9 +20,20 @@ import {
|
||||
useLlmWikiWikiPage,
|
||||
useLlmWikiRuntime,
|
||||
} from "@/strategy-components/api/llmwiki";
|
||||
import { exportLlmWikiToAssistant } from "@/strategy-components/api/assistant-knowledge";
|
||||
import {
|
||||
createLlmWikiExportPackage,
|
||||
listLlmWikiExportPackages,
|
||||
type LlmWikiExportPackageJob,
|
||||
} from "@/strategy-components/api/assistant-knowledge";
|
||||
|
||||
const LIST_PATH = "/page/workspace/knowledge";
|
||||
const RUNNING_EXPORT_STATUSES = new Set(["queued", "running"]);
|
||||
const EXPORT_STATUS_LABELS: Record<string, string> = {
|
||||
queued: "排队中",
|
||||
running: "整理中",
|
||||
completed: "已完成",
|
||||
failed: "失败",
|
||||
};
|
||||
|
||||
type DeerFlowThemePayload = {
|
||||
type: "deerflow-weknora-theme";
|
||||
@@ -94,6 +101,25 @@ function buildFrameUrl(framePath: string) {
|
||||
return new URL(`${backendBase || backendOrigin}/${framePath.replace(/^\/+/, "")}`);
|
||||
}
|
||||
|
||||
function formatBytes(value: number | undefined | null) {
|
||||
const bytes = Number(value || 0);
|
||||
if (!Number.isFinite(bytes) || bytes <= 0) return "—";
|
||||
const units = ["B", "KB", "MB", "GB", "TB"];
|
||||
let current = bytes;
|
||||
let unitIndex = 0;
|
||||
while (current >= 1024 && unitIndex < units.length - 1) {
|
||||
current /= 1024;
|
||||
unitIndex += 1;
|
||||
}
|
||||
return `${current >= 10 || unitIndex === 0 ? current.toFixed(0) : current.toFixed(1)} ${units[unitIndex]}`;
|
||||
}
|
||||
|
||||
function formatTime(value: string | undefined | null) {
|
||||
if (!value) return "—";
|
||||
const time = new Date(value);
|
||||
return Number.isNaN(time.getTime()) ? "—" : time.toLocaleString();
|
||||
}
|
||||
|
||||
export function WeKnoraKnowledgeDetailPage() {
|
||||
const {
|
||||
knowledge_base_id: strategyKnowledgeBaseId,
|
||||
@@ -106,16 +132,40 @@ export function WeKnoraKnowledgeDetailPage() {
|
||||
const [frameSrc, setFrameSrc] = useState("");
|
||||
const [frameVersion, setFrameVersion] = useState(0);
|
||||
const [frameError, setFrameError] = useState<string | null>(null);
|
||||
const [assistantImporting, setAssistantImporting] = useState(false);
|
||||
const [exportPackageLoading, setExportPackageLoading] = useState(false);
|
||||
const [exportJobsLoading, setExportJobsLoading] = useState(false);
|
||||
const [exportJobs, setExportJobs] = useState<LlmWikiExportPackageJob[]>([]);
|
||||
const isAdmin = getStoredAuth()?.system_role === "admin";
|
||||
const runtime = useLlmWikiRuntime();
|
||||
const details = useLlmWikiKnowledgeBase(knowledgeBaseId || null, !!knowledgeBaseId);
|
||||
|
||||
const refreshExportJobs = useCallback(
|
||||
async (silent = false) => {
|
||||
if (!knowledgeBaseId || !isAdmin) return;
|
||||
if (!silent) setExportJobsLoading(true);
|
||||
try {
|
||||
setExportJobs(await listLlmWikiExportPackages(knowledgeBaseId));
|
||||
} catch (error) {
|
||||
if (!silent) toast.error(error instanceof Error ? error.message : String(error));
|
||||
} finally {
|
||||
if (!silent) setExportJobsLoading(false);
|
||||
}
|
||||
},
|
||||
[isAdmin, knowledgeBaseId],
|
||||
);
|
||||
|
||||
const detailsReady = !!details.data;
|
||||
const requestedTab = searchParams.get("tab");
|
||||
const detailsWikiEnabled =
|
||||
details.data?.type === "wiki" ||
|
||||
details.data?.wiki_index_enabled === true ||
|
||||
details.data?.indexing_strategy?.wiki_enabled === true ||
|
||||
details.data?.capabilities?.wiki_enabled === true ||
|
||||
(details.data?.wiki_config && Object.keys(details.data.wiki_config).length > 0);
|
||||
const preferredTab =
|
||||
requestedTab === "documents" || requestedTab === "wiki" || requestedTab === "graph"
|
||||
? requestedTab
|
||||
: details.data?.indexing_strategy?.wiki_enabled
|
||||
: detailsWikiEnabled
|
||||
? "wiki"
|
||||
: "documents";
|
||||
const sourceChunkId = searchParams.get("sourceChunkId") || searchParams.get("chunkId") || "";
|
||||
@@ -182,6 +232,22 @@ export function WeKnoraKnowledgeDetailPage() {
|
||||
};
|
||||
}, [frameSrc]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!isAdmin || !knowledgeBaseId || runtime.data?.provider !== "weknora") {
|
||||
setExportJobs([]);
|
||||
return;
|
||||
}
|
||||
void refreshExportJobs();
|
||||
}, [isAdmin, knowledgeBaseId, refreshExportJobs, runtime.data?.provider]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!isAdmin || !knowledgeBaseId || !exportJobs.some((job) => RUNNING_EXPORT_STATUSES.has(job.status))) {
|
||||
return;
|
||||
}
|
||||
const timer = window.setInterval(() => void refreshExportJobs(true), 5000);
|
||||
return () => window.clearInterval(timer);
|
||||
}, [exportJobs, isAdmin, knowledgeBaseId, refreshExportJobs]);
|
||||
|
||||
if (nativeWikiPageUrl) {
|
||||
return <Navigate to={nativeWikiPageUrl} replace />;
|
||||
}
|
||||
@@ -235,8 +301,25 @@ export function WeKnoraKnowledgeDetailPage() {
|
||||
|
||||
const canWrite = details.data.can_write && !details.data.is_conversation_deposit;
|
||||
const canEditCurrentWikiArticle =
|
||||
canWrite && !!details.data.indexing_strategy?.wiki_enabled && !!wikiSlug;
|
||||
const canManageWiki = canWrite && !!details.data.indexing_strategy?.wiki_enabled;
|
||||
canWrite && !!detailsWikiEnabled && !!wikiSlug;
|
||||
const canManageWiki = canWrite && !!detailsWikiEnabled;
|
||||
|
||||
const createVectorizedPackage = () => {
|
||||
if (exportPackageLoading) return;
|
||||
setExportPackageLoading(true);
|
||||
void (async () => {
|
||||
try {
|
||||
const job = await createLlmWikiExportPackage(knowledgeBaseId);
|
||||
setExportJobs((current) => [job, ...current.filter((item) => item.id !== job.id)]);
|
||||
toast.success("已创建向量化数据包整理记录,完成后可在下方列表下载");
|
||||
void refreshExportJobs(true);
|
||||
} catch (error) {
|
||||
toast.error(error instanceof Error ? error.message : String(error));
|
||||
} finally {
|
||||
setExportPackageLoading(false);
|
||||
}
|
||||
})();
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex h-full min-h-0 flex-col bg-background">
|
||||
@@ -258,21 +341,15 @@ export function WeKnoraKnowledgeDetailPage() {
|
||||
)}
|
||||
</div>
|
||||
<div className="flex shrink-0 items-center gap-2">
|
||||
{getStoredAuth()?.system_role === "admin" && (
|
||||
{isAdmin && (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
disabled={assistantImporting}
|
||||
onClick={() => {
|
||||
setAssistantImporting(true);
|
||||
void exportLlmWikiToAssistant(knowledgeBaseId)
|
||||
.then(() => toast.success("已导入到助手知识库“知识梳理”"))
|
||||
.catch((error: unknown) => toast.error(error instanceof Error ? error.message : String(error)))
|
||||
.finally(() => setAssistantImporting(false));
|
||||
}}
|
||||
disabled={exportPackageLoading}
|
||||
onClick={createVectorizedPackage}
|
||||
>
|
||||
{assistantImporting ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Database className="mr-2 size-4" />}
|
||||
导入到助手知识库
|
||||
{exportPackageLoading ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Download className="mr-2 size-4" />}
|
||||
生成向量化数据包
|
||||
</Button>
|
||||
)}
|
||||
<Button
|
||||
@@ -286,6 +363,13 @@ export function WeKnoraKnowledgeDetailPage() {
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{canManageWiki && isAdmin && (
|
||||
<ExportPackageJobsPanel
|
||||
jobs={exportJobs}
|
||||
loading={exportJobsLoading}
|
||||
onRefresh={() => void refreshExportJobs()}
|
||||
/>
|
||||
)}
|
||||
{frameSrc && (
|
||||
<iframe
|
||||
key={`${frameSrc}:${frameVersion}`}
|
||||
@@ -313,6 +397,84 @@ export function WeKnoraKnowledgeDetailPage() {
|
||||
);
|
||||
}
|
||||
|
||||
function ExportPackageJobsPanel({
|
||||
jobs,
|
||||
loading,
|
||||
onRefresh,
|
||||
}: {
|
||||
jobs: LlmWikiExportPackageJob[];
|
||||
loading: boolean;
|
||||
onRefresh: () => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="shrink-0 border-b bg-muted/20 px-4 py-3">
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<div>
|
||||
<div className="text-sm font-medium">向量化数据包记录</div>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
每次点击都会生成一条后台整理记录;文件较大时,请等待状态变为“已完成”后再下载。
|
||||
</p>
|
||||
</div>
|
||||
<Button size="sm" variant="ghost" onClick={onRefresh} disabled={loading}>
|
||||
{loading ? <Loader2 className="mr-2 size-4 animate-spin" /> : <RefreshCw className="mr-2 size-4" />}
|
||||
刷新
|
||||
</Button>
|
||||
</div>
|
||||
<div className="max-h-44 space-y-2 overflow-y-auto pr-1">
|
||||
{jobs.map((job) => {
|
||||
const running = RUNNING_EXPORT_STATUSES.has(job.status);
|
||||
const completed = job.status === "completed";
|
||||
const canDownload = completed && !!job.package_download_url && job.file_exists !== false;
|
||||
const counts = job.counts ?? {};
|
||||
return (
|
||||
<div key={job.id} className="flex items-center gap-3 rounded-md border bg-card px-3 py-2 text-xs">
|
||||
<Badge
|
||||
variant={job.status === "failed" ? "destructive" : completed ? "default" : "secondary"}
|
||||
className="shrink-0"
|
||||
>
|
||||
{running && <Loader2 className="mr-1 inline size-3 animate-spin" />}
|
||||
{EXPORT_STATUS_LABELS[job.status] || job.status}
|
||||
</Badge>
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="flex flex-wrap items-center gap-x-3 gap-y-1">
|
||||
<span className="font-medium">任务 {job.id.slice(0, 8)}</span>
|
||||
<span className="text-muted-foreground">创建:{formatTime(job.created_at)}</span>
|
||||
<span className="text-muted-foreground">大小:{formatBytes(job.file_size_bytes)}</span>
|
||||
</div>
|
||||
<div className="mt-1 flex flex-wrap gap-x-3 gap-y-1 text-muted-foreground">
|
||||
<span>Wiki {counts.wiki_page ?? counts.wiki_pages ?? 0}</span>
|
||||
<span>文档 {counts.document ?? counts.documents ?? 0}</span>
|
||||
<span>片段 {counts.chunk ?? counts.chunks ?? 0}</span>
|
||||
<span>向量 {counts.vector ?? counts.vectors ?? 0}</span>
|
||||
<span>阶段:{job.phase}</span>
|
||||
{job.error && <span className="text-destructive">{job.error}</span>}
|
||||
</div>
|
||||
</div>
|
||||
{canDownload ? (
|
||||
<Button asChild size="sm" variant="outline" className="shrink-0">
|
||||
<a href={job.package_download_url} target="_blank" rel="noreferrer">
|
||||
<Download className="mr-2 size-4" />
|
||||
下载
|
||||
</a>
|
||||
</Button>
|
||||
) : (
|
||||
<Button size="sm" variant="outline" className="shrink-0" disabled>
|
||||
{completed && job.file_exists === false ? "文件不存在" : "等待完成"}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
{!jobs.length && (
|
||||
<div className="rounded-md border border-dashed bg-card px-3 py-6 text-center text-xs text-muted-foreground">
|
||||
暂无导出记录。点击上方“生成向量化数据包”后,任务会显示在这里。
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function CurrentWikiArticleActions({
|
||||
knowledgeBaseId,
|
||||
slug,
|
||||
|
||||
@@ -1623,23 +1623,40 @@ save fallback.
|
||||
## Skill/assistant knowledge invariants
|
||||
|
||||
Skill distillation is implemented in `deerflow.skill_knowledge` and
|
||||
`deerflow.persistence.skill_knowledge`; the single native assistant base is in
|
||||
`deerflow.persistence.skill_knowledge`; native assistant bases are in
|
||||
`deerflow.assistant_knowledge` and
|
||||
`deerflow.persistence.assistant_knowledge`. Gateway composition, remote adapters,
|
||||
the lease dispatcher, watcher/scheduler, and APIs remain in `app.gateway` so the
|
||||
harness-to-app import firewall is preserved.
|
||||
the lease dispatcher, and APIs remain in `app.gateway` so the harness-to-app
|
||||
import firewall is preserved.
|
||||
|
||||
Never execute a skill or read/upload/summarize its code and script files.
|
||||
Knowledge fingerprints exclude those files and Markdown code fences. All
|
||||
resyncs must stage and validate the knowledge digest before activation; the old
|
||||
snapshot remains authoritative on failure. A binding may have only one active
|
||||
item (`active_dedupe_key` plus atomic lease claims). Detach/delete operates on
|
||||
source contributions, not canonical rows, and shared entities/relations are
|
||||
retired only when their final contribution is gone.
|
||||
item (`active_dedupe_key` plus atomic lease claims). Skill distillation targets
|
||||
only writable WeKnora Wiki knowledge bases in this product version; reject direct
|
||||
assistant-base targets, non-Wiki/FAQ/conversation/readonly targets, and direct
|
||||
Neo4j writes. Detach/delete operates on source contributions, not canonical rows,
|
||||
and shared entities/relations are retired only when their final contribution is
|
||||
gone.
|
||||
|
||||
There is exactly one global assistant base (`知识梳理`). Ordinary users have
|
||||
read/search access and administrators own all mutations. WeKnora direct graph
|
||||
writes require `llmwiki.weknora.graph.direct_write_enabled`, backend-only
|
||||
credentials, and the fixed parameterized templates in
|
||||
`deerflow.skill_knowledge.graph_writer`; never accept Cypher from an API, model,
|
||||
skill, or frontend.
|
||||
Assistant knowledge has one global full-library base (`知识梳理总库`) and any
|
||||
number of admin-created custom bases. Ordinary users have read/search access and
|
||||
administrators own all mutations. Assistant bases only accept WeKnora export
|
||||
packages or imports triggered from a WeKnora mapping. Do not add direct skill
|
||||
imports or arbitrary local-file imports to assistant knowledge. Importing into a
|
||||
custom assistant base must also mirror the package into the global base for
|
||||
cross-source entity alignment.
|
||||
|
||||
## openGauss / GaussDB persistence invariant
|
||||
|
||||
`database.backend=gauss` is an application-ORM backend, parallel to `mysql`.
|
||||
It normalizes `database.gauss_url` to `opengauss+asyncpg://` and requires the
|
||||
`gauss` optional dependency (`asyncpg` plus `opengauss-sqlalchemy`). Never route
|
||||
a Gauss URL through SQLAlchemy's generic PostgreSQL dialect: the wire protocol
|
||||
may connect, but dialect initialization and schema DDL can fail against the
|
||||
Gauss server/version behavior. Keep the LangGraph checkpointer on the separately
|
||||
configured SQLite backend; the PostgreSQL checkpointer schema is not part of the
|
||||
Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
|
||||
names as timezone-aware PostgreSQL-family engines in portable types and
|
||||
dialect-specific session settings.
|
||||
@@ -13,8 +13,8 @@ FROM python:3.12-slim-bookworm AS builder
|
||||
ARG NODE_MAJOR=22
|
||||
ARG APT_MIRROR
|
||||
ARG UV_INDEX_URL
|
||||
# Optional extras to install (e.g. "postgres" for PostgreSQL support)
|
||||
# Usage: docker build --build-arg UV_EXTRAS=postgres ...
|
||||
# Optional extras to install (e.g. "postgres" or "gauss")
|
||||
# Usage: docker build --build-arg UV_EXTRAS=gauss ...
|
||||
ARG UV_EXTRAS
|
||||
|
||||
# Optionally override apt mirror for restricted networks (e.g. APT_MIRROR=mirrors.aliyun.com)
|
||||
|
||||
@@ -957,26 +957,54 @@ sandbox-directory recreation cannot discard a completed rewrite.
|
||||
## Skill knowledge distillation and assistant knowledge
|
||||
|
||||
The Gateway now exposes an administrator-only skill distillation pipeline at
|
||||
`/api/skill-knowledge`. A matrix job can send up to 200 skills to multiple
|
||||
WeKnora or global assistant-knowledge targets. The scanner never reads or
|
||||
uploads code/script files, strips Markdown code fences, blocks credential-like
|
||||
files and content, safely inspects ZIP members, and converts supported
|
||||
Office/PDF/spreadsheet files off the event loop. Jobs use database claims and
|
||||
renewable leases, stage snapshots before activation, retain the prior active
|
||||
version on failure, support per-item retry and entity/relation review, and keep
|
||||
per-binding contribution records for safe detach/delete.
|
||||
`/api/skill-knowledge`. Each request can send up to 200 skills to one writable
|
||||
WeKnora Wiki knowledge base; multi-base distillation is done by repeated
|
||||
operations so the local binding remains `skill_name + target_id`. The scanner
|
||||
never reads or uploads code/script files, strips Markdown code fences, blocks
|
||||
credential-like files and content, safely inspects ZIP members, and converts
|
||||
supported Office/PDF/spreadsheet files off the event loop. Jobs use database
|
||||
claims and renewable leases, stage snapshots before activation, retain the
|
||||
prior active version on failure, support per-item retry, and keep per-binding
|
||||
contribution records for safe detach/delete. Direct assistant-base targets,
|
||||
non-Wiki/FAQ/conversation/readonly targets, and direct Neo4j writes are rejected
|
||||
for this product version.
|
||||
|
||||
Bindings may opt into debounced file watching and cron synchronization through
|
||||
`/api/skill-knowledge/auto-sync-settings`. Only non-code knowledge digests can
|
||||
mark a binding stale. WeKnora Wiki, document, FAQ, conversation-archive and
|
||||
read-only sidecar targets have separate projections. Optional Neo4j direct
|
||||
writes are disabled by default under `llmwiki.weknora.graph`; when enabled they
|
||||
use only fixed parameterized templates and Neo4j's transactional HTTP endpoint.
|
||||
|
||||
`/api/assistant-knowledge` owns exactly one global DeerFlow-native base named
|
||||
`知识梳理`. Reads and search are available to authenticated users; initialization,
|
||||
local-file import, WeKnora export/import, source reimport/delete, and revision
|
||||
rollback are admin-only. Imports align canonical entities and relations while
|
||||
retaining source contributions, publish source/entity/relation Wiki revisions,
|
||||
and remove canonical knowledge only after its final contribution disappears.
|
||||
`/api/assistant-knowledge` manages DeerFlow-native assistant knowledge bases.
|
||||
There is one global full-library base named `知识梳理总库`, plus admin-created
|
||||
custom assistant bases. Reads and search are available to authenticated users;
|
||||
global initialization, custom-base creation, WeKnora export/import, source
|
||||
reimport, and source deletion are admin-only. Assistant bases only accept
|
||||
WeKnora export packages or imports triggered from a WeKnora mapping; direct
|
||||
skill import and arbitrary local-file import return 410. Importing into a custom
|
||||
assistant base also mirrors the same package into the global base, where
|
||||
canonical entities and relations are aligned across all imported sources.
|
||||
The database schema is introduced by Alembic revision `20260901_01`.
|
||||
|
||||
## openGauss / GaussDB application database
|
||||
|
||||
The application ORM can use a PG-compatible openGauss/GaussDB database through
|
||||
the official `opengauss-sqlalchemy` async dialect. Install the optional driver
|
||||
set before enabling it:
|
||||
|
||||
```bash
|
||||
uv sync --extra gauss
|
||||
```
|
||||
|
||||
Configure the business database independently from the LangGraph checkpointer:
|
||||
|
||||
```yaml
|
||||
database:
|
||||
backend: gauss
|
||||
gauss_url: $GAUSS_DATABASE_URL
|
||||
|
||||
checkpointer:
|
||||
type: sqlite
|
||||
connection_string: .deer-flow/data/deerflow.db
|
||||
```
|
||||
|
||||
`GAUSS_DATABASE_URL` should normally be
|
||||
`opengauss+asyncpg://user:password@host:26000/deerflow`. Plain
|
||||
`opengauss://`, `gaussdb://`, and PostgreSQL-style URLs are accepted and
|
||||
normalized to the openGauss dialect. Do not configure the LangGraph
|
||||
PostgreSQL checkpointer against GaussDB: its migrations are PostgreSQL-specific.
|
||||
For a production image, build with `--build-arg UV_EXTRAS=gauss`.
|
||||
@@ -362,13 +362,21 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
|
||||
|
||||
resolved_llmwiki = get_resolved_llmwiki_runtime(config)
|
||||
weknora_client = build_weknora_client(resolved_llmwiki) if resolved_llmwiki.weknora_enabled else None
|
||||
if local_index_config.enabled:
|
||||
embedding_config = local_index_config.embedding
|
||||
embedding_configured = bool(
|
||||
embedding_config.enabled
|
||||
and embedding_config.base_url
|
||||
and embedding_config.model.strip()
|
||||
)
|
||||
if embedding_configured:
|
||||
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
|
||||
|
||||
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(embedding_config)
|
||||
if local_index_config.enabled and app.state.llmwiki_embedding is not None:
|
||||
from deerflow.integrations.weknora.local_index.search import WikiVectorSearchService
|
||||
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
|
||||
|
||||
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(local_index_config.embedding)
|
||||
app.state.llmwiki_vector_cache = WikiVectorCache(
|
||||
app.state.llmwiki_index_store,
|
||||
max_knowledge_bases=local_index_config.cache_max_knowledge_bases,
|
||||
|
||||
@@ -40,6 +40,7 @@ _MAX_SOURCE_CHARS = 1400
|
||||
@dataclass(frozen=True)
|
||||
class LlmWikiRagContext:
|
||||
knowledge_base_ids: list[str]
|
||||
assistant_knowledge_base_ids: list[str]
|
||||
source_count: int
|
||||
prompt: str
|
||||
sources: list[dict[str, Any]]
|
||||
@@ -50,6 +51,10 @@ def _store(request: Request) -> LlmWikiStore | None:
|
||||
return getattr(request.app.state, "llmwiki_store", None)
|
||||
|
||||
|
||||
def _assistant_store(request: Request) -> Any | None:
|
||||
return getattr(request.app.state, "assistant_knowledge_store", None)
|
||||
|
||||
|
||||
async def _actor(request: Request) -> tuple[str, bool]:
|
||||
user = await get_optional_user_from_request(request)
|
||||
if user is None:
|
||||
@@ -76,6 +81,13 @@ def _body_selected_ids(body: Any) -> list[str]:
|
||||
return []
|
||||
|
||||
|
||||
def _body_selected_assistant_ids(body: Any) -> list[str]:
|
||||
for source in (getattr(body, "context", None), getattr(body, "metadata", None)):
|
||||
if isinstance(source, Mapping) and "assistant_knowledge_base_ids" in source:
|
||||
return _as_id_list(source.get("assistant_knowledge_base_ids"))
|
||||
return []
|
||||
|
||||
|
||||
def _assistant_agent_id(body: Any) -> str | None:
|
||||
assistant_id = validate_agent_id(getattr(body, "assistant_id", None))
|
||||
if assistant_id:
|
||||
@@ -106,8 +118,17 @@ def resolve_requested_llmwiki_knowledge_base_ids(body: Any) -> list[str]:
|
||||
return _body_selected_ids(body)
|
||||
|
||||
|
||||
def resolve_requested_assistant_knowledge_base_ids(body: Any) -> list[str]:
|
||||
"""Return assistant knowledge bases explicitly selected for the run."""
|
||||
|
||||
return _body_selected_assistant_ids(body)
|
||||
|
||||
|
||||
def llmwiki_selection_requested(body: Any) -> bool:
|
||||
return bool(resolve_requested_llmwiki_knowledge_base_ids(body))
|
||||
return bool(
|
||||
resolve_requested_llmwiki_knowledge_base_ids(body)
|
||||
or resolve_requested_assistant_knowledge_base_ids(body)
|
||||
)
|
||||
|
||||
|
||||
def _latest_human_query(graph_input: dict[str, Any]) -> str:
|
||||
@@ -191,18 +212,50 @@ def _source_from_result(result: dict[str, Any], mapping: dict[str, Any], index:
|
||||
}
|
||||
|
||||
|
||||
def _source_from_assistant_result(result: dict[str, Any], index: int) -> dict[str, Any]:
|
||||
title = str(result.get("title") or f"助手知识库片段 {index}")
|
||||
matched_section = result.get("matched_section") if isinstance(result.get("matched_section"), Mapping) else {}
|
||||
content = str(
|
||||
matched_section.get("content")
|
||||
or result.get("content_markdown")
|
||||
or result.get("content")
|
||||
or result.get("summary")
|
||||
or ""
|
||||
)
|
||||
snippet = str(result.get("summary") or content)
|
||||
raw = {
|
||||
**result,
|
||||
"source": "助手知识库",
|
||||
"url": "/page/workspace/knowledge/assistant",
|
||||
}
|
||||
return {
|
||||
"index": index,
|
||||
"id": result.get("id") or result.get("slug") or f"assistant:{index}",
|
||||
"title": title,
|
||||
"type": "assistant_knowledge",
|
||||
"snippet": _truncate(snippet, 360),
|
||||
"content": content,
|
||||
"source": "助手知识库",
|
||||
"score": float(result.get("score") or 0),
|
||||
"url": "/page/workspace/knowledge/assistant",
|
||||
"skillName": "assistant_knowledge_search",
|
||||
"mode": "citation",
|
||||
"raw": raw,
|
||||
}
|
||||
|
||||
|
||||
def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
|
||||
if not sources:
|
||||
return "当前用户选择了 LLMWiki 知识库,但 DeerFlow 未检索到可用片段。请直接说明没有检索到可用背景资料,不要编造知识库内容或引用编号。"
|
||||
return "当前用户选择了知识库,但 DeerFlow 未检索到可用片段。请直接说明没有检索到可用背景资料,不要编造知识库内容或引用编号。"
|
||||
|
||||
lines = [
|
||||
"当前用户选择了 LLMWiki 知识库。DeerFlow 已在模型回答前自动检索到以下背景资料。",
|
||||
"当前用户选择了知识库。DeerFlow 已在模型回答前自动检索到以下背景资料。",
|
||||
"请优先依据这些资料回答;如果资料不足,请明确说明不足之处,不要编造。",
|
||||
"引用规则:正文关键结论必须使用 [1]、[2] 这样的编号引用下方资料;只能引用存在的编号;不要在正文末尾重复输出完整参考文献列表,前端会展示引用卡片。",
|
||||
"",
|
||||
f"用户问题:{query}",
|
||||
"",
|
||||
f'<llmwiki_wiki_reference_batch source_count="{len(sources)}">',
|
||||
f'<knowledge_reference_batch source_count="{len(sources)}">',
|
||||
]
|
||||
for source in sources:
|
||||
meta = " / ".join(
|
||||
@@ -213,12 +266,12 @@ def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
|
||||
)
|
||||
if item
|
||||
)
|
||||
lines.append(f"[{source['index']}] {source.get('title') or 'LLMWiki 片段'}")
|
||||
lines.append(f"[{source['index']}] {source.get('title') or '知识库片段'}")
|
||||
if meta:
|
||||
lines.append(f"来源:{meta}")
|
||||
lines.append(_truncate(str(source.get("content") or source.get("snippet") or ""), _MAX_SOURCE_CHARS))
|
||||
lines.append("")
|
||||
lines.append("</llmwiki_wiki_reference_batch>")
|
||||
lines.append("</knowledge_reference_batch>")
|
||||
return "\n".join(lines).strip()
|
||||
|
||||
|
||||
@@ -229,52 +282,99 @@ async def build_llmwiki_rag_context(
|
||||
graph_input: dict[str, Any],
|
||||
) -> LlmWikiRagContext:
|
||||
selected_ids = resolve_requested_llmwiki_knowledge_base_ids(body)
|
||||
if not selected_ids:
|
||||
return LlmWikiRagContext([], 0, "", [], "")
|
||||
|
||||
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None))
|
||||
local_config = request.app.state.config.llmwiki.local_wiki_index
|
||||
if not (local_config.enabled and local_config.internal_search_enabled) and not runtime.weknora_enabled:
|
||||
return LlmWikiRagContext(selected_ids, 0, "", [], "")
|
||||
store = _store(request)
|
||||
if store is None:
|
||||
return LlmWikiRagContext(selected_ids, 0, "", [], "")
|
||||
assistant_selected_ids = resolve_requested_assistant_knowledge_base_ids(body)
|
||||
if not selected_ids and not assistant_selected_ids:
|
||||
return LlmWikiRagContext([], [], 0, "", [], "")
|
||||
|
||||
query = _latest_human_query(graph_input)
|
||||
if not query:
|
||||
return LlmWikiRagContext(selected_ids, 0, "", [], "")
|
||||
return LlmWikiRagContext(selected_ids, assistant_selected_ids, 0, "", [], "")
|
||||
|
||||
user_id, is_admin = await _actor(request)
|
||||
rows: list[dict[str, Any]] = []
|
||||
for mapping_id in selected_ids:
|
||||
row = await store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
|
||||
if row is None:
|
||||
raise HTTPException(status_code=404, detail=f"Knowledge base {mapping_id} not found")
|
||||
rows.append(row)
|
||||
search_service = getattr(request.app.state, "llmwiki_retrieval_service", None)
|
||||
if search_service is None:
|
||||
return LlmWikiRagContext(
|
||||
selected_ids,
|
||||
0,
|
||||
"Wiki 检索暂不可用。请明确说明系统检索不可用,不要表述为资料不存在。",
|
||||
[],
|
||||
query,
|
||||
)
|
||||
try:
|
||||
payload = await search_service.search(query, rows, top_k_pages=_MAX_SOURCES)
|
||||
except WikiIndexError as exc:
|
||||
logger.warning("Wiki-only pre-retrieval failed user=%s kb_count=%s code=%s", sanitize_log_param(user_id), len(rows), exc.code)
|
||||
return LlmWikiRagContext(selected_ids, 0, f"Wiki 检索暂不可用({exc.code})。请明确说明系统检索状态,不要声称知识库没有资料。", [], query)
|
||||
mapping_by_id = {str(row["id"]): row for row in rows}
|
||||
sources: list[dict[str, Any]] = []
|
||||
for result in payload.get("results") or []:
|
||||
mapping = mapping_by_id.get(str(result.get("knowledge_base_id") or ""))
|
||||
if mapping is None:
|
||||
continue
|
||||
sources.append(_source_from_result(result, mapping, len(sources) + 1))
|
||||
if len(sources) >= _MAX_SOURCES:
|
||||
break
|
||||
return LlmWikiRagContext(selected_ids, len(sources), _build_prompt(query, sources), sources, query)
|
||||
|
||||
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None))
|
||||
local_config = request.app.state.config.llmwiki.local_wiki_index
|
||||
user_id, is_admin = await _actor(request)
|
||||
|
||||
if selected_ids:
|
||||
llmwiki_available = (local_config.enabled and local_config.internal_search_enabled) or runtime.weknora_enabled
|
||||
store = _store(request)
|
||||
if not llmwiki_available or store is None:
|
||||
if not assistant_selected_ids:
|
||||
return LlmWikiRagContext(selected_ids, assistant_selected_ids, 0, "", [], query)
|
||||
else:
|
||||
rows: list[dict[str, Any]] = []
|
||||
for mapping_id in selected_ids:
|
||||
row = await store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
|
||||
if row is None:
|
||||
raise HTTPException(status_code=404, detail=f"Knowledge base {mapping_id} not found")
|
||||
rows.append(row)
|
||||
search_service = getattr(request.app.state, "llmwiki_retrieval_service", None)
|
||||
if search_service is None:
|
||||
if not assistant_selected_ids:
|
||||
return LlmWikiRagContext(
|
||||
selected_ids,
|
||||
assistant_selected_ids,
|
||||
0,
|
||||
"Wiki 检索暂不可用。请明确说明系统检索不可用,不要表述为资料不存在。",
|
||||
[],
|
||||
query,
|
||||
)
|
||||
try:
|
||||
payload = (
|
||||
await search_service.search(query, rows, top_k_pages=_MAX_SOURCES)
|
||||
if search_service is not None
|
||||
else {"results": []}
|
||||
)
|
||||
except WikiIndexError as exc:
|
||||
logger.warning(
|
||||
"Wiki-only pre-retrieval failed user=%s kb_count=%s code=%s",
|
||||
sanitize_log_param(user_id),
|
||||
len(rows),
|
||||
exc.code,
|
||||
)
|
||||
if not assistant_selected_ids:
|
||||
return LlmWikiRagContext(
|
||||
selected_ids,
|
||||
assistant_selected_ids,
|
||||
0,
|
||||
f"Wiki 检索暂不可用({exc.code})。请明确说明系统检索状态,不要声称知识库没有资料。",
|
||||
[],
|
||||
query,
|
||||
)
|
||||
payload = {"results": []}
|
||||
mapping_by_id = {str(row["id"]): row for row in rows}
|
||||
for result in payload.get("results") or []:
|
||||
mapping = mapping_by_id.get(str(result.get("knowledge_base_id") or ""))
|
||||
if mapping is None:
|
||||
continue
|
||||
sources.append(_source_from_result(result, mapping, len(sources) + 1))
|
||||
if len(sources) >= _MAX_SOURCES:
|
||||
break
|
||||
|
||||
if assistant_selected_ids and len(sources) < _MAX_SOURCES:
|
||||
assistant_store = _assistant_store(request)
|
||||
if assistant_store is not None:
|
||||
rows = await assistant_store.search_pages(
|
||||
base_ids=assistant_selected_ids,
|
||||
query=query,
|
||||
limit=_MAX_SOURCES - len(sources),
|
||||
include_chunks=True,
|
||||
embedding_client=getattr(request.app.state, "llmwiki_embedding", None),
|
||||
)
|
||||
for row in rows:
|
||||
sources.append(_source_from_assistant_result(row, len(sources) + 1))
|
||||
if len(sources) >= _MAX_SOURCES:
|
||||
break
|
||||
|
||||
return LlmWikiRagContext(
|
||||
selected_ids,
|
||||
assistant_selected_ids,
|
||||
len(sources),
|
||||
_build_prompt(query, sources),
|
||||
sources,
|
||||
query,
|
||||
)
|
||||
|
||||
|
||||
def attach_llmwiki_rag_to_graph_input(
|
||||
@@ -294,6 +394,7 @@ def attach_llmwiki_rag_to_graph_input(
|
||||
existing = str(extra.get("prompt_prefix") or "").strip()
|
||||
extra["prompt_prefix"] = f"{existing}\n\n{rag.prompt}".strip() if existing else rag.prompt
|
||||
extra["llmwiki_knowledge_base_ids"] = rag.knowledge_base_ids
|
||||
extra["assistant_knowledge_base_ids"] = rag.assistant_knowledge_base_ids
|
||||
extra["llmwiki_reference_query"] = rag.query
|
||||
if rag.sources:
|
||||
extra["llmwiki_reference_sources"] = rag.sources
|
||||
|
||||
@@ -461,7 +461,7 @@ async def _set_analytics_read_timeout(session) -> None:
|
||||
- MariaDB: ``max_statement_time`` (seconds)
|
||||
- MySQL <= 5.6: no per-statement timeout variable — skip it entirely
|
||||
(5.6 raises "Unknown system variable" otherwise)
|
||||
- PostgreSQL: ``statement_timeout`` (milliseconds)
|
||||
- PostgreSQL/openGauss: ``statement_timeout`` (milliseconds)
|
||||
"""
|
||||
conn = await session.connection()
|
||||
dialect = conn.dialect
|
||||
@@ -474,7 +474,7 @@ async def _set_analytics_read_timeout(session) -> None:
|
||||
await _try_set_session_value(session, f"SET SESSION max_execution_time = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
|
||||
# else: MySQL <= 5.6 has no statement-timeout variable — skip silently.
|
||||
await _try_set_session_value(session, f"SET SESSION innodb_lock_wait_timeout = {_LEADERBOARD_LOCK_WAIT_SECONDS}")
|
||||
elif name == "postgresql":
|
||||
elif name in {"postgresql", "opengauss"}:
|
||||
await _try_set_session_value(session, f"SET LOCAL statement_timeout = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
|
||||
|
||||
|
||||
|
||||
@@ -1,49 +1,50 @@
|
||||
import asyncio
|
||||
import logging
|
||||
import mimetypes
|
||||
import logging
|
||||
import mimetypes
|
||||
import time
|
||||
import zipfile
|
||||
import zipfile
|
||||
from collections import OrderedDict
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
from urllib.parse import quote
|
||||
|
||||
from urllib.parse import quote
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
from fastapi.responses import FileResponse, PlainTextResponse, Response
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.gateway.authz import require_permission
|
||||
from app.gateway.path_utils import aresolve_thread_virtual_path
|
||||
from deerflow.config.paths import get_paths
|
||||
from deerflow.runtime.thread_paths import aresolve_path_user_id
|
||||
from fastapi.responses import FileResponse, PlainTextResponse, Response
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.gateway.authz import require_permission
|
||||
from app.gateway.path_utils import aresolve_thread_virtual_path
|
||||
from deerflow.config.paths import get_paths
|
||||
from deerflow.runtime.thread_paths import aresolve_path_user_id
|
||||
from deerflow.runtime.user_context import get_effective_user_id
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api", tags=["artifacts"])
|
||||
|
||||
ACTIVE_CONTENT_MIME_TYPES = {
|
||||
"text/html",
|
||||
"application/xhtml+xml",
|
||||
"image/svg+xml",
|
||||
}
|
||||
|
||||
EDITABLE_TEXT_SUFFIXES = {
|
||||
".md",
|
||||
".markdown",
|
||||
".txt",
|
||||
".json",
|
||||
".yaml",
|
||||
".yml",
|
||||
".csv",
|
||||
".xml",
|
||||
".log",
|
||||
}
|
||||
|
||||
_VIRTUAL_USER_DATA_PREFIX = "mnt/user-data"
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api", tags=["artifacts"])
|
||||
|
||||
ACTIVE_CONTENT_MIME_TYPES = {
|
||||
"text/html",
|
||||
"application/xhtml+xml",
|
||||
"image/svg+xml",
|
||||
}
|
||||
|
||||
EDITABLE_TEXT_SUFFIXES = {
|
||||
".md",
|
||||
".markdown",
|
||||
".txt",
|
||||
".json",
|
||||
".yaml",
|
||||
".yml",
|
||||
".csv",
|
||||
".xml",
|
||||
".log",
|
||||
}
|
||||
|
||||
_VIRTUAL_USER_DATA_PREFIX = "mnt/user-data"
|
||||
|
||||
ArtifactLibraryKind = Literal["all", "document", "spreadsheet", "text_data", "archive", "other"]
|
||||
ArtifactLibrarySource = Literal["all", "general_qa", "agent"]
|
||||
|
||||
_MEDIA_MIME_PREFIXES = ("image/", "audio/", "video/")
|
||||
_MEDIA_SUFFIXES = {
|
||||
@@ -121,35 +122,37 @@ _ARTIFACT_LIBRARY_SCAN_CONCURRENCY = 8
|
||||
_ARTIFACT_LIBRARY_THREAD_BATCH_SIZE = 500
|
||||
_artifact_library_cache: OrderedDict[str, tuple[float, list[dict]]] = OrderedDict()
|
||||
|
||||
|
||||
def coerce_artifact_virtual_path(path: str) -> str:
|
||||
"""Normalize an artifacts URL path to a sandbox virtual path.
|
||||
|
||||
Streaming write previews may request a bare filename such as
|
||||
``未命名文件.md`` before the model has emitted a real
|
||||
``/mnt/user-data/outputs/...`` path. Mapping a single path segment into
|
||||
the thread outputs directory turns that into a normal 404 (file not written
|
||||
yet) instead of HTTP 400 ``Path must start with /mnt/user-data``.
|
||||
"""
|
||||
stripped = path.strip().lstrip("/").replace("\\", "/")
|
||||
if not stripped:
|
||||
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
||||
if stripped == _VIRTUAL_USER_DATA_PREFIX or stripped.startswith(_VIRTUAL_USER_DATA_PREFIX + "/"):
|
||||
return stripped
|
||||
if "/" in stripped or stripped in {".", ".."}:
|
||||
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
||||
return f"{_VIRTUAL_USER_DATA_PREFIX}/outputs/{stripped}"
|
||||
|
||||
|
||||
class ArtifactUpdateRequest(BaseModel):
|
||||
content: str
|
||||
|
||||
def coerce_artifact_virtual_path(path: str) -> str:
|
||||
"""Normalize an artifacts URL path to a sandbox virtual path.
|
||||
|
||||
Streaming write previews may request a bare filename such as
|
||||
``未命名文件.md`` before the model has emitted a real
|
||||
``/mnt/user-data/outputs/...`` path. Mapping a single path segment into
|
||||
the thread outputs directory turns that into a normal 404 (file not written
|
||||
yet) instead of HTTP 400 ``Path must start with /mnt/user-data``.
|
||||
"""
|
||||
stripped = path.strip().lstrip("/").replace("\\", "/")
|
||||
if not stripped:
|
||||
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
||||
if stripped == _VIRTUAL_USER_DATA_PREFIX or stripped.startswith(_VIRTUAL_USER_DATA_PREFIX + "/"):
|
||||
return stripped
|
||||
if "/" in stripped or stripped in {".", ".."}:
|
||||
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
||||
return f"{_VIRTUAL_USER_DATA_PREFIX}/outputs/{stripped}"
|
||||
|
||||
|
||||
class ArtifactUpdateRequest(BaseModel):
|
||||
content: str
|
||||
|
||||
|
||||
class ArtifactLibraryItem(BaseModel):
|
||||
thread_id: str
|
||||
thread_title: str
|
||||
agent_id: str | None = None
|
||||
agent_name: str | None = None
|
||||
thread_type: str | None = None
|
||||
source_type: Literal["general_qa", "agent"]
|
||||
path: str
|
||||
name: str
|
||||
size_bytes: int
|
||||
@@ -163,85 +166,85 @@ class ArtifactLibraryResponse(BaseModel):
|
||||
offset: int
|
||||
limit: int
|
||||
items: list[ArtifactLibraryItem]
|
||||
|
||||
|
||||
def _build_content_disposition(disposition_type: str, filename: str) -> str:
|
||||
"""Build an RFC 5987 encoded Content-Disposition header value."""
|
||||
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
|
||||
|
||||
|
||||
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
|
||||
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
|
||||
if extra_headers:
|
||||
headers.update(extra_headers)
|
||||
return headers
|
||||
|
||||
|
||||
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
|
||||
"""Check if file is text by examining content for null bytes."""
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
chunk = f.read(sample_size)
|
||||
# Text files shouldn't contain null bytes
|
||||
return b"\x00" not in chunk
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
|
||||
"""Extract a file from a .skill ZIP archive.
|
||||
|
||||
Args:
|
||||
zip_path: Path to the .skill file (ZIP archive).
|
||||
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
|
||||
|
||||
Returns:
|
||||
The file content as bytes, or None if not found.
|
||||
"""
|
||||
if not zipfile.is_zipfile(zip_path):
|
||||
return None
|
||||
|
||||
try:
|
||||
with zipfile.ZipFile(zip_path, "r") as zip_ref:
|
||||
# List all files in the archive
|
||||
namelist = zip_ref.namelist()
|
||||
|
||||
# Try direct path first
|
||||
if internal_path in namelist:
|
||||
return zip_ref.read(internal_path)
|
||||
|
||||
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
|
||||
for name in namelist:
|
||||
if name.endswith("/" + internal_path) or name == internal_path:
|
||||
return zip_ref.read(name)
|
||||
|
||||
# Not found
|
||||
return None
|
||||
except (zipfile.BadZipFile, KeyError):
|
||||
return None
|
||||
|
||||
|
||||
def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
|
||||
"""Walk an outputs directory and return artifact listing entries.
|
||||
|
||||
Each entry has ``path`` (virtual ``mnt/user-data/outputs/...``), ``name``,
|
||||
``size_bytes``, ``mime_type``, and ``modified_at`` (ISO 8601 UTC). Symlinks
|
||||
that escape the outputs root are dropped to keep the listing scoped to the
|
||||
thread. The result is sorted by ``modified_at`` descending (newest first).
|
||||
"""
|
||||
if not outputs_dir.exists() or not outputs_dir.is_dir():
|
||||
return []
|
||||
|
||||
outputs_root = outputs_dir.resolve()
|
||||
files: list[dict] = []
|
||||
for entry in outputs_root.rglob("*"):
|
||||
if not entry.is_file():
|
||||
continue
|
||||
try:
|
||||
resolved = entry.resolve()
|
||||
resolved.relative_to(outputs_root)
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
|
||||
|
||||
def _build_content_disposition(disposition_type: str, filename: str) -> str:
|
||||
"""Build an RFC 5987 encoded Content-Disposition header value."""
|
||||
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
|
||||
|
||||
|
||||
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
|
||||
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
|
||||
if extra_headers:
|
||||
headers.update(extra_headers)
|
||||
return headers
|
||||
|
||||
|
||||
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
|
||||
"""Check if file is text by examining content for null bytes."""
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
chunk = f.read(sample_size)
|
||||
# Text files shouldn't contain null bytes
|
||||
return b"\x00" not in chunk
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
|
||||
"""Extract a file from a .skill ZIP archive.
|
||||
|
||||
Args:
|
||||
zip_path: Path to the .skill file (ZIP archive).
|
||||
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
|
||||
|
||||
Returns:
|
||||
The file content as bytes, or None if not found.
|
||||
"""
|
||||
if not zipfile.is_zipfile(zip_path):
|
||||
return None
|
||||
|
||||
try:
|
||||
with zipfile.ZipFile(zip_path, "r") as zip_ref:
|
||||
# List all files in the archive
|
||||
namelist = zip_ref.namelist()
|
||||
|
||||
# Try direct path first
|
||||
if internal_path in namelist:
|
||||
return zip_ref.read(internal_path)
|
||||
|
||||
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
|
||||
for name in namelist:
|
||||
if name.endswith("/" + internal_path) or name == internal_path:
|
||||
return zip_ref.read(name)
|
||||
|
||||
# Not found
|
||||
return None
|
||||
except (zipfile.BadZipFile, KeyError):
|
||||
return None
|
||||
|
||||
|
||||
def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
|
||||
"""Walk an outputs directory and return artifact listing entries.
|
||||
|
||||
Each entry has ``path`` (virtual ``mnt/user-data/outputs/...``), ``name``,
|
||||
``size_bytes``, ``mime_type``, and ``modified_at`` (ISO 8601 UTC). Symlinks
|
||||
that escape the outputs root are dropped to keep the listing scoped to the
|
||||
thread. The result is sorted by ``modified_at`` descending (newest first).
|
||||
"""
|
||||
if not outputs_dir.exists() or not outputs_dir.is_dir():
|
||||
return []
|
||||
|
||||
outputs_root = outputs_dir.resolve()
|
||||
files: list[dict] = []
|
||||
for entry in outputs_root.rglob("*"):
|
||||
if not entry.is_file():
|
||||
continue
|
||||
try:
|
||||
resolved = entry.resolve()
|
||||
resolved.relative_to(outputs_root)
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
try:
|
||||
relative = resolved.relative_to(outputs_root).as_posix()
|
||||
mime_type, _ = mimetypes.guess_type(resolved.name)
|
||||
@@ -251,18 +254,18 @@ def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
|
||||
# is being scanned. Treat that transient entry as absent instead of
|
||||
# failing the complete thread/library listing.
|
||||
continue
|
||||
files.append(
|
||||
{
|
||||
"path": f"mnt/user-data/outputs/{relative}",
|
||||
"name": resolved.name,
|
||||
"size_bytes": stat.st_size,
|
||||
"mime_type": mime_type,
|
||||
files.append(
|
||||
{
|
||||
"path": f"mnt/user-data/outputs/{relative}",
|
||||
"name": resolved.name,
|
||||
"size_bytes": stat.st_size,
|
||||
"mime_type": mime_type,
|
||||
"modified_at": datetime.fromtimestamp(stat.st_mtime, tz=UTC).isoformat(),
|
||||
}
|
||||
)
|
||||
|
||||
files.sort(key=lambda f: f["modified_at"], reverse=True)
|
||||
return files
|
||||
}
|
||||
)
|
||||
|
||||
files.sort(key=lambda f: f["modified_at"], reverse=True)
|
||||
return files
|
||||
|
||||
|
||||
def artifact_library_kind(name: str) -> str:
|
||||
@@ -335,7 +338,10 @@ async def _scan_artifact_library(thread_store) -> list[dict]:
|
||||
|
||||
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
|
||||
agent_id = metadata.get("agent_id")
|
||||
agent_name = metadata.get("agent_name")
|
||||
thread_type = metadata.get("thread_type")
|
||||
normalized_agent_id = agent_id.strip() if isinstance(agent_id, str) else ""
|
||||
normalized_agent_name = agent_name.strip() if isinstance(agent_name, str) else ""
|
||||
title = str(row.get("display_name") or "新对话")
|
||||
items: list[dict] = []
|
||||
for file in files:
|
||||
@@ -345,8 +351,10 @@ async def _scan_artifact_library(thread_store) -> list[dict]:
|
||||
{
|
||||
"thread_id": thread_id,
|
||||
"thread_title": title,
|
||||
"agent_id": agent_id if isinstance(agent_id, str) and agent_id else None,
|
||||
"agent_id": normalized_agent_id or None,
|
||||
"agent_name": normalized_agent_name or None,
|
||||
"thread_type": thread_type if isinstance(thread_type, str) and thread_type else None,
|
||||
"source_type": "agent" if normalized_agent_id else "general_qa",
|
||||
**file,
|
||||
"file_kind": artifact_library_kind(str(file.get("name") or "")),
|
||||
}
|
||||
@@ -393,6 +401,7 @@ async def list_artifact_library(
|
||||
request: Request,
|
||||
query: str | None = Query(default=None, max_length=200),
|
||||
file_kind: ArtifactLibraryKind = Query(default="all"),
|
||||
source_type: ArtifactLibrarySource = Query(default="all"),
|
||||
offset: int = Query(default=0, ge=0),
|
||||
limit: int = Query(default=50, ge=1, le=200),
|
||||
refresh: bool = Query(default=False),
|
||||
@@ -404,14 +413,35 @@ async def list_artifact_library(
|
||||
explicit ``present_files`` call. Uploads and image/audio/video outputs are
|
||||
intentionally outside this collection.
|
||||
"""
|
||||
from app.gateway.deps import get_thread_store
|
||||
from app.gateway.deps import get_agent_store, get_thread_store
|
||||
|
||||
items = await _cached_artifact_library(get_thread_store(request), refresh=refresh)
|
||||
agent_ids = list(dict.fromkeys(str(item.get("agent_id") or "") for item in items if item.get("agent_id")))
|
||||
agent_names: dict[str, str] = {}
|
||||
if agent_ids:
|
||||
try:
|
||||
agents = await get_agent_store(request).list_by_ids(
|
||||
agent_ids,
|
||||
str(get_effective_user_id() or "default"),
|
||||
)
|
||||
agent_names = {str(agent.get("id") or ""): str(agent.get("name") or "").strip() for agent in agents if agent.get("id") and agent.get("name")}
|
||||
except Exception:
|
||||
logger.warning("Failed to resolve artifact library agent names", exc_info=True)
|
||||
|
||||
items_with_sources = [
|
||||
{
|
||||
**item,
|
||||
"agent_name": (agent_names.get(str(item.get("agent_id") or "")) or item.get("agent_name") or item.get("agent_id")),
|
||||
}
|
||||
for item in items
|
||||
]
|
||||
normalized_query = (query or "").strip().casefold()
|
||||
filtered = [
|
||||
item
|
||||
for item in items
|
||||
if (file_kind == "all" or item["file_kind"] == file_kind) and (not normalized_query or normalized_query in str(item.get("name") or "").casefold() or normalized_query in str(item.get("thread_title") or "").casefold())
|
||||
for item in items_with_sources
|
||||
if (file_kind == "all" or item["file_kind"] == file_kind)
|
||||
and (source_type == "all" or item["source_type"] == source_type)
|
||||
and (not normalized_query or normalized_query in str(item.get("name") or "").casefold() or normalized_query in str(item.get("thread_title") or "").casefold() or normalized_query in str(item.get("agent_name") or "").casefold())
|
||||
]
|
||||
page = filtered[offset : offset + limit]
|
||||
return ArtifactLibraryResponse(
|
||||
@@ -420,176 +450,176 @@ async def list_artifact_library(
|
||||
limit=limit,
|
||||
items=[ArtifactLibraryItem.model_validate(item) for item in page],
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/threads/{thread_id}/artifacts",
|
||||
summary="List Artifact Files",
|
||||
description="List all files under the thread's outputs directory (/mnt/user-data/outputs/). Returns an empty list if the thread has no outputs.",
|
||||
)
|
||||
@require_permission("threads", "read", owner_check=True)
|
||||
async def list_artifacts(thread_id: str, request: Request) -> dict:
|
||||
"""列出 thread 在 ``outputs/`` 下生成的产物文件。
|
||||
|
||||
圆桌 Step 3(``HighFidelityReport``)对每个**子席位** thread_id 调用本接口,
|
||||
展示各智能体写入沙箱 ``mnt/user-data/outputs/`` 的文件;协调智能体 thread
|
||||
通常无独立交付文件。
|
||||
|
||||
遍历路径::
|
||||
|
||||
.deer-flow/users/{user_id}/threads/{thread_id}/user-data/outputs/
|
||||
|
||||
返回 ``{ "thread_id", "files": [{ path, name, size_bytes, mime_type, modified_at }] }``,
|
||||
目录不存在时 ``files`` 为空数组(非 404)。
|
||||
"""
|
||||
outputs_dir = get_paths().sandbox_outputs_dir(thread_id, user_id=await aresolve_path_user_id(thread_id))
|
||||
return {"thread_id": thread_id, "files": collect_outputs_listing(outputs_dir)}
|
||||
|
||||
|
||||
@router.get(
|
||||
"/threads/{thread_id}/artifacts/{path:path}",
|
||||
summary="Get Artifact File",
|
||||
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
|
||||
)
|
||||
@require_permission("threads", "read", owner_check=True)
|
||||
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
|
||||
"""Get an artifact file by its path.
|
||||
|
||||
The endpoint automatically detects file types and returns appropriate content types.
|
||||
Use the `download` query parameter to force file download for non-active content.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID.
|
||||
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
|
||||
request: FastAPI request object (automatically injected).
|
||||
|
||||
Returns:
|
||||
The file content as a FileResponse with appropriate content type:
|
||||
- Active content (HTML/XHTML/SVG): Served as download attachment
|
||||
- Text files: Plain text with proper MIME type
|
||||
- Binary files: Inline display with download option
|
||||
|
||||
Raises:
|
||||
HTTPException:
|
||||
- 400 if path is invalid or not a file
|
||||
- 403 if access denied (path traversal detected)
|
||||
- 404 if file not found
|
||||
|
||||
Query Parameters:
|
||||
download (bool): If true, forces attachment download for file types that are
|
||||
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
|
||||
is always downloaded regardless of this flag.
|
||||
|
||||
Example:
|
||||
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
|
||||
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
|
||||
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
|
||||
"""
|
||||
path = coerce_artifact_virtual_path(path)
|
||||
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
|
||||
if ".skill/" in path:
|
||||
# Split the path at ".skill/" to get the ZIP file path and internal path
|
||||
skill_marker = ".skill/"
|
||||
marker_pos = path.find(skill_marker)
|
||||
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
|
||||
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
|
||||
|
||||
actual_skill_path = await aresolve_thread_virtual_path(thread_id, skill_file_path)
|
||||
|
||||
if not actual_skill_path.exists():
|
||||
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
|
||||
|
||||
if not actual_skill_path.is_file():
|
||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
|
||||
|
||||
# Extract the file from the .skill archive
|
||||
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
|
||||
if content is None:
|
||||
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
|
||||
|
||||
# Determine MIME type based on the internal file
|
||||
mime_type, _ = mimetypes.guess_type(internal_path)
|
||||
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
|
||||
cache_headers = {"Cache-Control": "private, max-age=300"}
|
||||
download_name = Path(internal_path).name or actual_skill_path.stem
|
||||
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
|
||||
|
||||
if mime_type and mime_type.startswith("text/"):
|
||||
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
|
||||
|
||||
# Default to plain text for unknown types that look like text
|
||||
try:
|
||||
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
|
||||
except UnicodeDecodeError:
|
||||
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
|
||||
|
||||
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
||||
|
||||
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
||||
|
||||
if not actual_path.exists():
|
||||
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
||||
|
||||
if not actual_path.is_file():
|
||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
||||
|
||||
mime_type, _ = mimetypes.guess_type(actual_path)
|
||||
|
||||
if download:
|
||||
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
||||
|
||||
# Always force download for active content types to prevent script execution
|
||||
# in the application origin when users open generated artifacts.
|
||||
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
||||
|
||||
if mime_type and mime_type.startswith("text/"):
|
||||
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
||||
|
||||
if is_text_file_by_content(actual_path):
|
||||
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
||||
|
||||
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/threads/{thread_id}/artifacts/{path:path}",
|
||||
summary="Update Artifact File",
|
||||
description="Update a text artifact file in the current thread workspace.",
|
||||
)
|
||||
@require_permission("threads", "write", owner_check=True)
|
||||
async def update_artifact(
|
||||
thread_id: str,
|
||||
path: str,
|
||||
request: Request,
|
||||
body: ArtifactUpdateRequest,
|
||||
) -> Response:
|
||||
"""Update a text artifact.
|
||||
|
||||
This endpoint is intentionally limited to normal text artifacts. Files inside
|
||||
.skill archives and active web content remain read-only from the artifact UI.
|
||||
"""
|
||||
if ".skill/" in path:
|
||||
raise HTTPException(status_code=400, detail="Skill archive contents are read-only.")
|
||||
|
||||
path = coerce_artifact_virtual_path(path)
|
||||
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
||||
logger.info(f"Updating artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
||||
|
||||
if not actual_path.exists():
|
||||
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
||||
|
||||
if not actual_path.is_file():
|
||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
||||
|
||||
mime_type, _ = mimetypes.guess_type(actual_path)
|
||||
suffix = actual_path.suffix.lower()
|
||||
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||
raise HTTPException(status_code=400, detail="Active web artifacts are read-only.")
|
||||
|
||||
if suffix not in EDITABLE_TEXT_SUFFIXES and not (mime_type and mime_type.startswith("text/")):
|
||||
raise HTTPException(status_code=400, detail="Only text artifacts can be edited.")
|
||||
|
||||
actual_path.write_text(body.content, encoding="utf-8")
|
||||
return PlainTextResponse(content=body.content, media_type=mime_type or "text/plain")
|
||||
|
||||
|
||||
@router.get(
|
||||
"/threads/{thread_id}/artifacts",
|
||||
summary="List Artifact Files",
|
||||
description="List all files under the thread's outputs directory (/mnt/user-data/outputs/). Returns an empty list if the thread has no outputs.",
|
||||
)
|
||||
@require_permission("threads", "read", owner_check=True)
|
||||
async def list_artifacts(thread_id: str, request: Request) -> dict:
|
||||
"""列出 thread 在 ``outputs/`` 下生成的产物文件。
|
||||
|
||||
圆桌 Step 3(``HighFidelityReport``)对每个**子席位** thread_id 调用本接口,
|
||||
展示各智能体写入沙箱 ``mnt/user-data/outputs/`` 的文件;协调智能体 thread
|
||||
通常无独立交付文件。
|
||||
|
||||
遍历路径::
|
||||
|
||||
.deer-flow/users/{user_id}/threads/{thread_id}/user-data/outputs/
|
||||
|
||||
返回 ``{ "thread_id", "files": [{ path, name, size_bytes, mime_type, modified_at }] }``,
|
||||
目录不存在时 ``files`` 为空数组(非 404)。
|
||||
"""
|
||||
outputs_dir = get_paths().sandbox_outputs_dir(thread_id, user_id=await aresolve_path_user_id(thread_id))
|
||||
return {"thread_id": thread_id, "files": collect_outputs_listing(outputs_dir)}
|
||||
|
||||
|
||||
@router.get(
|
||||
"/threads/{thread_id}/artifacts/{path:path}",
|
||||
summary="Get Artifact File",
|
||||
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
|
||||
)
|
||||
@require_permission("threads", "read", owner_check=True)
|
||||
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
|
||||
"""Get an artifact file by its path.
|
||||
|
||||
The endpoint automatically detects file types and returns appropriate content types.
|
||||
Use the `download` query parameter to force file download for non-active content.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID.
|
||||
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
|
||||
request: FastAPI request object (automatically injected).
|
||||
|
||||
Returns:
|
||||
The file content as a FileResponse with appropriate content type:
|
||||
- Active content (HTML/XHTML/SVG): Served as download attachment
|
||||
- Text files: Plain text with proper MIME type
|
||||
- Binary files: Inline display with download option
|
||||
|
||||
Raises:
|
||||
HTTPException:
|
||||
- 400 if path is invalid or not a file
|
||||
- 403 if access denied (path traversal detected)
|
||||
- 404 if file not found
|
||||
|
||||
Query Parameters:
|
||||
download (bool): If true, forces attachment download for file types that are
|
||||
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
|
||||
is always downloaded regardless of this flag.
|
||||
|
||||
Example:
|
||||
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
|
||||
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
|
||||
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
|
||||
"""
|
||||
path = coerce_artifact_virtual_path(path)
|
||||
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
|
||||
if ".skill/" in path:
|
||||
# Split the path at ".skill/" to get the ZIP file path and internal path
|
||||
skill_marker = ".skill/"
|
||||
marker_pos = path.find(skill_marker)
|
||||
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
|
||||
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
|
||||
|
||||
actual_skill_path = await aresolve_thread_virtual_path(thread_id, skill_file_path)
|
||||
|
||||
if not actual_skill_path.exists():
|
||||
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
|
||||
|
||||
if not actual_skill_path.is_file():
|
||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
|
||||
|
||||
# Extract the file from the .skill archive
|
||||
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
|
||||
if content is None:
|
||||
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
|
||||
|
||||
# Determine MIME type based on the internal file
|
||||
mime_type, _ = mimetypes.guess_type(internal_path)
|
||||
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
|
||||
cache_headers = {"Cache-Control": "private, max-age=300"}
|
||||
download_name = Path(internal_path).name or actual_skill_path.stem
|
||||
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
|
||||
|
||||
if mime_type and mime_type.startswith("text/"):
|
||||
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
|
||||
|
||||
# Default to plain text for unknown types that look like text
|
||||
try:
|
||||
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
|
||||
except UnicodeDecodeError:
|
||||
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
|
||||
|
||||
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
||||
|
||||
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
||||
|
||||
if not actual_path.exists():
|
||||
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
||||
|
||||
if not actual_path.is_file():
|
||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
||||
|
||||
mime_type, _ = mimetypes.guess_type(actual_path)
|
||||
|
||||
if download:
|
||||
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
||||
|
||||
# Always force download for active content types to prevent script execution
|
||||
# in the application origin when users open generated artifacts.
|
||||
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
||||
|
||||
if mime_type and mime_type.startswith("text/"):
|
||||
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
||||
|
||||
if is_text_file_by_content(actual_path):
|
||||
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
||||
|
||||
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/threads/{thread_id}/artifacts/{path:path}",
|
||||
summary="Update Artifact File",
|
||||
description="Update a text artifact file in the current thread workspace.",
|
||||
)
|
||||
@require_permission("threads", "write", owner_check=True)
|
||||
async def update_artifact(
|
||||
thread_id: str,
|
||||
path: str,
|
||||
request: Request,
|
||||
body: ArtifactUpdateRequest,
|
||||
) -> Response:
|
||||
"""Update a text artifact.
|
||||
|
||||
This endpoint is intentionally limited to normal text artifacts. Files inside
|
||||
.skill archives and active web content remain read-only from the artifact UI.
|
||||
"""
|
||||
if ".skill/" in path:
|
||||
raise HTTPException(status_code=400, detail="Skill archive contents are read-only.")
|
||||
|
||||
path = coerce_artifact_virtual_path(path)
|
||||
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
||||
logger.info(f"Updating artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
||||
|
||||
if not actual_path.exists():
|
||||
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
||||
|
||||
if not actual_path.is_file():
|
||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
||||
|
||||
mime_type, _ = mimetypes.guess_type(actual_path)
|
||||
suffix = actual_path.suffix.lower()
|
||||
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||
raise HTTPException(status_code=400, detail="Active web artifacts are read-only.")
|
||||
|
||||
if suffix not in EDITABLE_TEXT_SUFFIXES and not (mime_type and mime_type.startswith("text/")):
|
||||
raise HTTPException(status_code=400, detail="Only text artifacts can be edited.")
|
||||
|
||||
actual_path.write_text(body.content, encoding="utf-8")
|
||||
return PlainTextResponse(content=body.content, media_type=mime_type or "text/plain")
|
||||
File diff suppressed because it is too large.
Load diff
@@ -172,21 +172,15 @@ async def create_sync_job(request: Request, body: CreateSyncJobRequest) -> dict[
|
||||
targets: list[dict[str, Any]] = []
|
||||
for target in body.targets:
|
||||
data = target.model_dump()
|
||||
if target.target_type == "weknora":
|
||||
mapping = await request.app.state.llmwiki_store.get_authorized(target.target_id, actor or "system", write=True, is_admin=True)
|
||||
if mapping is None:
|
||||
raise HTTPException(status_code=404, detail=f"普通知识库不存在:{target.target_id}")
|
||||
data["target_name"] = target.target_name or str(mapping.get("name") or target.target_id)
|
||||
data["remote_business_key"] = target.remote_business_key or f"weknora:kb:{mapping['weknora_id']}"
|
||||
else:
|
||||
base = await request.app.state.assistant_knowledge_store.get_base()
|
||||
if base is None:
|
||||
base = await request.app.state.assistant_knowledge_store.initialize(created_by=actor)
|
||||
if target.target_id not in {base["id"], "global"}:
|
||||
raise HTTPException(status_code=404, detail="助手知识库仅允许选择全局知识梳理库")
|
||||
data["target_id"] = base["id"]
|
||||
data["target_name"] = base["name"]
|
||||
data["remote_business_key"] = f"assistant:base:{base['id']}"
|
||||
if target.target_type != "weknora":
|
||||
raise HTTPException(status_code=422, detail="不支持技能直接归纳到助手知识库,请先归纳到 WeKnora 普通知识库")
|
||||
if target.target_mode != "wiki":
|
||||
raise HTTPException(status_code=422, detail="技能归纳目标仅支持 WeKnora Wiki 知识库")
|
||||
mapping = await request.app.state.llmwiki_store.get_authorized(target.target_id, actor or "system", write=True, is_admin=True)
|
||||
if mapping is None:
|
||||
raise HTTPException(status_code=404, detail=f"普通知识库不存在:{target.target_id}")
|
||||
data["target_name"] = target.target_name or str(mapping.get("name") or target.target_id)
|
||||
data["remote_business_key"] = target.remote_business_key or f"weknora:kb:{mapping['weknora_id']}"
|
||||
targets.append(data)
|
||||
job = await _store(request).create_job(
|
||||
skill_names=body.skill_names,
|
||||
|
||||
@@ -405,6 +405,7 @@ _CONTEXT_CONFIGURABLE_KEYS: frozenset[str] = frozenset(
|
||||
"reasoning_effort",
|
||||
"rag_mode_enabled",
|
||||
"llmwiki_knowledge_base_ids",
|
||||
"assistant_knowledge_base_ids",
|
||||
"writing_mode",
|
||||
"writing_artifact_path",
|
||||
"canvas_artifact_title",
|
||||
@@ -1113,7 +1114,7 @@ async def start_run(
|
||||
if prefix_settings.enabled and prefix_settings.prompt_prefix.strip():
|
||||
graph_input = _prepend_prompt_prefix(graph_input, prefix_settings.prompt_prefix.strip())
|
||||
llmwiki_rag = (
|
||||
LlmWikiRagContext([], 0, "", [], "")
|
||||
LlmWikiRagContext([], [], 0, "", [], "")
|
||||
if fixed_answer is not None
|
||||
else await build_llmwiki_rag_context(
|
||||
request=request,
|
||||
@@ -1125,12 +1126,14 @@ async def start_run(
|
||||
graph_input = attach_llmwiki_rag_to_graph_input(graph_input, llmwiki_rag)
|
||||
effective_rag_mode = True
|
||||
run_metadata["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
|
||||
run_metadata["assistant_knowledge_base_ids"] = llmwiki_rag.assistant_knowledge_base_ids
|
||||
run_metadata["llmwiki_rag_source_count"] = llmwiki_rag.source_count
|
||||
try:
|
||||
await run_ctx.thread_store.update_metadata(
|
||||
thread_id,
|
||||
{
|
||||
"llmwiki_knowledge_base_ids": llmwiki_rag.knowledge_base_ids,
|
||||
"assistant_knowledge_base_ids": llmwiki_rag.assistant_knowledge_base_ids,
|
||||
"rag_mode_enabled": True,
|
||||
"rag_mode": True,
|
||||
},
|
||||
@@ -1150,6 +1153,8 @@ async def start_run(
|
||||
body_context["rag_mode_enabled"] = True
|
||||
if "llmwiki_knowledge_base_ids" not in body_context and llmwiki_rag.knowledge_base_ids:
|
||||
body_context["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
|
||||
if "assistant_knowledge_base_ids" not in body_context and llmwiki_rag.assistant_knowledge_base_ids:
|
||||
body_context["assistant_knowledge_base_ids"] = llmwiki_rag.assistant_knowledge_base_ids
|
||||
if not llmwiki_rag.knowledge_base_ids:
|
||||
existing_excluded = list(body_context.get("excluded_tools") or getattr(body, "excluded_tools", None) or [])
|
||||
if "llmwiki_search" not in existing_excluded:
|
||||
|
||||
@@ -429,7 +429,11 @@ auth_login:
|
||||
database:
|
||||
backend: sqlite
|
||||
sqlite_dir: .deer-flow/data
|
||||
pool_size: 20 # 基础连接池(postgres/mysql 生效;sqlite 忽略)
|
||||
# 高斯/OpenGauss 业务库示例(启用前先执行 uv sync --extra gauss):
|
||||
# backend: gauss
|
||||
# gauss_url: $GAUSS_DATABASE_URL # opengauss+asyncpg://user:pass@host:26000/deerflow
|
||||
# checkpointer 请继续使用下方 sqlite;不要把 PostgreSQL 专用 checkpointer 迁移直接用于高斯。
|
||||
pool_size: 20 # 基础连接池(postgres/gauss/mysql 生效;sqlite 忽略)
|
||||
max_overflow: 20 # 高峰期可临时超出 pool_size 的额外连接数,真实上限 = pool_size + max_overflow
|
||||
pool_timeout: 30 # 池满时等待空闲连接的秒数,超时报错
|
||||
pool_recycle: 1800 # 连接回收周期(秒);防火墙/pgbouncer/MySQL wait_timeout 静默断连兜底。多 worker 部署:每 worker 上限 = pool_size + max_overflow,总数 × worker 数 < DB max_connections
|
||||
@@ -564,9 +568,10 @@ llmwiki:
|
||||
password: ""
|
||||
database: "neo4j"
|
||||
timeout_seconds: 30
|
||||
# 本地 Wiki 向量化:只需把 enabled 改为 true,并填写下面 3 个 embedding 值。
|
||||
# 默认只在页面点击“向量化”时执行;auto_sync=true 才会后台自动增量同步,
|
||||
# 且服务启动时只启动定时器,不会立刻全量向量化。
|
||||
# 本地 Wiki 向量化:用于普通知识库“下载向量化数据包”和助手知识库向量检索。
|
||||
# 已有相同内容哈希 + 相同 embedding fingerprint 的 Wiki 向量会直接复用,不会重复编码。
|
||||
# 如需导出时自动补建向量,把 enabled 改为 true,并填写下面 embedding 服务地址。
|
||||
# auto_sync=true 才会后台自动增量同步;服务启动时只启动定时器,不会立刻全量向量化。
|
||||
local_wiki_index:
|
||||
enabled: false
|
||||
# 免登录 POST /api/knowledge/vector-search 返回的 Wiki 一键跳转链接前缀。
|
||||
@@ -574,7 +579,7 @@ llmwiki:
|
||||
auto_sync: false
|
||||
database_batch_size: 50
|
||||
embedding:
|
||||
model: bge-m3
|
||||
model: bge-embedding-m3
|
||||
base_url: ""
|
||||
api_key: ""
|
||||
|
||||
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
"""File-backed assistant knowledge import/export packages.
|
||||
|
||||
The assistant knowledge base is fed by WeKnora-exported data. Keep that
|
||||
boundary explicit: a WeKnora export is first materialized as a durable JSONL
|
||||
package, then imported asynchronously from that package. JSONL keeps large
|
||||
exports append-friendly and avoids treating the HTTP request body as the job
|
||||
state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Iterable, Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
PACKAGE_VERSION = 1
|
||||
PACKAGE_RECORD_TYPES = {
|
||||
"source",
|
||||
"wiki_page",
|
||||
"document",
|
||||
"chunk",
|
||||
"vector",
|
||||
"entity",
|
||||
"relation",
|
||||
"graph",
|
||||
}
|
||||
|
||||
|
||||
def package_path(root: Path, job_id: str) -> Path:
|
||||
safe = "".join(ch for ch in str(job_id) if ch.isalnum() or ch in {"-", "_"})
|
||||
return root / f"{safe}.jsonl"
|
||||
|
||||
|
||||
def write_jsonl_record(handle: Any, record_type: str, payload: Mapping[str, Any]) -> None:
|
||||
handle.write(json.dumps({"record_type": record_type, "payload": dict(payload)}, ensure_ascii=False))
|
||||
handle.write("\n")
|
||||
|
||||
|
||||
def write_package(path: Path, records: Iterable[tuple[str, Mapping[str, Any]]]) -> dict[str, int]:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
counts: dict[str, int] = {key: 0 for key in PACKAGE_RECORD_TYPES}
|
||||
tmp_path = path.with_suffix(path.suffix + ".tmp")
|
||||
with tmp_path.open("w", encoding="utf-8", newline="\n") as handle:
|
||||
write_jsonl_record(
|
||||
handle,
|
||||
"source",
|
||||
{
|
||||
"package_version": PACKAGE_VERSION,
|
||||
},
|
||||
)
|
||||
counts["source"] += 1
|
||||
for record_type, payload in records:
|
||||
if record_type not in PACKAGE_RECORD_TYPES:
|
||||
continue
|
||||
write_jsonl_record(handle, record_type, payload)
|
||||
counts[record_type] = counts.get(record_type, 0) + 1
|
||||
tmp_path.replace(path)
|
||||
return counts
|
||||
|
||||
|
||||
def read_package(path: Path) -> dict[str, Any]:
|
||||
package: dict[str, Any] = {
|
||||
"source": {},
|
||||
"wiki_pages": [],
|
||||
"documents": [],
|
||||
"chunks": [],
|
||||
"vectors": [],
|
||||
"entities": [],
|
||||
"relations": [],
|
||||
"graph": {"nodes": [], "edges": []},
|
||||
}
|
||||
with path.open("r", encoding="utf-8") as handle:
|
||||
for line in handle:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
record = json.loads(line)
|
||||
if not isinstance(record, dict):
|
||||
continue
|
||||
record_type = str(record.get("record_type") or "")
|
||||
payload = record.get("payload")
|
||||
if not isinstance(payload, dict):
|
||||
continue
|
||||
if record_type == "source":
|
||||
package["source"] = {**package.get("source", {}), **payload}
|
||||
elif record_type == "wiki_page":
|
||||
package["wiki_pages"].append(payload)
|
||||
elif record_type == "document":
|
||||
package["documents"].append(payload)
|
||||
elif record_type == "chunk":
|
||||
package["chunks"].append(payload)
|
||||
elif record_type == "vector":
|
||||
package["vectors"].append(payload)
|
||||
elif record_type == "entity":
|
||||
package["entities"].append(payload)
|
||||
elif record_type == "relation":
|
||||
package["relations"].append(payload)
|
||||
elif record_type == "graph":
|
||||
nodes = payload.get("nodes")
|
||||
edges = payload.get("edges")
|
||||
if isinstance(nodes, list):
|
||||
package["graph"]["nodes"].extend(item for item in nodes if isinstance(item, dict))
|
||||
if isinstance(edges, list):
|
||||
package["graph"]["edges"].extend(item for item in edges if isinstance(item, dict))
|
||||
for key, value in payload.items():
|
||||
if key not in {"nodes", "edges"}:
|
||||
package["graph"][key] = value
|
||||
return package
|
||||
+68
-21
@@ -11,10 +11,15 @@ blocking, making a unified file safe for both workloads. Writers
|
||||
that contend for the lock wait via the default 5-second sqlite3
|
||||
busy timeout rather than failing immediately.
|
||||
|
||||
Postgres mode: both use the same database URL but maintain independent
|
||||
connection pools with different lifecycles.
|
||||
|
||||
MySQL mode: application ORM data uses MySQL. LangGraph's checkpointer does
|
||||
Postgres mode: both use the same database URL but maintain independent
|
||||
connection pools with different lifecycles.
|
||||
|
||||
Gauss mode: application ORM data uses the official openGauss SQLAlchemy
|
||||
dialect over asyncpg. LangGraph's PostgreSQL checkpointer is intentionally not
|
||||
reused for GaussDB; configure the legacy ``checkpointer:`` section separately
|
||||
(normally sqlite).
|
||||
|
||||
MySQL mode: application ORM data uses MySQL. LangGraph's checkpointer does
|
||||
not have a built-in MySQL backend in this project, so deployments that choose
|
||||
MySQL for business tables should configure the legacy ``checkpointer:``
|
||||
section separately (usually sqlite) to preserve conversation checkpoints.
|
||||
@@ -42,24 +47,40 @@ from typing import Literal
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class DatabaseConfig(BaseModel):
|
||||
backend: Literal["memory", "sqlite", "postgres", "mysql"] = Field(
|
||||
default="memory",
|
||||
description=("Storage backend for application data. 'memory' for development (no persistence across restarts), 'sqlite' for single-node deployment, 'postgres' for production multi-node deployment, 'mysql' for MySQL-backed business tables."),
|
||||
class DatabaseConfig(BaseModel):
|
||||
backend: Literal["memory", "sqlite", "postgres", "gauss", "mysql"] = Field(
|
||||
default="memory",
|
||||
description=(
|
||||
"Storage backend for application data. 'memory' for development "
|
||||
"(no persistence across restarts), 'sqlite' for single-node "
|
||||
"deployment, 'postgres' for production multi-node deployment, "
|
||||
"'gauss' for openGauss/GaussDB business tables, and 'mysql' for "
|
||||
"MySQL-backed business tables."
|
||||
),
|
||||
)
|
||||
sqlite_dir: str = Field(
|
||||
default=".deer-flow/data",
|
||||
description=("Directory for the SQLite database file. Both checkpointer and application data share {sqlite_dir}/deerflow.db."),
|
||||
)
|
||||
postgres_url: str = Field(
|
||||
postgres_url: str = Field(
|
||||
default="",
|
||||
description=(
|
||||
"PostgreSQL connection URL, shared by checkpointer and app. "
|
||||
"Use $DATABASE_URL in config.yaml to reference .env. "
|
||||
"Example: postgresql://user:pass@host:5432/deerflow "
|
||||
"(the +asyncpg driver suffix is added automatically where needed)."
|
||||
),
|
||||
)
|
||||
),
|
||||
)
|
||||
gauss_url: str = Field(
|
||||
default="",
|
||||
description=(
|
||||
"openGauss/GaussDB connection URL for the application ORM. "
|
||||
"Use $GAUSS_DATABASE_URL in config.yaml to reference .env. "
|
||||
"Example: opengauss+asyncpg://user:pass@host:26000/deerflow. "
|
||||
"Plain opengauss:// or PostgreSQL-style URLs are normalized to the "
|
||||
"official asynchronous openGauss SQLAlchemy dialect."
|
||||
),
|
||||
)
|
||||
mysql_url: str = Field(
|
||||
default="",
|
||||
description=(
|
||||
@@ -75,26 +96,26 @@ class DatabaseConfig(BaseModel):
|
||||
)
|
||||
pool_size: int = Field(
|
||||
default=5,
|
||||
description="Connection pool size for the app ORM engine (postgres/mysql only).",
|
||||
description="Connection pool size for the app ORM engine (postgres/gauss/mysql only).",
|
||||
)
|
||||
max_overflow: int = Field(
|
||||
default=10,
|
||||
description=(
|
||||
"Extra connections allowed beyond pool_size under load (postgres/mysql only). "
|
||||
"Extra connections allowed beyond pool_size under load (postgres/gauss/mysql only). "
|
||||
"The real connection ceiling is pool_size + max_overflow. SQLAlchemy default is 10."
|
||||
),
|
||||
)
|
||||
pool_timeout: int = Field(
|
||||
default=30,
|
||||
description=(
|
||||
"Seconds a caller waits for a free pooled connection before erroring (postgres/mysql only). "
|
||||
"Seconds a caller waits for a free pooled connection before erroring (postgres/gauss/mysql only). "
|
||||
"SQLAlchemy default is 30."
|
||||
),
|
||||
)
|
||||
pool_recycle: int = Field(
|
||||
default=1800,
|
||||
description=(
|
||||
"Recycle pooled connections after this many seconds (postgres/mysql only). "
|
||||
"Recycle pooled connections after this many seconds (postgres/gauss/mysql only). "
|
||||
"Prevents stale-connection errors when a firewall, pgbouncer, or MySQL "
|
||||
"wait_timeout silently drops idle connections. pool_pre_ping catches most "
|
||||
"cases, but pool_recycle is the belt-and-suspenders safety net. "
|
||||
@@ -133,12 +154,38 @@ class DatabaseConfig(BaseModel):
|
||||
"""SQLAlchemy async URL for the application ORM engine."""
|
||||
if self.backend == "sqlite":
|
||||
return f"sqlite+aiosqlite:///{self.sqlite_path}"
|
||||
if self.backend == "postgres":
|
||||
url = self.postgres_url
|
||||
if url.startswith("postgresql://"):
|
||||
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||
return url
|
||||
if self.backend == "mysql":
|
||||
if self.backend == "postgres":
|
||||
url = self.postgres_url
|
||||
if url.startswith("postgresql://"):
|
||||
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||
return url
|
||||
if self.backend == "gauss":
|
||||
url = self.gauss_url.strip()
|
||||
if not url:
|
||||
raise ValueError("database.gauss_url is required for backend='gauss'")
|
||||
|
||||
# GaussDB speaks the PostgreSQL wire protocol, but its server
|
||||
# version string and DDL capabilities are not identical to
|
||||
# PostgreSQL. Always select the official openGauss dialect instead
|
||||
# of silently sending the URL through SQLAlchemy's generic PG
|
||||
# dialect (the latter commonly connects and then fails while
|
||||
# SQLAlchemy initializes/creates the schema).
|
||||
for prefix in (
|
||||
"opengauss://",
|
||||
"gaussdb://",
|
||||
"postgres://",
|
||||
"postgresql://",
|
||||
"postgresql+asyncpg://",
|
||||
):
|
||||
if url.startswith(prefix):
|
||||
return "opengauss+asyncpg://" + url[len(prefix) :]
|
||||
if url.startswith("opengauss+asyncpg://"):
|
||||
return url
|
||||
raise ValueError(
|
||||
"database.gauss_url must use opengauss://, "
|
||||
"opengauss+asyncpg://, or a PostgreSQL-style URL"
|
||||
)
|
||||
if self.backend == "mysql":
|
||||
url = self.mysql_url
|
||||
if not url:
|
||||
raise ValueError("database.mysql_url is required for backend='mysql'")
|
||||
|
||||
@@ -73,7 +73,7 @@ class WeKnoraDirectGraphConfig(BaseModel):
|
||||
class LocalWikiEmbeddingConfig(BaseModel):
|
||||
enabled: bool = True
|
||||
provider: str = "openai"
|
||||
model: str = "bge-m3"
|
||||
model: str = "bge-embedding-m3"
|
||||
base_url: str = ""
|
||||
api_key: str = Field(default="", repr=False)
|
||||
dimensions: int | None = Field(
|
||||
|
||||
+1
-1
@@ -494,7 +494,7 @@ class WeKnoraClient:
|
||||
await self._request(
|
||||
"GET",
|
||||
f"/api/v1/knowledgebase/{quote(knowledge_base_id, safe='')}/wiki/graph",
|
||||
params={"mode": "overview", "limit": max(1, min(limit, 500))},
|
||||
params={"mode": "overview", "limit": max(1, min(limit, 10000))},
|
||||
)
|
||||
)
|
||||
if not isinstance(value, dict):
|
||||
|
||||
+34
-4
@@ -1,7 +1,37 @@
|
||||
"""DeerFlow-local mirror and Wiki-only retrieval for WeKnora Wiki pages."""
|
||||
|
||||
from deerflow.integrations.weknora.local_index.retrieval import WikiRetrievalService, is_fully_vectorized
|
||||
from deerflow.integrations.weknora.local_index.search import WikiIndexError, WikiVectorSearchService
|
||||
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||
from __future__ import annotations
|
||||
|
||||
__all__ = ["WikiIndexError", "WikiRetrievalService", "WikiVectorSearchService", "WikiSyncService", "is_fully_vectorized"]
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from deerflow.integrations.weknora.local_index.retrieval import (
|
||||
WikiRetrievalService,
|
||||
is_fully_vectorized,
|
||||
)
|
||||
from deerflow.integrations.weknora.local_index.search import WikiIndexError, WikiVectorSearchService
|
||||
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||
|
||||
__all__ = [
|
||||
"WikiIndexError",
|
||||
"WikiRetrievalService",
|
||||
"WikiVectorSearchService",
|
||||
"WikiSyncService",
|
||||
"is_fully_vectorized",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str) -> Any:
|
||||
if name in {"WikiRetrievalService", "is_fully_vectorized"}:
|
||||
from deerflow.integrations.weknora.local_index import retrieval
|
||||
|
||||
return getattr(retrieval, name)
|
||||
if name in {"WikiIndexError", "WikiVectorSearchService"}:
|
||||
from deerflow.integrations.weknora.local_index import search
|
||||
|
||||
return getattr(search, name)
|
||||
if name == "WikiSyncService":
|
||||
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||
|
||||
return WikiSyncService
|
||||
raise AttributeError(name)
|
||||
+34
-3
@@ -1,10 +1,10 @@
|
||||
"""ORM model for the single global DeerFlow assistant knowledge base."""
|
||||
"""ORM model for DeerFlow-native assistant knowledge bases."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import Float, Integer, String, UniqueConstraint
|
||||
from sqlalchemy import Float, Index, Integer, LargeBinary, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from deerflow.persistence.base import Base
|
||||
@@ -19,7 +19,12 @@ class AssistantKnowledgeBaseRow(Base):
|
||||
__tablename__ = "assistant_knowledge_base"
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String(255), default="知识梳理")
|
||||
status: Mapped[str] = mapped_column(String(24), default="active", unique=True)
|
||||
description: Mapped[str | None] = mapped_column(String(2048), nullable=True)
|
||||
# ``global`` is the full-library aggregation base. ``custom`` bases are
|
||||
# user-created assistant knowledge bases fed from WeKnora export packages.
|
||||
# Nullable keeps startup column reconciliation safe for existing databases.
|
||||
base_kind: Mapped[str | None] = mapped_column(String(24), nullable=True, index=True)
|
||||
status: Mapped[str] = mapped_column(String(64), default="active")
|
||||
created_by: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
|
||||
updated_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now, onupdate=_now)
|
||||
@@ -91,6 +96,32 @@ class AssistantKnowledgeWikiRevisionRow(Base):
|
||||
published_at: Mapped[datetime | None] = mapped_column(BeijingDateTime(), nullable=True)
|
||||
|
||||
|
||||
class AssistantKnowledgeVectorRow(Base):
|
||||
__tablename__ = "assistant_knowledge_vectors"
|
||||
__table_args__ = (
|
||||
Index("ix_assistant_knowledge_vector_base_fingerprint", "base_id", "embedding_fingerprint"),
|
||||
Index("ix_assistant_knowledge_vector_page", "page_id"),
|
||||
Index("ix_assistant_knowledge_vector_source", "source_id"),
|
||||
)
|
||||
|
||||
id: Mapped[str] = mapped_column(String(36), primary_key=True)
|
||||
base_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||
page_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||
source_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||
import_job_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||
source_vector_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
source_chunk_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
section_index: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
heading: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
section_content: Mapped[str] = mapped_column(PortableLongText(), nullable=False, default="")
|
||||
content_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||
embedding_model: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
embedding_fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
||||
embedding_dimensions: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
vector_blob: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
|
||||
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
|
||||
|
||||
|
||||
class AssistantKnowledgeEntityRow(Base):
|
||||
__tablename__ = "assistant_knowledge_entities"
|
||||
__table_args__ = (UniqueConstraint("base_id", "entity_type", "normalized_name", name="uq_assistant_entity"),)
|
||||
|
||||
+807
-30
@@ -1,15 +1,21 @@
|
||||
"""Repository for the single global assistant knowledge base."""
|
||||
"""Repository for DeerFlow-native assistant knowledge bases."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import difflib
|
||||
import hashlib
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
import numpy as np
|
||||
from sqlalchemy import delete, func, or_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from deerflow.integrations.weknora.local_index.vector_codec import decode_vector, encode_vector, normalize_vector
|
||||
from deerflow.persistence.assistant_knowledge.model import (
|
||||
AssistantKnowledgeBaseRow,
|
||||
AssistantKnowledgeEntityContributionRow,
|
||||
@@ -18,11 +24,24 @@ from deerflow.persistence.assistant_knowledge.model import (
|
||||
AssistantKnowledgeRelationContributionRow,
|
||||
AssistantKnowledgeRelationRow,
|
||||
AssistantKnowledgeSourceRow,
|
||||
AssistantKnowledgeVectorRow,
|
||||
AssistantKnowledgeWikiPageRow,
|
||||
AssistantKnowledgeWikiRevisionRow,
|
||||
)
|
||||
from deerflow.skill_knowledge.merge import normalize_name, normalize_predicate
|
||||
|
||||
GLOBAL_BASE_NAME = "知识梳理总库"
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
VECTOR_FIELD_NAMES = (
|
||||
"embedding",
|
||||
"vector",
|
||||
"embedding_vector",
|
||||
"content_vector",
|
||||
"dense_vector",
|
||||
"dense",
|
||||
)
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
@@ -35,31 +54,246 @@ def _dict(row: Any) -> dict[str, Any]:
|
||||
data[key] = value.isoformat()
|
||||
elif key.endswith("_json"):
|
||||
data[key[:-5]] = data.pop(key) or ([] if "aliases" in key or "evidence" in key else {})
|
||||
if isinstance(row, AssistantKnowledgeBaseRow):
|
||||
raw_status = str(data.get("status") or "")
|
||||
data["status"] = "active" if raw_status.startswith("active") else raw_status
|
||||
data["base_kind"] = data.get("base_kind") or "custom"
|
||||
data["is_global"] = data["base_kind"] == "global"
|
||||
data["description"] = data.get("description") or ""
|
||||
return data
|
||||
|
||||
|
||||
def _stable_slug_part(value: Any) -> str:
|
||||
raw = str(value or "").strip()
|
||||
if not raw:
|
||||
raw = str(uuid4())
|
||||
return hashlib.sha256(raw.encode("utf-8")).hexdigest()[:24]
|
||||
|
||||
|
||||
def _content_hash(value: str) -> str:
|
||||
return hashlib.sha256(value.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _first_vector_value(raw: dict[str, Any]) -> Any:
|
||||
for key in VECTOR_FIELD_NAMES:
|
||||
value = raw.get(key)
|
||||
if value is not None:
|
||||
return value
|
||||
vectors = raw.get("vectors")
|
||||
if isinstance(vectors, dict):
|
||||
for key in VECTOR_FIELD_NAMES:
|
||||
value = vectors.get(key)
|
||||
if value is not None:
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
def _vector_blob_from_payload(raw: dict[str, Any]) -> tuple[bytes, int] | None:
|
||||
blob64 = raw.get("vector_blob_base64") or raw.get("embedding_blob_base64")
|
||||
dimensions = raw.get("embedding_dimensions") or raw.get("dimensions")
|
||||
if blob64 and dimensions:
|
||||
try:
|
||||
decoded = base64.b64decode(str(blob64), validate=True)
|
||||
vector = decode_vector(decoded, int(dimensions))
|
||||
return encode_vector(vector)
|
||||
except (ValueError, TypeError, binascii.Error):
|
||||
return None
|
||||
value = _first_vector_value(raw)
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, dict):
|
||||
value = _first_vector_value(value)
|
||||
if not isinstance(value, (list, tuple)):
|
||||
return None
|
||||
try:
|
||||
return encode_vector(value, dimensions=int(dimensions) if dimensions else None)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
class AssistantKnowledgeRepository:
|
||||
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||
self._sf = session_factory
|
||||
|
||||
async def get_base(self) -> dict[str, Any] | None:
|
||||
stmt = select(AssistantKnowledgeBaseRow).where(AssistantKnowledgeBaseRow.status == "active").limit(1)
|
||||
async with self._sf() as session:
|
||||
row = (await session.execute(stmt)).scalar_one_or_none()
|
||||
return _dict(row) if row else None
|
||||
async def _base_with_stats(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
row: AssistantKnowledgeBaseRow,
|
||||
) -> dict[str, Any]:
|
||||
data = _dict(row)
|
||||
data["source_count"] = int(
|
||||
(
|
||||
await session.execute(
|
||||
select(func.count(AssistantKnowledgeSourceRow.id)).where(
|
||||
AssistantKnowledgeSourceRow.base_id == row.id,
|
||||
AssistantKnowledgeSourceRow.status != "deleted",
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
or 0
|
||||
)
|
||||
data["page_count"] = int(
|
||||
(
|
||||
await session.execute(
|
||||
select(func.count(AssistantKnowledgeWikiPageRow.id)).where(
|
||||
AssistantKnowledgeWikiPageRow.base_id == row.id,
|
||||
AssistantKnowledgeWikiPageRow.status == "active",
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
or 0
|
||||
)
|
||||
data["entity_count"] = int(
|
||||
(
|
||||
await session.execute(
|
||||
select(func.count(AssistantKnowledgeEntityRow.id)).where(
|
||||
AssistantKnowledgeEntityRow.base_id == row.id,
|
||||
AssistantKnowledgeEntityRow.status == "active",
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
or 0
|
||||
)
|
||||
data["relation_count"] = int(
|
||||
(
|
||||
await session.execute(
|
||||
select(func.count(AssistantKnowledgeRelationRow.id)).where(
|
||||
AssistantKnowledgeRelationRow.base_id == row.id,
|
||||
AssistantKnowledgeRelationRow.status == "active",
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
or 0
|
||||
)
|
||||
return data
|
||||
|
||||
async def initialize(self, *, created_by: str | None, name: str = "知识梳理") -> dict[str, Any]:
|
||||
async def get_base(self, base_id: str | None = None) -> dict[str, Any] | None:
|
||||
async with self._sf() as session:
|
||||
if base_id:
|
||||
row = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeBaseRow).where(
|
||||
AssistantKnowledgeBaseRow.id == base_id,
|
||||
AssistantKnowledgeBaseRow.status != "deleted",
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
return await self._base_with_stats(session, row) if row else None
|
||||
|
||||
row = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeBaseRow)
|
||||
.where(
|
||||
AssistantKnowledgeBaseRow.status != "deleted",
|
||||
AssistantKnowledgeBaseRow.base_kind == "global",
|
||||
)
|
||||
.order_by(AssistantKnowledgeBaseRow.created_at.asc())
|
||||
.limit(1)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is None:
|
||||
# Backward compatibility: earlier builds had a single active row
|
||||
# and no ``base_kind``. Promote the first surviving base into the
|
||||
# global full-library aggregation base.
|
||||
row = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeBaseRow)
|
||||
.where(AssistantKnowledgeBaseRow.status != "deleted")
|
||||
.order_by(AssistantKnowledgeBaseRow.created_at.asc())
|
||||
.limit(1)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
row.base_kind = "global"
|
||||
if not row.description:
|
||||
row.description = "默认全库沉淀库,汇聚所有 WeKnora 导出的助手知识数据。"
|
||||
row.updated_at = _now()
|
||||
await session.flush()
|
||||
await session.commit()
|
||||
return await self._base_with_stats(session, row) if row else None
|
||||
|
||||
async def list_bases(self) -> list[dict[str, Any]]:
|
||||
await self.get_base()
|
||||
async with self._sf() as session:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeBaseRow)
|
||||
.where(AssistantKnowledgeBaseRow.status != "deleted")
|
||||
.order_by(
|
||||
AssistantKnowledgeBaseRow.base_kind.desc(),
|
||||
AssistantKnowledgeBaseRow.updated_at.desc(),
|
||||
)
|
||||
)
|
||||
).scalars()
|
||||
values = [await self._base_with_stats(session, row) for row in rows]
|
||||
global_values = [item for item in values if item.get("is_global")]
|
||||
custom_values = sorted(
|
||||
[item for item in values if not item.get("is_global")],
|
||||
key=lambda item: item.get("updated_at") or "",
|
||||
reverse=True,
|
||||
)
|
||||
return global_values + custom_values
|
||||
|
||||
async def initialize(self, *, created_by: str | None, name: str = GLOBAL_BASE_NAME) -> dict[str, Any]:
|
||||
existing = await self.get_base()
|
||||
if existing:
|
||||
return existing
|
||||
row = AssistantKnowledgeBaseRow(id=str(uuid4()), name=name.strip() or "知识梳理", status="active", created_by=created_by, created_at=_now(), updated_at=_now())
|
||||
row = AssistantKnowledgeBaseRow(
|
||||
id=str(uuid4()),
|
||||
name=name.strip() or GLOBAL_BASE_NAME,
|
||||
description="默认全库沉淀库,汇聚所有 WeKnora 导出的助手知识数据。",
|
||||
base_kind="global",
|
||||
status="active",
|
||||
created_by=created_by,
|
||||
created_at=_now(),
|
||||
updated_at=_now(),
|
||||
)
|
||||
async with self._sf() as session:
|
||||
session.add(row)
|
||||
await session.flush()
|
||||
data = _dict(row)
|
||||
data = await self._base_with_stats(session, row)
|
||||
await session.commit()
|
||||
return data
|
||||
|
||||
async def create_base(
|
||||
self,
|
||||
*,
|
||||
created_by: str | None,
|
||||
name: str,
|
||||
description: str = "",
|
||||
) -> dict[str, Any]:
|
||||
row_id = str(uuid4())
|
||||
row = AssistantKnowledgeBaseRow(
|
||||
id=row_id,
|
||||
name=name.strip(),
|
||||
description=description.strip(),
|
||||
base_kind="custom",
|
||||
# Some upgraded SQLite/MySQL databases may still carry the old
|
||||
# unique(status) constraint from the single-base implementation and
|
||||
# its original VARCHAR(24) length. Use a compact per-row active
|
||||
# marker internally and normalize it in API responses.
|
||||
status=f"active:{row_id[:17]}",
|
||||
created_by=created_by,
|
||||
created_at=_now(),
|
||||
updated_at=_now(),
|
||||
)
|
||||
async with self._sf() as session:
|
||||
session.add(row)
|
||||
await session.flush()
|
||||
data = await self._base_with_stats(session, row)
|
||||
await session.commit()
|
||||
return data
|
||||
|
||||
async def _ensure_base(self, base_id: str | None, *, created_by: str | None) -> dict[str, Any]:
|
||||
if base_id == "global":
|
||||
return await self.initialize(created_by=created_by)
|
||||
if base_id:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
raise ValueError("助手知识库不存在")
|
||||
return base
|
||||
return await self.initialize(created_by=created_by)
|
||||
|
||||
async def _upsert_source(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
@@ -181,8 +415,56 @@ class AssistantKnowledgeRepository:
|
||||
skill_name: str | None = None,
|
||||
mapping_id: str | None = None,
|
||||
upload_batch_id: str | None = None,
|
||||
base_id: str | None = None,
|
||||
mirror_to_global: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
base = await self.initialize(created_by=created_by)
|
||||
base = await self._ensure_base(base_id, created_by=created_by)
|
||||
result = await self._import_package_into_base(
|
||||
base=base,
|
||||
package=package,
|
||||
source_type=source_type,
|
||||
source_key=source_key,
|
||||
source_name=source_name,
|
||||
trigger=trigger,
|
||||
created_by=created_by,
|
||||
skill_name=skill_name,
|
||||
mapping_id=mapping_id,
|
||||
upload_batch_id=upload_batch_id,
|
||||
)
|
||||
if mirror_to_global and not base.get("is_global"):
|
||||
global_base = await self.initialize(created_by=created_by)
|
||||
mirror = await self._import_package_into_base(
|
||||
base=global_base,
|
||||
package=package,
|
||||
source_type=source_type,
|
||||
source_key=source_key,
|
||||
source_name=source_name,
|
||||
trigger=f"{trigger}:global_mirror"[:48],
|
||||
created_by=created_by,
|
||||
skill_name=skill_name,
|
||||
mapping_id=mapping_id,
|
||||
upload_batch_id=upload_batch_id,
|
||||
)
|
||||
result["mirrored_global_base_id"] = global_base["id"]
|
||||
result["mirrored_global_job_id"] = mirror["id"]
|
||||
return result
|
||||
|
||||
async def create_queued_import_job(
|
||||
self,
|
||||
*,
|
||||
source_type: str,
|
||||
source_key: str,
|
||||
source_name: str,
|
||||
trigger: str,
|
||||
created_by: str | None,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
skill_name: str | None = None,
|
||||
mapping_id: str | None = None,
|
||||
upload_batch_id: str | None = None,
|
||||
base_id: str | None = None,
|
||||
phase: str = "queued",
|
||||
) -> dict[str, Any]:
|
||||
base = await self._ensure_base(base_id, created_by=created_by)
|
||||
now = _now()
|
||||
async with self._sf() as session:
|
||||
source = await self._upsert_source(
|
||||
@@ -191,22 +473,18 @@ class AssistantKnowledgeRepository:
|
||||
source_type=source_type,
|
||||
source_key=source_key,
|
||||
source_name=source_name,
|
||||
metadata=package.get("source") or {},
|
||||
metadata=metadata or {},
|
||||
skill_name=skill_name,
|
||||
mapping_id=mapping_id,
|
||||
upload_batch_id=upload_batch_id,
|
||||
)
|
||||
# A re-import replaces this source's prior contributions while retaining
|
||||
# canonical entities/relations that are still supported by other sources.
|
||||
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source.id))
|
||||
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source.id))
|
||||
job = AssistantKnowledgeImportJobRow(
|
||||
id=str(uuid4()),
|
||||
base_id=base["id"],
|
||||
source_id=source.id,
|
||||
trigger=trigger,
|
||||
status="running",
|
||||
phase="aligning_entities",
|
||||
status="queued",
|
||||
phase=phase,
|
||||
counts_json={},
|
||||
created_by=created_by,
|
||||
created_at=now,
|
||||
@@ -214,6 +492,166 @@ class AssistantKnowledgeRepository:
|
||||
session.add(job)
|
||||
await session.flush()
|
||||
source.last_import_job_id = job.id
|
||||
result = _dict(job)
|
||||
await session.commit()
|
||||
return result
|
||||
|
||||
async def update_import_job(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
status: str | None = None,
|
||||
phase: str | None = None,
|
||||
counts: dict[str, Any] | None = None,
|
||||
error: str | None = None,
|
||||
completed: bool = False,
|
||||
) -> dict[str, Any] | None:
|
||||
async with self._sf() as session:
|
||||
job = await session.get(AssistantKnowledgeImportJobRow, job_id)
|
||||
if job is None:
|
||||
return None
|
||||
if status is not None:
|
||||
job.status = status
|
||||
if phase is not None:
|
||||
job.phase = phase
|
||||
if counts is not None:
|
||||
job.counts_json = counts
|
||||
if error is not None:
|
||||
job.last_error_message = error[:2048]
|
||||
elif status not in {None, "failed"}:
|
||||
job.last_error_message = None
|
||||
if completed:
|
||||
job.completed_at = _now()
|
||||
await session.flush()
|
||||
result = _dict(job)
|
||||
await session.commit()
|
||||
return result
|
||||
|
||||
async def import_package_for_job(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
package: dict[str, Any],
|
||||
created_by: str | None,
|
||||
mirror_to_global: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
async with self._sf() as session:
|
||||
job = await session.get(AssistantKnowledgeImportJobRow, job_id)
|
||||
if job is None:
|
||||
raise ValueError("导入任务不存在")
|
||||
source = await session.get(AssistantKnowledgeSourceRow, job.source_id)
|
||||
base_row = await session.get(AssistantKnowledgeBaseRow, job.base_id)
|
||||
if source is None or base_row is None:
|
||||
raise ValueError("导入任务来源不存在")
|
||||
base = _dict(base_row)
|
||||
source_type = source.source_type
|
||||
source_key = source.source_key
|
||||
source_name = source.source_name
|
||||
source_id = source.id
|
||||
trigger = job.trigger
|
||||
skill_name = source.skill_name
|
||||
mapping_id = source.knowledge_base_mapping_id
|
||||
upload_batch_id = source.upload_batch_id
|
||||
|
||||
result = await self._import_package_into_base(
|
||||
base=base,
|
||||
package=package,
|
||||
source_type=source_type,
|
||||
source_key=source_key,
|
||||
source_name=source_name,
|
||||
trigger=trigger,
|
||||
created_by=created_by,
|
||||
skill_name=skill_name,
|
||||
mapping_id=mapping_id,
|
||||
upload_batch_id=upload_batch_id,
|
||||
existing_source_id=source_id,
|
||||
existing_job_id=job_id,
|
||||
)
|
||||
if mirror_to_global and not base.get("is_global"):
|
||||
global_base = await self.initialize(created_by=created_by)
|
||||
mirror = await self._import_package_into_base(
|
||||
base=global_base,
|
||||
package=package,
|
||||
source_type=source_type,
|
||||
source_key=source_key,
|
||||
source_name=source_name,
|
||||
trigger=f"{trigger}:global_mirror"[:48],
|
||||
created_by=created_by,
|
||||
skill_name=skill_name,
|
||||
mapping_id=mapping_id,
|
||||
upload_batch_id=upload_batch_id,
|
||||
)
|
||||
result["mirrored_global_base_id"] = global_base["id"]
|
||||
result["mirrored_global_job_id"] = mirror["id"]
|
||||
return result
|
||||
|
||||
async def _import_package_into_base(
|
||||
self,
|
||||
*,
|
||||
base: dict[str, Any],
|
||||
package: dict[str, Any],
|
||||
source_type: str,
|
||||
source_key: str,
|
||||
source_name: str,
|
||||
trigger: str,
|
||||
created_by: str | None,
|
||||
skill_name: str | None = None,
|
||||
mapping_id: str | None = None,
|
||||
upload_batch_id: str | None = None,
|
||||
existing_source_id: str | None = None,
|
||||
existing_job_id: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
now = _now()
|
||||
async with self._sf() as session:
|
||||
if existing_source_id:
|
||||
source = await session.get(AssistantKnowledgeSourceRow, existing_source_id)
|
||||
if source is None or source.base_id != base["id"]:
|
||||
raise ValueError("导入任务来源不存在")
|
||||
source.source_name = source_name
|
||||
source.status = "active"
|
||||
source.metadata_json = package.get("source") or {}
|
||||
source.updated_at = now
|
||||
else:
|
||||
source = await self._upsert_source(
|
||||
session,
|
||||
base_id=base["id"],
|
||||
source_type=source_type,
|
||||
source_key=source_key,
|
||||
source_name=source_name,
|
||||
metadata=package.get("source") or {},
|
||||
skill_name=skill_name,
|
||||
mapping_id=mapping_id,
|
||||
upload_batch_id=upload_batch_id,
|
||||
)
|
||||
# A re-import replaces this source's prior contributions while retaining
|
||||
# canonical entities/relations that are still supported by other sources.
|
||||
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source.id))
|
||||
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source.id))
|
||||
await session.execute(delete(AssistantKnowledgeVectorRow).where(AssistantKnowledgeVectorRow.source_id == source.id))
|
||||
if existing_job_id:
|
||||
job = await session.get(AssistantKnowledgeImportJobRow, existing_job_id)
|
||||
if job is None or job.base_id != base["id"]:
|
||||
raise ValueError("导入任务不存在")
|
||||
job.source_id = source.id
|
||||
job.status = "running"
|
||||
job.phase = "aligning_entities"
|
||||
job.last_error_message = None
|
||||
job.completed_at = None
|
||||
else:
|
||||
job = AssistantKnowledgeImportJobRow(
|
||||
id=str(uuid4()),
|
||||
base_id=base["id"],
|
||||
source_id=source.id,
|
||||
trigger=trigger,
|
||||
status="running",
|
||||
phase="aligning_entities",
|
||||
counts_json={},
|
||||
created_by=created_by,
|
||||
created_at=now,
|
||||
)
|
||||
session.add(job)
|
||||
await session.flush()
|
||||
source.last_import_job_id = job.id
|
||||
|
||||
entity_map: dict[str, str] = {}
|
||||
entity_rows: dict[str, tuple[AssistantKnowledgeEntityRow, dict[str, Any]]] = {}
|
||||
@@ -363,6 +801,7 @@ class AssistantKnowledgeRepository:
|
||||
wiki = package["wiki"]
|
||||
page = wiki.get("skill_page") if isinstance(wiki, dict) else None
|
||||
pages = [page] if page else []
|
||||
page_ids_by_slug: dict[str, str] = {}
|
||||
for raw in pages:
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
@@ -404,16 +843,139 @@ class AssistantKnowledgeRepository:
|
||||
)
|
||||
session.add(revision)
|
||||
page.title = title
|
||||
page.page_type = str(raw.get("page_type") or "source")
|
||||
page.content_markdown = content
|
||||
page.summary = summary
|
||||
page.search_text = "\n".join([title, summary, content, " ".join(raw.get("aliases") or [])])
|
||||
page.current_revision_id = revision.id
|
||||
page.status = "active"
|
||||
page.updated_at = now
|
||||
page_ids_by_slug[slug] = page.id
|
||||
|
||||
chunks = list(package.get("chunks") or [])
|
||||
chunk_pages = 0
|
||||
imported_vectors = 0
|
||||
for index, raw in enumerate(chunks):
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
content = str(
|
||||
raw.get("content")
|
||||
or raw.get("section_content")
|
||||
or raw.get("text")
|
||||
or raw.get("chunk_content")
|
||||
or ""
|
||||
).strip()
|
||||
if not content:
|
||||
continue
|
||||
raw_chunk_id = str(raw.get("id") or raw.get("chunk_id") or raw.get("remote_id") or f"{source.id}:{index}")
|
||||
document_id = str(raw.get("source_document_id") or raw.get("knowledge_id") or raw.get("document_id") or "")
|
||||
title = str(
|
||||
raw.get("title")
|
||||
or raw.get("knowledge_title")
|
||||
or raw.get("filename")
|
||||
or raw.get("knowledge_filename")
|
||||
or f"{source_name} 片段 {index + 1}"
|
||||
)
|
||||
heading = str(raw.get("heading") or raw.get("section_title") or "")
|
||||
summary = content[:240]
|
||||
page_content = "\n".join(
|
||||
item
|
||||
for item in [
|
||||
f"# {title}",
|
||||
"",
|
||||
f"- 来源:{source_name}",
|
||||
f"- 文档ID:{document_id}" if document_id else "",
|
||||
f"- 片段ID:{raw_chunk_id}" if raw_chunk_id else "",
|
||||
f"- 标题层级:{heading}" if heading else "",
|
||||
"",
|
||||
content,
|
||||
]
|
||||
if item != ""
|
||||
)
|
||||
chunk_page = await self._publish_page(
|
||||
session,
|
||||
base_id=base["id"],
|
||||
slug=f"chunks/{source.id}/{_stable_slug_part(raw_chunk_id)}",
|
||||
title=title,
|
||||
page_type="chunk",
|
||||
content=page_content,
|
||||
summary=summary,
|
||||
search_terms=[raw_chunk_id, document_id, heading],
|
||||
source_id=source.id,
|
||||
import_job_id=job.id,
|
||||
created_by=created_by,
|
||||
change_type="chunk_import",
|
||||
)
|
||||
page_ids_by_slug[chunk_page.slug] = chunk_page.id
|
||||
vector_blob = _vector_blob_from_payload(raw)
|
||||
if vector_blob is not None:
|
||||
blob, dimensions = vector_blob
|
||||
session.add(
|
||||
AssistantKnowledgeVectorRow(
|
||||
id=str(uuid4()),
|
||||
base_id=base["id"],
|
||||
page_id=chunk_page.id,
|
||||
source_id=source.id,
|
||||
import_job_id=job.id,
|
||||
source_vector_id=str(raw.get("vector_id") or raw.get("embedding_id") or raw_chunk_id),
|
||||
source_chunk_id=raw_chunk_id,
|
||||
section_index=int(raw.get("section_index") or raw.get("chunk_index") or index),
|
||||
heading=heading or None,
|
||||
section_content=content,
|
||||
content_hash=str(raw.get("content_hash") or _content_hash(content)),
|
||||
embedding_model=str(raw.get("embedding_model") or raw.get("model") or ""),
|
||||
embedding_fingerprint=str(raw.get("embedding_fingerprint") or ""),
|
||||
embedding_dimensions=dimensions,
|
||||
vector_blob=blob,
|
||||
created_at=now,
|
||||
)
|
||||
)
|
||||
imported_vectors += 1
|
||||
chunk_pages += 1
|
||||
|
||||
for index, raw in enumerate(list(package.get("vectors") or [])):
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
slug = str(raw.get("wiki_slug") or raw.get("page_slug") or raw.get("slug") or "").strip("/")
|
||||
page_id = page_ids_by_slug.get(slug)
|
||||
if not page_id:
|
||||
continue
|
||||
content = str(raw.get("section_content") or raw.get("content") or raw.get("text") or "").strip()
|
||||
if not content:
|
||||
continue
|
||||
vector_blob = _vector_blob_from_payload(raw)
|
||||
if vector_blob is None:
|
||||
continue
|
||||
blob, dimensions = vector_blob
|
||||
raw_chunk_id = str(raw.get("source_chunk_id") or raw.get("chunk_id") or "")
|
||||
session.add(
|
||||
AssistantKnowledgeVectorRow(
|
||||
id=str(uuid4()),
|
||||
base_id=base["id"],
|
||||
page_id=page_id,
|
||||
source_id=source.id,
|
||||
import_job_id=job.id,
|
||||
source_vector_id=str(raw.get("id") or raw.get("vector_id") or f"{slug}:{index}"),
|
||||
source_chunk_id=raw_chunk_id or None,
|
||||
section_index=int(raw.get("section_index") or index),
|
||||
heading=str(raw.get("heading") or "") or None,
|
||||
section_content=content,
|
||||
content_hash=str(raw.get("content_hash") or _content_hash(content)),
|
||||
embedding_model=str(raw.get("embedding_model") or raw.get("model") or ""),
|
||||
embedding_fingerprint=str(raw.get("embedding_fingerprint") or ""),
|
||||
embedding_dimensions=dimensions,
|
||||
vector_blob=blob,
|
||||
created_at=now,
|
||||
)
|
||||
)
|
||||
imported_vectors += 1
|
||||
|
||||
counts = {
|
||||
"wiki_pages": len(pages) + len(entity_rows) + len(relation_rows),
|
||||
"documents": len(package.get("documents") or []),
|
||||
"chunks": len(package.get("chunks") or []),
|
||||
"chunks": len(chunks),
|
||||
"chunk_pages": chunk_pages,
|
||||
"vectors": imported_vectors,
|
||||
"entities": len(raw_entities),
|
||||
"relations": len(raw_relations),
|
||||
"graph_nodes": len((package.get("graph") or {}).get("nodes") or []),
|
||||
@@ -439,8 +1001,24 @@ class AssistantKnowledgeRepository:
|
||||
contribution = (await session.execute(select(AssistantKnowledgeRelationContributionRow.id).where(AssistantKnowledgeRelationContributionRow.relation_id == relation.id).limit(1))).scalar_one_or_none()
|
||||
relation.status = "active" if contribution else "deleted"
|
||||
|
||||
async def list_pages(self, *, query: str | None = None, page_type: str | None = None, limit: int = 100) -> list[dict[str, Any]]:
|
||||
stmt = select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.status == "active")
|
||||
async def list_pages(
|
||||
self,
|
||||
*,
|
||||
base_id: str | None = None,
|
||||
query: str | None = None,
|
||||
page_type: str | None = None,
|
||||
limit: int = 100,
|
||||
include_chunks: bool = False,
|
||||
) -> list[dict[str, Any]]:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
return []
|
||||
stmt = select(AssistantKnowledgeWikiPageRow).where(
|
||||
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||
AssistantKnowledgeWikiPageRow.status == "active",
|
||||
)
|
||||
if not include_chunks and not page_type:
|
||||
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
|
||||
if query:
|
||||
needle = f"%{query.strip()}%"
|
||||
stmt = stmt.where(or_(AssistantKnowledgeWikiPageRow.title.ilike(needle), AssistantKnowledgeWikiPageRow.search_text.ilike(needle)))
|
||||
@@ -450,15 +1028,178 @@ class AssistantKnowledgeRepository:
|
||||
async with self._sf() as session:
|
||||
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||
|
||||
async def get_page(self, slug: str) -> dict[str, Any] | None:
|
||||
stmt = select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/"), AssistantKnowledgeWikiPageRow.status == "active")
|
||||
async def search_pages(
|
||||
self,
|
||||
*,
|
||||
base_ids: list[str],
|
||||
query: str,
|
||||
limit: int = 8,
|
||||
include_chunks: bool = True,
|
||||
embedding_client: Any | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
resolved_base_ids: list[str] = []
|
||||
for base_id in list(dict.fromkeys(base_ids)):
|
||||
base = await self.get_base(base_id)
|
||||
if base is not None:
|
||||
resolved_base_ids.append(str(base["id"]))
|
||||
if not resolved_base_ids:
|
||||
return []
|
||||
if embedding_client is not None and query.strip():
|
||||
try:
|
||||
vector_results = await self._search_pages_by_vector(
|
||||
base_ids=resolved_base_ids,
|
||||
query=query,
|
||||
limit=limit,
|
||||
include_chunks=include_chunks,
|
||||
embedding_client=embedding_client,
|
||||
)
|
||||
if vector_results:
|
||||
return vector_results
|
||||
except Exception: # noqa: BLE001
|
||||
logger.warning("Assistant knowledge vector search failed; falling back to keyword search", exc_info=True)
|
||||
tokens = [token for token in query.strip().lower().split() if token]
|
||||
needle = f"%{query.strip()}%" if query.strip() else "%"
|
||||
stmt = select(AssistantKnowledgeWikiPageRow).where(
|
||||
AssistantKnowledgeWikiPageRow.base_id.in_(resolved_base_ids),
|
||||
AssistantKnowledgeWikiPageRow.status == "active",
|
||||
)
|
||||
if not include_chunks:
|
||||
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
|
||||
if query.strip():
|
||||
stmt = stmt.where(
|
||||
or_(
|
||||
AssistantKnowledgeWikiPageRow.title.ilike(needle),
|
||||
AssistantKnowledgeWikiPageRow.search_text.ilike(needle),
|
||||
*[
|
||||
AssistantKnowledgeWikiPageRow.search_text.ilike(f"%{token}%")
|
||||
for token in tokens[:8]
|
||||
],
|
||||
)
|
||||
)
|
||||
stmt = stmt.order_by(AssistantKnowledgeWikiPageRow.updated_at.desc()).limit(max(limit * 8, limit))
|
||||
async with self._sf() as session:
|
||||
rows = [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||
|
||||
def score(row: dict[str, Any]) -> float:
|
||||
text = f"{row.get('title') or ''}\n{row.get('summary') or ''}\n{row.get('search_text') or row.get('content_markdown') or ''}".lower()
|
||||
if not tokens:
|
||||
return 0.0
|
||||
value = 0.0
|
||||
title = str(row.get("title") or "").lower()
|
||||
for token in tokens:
|
||||
if token in title:
|
||||
value += 3.0
|
||||
value += min(text.count(token), 8) * 0.25
|
||||
if row.get("page_type") == "chunk":
|
||||
value += 0.5
|
||||
return value
|
||||
|
||||
rows.sort(key=score, reverse=True)
|
||||
results = rows[:limit]
|
||||
for row in results:
|
||||
row["score"] = score(row)
|
||||
return results
|
||||
|
||||
async def _search_pages_by_vector(
|
||||
self,
|
||||
*,
|
||||
base_ids: list[str],
|
||||
query: str,
|
||||
limit: int,
|
||||
include_chunks: bool,
|
||||
embedding_client: Any,
|
||||
) -> list[dict[str, Any]]:
|
||||
raw_query_vector = await embedding_client.embed_query(query)
|
||||
configured_dimensions = getattr(embedding_client, "dimensions", None)
|
||||
query_vector = normalize_vector(raw_query_vector, dimensions=configured_dimensions)
|
||||
dimensions = int(query_vector.size)
|
||||
stmt = (
|
||||
select(AssistantKnowledgeVectorRow, AssistantKnowledgeWikiPageRow)
|
||||
.join(
|
||||
AssistantKnowledgeWikiPageRow,
|
||||
AssistantKnowledgeWikiPageRow.id == AssistantKnowledgeVectorRow.page_id,
|
||||
)
|
||||
.where(
|
||||
AssistantKnowledgeVectorRow.base_id.in_(base_ids),
|
||||
AssistantKnowledgeVectorRow.embedding_dimensions == dimensions,
|
||||
AssistantKnowledgeWikiPageRow.status == "active",
|
||||
)
|
||||
.order_by(AssistantKnowledgeVectorRow.created_at.desc())
|
||||
)
|
||||
if not include_chunks:
|
||||
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
|
||||
async with self._sf() as session:
|
||||
rows = list((await session.execute(stmt)).all())
|
||||
if not rows:
|
||||
return []
|
||||
tokens = [token for token in query.strip().lower().split() if token]
|
||||
ranked: list[tuple[float, AssistantKnowledgeVectorRow, AssistantKnowledgeWikiPageRow]] = []
|
||||
for vector, page in rows:
|
||||
try:
|
||||
stored = decode_vector(vector.vector_blob, int(vector.embedding_dimensions))
|
||||
except ValueError:
|
||||
continue
|
||||
score = float(np.dot(query_vector, stored))
|
||||
if tokens:
|
||||
haystack = f"{page.title}\n{page.summary}\n{vector.heading or ''}\n{vector.section_content}".lower()
|
||||
score += min(sum(1 for token in tokens if token in haystack), 4) * 0.02
|
||||
ranked.append((score, vector, page))
|
||||
ranked.sort(key=lambda item: item[0], reverse=True)
|
||||
|
||||
results: list[dict[str, Any]] = []
|
||||
seen_pages: set[str] = set()
|
||||
for score, vector, page in ranked:
|
||||
if len(results) >= limit:
|
||||
break
|
||||
page_data = _dict(page)
|
||||
page_data["score"] = score
|
||||
page_data["content_markdown"] = vector.section_content or page_data.get("content_markdown") or ""
|
||||
page_data["summary"] = page_data.get("summary") or (vector.section_content[:240] if vector.section_content else "")
|
||||
page_data["retrieval_mode"] = "vector"
|
||||
page_data["matched_section"] = {
|
||||
"id": vector.id,
|
||||
"section_index": int(vector.section_index),
|
||||
"heading": vector.heading,
|
||||
"content": vector.section_content,
|
||||
"score": score,
|
||||
"source_chunk_id": vector.source_chunk_id,
|
||||
"embedding_model": vector.embedding_model,
|
||||
"embedding_fingerprint": vector.embedding_fingerprint,
|
||||
}
|
||||
dedupe_key = f"{page.id}:{vector.section_index}:{vector.content_hash}"
|
||||
if page.id in seen_pages and page.page_type != "chunk":
|
||||
continue
|
||||
seen_pages.add(page.id)
|
||||
seen_pages.add(dedupe_key)
|
||||
results.append(page_data)
|
||||
return results
|
||||
|
||||
async def get_page(self, slug: str, *, base_id: str | None = None) -> dict[str, Any] | None:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
return None
|
||||
stmt = select(AssistantKnowledgeWikiPageRow).where(
|
||||
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
|
||||
AssistantKnowledgeWikiPageRow.status == "active",
|
||||
)
|
||||
async with self._sf() as session:
|
||||
row = (await session.execute(stmt)).scalar_one_or_none()
|
||||
return _dict(row) if row else None
|
||||
|
||||
async def list_revisions(self, slug: str) -> list[dict[str, Any]]:
|
||||
async def list_revisions(self, slug: str, *, base_id: str | None = None) -> list[dict[str, Any]]:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
return []
|
||||
async with self._sf() as session:
|
||||
page = (await session.execute(select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/")))).scalar_one_or_none()
|
||||
page = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeWikiPageRow).where(
|
||||
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if page is None:
|
||||
return []
|
||||
stmt = select(AssistantKnowledgeWikiRevisionRow).where(AssistantKnowledgeWikiRevisionRow.page_id == page.id).order_by(AssistantKnowledgeWikiRevisionRow.revision_no.desc())
|
||||
@@ -484,10 +1225,27 @@ class AssistantKnowledgeRepository:
|
||||
)
|
||||
return {"left": left, "right": right, "diff": diff}
|
||||
|
||||
async def rollback(self, *, slug: str, revision_id: str, actor: str | None) -> dict[str, Any] | None:
|
||||
async def rollback(
|
||||
self,
|
||||
*,
|
||||
slug: str,
|
||||
revision_id: str,
|
||||
actor: str | None,
|
||||
base_id: str | None = None,
|
||||
) -> dict[str, Any] | None:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
return None
|
||||
now = _now()
|
||||
async with self._sf() as session:
|
||||
page = (await session.execute(select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/")))).scalar_one_or_none()
|
||||
page = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeWikiPageRow).where(
|
||||
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
source = await session.get(AssistantKnowledgeWikiRevisionRow, revision_id)
|
||||
if page is None or source is None or source.page_id != page.id:
|
||||
return None
|
||||
@@ -517,9 +1275,19 @@ class AssistantKnowledgeRepository:
|
||||
await session.commit()
|
||||
return data
|
||||
|
||||
async def list_sources(self) -> list[dict[str, Any]]:
|
||||
async def list_sources(self, *, base_id: str | None = None) -> list[dict[str, Any]]:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
return []
|
||||
async with self._sf() as session:
|
||||
stmt = select(AssistantKnowledgeSourceRow).where(AssistantKnowledgeSourceRow.status != "deleted").order_by(AssistantKnowledgeSourceRow.updated_at.desc())
|
||||
stmt = (
|
||||
select(AssistantKnowledgeSourceRow)
|
||||
.where(
|
||||
AssistantKnowledgeSourceRow.base_id == base["id"],
|
||||
AssistantKnowledgeSourceRow.status != "deleted",
|
||||
)
|
||||
.order_by(AssistantKnowledgeSourceRow.updated_at.desc())
|
||||
)
|
||||
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||
|
||||
async def get_source(self, source_id: str) -> dict[str, Any] | None:
|
||||
@@ -547,9 +1315,17 @@ class AssistantKnowledgeRepository:
|
||||
result["contributions"] = [_dict(value) for value in contributions]
|
||||
return result
|
||||
|
||||
async def list_jobs(self, limit: int = 100) -> list[dict[str, Any]]:
|
||||
async def list_jobs(self, limit: int = 100, *, base_id: str | None = None) -> list[dict[str, Any]]:
|
||||
base = await self.get_base(base_id)
|
||||
if base is None:
|
||||
return []
|
||||
async with self._sf() as session:
|
||||
stmt = select(AssistantKnowledgeImportJobRow).order_by(AssistantKnowledgeImportJobRow.created_at.desc()).limit(limit)
|
||||
stmt = (
|
||||
select(AssistantKnowledgeImportJobRow)
|
||||
.where(AssistantKnowledgeImportJobRow.base_id == base["id"])
|
||||
.order_by(AssistantKnowledgeImportJobRow.created_at.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||
|
||||
async def get_job(self, job_id: str) -> dict[str, Any] | None:
|
||||
@@ -565,6 +1341,7 @@ class AssistantKnowledgeRepository:
|
||||
source.status = "deleted"
|
||||
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source_id))
|
||||
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source_id))
|
||||
await session.execute(delete(AssistantKnowledgeVectorRow).where(AssistantKnowledgeVectorRow.source_id == source_id))
|
||||
pages = (
|
||||
await session.execute(
|
||||
select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.current_revision_id.in_(select(AssistantKnowledgeWikiRevisionRow.id).where(AssistantKnowledgeWikiRevisionRow.source_id == source_id)))
|
||||
|
||||
@@ -162,7 +162,7 @@ async def _auto_create_postgres_db(url: str) -> None:
|
||||
try:
|
||||
async with maint_engine.connect() as conn:
|
||||
await conn.execute(text(f'CREATE DATABASE "{db_name}"'))
|
||||
logger.info("Auto-created PostgreSQL database: %s", db_name)
|
||||
logger.info("Auto-created PostgreSQL-compatible database: %s", db_name)
|
||||
finally:
|
||||
await maint_engine.dispose()
|
||||
|
||||
@@ -216,10 +216,10 @@ def _ensure_orm_columns_sync(sync_conn) -> None:
|
||||
added column are created as well. Table-level ``UniqueConstraint`` objects
|
||||
that are declared on the model but missing from the table are also created
|
||||
(as a ``CREATE UNIQUE INDEX`` of the same name, which is what the migrations
|
||||
emit too, and works on SQLite/MySQL/PostgreSQL alike) — without this a newly
|
||||
emit too, and works on SQLite/MySQL/PostgreSQL/openGauss alike) — without this a newly
|
||||
declared uniqueness rule (e.g. ``active_dedupe_key``) would silently be absent
|
||||
on an existing database, weakening dedupe.
|
||||
Works on sqlite/mysql/postgres — the column DDL is compiled through the
|
||||
Works on sqlite/mysql/postgres/gauss — the column DDL is compiled through the
|
||||
active dialect, so portable types (``PortableJSON``/``PortableLongText``/
|
||||
``BeijingDateTime``) resolve to the right backend type. Everything is
|
||||
per-column best-effort: a failure is logged and startup continues.
|
||||
@@ -341,13 +341,13 @@ async def init_engine(
|
||||
"""Create the async engine and session factory, then auto-create tables.
|
||||
|
||||
Args:
|
||||
backend: "memory", "sqlite", "postgres", or "mysql".
|
||||
url: SQLAlchemy async URL (for sqlite/postgres/mysql).
|
||||
backend: "memory", "sqlite", "postgres", "gauss", or "mysql".
|
||||
url: SQLAlchemy async URL (for sqlite/postgres/gauss/mysql).
|
||||
echo: Echo SQL to log.
|
||||
pool_size: Base connection pool size (postgres/mysql).
|
||||
max_overflow: Extra connections allowed beyond pool_size under load (postgres/mysql).
|
||||
pool_timeout: Seconds to wait for a free pooled connection before erroring (postgres/mysql).
|
||||
pool_recycle: Recycle connections after N seconds (postgres/mysql). Prevents stale
|
||||
pool_size: Base connection pool size (postgres/gauss/mysql).
|
||||
max_overflow: Extra connections allowed beyond pool_size under load (postgres/gauss/mysql).
|
||||
pool_timeout: Seconds to wait for a free pooled connection before erroring (postgres/gauss/mysql).
|
||||
pool_recycle: Recycle connections after N seconds (postgres/gauss/mysql). Prevents stale
|
||||
connections dropped by firewalls / MySQL wait_timeout / pgbouncer.
|
||||
sqlite_dir: Directory to create for SQLite (ensured to exist).
|
||||
"""
|
||||
@@ -365,11 +365,21 @@ async def init_engine(
|
||||
except RuntimeError:
|
||||
_engine_loop = None
|
||||
|
||||
if backend == "postgres":
|
||||
try:
|
||||
import asyncpg # noqa: F401
|
||||
except ImportError:
|
||||
raise ImportError("database.backend is set to 'postgres' but asyncpg is not installed.\nInstall it with:\n uv sync --extra postgres\nOr switch to backend: sqlite in config.yaml for single-node deployment.") from None
|
||||
if backend in {"postgres", "gauss"}:
|
||||
try:
|
||||
import asyncpg # noqa: F401
|
||||
except ImportError:
|
||||
extra = "gauss" if backend == "gauss" else "postgres"
|
||||
raise ImportError(
|
||||
f"database.backend is set to {backend!r} but asyncpg is not installed.\n"
|
||||
f"Install it with:\n uv sync --extra {extra}\n"
|
||||
"Or switch to backend: sqlite in config.yaml for single-node deployment."
|
||||
) from None
|
||||
if backend == "gauss" and find_spec("opengauss_sqlalchemy") is None:
|
||||
raise ImportError(
|
||||
"database.backend is set to 'gauss' but the openGauss SQLAlchemy dialect "
|
||||
"is not installed.\nInstall it with:\n uv sync --extra gauss"
|
||||
)
|
||||
if backend == "mysql":
|
||||
_add_runtime_vendor_path()
|
||||
driver = _mysql_driver_name(url)
|
||||
@@ -409,7 +419,7 @@ async def init_engine(
|
||||
cursor.execute("PRAGMA foreign_keys=ON;")
|
||||
finally:
|
||||
cursor.close()
|
||||
elif backend == "postgres":
|
||||
elif backend in {"postgres", "gauss"}:
|
||||
_engine = create_async_engine(
|
||||
url,
|
||||
echo=echo,
|
||||
@@ -454,7 +464,7 @@ async def init_engine(
|
||||
async with _engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
except Exception as exc:
|
||||
if backend == "postgres" and "does not exist" in str(exc):
|
||||
if backend in {"postgres", "gauss"} and "does not exist" in str(exc).lower():
|
||||
# Database not yet created — attempt to auto-create it, then retry.
|
||||
await _auto_create_postgres_db(url)
|
||||
# Rebuild engine against the now-existing database
|
||||
@@ -497,7 +507,7 @@ async def init_engine(
|
||||
# cannot do on its own. Failures don't abort startup — see the helper.
|
||||
# await _run_alembic_upgrade(url)
|
||||
|
||||
logger.info("Persistence engine initialized: backend=%s", backend)
|
||||
logger.info("Persistence engine initialized: backend=%s dialect=%s", backend, _engine.dialect.name)
|
||||
|
||||
|
||||
async def init_engine_from_config(config) -> None:
|
||||
|
||||
+18
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import Any
|
||||
|
||||
|
||||
@@ -35,6 +36,23 @@ class LlmWikiIndexStore(ABC):
|
||||
@abstractmethod
|
||||
async def load_vector_snapshot(self, mapping_id: str, fingerprint: str) -> dict[str, Any]: ...
|
||||
|
||||
async def iter_vector_snapshot_rows(
|
||||
self,
|
||||
mapping_id: str,
|
||||
fingerprint: str,
|
||||
*,
|
||||
batch_size: int = 1000,
|
||||
) -> AsyncIterator[dict[str, Any]]:
|
||||
"""Iterate vector rows for streaming export.
|
||||
|
||||
Implementations with a database backend should override this method so
|
||||
very large vector packages do not need to be materialized in memory.
|
||||
"""
|
||||
snapshot = await self.load_vector_snapshot(mapping_id, fingerprint)
|
||||
for row in snapshot.get("rows") or []:
|
||||
if isinstance(row, dict):
|
||||
yield row
|
||||
|
||||
@abstractmethod
|
||||
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]: ...
|
||||
|
||||
|
||||
+36
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
@@ -181,6 +182,41 @@ class SqlLlmWikiIndexStore(LlmWikiIndexStore):
|
||||
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
|
||||
return {"revision": int(state.index_revision if state else 0), "fingerprint": fingerprint, "rows": rows}
|
||||
|
||||
async def iter_vector_snapshot_rows(
|
||||
self,
|
||||
mapping_id: str,
|
||||
fingerprint: str,
|
||||
*,
|
||||
batch_size: int = 1000,
|
||||
) -> AsyncIterator[dict[str, Any]]:
|
||||
batch = max(1, min(int(batch_size), 5000))
|
||||
offset = 0
|
||||
while True:
|
||||
stmt = (
|
||||
select(LlmWikiWikiVectorRow, LlmWikiWikiPageRow)
|
||||
.join(LlmWikiWikiPageRow, LlmWikiWikiPageRow.id == LlmWikiWikiVectorRow.page_id)
|
||||
.where(
|
||||
LlmWikiWikiVectorRow.knowledge_base_mapping_id == mapping_id,
|
||||
LlmWikiWikiVectorRow.embedding_fingerprint == fingerprint,
|
||||
LlmWikiWikiPageRow.index_status == "ready",
|
||||
LlmWikiWikiPageRow.is_remote_deleted.is_(False),
|
||||
LlmWikiWikiPageRow.status.in_(["published", "draft"]),
|
||||
)
|
||||
.order_by(LlmWikiWikiVectorRow.page_id, LlmWikiWikiVectorRow.section_index)
|
||||
.limit(batch)
|
||||
.offset(offset)
|
||||
)
|
||||
async with self._sf() as session:
|
||||
pairs = list((await session.execute(stmt)).all())
|
||||
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
|
||||
if not rows:
|
||||
break
|
||||
for row in rows:
|
||||
yield row
|
||||
if len(rows) < batch:
|
||||
break
|
||||
offset += len(rows)
|
||||
|
||||
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]:
|
||||
async with self._sf() as session:
|
||||
state = await session.get(LlmWikiWikiSyncStateRow, mapping_id)
|
||||
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
"""Add assistant knowledge vector table.
|
||||
|
||||
Revision ID: 20260902_01
|
||||
Revises: 20260901_01
|
||||
Create Date: 2026-09-02
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from alembic import op
|
||||
|
||||
from deerflow.persistence.assistant_knowledge.model import AssistantKnowledgeVectorRow
|
||||
|
||||
revision = "20260902_01"
|
||||
down_revision = "20260901_01"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
AssistantKnowledgeVectorRow.__table__.create(op.get_bind(), checkfirst=True)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
AssistantKnowledgeVectorRow.__table__.drop(op.get_bind(), checkfirst=True)
|
||||
+1
@@ -43,6 +43,7 @@ from deerflow.persistence.assistant_knowledge.model import (
|
||||
AssistantKnowledgeRelationContributionRow,
|
||||
AssistantKnowledgeRelationRow,
|
||||
AssistantKnowledgeSourceRow,
|
||||
AssistantKnowledgeVectorRow,
|
||||
AssistantKnowledgeWikiPageRow,
|
||||
AssistantKnowledgeWikiRevisionRow,
|
||||
)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from datetime import UTC, datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import JSON, DateTime, Text
|
||||
@@ -14,6 +14,12 @@ from sqlalchemy.types import TypeDecorator
|
||||
# fixed offset is exact and avoids depending on the `tzdata` package (which the
|
||||
# stdlib `zoneinfo` needs on Windows / minimal containers).
|
||||
BEIJING_TZ = timezone(timedelta(hours=8))
|
||||
_POSTGRES_FAMILY_DIALECTS = frozenset({"postgresql", "opengauss"})
|
||||
|
||||
|
||||
def _is_postgres_family(dialect) -> bool:
|
||||
"""Return whether *dialect* preserves timezone-aware timestamps natively."""
|
||||
return dialect.name in _POSTGRES_FAMILY_DIALECTS
|
||||
|
||||
|
||||
class PortableJSON(TypeDecorator):
|
||||
@@ -87,17 +93,17 @@ class BeijingDateTime(TypeDecorator):
|
||||
def process_bind_param(self, value: Any, dialect) -> Any:
|
||||
if not isinstance(value, datetime):
|
||||
return value
|
||||
if dialect.name == "postgresql":
|
||||
if _is_postgres_family(dialect):
|
||||
return value
|
||||
if value.tzinfo is None:
|
||||
# Naive values follow the codebase convention of being UTC.
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
value = value.replace(tzinfo=UTC)
|
||||
return value.astimezone(BEIJING_TZ).replace(tzinfo=None)
|
||||
|
||||
def process_result_value(self, value: Any, dialect) -> Any:
|
||||
if not isinstance(value, datetime):
|
||||
return value
|
||||
if dialect.name == "postgresql":
|
||||
if _is_postgres_family(dialect):
|
||||
return value
|
||||
if value.tzinfo is None:
|
||||
# Stored as naive Beijing wall-clock; re-attach the zone.
|
||||
|
||||
@@ -52,8 +52,12 @@ postgres = [
|
||||
"psycopg[binary]>=3.3.3",
|
||||
"psycopg-pool>=3.3.0",
|
||||
]
|
||||
mysql = ["asyncmy>=0.2.10"]
|
||||
pymupdf = ["pymupdf4llm>=0.0.17"]
|
||||
mysql = ["asyncmy>=0.2.10"]
|
||||
gauss = [
|
||||
"asyncpg>=0.29",
|
||||
"opengauss-sqlalchemy>=2.4,<3",
|
||||
]
|
||||
pymupdf = ["pymupdf4llm>=0.0.17"]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
|
||||
@@ -30,6 +30,7 @@ dependencies = [
|
||||
[project.optional-dependencies]
|
||||
postgres = ["deerflow-harness[postgres]"]
|
||||
mysql = ["deerflow-harness[mysql]"]
|
||||
gauss = ["deerflow-harness[gauss]"]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
|
||||
@@ -22,6 +22,18 @@ class _FakeThreadStore:
|
||||
return self.rows[offset : offset + limit]
|
||||
|
||||
|
||||
class _FakeAgentStore:
|
||||
async def list_by_ids(self, agent_ids: list[str], user_id: str) -> list[dict]:
|
||||
assert user_id == "owner"
|
||||
agents = {
|
||||
"analysis-agent": {
|
||||
"id": "analysis-agent",
|
||||
"name": "舆情分析智能体",
|
||||
}
|
||||
}
|
||||
return [agents[agent_id] for agent_id in agent_ids if agent_id in agents]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: Path, monkeypatch):
|
||||
thread_one = tmp_path / "thread-one" / "outputs"
|
||||
@@ -79,6 +91,7 @@ async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: P
|
||||
|
||||
assert [item["name"] for item in items] == ["证据.zip", "结论.md", "数据汇总.xlsx", "研判报告.docx"]
|
||||
assert [item["file_kind"] for item in items] == ["archive", "text_data", "spreadsheet", "document"]
|
||||
assert [item["source_type"] for item in items] == ["general_qa", "general_qa", "agent", "agent"]
|
||||
assert items[-1]["thread_title"] == "战场环境综合研判"
|
||||
assert items[-1]["agent_id"] == "analysis-agent"
|
||||
assert all(item["name"] not in {"配图.png", "录音.mp3"} for item in items)
|
||||
@@ -88,9 +101,12 @@ async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: P
|
||||
@pytest.mark.asyncio
|
||||
async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Path, monkeypatch):
|
||||
output_dir = tmp_path / "thread-one" / "outputs"
|
||||
agent_output_dir = tmp_path / "thread-agent" / "outputs"
|
||||
output_dir.mkdir(parents=True)
|
||||
agent_output_dir.mkdir(parents=True)
|
||||
(output_dir / "任务报告.md").write_text("report", encoding="utf-8")
|
||||
(output_dir / "任务数据.csv").write_text("a,b", encoding="utf-8")
|
||||
(agent_output_dir / "智能体结论.md").write_text("agent report", encoding="utf-8")
|
||||
|
||||
store = _FakeThreadStore(
|
||||
[
|
||||
@@ -98,7 +114,12 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
||||
"thread_id": "thread-one",
|
||||
"display_name": "任务研判对话",
|
||||
"metadata": {},
|
||||
}
|
||||
},
|
||||
{
|
||||
"thread_id": "thread-agent",
|
||||
"display_name": "智能体研判对话",
|
||||
"metadata": {"agent_id": "analysis-agent"},
|
||||
},
|
||||
]
|
||||
)
|
||||
|
||||
@@ -113,6 +134,7 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
||||
monkeypatch.setattr(artifacts, "aresolve_path_user_id", _resolve_owner)
|
||||
monkeypatch.setattr(artifacts, "get_effective_user_id", lambda: "owner")
|
||||
monkeypatch.setattr(deps, "get_thread_store", lambda _request: store)
|
||||
monkeypatch.setattr(deps, "get_agent_store", lambda _request: _FakeAgentStore())
|
||||
artifacts._artifact_library_cache.clear()
|
||||
|
||||
raw_endpoint = artifacts.list_artifact_library.__wrapped__
|
||||
@@ -120,6 +142,7 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
||||
request=SimpleNamespace(),
|
||||
query="研判对话",
|
||||
file_kind="text_data",
|
||||
source_type="general_qa",
|
||||
offset=0,
|
||||
limit=1,
|
||||
refresh=True,
|
||||
@@ -134,9 +157,24 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
||||
request=SimpleNamespace(),
|
||||
query="任务数据",
|
||||
file_kind="spreadsheet",
|
||||
source_type="all",
|
||||
offset=0,
|
||||
limit=50,
|
||||
refresh=False,
|
||||
)
|
||||
assert spreadsheet_response.total == 1
|
||||
assert spreadsheet_response.items[0].name == "任务数据.csv"
|
||||
|
||||
agent_response = await raw_endpoint(
|
||||
request=SimpleNamespace(),
|
||||
query=None,
|
||||
file_kind="all",
|
||||
source_type="agent",
|
||||
offset=0,
|
||||
limit=50,
|
||||
refresh=False,
|
||||
)
|
||||
assert agent_response.total == 1
|
||||
assert agent_response.items[0].source_type == "agent"
|
||||
assert agent_response.items[0].agent_name == "舆情分析智能体"
|
||||
assert agent_response.items[0].name == "智能体结论.md"
|
||||
@@ -11,6 +11,13 @@ from deerflow.persistence.assistant_knowledge.model import (
|
||||
from deerflow.persistence.base import Base
|
||||
|
||||
|
||||
class FakeEmbeddingClient:
|
||||
dimensions = 3
|
||||
|
||||
async def embed_query(self, _query: str) -> list[float]:
|
||||
return [1.0, 0.0, 0.0]
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def repository():
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||
@@ -94,3 +101,93 @@ async def test_delete_source_keeps_entity_until_last_contribution_is_removed(
|
||||
async with repository._sf() as session:
|
||||
entity = (await session.execute(select(AssistantKnowledgeEntityRow))).scalar_one()
|
||||
assert entity.status == "deleted"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_custom_base_import_is_also_mirrored_to_global_base(
|
||||
repository: AssistantKnowledgeRepository,
|
||||
) -> None:
|
||||
global_base = await repository.initialize(created_by="admin")
|
||||
custom_base = await repository.create_base(
|
||||
created_by="admin",
|
||||
name="专题助手库",
|
||||
description="只承接某个专题的 WeKnora 导出数据",
|
||||
)
|
||||
package = {
|
||||
"source": {"type": "weknora"},
|
||||
"wiki_pages": [{"slug": "wiki/topic", "title": "专题 Wiki", "content": "王五负责统筹。"}],
|
||||
"entities": [{"local_id": "e1", "type": "person", "name": "王五"}],
|
||||
"relations": [],
|
||||
}
|
||||
|
||||
job = await repository.import_package(
|
||||
package=package,
|
||||
source_type="weknora",
|
||||
source_key="kb-topic",
|
||||
source_name="专题普通库",
|
||||
trigger="manual_from_weknora",
|
||||
created_by="admin",
|
||||
mapping_id="kb-topic",
|
||||
base_id=custom_base["id"],
|
||||
)
|
||||
|
||||
assert job["base_id"] == custom_base["id"]
|
||||
assert job["mirrored_global_base_id"] == global_base["id"]
|
||||
assert len(await repository.list_pages(base_id=custom_base["id"], query="王五")) == 2
|
||||
assert len(await repository.list_pages(base_id=global_base["id"], query="王五")) == 2
|
||||
bases = await repository.list_bases()
|
||||
assert [base["id"] for base in bases[:2]] == [global_base["id"], custom_base["id"]]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_imported_vectors_are_reused_for_assistant_search(
|
||||
repository: AssistantKnowledgeRepository,
|
||||
) -> None:
|
||||
base = await repository.initialize(created_by="admin")
|
||||
job = await repository.import_package(
|
||||
package={
|
||||
"source": {"type": "weknora"},
|
||||
"wiki_pages": [{"slug": "wiki/vector", "title": "向量页", "content": "页面正文"}],
|
||||
"vectors": [
|
||||
{
|
||||
"wiki_slug": "wiki/vector",
|
||||
"section_index": 0,
|
||||
"section_content": "张三负责向量化检索。",
|
||||
"embedding_model": "bge-embedding-m3",
|
||||
"embedding_dimensions": 3,
|
||||
"vector": [1.0, 0.0, 0.0],
|
||||
},
|
||||
{
|
||||
"wiki_slug": "wiki/vector",
|
||||
"section_index": 1,
|
||||
"section_content": "无关片段。",
|
||||
"embedding_model": "bge-embedding-m3",
|
||||
"embedding_dimensions": 3,
|
||||
"vector": [0.0, 1.0, 0.0],
|
||||
},
|
||||
],
|
||||
},
|
||||
source_type="weknora",
|
||||
source_key="kb-vector",
|
||||
source_name="向量普通库",
|
||||
trigger="manual_package_upload",
|
||||
created_by="admin",
|
||||
)
|
||||
|
||||
assert job["counts"]["vectors"] == 2
|
||||
results = await repository.search_pages(
|
||||
base_ids=[base["id"]],
|
||||
query="这个查询不靠关键词",
|
||||
embedding_client=FakeEmbeddingClient(),
|
||||
)
|
||||
assert results[0]["retrieval_mode"] == "vector"
|
||||
assert results[0]["matched_section"]["embedding_model"] == "bge-embedding-m3"
|
||||
assert results[0]["content_markdown"] == "张三负责向量化检索。"
|
||||
|
||||
sources = await repository.list_sources()
|
||||
assert await repository.delete_source(sources[0]["id"])
|
||||
assert await repository.search_pages(
|
||||
base_ids=[base["id"]],
|
||||
query="这个查询不靠关键词",
|
||||
embedding_client=FakeEmbeddingClient(),
|
||||
) == []
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Regression coverage for the openGauss/GaussDB persistence backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy.engine import make_url
|
||||
from sqlalchemy.schema import CreateIndex, CreateTable
|
||||
|
||||
from deerflow.config.database_config import DatabaseConfig
|
||||
from deerflow.persistence.types import BeijingDateTime
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("configured", "expected"),
|
||||
[
|
||||
(
|
||||
"opengauss://user:pass@127.0.0.1:26000/deerflow",
|
||||
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
|
||||
),
|
||||
(
|
||||
"postgresql://user:pass@127.0.0.1:26000/deerflow?ssl=require",
|
||||
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow?ssl=require",
|
||||
),
|
||||
(
|
||||
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
|
||||
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_gauss_urls_always_select_the_opengauss_async_dialect(
|
||||
configured: str,
|
||||
expected: str,
|
||||
) -> None:
|
||||
config = DatabaseConfig(backend="gauss", gauss_url=configured)
|
||||
assert config.app_sqlalchemy_url == expected
|
||||
|
||||
|
||||
def test_gauss_backend_requires_a_supported_nonempty_url() -> None:
|
||||
with pytest.raises(ValueError, match="gauss_url is required"):
|
||||
_ = DatabaseConfig(backend="gauss").app_sqlalchemy_url
|
||||
|
||||
with pytest.raises(ValueError, match="must use opengauss"):
|
||||
_ = DatabaseConfig(
|
||||
backend="gauss",
|
||||
gauss_url="mysql://user:pass@127.0.0.1/deerflow",
|
||||
).app_sqlalchemy_url
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
DatabaseConfig(backend="unknown")
|
||||
|
||||
|
||||
def test_gauss_timestamps_keep_timezone_information() -> None:
|
||||
dialect = type("GaussDialect", (), {"name": "opengauss"})()
|
||||
value = datetime(2026, 9, 2, 8, 30, tzinfo=UTC)
|
||||
column_type = BeijingDateTime()
|
||||
|
||||
assert column_type.process_bind_param(value, dialect) is value
|
||||
assert column_type.process_result_value(value, dialect) is value
|
||||
|
||||
|
||||
def test_official_async_dialect_compiles_the_complete_orm_schema() -> None:
|
||||
"""Every registered table/index must compile before startup touches GaussDB."""
|
||||
pytest.importorskip("opengauss_sqlalchemy")
|
||||
|
||||
import deerflow.persistence.models # noqa: F401 -- register ORM metadata
|
||||
from deerflow.persistence.base import Base
|
||||
|
||||
dialect_cls = make_url("opengauss+asyncpg://user:pass@localhost/db").get_dialect(
|
||||
_is_async=True
|
||||
)
|
||||
dialect = dialect_cls()
|
||||
|
||||
assert dialect.__class__.__module__.startswith("opengauss_sqlalchemy")
|
||||
assert Base.metadata.sorted_tables
|
||||
for table in Base.metadata.sorted_tables:
|
||||
assert str(CreateTable(table).compile(dialect=dialect)).startswith("\nCREATE TABLE")
|
||||
for index in table.indexes:
|
||||
assert str(CreateIndex(index).compile(dialect=dialect)).startswith("CREATE")
|
||||
@@ -66,6 +66,9 @@ async def test_sql_store_atomically_replaces_vectors_and_marks_missing_deleted(t
|
||||
assert store.write_batch_size == 1
|
||||
assert len(snapshot["rows"]) == 3
|
||||
assert decode_vector(snapshot["rows"][0]["vector_blob"], 2).tolist() == pytest.approx([0.6, 0.8])
|
||||
streamed = [row async for row in store.iter_vector_snapshot_rows("mapping-1", "fp", batch_size=2)]
|
||||
assert [row["section_index"] for row in streamed] == [0, 1, 2]
|
||||
assert streamed[0]["page"]["slug"] == "people/alice"
|
||||
page_statuses = await store.list_page_index_status("mapping-1")
|
||||
assert len(page_statuses) == 1
|
||||
assert page_statuses[0] == {
|
||||
|
||||
Reference in new issue
Block a user