feat: improve assistant knowledge import and database compatibility
This commit is contained in:
1 parent
fbd8dec48f
commit
6b724bd00d
51 files changed
+4393
-1021
No files matched your search
@@ -88,6 +88,8 @@ skills/
|
|||||||
hermes-agent-main/
|
hermes-agent-main/
|
||||||
.gitignore
|
.gitignore
|
||||||
offline-backend-20260512/backend/tests/*
|
offline-backend-20260512/backend/tests/*
|
||||||
|
# Keep the openGauss URL/dialect/schema compatibility regression coverage versioned.
|
||||||
|
!offline-backend-20260512/backend/tests/test_gauss_database.py
|
||||||
!offline-backend-20260512/backend/tests/test_llmwiki_weknora.py
|
!offline-backend-20260512/backend/tests/test_llmwiki_weknora.py
|
||||||
!offline-backend-20260512/backend/tests/test_health_auth_bypass.py
|
!offline-backend-20260512/backend/tests/test_health_auth_bypass.py
|
||||||
# Keep formal Word OOXML and embedded-font regression coverage versioned.
|
# Keep formal Word OOXML and embedded-font regression coverage versioned.
|
||||||
|
|||||||
@@ -1,8 +1,10 @@
|
|||||||
# 技能归纳、WeKnora 普通知识库与助手知识库:实现设计
|
# 技能归纳、WeKnora 普通知识库与助手知识库:实现设计
|
||||||
|
|
||||||
本文档用于指导大模型或开发者实现知识库体系改造。整体包含两条线:一条是“将一个或多个技能归纳到一个或多个目标知识库”;另一条是新增一个自研的全局助手知识库,用于承接 WeKnora 已处理、已向量化后的 Wiki、实体和关系数据,也支持技能和管理员本地数据直接导入,并做统一实体对齐、Wiki 展示和版本管理。
|
本文档用于指导大模型或开发者实现知识库体系改造。整体包含两条线:一条是“将一个或多个技能归纳到一个 WeKnora Wiki 普通知识库”;另一条是新增自研助手知识库体系。助手知识库像普通知识库一样可以创建多个库,同时系统内置一个全局大库,用于统计全库数据、承接所有 WeKnora 导入沉淀、做全局实体对齐、Wiki 展示和导入审计。
|
||||||
|
|
||||||
本文档中,目标知识库既可以是 WeKnora 普通知识库,也可以是自研助手知识库。普通知识库负责向量化、检索、Wiki 和图谱;助手知识库负责全局知识梳理、实体对齐、Wiki 展示、搜索和版本管理。
|
本文档中,技能归纳目标只能是 WeKnora Wiki 普通知识库;助手知识库不接收技能直接归纳,也不接收任意本地文件。普通知识库负责向量化、检索、Wiki 和图谱;助手知识库负责自研存储、WeKnora 导入、实体对齐、Wiki 展示、搜索和导入审计。无论 WeKnora 数据导入到哪个助手知识库,最终都必须沉淀一份到系统全局大库。
|
||||||
|
|
||||||
|
> 最终约束:如果本文历史章节与本节冲突,以本节和“2. 当前边界”中的约束为准。
|
||||||
|
|
||||||
## 1. 背景与目标
|
## 1. 背景与目标
|
||||||
|
|
||||||
@@ -14,57 +16,59 @@
|
|||||||
|
|
||||||
本次要新增一条管理员工作流:
|
本次要新增一条管理员工作流:
|
||||||
|
|
||||||
管理员在“技能管理”页面选择多个技能,再选择一个或多个目标知识库,点击归纳。系统将每个技能作为独立来源读取、解析、抽取和归纳,把结果写入对应知识库,并在技能列表和技能详情中展示“该技能已被哪些知识库归纳”。
|
管理员在“技能管理”页面选择多个技能,再选择一个目标 WeKnora Wiki 普通知识库,点击归纳。系统将每个技能作为独立来源读取、解析、抽取和归纳,把结果写入对应 WeKnora 知识库,并在技能列表和技能详情中展示“该技能已被哪些知识库归纳”。如果需要归纳到多个普通知识库,通过多次操作完成。
|
||||||
|
|
||||||
用户已确认的业务约束:
|
用户已确认的业务约束:
|
||||||
|
|
||||||
- 一次操作可以选择多个技能,也可以选择多个目标知识库;系统按 `技能 x 目标知识库` 生成处理矩阵。
|
- 一次操作可以选择多个技能,但只能选择一个目标 WeKnora Wiki 普通知识库。
|
||||||
- 同一个技能可以在不同时间归纳到多个知识库,所以最终关系是技能与知识库多对多。
|
- 同一个技能可以在不同时间归纳到多个知识库,所以最终关系是技能与知识库多对多。
|
||||||
- V1 支持归纳到多种目标:Wiki WeKnora、非 Wiki WeKnora、FAQ 知识库、系统会话存档知识库、只读知识库,以及自研助手知识库。
|
- 多知识库归纳通过多次操作完成;每次操作形成一批 `技能 x 单目标知识库` 任务。
|
||||||
|
- V1 只支持归纳到可写 WeKnora Wiki 普通知识库;不支持非 Wiki、FAQ、系统会话存档、只读知识库作为技能归纳目标。
|
||||||
- 技能归纳时只抽取数据、业务说明、实体、概念、对象、事件、关系等知识;Python、TypeScript、Shell 等代码和脚本类文件完全排除,不读取注释、不抽取函数、不上传、不进入 prompt。
|
- 技能归纳时只抽取数据、业务说明、实体、概念、对象、事件、关系等知识;Python、TypeScript、Shell 等代码和脚本类文件完全排除,不读取注释、不抽取函数、不上传、不进入 prompt。
|
||||||
- 技能里的 Markdown、业务数据文件、表格、PDF、Office、图片等要纳入处理范围。
|
- 技能里的 Markdown、业务数据文件、表格、PDF、Office、图片等要纳入处理范围。
|
||||||
- 需要同步三类内容:原始资料、归纳后的 Wiki、实体与关系。
|
- 需要同步三类内容:原始资料、归纳后的 Wiki、实体与关系。
|
||||||
- 关系来源包括 Markdown 自然语言描述、目录和文件引用、模型自动推断。
|
- 关系来源包括 Markdown 自然语言描述、目录和文件引用、模型自动推断。
|
||||||
- 允许模型推断关系,同时前端需要提供逐条实体/关系预览审核能力。
|
- 允许模型推断关系,但前端不提供逐条实体/关系预览审核。
|
||||||
- 支持手动重新归纳,也支持自动文件监听、定时后台同步和技能变更后的自动远程写入。
|
- 支持管理员手动点击“重新归纳”;不新增自动文件监听、定时后台同步或技能变更自动远程写入。
|
||||||
- 解除关联时弹窗让用户选择:保留知识库内容,或删除该技能贡献的内容。
|
- 解除关联时弹窗让用户选择:保留知识库内容,或删除该技能贡献的内容。
|
||||||
- 权限仅管理员。
|
- 权限仅管理员。
|
||||||
- 总技能数约 200 个,需要支持批量进度、后台执行、失败重试。
|
- 总技能数约 200 个,需要支持批量进度、后台执行、失败重试。
|
||||||
- 同一目标知识库内,不同技能抽取出的同一实体需要合并。
|
- 同一目标知识库内,不同技能抽取出的同一实体需要合并。
|
||||||
- 重归纳时先生成新版本,成功后再替换旧版本;失败时旧版本继续可用。
|
- 重归纳时先生成新版本,成功后再替换旧版本;失败时旧版本继续可用。
|
||||||
- 新增知识库类型区分:普通知识库是 WeKnora,助手知识库是自研存储。
|
- 新增知识库类型区分:普通知识库是 WeKnora,助手知识库是自研存储。
|
||||||
- 助手知识库只有一个全局大库,建议命名为“知识梳理”,所有用户可见。
|
- 助手知识库支持创建多个库;其中有且只有一个系统全局大库,建议命名为“知识梳理总库”,所有用户可见。
|
||||||
- 助手知识库只提供 Wiki 页面展示、搜索和数据导入;不落到 WeKnora。
|
- 助手知识库只提供 Wiki 页面展示、搜索和数据导入;不落到 WeKnora。
|
||||||
- 助手知识库数据来源包括 WeKnora 导出、技能直接归纳、管理员本地文件导入。
|
- 助手知识库数据来源只能是 WeKnora 导出包或从某个 WeKnora 知识库触发导入。
|
||||||
- WeKnora 已向量化和处理完成的 Wiki、文档、实体、关系、图谱数据默认沉淀到助手知识库。
|
- WeKnora 已向量化和处理完成的 Wiki、文档、实体、关系、图谱数据默认沉淀到助手知识库;如果导入的是某个自建助手库,也必须同步沉淀到全局大库。
|
||||||
- 普通知识库需要提供导出或“导入到助手知识库”的功能,把 WeKnora 里已经处理好的数据导入助手知识库;导出数据需要包含 Wiki、文档、chunk、实体、关系、图谱。
|
- 普通知识库需要提供导出或“导入到助手知识库”的功能,把 WeKnora 里已经处理好的数据导入指定助手知识库;导出数据需要包含 Wiki、文档、chunk、实体、关系、图谱。
|
||||||
- 即使没有配置 WeKnora,也要默认展示助手知识库的创建或初始化入口。
|
- 即使没有配置 WeKnora 地址,也要默认展示助手知识库列表和创建入口,而不是展示空的普通知识库页面。
|
||||||
- 管理员可切换知识库产品版本:`v1` / `v2`,默认 `v1`。
|
- 管理员可切换知识库产品版本:`v1` / `v2`,默认 `v1`。
|
||||||
- 助手知识库普通用户只读,管理员可导入、编辑、删除、查看版本和回滚 Wiki 页面。
|
- 助手知识库普通用户只读,管理员可导入、编辑、删除、查看版本和回滚 Wiki 页面。
|
||||||
|
|
||||||
## 2. 必须支持的扩展能力
|
## 2. 当前边界
|
||||||
|
|
||||||
以下能力原先按非目标处理,最新需求已确认全部要支持。实现时必须把它们作为正式目标,并加上必要的权限、状态和审计护栏。
|
以下能力明确不做,开发实现和验收必须按这些边界收敛:
|
||||||
|
|
||||||
- 支持一次选择多个目标知识库。一个归纳任务可包含多个技能和多个目标,后端按矩阵拆分 item。
|
- 不支持一次选择多个目标知识库。多知识库归纳通过多次操作完成。
|
||||||
- 支持非 Wiki 知识库、FAQ 知识库、系统会话存档知识库、只读知识库作为目标。不同目标使用不同写入 adapter;只读目标使用本地 sidecar 投影和待推送状态。
|
- 不支持非 Wiki 知识库、FAQ 知识库、系统会话存档知识库、只读知识库作为技能归纳目标。
|
||||||
- 支持直接写 WeKnora 的 Neo4j 或底层图数据库,但必须通过受控 graph writer、显式配置开关、事务和审计,不允许 LLM 生成任意 Cypher 直接执行。
|
- 不直接写 WeKnora 的 Neo4j 或底层图数据库。
|
||||||
- 支持自动文件监听、定时后台同步、技能变更自动远程写入。自动写入仍必须走 staged 版本和失败保留旧版本机制。
|
- 不新增自动文件监听、定时后台同步或技能变更自动远程写入。
|
||||||
- 前端提供逐条实体/关系预览审核,管理员可批准、拒绝、编辑、批量处理。
|
- 不在前端提供逐条实体/关系预览审核。
|
||||||
- 前端和 API 暴露远端 WeKnora ID,并可作为跨系统业务 key 使用;本地仍保留 surrogate id 以保证数据库关系稳定。
|
- 不把远端 WeKnora 内部 ID 暴露为前端业务主键。
|
||||||
- 支持技能直接归纳到助手知识库,不必先经过 WeKnora。
|
- 不支持技能直接归纳到助手知识库。技能先进入 WeKnora 普通知识库,再由 WeKnora 导出沉淀到助手知识库。
|
||||||
- 支持管理员向助手知识库导入任意本地文件,导入前必须做文件类型、安全扫描和敏感信息阻断。
|
- 不支持管理员向助手知识库导入任意本地文件;手动导入也必须是 WeKnora 导出包或从某个 WeKnora 知识库触发。
|
||||||
- 助手知识库 V1 支持 Wiki 页面版本管理、版本对比、发布、回滚和导入审计。
|
- 助手知识库 V1 不做页面版本管理,只保留当前合并后的 Wiki 状态和导入审计记录。
|
||||||
|
|
||||||
## 3. 术语
|
## 3. 术语
|
||||||
|
|
||||||
| 术语 | 含义 |
|
| 术语 | 含义 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| 技能 | DeerFlow 技能目录,通常包含 `SKILL.md`、代码、配置、示例、文档、数据文件等。 |
|
| 技能 | DeerFlow 技能目录,通常包含 `SKILL.md`、代码、配置、示例、文档、数据文件等。 |
|
||||||
| 目标知识库 | 管理员选择的一个或多个知识库,可以是 WeKnora 普通知识库,也可以是自研助手知识库。 |
|
| 目标知识库 | 管理员选择的一个 WeKnora Wiki 普通知识库。助手知识库不是技能归纳目标。 |
|
||||||
| 普通知识库 | WeKnora 知识库,负责向量化、Wiki、文档检索和图谱能力。 |
|
| 普通知识库 | WeKnora 知识库,负责向量化、Wiki、文档检索和图谱能力。 |
|
||||||
| 助手知识库 | DeerFlow 自研知识库,只维护一个全局大库,用于 Wiki 展示、搜索、WeKnora 导入、技能直接导入、本地文件导入和实体对齐。 |
|
| 助手知识库 | DeerFlow 自研知识库类型,支持创建多个库,用于 Wiki 展示、搜索、WeKnora 导入和实体对齐。 |
|
||||||
| 知识梳理底库 | 助手知识库的默认全局库名,所有用户可见,普通用户只读。 |
|
| 知识梳理总库 | 助手知识库体系中的系统全局大库,有且只有一个,自动统计和沉淀所有助手库与普通知识库导入数据,做全局实体对齐。 |
|
||||||
|
| 自建助手库 | 管理员创建的普通助手知识库,类似普通知识库列表中的一个库,可导入 WeKnora 普通知识库导出数据。 |
|
||||||
| 产品版本 | 知识库产品形态开关。`v1` 默认为普通知识库体验,`v2` 默认为助手知识库体验。 |
|
| 产品版本 | 知识库产品形态开关。`v1` 默认为普通知识库体验,`v2` 默认为助手知识库体验。 |
|
||||||
| WeKnora 导出包 | 从普通知识库导出的已处理知识集合,包含 Wiki 页面、文档、chunk、实体、关系、图谱等。 |
|
| WeKnora 导出包 | 从普通知识库导出的已处理知识集合,包含 Wiki 页面、文档、chunk、实体、关系、图谱等。 |
|
||||||
| 绑定 | 一个技能与一个目标知识库之间的持久关联。`skill_name + target_type + target_id` 唯一。 |
|
| 绑定 | 一个技能与一个目标知识库之间的持久关联。`skill_name + target_type + target_id` 唯一。 |
|
||||||
@@ -81,37 +85,35 @@
|
|||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart LR
|
flowchart LR
|
||||||
A[管理员选择多个技能] --> B[选择一个或多个目标知识库]
|
A[管理员选择多个技能] --> B[选择一个 WeKnora Wiki 目标知识库]
|
||||||
B --> C[创建归纳任务矩阵]
|
B --> C[创建归纳任务]
|
||||||
C --> D[逐技能扫描与解析]
|
C --> D[逐技能扫描与解析]
|
||||||
D --> E[生成技能快照]
|
D --> E[生成技能快照]
|
||||||
E --> F[LLM 归纳 Wiki 与关系抽取]
|
E --> F[LLM 归纳 Wiki 与关系抽取]
|
||||||
F --> R[前端实体/关系预览审核]
|
F --> G[按目标 WeKnora Wiki 合并实体/关系]
|
||||||
R --> G[按目标知识库合并实体/关系]
|
G --> H[写入 WeKnora 原始资料与 Wiki]
|
||||||
G --> H[写入 WeKnora 或本地助手库]
|
G --> I[写入关系投影文档]
|
||||||
G --> I[写入 Wiki/FAQ/会话存档/sidecar]
|
|
||||||
G --> J[写入 WeKnora GraphRAG 或 Neo4j]
|
|
||||||
H --> K[校验远端写入]
|
H --> K[校验远端写入]
|
||||||
I --> K
|
I --> K
|
||||||
J --> K
|
|
||||||
K --> L[本地原子激活新版本]
|
K --> L[本地原子激活新版本]
|
||||||
L --> M[技能页展示已归纳知识库]
|
L --> M[技能页展示已归纳知识库]
|
||||||
K --> N[WeKnora 导出已处理数据]
|
K --> N[WeKnora 导出已处理数据]
|
||||||
N --> O[沉淀到助手知识库]
|
N --> O[导入指定助手知识库]
|
||||||
O --> P[全局实体对齐]
|
O --> P[同步沉淀到知识梳理总库]
|
||||||
P --> Q[知识梳理 Wiki 展示与搜索]
|
P --> Q[全局实体对齐]
|
||||||
|
Q --> S[总库 Wiki 展示与搜索]
|
||||||
```
|
```
|
||||||
|
|
||||||
关键原则:
|
关键原则:
|
||||||
|
|
||||||
- DeerFlow 本地数据库保存权威绑定关系、版本状态、来源贡献和远端 artifact 映射。
|
- DeerFlow 本地数据库保存权威绑定关系、版本状态、来源贡献和远端 artifact 映射。
|
||||||
- WeKnora 保存可检索知识内容,包括原文、Wiki 页面、FAQ、会话存档、关系投影文档和图谱。
|
- WeKnora 保存可检索知识内容,包括原文、Wiki 页面、FAQ、会话存档、关系投影文档和图谱。
|
||||||
- 对 WeKnora 图谱能力采取能力探测和分层写入:优先使用 WeKnora API;需要底层图写入时,使用受控 Neo4j graph writer。
|
- 不直接写 WeKnora 底层图数据库;关系以 WeKnora API 能力和关系投影文档承接。
|
||||||
- 重归纳使用 staged artifact,远端写入和校验成功后,本地才激活新版本。
|
- 重归纳使用 staged artifact,远端写入和校验成功后,本地才激活新版本。
|
||||||
- 删除或解除关联只影响该技能在目标知识库中的贡献,不误删其他技能共享的实体和关系。
|
- 删除或解除关联只影响该技能在目标知识库中的贡献,不误删其他技能共享的实体和关系。
|
||||||
- 助手知识库只有一个全局库,可以接收技能直接归纳、WeKnora 导出和管理员本地文件导入。
|
- 助手知识库可以创建多个库;系统全局大库只有一个,负责汇总全库数据和做跨库实体对齐。
|
||||||
- WeKnora 的已处理结果默认进入助手知识库,由助手底库做全局实体对齐、Wiki 展示、搜索和版本管理。
|
- WeKnora 的已处理结果可以导入指定助手知识库;无论是否指定自建助手库,都必须同步沉淀到知识梳理总库。
|
||||||
- 未配置 WeKnora 时,仍展示助手知识库创建或初始化能力;与 WeKnora 相关的归纳和导入动作置灰或隐藏。
|
- 未配置 WeKnora 地址时,默认进入助手知识库页面,展示助手知识库列表、创建入口和知识梳理总库;与 WeKnora 相关的导出导入动作置灰或隐藏。
|
||||||
|
|
||||||
## 5. 现有代码落点
|
## 5. 现有代码落点
|
||||||
|
|
||||||
@@ -193,6 +195,14 @@ items = selected_skills x selected_targets
|
|||||||
|
|
||||||
每个 item 对应一个 `binding`,一个技能归纳到三个目标知识库,就生成三个 binding。父 job 负责展示整体进度,item 独立成功、失败、取消和重试。
|
每个 item 对应一个 `binding`,一个技能归纳到三个目标知识库,就生成三个 binding。父 job 负责展示整体进度,item 独立成功、失败、取消和重试。
|
||||||
|
|
||||||
|
当目标是自建助手库时,item 写入顺序是:
|
||||||
|
|
||||||
|
```text
|
||||||
|
写入目标助手库 -> 生成目标库 Wiki/实体/关系 -> 创建总库沉淀任务 -> 总库实体对齐和统计
|
||||||
|
```
|
||||||
|
|
||||||
|
总库沉淀失败不能让目标助手库写入结果消失,但 item 必须标记为 `partial_failed`,并允许单独重试“沉淀到总库”。
|
||||||
|
|
||||||
目标知识库需要通过 adapter 写入:
|
目标知识库需要通过 adapter 写入:
|
||||||
|
|
||||||
| 目标类型 | 写入策略 | Wiki | 关系/图谱 |
|
| 目标类型 | 写入策略 | Wiki | 关系/图谱 |
|
||||||
@@ -202,7 +212,7 @@ items = selected_skills x selected_targets
|
|||||||
| WeKnora FAQ 知识库 | 把归纳结果转换为 FAQ 条目、来源文档和关系摘要 | 不写或只写 sidecar Wiki | 关系作为 FAQ metadata、projection 文档和图谱边 |
|
| WeKnora FAQ 知识库 | 把归纳结果转换为 FAQ 条目、来源文档和关系摘要 | 不写或只写 sidecar Wiki | 关系作为 FAQ metadata、projection 文档和图谱边 |
|
||||||
| 系统会话存档知识库 | 把技能归纳结果作为系统会话归档条目写入 | sidecar Wiki 展示 | 关系作为归档 metadata 和图谱边 |
|
| 系统会话存档知识库 | 把技能归纳结果作为系统会话归档条目写入 | sidecar Wiki 展示 | 关系作为归档 metadata 和图谱边 |
|
||||||
| 只读 WeKnora 知识库 | 不修改远端原库;写本地 sidecar artifact,并生成待推送状态 | sidecar Wiki 展示 | sidecar graph + 待推送 Neo4j plan |
|
| 只读 WeKnora 知识库 | 不修改远端原库;写本地 sidecar artifact,并生成待推送状态 | sidecar Wiki 展示 | sidecar graph + 待推送 Neo4j plan |
|
||||||
| 助手知识库 | 写入自研底库、实体对齐、生成 Wiki 页面版本 | 写自研 Wiki | 写自研实体/关系表 |
|
| 助手知识库 | 写入指定助手库、实体对齐、生成 Wiki 页面版本;同时异步沉淀到知识梳理总库 | 写自研 Wiki | 写自研实体/关系表;总库做跨库实体对齐 |
|
||||||
|
|
||||||
只读目标说明:
|
只读目标说明:
|
||||||
|
|
||||||
|
|||||||
@@ -13,6 +13,8 @@ import {
|
|||||||
} from "@/core/system-settings/api";
|
} from "@/core/system-settings/api";
|
||||||
|
|
||||||
export const KNOWLEDGE_PRODUCT_KEY = ["system-settings", "knowledge-base-product-version"] as const;
|
export const KNOWLEDGE_PRODUCT_KEY = ["system-settings", "knowledge-base-product-version"] as const;
|
||||||
|
export const KNOWLEDGE_NORMAL_VIEW_VALUE = "normal";
|
||||||
|
export const KNOWLEDGE_NORMAL_VIEW_SEARCH = `?view=${KNOWLEDGE_NORMAL_VIEW_VALUE}`;
|
||||||
|
|
||||||
export function useKnowledgeBaseProductVersion() {
|
export function useKnowledgeBaseProductVersion() {
|
||||||
return useQuery({ queryKey: KNOWLEDGE_PRODUCT_KEY, queryFn: loadKnowledgeBaseProductVersion, staleTime: 60_000 });
|
return useQuery({ queryKey: KNOWLEDGE_PRODUCT_KEY, queryFn: loadKnowledgeBaseProductVersion, staleTime: 60_000 });
|
||||||
@@ -35,7 +37,7 @@ export function KnowledgeProductNav() {
|
|||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="flex items-center gap-2 border-b bg-card px-6 py-2">
|
<div className="flex items-center gap-2 border-b bg-card px-6 py-2">
|
||||||
<Button size="sm" variant={assistant ? "ghost" : "secondary"} onClick={() => navigate("/page/workspace/knowledge")}>
|
<Button size="sm" variant={assistant ? "ghost" : "secondary"} onClick={() => navigate(`/page/workspace/knowledge${KNOWLEDGE_NORMAL_VIEW_SEARCH}`)}>
|
||||||
<BookOpen />普通知识库
|
<BookOpen />普通知识库
|
||||||
</Button>
|
</Button>
|
||||||
<Button size="sm" variant={assistant ? "secondary" : "ghost"} onClick={() => navigate("/page/workspace/knowledge/assistant")}>
|
<Button size="sm" variant={assistant ? "secondary" : "ghost"} onClick={() => navigate("/page/workspace/knowledge/assistant")}>
|
||||||
|
|||||||
@@ -22,6 +22,10 @@ import {
|
|||||||
} from "@/components/ui/dropdown-menu";
|
} from "@/components/ui/dropdown-menu";
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
import { cn } from "@/lib/utils";
|
import { cn } from "@/lib/utils";
|
||||||
|
import {
|
||||||
|
type AssistantKnowledgeBase,
|
||||||
|
useAssistantKnowledgeBases,
|
||||||
|
} from "@/strategy-components/api/assistant-knowledge";
|
||||||
import {
|
import {
|
||||||
type LlmWikiKnowledgeBase,
|
type LlmWikiKnowledgeBase,
|
||||||
useAgentLlmWikiKnowledgeBases,
|
useAgentLlmWikiKnowledgeBases,
|
||||||
@@ -66,10 +70,26 @@ function SpaceIcon({ item }: { item: LlmWikiKnowledgeBase }) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function AssistantSpaceIcon({ item }: { item: AssistantKnowledgeBase }) {
|
||||||
|
return (
|
||||||
|
<span
|
||||||
|
className={cn(
|
||||||
|
"flex size-7 shrink-0 items-center justify-center rounded-md",
|
||||||
|
item.is_global ? "bg-amber-500/10 text-amber-600 dark:text-amber-400" : "bg-sky-500/10 text-sky-600 dark:text-sky-400",
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
<Database className="size-3.5" />
|
||||||
|
</span>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
function SelectionContent({
|
function SelectionContent({
|
||||||
rows,
|
rows,
|
||||||
|
assistantRows,
|
||||||
value,
|
value,
|
||||||
|
assistantValue,
|
||||||
onChange,
|
onChange,
|
||||||
|
onAssistantChange,
|
||||||
search,
|
search,
|
||||||
onSearchChange,
|
onSearchChange,
|
||||||
loading,
|
loading,
|
||||||
@@ -79,8 +99,11 @@ function SelectionContent({
|
|||||||
readOnly,
|
readOnly,
|
||||||
}: {
|
}: {
|
||||||
rows: LlmWikiKnowledgeBase[];
|
rows: LlmWikiKnowledgeBase[];
|
||||||
|
assistantRows: AssistantKnowledgeBase[];
|
||||||
value: string[];
|
value: string[];
|
||||||
|
assistantValue: string[];
|
||||||
onChange: (value: string[]) => void;
|
onChange: (value: string[]) => void;
|
||||||
|
onAssistantChange: (value: string[]) => void;
|
||||||
search: string;
|
search: string;
|
||||||
onSearchChange: (value: string) => void;
|
onSearchChange: (value: string) => void;
|
||||||
loading: boolean;
|
loading: boolean;
|
||||||
@@ -90,6 +113,7 @@ function SelectionContent({
|
|||||||
readOnly: boolean;
|
readOnly: boolean;
|
||||||
}) {
|
}) {
|
||||||
const selected = useMemo(() => new Set(value), [value]);
|
const selected = useMemo(() => new Set(value), [value]);
|
||||||
|
const selectedAssistant = useMemo(() => new Set(assistantValue), [assistantValue]);
|
||||||
const filtered = useMemo(() => {
|
const filtered = useMemo(() => {
|
||||||
const query = search.trim().toLocaleLowerCase();
|
const query = search.trim().toLocaleLowerCase();
|
||||||
if (!query) return rows;
|
if (!query) return rows;
|
||||||
@@ -97,11 +121,28 @@ function SelectionContent({
|
|||||||
`${item.name} ${item.description}`.toLocaleLowerCase().includes(query),
|
`${item.name} ${item.description}`.toLocaleLowerCase().includes(query),
|
||||||
);
|
);
|
||||||
}, [rows, search]);
|
}, [rows, search]);
|
||||||
|
const filteredAssistant = useMemo(() => {
|
||||||
|
const query = search.trim().toLocaleLowerCase();
|
||||||
|
if (!query) return assistantRows;
|
||||||
|
return assistantRows.filter((item) =>
|
||||||
|
`${item.name} ${item.description ?? ""}`.toLocaleLowerCase().includes(query),
|
||||||
|
);
|
||||||
|
}, [assistantRows, search]);
|
||||||
const groups = useMemo(() => groupRows(filtered), [filtered]);
|
const groups = useMemo(() => groupRows(filtered), [filtered]);
|
||||||
const toggle = (id: string) => {
|
const toggle = (id: string) => {
|
||||||
if (readOnly) return;
|
if (readOnly) return;
|
||||||
onChange(selected.has(id) ? value.filter((item) => item !== id) : [...value, id]);
|
onChange(selected.has(id) ? value.filter((item) => item !== id) : [...value, id]);
|
||||||
};
|
};
|
||||||
|
const toggleAssistant = (id: string) => {
|
||||||
|
if (readOnly) return;
|
||||||
|
onAssistantChange(
|
||||||
|
selectedAssistant.has(id)
|
||||||
|
? assistantValue.filter((item) => item !== id)
|
||||||
|
: [...assistantValue, id],
|
||||||
|
);
|
||||||
|
};
|
||||||
|
const selectedCount = value.length + assistantValue.length;
|
||||||
|
const hasRows = groups.length > 0 || filteredAssistant.length > 0;
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<>
|
<>
|
||||||
@@ -127,7 +168,7 @@ function SelectionContent({
|
|||||||
<div className="m-2 rounded-lg border border-destructive/30 bg-destructive/5 p-3 text-xs text-destructive">
|
<div className="m-2 rounded-lg border border-destructive/30 bg-destructive/5 p-3 text-xs text-destructive">
|
||||||
{error instanceof Error ? error.message : "知识空间读取失败"}
|
{error instanceof Error ? error.message : "知识空间读取失败"}
|
||||||
</div>
|
</div>
|
||||||
) : groups.length === 0 ? (
|
) : !hasRows ? (
|
||||||
<div className="flex h-28 items-center justify-center px-4 text-center text-xs text-muted-foreground">
|
<div className="flex h-28 items-center justify-center px-4 text-center text-xs text-muted-foreground">
|
||||||
{search
|
{search
|
||||||
? "没有匹配的知识空间"
|
? "没有匹配的知识空间"
|
||||||
@@ -136,91 +177,181 @@ function SelectionContent({
|
|||||||
: "当前账号暂无可用知识空间"}
|
: "当前账号暂无可用知识空间"}
|
||||||
</div>
|
</div>
|
||||||
) : (
|
) : (
|
||||||
groups.map((group, groupIndex) => (
|
<>
|
||||||
<div key={group.key}>
|
{groups.map((group, groupIndex) => (
|
||||||
{groupIndex > 0 && (
|
<div key={group.key}>
|
||||||
variant === "popover"
|
{groupIndex > 0 && (
|
||||||
? <DropdownMenuSeparator className="my-1.5" />
|
variant === "popover"
|
||||||
: <div className="my-1.5 border-t" />
|
? <DropdownMenuSeparator className="my-1.5" />
|
||||||
)}
|
: <div className="my-1.5 border-t" />
|
||||||
{variant === "popover" ? (
|
)}
|
||||||
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
{variant === "popover" ? (
|
||||||
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||||
</DropdownMenuLabel>
|
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
||||||
) : (
|
</DropdownMenuLabel>
|
||||||
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
|
||||||
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{group.rows.map((item) => {
|
|
||||||
const rowContent = (
|
|
||||||
<>
|
|
||||||
<SpaceIcon item={item} />
|
|
||||||
<div className="min-w-0 flex-1">
|
|
||||||
<div className="truncate text-xs font-medium">{item.name}</div>
|
|
||||||
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
|
|
||||||
{item.type === "faq" ? "FAQ" : "文档库"} · {item.knowledge_count ?? 0} 项知识
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
{item.publication_status === "published" && (
|
|
||||||
<Badge variant="outline" className="h-4 px-1 text-[9px]">公共</Badge>
|
|
||||||
)}
|
|
||||||
</>
|
|
||||||
);
|
|
||||||
return variant === "popover" ? (
|
|
||||||
<DropdownMenuCheckboxItem
|
|
||||||
key={item.id}
|
|
||||||
checked={selected.has(item.id)}
|
|
||||||
disabled={readOnly}
|
|
||||||
onCheckedChange={() => toggle(item.id)}
|
|
||||||
onSelect={(event) => event.preventDefault()}
|
|
||||||
className={cn(
|
|
||||||
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
|
|
||||||
selected.has(item.id) && "bg-primary/8 focus:bg-primary/12",
|
|
||||||
)}
|
|
||||||
>
|
|
||||||
{rowContent}
|
|
||||||
</DropdownMenuCheckboxItem>
|
|
||||||
) : (
|
) : (
|
||||||
<button
|
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||||
key={item.id}
|
{group.key === "personal" ? <Folder className="size-3" /> : <Globe2 className="size-3" />}{group.label}
|
||||||
type="button"
|
</div>
|
||||||
role="checkbox"
|
)}
|
||||||
aria-checked={selected.has(item.id)}
|
{group.rows.map((item) => {
|
||||||
disabled={readOnly}
|
const rowContent = (
|
||||||
onClick={() => toggle(item.id)}
|
<>
|
||||||
className={cn(
|
<SpaceIcon item={item} />
|
||||||
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
|
<div className="min-w-0 flex-1">
|
||||||
selected.has(item.id) && "bg-primary/8",
|
<div className="truncate text-xs font-medium">{item.name}</div>
|
||||||
)}
|
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
|
||||||
>
|
{item.type === "faq" ? "FAQ" : "文档库"} · {item.knowledge_count ?? 0} 项知识
|
||||||
<span className={cn(
|
</div>
|
||||||
"flex size-4 shrink-0 items-center justify-center rounded border",
|
</div>
|
||||||
selected.has(item.id) && "border-primary bg-primary text-primary-foreground",
|
{item.publication_status === "published" && (
|
||||||
)}>
|
<Badge variant="outline" className="h-4 px-1 text-[9px]">公共</Badge>
|
||||||
{selected.has(item.id) && <Check className="size-3" />}
|
)}
|
||||||
</span>
|
</>
|
||||||
{rowContent}
|
);
|
||||||
</button>
|
return variant === "popover" ? (
|
||||||
);
|
<DropdownMenuCheckboxItem
|
||||||
})}
|
key={item.id}
|
||||||
</div>
|
checked={selected.has(item.id)}
|
||||||
))
|
disabled={readOnly}
|
||||||
|
onCheckedChange={() => toggle(item.id)}
|
||||||
|
onSelect={(event) => event.preventDefault()}
|
||||||
|
className={cn(
|
||||||
|
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
|
||||||
|
selected.has(item.id) && "bg-primary/8 focus:bg-primary/12",
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
{rowContent}
|
||||||
|
</DropdownMenuCheckboxItem>
|
||||||
|
) : (
|
||||||
|
<button
|
||||||
|
key={item.id}
|
||||||
|
type="button"
|
||||||
|
role="checkbox"
|
||||||
|
aria-checked={selected.has(item.id)}
|
||||||
|
disabled={readOnly}
|
||||||
|
onClick={() => toggle(item.id)}
|
||||||
|
className={cn(
|
||||||
|
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
|
||||||
|
selected.has(item.id) && "bg-primary/8",
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
<span className={cn(
|
||||||
|
"flex size-4 shrink-0 items-center justify-center rounded border",
|
||||||
|
selected.has(item.id) && "border-primary bg-primary text-primary-foreground",
|
||||||
|
)}>
|
||||||
|
{selected.has(item.id) && <Check className="size-3" />}
|
||||||
|
</span>
|
||||||
|
{rowContent}
|
||||||
|
</button>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
{filteredAssistant.length > 0 && (
|
||||||
|
<div>
|
||||||
|
{(groups.length > 0) && (
|
||||||
|
variant === "popover"
|
||||||
|
? <DropdownMenuSeparator className="my-1.5" />
|
||||||
|
: <div className="my-1.5 border-t" />
|
||||||
|
)}
|
||||||
|
{variant === "popover" ? (
|
||||||
|
<DropdownMenuLabel className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||||
|
<Database className="size-3" />助手知识库
|
||||||
|
</DropdownMenuLabel>
|
||||||
|
) : (
|
||||||
|
<div className="flex items-center gap-1.5 px-2 py-1 text-[11px] font-semibold text-muted-foreground">
|
||||||
|
<Database className="size-3" />助手知识库
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{filteredAssistant.map((item) => {
|
||||||
|
const rowContent = (
|
||||||
|
<>
|
||||||
|
<AssistantSpaceIcon item={item} />
|
||||||
|
<div className="min-w-0 flex-1">
|
||||||
|
<div className="truncate text-xs font-medium">{item.name}</div>
|
||||||
|
<div className="mt-0.5 truncate text-[10px] text-muted-foreground">
|
||||||
|
Wiki · {item.page_count ?? 0} 页 · {item.entity_count ?? 0} 实体
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<Badge variant="outline" className="h-4 px-1 text-[9px]">
|
||||||
|
{item.is_global ? "总库" : "助手"}
|
||||||
|
</Badge>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
return variant === "popover" ? (
|
||||||
|
<DropdownMenuCheckboxItem
|
||||||
|
key={`assistant:${item.id}`}
|
||||||
|
checked={selectedAssistant.has(item.id)}
|
||||||
|
disabled={readOnly}
|
||||||
|
onCheckedChange={() => toggleAssistant(item.id)}
|
||||||
|
onSelect={(event) => event.preventDefault()}
|
||||||
|
className={cn(
|
||||||
|
"my-0.5 min-h-10 rounded-md py-1.5 pr-2 pl-7",
|
||||||
|
selectedAssistant.has(item.id) && "bg-primary/8 focus:bg-primary/12",
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
{rowContent}
|
||||||
|
</DropdownMenuCheckboxItem>
|
||||||
|
) : (
|
||||||
|
<button
|
||||||
|
key={`assistant:${item.id}`}
|
||||||
|
type="button"
|
||||||
|
role="checkbox"
|
||||||
|
aria-checked={selectedAssistant.has(item.id)}
|
||||||
|
disabled={readOnly}
|
||||||
|
onClick={() => toggleAssistant(item.id)}
|
||||||
|
className={cn(
|
||||||
|
"my-0.5 flex min-h-11 w-full items-center gap-2 rounded-md px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-primary",
|
||||||
|
selectedAssistant.has(item.id) && "bg-primary/8",
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
<span className={cn(
|
||||||
|
"flex size-4 shrink-0 items-center justify-center rounded border",
|
||||||
|
selectedAssistant.has(item.id) && "border-primary bg-primary text-primary-foreground",
|
||||||
|
)}>
|
||||||
|
{selectedAssistant.has(item.id) && <Check className="size-3" />}
|
||||||
|
</span>
|
||||||
|
{rowContent}
|
||||||
|
</button>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="flex items-center justify-between gap-2 border-t bg-muted/20 px-2.5 py-2">
|
<div className="flex items-center justify-between gap-2 border-t bg-muted/20 px-2.5 py-2">
|
||||||
<span className="text-[11px] text-muted-foreground">
|
<span className="text-[11px] text-muted-foreground">
|
||||||
{readOnly ? `本对话已选 ${value.length} 个` : `已选择 ${value.length} 个`}
|
{readOnly ? `本对话已选 ${selectedCount} 个` : `已选择 ${selectedCount} 个`}
|
||||||
</span>
|
</span>
|
||||||
{readOnly ? (
|
{readOnly ? (
|
||||||
<span className="text-[11px] text-muted-foreground">对话开始后不可更改</span>
|
<span className="text-[11px] text-muted-foreground">对话开始后不可更改</span>
|
||||||
) : (
|
) : (
|
||||||
<div className="flex items-center gap-1">
|
<div className="flex items-center gap-1">
|
||||||
<Button type="button" variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => onChange(filtered.map((item) => item.id))}>
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="ghost"
|
||||||
|
size="sm"
|
||||||
|
className="h-7 px-2 text-xs"
|
||||||
|
onClick={() => {
|
||||||
|
onChange(filtered.map((item) => item.id));
|
||||||
|
onAssistantChange(filteredAssistant.map((item) => item.id));
|
||||||
|
}}
|
||||||
|
>
|
||||||
全选
|
全选
|
||||||
</Button>
|
</Button>
|
||||||
<Button type="button" variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => onChange([])}>
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="ghost"
|
||||||
|
size="sm"
|
||||||
|
className="h-7 px-2 text-xs"
|
||||||
|
onClick={() => {
|
||||||
|
onChange([]);
|
||||||
|
onAssistantChange([]);
|
||||||
|
}}
|
||||||
|
>
|
||||||
清空
|
清空
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
@@ -232,7 +363,9 @@ function SelectionContent({
|
|||||||
|
|
||||||
export function LlmWikiKnowledgeSelector({
|
export function LlmWikiKnowledgeSelector({
|
||||||
value,
|
value,
|
||||||
|
assistantValue = [],
|
||||||
onChange,
|
onChange,
|
||||||
|
onAssistantChange = () => undefined,
|
||||||
source = "all",
|
source = "all",
|
||||||
agentId,
|
agentId,
|
||||||
autoSelectAll = false,
|
autoSelectAll = false,
|
||||||
@@ -243,7 +376,9 @@ export function LlmWikiKnowledgeSelector({
|
|||||||
variant = "popover",
|
variant = "popover",
|
||||||
}: {
|
}: {
|
||||||
value: string[];
|
value: string[];
|
||||||
|
assistantValue?: string[];
|
||||||
onChange: (value: string[]) => void;
|
onChange: (value: string[]) => void;
|
||||||
|
onAssistantChange?: (value: string[]) => void;
|
||||||
source?: SelectorSource;
|
source?: SelectorSource;
|
||||||
agentId?: string | null;
|
agentId?: string | null;
|
||||||
autoSelectAll?: boolean;
|
autoSelectAll?: boolean;
|
||||||
@@ -257,28 +392,39 @@ export function LlmWikiKnowledgeSelector({
|
|||||||
const [open, setOpen] = useState(false);
|
const [open, setOpen] = useState(false);
|
||||||
const [search, setSearch] = useState("");
|
const [search, setSearch] = useState("");
|
||||||
const runtime = useLlmWikiRuntime();
|
const runtime = useLlmWikiRuntime();
|
||||||
|
const weknoraEnabled = runtime.data?.provider === "weknora";
|
||||||
const all = useLlmWikiKnowledgeBases(
|
const all = useLlmWikiKnowledgeBases(
|
||||||
"all",
|
"all",
|
||||||
runtime.data?.provider === "weknora",
|
weknoraEnabled,
|
||||||
);
|
);
|
||||||
const agent = useAgentLlmWikiKnowledgeBases(
|
const agent = useAgentLlmWikiKnowledgeBases(
|
||||||
agentId,
|
agentId,
|
||||||
runtime.data?.provider === "weknora" && source === "agent" && !!agentId,
|
weknoraEnabled && source === "agent" && !!agentId,
|
||||||
);
|
);
|
||||||
const rows = all.data ?? [];
|
const assistantBases = useAssistantKnowledgeBases();
|
||||||
|
const rows = weknoraEnabled ? (all.data ?? []) : [];
|
||||||
|
const assistantRows = assistantBases.data ?? [];
|
||||||
const defaultIds = useMemo(() => {
|
const defaultIds = useMemo(() => {
|
||||||
if (source !== "agent") {
|
if (source !== "agent") {
|
||||||
return rows.map((item) => item.id);
|
return rows.map((item) => item.id);
|
||||||
}
|
}
|
||||||
return (agent.data ?? []).map((item) => item.id);
|
return (agent.data ?? []).map((item) => item.id);
|
||||||
}, [agent.data, rows, source]);
|
}, [agent.data, rows, source]);
|
||||||
const loading = all.isLoading || (source === "agent" && !!agentId && agent.isLoading);
|
const loading = runtime.isLoading || (weknoraEnabled && all.isLoading) || assistantBases.isLoading || (source === "agent" && !!agentId && agent.isLoading);
|
||||||
const error = all.error ?? (source === "agent" ? agent.error : null);
|
const error = (weknoraEnabled ? all.error ?? (source === "agent" ? agent.error : null) : null) ?? assistantBases.error;
|
||||||
const availableIds = useMemo(() => new Set(rows.map((item) => item.id)), [rows]);
|
const availableIds = useMemo(() => new Set(rows.map((item) => item.id)), [rows]);
|
||||||
|
const assistantAvailableIds = useMemo(
|
||||||
|
() => new Set(assistantRows.map((item) => item.id)),
|
||||||
|
[assistantRows],
|
||||||
|
);
|
||||||
const initializationKey = `${source}:${agentId ?? "all"}:${defaultIds.join(",")}`;
|
const initializationKey = `${source}:${agentId ?? "all"}:${defaultIds.join(",")}`;
|
||||||
const initialized = useRef<string | null>(null);
|
const initialized = useRef<string | null>(null);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
|
if (!runtime.isLoading && !weknoraEnabled) {
|
||||||
|
if (value.length > 0) onChange([]);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (!all.data || (source === "agent" && agentId && !agent.data) || initialized.current === initializationKey) return;
|
if (!all.data || (source === "agent" && agentId && !agent.data) || initialized.current === initializationKey) return;
|
||||||
initialized.current = initializationKey;
|
initialized.current = initializationKey;
|
||||||
const valid = value.filter((id) => availableIds.has(id));
|
const valid = value.filter((id) => availableIds.has(id));
|
||||||
@@ -288,19 +434,30 @@ export function LlmWikiKnowledgeSelector({
|
|||||||
if (next.length !== value.length || next.some((id, index) => id !== value[index])) {
|
if (next.length !== value.length || next.some((id, index) => id !== value[index])) {
|
||||||
onChange(next);
|
onChange(next);
|
||||||
}
|
}
|
||||||
}, [agent.data, agentId, all.data, autoSelectAll, availableIds, defaultIds, initializationKey, onChange, source, value]);
|
}, [agent.data, agentId, all.data, autoSelectAll, availableIds, defaultIds, initializationKey, onChange, runtime.isLoading, source, value, weknoraEnabled]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!assistantBases.data) return;
|
||||||
|
const valid = assistantValue.filter((id) => assistantAvailableIds.has(id));
|
||||||
|
if (valid.length !== assistantValue.length || valid.some((id, index) => id !== assistantValue[index])) {
|
||||||
|
onAssistantChange(valid);
|
||||||
|
}
|
||||||
|
}, [assistantAvailableIds, assistantBases.data, assistantValue, onAssistantChange]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!open) setSearch("");
|
if (!open) setSearch("");
|
||||||
}, [open]);
|
}, [open]);
|
||||||
|
|
||||||
if (runtime.isError || runtime.data?.provider !== "weknora") return null;
|
if ((runtime.isError || !weknoraEnabled) && !assistantBases.isLoading && assistantRows.length === 0) return null;
|
||||||
|
|
||||||
const content = (
|
const content = (
|
||||||
<SelectionContent
|
<SelectionContent
|
||||||
rows={rows}
|
rows={rows}
|
||||||
|
assistantRows={assistantRows}
|
||||||
value={value}
|
value={value}
|
||||||
|
assistantValue={assistantValue}
|
||||||
onChange={onChange}
|
onChange={onChange}
|
||||||
|
onAssistantChange={onAssistantChange}
|
||||||
search={search}
|
search={search}
|
||||||
onSearchChange={setSearch}
|
onSearchChange={setSearch}
|
||||||
loading={loading}
|
loading={loading}
|
||||||
@@ -323,7 +480,7 @@ export function LlmWikiKnowledgeSelector({
|
|||||||
新建该智能体对话时,只能从已绑定空间中选择本次检索范围。
|
新建该智能体对话时,只能从已绑定空间中选择本次检索范围。
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
<Badge variant="secondary" className="shrink-0">{value.length} 个</Badge>
|
<Badge variant="secondary" className="shrink-0">{value.length + assistantValue.length} 个</Badge>
|
||||||
</div>
|
</div>
|
||||||
{content}
|
{content}
|
||||||
</div>
|
</div>
|
||||||
@@ -333,10 +490,15 @@ export function LlmWikiKnowledgeSelector({
|
|||||||
const selectedNames = value
|
const selectedNames = value
|
||||||
.map((id) => rows.find((item) => item.id === id)?.name)
|
.map((id) => rows.find((item) => item.id === id)?.name)
|
||||||
.filter((name): name is string => !!name);
|
.filter((name): name is string => !!name);
|
||||||
const selectedLabel = selectedNames.length > 0
|
const selectedAssistantNames = assistantValue
|
||||||
? selectedNames.join("、")
|
.map((id) => assistantRows.find((item) => item.id === id)?.name)
|
||||||
: value.length > 0
|
.filter((name): name is string => !!name);
|
||||||
? `已选 ${value.length} 个知识空间`
|
const allSelectedNames = [...selectedNames, ...selectedAssistantNames];
|
||||||
|
const selectedCount = value.length + assistantValue.length;
|
||||||
|
const selectedLabel = allSelectedNames.length > 0
|
||||||
|
? allSelectedNames.join("、")
|
||||||
|
: selectedCount > 0
|
||||||
|
? `已选 ${selectedCount} 个知识空间`
|
||||||
: label;
|
: label;
|
||||||
|
|
||||||
return (
|
return (
|
||||||
@@ -348,20 +510,20 @@ export function LlmWikiKnowledgeSelector({
|
|||||||
size="sm"
|
size="sm"
|
||||||
className={cn(
|
className={cn(
|
||||||
"h-8 max-w-52 gap-1.5 rounded-full border-0 bg-transparent px-2.5 text-muted-foreground shadow-none hover:bg-muted/70 hover:text-foreground",
|
"h-8 max-w-52 gap-1.5 rounded-full border-0 bg-transparent px-2.5 text-muted-foreground shadow-none hover:bg-muted/70 hover:text-foreground",
|
||||||
value.length > 0 && "bg-primary/8 text-primary hover:bg-primary/12 hover:text-primary",
|
selectedCount > 0 && "bg-primary/8 text-primary hover:bg-primary/12 hover:text-primary",
|
||||||
className,
|
className,
|
||||||
)}
|
)}
|
||||||
disabled={disabled || loading}
|
disabled={disabled || loading}
|
||||||
title={value.length > 0 ? selectedLabel : undefined}
|
title={selectedCount > 0 ? selectedLabel : undefined}
|
||||||
>
|
>
|
||||||
{loading ? <Loader2 className="size-3.5 animate-spin" /> : <Database className="size-3.5" />}
|
{loading ? <Loader2 className="size-3.5 animate-spin" /> : <Database className="size-3.5" />}
|
||||||
<span className="truncate text-xs font-normal">{selectedLabel}</span>
|
<span className="truncate text-xs font-normal">{selectedLabel}</span>
|
||||||
{value.length > 0 && (
|
{selectedCount > 0 && (
|
||||||
<span className="flex size-4 items-center justify-center rounded-full bg-primary text-[9px] font-semibold text-primary-foreground">
|
<span className="flex size-4 items-center justify-center rounded-full bg-primary text-[9px] font-semibold text-primary-foreground">
|
||||||
{value.length}
|
{selectedCount}
|
||||||
</span>
|
</span>
|
||||||
)}
|
)}
|
||||||
{value.length > 0 && value.length === rows.length && <Check className="size-3" />}
|
{selectedCount > 0 && selectedCount === rows.length + assistantRows.length && <Check className="size-3" />}
|
||||||
</Button>
|
</Button>
|
||||||
</DropdownMenuTrigger>
|
</DropdownMenuTrigger>
|
||||||
<DropdownMenuContent align="start" sideOffset={8} className="w-80 overflow-hidden rounded-xl p-0 shadow-xl">
|
<DropdownMenuContent align="start" sideOffset={8} className="w-80 overflow-hidden rounded-xl p-0 shadow-xl">
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { CheckCircle2, Database, Loader2, RefreshCw, Sparkles } from "lucide-react";
|
import { Database, Loader2, RefreshCw, Search, Sparkles } from "lucide-react";
|
||||||
import { useEffect, useMemo, useState } from "react";
|
import { useEffect, useMemo, useState } from "react";
|
||||||
import { toast } from "sonner";
|
import { toast } from "sonner";
|
||||||
|
|
||||||
@@ -10,18 +10,12 @@ import { Progress } from "@/components/ui/progress";
|
|||||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||||
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
|
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
|
||||||
import { useAllSkillsAdmin } from "@/core/skills/hooks";
|
import { useAllSkillsAdmin } from "@/core/skills/hooks";
|
||||||
import {
|
|
||||||
useAssistantKnowledgeActions,
|
|
||||||
useAssistantKnowledgeBase,
|
|
||||||
} from "@/strategy-components/api/assistant-knowledge";
|
|
||||||
import { useLlmWikiKnowledgeBases, useLlmWikiRuntime } from "@/strategy-components/api/llmwiki";
|
import { useLlmWikiKnowledgeBases, useLlmWikiRuntime } from "@/strategy-components/api/llmwiki";
|
||||||
import {
|
import {
|
||||||
type SkillKnowledgeTarget,
|
type SkillKnowledgeTarget,
|
||||||
useSkillKnowledgeActions,
|
useSkillKnowledgeActions,
|
||||||
useSkillKnowledgeAutoSyncSettings,
|
|
||||||
useSkillKnowledgeBindings,
|
useSkillKnowledgeBindings,
|
||||||
useSkillKnowledgeJobs,
|
useSkillKnowledgeJobs,
|
||||||
useSkillKnowledgeReviews,
|
|
||||||
} from "@/strategy-components/api/skill-knowledge";
|
} from "@/strategy-components/api/skill-knowledge";
|
||||||
|
|
||||||
const PHASE_LABELS: Record<string, string> = {
|
const PHASE_LABELS: Record<string, string> = {
|
||||||
@@ -39,53 +33,82 @@ export function SkillKnowledgePanel() {
|
|||||||
const { skills, isLoading } = useAllSkillsAdmin(true);
|
const { skills, isLoading } = useAllSkillsAdmin(true);
|
||||||
const liveSkills = useMemo(() => skills.filter((skill) => (skill.state ?? "active") === "active"), [skills]);
|
const liveSkills = useMemo(() => skills.filter((skill) => (skill.state ?? "active") === "active"), [skills]);
|
||||||
const [selectedSkills, setSelectedSkills] = useState<string[]>([]);
|
const [selectedSkills, setSelectedSkills] = useState<string[]>([]);
|
||||||
const [selectedTargetKeys, setSelectedTargetKeys] = useState<string[]>([]);
|
const [skillQuery, setSkillQuery] = useState("");
|
||||||
const [reviewMode, setReviewMode] = useState<"required" | "auto_high_confidence" | "off">("auto_high_confidence");
|
const [selectedTargetKey, setSelectedTargetKey] = useState<string | null>(null);
|
||||||
|
const [reviewMode, setReviewMode] = useState<"auto_high_confidence" | "off">("auto_high_confidence");
|
||||||
const runtime = useLlmWikiRuntime();
|
const runtime = useLlmWikiRuntime();
|
||||||
const knowledgeBases = useLlmWikiKnowledgeBases("all", runtime.data?.provider === "weknora");
|
const knowledgeBases = useLlmWikiKnowledgeBases("all", runtime.data?.provider === "weknora");
|
||||||
const assistantBase = useAssistantKnowledgeBase();
|
|
||||||
const assistantActions = useAssistantKnowledgeActions();
|
|
||||||
const bindings = useSkillKnowledgeBindings();
|
const bindings = useSkillKnowledgeBindings();
|
||||||
const jobs = useSkillKnowledgeJobs();
|
const jobs = useSkillKnowledgeJobs();
|
||||||
const reviews = useSkillKnowledgeReviews();
|
|
||||||
const actions = useSkillKnowledgeActions();
|
const actions = useSkillKnowledgeActions();
|
||||||
const autoSettings = useSkillKnowledgeAutoSyncSettings();
|
|
||||||
const [scheduleCron, setScheduleCron] = useState("0 */6 * * *");
|
|
||||||
const [debounceSeconds, setDebounceSeconds] = useState(120);
|
|
||||||
|
|
||||||
useEffect(() => {
|
|
||||||
if (!autoSettings.data) return;
|
|
||||||
setScheduleCron(autoSettings.data.schedule_cron);
|
|
||||||
setDebounceSeconds(autoSettings.data.debounce_seconds);
|
|
||||||
}, [autoSettings.data]);
|
|
||||||
|
|
||||||
const targets = useMemo<SkillKnowledgeTarget[]>(() => {
|
const targets = useMemo<SkillKnowledgeTarget[]>(() => {
|
||||||
const rows: SkillKnowledgeTarget[] = (knowledgeBases.data ?? []).map((kb) => ({
|
return (knowledgeBases.data ?? [])
|
||||||
target_type: "weknora",
|
.filter((kb) => {
|
||||||
target_mode: kb.can_write ? kb.type === "faq" ? "faq" : kb.type === "wiki" ? "wiki" : "document" : "readonly_sidecar",
|
const strategy = kb.indexing_strategy ?? {};
|
||||||
target_id: kb.id,
|
const capabilities = kb.capabilities ?? {};
|
||||||
target_name: kb.name,
|
const wikiEnabled =
|
||||||
remote_business_key: `weknora:kb:${kb.id}`,
|
kb.type === "wiki" ||
|
||||||
readonly: !kb.can_write,
|
kb.wiki_index_enabled === true ||
|
||||||
}));
|
strategy.wiki_enabled === true ||
|
||||||
if (assistantBase.data) {
|
capabilities.wiki_enabled === true ||
|
||||||
rows.unshift({
|
(kb.wiki_config && Object.keys(kb.wiki_config).length > 0);
|
||||||
target_type: "assistant",
|
return kb.can_write && kb.remote_status !== "missing" && !kb.is_conversation_deposit && kb.type !== "faq" && wikiEnabled;
|
||||||
target_mode: "assistant",
|
})
|
||||||
target_id: assistantBase.data.id,
|
.map((kb) => ({
|
||||||
target_name: assistantBase.data.name,
|
target_type: "weknora",
|
||||||
remote_business_key: `assistant:base:${assistantBase.data.id}`,
|
target_mode: "wiki",
|
||||||
});
|
target_id: kb.id,
|
||||||
|
target_name: kb.name,
|
||||||
|
remote_business_key: `weknora:kb:${kb.id}`,
|
||||||
|
}));
|
||||||
|
}, [knowledgeBases.data]);
|
||||||
|
|
||||||
|
const filteredSkills = useMemo(() => {
|
||||||
|
const needle = skillQuery.trim().toLowerCase();
|
||||||
|
if (!needle) return liveSkills;
|
||||||
|
return liveSkills.filter((skill) => {
|
||||||
|
const tagText = (skill.tags ?? []).map((tag) => tag.name).join(" ");
|
||||||
|
const haystack = [
|
||||||
|
skill.name,
|
||||||
|
skill.name_zh,
|
||||||
|
skill.description,
|
||||||
|
skill.category,
|
||||||
|
tagText,
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join("\n")
|
||||||
|
.toLowerCase();
|
||||||
|
return haystack.includes(needle);
|
||||||
|
});
|
||||||
|
}, [liveSkills, skillQuery]);
|
||||||
|
|
||||||
|
const filteredSkillNames = useMemo(() => filteredSkills.map((skill) => skill.name), [filteredSkills]);
|
||||||
|
const allFilteredSelected = filteredSkillNames.length > 0 && filteredSkillNames.every((name) => selectedSkills.includes(name));
|
||||||
|
|
||||||
|
const toggleFilteredSkills = () => {
|
||||||
|
if (!filteredSkillNames.length) return;
|
||||||
|
setSelectedSkills((current) => {
|
||||||
|
if (allFilteredSelected) {
|
||||||
|
const filtered = new Set(filteredSkillNames);
|
||||||
|
return current.filter((name) => !filtered.has(name));
|
||||||
|
}
|
||||||
|
return [...current, ...filteredSkillNames.filter((name) => !current.includes(name))];
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (selectedTargetKey && !targets.some((target) => `${target.target_type}:${target.target_id}` === selectedTargetKey)) {
|
||||||
|
setSelectedTargetKey(null);
|
||||||
}
|
}
|
||||||
return rows;
|
}, [selectedTargetKey, targets]);
|
||||||
}, [assistantBase.data, knowledgeBases.data]);
|
|
||||||
|
|
||||||
const toggle = (values: string[], value: string, setter: (next: string[]) => void) =>
|
const toggle = (values: string[], value: string, setter: (next: string[]) => void) =>
|
||||||
setter(values.includes(value) ? values.filter((item) => item !== value) : [...values, value]);
|
setter(values.includes(value) ? values.filter((item) => item !== value) : [...values, value]);
|
||||||
|
|
||||||
const submit = async () => {
|
const submit = async () => {
|
||||||
const selectedTargets = targets.filter((target) =>
|
const selectedTargets = targets.filter((target) =>
|
||||||
selectedTargetKeys.includes(`${target.target_type}:${target.target_id}`),
|
selectedTargetKey === `${target.target_type}:${target.target_id}`,
|
||||||
);
|
);
|
||||||
if (!selectedSkills.length || !selectedTargets.length) {
|
if (!selectedSkills.length || !selectedTargets.length) {
|
||||||
toast.error("请至少选择一个技能和一个目标知识库");
|
toast.error("请至少选择一个技能和一个目标知识库");
|
||||||
@@ -108,46 +131,76 @@ export function SkillKnowledgePanel() {
|
|||||||
<div className="grid size-full min-h-0 grid-cols-[minmax(320px,0.9fr)_minmax(420px,1.1fr)] gap-4 overflow-hidden p-6">
|
<div className="grid size-full min-h-0 grid-cols-[minmax(320px,0.9fr)_minmax(420px,1.1fr)] gap-4 overflow-hidden p-6">
|
||||||
<section className="flex min-h-0 flex-col rounded-lg border bg-card">
|
<section className="flex min-h-0 flex-col rounded-lg border bg-card">
|
||||||
<div className="border-b p-4">
|
<div className="border-b p-4">
|
||||||
<h2 className="font-semibold">批量归纳</h2>
|
<h2 className="font-semibold">技能到知识库管理</h2>
|
||||||
<p className="mt-1 text-xs text-muted-foreground">按“技能 × 目标知识库”生成独立后台任务;代码与脚本不会被读取。</p>
|
<p className="mt-1 text-xs text-muted-foreground">一次选择多个技能归纳到一个 WeKnora Wiki 知识库;代码与脚本不会被读取。</p>
|
||||||
</div>
|
</div>
|
||||||
<ScrollArea className="min-h-0 flex-1">
|
<ScrollArea className="min-h-0 flex-1">
|
||||||
<div className="space-y-5 p-4">
|
<div className="space-y-5 p-4">
|
||||||
<div>
|
<div>
|
||||||
<div className="mb-2 flex items-center justify-between">
|
<div className="mb-2 flex items-center justify-between">
|
||||||
<span className="text-sm font-medium">1. 选择技能({selectedSkills.length})</span>
|
<span className="text-sm font-medium">1. 选择技能({selectedSkills.length})</span>
|
||||||
<Button variant="ghost" size="sm" onClick={() => setSelectedSkills(selectedSkills.length === liveSkills.length ? [] : liveSkills.map((skill) => skill.name))}>全选/清空</Button>
|
<Button variant="ghost" size="sm" onClick={toggleFilteredSkills}>
|
||||||
|
{allFilteredSelected ? "清空当前结果" : "选择当前结果"}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
<div className="relative mb-2">
|
||||||
|
<Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
|
||||||
|
<Input
|
||||||
|
value={skillQuery}
|
||||||
|
onChange={(event) => setSkillQuery(event.target.value)}
|
||||||
|
placeholder="按技能中文名、英文名、描述、分类或标签搜索"
|
||||||
|
className="pl-8"
|
||||||
|
/>
|
||||||
</div>
|
</div>
|
||||||
<div className="max-h-60 space-y-1 overflow-y-auto rounded-md border p-2">
|
<div className="max-h-60 space-y-1 overflow-y-auto rounded-md border p-2">
|
||||||
{isLoading ? <Loader2 className="m-4 size-5 animate-spin" /> : liveSkills.map((skill) => (
|
{isLoading ? <Loader2 className="m-4 size-5 animate-spin" /> : filteredSkills.map((skill) => (
|
||||||
<label key={skill.name} className="flex cursor-pointer items-center gap-2 rounded px-2 py-1.5 hover:bg-muted">
|
<label key={skill.name} className="flex cursor-pointer items-center gap-2 rounded px-2 py-1.5 hover:bg-muted">
|
||||||
<Checkbox checked={selectedSkills.includes(skill.name)} onCheckedChange={() => toggle(selectedSkills, skill.name, setSelectedSkills)} />
|
<Checkbox checked={selectedSkills.includes(skill.name)} onCheckedChange={() => toggle(selectedSkills, skill.name, setSelectedSkills)} />
|
||||||
<span className="truncate text-sm">{skill.name_zh || skill.name}</span>
|
<span className="truncate text-sm">{skill.name_zh || skill.name}</span>
|
||||||
<span className="ml-auto truncate text-xs text-muted-foreground">{skill.name}</span>
|
<span className="ml-auto truncate text-xs text-muted-foreground">{skill.name}</span>
|
||||||
</label>
|
</label>
|
||||||
))}
|
))}
|
||||||
|
{!isLoading && !filteredSkills.length && (
|
||||||
|
<p className="p-3 text-xs text-muted-foreground">没有匹配的技能,请换个关键词。</p>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
<p className="mt-1 text-xs text-muted-foreground">
|
||||||
|
当前显示 {filteredSkills.length}/{liveSkills.length} 个技能。
|
||||||
|
</p>
|
||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<div className="mb-2 flex items-center justify-between">
|
<div className="mb-2 flex items-center justify-between">
|
||||||
<span className="text-sm font-medium">2. 选择目标({selectedTargetKeys.length})</span>
|
<span className="text-sm font-medium">2. 选择目标知识库({selectedTargetKey ? 1 : 0})</span>
|
||||||
{!assistantBase.data && (
|
|
||||||
<Button size="sm" variant="outline" disabled={assistantActions.initialize.isPending} onClick={() => assistantActions.initialize.mutate("知识梳理")}>创建知识梳理库</Button>
|
|
||||||
)}
|
|
||||||
</div>
|
</div>
|
||||||
<div className="space-y-1 rounded-md border p-2">
|
<div className="space-y-1 rounded-md border p-2">
|
||||||
{targets.map((target) => {
|
{targets.map((target) => {
|
||||||
const key = `${target.target_type}:${target.target_id}`;
|
const key = `${target.target_type}:${target.target_id}`;
|
||||||
return (
|
return (
|
||||||
<label key={key} className="flex cursor-pointer items-center gap-2 rounded px-2 py-2 hover:bg-muted">
|
<label key={key} className="flex cursor-pointer items-center gap-2 rounded px-2 py-2 hover:bg-muted">
|
||||||
<Checkbox checked={selectedTargetKeys.includes(key)} onCheckedChange={() => toggle(selectedTargetKeys, key, setSelectedTargetKeys)} />
|
<Checkbox
|
||||||
|
checked={selectedTargetKey === key}
|
||||||
|
onCheckedChange={() => setSelectedTargetKey((current) => (current === key ? null : key))}
|
||||||
|
/>
|
||||||
<Database className="size-4 text-muted-foreground" />
|
<Database className="size-4 text-muted-foreground" />
|
||||||
<span className="text-sm">{target.target_name}</span>
|
<span className="text-sm">{target.target_name}</span>
|
||||||
<Badge variant="outline" className="ml-auto text-[10px]">{target.target_type === "assistant" ? "助手库" : target.target_mode === "readonly_sidecar" ? "只读·本地投影" : target.target_mode}</Badge>
|
<Badge variant="outline" className="ml-auto text-[10px]">WeKnora Wiki</Badge>
|
||||||
</label>
|
</label>
|
||||||
);
|
);
|
||||||
})}
|
})}
|
||||||
{!targets.length && <p className="p-3 text-xs text-muted-foreground">未配置普通知识库;可先创建全局“知识梳理”助手库。</p>}
|
{(runtime.isLoading || knowledgeBases.isLoading) && <Loader2 className="m-4 size-5 animate-spin" />}
|
||||||
|
{!runtime.isLoading && !knowledgeBases.isLoading && knowledgeBases.error && (
|
||||||
|
<p className="p-3 text-xs text-destructive">
|
||||||
|
读取 WeKnora 普通知识库失败:
|
||||||
|
{knowledgeBases.error instanceof Error ? knowledgeBases.error.message : String(knowledgeBases.error)}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
{!runtime.isLoading && !knowledgeBases.isLoading && !knowledgeBases.error && !targets.length && (
|
||||||
|
<p className="p-3 text-xs text-muted-foreground">
|
||||||
|
{runtime.data?.provider !== "weknora"
|
||||||
|
? "当前未配置 WeKnora 普通知识库地址;技能需要先归纳到普通知识库,再导入助手知识库。"
|
||||||
|
: "当前没有可写且已开启 Wiki 的普通知识库;请确认该库不是 FAQ/会话沉淀库,并已开启 Wiki。"}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div className="flex items-center gap-3">
|
<div className="flex items-center gap-3">
|
||||||
@@ -156,14 +209,13 @@ export function SkillKnowledgePanel() {
|
|||||||
<SelectTrigger className="w-52"><SelectValue /></SelectTrigger>
|
<SelectTrigger className="w-52"><SelectValue /></SelectTrigger>
|
||||||
<SelectContent>
|
<SelectContent>
|
||||||
<SelectItem value="auto_high_confidence">高置信自动通过</SelectItem>
|
<SelectItem value="auto_high_confidence">高置信自动通过</SelectItem>
|
||||||
<SelectItem value="required">全部逐条审核</SelectItem>
|
|
||||||
<SelectItem value="off">不阻塞写入</SelectItem>
|
<SelectItem value="off">不阻塞写入</SelectItem>
|
||||||
</SelectContent>
|
</SelectContent>
|
||||||
</Select>
|
</Select>
|
||||||
</div>
|
</div>
|
||||||
<Button className="w-full" disabled={actions.createJob.isPending} onClick={() => void submit()}>
|
<Button className="w-full" disabled={actions.createJob.isPending} onClick={() => void submit()}>
|
||||||
{actions.createJob.isPending ? <Loader2 className="animate-spin" /> : <Sparkles />}
|
{actions.createJob.isPending ? <Loader2 className="animate-spin" /> : <Sparkles />}
|
||||||
开始归纳({selectedSkills.length * selectedTargetKeys.length} 项)
|
开始归纳({selectedSkills.length * (selectedTargetKey ? 1 : 0)} 项)
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
</ScrollArea>
|
</ScrollArea>
|
||||||
@@ -199,26 +251,9 @@ export function SkillKnowledgePanel() {
|
|||||||
</div>
|
</div>
|
||||||
</ScrollArea>
|
</ScrollArea>
|
||||||
</div>
|
</div>
|
||||||
{(reviews.data?.length ?? 0) > 0 && (
|
|
||||||
<div className="max-h-64 overflow-auto rounded-lg border bg-card p-4">
|
|
||||||
<div className="mb-3 flex items-center"><h2 className="font-semibold">待审核({reviews.data!.length})</h2><Button size="sm" className="ml-auto" onClick={() => actions.review.mutate({ review_ids: reviews.data!.map((row) => row.id), action: "approve" })}><CheckCircle2 />批量批准</Button></div>
|
|
||||||
{reviews.data!.map((review) => <div key={review.id} className="mb-2 flex items-center gap-2 rounded border p-2 text-xs"><Badge variant="outline">{review.object_type}</Badge><code className="truncate">{String(review.original.name || review.original.predicate || review.object_key)}</code><Button size="sm" variant="ghost" className="ml-auto" onClick={() => actions.review.mutate({ review_ids: [review.id], action: "approve" })}>批准</Button><Button size="sm" variant="ghost" onClick={() => actions.review.mutate({ review_ids: [review.id], action: "reject" })}>拒绝</Button></div>)}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
<div className="max-h-80 overflow-auto rounded-lg border bg-card p-3 text-xs">
|
<div className="max-h-80 overflow-auto rounded-lg border bg-card p-3 text-xs">
|
||||||
<div className="mb-3 flex flex-wrap items-center gap-3 border-b pb-3">
|
|
||||||
<span className="font-medium">自动同步</span>
|
|
||||||
{autoSettings.data && <>
|
|
||||||
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.watch_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ watch_enabled: checked === true })} />文件监听</label>
|
|
||||||
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.schedule_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ schedule_enabled: checked === true })} />定时同步</label>
|
|
||||||
<label className="flex items-center gap-1"><Checkbox checked={autoSettings.data.write_remote_enabled} onCheckedChange={(checked) => actions.updateAutoSyncSettings.mutate({ write_remote_enabled: checked === true })} />自动写入目标</label>
|
|
||||||
<Input className="h-8 w-32" value={scheduleCron} onChange={(event) => setScheduleCron(event.target.value)} aria-label="自动同步 cron" />
|
|
||||||
<Input className="h-8 w-24" type="number" min={10} value={debounceSeconds} onChange={(event) => setDebounceSeconds(Number(event.target.value))} aria-label="防抖秒数" />
|
|
||||||
<Button size="sm" variant="outline" onClick={() => actions.updateAutoSyncSettings.mutate({ schedule_cron: scheduleCron, debounce_seconds: debounceSeconds }, { onSuccess: () => toast.success("自动同步设置已保存"), onError: (error) => toast.error(error.message) })}>保存</Button>
|
|
||||||
</>}
|
|
||||||
</div>
|
|
||||||
<div className="mb-2 text-muted-foreground">当前有效绑定:{bindings.data?.length ?? 0};同名同类型实体按来源贡献合并。</div>
|
<div className="mb-2 text-muted-foreground">当前有效绑定:{bindings.data?.length ?? 0};同名同类型实体按来源贡献合并。</div>
|
||||||
<div className="space-y-2">{bindings.data?.map((binding) => <div key={binding.id} className="flex flex-wrap items-center gap-2 rounded border p-2"><span className="max-w-32 truncate font-medium">{binding.skill_name}</span><span className="max-w-40 truncate text-muted-foreground">→ {binding.target_name}</span><Badge variant="outline">{binding.status}</Badge><code className="max-w-40 truncate text-[10px] text-muted-foreground">{binding.remote_business_key}</code><label className="ml-auto flex items-center gap-1"><Checkbox checked={binding.auto_sync_enabled} onCheckedChange={(checked) => actions.setAutoSync.mutate({ id: binding.id, enabled: checked === true })} />自动</label><Button size="sm" variant="outline" disabled={actions.resync.isPending} onClick={() => actions.resync.mutate(binding.id)}>重归纳</Button><Button size="sm" variant="ghost" onClick={() => { if (window.confirm("解除关联并保留目标知识内容?")) actions.detach.mutate({ id: binding.id, deleteContributions: false }); }}>解绑并保留</Button><Button size="sm" variant="destructive" onClick={() => { if (window.confirm("解除关联并删除此技能贡献?共享知识仍会保留其他来源。")) actions.detach.mutate({ id: binding.id, deleteContributions: true }); }}>解绑并删除贡献</Button></div>)}</div>
|
<div className="space-y-2">{bindings.data?.map((binding) => <div key={binding.id} className="flex flex-wrap items-center gap-2 rounded border p-2"><span className="max-w-32 truncate font-medium">{binding.skill_name}</span><span className="max-w-40 truncate text-muted-foreground">→ {binding.target_name}</span><Badge variant="outline">{binding.status}</Badge><code className="max-w-40 truncate text-[10px] text-muted-foreground">{binding.remote_business_key}</code><Button size="sm" variant="outline" className="ml-auto" disabled={actions.resync.isPending} onClick={() => actions.resync.mutate(binding.id)}>重归纳</Button><Button size="sm" variant="ghost" onClick={() => { if (window.confirm("解除关联并保留目标知识内容?")) actions.detach.mutate({ id: binding.id, deleteContributions: false }); }}>解绑并保留</Button><Button size="sm" variant="destructive" onClick={() => { if (window.confirm("解除关联并删除此技能贡献?共享知识仍会保留其他来源。")) actions.detach.mutate({ id: binding.id, deleteContributions: true }); }}>解绑并删除贡献</Button></div>)}</div>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import {
|
|||||||
SidebarMenuItem,
|
SidebarMenuItem,
|
||||||
} from "@/components/ui/sidebar";
|
} from "@/components/ui/sidebar";
|
||||||
import { useI18n } from "@/core/i18n/hooks";
|
import { useI18n } from "@/core/i18n/hooks";
|
||||||
import { BookOpen, BotIcon, BrainIcon, FileSearch, ListChecks, MessageSquarePlus, MessagesSquare, PenLine, Plus, SparklesIcon, Telescope, Users } from "lucide-react";
|
import { BookOpen, BotIcon, BrainIcon, FileSearch, Library, ListChecks, MessageSquarePlus, MessagesSquare, PenLine, Plus, SparklesIcon, Telescope, Users } from "lucide-react";
|
||||||
|
|
||||||
import { KnowledgeRagToggle } from "./knowledge-rag-toggle";
|
import { KnowledgeRagToggle } from "./knowledge-rag-toggle";
|
||||||
import { SidebarCollapseButton } from "./workspace-header";
|
import { SidebarCollapseButton } from "./workspace-header";
|
||||||
@@ -65,6 +65,13 @@ export const WORKSPACE_COMPACT_NAV_ITEMS: WorkspaceCompactNavItem[] = [
|
|||||||
path: "/page/workspace/chats",
|
path: "/page/workspace/chats",
|
||||||
match: (pathname) => pathname === "/page/workspace/chats",
|
match: (pathname) => pathname === "/page/workspace/chats",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
id: "workspace-library",
|
||||||
|
label: "资料库",
|
||||||
|
icon: Library,
|
||||||
|
path: "/page/workspace/library",
|
||||||
|
match: (pathname) => pathname === "/page/workspace/library",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
id: "workspace-agents",
|
id: "workspace-agents",
|
||||||
label: "智能体",
|
label: "智能体",
|
||||||
|
|||||||
@@ -229,6 +229,14 @@ export function WorkspaceNavMenu() {
|
|||||||
技能管理
|
技能管理
|
||||||
</DropdownMenuItem>
|
</DropdownMenuItem>
|
||||||
)}
|
)}
|
||||||
|
{isAdmin && (
|
||||||
|
<DropdownMenuItem
|
||||||
|
onClick={() => navigate("/page/workspace/admin/skills?tab=knowledge")}
|
||||||
|
>
|
||||||
|
<DatabaseZapIcon />
|
||||||
|
技能到知识库管理
|
||||||
|
</DropdownMenuItem>
|
||||||
|
)}
|
||||||
{isAdmin && (
|
{isAdmin && (
|
||||||
<DropdownMenuItem
|
<DropdownMenuItem
|
||||||
onClick={() => navigate("/page/workspace/admin/models")}
|
onClick={() => navigate("/page/workspace/admin/models")}
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import { apiFetch } from "../api";
|
import { apiFetch } from "../api";
|
||||||
import { getFileName } from "../utils/files";
|
import { downloadMarkdownAsDocx } from "../scheduled-tasks/markdown-docx";
|
||||||
|
import { getFileName } from "../utils/files";
|
||||||
|
|
||||||
import { urlOfArtifact } from "./utils";
|
import { urlOfArtifact } from "./utils";
|
||||||
|
|
||||||
@@ -39,7 +40,7 @@ async function errorMessageFromResponse(res: Response): Promise<string> {
|
|||||||
return res.statusText || "Request failed";
|
return res.statusText || "Request failed";
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function downloadArtifactFile({
|
export async function downloadArtifactFile({
|
||||||
filepath,
|
filepath,
|
||||||
threadId,
|
threadId,
|
||||||
isMock,
|
isMock,
|
||||||
@@ -73,8 +74,26 @@ export async function downloadArtifactFile({
|
|||||||
document.body.appendChild(a);
|
document.body.appendChild(a);
|
||||||
a.click();
|
a.click();
|
||||||
a.remove();
|
a.remove();
|
||||||
window.setTimeout(() => URL.revokeObjectURL(objectUrl), 30_000);
|
window.setTimeout(() => URL.revokeObjectURL(objectUrl), 30_000);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export async function downloadArtifactMarkdownAsWord({
|
||||||
|
filepath,
|
||||||
|
threadId,
|
||||||
|
}: {
|
||||||
|
filepath: string;
|
||||||
|
threadId: string;
|
||||||
|
}): Promise<void> {
|
||||||
|
const url = urlOfArtifact({ filepath, threadId, download: false });
|
||||||
|
const res = await apiFetch(url);
|
||||||
|
if (!res.ok) {
|
||||||
|
throw new Error(await errorMessageFromResponse(res));
|
||||||
|
}
|
||||||
|
|
||||||
|
const fileName = getFileName(filepath);
|
||||||
|
const title = fileName.replace(/\.(md|markdown)$/i, "");
|
||||||
|
await downloadMarkdownAsDocx(await res.text(), fileName, title);
|
||||||
|
}
|
||||||
|
|
||||||
export async function openArtifactInNewTab({
|
export async function openArtifactInNewTab({
|
||||||
filepath,
|
filepath,
|
||||||
|
|||||||
@@ -9,11 +9,15 @@ export type ArtifactLibraryKind =
|
|||||||
| "archive"
|
| "archive"
|
||||||
| "other";
|
| "other";
|
||||||
|
|
||||||
|
export type ArtifactLibrarySource = "all" | "general_qa" | "agent";
|
||||||
|
|
||||||
export interface ArtifactLibraryItem {
|
export interface ArtifactLibraryItem {
|
||||||
threadId: string;
|
threadId: string;
|
||||||
threadTitle: string;
|
threadTitle: string;
|
||||||
agentId: string | null;
|
agentId: string | null;
|
||||||
|
agentName: string | null;
|
||||||
threadType: string | null;
|
threadType: string | null;
|
||||||
|
sourceType: Exclude<ArtifactLibrarySource, "all">;
|
||||||
path: string;
|
path: string;
|
||||||
name: string;
|
name: string;
|
||||||
sizeBytes: number;
|
sizeBytes: number;
|
||||||
@@ -32,6 +36,7 @@ export interface ArtifactLibraryResponse {
|
|||||||
export async function fetchArtifactLibrary({
|
export async function fetchArtifactLibrary({
|
||||||
query,
|
query,
|
||||||
fileKind,
|
fileKind,
|
||||||
|
sourceType,
|
||||||
offset,
|
offset,
|
||||||
limit,
|
limit,
|
||||||
refresh = false,
|
refresh = false,
|
||||||
@@ -39,6 +44,7 @@ export async function fetchArtifactLibrary({
|
|||||||
}: {
|
}: {
|
||||||
query?: string;
|
query?: string;
|
||||||
fileKind: ArtifactLibraryKind;
|
fileKind: ArtifactLibraryKind;
|
||||||
|
sourceType: ArtifactLibrarySource;
|
||||||
offset: number;
|
offset: number;
|
||||||
limit: number;
|
limit: number;
|
||||||
refresh?: boolean;
|
refresh?: boolean;
|
||||||
@@ -46,6 +52,7 @@ export async function fetchArtifactLibrary({
|
|||||||
}): Promise<ArtifactLibraryResponse> {
|
}): Promise<ArtifactLibraryResponse> {
|
||||||
const search = new URLSearchParams({
|
const search = new URLSearchParams({
|
||||||
file_kind: fileKind,
|
file_kind: fileKind,
|
||||||
|
source_type: sourceType,
|
||||||
offset: String(offset),
|
offset: String(offset),
|
||||||
limit: String(limit),
|
limit: String(limit),
|
||||||
});
|
});
|
||||||
@@ -72,7 +79,9 @@ export async function fetchArtifactLibrary({
|
|||||||
thread_id: string;
|
thread_id: string;
|
||||||
thread_title: string;
|
thread_title: string;
|
||||||
agent_id?: string | null;
|
agent_id?: string | null;
|
||||||
|
agent_name?: string | null;
|
||||||
thread_type?: string | null;
|
thread_type?: string | null;
|
||||||
|
source_type?: Exclude<ArtifactLibrarySource, "all">;
|
||||||
path: string;
|
path: string;
|
||||||
name: string;
|
name: string;
|
||||||
size_bytes: number;
|
size_bytes: number;
|
||||||
@@ -90,7 +99,9 @@ export async function fetchArtifactLibrary({
|
|||||||
threadId: item.thread_id,
|
threadId: item.thread_id,
|
||||||
threadTitle: item.thread_title,
|
threadTitle: item.thread_title,
|
||||||
agentId: item.agent_id ?? null,
|
agentId: item.agent_id ?? null,
|
||||||
|
agentName: item.agent_name ?? item.agent_id ?? null,
|
||||||
threadType: item.thread_type ?? null,
|
threadType: item.thread_type ?? null,
|
||||||
|
sourceType: item.source_type ?? (item.agent_id ? "agent" : "general_qa"),
|
||||||
path: item.path,
|
path: item.path,
|
||||||
name: item.name,
|
name: item.name,
|
||||||
sizeBytes: item.size_bytes,
|
sizeBytes: item.size_bytes,
|
||||||
|
|||||||
@@ -1293,15 +1293,21 @@ export function useThreadStream({
|
|||||||
...context,
|
...context,
|
||||||
...extraContext,
|
...extraContext,
|
||||||
};
|
};
|
||||||
const selectedLlmWikiIds = Array.isArray(
|
const selectedLlmWikiIds = Array.isArray(
|
||||||
mergedContext.llmwiki_knowledge_base_ids,
|
mergedContext.llmwiki_knowledge_base_ids,
|
||||||
)
|
)
|
||||||
? mergedContext.llmwiki_knowledge_base_ids
|
? mergedContext.llmwiki_knowledge_base_ids
|
||||||
: [];
|
: [];
|
||||||
const ragModeEnabled =
|
const selectedAssistantKnowledgeIds = Array.isArray(
|
||||||
localSettings.display.ragModeEnabled ||
|
mergedContext.assistant_knowledge_base_ids,
|
||||||
isThreadRagModeLocked(threadId) ||
|
)
|
||||||
selectedLlmWikiIds.length > 0;
|
? mergedContext.assistant_knowledge_base_ids
|
||||||
|
: [];
|
||||||
|
const ragModeEnabled =
|
||||||
|
localSettings.display.ragModeEnabled ||
|
||||||
|
isThreadRagModeLocked(threadId) ||
|
||||||
|
selectedLlmWikiIds.length > 0 ||
|
||||||
|
selectedAssistantKnowledgeIds.length > 0;
|
||||||
if (ragModeEnabled) {
|
if (ragModeEnabled) {
|
||||||
lockThreadRagMode(threadId);
|
lockThreadRagMode(threadId);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -27,6 +27,8 @@ export interface AgentThreadContext extends Record<string, unknown> {
|
|||||||
knowledge_rag_enabled?: boolean;
|
knowledge_rag_enabled?: boolean;
|
||||||
/** DeerFlow LLMWiki mapping ids selected for this conversation. */
|
/** DeerFlow LLMWiki mapping ids selected for this conversation. */
|
||||||
llmwiki_knowledge_base_ids?: string[];
|
llmwiki_knowledge_base_ids?: string[];
|
||||||
|
/** Assistant knowledge-base ids selected for this conversation. */
|
||||||
|
assistant_knowledge_base_ids?: string[];
|
||||||
/** Direct RAG modes handled in the frontend; these requests never enter Lead Agent. */
|
/** Direct RAG modes handled in the frontend; these requests never enter Lead Agent. */
|
||||||
execution_profile?: "single_round_rag" | "multi_round_rag";
|
execution_profile?: "single_round_rag" | "multi_round_rag";
|
||||||
agent_id?: string;
|
agent_id?: string;
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ import {
|
|||||||
SparklesIcon,
|
SparklesIcon,
|
||||||
} from "lucide-react";
|
} from "lucide-react";
|
||||||
import { useEffect, useMemo, useState } from "react";
|
import { useEffect, useMemo, useState } from "react";
|
||||||
|
import { useSearchParams } from "react-router-dom";
|
||||||
|
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
@@ -29,10 +30,22 @@ import { SkillAddressesPanel } from "./SkillAddressesPage";
|
|||||||
const PAGE_SIZE = 10;
|
const PAGE_SIZE = 10;
|
||||||
|
|
||||||
export default function AdminSkillsPage() {
|
export default function AdminSkillsPage() {
|
||||||
|
const [searchParams, setSearchParams] = useSearchParams();
|
||||||
const [search, setSearch] = useState("");
|
const [search, setSearch] = useState("");
|
||||||
const [page, setPage] = useState(1);
|
const [page, setPage] = useState(1);
|
||||||
const { skills, isLoading, error } = useAllSkillsAdmin(true);
|
const { skills, isLoading, error } = useAllSkillsAdmin(true);
|
||||||
const knowledgeBindings = useSkillKnowledgeBindings();
|
const knowledgeBindings = useSkillKnowledgeBindings();
|
||||||
|
const activeTab = useMemo(() => {
|
||||||
|
const tab = searchParams.get("tab");
|
||||||
|
return tab === "addresses" || tab === "knowledge" ? tab : "skills";
|
||||||
|
}, [searchParams]);
|
||||||
|
|
||||||
|
const changeTab = (value: string) => {
|
||||||
|
const next = new URLSearchParams(searchParams);
|
||||||
|
if (value === "skills") next.delete("tab");
|
||||||
|
else next.set("tab", value);
|
||||||
|
setSearchParams(next, { replace: true });
|
||||||
|
};
|
||||||
|
|
||||||
const storedAuth = useMemo(() => getStoredAuth(), []);
|
const storedAuth = useMemo(() => getStoredAuth(), []);
|
||||||
const currentUserId = storedAuth?.user_id ?? null;
|
const currentUserId = storedAuth?.user_id ?? null;
|
||||||
@@ -60,7 +73,8 @@ export default function AdminSkillsPage() {
|
|||||||
<WorkspaceHeader />
|
<WorkspaceHeader />
|
||||||
<WorkspaceBody>
|
<WorkspaceBody>
|
||||||
<Tabs
|
<Tabs
|
||||||
defaultValue="skills"
|
value={activeTab}
|
||||||
|
onValueChange={changeTab}
|
||||||
className="flex size-full min-h-0 flex-col gap-0"
|
className="flex size-full min-h-0 flex-col gap-0"
|
||||||
>
|
>
|
||||||
<div className="flex items-center justify-between gap-4 border-b px-6 pt-4">
|
<div className="flex items-center justify-between gap-4 border-b px-6 pt-4">
|
||||||
@@ -77,7 +91,7 @@ export default function AdminSkillsPage() {
|
|||||||
</TabsTrigger>
|
</TabsTrigger>
|
||||||
<TabsTrigger value="knowledge">
|
<TabsTrigger value="knowledge">
|
||||||
<DatabaseZap />
|
<DatabaseZap />
|
||||||
技能知识归纳
|
技能到知识库管理
|
||||||
</TabsTrigger>
|
</TabsTrigger>
|
||||||
</TabsList>
|
</TabsList>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -92,6 +92,26 @@ function readPendingKnowledgeSelection(agentId: string | undefined): string[] |
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function readPendingAssistantKnowledgeSelection(agentId: string | undefined): string[] | null {
|
||||||
|
if (typeof window === "undefined" || !agentId) return null;
|
||||||
|
try {
|
||||||
|
const raw = sessionStorage.getItem("pendingAgentMessage");
|
||||||
|
if (!raw) return null;
|
||||||
|
const parsed = JSON.parse(raw) as {
|
||||||
|
agentId?: string;
|
||||||
|
assistantKnowledgeBaseIds?: unknown;
|
||||||
|
};
|
||||||
|
if (parsed.agentId !== agentId || !Array.isArray(parsed.assistantKnowledgeBaseIds)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return parsed.assistantKnowledgeBaseIds
|
||||||
|
.map((item) => String(item).trim())
|
||||||
|
.filter(Boolean);
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
function stringifyForHiddenContext(value: unknown): string {
|
function stringifyForHiddenContext(value: unknown): string {
|
||||||
if (typeof value === "string") return value;
|
if (typeof value === "string") return value;
|
||||||
try {
|
try {
|
||||||
@@ -270,11 +290,17 @@ export default function AgentChatPage({
|
|||||||
const pendingKnowledgeSelection = useRef(
|
const pendingKnowledgeSelection = useRef(
|
||||||
readPendingKnowledgeSelection(agent_id),
|
readPendingKnowledgeSelection(agent_id),
|
||||||
);
|
);
|
||||||
|
const pendingAssistantKnowledgeSelection = useRef(
|
||||||
|
readPendingAssistantKnowledgeSelection(agent_id),
|
||||||
|
);
|
||||||
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>(
|
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>(
|
||||||
() => pendingKnowledgeSelection.current ?? [],
|
() => pendingKnowledgeSelection.current ?? [],
|
||||||
);
|
);
|
||||||
|
const [selectedAssistantKnowledgeBaseIds, setSelectedAssistantKnowledgeBaseIds] = useState<string[]>(
|
||||||
|
() => pendingAssistantKnowledgeSelection.current ?? [],
|
||||||
|
);
|
||||||
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(
|
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(
|
||||||
() => pendingKnowledgeSelection.current !== null,
|
() => pendingKnowledgeSelection.current !== null || pendingAssistantKnowledgeSelection.current !== null,
|
||||||
);
|
);
|
||||||
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
|
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
|
||||||
const knowledgeSelectionThread = useRef<string | null>(null);
|
const knowledgeSelectionThread = useRef<string | null>(null);
|
||||||
@@ -285,6 +311,10 @@ export default function AgentChatPage({
|
|||||||
setSelectedKnowledgeBaseIds(ids);
|
setSelectedKnowledgeBaseIds(ids);
|
||||||
setKnowledgeSelectionReady(true);
|
setKnowledgeSelectionReady(true);
|
||||||
}, []);
|
}, []);
|
||||||
|
const changeAssistantKnowledgeSelection = useCallback((ids: string[]) => {
|
||||||
|
setSelectedAssistantKnowledgeBaseIds(ids);
|
||||||
|
setKnowledgeSelectionReady(true);
|
||||||
|
}, []);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
activeThreadIdRef.current = threadId;
|
activeThreadIdRef.current = threadId;
|
||||||
@@ -294,17 +324,23 @@ export default function AgentChatPage({
|
|||||||
if (knowledgeSelectionAgent.current === agent_id) return;
|
if (knowledgeSelectionAgent.current === agent_id) return;
|
||||||
knowledgeSelectionAgent.current = agent_id;
|
knowledgeSelectionAgent.current = agent_id;
|
||||||
const pending = readPendingKnowledgeSelection(agent_id);
|
const pending = readPendingKnowledgeSelection(agent_id);
|
||||||
|
const pendingAssistant = readPendingAssistantKnowledgeSelection(agent_id);
|
||||||
pendingKnowledgeSelection.current = pending;
|
pendingKnowledgeSelection.current = pending;
|
||||||
|
pendingAssistantKnowledgeSelection.current = pendingAssistant;
|
||||||
knowledgeSelectionThread.current = null;
|
knowledgeSelectionThread.current = null;
|
||||||
setSelectedKnowledgeBaseIds(pending ?? []);
|
setSelectedKnowledgeBaseIds(pending ?? []);
|
||||||
setKnowledgeSelectionReady(pending !== null);
|
setSelectedAssistantKnowledgeBaseIds(pendingAssistant ?? []);
|
||||||
|
setKnowledgeSelectionReady(pending !== null || pendingAssistant !== null);
|
||||||
}, [agent_id]);
|
}, [agent_id]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (isNewThread) {
|
if (isNewThread) {
|
||||||
if (knowledgeSelectionThread.current !== "__new__") {
|
if (knowledgeSelectionThread.current !== "__new__") {
|
||||||
setSelectedKnowledgeBaseIds(pendingKnowledgeSelection.current ?? []);
|
setSelectedKnowledgeBaseIds(pendingKnowledgeSelection.current ?? []);
|
||||||
setKnowledgeSelectionReady(pendingKnowledgeSelection.current !== null);
|
setSelectedAssistantKnowledgeBaseIds(pendingAssistantKnowledgeSelection.current ?? []);
|
||||||
|
setKnowledgeSelectionReady(
|
||||||
|
pendingKnowledgeSelection.current !== null || pendingAssistantKnowledgeSelection.current !== null,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
knowledgeSelectionThread.current = "__new__";
|
knowledgeSelectionThread.current = "__new__";
|
||||||
setKnowledgeSelectionLoading(false);
|
setKnowledgeSelectionLoading(false);
|
||||||
@@ -319,18 +355,29 @@ export default function AgentChatPage({
|
|||||||
if (knowledgeSelectionThread.current !== threadId) return;
|
if (knowledgeSelectionThread.current !== threadId) return;
|
||||||
const metadata = record.metadata as Record<string, unknown> | undefined;
|
const metadata = record.metadata as Record<string, unknown> | undefined;
|
||||||
const raw = metadata?.llmwiki_knowledge_base_ids;
|
const raw = metadata?.llmwiki_knowledge_base_ids;
|
||||||
|
const assistantRaw = metadata?.assistant_knowledge_base_ids;
|
||||||
|
const llmwikiIds = Array.isArray(raw)
|
||||||
|
? raw.map((item) => String(item).trim()).filter(Boolean)
|
||||||
|
: [];
|
||||||
|
const assistantIds = Array.isArray(assistantRaw)
|
||||||
|
? assistantRaw.map((item) => String(item).trim()).filter(Boolean)
|
||||||
|
: [];
|
||||||
if (Array.isArray(raw)) {
|
if (Array.isArray(raw)) {
|
||||||
setSelectedKnowledgeBaseIds(
|
setSelectedKnowledgeBaseIds(llmwikiIds);
|
||||||
raw.map((item) => String(item).trim()).filter(Boolean),
|
|
||||||
);
|
|
||||||
setKnowledgeSelectionReady(true);
|
|
||||||
} else {
|
} else {
|
||||||
setSelectedKnowledgeBaseIds([]);
|
setSelectedKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
|
||||||
}
|
}
|
||||||
|
if (Array.isArray(assistantRaw)) {
|
||||||
|
setSelectedAssistantKnowledgeBaseIds(assistantIds);
|
||||||
|
} else {
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
|
}
|
||||||
|
setKnowledgeSelectionReady(Array.isArray(raw) || Array.isArray(assistantRaw));
|
||||||
})
|
})
|
||||||
.catch(() => {
|
.catch(() => {
|
||||||
if (knowledgeSelectionThread.current === threadId) {
|
if (knowledgeSelectionThread.current === threadId) {
|
||||||
|
setSelectedKnowledgeBaseIds([]);
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
setKnowledgeSelectionReady(false);
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -353,10 +400,13 @@ export default function AgentChatPage({
|
|||||||
() => ({
|
() => ({
|
||||||
...(taskId ? taskThreadMetadata(taskContext) : {}),
|
...(taskId ? taskThreadMetadata(taskContext) : {}),
|
||||||
...(knowledgeSelectionReady
|
...(knowledgeSelectionReady
|
||||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
? {
|
||||||
|
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||||
|
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||||
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}),
|
}),
|
||||||
[knowledgeSelectionReady, selectedKnowledgeBaseIds, taskContext, taskId],
|
[knowledgeSelectionReady, selectedAssistantKnowledgeBaseIds, selectedKnowledgeBaseIds, taskContext, taskId],
|
||||||
);
|
);
|
||||||
const [thread, sendMessage] = useThreadStream({
|
const [thread, sendMessage] = useThreadStream({
|
||||||
threadId: isNewThread ? undefined : threadId,
|
threadId: isNewThread ? undefined : threadId,
|
||||||
@@ -365,7 +415,10 @@ export default function AgentChatPage({
|
|||||||
...settings.context,
|
...settings.context,
|
||||||
agent_id,
|
agent_id,
|
||||||
...(knowledgeSelectionReady
|
...(knowledgeSelectionReady
|
||||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
? {
|
||||||
|
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||||
|
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||||
|
}
|
||||||
: {}),
|
: {}),
|
||||||
},
|
},
|
||||||
// 任务闸门:新建 thread 打 metadata.taskId 标签,供任务对话列表(1:N)按任务过滤;
|
// 任务闸门:新建 thread 打 metadata.taskId 标签,供任务对话列表(1:N)按任务过滤;
|
||||||
@@ -713,9 +766,13 @@ export default function AgentChatPage({
|
|||||||
source="agent"
|
source="agent"
|
||||||
agentId={agent_id}
|
agentId={agent_id}
|
||||||
value={selectedKnowledgeBaseIds}
|
value={selectedKnowledgeBaseIds}
|
||||||
|
assistantValue={selectedAssistantKnowledgeBaseIds}
|
||||||
onChange={changeKnowledgeSelection}
|
onChange={changeKnowledgeSelection}
|
||||||
|
onAssistantChange={changeAssistantKnowledgeSelection}
|
||||||
autoSelectAll={
|
autoSelectAll={
|
||||||
isNewThread && pendingKnowledgeSelection.current === null
|
isNewThread &&
|
||||||
|
pendingKnowledgeSelection.current === null &&
|
||||||
|
pendingAssistantKnowledgeSelection.current === null
|
||||||
}
|
}
|
||||||
disabled={isDemoMode || knowledgeSelectionLoading}
|
disabled={isDemoMode || knowledgeSelectionLoading}
|
||||||
label="知识空间"
|
label="知识空间"
|
||||||
@@ -781,7 +838,9 @@ export default function AgentChatPage({
|
|||||||
source="agent"
|
source="agent"
|
||||||
agentId={agent_id}
|
agentId={agent_id}
|
||||||
value={selectedKnowledgeBaseIds}
|
value={selectedKnowledgeBaseIds}
|
||||||
|
assistantValue={selectedAssistantKnowledgeBaseIds}
|
||||||
onChange={changeKnowledgeSelection}
|
onChange={changeKnowledgeSelection}
|
||||||
|
onAssistantChange={changeAssistantKnowledgeSelection}
|
||||||
disabled={isDemoMode || knowledgeSelectionLoading}
|
disabled={isDemoMode || knowledgeSelectionLoading}
|
||||||
readOnly
|
readOnly
|
||||||
label="知识空间"
|
label="知识空间"
|
||||||
|
|||||||
@@ -3,12 +3,15 @@
|
|||||||
import { useQuery } from "@tanstack/react-query";
|
import { useQuery } from "@tanstack/react-query";
|
||||||
import {
|
import {
|
||||||
ArrowDownWideNarrow,
|
ArrowDownWideNarrow,
|
||||||
|
Bot,
|
||||||
ChevronLeft,
|
ChevronLeft,
|
||||||
ChevronRight,
|
ChevronRight,
|
||||||
Download,
|
Download,
|
||||||
|
FileText,
|
||||||
FileX2,
|
FileX2,
|
||||||
Library,
|
Library,
|
||||||
Loader2,
|
Loader2,
|
||||||
|
MessagesSquare,
|
||||||
MessageSquareShare,
|
MessageSquareShare,
|
||||||
RefreshCw,
|
RefreshCw,
|
||||||
Search,
|
Search,
|
||||||
@@ -20,17 +23,28 @@ import { toast } from "sonner";
|
|||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||||
|
import {
|
||||||
|
Select,
|
||||||
|
SelectContent,
|
||||||
|
SelectItem,
|
||||||
|
SelectTrigger,
|
||||||
|
SelectValue,
|
||||||
|
} from "@/components/ui/select";
|
||||||
import { Skeleton } from "@/components/ui/skeleton";
|
import { Skeleton } from "@/components/ui/skeleton";
|
||||||
import {
|
import {
|
||||||
WorkspaceBody,
|
WorkspaceBody,
|
||||||
WorkspaceContainer,
|
WorkspaceContainer,
|
||||||
WorkspaceHeader,
|
WorkspaceHeader,
|
||||||
} from "@/components/workspace/workspace-container";
|
} from "@/components/workspace/workspace-container";
|
||||||
import { downloadArtifactFile } from "@/core/artifacts/download";
|
import {
|
||||||
|
downloadArtifactFile,
|
||||||
|
downloadArtifactMarkdownAsWord,
|
||||||
|
} from "@/core/artifacts/download";
|
||||||
import {
|
import {
|
||||||
fetchArtifactLibrary,
|
fetchArtifactLibrary,
|
||||||
type ArtifactLibraryItem,
|
type ArtifactLibraryItem,
|
||||||
type ArtifactLibraryKind,
|
type ArtifactLibraryKind,
|
||||||
|
type ArtifactLibrarySource,
|
||||||
} from "@/core/artifacts/library";
|
} from "@/core/artifacts/library";
|
||||||
import { useI18n } from "@/core/i18n/hooks";
|
import { useI18n } from "@/core/i18n/hooks";
|
||||||
import { bootstrapResumePath, pathOfThread } from "@/core/threads/utils";
|
import { bootstrapResumePath, pathOfThread } from "@/core/threads/utils";
|
||||||
@@ -53,6 +67,28 @@ const FILTERS: Array<{ value: ArtifactLibraryKind; label: string }> = [
|
|||||||
{ value: "other", label: "其他" },
|
{ value: "other", label: "其他" },
|
||||||
];
|
];
|
||||||
|
|
||||||
|
const SOURCE_FILTERS: Array<{
|
||||||
|
value: ArtifactLibrarySource;
|
||||||
|
label: string;
|
||||||
|
}> = [
|
||||||
|
{ value: "all", label: "全部来源" },
|
||||||
|
{ value: "general_qa", label: "通用问答" },
|
||||||
|
{ value: "agent", label: "智能体" },
|
||||||
|
];
|
||||||
|
|
||||||
|
type DownloadFormat = "original" | "word";
|
||||||
|
|
||||||
|
function isMarkdownArtifact(name: string): boolean {
|
||||||
|
return /\.(md|markdown)$/i.test(name);
|
||||||
|
}
|
||||||
|
|
||||||
|
function artifactDownloadKey(
|
||||||
|
item: ArtifactLibraryItem,
|
||||||
|
format: DownloadFormat,
|
||||||
|
): string {
|
||||||
|
return `${item.threadId}:${item.path}:${format}`;
|
||||||
|
}
|
||||||
|
|
||||||
function formatBytes(bytes: number): string {
|
function formatBytes(bytes: number): string {
|
||||||
if (!Number.isFinite(bytes) || bytes <= 0) return "0 B";
|
if (!Number.isFinite(bytes) || bytes <= 0) return "0 B";
|
||||||
const units = ["B", "KB", "MB", "GB"];
|
const units = ["B", "KB", "MB", "GB"];
|
||||||
@@ -83,12 +119,19 @@ function FileRow({
|
|||||||
onDownload,
|
onDownload,
|
||||||
}: {
|
}: {
|
||||||
item: ArtifactLibraryItem;
|
item: ArtifactLibraryItem;
|
||||||
downloading: boolean;
|
downloading: DownloadFormat | null;
|
||||||
onDownload: (item: ArtifactLibraryItem) => void;
|
onDownload: (item: ArtifactLibraryItem, format: DownloadFormat) => void;
|
||||||
}) {
|
}) {
|
||||||
const sourcePath = sourceConversationPath(item);
|
const sourcePath = sourceConversationPath(item);
|
||||||
const displayName = desensitize(item.name);
|
const displayName = desensitize(item.name);
|
||||||
const sourceTitle = desensitize(item.threadTitle || "新对话");
|
const sourceTitle = desensitize(item.threadTitle || "新对话");
|
||||||
|
const sourceLabel = item.sourceType === "agent" ? "智能体" : "通用问答";
|
||||||
|
const sourceName =
|
||||||
|
item.sourceType === "agent"
|
||||||
|
? desensitize(item.agentName || item.agentId || "未知智能体")
|
||||||
|
: sourceLabel;
|
||||||
|
const SourceIcon = item.sourceType === "agent" ? Bot : MessagesSquare;
|
||||||
|
const busy = downloading !== null;
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<tr className="group border-b transition-colors hover:bg-muted/35">
|
<tr className="group border-b transition-colors hover:bg-muted/35">
|
||||||
@@ -106,7 +149,8 @@ function FileRow({
|
|||||||
{displayName}
|
{displayName}
|
||||||
</span>
|
</span>
|
||||||
<span className="mt-1 block truncate text-xs text-muted-foreground lg:hidden">
|
<span className="mt-1 block truncate text-xs text-muted-foreground lg:hidden">
|
||||||
{getFileExtensionDisplayName(item.name)} · 来自「{sourceTitle}」
|
{getFileExtensionDisplayName(item.name)} · {sourceName} · 来自「
|
||||||
|
{sourceTitle}」
|
||||||
</span>
|
</span>
|
||||||
<span className="mt-1 hidden text-xs text-muted-foreground lg:block">
|
<span className="mt-1 hidden text-xs text-muted-foreground lg:block">
|
||||||
{getFileExtensionDisplayName(item.name)} 文件
|
{getFileExtensionDisplayName(item.name)} 文件
|
||||||
@@ -115,13 +159,24 @@ function FileRow({
|
|||||||
</Link>
|
</Link>
|
||||||
</td>
|
</td>
|
||||||
<td className="hidden min-w-0 px-3 py-3 lg:table-cell">
|
<td className="hidden min-w-0 px-3 py-3 lg:table-cell">
|
||||||
<Link
|
<div className="flex min-w-0 items-center gap-2.5">
|
||||||
href={sourcePath}
|
<span className="flex size-8 shrink-0 items-center justify-center rounded-md bg-muted text-muted-foreground">
|
||||||
className="block truncate text-sm text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
|
<SourceIcon className="size-4" />
|
||||||
title={`打开原对话:${sourceTitle}`}
|
</span>
|
||||||
>
|
<span className="min-w-0">
|
||||||
{sourceTitle}
|
<span className="block truncate text-sm font-medium text-foreground">
|
||||||
</Link>
|
{sourceName}
|
||||||
|
</span>
|
||||||
|
<Link
|
||||||
|
href={sourcePath}
|
||||||
|
className="block truncate text-xs text-muted-foreground underline-offset-4 hover:text-foreground hover:underline"
|
||||||
|
title={`打开原对话:${sourceTitle}`}
|
||||||
|
>
|
||||||
|
{item.sourceType === "agent" ? "智能体 · " : ""}
|
||||||
|
{sourceTitle}
|
||||||
|
</Link>
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
</td>
|
</td>
|
||||||
<td className="whitespace-nowrap px-3 py-3 text-sm text-muted-foreground">
|
<td className="whitespace-nowrap px-3 py-3 text-sm text-muted-foreground">
|
||||||
{formatDateTime(item.modifiedAt)}
|
{formatDateTime(item.modifiedAt)}
|
||||||
@@ -136,17 +191,36 @@ function FileRow({
|
|||||||
size="icon"
|
size="icon"
|
||||||
variant="ghost"
|
variant="ghost"
|
||||||
className="size-8"
|
className="size-8"
|
||||||
disabled={downloading}
|
disabled={busy}
|
||||||
onClick={() => onDownload(item)}
|
onClick={() => onDownload(item, "original")}
|
||||||
title={`下载 ${displayName}`}
|
title={`下载原文件:${displayName}`}
|
||||||
aria-label={`下载 ${displayName}`}
|
aria-label={`下载原文件:${displayName}`}
|
||||||
>
|
>
|
||||||
{downloading ? (
|
{downloading === "original" ? (
|
||||||
<Loader2 className="size-4 animate-spin" />
|
<Loader2 className="size-4 animate-spin" />
|
||||||
) : (
|
) : (
|
||||||
<Download className="size-4" />
|
<Download className="size-4" />
|
||||||
)}
|
)}
|
||||||
</Button>
|
</Button>
|
||||||
|
{isMarkdownArtifact(item.name) && (
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
size="sm"
|
||||||
|
variant="ghost"
|
||||||
|
className="h-8 gap-1 px-2 text-xs"
|
||||||
|
disabled={busy}
|
||||||
|
onClick={() => onDownload(item, "word")}
|
||||||
|
title={`下载为 Word:${displayName}`}
|
||||||
|
aria-label={`下载为 Word:${displayName}`}
|
||||||
|
>
|
||||||
|
{downloading === "word" ? (
|
||||||
|
<Loader2 className="size-4 animate-spin" />
|
||||||
|
) : (
|
||||||
|
<FileText className="size-4" />
|
||||||
|
)}
|
||||||
|
Word
|
||||||
|
</Button>
|
||||||
|
)}
|
||||||
<Button
|
<Button
|
||||||
asChild
|
asChild
|
||||||
size="icon"
|
size="icon"
|
||||||
@@ -189,6 +263,7 @@ export default function ArtifactLibraryPage() {
|
|||||||
const [search, setSearch] = useState("");
|
const [search, setSearch] = useState("");
|
||||||
const [debouncedSearch, setDebouncedSearch] = useState("");
|
const [debouncedSearch, setDebouncedSearch] = useState("");
|
||||||
const [fileKind, setFileKind] = useState<ArtifactLibraryKind>("all");
|
const [fileKind, setFileKind] = useState<ArtifactLibraryKind>("all");
|
||||||
|
const [sourceType, setSourceType] = useState<ArtifactLibrarySource>("all");
|
||||||
const [page, setPage] = useState(1);
|
const [page, setPage] = useState(1);
|
||||||
const [downloadingKey, setDownloadingKey] = useState<string | null>(null);
|
const [downloadingKey, setDownloadingKey] = useState<string | null>(null);
|
||||||
const forceRefreshRef = useRef(false);
|
const forceRefreshRef = useRef(false);
|
||||||
@@ -204,17 +279,18 @@ export default function ArtifactLibraryPage() {
|
|||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
setPage(1);
|
setPage(1);
|
||||||
}, [debouncedSearch, fileKind]);
|
}, [debouncedSearch, fileKind, sourceType]);
|
||||||
|
|
||||||
const offset = (page - 1) * PAGE_SIZE;
|
const offset = (page - 1) * PAGE_SIZE;
|
||||||
const libraryQuery = useQuery({
|
const libraryQuery = useQuery({
|
||||||
queryKey: ["artifact-library", debouncedSearch, fileKind, page],
|
queryKey: ["artifact-library", debouncedSearch, fileKind, sourceType, page],
|
||||||
queryFn: async ({ signal }) => {
|
queryFn: async ({ signal }) => {
|
||||||
const refresh = forceRefreshRef.current;
|
const refresh = forceRefreshRef.current;
|
||||||
try {
|
try {
|
||||||
return await fetchArtifactLibrary({
|
return await fetchArtifactLibrary({
|
||||||
query: debouncedSearch,
|
query: debouncedSearch,
|
||||||
fileKind,
|
fileKind,
|
||||||
|
sourceType,
|
||||||
offset,
|
offset,
|
||||||
limit: PAGE_SIZE,
|
limit: PAGE_SIZE,
|
||||||
refresh,
|
refresh,
|
||||||
@@ -238,19 +314,39 @@ export default function ArtifactLibraryPage() {
|
|||||||
}, [page, totalPages]);
|
}, [page, totalPages]);
|
||||||
|
|
||||||
const itemKeys = useMemo(
|
const itemKeys = useMemo(
|
||||||
() => new Set(items.map((item) => `${item.threadId}:${item.path}`)),
|
() =>
|
||||||
|
new Set(
|
||||||
|
items.flatMap((item) => [
|
||||||
|
artifactDownloadKey(item, "original"),
|
||||||
|
artifactDownloadKey(item, "word"),
|
||||||
|
]),
|
||||||
|
),
|
||||||
[items],
|
[items],
|
||||||
);
|
);
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (downloadingKey && !itemKeys.has(downloadingKey)) setDownloadingKey(null);
|
if (downloadingKey && !itemKeys.has(downloadingKey)) setDownloadingKey(null);
|
||||||
}, [downloadingKey, itemKeys]);
|
}, [downloadingKey, itemKeys]);
|
||||||
|
|
||||||
const handleDownload = async (item: ArtifactLibraryItem) => {
|
const handleDownload = async (
|
||||||
const key = `${item.threadId}:${item.path}`;
|
item: ArtifactLibraryItem,
|
||||||
|
format: DownloadFormat,
|
||||||
|
) => {
|
||||||
|
const key = artifactDownloadKey(item, format);
|
||||||
if (downloadingKey) return;
|
if (downloadingKey) return;
|
||||||
setDownloadingKey(key);
|
setDownloadingKey(key);
|
||||||
try {
|
try {
|
||||||
await downloadArtifactFile({ filepath: item.path, threadId: item.threadId });
|
if (format === "word") {
|
||||||
|
await downloadArtifactMarkdownAsWord({
|
||||||
|
filepath: item.path,
|
||||||
|
threadId: item.threadId,
|
||||||
|
});
|
||||||
|
toast.success("Word 文件已生成");
|
||||||
|
} else {
|
||||||
|
await downloadArtifactFile({
|
||||||
|
filepath: item.path,
|
||||||
|
threadId: item.threadId,
|
||||||
|
});
|
||||||
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error(error);
|
console.error(error);
|
||||||
toast.error(error instanceof Error ? error.message : "下载失败");
|
toast.error(error instanceof Error ? error.message : "下载失败");
|
||||||
@@ -298,26 +394,49 @@ export default function ArtifactLibraryPage() {
|
|||||||
<Input
|
<Input
|
||||||
type="search"
|
type="search"
|
||||||
className="h-11 pl-9"
|
className="h-11 pl-9"
|
||||||
placeholder="搜索文件名或来源对话"
|
placeholder="搜索文件名、来源对话或智能体名称"
|
||||||
value={search}
|
value={search}
|
||||||
onChange={(event) => setSearch(event.target.value)}
|
onChange={(event) => setSearch(event.target.value)}
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="flex flex-wrap items-center justify-between gap-3">
|
<div className="flex flex-wrap items-center justify-between gap-4">
|
||||||
<div className="flex flex-wrap gap-1" aria-label="文件类型筛选">
|
<div className="flex flex-wrap items-center gap-3">
|
||||||
{FILTERS.map((filter) => (
|
<div className="flex flex-wrap gap-1" aria-label="文件类型筛选">
|
||||||
<Button
|
{FILTERS.map((filter) => (
|
||||||
key={filter.value}
|
<Button
|
||||||
type="button"
|
key={filter.value}
|
||||||
|
type="button"
|
||||||
|
size="sm"
|
||||||
|
variant={fileKind === filter.value ? "secondary" : "ghost"}
|
||||||
|
aria-pressed={fileKind === filter.value}
|
||||||
|
onClick={() => setFileKind(filter.value)}
|
||||||
|
>
|
||||||
|
{filter.label}
|
||||||
|
</Button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
<Select
|
||||||
|
value={sourceType}
|
||||||
|
onValueChange={(value) =>
|
||||||
|
setSourceType(value as ArtifactLibrarySource)
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<SelectTrigger
|
||||||
size="sm"
|
size="sm"
|
||||||
variant={fileKind === filter.value ? "secondary" : "ghost"}
|
className="w-32"
|
||||||
aria-pressed={fileKind === filter.value}
|
aria-label="筛选文件来源"
|
||||||
onClick={() => setFileKind(filter.value)}
|
|
||||||
>
|
>
|
||||||
{filter.label}
|
<SelectValue />
|
||||||
</Button>
|
</SelectTrigger>
|
||||||
))}
|
<SelectContent>
|
||||||
|
{SOURCE_FILTERS.map((filter) => (
|
||||||
|
<SelectItem key={filter.value} value={filter.value}>
|
||||||
|
{filter.label}
|
||||||
|
</SelectItem>
|
||||||
|
))}
|
||||||
|
</SelectContent>
|
||||||
|
</Select>
|
||||||
</div>
|
</div>
|
||||||
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
||||||
<ArrowDownWideNarrow className="size-4" />
|
<ArrowDownWideNarrow className="size-4" />
|
||||||
@@ -337,11 +456,11 @@ export default function ArtifactLibraryPage() {
|
|||||||
<table className="w-full table-fixed border-collapse">
|
<table className="w-full table-fixed border-collapse">
|
||||||
<thead>
|
<thead>
|
||||||
<tr className="border-b text-left text-xs font-normal text-muted-foreground">
|
<tr className="border-b text-left text-xs font-normal text-muted-foreground">
|
||||||
<th className="w-[48%] py-3 pr-4 pl-1 font-normal lg:w-[38%]">文件名</th>
|
<th className="w-[48%] py-3 pr-4 pl-1 font-normal lg:w-[35%]">文件名</th>
|
||||||
<th className="hidden w-[27%] px-3 py-3 font-normal lg:table-cell">来源对话</th>
|
<th className="hidden w-[25%] px-3 py-3 font-normal lg:table-cell">来源</th>
|
||||||
<th className="w-[34%] px-3 py-3 font-normal md:w-[22%] lg:w-[17%]">生成时间</th>
|
<th className="w-[34%] px-3 py-3 font-normal md:w-[22%] lg:w-[16%]">生成时间</th>
|
||||||
<th className="hidden w-[13%] px-3 py-3 font-normal md:table-cell">大小</th>
|
<th className="hidden w-[13%] px-3 py-3 font-normal lg:w-[10%] md:table-cell">大小</th>
|
||||||
<th className="w-[18%] py-3 pr-1 pl-3 text-right font-normal md:w-[10%]">
|
<th className="w-[18%] py-3 pr-1 pl-3 text-right font-normal md:w-[17%] lg:w-[14%]">
|
||||||
<span className="sr-only">操作</span>
|
<span className="sr-only">操作</span>
|
||||||
</th>
|
</th>
|
||||||
</tr>
|
</tr>
|
||||||
@@ -352,12 +471,19 @@ export default function ArtifactLibraryPage() {
|
|||||||
) : (
|
) : (
|
||||||
items.map((item) => {
|
items.map((item) => {
|
||||||
const key = `${item.threadId}:${item.path}`;
|
const key = `${item.threadId}:${item.path}`;
|
||||||
|
const activeDownload = downloadingKey?.startsWith(`${key}:`)
|
||||||
|
? downloadingKey.endsWith(":word")
|
||||||
|
? "word"
|
||||||
|
: "original"
|
||||||
|
: null;
|
||||||
return (
|
return (
|
||||||
<FileRow
|
<FileRow
|
||||||
key={key}
|
key={key}
|
||||||
item={item}
|
item={item}
|
||||||
downloading={downloadingKey === key}
|
downloading={activeDownload}
|
||||||
onDownload={(target) => void handleDownload(target)}
|
onDownload={(target, format) =>
|
||||||
|
void handleDownload(target, format)
|
||||||
|
}
|
||||||
/>
|
/>
|
||||||
);
|
);
|
||||||
})
|
})
|
||||||
@@ -388,7 +514,7 @@ export default function ArtifactLibraryPage() {
|
|||||||
<FileX2 className="size-9 text-muted-foreground" />
|
<FileX2 className="size-9 text-muted-foreground" />
|
||||||
<div>
|
<div>
|
||||||
<p className="text-sm font-medium">
|
<p className="text-sm font-medium">
|
||||||
{debouncedSearch || fileKind !== "all"
|
{debouncedSearch || fileKind !== "all" || sourceType !== "all"
|
||||||
? "没有找到匹配的文件"
|
? "没有找到匹配的文件"
|
||||||
: "还没有生成文件"}
|
: "还没有生成文件"}
|
||||||
</p>
|
</p>
|
||||||
|
|||||||
@@ -1,74 +1,448 @@
|
|||||||
import { Database, FileUp, History, Loader2, RefreshCw, Search, ShieldCheck, Trash2 } from "lucide-react";
|
import { BookOpen, Database, Download, Loader2, Plus, RefreshCw, Search, ShieldCheck, Trash2, Upload } from "lucide-react";
|
||||||
import { useMemo, useRef, useState } from "react";
|
import { useEffect, useMemo, useRef, useState } from "react";
|
||||||
import { toast } from "sonner";
|
import { toast } from "sonner";
|
||||||
|
|
||||||
import { Badge } from "@/components/ui/badge";
|
import { Badge } from "@/components/ui/badge";
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
|
import {
|
||||||
|
Dialog,
|
||||||
|
DialogContent,
|
||||||
|
DialogDescription,
|
||||||
|
DialogFooter,
|
||||||
|
DialogHeader,
|
||||||
|
DialogTitle,
|
||||||
|
} from "@/components/ui/dialog";
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
|
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
|
||||||
import { MarkdownContent } from "@/components/workspace/messages/markdown-content";
|
import { Textarea } from "@/components/ui/textarea";
|
||||||
import { KnowledgeProductNav } from "@/components/workspace/knowledge-product-nav";
|
import { KnowledgeProductNav } from "@/components/workspace/knowledge-product-nav";
|
||||||
|
import { MarkdownContent } from "@/components/workspace/messages/markdown-content";
|
||||||
import { getStoredAuth } from "@/core/auth";
|
import { getStoredAuth } from "@/core/auth";
|
||||||
import {
|
import {
|
||||||
listAssistantWikiRevisions,
|
type AssistantKnowledgeBase,
|
||||||
type AssistantWikiPage,
|
type AssistantWikiPage,
|
||||||
type AssistantWikiRevision,
|
|
||||||
useAssistantImportJobs,
|
useAssistantImportJobs,
|
||||||
useAssistantKnowledgeActions,
|
useAssistantKnowledgeActions,
|
||||||
useAssistantKnowledgeBase,
|
useAssistantKnowledgeBases,
|
||||||
useAssistantKnowledgeSources,
|
useAssistantKnowledgeSources,
|
||||||
useAssistantWikiPages,
|
useAssistantWikiPages,
|
||||||
} from "@/strategy-components/api/assistant-knowledge";
|
} from "@/strategy-components/api/assistant-knowledge";
|
||||||
|
|
||||||
|
function baseSubtitle(base: AssistantKnowledgeBase) {
|
||||||
|
if (base.is_global) return "全库沉淀总库 · 所有 WeKnora 导入数据都会同步进入这里并做实体对齐";
|
||||||
|
return base.description || "自建助手知识库 · 可上传普通知识库导出的 Wiki 向量包";
|
||||||
|
}
|
||||||
|
|
||||||
|
function EmptyInitializeCard({
|
||||||
|
isAdmin,
|
||||||
|
onInitialize,
|
||||||
|
pending,
|
||||||
|
}: {
|
||||||
|
isAdmin: boolean;
|
||||||
|
onInitialize: () => void;
|
||||||
|
pending: boolean;
|
||||||
|
}) {
|
||||||
|
return (
|
||||||
|
<div className="flex flex-1 items-center justify-center p-8">
|
||||||
|
<div className="max-w-lg rounded-xl border bg-card p-8 text-center">
|
||||||
|
<Database className="mx-auto mb-4 size-10 text-primary" />
|
||||||
|
<h1 className="text-xl font-semibold">初始化助手知识库</h1>
|
||||||
|
<p className="mt-2 text-sm text-muted-foreground">
|
||||||
|
未配置 WeKnora 时会默认展示助手知识库。首次使用需要创建“知识梳理总库”,后续从 WeKnora 导入的数据都会沉淀到这里。
|
||||||
|
</p>
|
||||||
|
{isAdmin ? (
|
||||||
|
<Button className="mt-5" disabled={pending} onClick={onInitialize}>
|
||||||
|
{pending ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Database className="mr-2 size-4" />}
|
||||||
|
创建知识梳理总库
|
||||||
|
</Button>
|
||||||
|
) : (
|
||||||
|
<p className="mt-5 text-sm">请联系管理员初始化。</p>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
export default function AssistantKnowledgePage() {
|
export default function AssistantKnowledgePage() {
|
||||||
const isAdmin = getStoredAuth()?.system_role === "admin";
|
const isAdmin = getStoredAuth()?.system_role === "admin";
|
||||||
const [query, setQuery] = useState("");
|
const [query, setQuery] = useState("");
|
||||||
|
const [selectedBaseId, setSelectedBaseId] = useState<string | null>(null);
|
||||||
const [selected, setSelected] = useState<AssistantWikiPage | null>(null);
|
const [selected, setSelected] = useState<AssistantWikiPage | null>(null);
|
||||||
const [revisions, setRevisions] = useState<AssistantWikiRevision[]>([]);
|
const [createOpen, setCreateOpen] = useState(false);
|
||||||
const fileRef = useRef<HTMLInputElement>(null);
|
const [createDraft, setCreateDraft] = useState({ name: "", description: "" });
|
||||||
const base = useAssistantKnowledgeBase();
|
const packageInputRef = useRef<HTMLInputElement | null>(null);
|
||||||
const pages = useAssistantWikiPages(query);
|
const bases = useAssistantKnowledgeBases();
|
||||||
const sources = useAssistantKnowledgeSources();
|
|
||||||
const jobs = useAssistantImportJobs();
|
|
||||||
const actions = useAssistantKnowledgeActions();
|
const actions = useAssistantKnowledgeActions();
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const values = bases.data ?? [];
|
||||||
|
if (!values.length) {
|
||||||
|
setSelectedBaseId(null);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (selectedBaseId && values.some((base) => base.id === selectedBaseId)) return;
|
||||||
|
setSelectedBaseId((values.find((base) => base.is_global) ?? values[0]).id);
|
||||||
|
}, [bases.data, selectedBaseId]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
setSelected(null);
|
||||||
|
}, [selectedBaseId]);
|
||||||
|
|
||||||
|
const selectedBase = useMemo(
|
||||||
|
() => (bases.data ?? []).find((base) => base.id === selectedBaseId) ?? null,
|
||||||
|
[bases.data, selectedBaseId],
|
||||||
|
);
|
||||||
|
const pages = useAssistantWikiPages(query, selectedBaseId);
|
||||||
|
const sources = useAssistantKnowledgeSources(selectedBaseId);
|
||||||
|
const jobs = useAssistantImportJobs(selectedBaseId);
|
||||||
const activePage = selected ?? pages.data?.[0] ?? null;
|
const activePage = selected ?? pages.data?.[0] ?? null;
|
||||||
|
const counts = useMemo(
|
||||||
|
() => ({
|
||||||
|
pages: pages.data?.length ?? selectedBase?.page_count ?? 0,
|
||||||
|
sources: sources.data?.length ?? selectedBase?.source_count ?? 0,
|
||||||
|
jobs: jobs.data?.length ?? 0,
|
||||||
|
}),
|
||||||
|
[jobs.data, pages.data, selectedBase?.page_count, selectedBase?.source_count, sources.data],
|
||||||
|
);
|
||||||
|
|
||||||
const counts = useMemo(() => ({ pages: pages.data?.length ?? 0, sources: sources.data?.length ?? 0, jobs: jobs.data?.length ?? 0 }), [pages.data, sources.data, jobs.data]);
|
const initializeGlobal = () => {
|
||||||
|
actions.initialize.mutate("知识梳理总库", {
|
||||||
|
onSuccess: (base) => {
|
||||||
|
setSelectedBaseId(base.id);
|
||||||
|
toast.success("知识梳理总库已创建");
|
||||||
|
},
|
||||||
|
onError: (error) => toast.error(error.message),
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
if (base.isLoading) return <div className="flex size-full flex-col"><KnowledgeProductNav /><div className="flex flex-1 items-center justify-center"><Loader2 className="animate-spin" /></div></div>;
|
const createBase = async () => {
|
||||||
if (!base.data) {
|
const name = createDraft.name.trim();
|
||||||
return <div className="flex size-full flex-col"><KnowledgeProductNav /><div className="flex flex-1 items-center justify-center p-8"><div className="max-w-lg rounded-xl border bg-card p-8 text-center"><Database className="mx-auto mb-4 size-10 text-primary" /><h1 className="text-xl font-semibold">创建“知识梳理”助手知识库</h1><p className="mt-2 text-sm text-muted-foreground">无需配置普通知识库服务。创建后所有用户可查看 Wiki 与搜索,只有管理员可导入和维护。</p>{isAdmin ? <Button className="mt-5" disabled={actions.initialize.isPending} onClick={() => actions.initialize.mutate("知识梳理")}>创建知识梳理库</Button> : <p className="mt-5 text-sm">请联系管理员初始化。</p>}</div></div></div>;
|
if (!name) {
|
||||||
}
|
toast.error("请输入助手知识库名称");
|
||||||
|
return;
|
||||||
const uploadFiles = async (files: FileList | null) => {
|
}
|
||||||
if (!files?.length) return;
|
|
||||||
try {
|
try {
|
||||||
await actions.upload.mutateAsync(Array.from(files));
|
const base = await actions.createBase.mutateAsync({
|
||||||
toast.success("文件已安全扫描并导入");
|
name,
|
||||||
|
description: createDraft.description.trim(),
|
||||||
|
});
|
||||||
|
setSelectedBaseId(base.id);
|
||||||
|
setCreateDraft({ name: "", description: "" });
|
||||||
|
setCreateOpen(false);
|
||||||
|
toast.success("助手知识库已创建");
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
toast.error(error instanceof Error ? error.message : String(error));
|
toast.error(error instanceof Error ? error.message : String(error));
|
||||||
} finally {
|
|
||||||
if (fileRef.current) fileRef.current.value = "";
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const openHistory = async () => {
|
const uploadPackage = async (file: File | null | undefined) => {
|
||||||
if (!activePage) return;
|
if (!file) return;
|
||||||
setRevisions(await listAssistantWikiRevisions(activePage.slug));
|
try {
|
||||||
|
await actions.uploadWeKnoraPackage.mutateAsync({
|
||||||
|
file,
|
||||||
|
assistantBaseId: selectedBaseId,
|
||||||
|
});
|
||||||
|
toast.success("导出包已上传,后台正在异步导入");
|
||||||
|
} catch (error) {
|
||||||
|
toast.error(error instanceof Error ? error.message : String(error));
|
||||||
|
} finally {
|
||||||
|
if (packageInputRef.current) packageInputRef.current.value = "";
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
if (bases.isLoading) {
|
||||||
|
return (
|
||||||
|
<div className="flex size-full flex-col">
|
||||||
|
<KnowledgeProductNav />
|
||||||
|
<div className="flex flex-1 items-center justify-center">
|
||||||
|
<Loader2 className="animate-spin" />
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!bases.data?.length) {
|
||||||
|
return (
|
||||||
|
<div className="flex size-full flex-col">
|
||||||
|
<KnowledgeProductNav />
|
||||||
|
<EmptyInitializeCard
|
||||||
|
isAdmin={isAdmin}
|
||||||
|
pending={actions.initialize.isPending}
|
||||||
|
onInitialize={initializeGlobal}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="flex size-full min-h-0 flex-col bg-background">
|
<div className="flex size-full min-h-0 flex-col bg-background">
|
||||||
<KnowledgeProductNav />
|
<KnowledgeProductNav />
|
||||||
<header className="flex items-center gap-4 border-b px-6 py-4"><div><h1 className="text-xl font-semibold">{base.data.name}</h1><p className="text-xs text-muted-foreground">全局助手知识库 · 普通用户只读</p></div><Badge variant="outline"><ShieldCheck className="mr-1 size-3" />实体对齐与版本管理</Badge><div className="ml-auto flex gap-2">{isAdmin && <><input ref={fileRef} className="hidden" type="file" multiple onChange={(event) => void uploadFiles(event.target.files)} /><Button variant="outline" onClick={() => fileRef.current?.click()}><FileUp />导入本地文件</Button></>}<Button variant="outline" disabled={!activePage} onClick={() => void openHistory()}><History />页面版本</Button></div></header>
|
<div className="grid min-h-0 flex-1 grid-cols-[280px_1fr]">
|
||||||
<Tabs defaultValue="wiki" className="flex min-h-0 flex-1 flex-col">
|
<aside className="flex min-h-0 flex-col border-r bg-muted/20">
|
||||||
<div className="flex items-center gap-4 border-b px-6"><TabsList className="my-2"><TabsTrigger value="wiki">Wiki({counts.pages})</TabsTrigger><TabsTrigger value="sources">来源({counts.sources})</TabsTrigger><TabsTrigger value="jobs">导入记录({counts.jobs})</TabsTrigger><TabsTrigger value="revisions">版本({revisions.length})</TabsTrigger></TabsList><div className="relative ml-auto w-72"><Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" /><Input value={query} onChange={(event) => setQuery(event.target.value)} placeholder="搜索标题、实体别名和正文" className="pl-8" /></div></div>
|
<div className="border-b p-4">
|
||||||
<TabsContent value="wiki" className="mt-0 grid min-h-0 flex-1 grid-cols-[300px_1fr]"><ScrollArea className="border-r"><div className="space-y-1 p-3">{pages.data?.map((page) => <button key={page.id} type="button" onClick={() => setSelected(page)} className={`w-full rounded-md px-3 py-2 text-left ${activePage?.id === page.id ? "bg-primary/10 text-primary" : "hover:bg-muted"}`}><div className="truncate text-sm font-medium">{page.title}</div><div className="mt-1 flex items-center gap-2 text-[11px] text-muted-foreground"><span>{page.page_type}</span><span>{new Date(page.updated_at).toLocaleString()}</span></div></button>)}</div></ScrollArea><ScrollArea><article className="mx-auto max-w-5xl p-8">{activePage ? <><h1 className="mb-1 text-2xl font-semibold">{activePage.title}</h1><p className="mb-6 text-xs text-muted-foreground">{activePage.slug}</p><MarkdownContent content={activePage.content_markdown} isLoading={false} rehypePlugins={[]} /></> : <p className="text-sm text-muted-foreground">暂无 Wiki 页面。</p>}</article></ScrollArea></TabsContent>
|
<div className="flex items-center justify-between">
|
||||||
<TabsContent value="sources" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">{sources.data?.map((source) => <div key={source.id} className="rounded-lg border bg-card p-4"><div className="flex items-center gap-2"><Database className="size-4" /><span className="font-medium">{source.source_name}</span><Badge variant="outline" className="ml-auto">{source.source_type}</Badge></div><p className="mt-3 text-xs text-muted-foreground">业务键:{source.source_key}</p>{isAdmin && <div className="mt-4 flex justify-end gap-2"><Button size="sm" variant="outline" disabled={source.source_type === "local_file" || actions.reimportSource.isPending} onClick={() => actions.reimportSource.mutate(source.id, { onSuccess: () => toast.success("已发起重新导入"), onError: (error) => toast.error(error.message) })}><RefreshCw />重新导入</Button><Button size="sm" variant="destructive" disabled={actions.deleteSource.isPending} onClick={() => { if (window.confirm(`确认删除来源“${source.source_name}”的贡献?仍被其他来源引用的知识会保留。`)) actions.deleteSource.mutate(source.id, { onSuccess: () => toast.success("来源贡献已删除"), onError: (error) => toast.error(error.message) }); }}><Trash2 />删除来源</Button></div>}</div>)}</div></TabsContent>
|
<div>
|
||||||
<TabsContent value="jobs" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="space-y-3">{jobs.data?.map((job) => <div key={job.id} className="flex items-center gap-3 rounded-lg border bg-card p-4"><Badge>{job.status}</Badge><span className="text-sm">{job.trigger}</span><span className="text-xs text-muted-foreground">{Object.entries(job.counts || {}).map(([key, value]) => `${key} ${value}`).join(" · ")}</span><span className="ml-auto text-xs">{new Date(job.created_at).toLocaleString()}</span></div>)}</div></TabsContent>
|
<h2 className="text-sm font-semibold">助手知识库</h2>
|
||||||
<TabsContent value="revisions" className="mt-0 min-h-0 flex-1 overflow-auto p-6"><div className="space-y-3">{revisions.map((revision) => <div key={revision.id} className="flex items-center gap-3 rounded-lg border bg-card p-4"><Badge variant="outline">v{revision.revision_no}</Badge><span className="text-sm">{revision.change_type}</span><span className="text-xs text-muted-foreground">{revision.summary}</span><span className="ml-auto text-xs">{new Date(revision.created_at).toLocaleString()}</span>{isAdmin && activePage && revision.id !== activePage.current_revision_id && <Button size="sm" variant="outline" onClick={() => actions.rollback.mutate({ slug: activePage.slug, revisionId: revision.id })}>回滚到此版本</Button>}</div>)}{!revisions.length && <p className="text-sm text-muted-foreground">请先在 Wiki 页选择页面并点击“页面版本”。</p>}</div></TabsContent>
|
<p className="mt-1 text-xs text-muted-foreground">总库置顶,自建库可上传 WeKnora 导出包</p>
|
||||||
</Tabs>
|
</div>
|
||||||
|
{isAdmin && (
|
||||||
|
<Button size="icon" variant="outline" onClick={() => setCreateOpen(true)}>
|
||||||
|
<Plus className="size-4" />
|
||||||
|
</Button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<ScrollArea className="min-h-0 flex-1">
|
||||||
|
<div className="space-y-2 p-3">
|
||||||
|
{(bases.data ?? []).map((base) => (
|
||||||
|
<button
|
||||||
|
key={base.id}
|
||||||
|
type="button"
|
||||||
|
onClick={() => setSelectedBaseId(base.id)}
|
||||||
|
className={`w-full rounded-lg border px-3 py-3 text-left transition ${
|
||||||
|
selectedBaseId === base.id ? "border-primary bg-primary/10 text-primary" : "bg-card hover:bg-muted"
|
||||||
|
}`}
|
||||||
|
>
|
||||||
|
<div className="flex items-center gap-2">
|
||||||
|
<Database className="size-4 shrink-0" />
|
||||||
|
<span className="min-w-0 flex-1 truncate text-sm font-medium">{base.name}</span>
|
||||||
|
{base.is_global && <Badge variant="outline" className="text-[10px]">总库</Badge>}
|
||||||
|
</div>
|
||||||
|
<div className="mt-2 flex gap-3 text-[11px] text-muted-foreground">
|
||||||
|
<span>{base.page_count ?? 0} 页</span>
|
||||||
|
<span>{base.source_count ?? 0} 来源</span>
|
||||||
|
<span>{base.entity_count ?? 0} 实体</span>
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
</ScrollArea>
|
||||||
|
</aside>
|
||||||
|
|
||||||
|
<main className="flex min-h-0 flex-col">
|
||||||
|
<header className="flex items-center gap-4 border-b px-6 py-4">
|
||||||
|
<div className="min-w-0">
|
||||||
|
<h1 className="truncate text-xl font-semibold">{selectedBase?.name}</h1>
|
||||||
|
<p className="mt-1 truncate text-xs text-muted-foreground">
|
||||||
|
{selectedBase ? baseSubtitle(selectedBase) : "请选择助手知识库"}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<Badge variant="outline">
|
||||||
|
<ShieldCheck className="mr-1 size-3" />
|
||||||
|
实体对齐
|
||||||
|
</Badge>
|
||||||
|
{isAdmin && (
|
||||||
|
<div className="ml-auto flex items-center gap-2">
|
||||||
|
<input
|
||||||
|
ref={packageInputRef}
|
||||||
|
type="file"
|
||||||
|
accept=".jsonl,.ndjson,application/x-ndjson,application/json"
|
||||||
|
className="hidden"
|
||||||
|
onChange={(event) => void uploadPackage(event.target.files?.[0])}
|
||||||
|
/>
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
disabled={actions.uploadWeKnoraPackage.isPending || !selectedBaseId}
|
||||||
|
onClick={() => packageInputRef.current?.click()}
|
||||||
|
>
|
||||||
|
{actions.uploadWeKnoraPackage.isPending ? (
|
||||||
|
<Loader2 className="mr-2 size-4 animate-spin" />
|
||||||
|
) : (
|
||||||
|
<Upload className="mr-2 size-4" />
|
||||||
|
)}
|
||||||
|
上传 WeKnora 导出包
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<Tabs defaultValue="wiki" className="flex min-h-0 flex-1 flex-col">
|
||||||
|
<div className="flex items-center gap-4 border-b px-6">
|
||||||
|
<TabsList className="my-2">
|
||||||
|
<TabsTrigger value="wiki">Wiki({counts.pages})</TabsTrigger>
|
||||||
|
<TabsTrigger value="sources">来源({counts.sources})</TabsTrigger>
|
||||||
|
<TabsTrigger value="jobs">导入记录({counts.jobs})</TabsTrigger>
|
||||||
|
</TabsList>
|
||||||
|
<div className="relative ml-auto w-72">
|
||||||
|
<Search className="absolute left-2.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
|
||||||
|
<Input
|
||||||
|
value={query}
|
||||||
|
onChange={(event) => setQuery(event.target.value)}
|
||||||
|
placeholder="搜索标题、实体别名和正文"
|
||||||
|
className="pl-8"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<TabsContent value="wiki" className="mt-0 grid min-h-0 flex-1 grid-cols-[300px_1fr]">
|
||||||
|
<ScrollArea className="border-r">
|
||||||
|
<div className="space-y-1 p-3">
|
||||||
|
{pages.isLoading && <Loader2 className="m-4 size-5 animate-spin" />}
|
||||||
|
{pages.data?.map((page) => (
|
||||||
|
<button
|
||||||
|
key={page.id}
|
||||||
|
type="button"
|
||||||
|
onClick={() => setSelected(page)}
|
||||||
|
className={`w-full rounded-md px-3 py-2 text-left ${
|
||||||
|
activePage?.id === page.id ? "bg-primary/10 text-primary" : "hover:bg-muted"
|
||||||
|
}`}
|
||||||
|
>
|
||||||
|
<div className="truncate text-sm font-medium">{page.title}</div>
|
||||||
|
<div className="mt-1 flex items-center gap-2 text-[11px] text-muted-foreground">
|
||||||
|
<span>{page.page_type}</span>
|
||||||
|
<span>{new Date(page.updated_at).toLocaleString()}</span>
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
{!pages.isLoading && !pages.data?.length && (
|
||||||
|
<div className="p-4 text-xs text-muted-foreground">
|
||||||
|
暂无 Wiki 页面。请先到普通知识库详情页下载向量化数据包,再在当前助手知识库上传。
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</ScrollArea>
|
||||||
|
<ScrollArea>
|
||||||
|
<article className="mx-auto max-w-5xl p-8">
|
||||||
|
{activePage ? (
|
||||||
|
<>
|
||||||
|
<h1 className="mb-1 flex items-center gap-2 text-2xl font-semibold">
|
||||||
|
<BookOpen className="size-5 text-primary" />
|
||||||
|
{activePage.title}
|
||||||
|
</h1>
|
||||||
|
<p className="mb-6 text-xs text-muted-foreground">{activePage.slug}</p>
|
||||||
|
<MarkdownContent content={activePage.content_markdown} isLoading={false} rehypePlugins={[]} />
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<p className="text-sm text-muted-foreground">暂无 Wiki 页面。</p>
|
||||||
|
)}
|
||||||
|
</article>
|
||||||
|
</ScrollArea>
|
||||||
|
</TabsContent>
|
||||||
|
|
||||||
|
<TabsContent value="sources" className="mt-0 min-h-0 flex-1 overflow-auto p-6">
|
||||||
|
<div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">
|
||||||
|
{sources.data?.map((source) => (
|
||||||
|
<div key={source.id} className="rounded-lg border bg-card p-4">
|
||||||
|
<div className="flex items-center gap-2">
|
||||||
|
<Database className="size-4" />
|
||||||
|
<span className="truncate font-medium">{source.source_name}</span>
|
||||||
|
<Badge variant="outline" className="ml-auto">{source.source_type}</Badge>
|
||||||
|
</div>
|
||||||
|
<p className="mt-3 text-xs text-muted-foreground">业务键:{source.source_key}</p>
|
||||||
|
{isAdmin && (
|
||||||
|
<div className="mt-4 flex justify-end gap-2">
|
||||||
|
<Button
|
||||||
|
size="sm"
|
||||||
|
variant="outline"
|
||||||
|
disabled={source.source_type !== "weknora" || actions.reimportSource.isPending}
|
||||||
|
onClick={() =>
|
||||||
|
actions.reimportSource.mutate(source.id, {
|
||||||
|
onSuccess: () => toast.success("已发起重新导入"),
|
||||||
|
onError: (error) => toast.error(error.message),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<RefreshCw className="mr-2 size-4" />
|
||||||
|
重新导入
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
size="sm"
|
||||||
|
variant="destructive"
|
||||||
|
disabled={actions.deleteSource.isPending}
|
||||||
|
onClick={() => {
|
||||||
|
if (window.confirm(`确认删除来源“${source.source_name}”的贡献?仍被其他来源引用的知识会保留。`)) {
|
||||||
|
actions.deleteSource.mutate(source.id, {
|
||||||
|
onSuccess: () => toast.success("来源贡献已删除"),
|
||||||
|
onError: (error) => toast.error(error.message),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Trash2 className="mr-2 size-4" />
|
||||||
|
删除来源
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
{!sources.data?.length && <p className="text-sm text-muted-foreground">暂无来源。</p>}
|
||||||
|
</div>
|
||||||
|
</TabsContent>
|
||||||
|
|
||||||
|
<TabsContent value="jobs" className="mt-0 min-h-0 flex-1 overflow-auto p-6">
|
||||||
|
<div className="space-y-3">
|
||||||
|
{jobs.data?.map((job) => (
|
||||||
|
<div key={job.id} className="rounded-lg border bg-card p-4">
|
||||||
|
<div className="flex items-center gap-3">
|
||||||
|
<Badge>{job.status}</Badge>
|
||||||
|
<span className="text-sm">{job.trigger}</span>
|
||||||
|
<span className="text-xs text-muted-foreground">阶段:{job.phase}</span>
|
||||||
|
<span className="ml-auto text-xs">{new Date(job.created_at).toLocaleString()}</span>
|
||||||
|
{isAdmin && job.package_download_url && (
|
||||||
|
<Button asChild size="sm" variant="outline">
|
||||||
|
<a href={job.package_download_url} target="_blank" rel="noreferrer">
|
||||||
|
<Download className="mr-2 size-4" />
|
||||||
|
下载导出包
|
||||||
|
</a>
|
||||||
|
</Button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
<div className="mt-2 text-xs text-muted-foreground">
|
||||||
|
{Object.entries(job.counts || {}).map(([key, value]) => `${key} ${value}`).join(" · ") || "暂无统计"}
|
||||||
|
</div>
|
||||||
|
{job.last_error_message && (
|
||||||
|
<div className="mt-2 rounded-md bg-destructive/10 px-3 py-2 text-xs text-destructive">
|
||||||
|
{job.last_error_message}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
{!jobs.data?.length && <p className="text-sm text-muted-foreground">暂无导入记录。</p>}
|
||||||
|
</div>
|
||||||
|
</TabsContent>
|
||||||
|
</Tabs>
|
||||||
|
</main>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<Dialog open={createOpen} onOpenChange={setCreateOpen}>
|
||||||
|
<DialogContent>
|
||||||
|
<DialogHeader>
|
||||||
|
<DialogTitle>新建助手知识库</DialogTitle>
|
||||||
|
<DialogDescription>
|
||||||
|
自建助手知识库用于承接从 WeKnora 普通知识库下载的 Wiki 向量包;导入时同一份数据也会沉淀到“知识梳理总库”。
|
||||||
|
</DialogDescription>
|
||||||
|
</DialogHeader>
|
||||||
|
<div className="space-y-4">
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<label className="text-sm font-medium">名称</label>
|
||||||
|
<Input
|
||||||
|
value={createDraft.name}
|
||||||
|
onChange={(event) => setCreateDraft((prev) => ({ ...prev, name: event.target.value }))}
|
||||||
|
placeholder="例如:政策资料库"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<label className="text-sm font-medium">描述</label>
|
||||||
|
<Textarea
|
||||||
|
value={createDraft.description}
|
||||||
|
onChange={(event) => setCreateDraft((prev) => ({ ...prev, description: event.target.value }))}
|
||||||
|
placeholder="可选,用于说明这个助手知识库的范围"
|
||||||
|
rows={3}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<DialogFooter>
|
||||||
|
<Button variant="outline" onClick={() => setCreateOpen(false)}>
|
||||||
|
取消
|
||||||
|
</Button>
|
||||||
|
<Button disabled={actions.createBase.isPending} onClick={() => void createBase()}>
|
||||||
|
{actions.createBase.isPending ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Plus className="mr-2 size-4" />}
|
||||||
|
创建
|
||||||
|
</Button>
|
||||||
|
</DialogFooter>
|
||||||
|
</DialogContent>
|
||||||
|
</Dialog>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
@@ -132,6 +132,7 @@ export default function ChatPage() {
|
|||||||
// per-thread header toggle so stale localStorage can't re-enable it.
|
// per-thread header toggle so stale localStorage can't re-enable it.
|
||||||
const [knowledgeRagEnabled] = useKnowledgeRagEnabled();
|
const [knowledgeRagEnabled] = useKnowledgeRagEnabled();
|
||||||
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>([]);
|
const [selectedKnowledgeBaseIds, setSelectedKnowledgeBaseIds] = useState<string[]>([]);
|
||||||
|
const [selectedAssistantKnowledgeBaseIds, setSelectedAssistantKnowledgeBaseIds] = useState<string[]>([]);
|
||||||
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(false);
|
const [knowledgeSelectionReady, setKnowledgeSelectionReady] = useState(false);
|
||||||
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
|
const [knowledgeSelectionLoading, setKnowledgeSelectionLoading] = useState(false);
|
||||||
const knowledgeSelectionThread = useRef<string | null>(null);
|
const knowledgeSelectionThread = useRef<string | null>(null);
|
||||||
@@ -140,18 +141,26 @@ export default function ChatPage() {
|
|||||||
setSelectedKnowledgeBaseIds(ids);
|
setSelectedKnowledgeBaseIds(ids);
|
||||||
setKnowledgeSelectionReady(true);
|
setKnowledgeSelectionReady(true);
|
||||||
}, []);
|
}, []);
|
||||||
|
const changeAssistantKnowledgeSelection = useCallback((ids: string[]) => {
|
||||||
|
setSelectedAssistantKnowledgeBaseIds(ids);
|
||||||
|
setKnowledgeSelectionReady(true);
|
||||||
|
}, []);
|
||||||
const streamContext = useMemo(
|
const streamContext = useMemo(
|
||||||
() => ({
|
() => ({
|
||||||
...settings.context,
|
...settings.context,
|
||||||
knowledge_rag_enabled: knowledgeRagEnabled,
|
knowledge_rag_enabled: knowledgeRagEnabled,
|
||||||
...(knowledgeSelectionReady
|
...(knowledgeSelectionReady
|
||||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
? {
|
||||||
|
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||||
|
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||||
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}),
|
}),
|
||||||
[
|
[
|
||||||
settings.context,
|
settings.context,
|
||||||
knowledgeRagEnabled,
|
knowledgeRagEnabled,
|
||||||
knowledgeSelectionReady,
|
knowledgeSelectionReady,
|
||||||
|
selectedAssistantKnowledgeBaseIds,
|
||||||
selectedKnowledgeBaseIds,
|
selectedKnowledgeBaseIds,
|
||||||
],
|
],
|
||||||
);
|
);
|
||||||
@@ -190,6 +199,7 @@ export default function ChatPage() {
|
|||||||
setHasStartedConversation(false);
|
setHasStartedConversation(false);
|
||||||
setPromptPrefixOn(false);
|
setPromptPrefixOn(false);
|
||||||
setSelectedKnowledgeBaseIds([]);
|
setSelectedKnowledgeBaseIds([]);
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
setKnowledgeSelectionReady(false);
|
||||||
setKnowledgeSelectionLoading(false);
|
setKnowledgeSelectionLoading(false);
|
||||||
knowledgeSelectionThread.current = "__new__";
|
knowledgeSelectionThread.current = "__new__";
|
||||||
@@ -215,18 +225,29 @@ export default function ChatPage() {
|
|||||||
if (knowledgeSelectionThread.current !== threadId) return;
|
if (knowledgeSelectionThread.current !== threadId) return;
|
||||||
const metadata = record.metadata as Record<string, unknown> | undefined;
|
const metadata = record.metadata as Record<string, unknown> | undefined;
|
||||||
const raw = metadata?.llmwiki_knowledge_base_ids;
|
const raw = metadata?.llmwiki_knowledge_base_ids;
|
||||||
|
const assistantRaw = metadata?.assistant_knowledge_base_ids;
|
||||||
|
const llmwikiIds = Array.isArray(raw)
|
||||||
|
? raw.map((item) => String(item).trim()).filter(Boolean)
|
||||||
|
: [];
|
||||||
|
const assistantIds = Array.isArray(assistantRaw)
|
||||||
|
? assistantRaw.map((item) => String(item).trim()).filter(Boolean)
|
||||||
|
: [];
|
||||||
if (Array.isArray(raw)) {
|
if (Array.isArray(raw)) {
|
||||||
setSelectedKnowledgeBaseIds(
|
setSelectedKnowledgeBaseIds(llmwikiIds);
|
||||||
raw.map((item) => String(item).trim()).filter(Boolean),
|
|
||||||
);
|
|
||||||
setKnowledgeSelectionReady(true);
|
|
||||||
} else {
|
} else {
|
||||||
setSelectedKnowledgeBaseIds([]);
|
setSelectedKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
|
||||||
}
|
}
|
||||||
|
if (Array.isArray(assistantRaw)) {
|
||||||
|
setSelectedAssistantKnowledgeBaseIds(assistantIds);
|
||||||
|
} else {
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
|
}
|
||||||
|
setKnowledgeSelectionReady(Array.isArray(raw) || Array.isArray(assistantRaw));
|
||||||
})
|
})
|
||||||
.catch(() => {
|
.catch(() => {
|
||||||
if (knowledgeSelectionThread.current === threadId) {
|
if (knowledgeSelectionThread.current === threadId) {
|
||||||
|
setSelectedKnowledgeBaseIds([]);
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
setKnowledgeSelectionReady(false);
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -249,6 +270,7 @@ export default function ChatPage() {
|
|||||||
if (!known.some((a) => a.id === selectedAgentId)) {
|
if (!known.some((a) => a.id === selectedAgentId)) {
|
||||||
setSelectedAgentId(null);
|
setSelectedAgentId(null);
|
||||||
setSelectedKnowledgeBaseIds([]);
|
setSelectedKnowledgeBaseIds([]);
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
setKnowledgeSelectionReady(false);
|
||||||
}
|
}
|
||||||
}, [selectedAgentId, selectorAgents, mineAgents, squareAgents, selectorLoading]);
|
}, [selectedAgentId, selectorAgents, mineAgents, squareAgents, selectorLoading]);
|
||||||
@@ -259,7 +281,10 @@ export default function ChatPage() {
|
|||||||
threadId: isNewThread ? undefined : threadId,
|
threadId: isNewThread ? undefined : threadId,
|
||||||
context: streamContext,
|
context: streamContext,
|
||||||
threadMetadata: knowledgeSelectionReady
|
threadMetadata: knowledgeSelectionReady
|
||||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
? {
|
||||||
|
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||||
|
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||||
|
}
|
||||||
: undefined,
|
: undefined,
|
||||||
isMock,
|
isMock,
|
||||||
onStart: (createdThreadId) => {
|
onStart: (createdThreadId) => {
|
||||||
@@ -434,7 +459,10 @@ export default function ChatPage() {
|
|||||||
metadata: {
|
metadata: {
|
||||||
agent_id: selectedAgentId,
|
agent_id: selectedAgentId,
|
||||||
...(knowledgeSelectionReady
|
...(knowledgeSelectionReady
|
||||||
? { llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds }
|
? {
|
||||||
|
llmwiki_knowledge_base_ids: selectedKnowledgeBaseIds,
|
||||||
|
assistant_knowledge_base_ids: selectedAssistantKnowledgeBaseIds,
|
||||||
|
}
|
||||||
: {}),
|
: {}),
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -445,7 +473,10 @@ export default function ChatPage() {
|
|||||||
threadId: created.thread_id,
|
threadId: created.thread_id,
|
||||||
text,
|
text,
|
||||||
...(knowledgeSelectionReady
|
...(knowledgeSelectionReady
|
||||||
? { knowledgeBaseIds: selectedKnowledgeBaseIds }
|
? {
|
||||||
|
knowledgeBaseIds: selectedKnowledgeBaseIds,
|
||||||
|
assistantKnowledgeBaseIds: selectedAssistantKnowledgeBaseIds,
|
||||||
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
@@ -505,6 +536,7 @@ export default function ChatPage() {
|
|||||||
hideAdminPrompt,
|
hideAdminPrompt,
|
||||||
preferencesLoading,
|
preferencesLoading,
|
||||||
selectedAgentId,
|
selectedAgentId,
|
||||||
|
selectedAssistantKnowledgeBaseIds,
|
||||||
selectedKnowledgeBaseIds,
|
selectedKnowledgeBaseIds,
|
||||||
knowledgeSelectionReady,
|
knowledgeSelectionReady,
|
||||||
agentHandoffPending,
|
agentHandoffPending,
|
||||||
@@ -608,6 +640,7 @@ export default function ChatPage() {
|
|||||||
const handleAgentChange = useCallback((agentId: string | null) => {
|
const handleAgentChange = useCallback((agentId: string | null) => {
|
||||||
setSelectedAgentId(agentId);
|
setSelectedAgentId(agentId);
|
||||||
setSelectedKnowledgeBaseIds([]);
|
setSelectedKnowledgeBaseIds([]);
|
||||||
|
setSelectedAssistantKnowledgeBaseIds([]);
|
||||||
setKnowledgeSelectionReady(false);
|
setKnowledgeSelectionReady(false);
|
||||||
saveLastAgentId(agentId);
|
saveLastAgentId(agentId);
|
||||||
}, []);
|
}, []);
|
||||||
@@ -842,7 +875,9 @@ export default function ChatPage() {
|
|||||||
source={selectedAgentId ? "agent" : "all"}
|
source={selectedAgentId ? "agent" : "all"}
|
||||||
agentId={selectedAgentId}
|
agentId={selectedAgentId}
|
||||||
value={selectedKnowledgeBaseIds}
|
value={selectedKnowledgeBaseIds}
|
||||||
|
assistantValue={selectedAssistantKnowledgeBaseIds}
|
||||||
onChange={changeKnowledgeSelection}
|
onChange={changeKnowledgeSelection}
|
||||||
|
onAssistantChange={changeAssistantKnowledgeSelection}
|
||||||
autoSelectAll={isLandingState && !!selectedAgentId}
|
autoSelectAll={isLandingState && !!selectedAgentId}
|
||||||
disabled={
|
disabled={
|
||||||
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" ||
|
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" ||
|
||||||
|
|||||||
@@ -3,7 +3,11 @@ import { useState } from "react";
|
|||||||
import { Navigate, useParams, useSearchParams } from "react-router-dom";
|
import { Navigate, useParams, useSearchParams } from "react-router-dom";
|
||||||
|
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
import { KnowledgeProductNav, useKnowledgeBaseProductVersion } from "@/components/workspace/knowledge-product-nav";
|
import {
|
||||||
|
KNOWLEDGE_NORMAL_VIEW_VALUE,
|
||||||
|
KnowledgeProductNav,
|
||||||
|
useKnowledgeBaseProductVersion,
|
||||||
|
} from "@/components/workspace/knowledge-product-nav";
|
||||||
import {
|
import {
|
||||||
Dialog,
|
Dialog,
|
||||||
DialogContent,
|
DialogContent,
|
||||||
@@ -73,6 +77,7 @@ export default function LlmWikiKnowledgePage({
|
|||||||
const runtime = useLlmWikiRuntime();
|
const runtime = useLlmWikiRuntime();
|
||||||
const productVersion = useKnowledgeBaseProductVersion();
|
const productVersion = useKnowledgeBaseProductVersion();
|
||||||
const wikiSlug = searchParams.get("wikiSlug")?.trim() || "";
|
const wikiSlug = searchParams.get("wikiSlug")?.trim() || "";
|
||||||
|
const explicitNormalView = searchParams.get("view") === KNOWLEDGE_NORMAL_VIEW_VALUE;
|
||||||
|
|
||||||
if (mode === "detail" && noteId && wikiSlug) {
|
if (mode === "detail" && noteId && wikiSlug) {
|
||||||
return <Navigate to={buildWeKnoraWikiPagePath(noteId, wikiSlug)} replace />;
|
return <Navigate to={buildWeKnoraWikiPagePath(noteId, wikiSlug)} replace />;
|
||||||
@@ -82,15 +87,18 @@ export default function LlmWikiKnowledgePage({
|
|||||||
return <RuntimeLoading />;
|
return <RuntimeLoading />;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (mode === "list" && productVersion.data === "v2") {
|
if (mode === "list" && productVersion.data === "v2" && !explicitNormalView) {
|
||||||
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
|
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (runtime.isError || runtime.data?.provider !== "weknora") {
|
if (runtime.isError || runtime.data?.provider !== "weknora") {
|
||||||
|
if (mode === "list" || mode === "detail") {
|
||||||
|
return <Navigate to="/page/workspace/knowledge/assistant" replace />;
|
||||||
|
}
|
||||||
if (mode === "graph") {
|
if (mode === "graph") {
|
||||||
return <KnowledgeGraphPage />;
|
return <KnowledgeGraphPage />;
|
||||||
}
|
}
|
||||||
return mode === "list" ? <div className="flex size-full min-h-0 flex-col"><KnowledgeProductNav /><div className="min-h-0 flex-1"><LegacyKnowledgePage /></div></div> : <LegacyKnowledgePage />;
|
return <LegacyKnowledgePage />;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (mode === "graph") {
|
if (mode === "graph") {
|
||||||
|
|||||||
@@ -450,7 +450,13 @@ export default function WorkspaceRoutes({
|
|||||||
<Route
|
<Route
|
||||||
path="library"
|
path="library"
|
||||||
element={
|
element={
|
||||||
<WorkspaceLayout {...layoutProps} workspaceSidebarHidden>
|
<WorkspaceLayout
|
||||||
|
{...layoutProps}
|
||||||
|
workspaceSidebarHidden={
|
||||||
|
layoutProps.workspaceSidebarHidden ||
|
||||||
|
!layoutProps.pageSidebarHidden
|
||||||
|
}
|
||||||
|
>
|
||||||
<ArtifactLibraryPage />
|
<ArtifactLibraryPage />
|
||||||
</WorkspaceLayout>
|
</WorkspaceLayout>
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,13 +8,21 @@ const BASE = () => `${getBackendBaseURL()}/api/assistant-knowledge`;
|
|||||||
export interface AssistantKnowledgeBase {
|
export interface AssistantKnowledgeBase {
|
||||||
id: string;
|
id: string;
|
||||||
name: string;
|
name: string;
|
||||||
|
description?: string;
|
||||||
|
base_kind?: "global" | "custom";
|
||||||
|
is_global?: boolean;
|
||||||
status: string;
|
status: string;
|
||||||
|
source_count?: number;
|
||||||
|
page_count?: number;
|
||||||
|
entity_count?: number;
|
||||||
|
relation_count?: number;
|
||||||
created_at: string;
|
created_at: string;
|
||||||
updated_at: string;
|
updated_at: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface AssistantWikiPage {
|
export interface AssistantWikiPage {
|
||||||
id: string;
|
id: string;
|
||||||
|
base_id: string;
|
||||||
slug: string;
|
slug: string;
|
||||||
title: string;
|
title: string;
|
||||||
page_type: string;
|
page_type: string;
|
||||||
@@ -37,6 +45,7 @@ export interface AssistantWikiRevision {
|
|||||||
|
|
||||||
export interface AssistantKnowledgeSource {
|
export interface AssistantKnowledgeSource {
|
||||||
id: string;
|
id: string;
|
||||||
|
base_id: string;
|
||||||
source_type: "weknora" | "skill" | "local_file";
|
source_type: "weknora" | "skill" | "local_file";
|
||||||
source_name: string;
|
source_name: string;
|
||||||
source_key: string;
|
source_key: string;
|
||||||
@@ -46,12 +55,34 @@ export interface AssistantKnowledgeSource {
|
|||||||
|
|
||||||
export interface AssistantImportJob {
|
export interface AssistantImportJob {
|
||||||
id: string;
|
id: string;
|
||||||
|
base_id: string;
|
||||||
source_id: string;
|
source_id: string;
|
||||||
|
mirrored_global_base_id?: string;
|
||||||
|
mirrored_global_job_id?: string;
|
||||||
|
package_download_url?: string;
|
||||||
trigger: string;
|
trigger: string;
|
||||||
status: string;
|
status: string;
|
||||||
phase: string;
|
phase: string;
|
||||||
counts: Record<string, number>;
|
counts: Record<string, number>;
|
||||||
|
last_error_message?: string | null;
|
||||||
created_at: string;
|
created_at: string;
|
||||||
|
completed_at?: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface LlmWikiExportPackageJob {
|
||||||
|
id: string;
|
||||||
|
mapping_id: string;
|
||||||
|
status: string;
|
||||||
|
phase: string;
|
||||||
|
counts: Record<string, number>;
|
||||||
|
package_download_url?: string;
|
||||||
|
status_url?: string;
|
||||||
|
error?: string | null;
|
||||||
|
created_at?: string | null;
|
||||||
|
updated_at?: string | null;
|
||||||
|
completed_at?: string | null;
|
||||||
|
file_exists?: boolean;
|
||||||
|
file_size_bytes?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function readJson<T>(response: Response): Promise<T> {
|
async function readJson<T>(response: Response): Promise<T> {
|
||||||
@@ -65,12 +96,41 @@ async function readJson<T>(response: Response): Promise<T> {
|
|||||||
return (await response.json()) as T;
|
return (await response.json()) as T;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function normalizeImportJob(job: AssistantImportJob): AssistantImportJob {
|
||||||
|
const url = normalizeBackendUrl(job.package_download_url);
|
||||||
|
return url ? { ...job, package_download_url: url } : job;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeBackendUrl(url: string | undefined | null): string | undefined {
|
||||||
|
if (!url || /^https?:\/\//i.test(url)) return url || undefined;
|
||||||
|
return `${getBackendBaseURL()}${url.startsWith("/") ? url : `/${url}`}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeExportPackageJob(job: LlmWikiExportPackageJob): LlmWikiExportPackageJob {
|
||||||
|
return {
|
||||||
|
...job,
|
||||||
|
package_download_url: normalizeBackendUrl(job.package_download_url),
|
||||||
|
status_url: normalizeBackendUrl(job.status_url),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
export async function getAssistantKnowledgeBase() {
|
export async function getAssistantKnowledgeBase() {
|
||||||
const data = await readJson<{ base: AssistantKnowledgeBase | null }>(await apiFetch(`${BASE()}/base`));
|
const data = await readJson<{ base: AssistantKnowledgeBase | null }>(await apiFetch(`${BASE()}/base`));
|
||||||
return data.base;
|
return data.base;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function initializeAssistantKnowledgeBase(name = "知识梳理") {
|
export async function getAssistantKnowledgeBaseById(baseId: string) {
|
||||||
|
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
||||||
|
await apiFetch(`${BASE()}/bases/${encodeURIComponent(baseId)}`),
|
||||||
|
);
|
||||||
|
return data.base;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function listAssistantKnowledgeBases() {
|
||||||
|
return (await readJson<{ bases: AssistantKnowledgeBase[] }>(await apiFetch(`${BASE()}/bases`))).bases;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function initializeAssistantKnowledgeBase(name = "知识梳理总库") {
|
||||||
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
||||||
await apiFetch(`${BASE()}/base/initialize`, {
|
await apiFetch(`${BASE()}/base/initialize`, {
|
||||||
method: "POST",
|
method: "POST",
|
||||||
@@ -81,58 +141,124 @@ export async function initializeAssistantKnowledgeBase(name = "知识梳理") {
|
|||||||
return data.base;
|
return data.base;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function listAssistantWikiPages(q = "") {
|
export async function createAssistantKnowledgeBase(input: { name: string; description?: string }) {
|
||||||
|
const data = await readJson<{ base: AssistantKnowledgeBase }>(
|
||||||
|
await apiFetch(`${BASE()}/bases`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(input),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
return data.base;
|
||||||
|
}
|
||||||
|
|
||||||
|
function withBaseParam(baseId?: string | null) {
|
||||||
const params = new URLSearchParams();
|
const params = new URLSearchParams();
|
||||||
|
if (baseId) params.set("base_id", baseId);
|
||||||
|
return params;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function listAssistantWikiPages(q = "", baseId?: string | null) {
|
||||||
|
const params = withBaseParam(baseId);
|
||||||
if (q.trim()) params.set("q", q.trim());
|
if (q.trim()) params.set("q", q.trim());
|
||||||
return (await readJson<{ pages: AssistantWikiPage[] }>(await apiFetch(`${BASE()}/wiki-pages?${params}`))).pages;
|
return (await readJson<{ pages: AssistantWikiPage[] }>(await apiFetch(`${BASE()}/wiki-pages?${params}`))).pages;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function listAssistantKnowledgeSources() {
|
export async function listAssistantKnowledgeSources(baseId?: string | null) {
|
||||||
return (await readJson<{ sources: AssistantKnowledgeSource[] }>(await apiFetch(`${BASE()}/sources`))).sources;
|
const params = withBaseParam(baseId);
|
||||||
|
return (await readJson<{ sources: AssistantKnowledgeSource[] }>(await apiFetch(`${BASE()}/sources?${params}`))).sources;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function listAssistantImportJobs() {
|
export async function listAssistantImportJobs(baseId?: string | null) {
|
||||||
return (await readJson<{ jobs: AssistantImportJob[] }>(await apiFetch(`${BASE()}/import-jobs`))).jobs;
|
const params = withBaseParam(baseId);
|
||||||
|
return (
|
||||||
|
await readJson<{ jobs: AssistantImportJob[] }>(await apiFetch(`${BASE()}/import-jobs?${params}`))
|
||||||
|
).jobs.map(normalizeImportJob);
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function listAssistantWikiRevisions(slug: string) {
|
export async function listAssistantWikiRevisions(slug: string, baseId?: string | null) {
|
||||||
|
const params = withBaseParam(baseId);
|
||||||
return (await readJson<{ revisions: AssistantWikiRevision[] }>(
|
return (await readJson<{ revisions: AssistantWikiRevision[] }>(
|
||||||
await apiFetch(`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions`),
|
await apiFetch(`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions?${params}`),
|
||||||
)).revisions;
|
)).revisions;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function rollbackAssistantWikiRevision(slug: string, revisionId: string) {
|
export async function rollbackAssistantWikiRevision(slug: string, revisionId: string, baseId?: string | null) {
|
||||||
|
const params = withBaseParam(baseId);
|
||||||
return readJson<AssistantWikiRevision>(
|
return readJson<AssistantWikiRevision>(
|
||||||
await apiFetch(
|
await apiFetch(
|
||||||
`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions/${encodeURIComponent(revisionId)}/rollback`,
|
`${BASE()}/wiki-pages/${encodeURIComponent(slug)}/revisions/${encodeURIComponent(revisionId)}/rollback?${params}`,
|
||||||
{ method: "POST" },
|
{ method: "POST" },
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function uploadAssistantKnowledgeFiles(files: File[]) {
|
export async function exportLlmWikiToAssistant(mappingId: string, assistantBaseId?: string | null) {
|
||||||
const form = new FormData();
|
return normalizeImportJob(await readJson<AssistantImportJob>(
|
||||||
files.forEach((file) => form.append("files", file));
|
await apiFetch(
|
||||||
return readJson<{ batch_id: string; jobs: AssistantImportJob[] }>(
|
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-to-assistant`,
|
||||||
await apiFetch(`${BASE()}/imports/files`, { method: "POST", body: form }),
|
{
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ assistant_base_id: assistantBaseId || null }),
|
||||||
|
},
|
||||||
|
),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function createLlmWikiExportPackage(mappingId: string) {
|
||||||
|
return normalizeExportPackageJob(
|
||||||
|
await readJson<LlmWikiExportPackageJob>(
|
||||||
|
await apiFetch(
|
||||||
|
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package`,
|
||||||
|
{ method: "POST" },
|
||||||
|
),
|
||||||
|
),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function exportLlmWikiToAssistant(mappingId: string) {
|
export async function listLlmWikiExportPackages(mappingId: string) {
|
||||||
return readJson<AssistantImportJob>(
|
return (
|
||||||
await apiFetch(
|
await readJson<{ jobs: LlmWikiExportPackageJob[] }>(
|
||||||
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-to-assistant`,
|
await apiFetch(
|
||||||
{ method: "POST" },
|
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package`,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
).jobs.map(normalizeExportPackageJob);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function getLlmWikiExportPackageStatus(mappingId: string, jobId: string) {
|
||||||
|
return normalizeExportPackageJob(
|
||||||
|
await readJson<LlmWikiExportPackageJob>(
|
||||||
|
await apiFetch(
|
||||||
|
`${getBackendBaseURL()}/api/llmwiki/knowledge-bases/${encodeURIComponent(mappingId)}/export-package/${encodeURIComponent(jobId)}`,
|
||||||
|
),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function reimportAssistantKnowledgeSource(sourceId: string) {
|
export async function reimportAssistantKnowledgeSource(sourceId: string) {
|
||||||
return readJson<AssistantImportJob>(
|
return normalizeImportJob(await readJson<AssistantImportJob>(
|
||||||
await apiFetch(`${BASE()}/sources/${encodeURIComponent(sourceId)}/reimport`, {
|
await apiFetch(`${BASE()}/sources/${encodeURIComponent(sourceId)}/reimport`, {
|
||||||
method: "POST",
|
method: "POST",
|
||||||
}),
|
}),
|
||||||
);
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getAssistantImportPackageUrl(jobId: string) {
|
||||||
|
return `${BASE()}/import-jobs/${encodeURIComponent(jobId)}/package`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function uploadAssistantWeKnoraPackage(input: { file: File; assistantBaseId?: string | null }) {
|
||||||
|
const body = new FormData();
|
||||||
|
body.set("file", input.file);
|
||||||
|
if (input.assistantBaseId) body.set("assistant_base_id", input.assistantBaseId);
|
||||||
|
return normalizeImportJob(await readJson<AssistantImportJob>(
|
||||||
|
await apiFetch(`${BASE()}/imports/weknora-package`, {
|
||||||
|
method: "POST",
|
||||||
|
body,
|
||||||
|
}),
|
||||||
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function deleteAssistantKnowledgeSource(sourceId: string) {
|
export async function deleteAssistantKnowledgeSource(sourceId: string) {
|
||||||
@@ -144,20 +270,40 @@ export async function deleteAssistantKnowledgeSource(sourceId: string) {
|
|||||||
|
|
||||||
const KEY = "assistant-knowledge";
|
const KEY = "assistant-knowledge";
|
||||||
|
|
||||||
export function useAssistantKnowledgeBase() {
|
export function useAssistantKnowledgeBase(baseId?: string | null) {
|
||||||
return useQuery({ queryKey: [KEY, "base"], queryFn: getAssistantKnowledgeBase });
|
return useQuery({
|
||||||
|
queryKey: [KEY, "base", baseId ?? "global"],
|
||||||
|
queryFn: () => (baseId ? getAssistantKnowledgeBaseById(baseId) : getAssistantKnowledgeBase()),
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
export function useAssistantWikiPages(query = "") {
|
export function useAssistantKnowledgeBases() {
|
||||||
return useQuery({ queryKey: [KEY, "pages", query], queryFn: () => listAssistantWikiPages(query) });
|
return useQuery({ queryKey: [KEY, "bases"], queryFn: listAssistantKnowledgeBases });
|
||||||
}
|
}
|
||||||
|
|
||||||
export function useAssistantKnowledgeSources() {
|
export function useAssistantWikiPages(query = "", baseId?: string | null) {
|
||||||
return useQuery({ queryKey: [KEY, "sources"], queryFn: listAssistantKnowledgeSources });
|
return useQuery({
|
||||||
|
queryKey: [KEY, "pages", baseId ?? "global", query],
|
||||||
|
queryFn: () => listAssistantWikiPages(query, baseId),
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
export function useAssistantImportJobs() {
|
export function useAssistantKnowledgeSources(baseId?: string | null) {
|
||||||
return useQuery({ queryKey: [KEY, "jobs"], queryFn: listAssistantImportJobs });
|
return useQuery({
|
||||||
|
queryKey: [KEY, "sources", baseId ?? "global"],
|
||||||
|
queryFn: () => listAssistantKnowledgeSources(baseId),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function useAssistantImportJobs(baseId?: string | null) {
|
||||||
|
return useQuery({
|
||||||
|
queryKey: [KEY, "jobs", baseId ?? "global"],
|
||||||
|
queryFn: () => listAssistantImportJobs(baseId),
|
||||||
|
refetchInterval: (query) =>
|
||||||
|
(query.state.data ?? []).some((job) => ["queued", "running"].includes(job.status))
|
||||||
|
? 3000
|
||||||
|
: false,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
export function useAssistantKnowledgeActions() {
|
export function useAssistantKnowledgeActions() {
|
||||||
@@ -165,12 +311,13 @@ export function useAssistantKnowledgeActions() {
|
|||||||
const invalidate = () => client.invalidateQueries({ queryKey: [KEY] });
|
const invalidate = () => client.invalidateQueries({ queryKey: [KEY] });
|
||||||
return {
|
return {
|
||||||
initialize: useMutation({ mutationFn: initializeAssistantKnowledgeBase, onSuccess: invalidate }),
|
initialize: useMutation({ mutationFn: initializeAssistantKnowledgeBase, onSuccess: invalidate }),
|
||||||
upload: useMutation({ mutationFn: uploadAssistantKnowledgeFiles, onSuccess: invalidate }),
|
createBase: useMutation({ mutationFn: createAssistantKnowledgeBase, onSuccess: invalidate }),
|
||||||
|
uploadWeKnoraPackage: useMutation({ mutationFn: uploadAssistantWeKnoraPackage, onSuccess: invalidate }),
|
||||||
reimportSource: useMutation({ mutationFn: reimportAssistantKnowledgeSource, onSuccess: invalidate }),
|
reimportSource: useMutation({ mutationFn: reimportAssistantKnowledgeSource, onSuccess: invalidate }),
|
||||||
deleteSource: useMutation({ mutationFn: deleteAssistantKnowledgeSource, onSuccess: invalidate }),
|
deleteSource: useMutation({ mutationFn: deleteAssistantKnowledgeSource, onSuccess: invalidate }),
|
||||||
rollback: useMutation({
|
rollback: useMutation({
|
||||||
mutationFn: ({ slug, revisionId }: { slug: string; revisionId: string }) =>
|
mutationFn: ({ slug, revisionId, baseId }: { slug: string; revisionId: string; baseId?: string | null }) =>
|
||||||
rollbackAssistantWikiRevision(slug, revisionId),
|
rollbackAssistantWikiRevision(slug, revisionId, baseId),
|
||||||
onSuccess: invalidate,
|
onSuccess: invalidate,
|
||||||
}),
|
}),
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -40,6 +40,9 @@ export interface LlmWikiKnowledgeBase {
|
|||||||
share_count?: number;
|
share_count?: number;
|
||||||
is_processing?: boolean;
|
is_processing?: boolean;
|
||||||
is_temporary?: boolean;
|
is_temporary?: boolean;
|
||||||
|
wiki_index_enabled?: boolean;
|
||||||
|
external_search_enabled?: boolean;
|
||||||
|
external_search_updated_at?: string | null;
|
||||||
embedding_model_id?: string | null;
|
embedding_model_id?: string | null;
|
||||||
vector_store_id?: string | null;
|
vector_store_id?: string | null;
|
||||||
vector_store_name?: string | null;
|
vector_store_name?: string | null;
|
||||||
|
|||||||
+186
-24
@@ -1,15 +1,11 @@
|
|||||||
import { AlertCircle, Database, Edit3, FileText, Loader2, RefreshCw, Trash2 } from "lucide-react";
|
import { AlertCircle, Download, Edit3, FileText, Loader2, RefreshCw, Trash2 } from "lucide-react";
|
||||||
import { useEffect, useRef, useState } from "react";
|
import { useCallback, useEffect, useRef, useState } from "react";
|
||||||
import { Navigate, useNavigate, useParams, useSearchParams } from "react-router-dom";
|
import { Navigate, useNavigate, useParams, useSearchParams } from "react-router-dom";
|
||||||
import { toast } from "sonner";
|
import { toast } from "sonner";
|
||||||
|
|
||||||
|
import { Badge } from "@/components/ui/badge";
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
import {
|
import { Dialog, DialogContent, DialogHeader, DialogTitle } from "@/components/ui/dialog";
|
||||||
Dialog,
|
|
||||||
DialogContent,
|
|
||||||
DialogHeader,
|
|
||||||
DialogTitle,
|
|
||||||
} from "@/components/ui/dialog";
|
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
import { Textarea } from "@/components/ui/textarea";
|
import { Textarea } from "@/components/ui/textarea";
|
||||||
import { MarkdownCodeEditor } from "@/components/wiki/MarkdownCodeEditor";
|
import { MarkdownCodeEditor } from "@/components/wiki/MarkdownCodeEditor";
|
||||||
@@ -24,9 +20,20 @@ import {
|
|||||||
useLlmWikiWikiPage,
|
useLlmWikiWikiPage,
|
||||||
useLlmWikiRuntime,
|
useLlmWikiRuntime,
|
||||||
} from "@/strategy-components/api/llmwiki";
|
} from "@/strategy-components/api/llmwiki";
|
||||||
import { exportLlmWikiToAssistant } from "@/strategy-components/api/assistant-knowledge";
|
import {
|
||||||
|
createLlmWikiExportPackage,
|
||||||
|
listLlmWikiExportPackages,
|
||||||
|
type LlmWikiExportPackageJob,
|
||||||
|
} from "@/strategy-components/api/assistant-knowledge";
|
||||||
|
|
||||||
const LIST_PATH = "/page/workspace/knowledge";
|
const LIST_PATH = "/page/workspace/knowledge";
|
||||||
|
const RUNNING_EXPORT_STATUSES = new Set(["queued", "running"]);
|
||||||
|
const EXPORT_STATUS_LABELS: Record<string, string> = {
|
||||||
|
queued: "排队中",
|
||||||
|
running: "整理中",
|
||||||
|
completed: "已完成",
|
||||||
|
failed: "失败",
|
||||||
|
};
|
||||||
|
|
||||||
type DeerFlowThemePayload = {
|
type DeerFlowThemePayload = {
|
||||||
type: "deerflow-weknora-theme";
|
type: "deerflow-weknora-theme";
|
||||||
@@ -94,6 +101,25 @@ function buildFrameUrl(framePath: string) {
|
|||||||
return new URL(`${backendBase || backendOrigin}/${framePath.replace(/^\/+/, "")}`);
|
return new URL(`${backendBase || backendOrigin}/${framePath.replace(/^\/+/, "")}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function formatBytes(value: number | undefined | null) {
|
||||||
|
const bytes = Number(value || 0);
|
||||||
|
if (!Number.isFinite(bytes) || bytes <= 0) return "—";
|
||||||
|
const units = ["B", "KB", "MB", "GB", "TB"];
|
||||||
|
let current = bytes;
|
||||||
|
let unitIndex = 0;
|
||||||
|
while (current >= 1024 && unitIndex < units.length - 1) {
|
||||||
|
current /= 1024;
|
||||||
|
unitIndex += 1;
|
||||||
|
}
|
||||||
|
return `${current >= 10 || unitIndex === 0 ? current.toFixed(0) : current.toFixed(1)} ${units[unitIndex]}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatTime(value: string | undefined | null) {
|
||||||
|
if (!value) return "—";
|
||||||
|
const time = new Date(value);
|
||||||
|
return Number.isNaN(time.getTime()) ? "—" : time.toLocaleString();
|
||||||
|
}
|
||||||
|
|
||||||
export function WeKnoraKnowledgeDetailPage() {
|
export function WeKnoraKnowledgeDetailPage() {
|
||||||
const {
|
const {
|
||||||
knowledge_base_id: strategyKnowledgeBaseId,
|
knowledge_base_id: strategyKnowledgeBaseId,
|
||||||
@@ -106,16 +132,40 @@ export function WeKnoraKnowledgeDetailPage() {
|
|||||||
const [frameSrc, setFrameSrc] = useState("");
|
const [frameSrc, setFrameSrc] = useState("");
|
||||||
const [frameVersion, setFrameVersion] = useState(0);
|
const [frameVersion, setFrameVersion] = useState(0);
|
||||||
const [frameError, setFrameError] = useState<string | null>(null);
|
const [frameError, setFrameError] = useState<string | null>(null);
|
||||||
const [assistantImporting, setAssistantImporting] = useState(false);
|
const [exportPackageLoading, setExportPackageLoading] = useState(false);
|
||||||
|
const [exportJobsLoading, setExportJobsLoading] = useState(false);
|
||||||
|
const [exportJobs, setExportJobs] = useState<LlmWikiExportPackageJob[]>([]);
|
||||||
|
const isAdmin = getStoredAuth()?.system_role === "admin";
|
||||||
const runtime = useLlmWikiRuntime();
|
const runtime = useLlmWikiRuntime();
|
||||||
const details = useLlmWikiKnowledgeBase(knowledgeBaseId || null, !!knowledgeBaseId);
|
const details = useLlmWikiKnowledgeBase(knowledgeBaseId || null, !!knowledgeBaseId);
|
||||||
|
|
||||||
|
const refreshExportJobs = useCallback(
|
||||||
|
async (silent = false) => {
|
||||||
|
if (!knowledgeBaseId || !isAdmin) return;
|
||||||
|
if (!silent) setExportJobsLoading(true);
|
||||||
|
try {
|
||||||
|
setExportJobs(await listLlmWikiExportPackages(knowledgeBaseId));
|
||||||
|
} catch (error) {
|
||||||
|
if (!silent) toast.error(error instanceof Error ? error.message : String(error));
|
||||||
|
} finally {
|
||||||
|
if (!silent) setExportJobsLoading(false);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[isAdmin, knowledgeBaseId],
|
||||||
|
);
|
||||||
|
|
||||||
const detailsReady = !!details.data;
|
const detailsReady = !!details.data;
|
||||||
const requestedTab = searchParams.get("tab");
|
const requestedTab = searchParams.get("tab");
|
||||||
|
const detailsWikiEnabled =
|
||||||
|
details.data?.type === "wiki" ||
|
||||||
|
details.data?.wiki_index_enabled === true ||
|
||||||
|
details.data?.indexing_strategy?.wiki_enabled === true ||
|
||||||
|
details.data?.capabilities?.wiki_enabled === true ||
|
||||||
|
(details.data?.wiki_config && Object.keys(details.data.wiki_config).length > 0);
|
||||||
const preferredTab =
|
const preferredTab =
|
||||||
requestedTab === "documents" || requestedTab === "wiki" || requestedTab === "graph"
|
requestedTab === "documents" || requestedTab === "wiki" || requestedTab === "graph"
|
||||||
? requestedTab
|
? requestedTab
|
||||||
: details.data?.indexing_strategy?.wiki_enabled
|
: detailsWikiEnabled
|
||||||
? "wiki"
|
? "wiki"
|
||||||
: "documents";
|
: "documents";
|
||||||
const sourceChunkId = searchParams.get("sourceChunkId") || searchParams.get("chunkId") || "";
|
const sourceChunkId = searchParams.get("sourceChunkId") || searchParams.get("chunkId") || "";
|
||||||
@@ -182,6 +232,22 @@ export function WeKnoraKnowledgeDetailPage() {
|
|||||||
};
|
};
|
||||||
}, [frameSrc]);
|
}, [frameSrc]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!isAdmin || !knowledgeBaseId || runtime.data?.provider !== "weknora") {
|
||||||
|
setExportJobs([]);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
void refreshExportJobs();
|
||||||
|
}, [isAdmin, knowledgeBaseId, refreshExportJobs, runtime.data?.provider]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!isAdmin || !knowledgeBaseId || !exportJobs.some((job) => RUNNING_EXPORT_STATUSES.has(job.status))) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const timer = window.setInterval(() => void refreshExportJobs(true), 5000);
|
||||||
|
return () => window.clearInterval(timer);
|
||||||
|
}, [exportJobs, isAdmin, knowledgeBaseId, refreshExportJobs]);
|
||||||
|
|
||||||
if (nativeWikiPageUrl) {
|
if (nativeWikiPageUrl) {
|
||||||
return <Navigate to={nativeWikiPageUrl} replace />;
|
return <Navigate to={nativeWikiPageUrl} replace />;
|
||||||
}
|
}
|
||||||
@@ -235,8 +301,25 @@ export function WeKnoraKnowledgeDetailPage() {
|
|||||||
|
|
||||||
const canWrite = details.data.can_write && !details.data.is_conversation_deposit;
|
const canWrite = details.data.can_write && !details.data.is_conversation_deposit;
|
||||||
const canEditCurrentWikiArticle =
|
const canEditCurrentWikiArticle =
|
||||||
canWrite && !!details.data.indexing_strategy?.wiki_enabled && !!wikiSlug;
|
canWrite && !!detailsWikiEnabled && !!wikiSlug;
|
||||||
const canManageWiki = canWrite && !!details.data.indexing_strategy?.wiki_enabled;
|
const canManageWiki = canWrite && !!detailsWikiEnabled;
|
||||||
|
|
||||||
|
const createVectorizedPackage = () => {
|
||||||
|
if (exportPackageLoading) return;
|
||||||
|
setExportPackageLoading(true);
|
||||||
|
void (async () => {
|
||||||
|
try {
|
||||||
|
const job = await createLlmWikiExportPackage(knowledgeBaseId);
|
||||||
|
setExportJobs((current) => [job, ...current.filter((item) => item.id !== job.id)]);
|
||||||
|
toast.success("已创建向量化数据包整理记录,完成后可在下方列表下载");
|
||||||
|
void refreshExportJobs(true);
|
||||||
|
} catch (error) {
|
||||||
|
toast.error(error instanceof Error ? error.message : String(error));
|
||||||
|
} finally {
|
||||||
|
setExportPackageLoading(false);
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
};
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="flex h-full min-h-0 flex-col bg-background">
|
<div className="flex h-full min-h-0 flex-col bg-background">
|
||||||
@@ -258,21 +341,15 @@ export function WeKnoraKnowledgeDetailPage() {
|
|||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
<div className="flex shrink-0 items-center gap-2">
|
<div className="flex shrink-0 items-center gap-2">
|
||||||
{getStoredAuth()?.system_role === "admin" && (
|
{isAdmin && (
|
||||||
<Button
|
<Button
|
||||||
size="sm"
|
size="sm"
|
||||||
variant="outline"
|
variant="outline"
|
||||||
disabled={assistantImporting}
|
disabled={exportPackageLoading}
|
||||||
onClick={() => {
|
onClick={createVectorizedPackage}
|
||||||
setAssistantImporting(true);
|
|
||||||
void exportLlmWikiToAssistant(knowledgeBaseId)
|
|
||||||
.then(() => toast.success("已导入到助手知识库“知识梳理”"))
|
|
||||||
.catch((error: unknown) => toast.error(error instanceof Error ? error.message : String(error)))
|
|
||||||
.finally(() => setAssistantImporting(false));
|
|
||||||
}}
|
|
||||||
>
|
>
|
||||||
{assistantImporting ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Database className="mr-2 size-4" />}
|
{exportPackageLoading ? <Loader2 className="mr-2 size-4 animate-spin" /> : <Download className="mr-2 size-4" />}
|
||||||
导入到助手知识库
|
生成向量化数据包
|
||||||
</Button>
|
</Button>
|
||||||
)}
|
)}
|
||||||
<Button
|
<Button
|
||||||
@@ -286,6 +363,13 @@ export function WeKnoraKnowledgeDetailPage() {
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
{canManageWiki && isAdmin && (
|
||||||
|
<ExportPackageJobsPanel
|
||||||
|
jobs={exportJobs}
|
||||||
|
loading={exportJobsLoading}
|
||||||
|
onRefresh={() => void refreshExportJobs()}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
{frameSrc && (
|
{frameSrc && (
|
||||||
<iframe
|
<iframe
|
||||||
key={`${frameSrc}:${frameVersion}`}
|
key={`${frameSrc}:${frameVersion}`}
|
||||||
@@ -313,6 +397,84 @@ export function WeKnoraKnowledgeDetailPage() {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function ExportPackageJobsPanel({
|
||||||
|
jobs,
|
||||||
|
loading,
|
||||||
|
onRefresh,
|
||||||
|
}: {
|
||||||
|
jobs: LlmWikiExportPackageJob[];
|
||||||
|
loading: boolean;
|
||||||
|
onRefresh: () => void;
|
||||||
|
}) {
|
||||||
|
return (
|
||||||
|
<div className="shrink-0 border-b bg-muted/20 px-4 py-3">
|
||||||
|
<div className="mb-2 flex items-center justify-between">
|
||||||
|
<div>
|
||||||
|
<div className="text-sm font-medium">向量化数据包记录</div>
|
||||||
|
<p className="text-xs text-muted-foreground">
|
||||||
|
每次点击都会生成一条后台整理记录;文件较大时,请等待状态变为“已完成”后再下载。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<Button size="sm" variant="ghost" onClick={onRefresh} disabled={loading}>
|
||||||
|
{loading ? <Loader2 className="mr-2 size-4 animate-spin" /> : <RefreshCw className="mr-2 size-4" />}
|
||||||
|
刷新
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
<div className="max-h-44 space-y-2 overflow-y-auto pr-1">
|
||||||
|
{jobs.map((job) => {
|
||||||
|
const running = RUNNING_EXPORT_STATUSES.has(job.status);
|
||||||
|
const completed = job.status === "completed";
|
||||||
|
const canDownload = completed && !!job.package_download_url && job.file_exists !== false;
|
||||||
|
const counts = job.counts ?? {};
|
||||||
|
return (
|
||||||
|
<div key={job.id} className="flex items-center gap-3 rounded-md border bg-card px-3 py-2 text-xs">
|
||||||
|
<Badge
|
||||||
|
variant={job.status === "failed" ? "destructive" : completed ? "default" : "secondary"}
|
||||||
|
className="shrink-0"
|
||||||
|
>
|
||||||
|
{running && <Loader2 className="mr-1 inline size-3 animate-spin" />}
|
||||||
|
{EXPORT_STATUS_LABELS[job.status] || job.status}
|
||||||
|
</Badge>
|
||||||
|
<div className="min-w-0 flex-1">
|
||||||
|
<div className="flex flex-wrap items-center gap-x-3 gap-y-1">
|
||||||
|
<span className="font-medium">任务 {job.id.slice(0, 8)}</span>
|
||||||
|
<span className="text-muted-foreground">创建:{formatTime(job.created_at)}</span>
|
||||||
|
<span className="text-muted-foreground">大小:{formatBytes(job.file_size_bytes)}</span>
|
||||||
|
</div>
|
||||||
|
<div className="mt-1 flex flex-wrap gap-x-3 gap-y-1 text-muted-foreground">
|
||||||
|
<span>Wiki {counts.wiki_page ?? counts.wiki_pages ?? 0}</span>
|
||||||
|
<span>文档 {counts.document ?? counts.documents ?? 0}</span>
|
||||||
|
<span>片段 {counts.chunk ?? counts.chunks ?? 0}</span>
|
||||||
|
<span>向量 {counts.vector ?? counts.vectors ?? 0}</span>
|
||||||
|
<span>阶段:{job.phase}</span>
|
||||||
|
{job.error && <span className="text-destructive">{job.error}</span>}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{canDownload ? (
|
||||||
|
<Button asChild size="sm" variant="outline" className="shrink-0">
|
||||||
|
<a href={job.package_download_url} target="_blank" rel="noreferrer">
|
||||||
|
<Download className="mr-2 size-4" />
|
||||||
|
下载
|
||||||
|
</a>
|
||||||
|
</Button>
|
||||||
|
) : (
|
||||||
|
<Button size="sm" variant="outline" className="shrink-0" disabled>
|
||||||
|
{completed && job.file_exists === false ? "文件不存在" : "等待完成"}
|
||||||
|
</Button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
{!jobs.length && (
|
||||||
|
<div className="rounded-md border border-dashed bg-card px-3 py-6 text-center text-xs text-muted-foreground">
|
||||||
|
暂无导出记录。点击上方“生成向量化数据包”后,任务会显示在这里。
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
function CurrentWikiArticleActions({
|
function CurrentWikiArticleActions({
|
||||||
knowledgeBaseId,
|
knowledgeBaseId,
|
||||||
slug,
|
slug,
|
||||||
|
|||||||
@@ -1623,23 +1623,40 @@ save fallback.
|
|||||||
## Skill/assistant knowledge invariants
|
## Skill/assistant knowledge invariants
|
||||||
|
|
||||||
Skill distillation is implemented in `deerflow.skill_knowledge` and
|
Skill distillation is implemented in `deerflow.skill_knowledge` and
|
||||||
`deerflow.persistence.skill_knowledge`; the single native assistant base is in
|
`deerflow.persistence.skill_knowledge`; native assistant bases are in
|
||||||
`deerflow.assistant_knowledge` and
|
`deerflow.assistant_knowledge` and
|
||||||
`deerflow.persistence.assistant_knowledge`. Gateway composition, remote adapters,
|
`deerflow.persistence.assistant_knowledge`. Gateway composition, remote adapters,
|
||||||
the lease dispatcher, watcher/scheduler, and APIs remain in `app.gateway` so the
|
the lease dispatcher, and APIs remain in `app.gateway` so the harness-to-app
|
||||||
harness-to-app import firewall is preserved.
|
import firewall is preserved.
|
||||||
|
|
||||||
Never execute a skill or read/upload/summarize its code and script files.
|
Never execute a skill or read/upload/summarize its code and script files.
|
||||||
Knowledge fingerprints exclude those files and Markdown code fences. All
|
Knowledge fingerprints exclude those files and Markdown code fences. All
|
||||||
resyncs must stage and validate the knowledge digest before activation; the old
|
resyncs must stage and validate the knowledge digest before activation; the old
|
||||||
snapshot remains authoritative on failure. A binding may have only one active
|
snapshot remains authoritative on failure. A binding may have only one active
|
||||||
item (`active_dedupe_key` plus atomic lease claims). Detach/delete operates on
|
item (`active_dedupe_key` plus atomic lease claims). Skill distillation targets
|
||||||
source contributions, not canonical rows, and shared entities/relations are
|
only writable WeKnora Wiki knowledge bases in this product version; reject direct
|
||||||
retired only when their final contribution is gone.
|
assistant-base targets, non-Wiki/FAQ/conversation/readonly targets, and direct
|
||||||
|
Neo4j writes. Detach/delete operates on source contributions, not canonical rows,
|
||||||
|
and shared entities/relations are retired only when their final contribution is
|
||||||
|
gone.
|
||||||
|
|
||||||
There is exactly one global assistant base (`知识梳理`). Ordinary users have
|
Assistant knowledge has one global full-library base (`知识梳理总库`) and any
|
||||||
read/search access and administrators own all mutations. WeKnora direct graph
|
number of admin-created custom bases. Ordinary users have read/search access and
|
||||||
writes require `llmwiki.weknora.graph.direct_write_enabled`, backend-only
|
administrators own all mutations. Assistant bases only accept WeKnora export
|
||||||
credentials, and the fixed parameterized templates in
|
packages or imports triggered from a WeKnora mapping. Do not add direct skill
|
||||||
`deerflow.skill_knowledge.graph_writer`; never accept Cypher from an API, model,
|
imports or arbitrary local-file imports to assistant knowledge. Importing into a
|
||||||
skill, or frontend.
|
custom assistant base must also mirror the package into the global base for
|
||||||
|
cross-source entity alignment.
|
||||||
|
|
||||||
|
## openGauss / GaussDB persistence invariant
|
||||||
|
|
||||||
|
`database.backend=gauss` is an application-ORM backend, parallel to `mysql`.
|
||||||
|
It normalizes `database.gauss_url` to `opengauss+asyncpg://` and requires the
|
||||||
|
`gauss` optional dependency (`asyncpg` plus `opengauss-sqlalchemy`). Never route
|
||||||
|
a Gauss URL through SQLAlchemy's generic PostgreSQL dialect: the wire protocol
|
||||||
|
may connect, but dialect initialization and schema DDL can fail against the
|
||||||
|
Gauss server/version behavior. Keep the LangGraph checkpointer on the separately
|
||||||
|
configured SQLite backend; the PostgreSQL checkpointer schema is not part of the
|
||||||
|
Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
|
||||||
|
names as timezone-aware PostgreSQL-family engines in portable types and
|
||||||
|
dialect-specific session settings.
|
||||||
@@ -13,8 +13,8 @@ FROM python:3.12-slim-bookworm AS builder
|
|||||||
ARG NODE_MAJOR=22
|
ARG NODE_MAJOR=22
|
||||||
ARG APT_MIRROR
|
ARG APT_MIRROR
|
||||||
ARG UV_INDEX_URL
|
ARG UV_INDEX_URL
|
||||||
# Optional extras to install (e.g. "postgres" for PostgreSQL support)
|
# Optional extras to install (e.g. "postgres" or "gauss")
|
||||||
# Usage: docker build --build-arg UV_EXTRAS=postgres ...
|
# Usage: docker build --build-arg UV_EXTRAS=gauss ...
|
||||||
ARG UV_EXTRAS
|
ARG UV_EXTRAS
|
||||||
|
|
||||||
# Optionally override apt mirror for restricted networks (e.g. APT_MIRROR=mirrors.aliyun.com)
|
# Optionally override apt mirror for restricted networks (e.g. APT_MIRROR=mirrors.aliyun.com)
|
||||||
|
|||||||
@@ -957,26 +957,54 @@ sandbox-directory recreation cannot discard a completed rewrite.
|
|||||||
## Skill knowledge distillation and assistant knowledge
|
## Skill knowledge distillation and assistant knowledge
|
||||||
|
|
||||||
The Gateway now exposes an administrator-only skill distillation pipeline at
|
The Gateway now exposes an administrator-only skill distillation pipeline at
|
||||||
`/api/skill-knowledge`. A matrix job can send up to 200 skills to multiple
|
`/api/skill-knowledge`. Each request can send up to 200 skills to one writable
|
||||||
WeKnora or global assistant-knowledge targets. The scanner never reads or
|
WeKnora Wiki knowledge base; multi-base distillation is done by repeated
|
||||||
uploads code/script files, strips Markdown code fences, blocks credential-like
|
operations so the local binding remains `skill_name + target_id`. The scanner
|
||||||
files and content, safely inspects ZIP members, and converts supported
|
never reads or uploads code/script files, strips Markdown code fences, blocks
|
||||||
Office/PDF/spreadsheet files off the event loop. Jobs use database claims and
|
credential-like files and content, safely inspects ZIP members, and converts
|
||||||
renewable leases, stage snapshots before activation, retain the prior active
|
supported Office/PDF/spreadsheet files off the event loop. Jobs use database
|
||||||
version on failure, support per-item retry and entity/relation review, and keep
|
claims and renewable leases, stage snapshots before activation, retain the
|
||||||
per-binding contribution records for safe detach/delete.
|
prior active version on failure, support per-item retry, and keep per-binding
|
||||||
|
contribution records for safe detach/delete. Direct assistant-base targets,
|
||||||
|
non-Wiki/FAQ/conversation/readonly targets, and direct Neo4j writes are rejected
|
||||||
|
for this product version.
|
||||||
|
|
||||||
Bindings may opt into debounced file watching and cron synchronization through
|
`/api/assistant-knowledge` manages DeerFlow-native assistant knowledge bases.
|
||||||
`/api/skill-knowledge/auto-sync-settings`. Only non-code knowledge digests can
|
There is one global full-library base named `知识梳理总库`, plus admin-created
|
||||||
mark a binding stale. WeKnora Wiki, document, FAQ, conversation-archive and
|
custom assistant bases. Reads and search are available to authenticated users;
|
||||||
read-only sidecar targets have separate projections. Optional Neo4j direct
|
global initialization, custom-base creation, WeKnora export/import, source
|
||||||
writes are disabled by default under `llmwiki.weknora.graph`; when enabled they
|
reimport, and source deletion are admin-only. Assistant bases only accept
|
||||||
use only fixed parameterized templates and Neo4j's transactional HTTP endpoint.
|
WeKnora export packages or imports triggered from a WeKnora mapping; direct
|
||||||
|
skill import and arbitrary local-file import return 410. Importing into a custom
|
||||||
`/api/assistant-knowledge` owns exactly one global DeerFlow-native base named
|
assistant base also mirrors the same package into the global base, where
|
||||||
`知识梳理`. Reads and search are available to authenticated users; initialization,
|
canonical entities and relations are aligned across all imported sources.
|
||||||
local-file import, WeKnora export/import, source reimport/delete, and revision
|
|
||||||
rollback are admin-only. Imports align canonical entities and relations while
|
|
||||||
retaining source contributions, publish source/entity/relation Wiki revisions,
|
|
||||||
and remove canonical knowledge only after its final contribution disappears.
|
|
||||||
The database schema is introduced by Alembic revision `20260901_01`.
|
The database schema is introduced by Alembic revision `20260901_01`.
|
||||||
|
|
||||||
|
## openGauss / GaussDB application database
|
||||||
|
|
||||||
|
The application ORM can use a PG-compatible openGauss/GaussDB database through
|
||||||
|
the official `opengauss-sqlalchemy` async dialect. Install the optional driver
|
||||||
|
set before enabling it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv sync --extra gauss
|
||||||
|
```
|
||||||
|
|
||||||
|
Configure the business database independently from the LangGraph checkpointer:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
database:
|
||||||
|
backend: gauss
|
||||||
|
gauss_url: $GAUSS_DATABASE_URL
|
||||||
|
|
||||||
|
checkpointer:
|
||||||
|
type: sqlite
|
||||||
|
connection_string: .deer-flow/data/deerflow.db
|
||||||
|
```
|
||||||
|
|
||||||
|
`GAUSS_DATABASE_URL` should normally be
|
||||||
|
`opengauss+asyncpg://user:password@host:26000/deerflow`. Plain
|
||||||
|
`opengauss://`, `gaussdb://`, and PostgreSQL-style URLs are accepted and
|
||||||
|
normalized to the openGauss dialect. Do not configure the LangGraph
|
||||||
|
PostgreSQL checkpointer against GaussDB: its migrations are PostgreSQL-specific.
|
||||||
|
For a production image, build with `--build-arg UV_EXTRAS=gauss`.
|
||||||
@@ -362,13 +362,21 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
|
|||||||
|
|
||||||
resolved_llmwiki = get_resolved_llmwiki_runtime(config)
|
resolved_llmwiki = get_resolved_llmwiki_runtime(config)
|
||||||
weknora_client = build_weknora_client(resolved_llmwiki) if resolved_llmwiki.weknora_enabled else None
|
weknora_client = build_weknora_client(resolved_llmwiki) if resolved_llmwiki.weknora_enabled else None
|
||||||
if local_index_config.enabled:
|
embedding_config = local_index_config.embedding
|
||||||
|
embedding_configured = bool(
|
||||||
|
embedding_config.enabled
|
||||||
|
and embedding_config.base_url
|
||||||
|
and embedding_config.model.strip()
|
||||||
|
)
|
||||||
|
if embedding_configured:
|
||||||
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
|
from deerflow.integrations.weknora.local_index.embedding import StrictWikiEmbeddingClient
|
||||||
|
|
||||||
|
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(embedding_config)
|
||||||
|
if local_index_config.enabled and app.state.llmwiki_embedding is not None:
|
||||||
from deerflow.integrations.weknora.local_index.search import WikiVectorSearchService
|
from deerflow.integrations.weknora.local_index.search import WikiVectorSearchService
|
||||||
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||||
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
|
from deerflow.integrations.weknora.local_index.vector_cache import WikiVectorCache
|
||||||
|
|
||||||
app.state.llmwiki_embedding = StrictWikiEmbeddingClient(local_index_config.embedding)
|
|
||||||
app.state.llmwiki_vector_cache = WikiVectorCache(
|
app.state.llmwiki_vector_cache = WikiVectorCache(
|
||||||
app.state.llmwiki_index_store,
|
app.state.llmwiki_index_store,
|
||||||
max_knowledge_bases=local_index_config.cache_max_knowledge_bases,
|
max_knowledge_bases=local_index_config.cache_max_knowledge_bases,
|
||||||
|
|||||||
@@ -40,6 +40,7 @@ _MAX_SOURCE_CHARS = 1400
|
|||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class LlmWikiRagContext:
|
class LlmWikiRagContext:
|
||||||
knowledge_base_ids: list[str]
|
knowledge_base_ids: list[str]
|
||||||
|
assistant_knowledge_base_ids: list[str]
|
||||||
source_count: int
|
source_count: int
|
||||||
prompt: str
|
prompt: str
|
||||||
sources: list[dict[str, Any]]
|
sources: list[dict[str, Any]]
|
||||||
@@ -50,6 +51,10 @@ def _store(request: Request) -> LlmWikiStore | None:
|
|||||||
return getattr(request.app.state, "llmwiki_store", None)
|
return getattr(request.app.state, "llmwiki_store", None)
|
||||||
|
|
||||||
|
|
||||||
|
def _assistant_store(request: Request) -> Any | None:
|
||||||
|
return getattr(request.app.state, "assistant_knowledge_store", None)
|
||||||
|
|
||||||
|
|
||||||
async def _actor(request: Request) -> tuple[str, bool]:
|
async def _actor(request: Request) -> tuple[str, bool]:
|
||||||
user = await get_optional_user_from_request(request)
|
user = await get_optional_user_from_request(request)
|
||||||
if user is None:
|
if user is None:
|
||||||
@@ -76,6 +81,13 @@ def _body_selected_ids(body: Any) -> list[str]:
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _body_selected_assistant_ids(body: Any) -> list[str]:
|
||||||
|
for source in (getattr(body, "context", None), getattr(body, "metadata", None)):
|
||||||
|
if isinstance(source, Mapping) and "assistant_knowledge_base_ids" in source:
|
||||||
|
return _as_id_list(source.get("assistant_knowledge_base_ids"))
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
def _assistant_agent_id(body: Any) -> str | None:
|
def _assistant_agent_id(body: Any) -> str | None:
|
||||||
assistant_id = validate_agent_id(getattr(body, "assistant_id", None))
|
assistant_id = validate_agent_id(getattr(body, "assistant_id", None))
|
||||||
if assistant_id:
|
if assistant_id:
|
||||||
@@ -106,8 +118,17 @@ def resolve_requested_llmwiki_knowledge_base_ids(body: Any) -> list[str]:
|
|||||||
return _body_selected_ids(body)
|
return _body_selected_ids(body)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_requested_assistant_knowledge_base_ids(body: Any) -> list[str]:
|
||||||
|
"""Return assistant knowledge bases explicitly selected for the run."""
|
||||||
|
|
||||||
|
return _body_selected_assistant_ids(body)
|
||||||
|
|
||||||
|
|
||||||
def llmwiki_selection_requested(body: Any) -> bool:
|
def llmwiki_selection_requested(body: Any) -> bool:
|
||||||
return bool(resolve_requested_llmwiki_knowledge_base_ids(body))
|
return bool(
|
||||||
|
resolve_requested_llmwiki_knowledge_base_ids(body)
|
||||||
|
or resolve_requested_assistant_knowledge_base_ids(body)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _latest_human_query(graph_input: dict[str, Any]) -> str:
|
def _latest_human_query(graph_input: dict[str, Any]) -> str:
|
||||||
@@ -191,18 +212,50 @@ def _source_from_result(result: dict[str, Any], mapping: dict[str, Any], index:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _source_from_assistant_result(result: dict[str, Any], index: int) -> dict[str, Any]:
|
||||||
|
title = str(result.get("title") or f"助手知识库片段 {index}")
|
||||||
|
matched_section = result.get("matched_section") if isinstance(result.get("matched_section"), Mapping) else {}
|
||||||
|
content = str(
|
||||||
|
matched_section.get("content")
|
||||||
|
or result.get("content_markdown")
|
||||||
|
or result.get("content")
|
||||||
|
or result.get("summary")
|
||||||
|
or ""
|
||||||
|
)
|
||||||
|
snippet = str(result.get("summary") or content)
|
||||||
|
raw = {
|
||||||
|
**result,
|
||||||
|
"source": "助手知识库",
|
||||||
|
"url": "/page/workspace/knowledge/assistant",
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"index": index,
|
||||||
|
"id": result.get("id") or result.get("slug") or f"assistant:{index}",
|
||||||
|
"title": title,
|
||||||
|
"type": "assistant_knowledge",
|
||||||
|
"snippet": _truncate(snippet, 360),
|
||||||
|
"content": content,
|
||||||
|
"source": "助手知识库",
|
||||||
|
"score": float(result.get("score") or 0),
|
||||||
|
"url": "/page/workspace/knowledge/assistant",
|
||||||
|
"skillName": "assistant_knowledge_search",
|
||||||
|
"mode": "citation",
|
||||||
|
"raw": raw,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
|
def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
|
||||||
if not sources:
|
if not sources:
|
||||||
return "当前用户选择了 LLMWiki 知识库,但 DeerFlow 未检索到可用片段。请直接说明没有检索到可用背景资料,不要编造知识库内容或引用编号。"
|
return "当前用户选择了知识库,但 DeerFlow 未检索到可用片段。请直接说明没有检索到可用背景资料,不要编造知识库内容或引用编号。"
|
||||||
|
|
||||||
lines = [
|
lines = [
|
||||||
"当前用户选择了 LLMWiki 知识库。DeerFlow 已在模型回答前自动检索到以下背景资料。",
|
"当前用户选择了知识库。DeerFlow 已在模型回答前自动检索到以下背景资料。",
|
||||||
"请优先依据这些资料回答;如果资料不足,请明确说明不足之处,不要编造。",
|
"请优先依据这些资料回答;如果资料不足,请明确说明不足之处,不要编造。",
|
||||||
"引用规则:正文关键结论必须使用 [1]、[2] 这样的编号引用下方资料;只能引用存在的编号;不要在正文末尾重复输出完整参考文献列表,前端会展示引用卡片。",
|
"引用规则:正文关键结论必须使用 [1]、[2] 这样的编号引用下方资料;只能引用存在的编号;不要在正文末尾重复输出完整参考文献列表,前端会展示引用卡片。",
|
||||||
"",
|
"",
|
||||||
f"用户问题:{query}",
|
f"用户问题:{query}",
|
||||||
"",
|
"",
|
||||||
f'<llmwiki_wiki_reference_batch source_count="{len(sources)}">',
|
f'<knowledge_reference_batch source_count="{len(sources)}">',
|
||||||
]
|
]
|
||||||
for source in sources:
|
for source in sources:
|
||||||
meta = " / ".join(
|
meta = " / ".join(
|
||||||
@@ -213,12 +266,12 @@ def _build_prompt(query: str, sources: list[dict[str, Any]]) -> str:
|
|||||||
)
|
)
|
||||||
if item
|
if item
|
||||||
)
|
)
|
||||||
lines.append(f"[{source['index']}] {source.get('title') or 'LLMWiki 片段'}")
|
lines.append(f"[{source['index']}] {source.get('title') or '知识库片段'}")
|
||||||
if meta:
|
if meta:
|
||||||
lines.append(f"来源:{meta}")
|
lines.append(f"来源:{meta}")
|
||||||
lines.append(_truncate(str(source.get("content") or source.get("snippet") or ""), _MAX_SOURCE_CHARS))
|
lines.append(_truncate(str(source.get("content") or source.get("snippet") or ""), _MAX_SOURCE_CHARS))
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("</llmwiki_wiki_reference_batch>")
|
lines.append("</knowledge_reference_batch>")
|
||||||
return "\n".join(lines).strip()
|
return "\n".join(lines).strip()
|
||||||
|
|
||||||
|
|
||||||
@@ -229,52 +282,99 @@ async def build_llmwiki_rag_context(
|
|||||||
graph_input: dict[str, Any],
|
graph_input: dict[str, Any],
|
||||||
) -> LlmWikiRagContext:
|
) -> LlmWikiRagContext:
|
||||||
selected_ids = resolve_requested_llmwiki_knowledge_base_ids(body)
|
selected_ids = resolve_requested_llmwiki_knowledge_base_ids(body)
|
||||||
if not selected_ids:
|
assistant_selected_ids = resolve_requested_assistant_knowledge_base_ids(body)
|
||||||
return LlmWikiRagContext([], 0, "", [], "")
|
if not selected_ids and not assistant_selected_ids:
|
||||||
|
return LlmWikiRagContext([], [], 0, "", [], "")
|
||||||
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None))
|
|
||||||
local_config = request.app.state.config.llmwiki.local_wiki_index
|
|
||||||
if not (local_config.enabled and local_config.internal_search_enabled) and not runtime.weknora_enabled:
|
|
||||||
return LlmWikiRagContext(selected_ids, 0, "", [], "")
|
|
||||||
store = _store(request)
|
|
||||||
if store is None:
|
|
||||||
return LlmWikiRagContext(selected_ids, 0, "", [], "")
|
|
||||||
|
|
||||||
query = _latest_human_query(graph_input)
|
query = _latest_human_query(graph_input)
|
||||||
if not query:
|
if not query:
|
||||||
return LlmWikiRagContext(selected_ids, 0, "", [], "")
|
return LlmWikiRagContext(selected_ids, assistant_selected_ids, 0, "", [], "")
|
||||||
|
|
||||||
user_id, is_admin = await _actor(request)
|
|
||||||
rows: list[dict[str, Any]] = []
|
|
||||||
for mapping_id in selected_ids:
|
|
||||||
row = await store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
|
|
||||||
if row is None:
|
|
||||||
raise HTTPException(status_code=404, detail=f"Knowledge base {mapping_id} not found")
|
|
||||||
rows.append(row)
|
|
||||||
search_service = getattr(request.app.state, "llmwiki_retrieval_service", None)
|
|
||||||
if search_service is None:
|
|
||||||
return LlmWikiRagContext(
|
|
||||||
selected_ids,
|
|
||||||
0,
|
|
||||||
"Wiki 检索暂不可用。请明确说明系统检索不可用,不要表述为资料不存在。",
|
|
||||||
[],
|
|
||||||
query,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
payload = await search_service.search(query, rows, top_k_pages=_MAX_SOURCES)
|
|
||||||
except WikiIndexError as exc:
|
|
||||||
logger.warning("Wiki-only pre-retrieval failed user=%s kb_count=%s code=%s", sanitize_log_param(user_id), len(rows), exc.code)
|
|
||||||
return LlmWikiRagContext(selected_ids, 0, f"Wiki 检索暂不可用({exc.code})。请明确说明系统检索状态,不要声称知识库没有资料。", [], query)
|
|
||||||
mapping_by_id = {str(row["id"]): row for row in rows}
|
|
||||||
sources: list[dict[str, Any]] = []
|
sources: list[dict[str, Any]] = []
|
||||||
for result in payload.get("results") or []:
|
|
||||||
mapping = mapping_by_id.get(str(result.get("knowledge_base_id") or ""))
|
runtime = get_resolved_llmwiki_runtime(getattr(request.app.state, "config", None))
|
||||||
if mapping is None:
|
local_config = request.app.state.config.llmwiki.local_wiki_index
|
||||||
continue
|
user_id, is_admin = await _actor(request)
|
||||||
sources.append(_source_from_result(result, mapping, len(sources) + 1))
|
|
||||||
if len(sources) >= _MAX_SOURCES:
|
if selected_ids:
|
||||||
break
|
llmwiki_available = (local_config.enabled and local_config.internal_search_enabled) or runtime.weknora_enabled
|
||||||
return LlmWikiRagContext(selected_ids, len(sources), _build_prompt(query, sources), sources, query)
|
store = _store(request)
|
||||||
|
if not llmwiki_available or store is None:
|
||||||
|
if not assistant_selected_ids:
|
||||||
|
return LlmWikiRagContext(selected_ids, assistant_selected_ids, 0, "", [], query)
|
||||||
|
else:
|
||||||
|
rows: list[dict[str, Any]] = []
|
||||||
|
for mapping_id in selected_ids:
|
||||||
|
row = await store.get_authorized(mapping_id, user_id, write=False, is_admin=is_admin)
|
||||||
|
if row is None:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Knowledge base {mapping_id} not found")
|
||||||
|
rows.append(row)
|
||||||
|
search_service = getattr(request.app.state, "llmwiki_retrieval_service", None)
|
||||||
|
if search_service is None:
|
||||||
|
if not assistant_selected_ids:
|
||||||
|
return LlmWikiRagContext(
|
||||||
|
selected_ids,
|
||||||
|
assistant_selected_ids,
|
||||||
|
0,
|
||||||
|
"Wiki 检索暂不可用。请明确说明系统检索不可用,不要表述为资料不存在。",
|
||||||
|
[],
|
||||||
|
query,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
payload = (
|
||||||
|
await search_service.search(query, rows, top_k_pages=_MAX_SOURCES)
|
||||||
|
if search_service is not None
|
||||||
|
else {"results": []}
|
||||||
|
)
|
||||||
|
except WikiIndexError as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Wiki-only pre-retrieval failed user=%s kb_count=%s code=%s",
|
||||||
|
sanitize_log_param(user_id),
|
||||||
|
len(rows),
|
||||||
|
exc.code,
|
||||||
|
)
|
||||||
|
if not assistant_selected_ids:
|
||||||
|
return LlmWikiRagContext(
|
||||||
|
selected_ids,
|
||||||
|
assistant_selected_ids,
|
||||||
|
0,
|
||||||
|
f"Wiki 检索暂不可用({exc.code})。请明确说明系统检索状态,不要声称知识库没有资料。",
|
||||||
|
[],
|
||||||
|
query,
|
||||||
|
)
|
||||||
|
payload = {"results": []}
|
||||||
|
mapping_by_id = {str(row["id"]): row for row in rows}
|
||||||
|
for result in payload.get("results") or []:
|
||||||
|
mapping = mapping_by_id.get(str(result.get("knowledge_base_id") or ""))
|
||||||
|
if mapping is None:
|
||||||
|
continue
|
||||||
|
sources.append(_source_from_result(result, mapping, len(sources) + 1))
|
||||||
|
if len(sources) >= _MAX_SOURCES:
|
||||||
|
break
|
||||||
|
|
||||||
|
if assistant_selected_ids and len(sources) < _MAX_SOURCES:
|
||||||
|
assistant_store = _assistant_store(request)
|
||||||
|
if assistant_store is not None:
|
||||||
|
rows = await assistant_store.search_pages(
|
||||||
|
base_ids=assistant_selected_ids,
|
||||||
|
query=query,
|
||||||
|
limit=_MAX_SOURCES - len(sources),
|
||||||
|
include_chunks=True,
|
||||||
|
embedding_client=getattr(request.app.state, "llmwiki_embedding", None),
|
||||||
|
)
|
||||||
|
for row in rows:
|
||||||
|
sources.append(_source_from_assistant_result(row, len(sources) + 1))
|
||||||
|
if len(sources) >= _MAX_SOURCES:
|
||||||
|
break
|
||||||
|
|
||||||
|
return LlmWikiRagContext(
|
||||||
|
selected_ids,
|
||||||
|
assistant_selected_ids,
|
||||||
|
len(sources),
|
||||||
|
_build_prompt(query, sources),
|
||||||
|
sources,
|
||||||
|
query,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def attach_llmwiki_rag_to_graph_input(
|
def attach_llmwiki_rag_to_graph_input(
|
||||||
@@ -294,6 +394,7 @@ def attach_llmwiki_rag_to_graph_input(
|
|||||||
existing = str(extra.get("prompt_prefix") or "").strip()
|
existing = str(extra.get("prompt_prefix") or "").strip()
|
||||||
extra["prompt_prefix"] = f"{existing}\n\n{rag.prompt}".strip() if existing else rag.prompt
|
extra["prompt_prefix"] = f"{existing}\n\n{rag.prompt}".strip() if existing else rag.prompt
|
||||||
extra["llmwiki_knowledge_base_ids"] = rag.knowledge_base_ids
|
extra["llmwiki_knowledge_base_ids"] = rag.knowledge_base_ids
|
||||||
|
extra["assistant_knowledge_base_ids"] = rag.assistant_knowledge_base_ids
|
||||||
extra["llmwiki_reference_query"] = rag.query
|
extra["llmwiki_reference_query"] = rag.query
|
||||||
if rag.sources:
|
if rag.sources:
|
||||||
extra["llmwiki_reference_sources"] = rag.sources
|
extra["llmwiki_reference_sources"] = rag.sources
|
||||||
|
|||||||
@@ -461,7 +461,7 @@ async def _set_analytics_read_timeout(session) -> None:
|
|||||||
- MariaDB: ``max_statement_time`` (seconds)
|
- MariaDB: ``max_statement_time`` (seconds)
|
||||||
- MySQL <= 5.6: no per-statement timeout variable — skip it entirely
|
- MySQL <= 5.6: no per-statement timeout variable — skip it entirely
|
||||||
(5.6 raises "Unknown system variable" otherwise)
|
(5.6 raises "Unknown system variable" otherwise)
|
||||||
- PostgreSQL: ``statement_timeout`` (milliseconds)
|
- PostgreSQL/openGauss: ``statement_timeout`` (milliseconds)
|
||||||
"""
|
"""
|
||||||
conn = await session.connection()
|
conn = await session.connection()
|
||||||
dialect = conn.dialect
|
dialect = conn.dialect
|
||||||
@@ -474,7 +474,7 @@ async def _set_analytics_read_timeout(session) -> None:
|
|||||||
await _try_set_session_value(session, f"SET SESSION max_execution_time = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
|
await _try_set_session_value(session, f"SET SESSION max_execution_time = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
|
||||||
# else: MySQL <= 5.6 has no statement-timeout variable — skip silently.
|
# else: MySQL <= 5.6 has no statement-timeout variable — skip silently.
|
||||||
await _try_set_session_value(session, f"SET SESSION innodb_lock_wait_timeout = {_LEADERBOARD_LOCK_WAIT_SECONDS}")
|
await _try_set_session_value(session, f"SET SESSION innodb_lock_wait_timeout = {_LEADERBOARD_LOCK_WAIT_SECONDS}")
|
||||||
elif name == "postgresql":
|
elif name in {"postgresql", "opengauss"}:
|
||||||
await _try_set_session_value(session, f"SET LOCAL statement_timeout = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
|
await _try_set_session_value(session, f"SET LOCAL statement_timeout = {_LEADERBOARD_QUERY_TIMEOUT_MS}")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,49 +1,50 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
import logging
|
import logging
|
||||||
import mimetypes
|
import mimetypes
|
||||||
import time
|
import time
|
||||||
import zipfile
|
import zipfile
|
||||||
from collections import OrderedDict
|
from collections import OrderedDict
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Literal
|
from typing import Literal
|
||||||
from urllib.parse import quote
|
from urllib.parse import quote
|
||||||
|
|
||||||
from fastapi import APIRouter, HTTPException, Query, Request
|
from fastapi import APIRouter, HTTPException, Query, Request
|
||||||
from fastapi.responses import FileResponse, PlainTextResponse, Response
|
from fastapi.responses import FileResponse, PlainTextResponse, Response
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from app.gateway.authz import require_permission
|
from app.gateway.authz import require_permission
|
||||||
from app.gateway.path_utils import aresolve_thread_virtual_path
|
from app.gateway.path_utils import aresolve_thread_virtual_path
|
||||||
from deerflow.config.paths import get_paths
|
from deerflow.config.paths import get_paths
|
||||||
from deerflow.runtime.thread_paths import aresolve_path_user_id
|
from deerflow.runtime.thread_paths import aresolve_path_user_id
|
||||||
from deerflow.runtime.user_context import get_effective_user_id
|
from deerflow.runtime.user_context import get_effective_user_id
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
router = APIRouter(prefix="/api", tags=["artifacts"])
|
router = APIRouter(prefix="/api", tags=["artifacts"])
|
||||||
|
|
||||||
ACTIVE_CONTENT_MIME_TYPES = {
|
ACTIVE_CONTENT_MIME_TYPES = {
|
||||||
"text/html",
|
"text/html",
|
||||||
"application/xhtml+xml",
|
"application/xhtml+xml",
|
||||||
"image/svg+xml",
|
"image/svg+xml",
|
||||||
}
|
}
|
||||||
|
|
||||||
EDITABLE_TEXT_SUFFIXES = {
|
EDITABLE_TEXT_SUFFIXES = {
|
||||||
".md",
|
".md",
|
||||||
".markdown",
|
".markdown",
|
||||||
".txt",
|
".txt",
|
||||||
".json",
|
".json",
|
||||||
".yaml",
|
".yaml",
|
||||||
".yml",
|
".yml",
|
||||||
".csv",
|
".csv",
|
||||||
".xml",
|
".xml",
|
||||||
".log",
|
".log",
|
||||||
}
|
}
|
||||||
|
|
||||||
_VIRTUAL_USER_DATA_PREFIX = "mnt/user-data"
|
_VIRTUAL_USER_DATA_PREFIX = "mnt/user-data"
|
||||||
|
|
||||||
ArtifactLibraryKind = Literal["all", "document", "spreadsheet", "text_data", "archive", "other"]
|
ArtifactLibraryKind = Literal["all", "document", "spreadsheet", "text_data", "archive", "other"]
|
||||||
|
ArtifactLibrarySource = Literal["all", "general_qa", "agent"]
|
||||||
|
|
||||||
_MEDIA_MIME_PREFIXES = ("image/", "audio/", "video/")
|
_MEDIA_MIME_PREFIXES = ("image/", "audio/", "video/")
|
||||||
_MEDIA_SUFFIXES = {
|
_MEDIA_SUFFIXES = {
|
||||||
@@ -121,35 +122,37 @@ _ARTIFACT_LIBRARY_SCAN_CONCURRENCY = 8
|
|||||||
_ARTIFACT_LIBRARY_THREAD_BATCH_SIZE = 500
|
_ARTIFACT_LIBRARY_THREAD_BATCH_SIZE = 500
|
||||||
_artifact_library_cache: OrderedDict[str, tuple[float, list[dict]]] = OrderedDict()
|
_artifact_library_cache: OrderedDict[str, tuple[float, list[dict]]] = OrderedDict()
|
||||||
|
|
||||||
|
|
||||||
def coerce_artifact_virtual_path(path: str) -> str:
|
def coerce_artifact_virtual_path(path: str) -> str:
|
||||||
"""Normalize an artifacts URL path to a sandbox virtual path.
|
"""Normalize an artifacts URL path to a sandbox virtual path.
|
||||||
|
|
||||||
Streaming write previews may request a bare filename such as
|
Streaming write previews may request a bare filename such as
|
||||||
``未命名文件.md`` before the model has emitted a real
|
``未命名文件.md`` before the model has emitted a real
|
||||||
``/mnt/user-data/outputs/...`` path. Mapping a single path segment into
|
``/mnt/user-data/outputs/...`` path. Mapping a single path segment into
|
||||||
the thread outputs directory turns that into a normal 404 (file not written
|
the thread outputs directory turns that into a normal 404 (file not written
|
||||||
yet) instead of HTTP 400 ``Path must start with /mnt/user-data``.
|
yet) instead of HTTP 400 ``Path must start with /mnt/user-data``.
|
||||||
"""
|
"""
|
||||||
stripped = path.strip().lstrip("/").replace("\\", "/")
|
stripped = path.strip().lstrip("/").replace("\\", "/")
|
||||||
if not stripped:
|
if not stripped:
|
||||||
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
||||||
if stripped == _VIRTUAL_USER_DATA_PREFIX or stripped.startswith(_VIRTUAL_USER_DATA_PREFIX + "/"):
|
if stripped == _VIRTUAL_USER_DATA_PREFIX or stripped.startswith(_VIRTUAL_USER_DATA_PREFIX + "/"):
|
||||||
return stripped
|
return stripped
|
||||||
if "/" in stripped or stripped in {".", ".."}:
|
if "/" in stripped or stripped in {".", ".."}:
|
||||||
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
raise HTTPException(status_code=400, detail="Path must start with /mnt/user-data")
|
||||||
return f"{_VIRTUAL_USER_DATA_PREFIX}/outputs/{stripped}"
|
return f"{_VIRTUAL_USER_DATA_PREFIX}/outputs/{stripped}"
|
||||||
|
|
||||||
|
|
||||||
class ArtifactUpdateRequest(BaseModel):
|
class ArtifactUpdateRequest(BaseModel):
|
||||||
content: str
|
content: str
|
||||||
|
|
||||||
|
|
||||||
class ArtifactLibraryItem(BaseModel):
|
class ArtifactLibraryItem(BaseModel):
|
||||||
thread_id: str
|
thread_id: str
|
||||||
thread_title: str
|
thread_title: str
|
||||||
agent_id: str | None = None
|
agent_id: str | None = None
|
||||||
|
agent_name: str | None = None
|
||||||
thread_type: str | None = None
|
thread_type: str | None = None
|
||||||
|
source_type: Literal["general_qa", "agent"]
|
||||||
path: str
|
path: str
|
||||||
name: str
|
name: str
|
||||||
size_bytes: int
|
size_bytes: int
|
||||||
@@ -163,85 +166,85 @@ class ArtifactLibraryResponse(BaseModel):
|
|||||||
offset: int
|
offset: int
|
||||||
limit: int
|
limit: int
|
||||||
items: list[ArtifactLibraryItem]
|
items: list[ArtifactLibraryItem]
|
||||||
|
|
||||||
|
|
||||||
def _build_content_disposition(disposition_type: str, filename: str) -> str:
|
def _build_content_disposition(disposition_type: str, filename: str) -> str:
|
||||||
"""Build an RFC 5987 encoded Content-Disposition header value."""
|
"""Build an RFC 5987 encoded Content-Disposition header value."""
|
||||||
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
|
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
|
||||||
|
|
||||||
|
|
||||||
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
|
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
|
||||||
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
|
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
|
||||||
if extra_headers:
|
if extra_headers:
|
||||||
headers.update(extra_headers)
|
headers.update(extra_headers)
|
||||||
return headers
|
return headers
|
||||||
|
|
||||||
|
|
||||||
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
|
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
|
||||||
"""Check if file is text by examining content for null bytes."""
|
"""Check if file is text by examining content for null bytes."""
|
||||||
try:
|
try:
|
||||||
with open(path, "rb") as f:
|
with open(path, "rb") as f:
|
||||||
chunk = f.read(sample_size)
|
chunk = f.read(sample_size)
|
||||||
# Text files shouldn't contain null bytes
|
# Text files shouldn't contain null bytes
|
||||||
return b"\x00" not in chunk
|
return b"\x00" not in chunk
|
||||||
except Exception:
|
except Exception:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
|
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
|
||||||
"""Extract a file from a .skill ZIP archive.
|
"""Extract a file from a .skill ZIP archive.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
zip_path: Path to the .skill file (ZIP archive).
|
zip_path: Path to the .skill file (ZIP archive).
|
||||||
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
|
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The file content as bytes, or None if not found.
|
The file content as bytes, or None if not found.
|
||||||
"""
|
"""
|
||||||
if not zipfile.is_zipfile(zip_path):
|
if not zipfile.is_zipfile(zip_path):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with zipfile.ZipFile(zip_path, "r") as zip_ref:
|
with zipfile.ZipFile(zip_path, "r") as zip_ref:
|
||||||
# List all files in the archive
|
# List all files in the archive
|
||||||
namelist = zip_ref.namelist()
|
namelist = zip_ref.namelist()
|
||||||
|
|
||||||
# Try direct path first
|
# Try direct path first
|
||||||
if internal_path in namelist:
|
if internal_path in namelist:
|
||||||
return zip_ref.read(internal_path)
|
return zip_ref.read(internal_path)
|
||||||
|
|
||||||
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
|
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
|
||||||
for name in namelist:
|
for name in namelist:
|
||||||
if name.endswith("/" + internal_path) or name == internal_path:
|
if name.endswith("/" + internal_path) or name == internal_path:
|
||||||
return zip_ref.read(name)
|
return zip_ref.read(name)
|
||||||
|
|
||||||
# Not found
|
# Not found
|
||||||
return None
|
return None
|
||||||
except (zipfile.BadZipFile, KeyError):
|
except (zipfile.BadZipFile, KeyError):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
|
def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
|
||||||
"""Walk an outputs directory and return artifact listing entries.
|
"""Walk an outputs directory and return artifact listing entries.
|
||||||
|
|
||||||
Each entry has ``path`` (virtual ``mnt/user-data/outputs/...``), ``name``,
|
Each entry has ``path`` (virtual ``mnt/user-data/outputs/...``), ``name``,
|
||||||
``size_bytes``, ``mime_type``, and ``modified_at`` (ISO 8601 UTC). Symlinks
|
``size_bytes``, ``mime_type``, and ``modified_at`` (ISO 8601 UTC). Symlinks
|
||||||
that escape the outputs root are dropped to keep the listing scoped to the
|
that escape the outputs root are dropped to keep the listing scoped to the
|
||||||
thread. The result is sorted by ``modified_at`` descending (newest first).
|
thread. The result is sorted by ``modified_at`` descending (newest first).
|
||||||
"""
|
"""
|
||||||
if not outputs_dir.exists() or not outputs_dir.is_dir():
|
if not outputs_dir.exists() or not outputs_dir.is_dir():
|
||||||
return []
|
return []
|
||||||
|
|
||||||
outputs_root = outputs_dir.resolve()
|
outputs_root = outputs_dir.resolve()
|
||||||
files: list[dict] = []
|
files: list[dict] = []
|
||||||
for entry in outputs_root.rglob("*"):
|
for entry in outputs_root.rglob("*"):
|
||||||
if not entry.is_file():
|
if not entry.is_file():
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
resolved = entry.resolve()
|
resolved = entry.resolve()
|
||||||
resolved.relative_to(outputs_root)
|
resolved.relative_to(outputs_root)
|
||||||
except (OSError, ValueError):
|
except (OSError, ValueError):
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
relative = resolved.relative_to(outputs_root).as_posix()
|
relative = resolved.relative_to(outputs_root).as_posix()
|
||||||
mime_type, _ = mimetypes.guess_type(resolved.name)
|
mime_type, _ = mimetypes.guess_type(resolved.name)
|
||||||
@@ -251,18 +254,18 @@ def collect_outputs_listing(outputs_dir: Path) -> list[dict]:
|
|||||||
# is being scanned. Treat that transient entry as absent instead of
|
# is being scanned. Treat that transient entry as absent instead of
|
||||||
# failing the complete thread/library listing.
|
# failing the complete thread/library listing.
|
||||||
continue
|
continue
|
||||||
files.append(
|
files.append(
|
||||||
{
|
{
|
||||||
"path": f"mnt/user-data/outputs/{relative}",
|
"path": f"mnt/user-data/outputs/{relative}",
|
||||||
"name": resolved.name,
|
"name": resolved.name,
|
||||||
"size_bytes": stat.st_size,
|
"size_bytes": stat.st_size,
|
||||||
"mime_type": mime_type,
|
"mime_type": mime_type,
|
||||||
"modified_at": datetime.fromtimestamp(stat.st_mtime, tz=UTC).isoformat(),
|
"modified_at": datetime.fromtimestamp(stat.st_mtime, tz=UTC).isoformat(),
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
files.sort(key=lambda f: f["modified_at"], reverse=True)
|
files.sort(key=lambda f: f["modified_at"], reverse=True)
|
||||||
return files
|
return files
|
||||||
|
|
||||||
|
|
||||||
def artifact_library_kind(name: str) -> str:
|
def artifact_library_kind(name: str) -> str:
|
||||||
@@ -335,7 +338,10 @@ async def _scan_artifact_library(thread_store) -> list[dict]:
|
|||||||
|
|
||||||
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
|
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
|
||||||
agent_id = metadata.get("agent_id")
|
agent_id = metadata.get("agent_id")
|
||||||
|
agent_name = metadata.get("agent_name")
|
||||||
thread_type = metadata.get("thread_type")
|
thread_type = metadata.get("thread_type")
|
||||||
|
normalized_agent_id = agent_id.strip() if isinstance(agent_id, str) else ""
|
||||||
|
normalized_agent_name = agent_name.strip() if isinstance(agent_name, str) else ""
|
||||||
title = str(row.get("display_name") or "新对话")
|
title = str(row.get("display_name") or "新对话")
|
||||||
items: list[dict] = []
|
items: list[dict] = []
|
||||||
for file in files:
|
for file in files:
|
||||||
@@ -345,8 +351,10 @@ async def _scan_artifact_library(thread_store) -> list[dict]:
|
|||||||
{
|
{
|
||||||
"thread_id": thread_id,
|
"thread_id": thread_id,
|
||||||
"thread_title": title,
|
"thread_title": title,
|
||||||
"agent_id": agent_id if isinstance(agent_id, str) and agent_id else None,
|
"agent_id": normalized_agent_id or None,
|
||||||
|
"agent_name": normalized_agent_name or None,
|
||||||
"thread_type": thread_type if isinstance(thread_type, str) and thread_type else None,
|
"thread_type": thread_type if isinstance(thread_type, str) and thread_type else None,
|
||||||
|
"source_type": "agent" if normalized_agent_id else "general_qa",
|
||||||
**file,
|
**file,
|
||||||
"file_kind": artifact_library_kind(str(file.get("name") or "")),
|
"file_kind": artifact_library_kind(str(file.get("name") or "")),
|
||||||
}
|
}
|
||||||
@@ -393,6 +401,7 @@ async def list_artifact_library(
|
|||||||
request: Request,
|
request: Request,
|
||||||
query: str | None = Query(default=None, max_length=200),
|
query: str | None = Query(default=None, max_length=200),
|
||||||
file_kind: ArtifactLibraryKind = Query(default="all"),
|
file_kind: ArtifactLibraryKind = Query(default="all"),
|
||||||
|
source_type: ArtifactLibrarySource = Query(default="all"),
|
||||||
offset: int = Query(default=0, ge=0),
|
offset: int = Query(default=0, ge=0),
|
||||||
limit: int = Query(default=50, ge=1, le=200),
|
limit: int = Query(default=50, ge=1, le=200),
|
||||||
refresh: bool = Query(default=False),
|
refresh: bool = Query(default=False),
|
||||||
@@ -404,14 +413,35 @@ async def list_artifact_library(
|
|||||||
explicit ``present_files`` call. Uploads and image/audio/video outputs are
|
explicit ``present_files`` call. Uploads and image/audio/video outputs are
|
||||||
intentionally outside this collection.
|
intentionally outside this collection.
|
||||||
"""
|
"""
|
||||||
from app.gateway.deps import get_thread_store
|
from app.gateway.deps import get_agent_store, get_thread_store
|
||||||
|
|
||||||
items = await _cached_artifact_library(get_thread_store(request), refresh=refresh)
|
items = await _cached_artifact_library(get_thread_store(request), refresh=refresh)
|
||||||
|
agent_ids = list(dict.fromkeys(str(item.get("agent_id") or "") for item in items if item.get("agent_id")))
|
||||||
|
agent_names: dict[str, str] = {}
|
||||||
|
if agent_ids:
|
||||||
|
try:
|
||||||
|
agents = await get_agent_store(request).list_by_ids(
|
||||||
|
agent_ids,
|
||||||
|
str(get_effective_user_id() or "default"),
|
||||||
|
)
|
||||||
|
agent_names = {str(agent.get("id") or ""): str(agent.get("name") or "").strip() for agent in agents if agent.get("id") and agent.get("name")}
|
||||||
|
except Exception:
|
||||||
|
logger.warning("Failed to resolve artifact library agent names", exc_info=True)
|
||||||
|
|
||||||
|
items_with_sources = [
|
||||||
|
{
|
||||||
|
**item,
|
||||||
|
"agent_name": (agent_names.get(str(item.get("agent_id") or "")) or item.get("agent_name") or item.get("agent_id")),
|
||||||
|
}
|
||||||
|
for item in items
|
||||||
|
]
|
||||||
normalized_query = (query or "").strip().casefold()
|
normalized_query = (query or "").strip().casefold()
|
||||||
filtered = [
|
filtered = [
|
||||||
item
|
item
|
||||||
for item in items
|
for item in items_with_sources
|
||||||
if (file_kind == "all" or item["file_kind"] == file_kind) and (not normalized_query or normalized_query in str(item.get("name") or "").casefold() or normalized_query in str(item.get("thread_title") or "").casefold())
|
if (file_kind == "all" or item["file_kind"] == file_kind)
|
||||||
|
and (source_type == "all" or item["source_type"] == source_type)
|
||||||
|
and (not normalized_query or normalized_query in str(item.get("name") or "").casefold() or normalized_query in str(item.get("thread_title") or "").casefold() or normalized_query in str(item.get("agent_name") or "").casefold())
|
||||||
]
|
]
|
||||||
page = filtered[offset : offset + limit]
|
page = filtered[offset : offset + limit]
|
||||||
return ArtifactLibraryResponse(
|
return ArtifactLibraryResponse(
|
||||||
@@ -420,176 +450,176 @@ async def list_artifact_library(
|
|||||||
limit=limit,
|
limit=limit,
|
||||||
items=[ArtifactLibraryItem.model_validate(item) for item in page],
|
items=[ArtifactLibraryItem.model_validate(item) for item in page],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@router.get(
|
@router.get(
|
||||||
"/threads/{thread_id}/artifacts",
|
"/threads/{thread_id}/artifacts",
|
||||||
summary="List Artifact Files",
|
summary="List Artifact Files",
|
||||||
description="List all files under the thread's outputs directory (/mnt/user-data/outputs/). Returns an empty list if the thread has no outputs.",
|
description="List all files under the thread's outputs directory (/mnt/user-data/outputs/). Returns an empty list if the thread has no outputs.",
|
||||||
)
|
)
|
||||||
@require_permission("threads", "read", owner_check=True)
|
@require_permission("threads", "read", owner_check=True)
|
||||||
async def list_artifacts(thread_id: str, request: Request) -> dict:
|
async def list_artifacts(thread_id: str, request: Request) -> dict:
|
||||||
"""列出 thread 在 ``outputs/`` 下生成的产物文件。
|
"""列出 thread 在 ``outputs/`` 下生成的产物文件。
|
||||||
|
|
||||||
圆桌 Step 3(``HighFidelityReport``)对每个**子席位** thread_id 调用本接口,
|
圆桌 Step 3(``HighFidelityReport``)对每个**子席位** thread_id 调用本接口,
|
||||||
展示各智能体写入沙箱 ``mnt/user-data/outputs/`` 的文件;协调智能体 thread
|
展示各智能体写入沙箱 ``mnt/user-data/outputs/`` 的文件;协调智能体 thread
|
||||||
通常无独立交付文件。
|
通常无独立交付文件。
|
||||||
|
|
||||||
遍历路径::
|
遍历路径::
|
||||||
|
|
||||||
.deer-flow/users/{user_id}/threads/{thread_id}/user-data/outputs/
|
.deer-flow/users/{user_id}/threads/{thread_id}/user-data/outputs/
|
||||||
|
|
||||||
返回 ``{ "thread_id", "files": [{ path, name, size_bytes, mime_type, modified_at }] }``,
|
返回 ``{ "thread_id", "files": [{ path, name, size_bytes, mime_type, modified_at }] }``,
|
||||||
目录不存在时 ``files`` 为空数组(非 404)。
|
目录不存在时 ``files`` 为空数组(非 404)。
|
||||||
"""
|
"""
|
||||||
outputs_dir = get_paths().sandbox_outputs_dir(thread_id, user_id=await aresolve_path_user_id(thread_id))
|
outputs_dir = get_paths().sandbox_outputs_dir(thread_id, user_id=await aresolve_path_user_id(thread_id))
|
||||||
return {"thread_id": thread_id, "files": collect_outputs_listing(outputs_dir)}
|
return {"thread_id": thread_id, "files": collect_outputs_listing(outputs_dir)}
|
||||||
|
|
||||||
|
|
||||||
@router.get(
|
@router.get(
|
||||||
"/threads/{thread_id}/artifacts/{path:path}",
|
"/threads/{thread_id}/artifacts/{path:path}",
|
||||||
summary="Get Artifact File",
|
summary="Get Artifact File",
|
||||||
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
|
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
|
||||||
)
|
)
|
||||||
@require_permission("threads", "read", owner_check=True)
|
@require_permission("threads", "read", owner_check=True)
|
||||||
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
|
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
|
||||||
"""Get an artifact file by its path.
|
"""Get an artifact file by its path.
|
||||||
|
|
||||||
The endpoint automatically detects file types and returns appropriate content types.
|
The endpoint automatically detects file types and returns appropriate content types.
|
||||||
Use the `download` query parameter to force file download for non-active content.
|
Use the `download` query parameter to force file download for non-active content.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
thread_id: The thread ID.
|
thread_id: The thread ID.
|
||||||
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
|
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
|
||||||
request: FastAPI request object (automatically injected).
|
request: FastAPI request object (automatically injected).
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The file content as a FileResponse with appropriate content type:
|
The file content as a FileResponse with appropriate content type:
|
||||||
- Active content (HTML/XHTML/SVG): Served as download attachment
|
- Active content (HTML/XHTML/SVG): Served as download attachment
|
||||||
- Text files: Plain text with proper MIME type
|
- Text files: Plain text with proper MIME type
|
||||||
- Binary files: Inline display with download option
|
- Binary files: Inline display with download option
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
HTTPException:
|
HTTPException:
|
||||||
- 400 if path is invalid or not a file
|
- 400 if path is invalid or not a file
|
||||||
- 403 if access denied (path traversal detected)
|
- 403 if access denied (path traversal detected)
|
||||||
- 404 if file not found
|
- 404 if file not found
|
||||||
|
|
||||||
Query Parameters:
|
Query Parameters:
|
||||||
download (bool): If true, forces attachment download for file types that are
|
download (bool): If true, forces attachment download for file types that are
|
||||||
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
|
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
|
||||||
is always downloaded regardless of this flag.
|
is always downloaded regardless of this flag.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
|
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
|
||||||
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
|
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
|
||||||
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
|
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
|
||||||
"""
|
"""
|
||||||
path = coerce_artifact_virtual_path(path)
|
path = coerce_artifact_virtual_path(path)
|
||||||
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
|
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
|
||||||
if ".skill/" in path:
|
if ".skill/" in path:
|
||||||
# Split the path at ".skill/" to get the ZIP file path and internal path
|
# Split the path at ".skill/" to get the ZIP file path and internal path
|
||||||
skill_marker = ".skill/"
|
skill_marker = ".skill/"
|
||||||
marker_pos = path.find(skill_marker)
|
marker_pos = path.find(skill_marker)
|
||||||
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
|
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
|
||||||
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
|
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
|
||||||
|
|
||||||
actual_skill_path = await aresolve_thread_virtual_path(thread_id, skill_file_path)
|
actual_skill_path = await aresolve_thread_virtual_path(thread_id, skill_file_path)
|
||||||
|
|
||||||
if not actual_skill_path.exists():
|
if not actual_skill_path.exists():
|
||||||
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
|
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
|
||||||
|
|
||||||
if not actual_skill_path.is_file():
|
if not actual_skill_path.is_file():
|
||||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
|
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
|
||||||
|
|
||||||
# Extract the file from the .skill archive
|
# Extract the file from the .skill archive
|
||||||
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
|
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
|
||||||
if content is None:
|
if content is None:
|
||||||
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
|
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
|
||||||
|
|
||||||
# Determine MIME type based on the internal file
|
# Determine MIME type based on the internal file
|
||||||
mime_type, _ = mimetypes.guess_type(internal_path)
|
mime_type, _ = mimetypes.guess_type(internal_path)
|
||||||
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
|
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
|
||||||
cache_headers = {"Cache-Control": "private, max-age=300"}
|
cache_headers = {"Cache-Control": "private, max-age=300"}
|
||||||
download_name = Path(internal_path).name or actual_skill_path.stem
|
download_name = Path(internal_path).name or actual_skill_path.stem
|
||||||
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||||
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
|
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
|
||||||
|
|
||||||
if mime_type and mime_type.startswith("text/"):
|
if mime_type and mime_type.startswith("text/"):
|
||||||
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
|
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
|
||||||
|
|
||||||
# Default to plain text for unknown types that look like text
|
# Default to plain text for unknown types that look like text
|
||||||
try:
|
try:
|
||||||
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
|
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
|
||||||
except UnicodeDecodeError:
|
except UnicodeDecodeError:
|
||||||
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
|
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
|
||||||
|
|
||||||
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
||||||
|
|
||||||
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
||||||
|
|
||||||
if not actual_path.exists():
|
if not actual_path.exists():
|
||||||
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
||||||
|
|
||||||
if not actual_path.is_file():
|
if not actual_path.is_file():
|
||||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
||||||
|
|
||||||
mime_type, _ = mimetypes.guess_type(actual_path)
|
mime_type, _ = mimetypes.guess_type(actual_path)
|
||||||
|
|
||||||
if download:
|
if download:
|
||||||
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
||||||
|
|
||||||
# Always force download for active content types to prevent script execution
|
# Always force download for active content types to prevent script execution
|
||||||
# in the application origin when users open generated artifacts.
|
# in the application origin when users open generated artifacts.
|
||||||
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||||
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
|
||||||
|
|
||||||
if mime_type and mime_type.startswith("text/"):
|
if mime_type and mime_type.startswith("text/"):
|
||||||
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
||||||
|
|
||||||
if is_text_file_by_content(actual_path):
|
if is_text_file_by_content(actual_path):
|
||||||
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
|
||||||
|
|
||||||
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
|
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
|
||||||
|
|
||||||
|
|
||||||
@router.patch(
|
@router.patch(
|
||||||
"/threads/{thread_id}/artifacts/{path:path}",
|
"/threads/{thread_id}/artifacts/{path:path}",
|
||||||
summary="Update Artifact File",
|
summary="Update Artifact File",
|
||||||
description="Update a text artifact file in the current thread workspace.",
|
description="Update a text artifact file in the current thread workspace.",
|
||||||
)
|
)
|
||||||
@require_permission("threads", "write", owner_check=True)
|
@require_permission("threads", "write", owner_check=True)
|
||||||
async def update_artifact(
|
async def update_artifact(
|
||||||
thread_id: str,
|
thread_id: str,
|
||||||
path: str,
|
path: str,
|
||||||
request: Request,
|
request: Request,
|
||||||
body: ArtifactUpdateRequest,
|
body: ArtifactUpdateRequest,
|
||||||
) -> Response:
|
) -> Response:
|
||||||
"""Update a text artifact.
|
"""Update a text artifact.
|
||||||
|
|
||||||
This endpoint is intentionally limited to normal text artifacts. Files inside
|
This endpoint is intentionally limited to normal text artifacts. Files inside
|
||||||
.skill archives and active web content remain read-only from the artifact UI.
|
.skill archives and active web content remain read-only from the artifact UI.
|
||||||
"""
|
"""
|
||||||
if ".skill/" in path:
|
if ".skill/" in path:
|
||||||
raise HTTPException(status_code=400, detail="Skill archive contents are read-only.")
|
raise HTTPException(status_code=400, detail="Skill archive contents are read-only.")
|
||||||
|
|
||||||
path = coerce_artifact_virtual_path(path)
|
path = coerce_artifact_virtual_path(path)
|
||||||
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
actual_path = await aresolve_thread_virtual_path(thread_id, path)
|
||||||
logger.info(f"Updating artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
logger.info(f"Updating artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
|
||||||
|
|
||||||
if not actual_path.exists():
|
if not actual_path.exists():
|
||||||
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
|
||||||
|
|
||||||
if not actual_path.is_file():
|
if not actual_path.is_file():
|
||||||
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
|
||||||
|
|
||||||
mime_type, _ = mimetypes.guess_type(actual_path)
|
mime_type, _ = mimetypes.guess_type(actual_path)
|
||||||
suffix = actual_path.suffix.lower()
|
suffix = actual_path.suffix.lower()
|
||||||
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
|
||||||
raise HTTPException(status_code=400, detail="Active web artifacts are read-only.")
|
raise HTTPException(status_code=400, detail="Active web artifacts are read-only.")
|
||||||
|
|
||||||
if suffix not in EDITABLE_TEXT_SUFFIXES and not (mime_type and mime_type.startswith("text/")):
|
if suffix not in EDITABLE_TEXT_SUFFIXES and not (mime_type and mime_type.startswith("text/")):
|
||||||
raise HTTPException(status_code=400, detail="Only text artifacts can be edited.")
|
raise HTTPException(status_code=400, detail="Only text artifacts can be edited.")
|
||||||
|
|
||||||
actual_path.write_text(body.content, encoding="utf-8")
|
actual_path.write_text(body.content, encoding="utf-8")
|
||||||
return PlainTextResponse(content=body.content, media_type=mime_type or "text/plain")
|
return PlainTextResponse(content=body.content, media_type=mime_type or "text/plain")
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -172,21 +172,15 @@ async def create_sync_job(request: Request, body: CreateSyncJobRequest) -> dict[
|
|||||||
targets: list[dict[str, Any]] = []
|
targets: list[dict[str, Any]] = []
|
||||||
for target in body.targets:
|
for target in body.targets:
|
||||||
data = target.model_dump()
|
data = target.model_dump()
|
||||||
if target.target_type == "weknora":
|
if target.target_type != "weknora":
|
||||||
mapping = await request.app.state.llmwiki_store.get_authorized(target.target_id, actor or "system", write=True, is_admin=True)
|
raise HTTPException(status_code=422, detail="不支持技能直接归纳到助手知识库,请先归纳到 WeKnora 普通知识库")
|
||||||
if mapping is None:
|
if target.target_mode != "wiki":
|
||||||
raise HTTPException(status_code=404, detail=f"普通知识库不存在:{target.target_id}")
|
raise HTTPException(status_code=422, detail="技能归纳目标仅支持 WeKnora Wiki 知识库")
|
||||||
data["target_name"] = target.target_name or str(mapping.get("name") or target.target_id)
|
mapping = await request.app.state.llmwiki_store.get_authorized(target.target_id, actor or "system", write=True, is_admin=True)
|
||||||
data["remote_business_key"] = target.remote_business_key or f"weknora:kb:{mapping['weknora_id']}"
|
if mapping is None:
|
||||||
else:
|
raise HTTPException(status_code=404, detail=f"普通知识库不存在:{target.target_id}")
|
||||||
base = await request.app.state.assistant_knowledge_store.get_base()
|
data["target_name"] = target.target_name or str(mapping.get("name") or target.target_id)
|
||||||
if base is None:
|
data["remote_business_key"] = target.remote_business_key or f"weknora:kb:{mapping['weknora_id']}"
|
||||||
base = await request.app.state.assistant_knowledge_store.initialize(created_by=actor)
|
|
||||||
if target.target_id not in {base["id"], "global"}:
|
|
||||||
raise HTTPException(status_code=404, detail="助手知识库仅允许选择全局知识梳理库")
|
|
||||||
data["target_id"] = base["id"]
|
|
||||||
data["target_name"] = base["name"]
|
|
||||||
data["remote_business_key"] = f"assistant:base:{base['id']}"
|
|
||||||
targets.append(data)
|
targets.append(data)
|
||||||
job = await _store(request).create_job(
|
job = await _store(request).create_job(
|
||||||
skill_names=body.skill_names,
|
skill_names=body.skill_names,
|
||||||
|
|||||||
@@ -405,6 +405,7 @@ _CONTEXT_CONFIGURABLE_KEYS: frozenset[str] = frozenset(
|
|||||||
"reasoning_effort",
|
"reasoning_effort",
|
||||||
"rag_mode_enabled",
|
"rag_mode_enabled",
|
||||||
"llmwiki_knowledge_base_ids",
|
"llmwiki_knowledge_base_ids",
|
||||||
|
"assistant_knowledge_base_ids",
|
||||||
"writing_mode",
|
"writing_mode",
|
||||||
"writing_artifact_path",
|
"writing_artifact_path",
|
||||||
"canvas_artifact_title",
|
"canvas_artifact_title",
|
||||||
@@ -1113,7 +1114,7 @@ async def start_run(
|
|||||||
if prefix_settings.enabled and prefix_settings.prompt_prefix.strip():
|
if prefix_settings.enabled and prefix_settings.prompt_prefix.strip():
|
||||||
graph_input = _prepend_prompt_prefix(graph_input, prefix_settings.prompt_prefix.strip())
|
graph_input = _prepend_prompt_prefix(graph_input, prefix_settings.prompt_prefix.strip())
|
||||||
llmwiki_rag = (
|
llmwiki_rag = (
|
||||||
LlmWikiRagContext([], 0, "", [], "")
|
LlmWikiRagContext([], [], 0, "", [], "")
|
||||||
if fixed_answer is not None
|
if fixed_answer is not None
|
||||||
else await build_llmwiki_rag_context(
|
else await build_llmwiki_rag_context(
|
||||||
request=request,
|
request=request,
|
||||||
@@ -1125,12 +1126,14 @@ async def start_run(
|
|||||||
graph_input = attach_llmwiki_rag_to_graph_input(graph_input, llmwiki_rag)
|
graph_input = attach_llmwiki_rag_to_graph_input(graph_input, llmwiki_rag)
|
||||||
effective_rag_mode = True
|
effective_rag_mode = True
|
||||||
run_metadata["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
|
run_metadata["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
|
||||||
|
run_metadata["assistant_knowledge_base_ids"] = llmwiki_rag.assistant_knowledge_base_ids
|
||||||
run_metadata["llmwiki_rag_source_count"] = llmwiki_rag.source_count
|
run_metadata["llmwiki_rag_source_count"] = llmwiki_rag.source_count
|
||||||
try:
|
try:
|
||||||
await run_ctx.thread_store.update_metadata(
|
await run_ctx.thread_store.update_metadata(
|
||||||
thread_id,
|
thread_id,
|
||||||
{
|
{
|
||||||
"llmwiki_knowledge_base_ids": llmwiki_rag.knowledge_base_ids,
|
"llmwiki_knowledge_base_ids": llmwiki_rag.knowledge_base_ids,
|
||||||
|
"assistant_knowledge_base_ids": llmwiki_rag.assistant_knowledge_base_ids,
|
||||||
"rag_mode_enabled": True,
|
"rag_mode_enabled": True,
|
||||||
"rag_mode": True,
|
"rag_mode": True,
|
||||||
},
|
},
|
||||||
@@ -1150,6 +1153,8 @@ async def start_run(
|
|||||||
body_context["rag_mode_enabled"] = True
|
body_context["rag_mode_enabled"] = True
|
||||||
if "llmwiki_knowledge_base_ids" not in body_context and llmwiki_rag.knowledge_base_ids:
|
if "llmwiki_knowledge_base_ids" not in body_context and llmwiki_rag.knowledge_base_ids:
|
||||||
body_context["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
|
body_context["llmwiki_knowledge_base_ids"] = llmwiki_rag.knowledge_base_ids
|
||||||
|
if "assistant_knowledge_base_ids" not in body_context and llmwiki_rag.assistant_knowledge_base_ids:
|
||||||
|
body_context["assistant_knowledge_base_ids"] = llmwiki_rag.assistant_knowledge_base_ids
|
||||||
if not llmwiki_rag.knowledge_base_ids:
|
if not llmwiki_rag.knowledge_base_ids:
|
||||||
existing_excluded = list(body_context.get("excluded_tools") or getattr(body, "excluded_tools", None) or [])
|
existing_excluded = list(body_context.get("excluded_tools") or getattr(body, "excluded_tools", None) or [])
|
||||||
if "llmwiki_search" not in existing_excluded:
|
if "llmwiki_search" not in existing_excluded:
|
||||||
|
|||||||
@@ -429,7 +429,11 @@ auth_login:
|
|||||||
database:
|
database:
|
||||||
backend: sqlite
|
backend: sqlite
|
||||||
sqlite_dir: .deer-flow/data
|
sqlite_dir: .deer-flow/data
|
||||||
pool_size: 20 # 基础连接池(postgres/mysql 生效;sqlite 忽略)
|
# 高斯/OpenGauss 业务库示例(启用前先执行 uv sync --extra gauss):
|
||||||
|
# backend: gauss
|
||||||
|
# gauss_url: $GAUSS_DATABASE_URL # opengauss+asyncpg://user:pass@host:26000/deerflow
|
||||||
|
# checkpointer 请继续使用下方 sqlite;不要把 PostgreSQL 专用 checkpointer 迁移直接用于高斯。
|
||||||
|
pool_size: 20 # 基础连接池(postgres/gauss/mysql 生效;sqlite 忽略)
|
||||||
max_overflow: 20 # 高峰期可临时超出 pool_size 的额外连接数,真实上限 = pool_size + max_overflow
|
max_overflow: 20 # 高峰期可临时超出 pool_size 的额外连接数,真实上限 = pool_size + max_overflow
|
||||||
pool_timeout: 30 # 池满时等待空闲连接的秒数,超时报错
|
pool_timeout: 30 # 池满时等待空闲连接的秒数,超时报错
|
||||||
pool_recycle: 1800 # 连接回收周期(秒);防火墙/pgbouncer/MySQL wait_timeout 静默断连兜底。多 worker 部署:每 worker 上限 = pool_size + max_overflow,总数 × worker 数 < DB max_connections
|
pool_recycle: 1800 # 连接回收周期(秒);防火墙/pgbouncer/MySQL wait_timeout 静默断连兜底。多 worker 部署:每 worker 上限 = pool_size + max_overflow,总数 × worker 数 < DB max_connections
|
||||||
@@ -564,9 +568,10 @@ llmwiki:
|
|||||||
password: ""
|
password: ""
|
||||||
database: "neo4j"
|
database: "neo4j"
|
||||||
timeout_seconds: 30
|
timeout_seconds: 30
|
||||||
# 本地 Wiki 向量化:只需把 enabled 改为 true,并填写下面 3 个 embedding 值。
|
# 本地 Wiki 向量化:用于普通知识库“下载向量化数据包”和助手知识库向量检索。
|
||||||
# 默认只在页面点击“向量化”时执行;auto_sync=true 才会后台自动增量同步,
|
# 已有相同内容哈希 + 相同 embedding fingerprint 的 Wiki 向量会直接复用,不会重复编码。
|
||||||
# 且服务启动时只启动定时器,不会立刻全量向量化。
|
# 如需导出时自动补建向量,把 enabled 改为 true,并填写下面 embedding 服务地址。
|
||||||
|
# auto_sync=true 才会后台自动增量同步;服务启动时只启动定时器,不会立刻全量向量化。
|
||||||
local_wiki_index:
|
local_wiki_index:
|
||||||
enabled: false
|
enabled: false
|
||||||
# 免登录 POST /api/knowledge/vector-search 返回的 Wiki 一键跳转链接前缀。
|
# 免登录 POST /api/knowledge/vector-search 返回的 Wiki 一键跳转链接前缀。
|
||||||
@@ -574,7 +579,7 @@ llmwiki:
|
|||||||
auto_sync: false
|
auto_sync: false
|
||||||
database_batch_size: 50
|
database_batch_size: 50
|
||||||
embedding:
|
embedding:
|
||||||
model: bge-m3
|
model: bge-embedding-m3
|
||||||
base_url: ""
|
base_url: ""
|
||||||
api_key: ""
|
api_key: ""
|
||||||
|
|
||||||
|
|||||||
+109
@@ -0,0 +1,109 @@
|
|||||||
|
"""File-backed assistant knowledge import/export packages.
|
||||||
|
|
||||||
|
The assistant knowledge base is fed by WeKnora-exported data. Keep that
|
||||||
|
boundary explicit: a WeKnora export is first materialized as a durable JSONL
|
||||||
|
package, then imported asynchronously from that package. JSONL keeps large
|
||||||
|
exports append-friendly and avoids treating the HTTP request body as the job
|
||||||
|
state.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from collections.abc import Iterable, Mapping
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
PACKAGE_VERSION = 1
|
||||||
|
PACKAGE_RECORD_TYPES = {
|
||||||
|
"source",
|
||||||
|
"wiki_page",
|
||||||
|
"document",
|
||||||
|
"chunk",
|
||||||
|
"vector",
|
||||||
|
"entity",
|
||||||
|
"relation",
|
||||||
|
"graph",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def package_path(root: Path, job_id: str) -> Path:
|
||||||
|
safe = "".join(ch for ch in str(job_id) if ch.isalnum() or ch in {"-", "_"})
|
||||||
|
return root / f"{safe}.jsonl"
|
||||||
|
|
||||||
|
|
||||||
|
def write_jsonl_record(handle: Any, record_type: str, payload: Mapping[str, Any]) -> None:
|
||||||
|
handle.write(json.dumps({"record_type": record_type, "payload": dict(payload)}, ensure_ascii=False))
|
||||||
|
handle.write("\n")
|
||||||
|
|
||||||
|
|
||||||
|
def write_package(path: Path, records: Iterable[tuple[str, Mapping[str, Any]]]) -> dict[str, int]:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
counts: dict[str, int] = {key: 0 for key in PACKAGE_RECORD_TYPES}
|
||||||
|
tmp_path = path.with_suffix(path.suffix + ".tmp")
|
||||||
|
with tmp_path.open("w", encoding="utf-8", newline="\n") as handle:
|
||||||
|
write_jsonl_record(
|
||||||
|
handle,
|
||||||
|
"source",
|
||||||
|
{
|
||||||
|
"package_version": PACKAGE_VERSION,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
counts["source"] += 1
|
||||||
|
for record_type, payload in records:
|
||||||
|
if record_type not in PACKAGE_RECORD_TYPES:
|
||||||
|
continue
|
||||||
|
write_jsonl_record(handle, record_type, payload)
|
||||||
|
counts[record_type] = counts.get(record_type, 0) + 1
|
||||||
|
tmp_path.replace(path)
|
||||||
|
return counts
|
||||||
|
|
||||||
|
|
||||||
|
def read_package(path: Path) -> dict[str, Any]:
|
||||||
|
package: dict[str, Any] = {
|
||||||
|
"source": {},
|
||||||
|
"wiki_pages": [],
|
||||||
|
"documents": [],
|
||||||
|
"chunks": [],
|
||||||
|
"vectors": [],
|
||||||
|
"entities": [],
|
||||||
|
"relations": [],
|
||||||
|
"graph": {"nodes": [], "edges": []},
|
||||||
|
}
|
||||||
|
with path.open("r", encoding="utf-8") as handle:
|
||||||
|
for line in handle:
|
||||||
|
line = line.strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
record = json.loads(line)
|
||||||
|
if not isinstance(record, dict):
|
||||||
|
continue
|
||||||
|
record_type = str(record.get("record_type") or "")
|
||||||
|
payload = record.get("payload")
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
continue
|
||||||
|
if record_type == "source":
|
||||||
|
package["source"] = {**package.get("source", {}), **payload}
|
||||||
|
elif record_type == "wiki_page":
|
||||||
|
package["wiki_pages"].append(payload)
|
||||||
|
elif record_type == "document":
|
||||||
|
package["documents"].append(payload)
|
||||||
|
elif record_type == "chunk":
|
||||||
|
package["chunks"].append(payload)
|
||||||
|
elif record_type == "vector":
|
||||||
|
package["vectors"].append(payload)
|
||||||
|
elif record_type == "entity":
|
||||||
|
package["entities"].append(payload)
|
||||||
|
elif record_type == "relation":
|
||||||
|
package["relations"].append(payload)
|
||||||
|
elif record_type == "graph":
|
||||||
|
nodes = payload.get("nodes")
|
||||||
|
edges = payload.get("edges")
|
||||||
|
if isinstance(nodes, list):
|
||||||
|
package["graph"]["nodes"].extend(item for item in nodes if isinstance(item, dict))
|
||||||
|
if isinstance(edges, list):
|
||||||
|
package["graph"]["edges"].extend(item for item in edges if isinstance(item, dict))
|
||||||
|
for key, value in payload.items():
|
||||||
|
if key not in {"nodes", "edges"}:
|
||||||
|
package["graph"][key] = value
|
||||||
|
return package
|
||||||
+68
-21
@@ -11,10 +11,15 @@ blocking, making a unified file safe for both workloads. Writers
|
|||||||
that contend for the lock wait via the default 5-second sqlite3
|
that contend for the lock wait via the default 5-second sqlite3
|
||||||
busy timeout rather than failing immediately.
|
busy timeout rather than failing immediately.
|
||||||
|
|
||||||
Postgres mode: both use the same database URL but maintain independent
|
Postgres mode: both use the same database URL but maintain independent
|
||||||
connection pools with different lifecycles.
|
connection pools with different lifecycles.
|
||||||
|
|
||||||
MySQL mode: application ORM data uses MySQL. LangGraph's checkpointer does
|
Gauss mode: application ORM data uses the official openGauss SQLAlchemy
|
||||||
|
dialect over asyncpg. LangGraph's PostgreSQL checkpointer is intentionally not
|
||||||
|
reused for GaussDB; configure the legacy ``checkpointer:`` section separately
|
||||||
|
(normally sqlite).
|
||||||
|
|
||||||
|
MySQL mode: application ORM data uses MySQL. LangGraph's checkpointer does
|
||||||
not have a built-in MySQL backend in this project, so deployments that choose
|
not have a built-in MySQL backend in this project, so deployments that choose
|
||||||
MySQL for business tables should configure the legacy ``checkpointer:``
|
MySQL for business tables should configure the legacy ``checkpointer:``
|
||||||
section separately (usually sqlite) to preserve conversation checkpoints.
|
section separately (usually sqlite) to preserve conversation checkpoints.
|
||||||
@@ -42,24 +47,40 @@ from typing import Literal
|
|||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
|
||||||
class DatabaseConfig(BaseModel):
|
class DatabaseConfig(BaseModel):
|
||||||
backend: Literal["memory", "sqlite", "postgres", "mysql"] = Field(
|
backend: Literal["memory", "sqlite", "postgres", "gauss", "mysql"] = Field(
|
||||||
default="memory",
|
default="memory",
|
||||||
description=("Storage backend for application data. 'memory' for development (no persistence across restarts), 'sqlite' for single-node deployment, 'postgres' for production multi-node deployment, 'mysql' for MySQL-backed business tables."),
|
description=(
|
||||||
|
"Storage backend for application data. 'memory' for development "
|
||||||
|
"(no persistence across restarts), 'sqlite' for single-node "
|
||||||
|
"deployment, 'postgres' for production multi-node deployment, "
|
||||||
|
"'gauss' for openGauss/GaussDB business tables, and 'mysql' for "
|
||||||
|
"MySQL-backed business tables."
|
||||||
|
),
|
||||||
)
|
)
|
||||||
sqlite_dir: str = Field(
|
sqlite_dir: str = Field(
|
||||||
default=".deer-flow/data",
|
default=".deer-flow/data",
|
||||||
description=("Directory for the SQLite database file. Both checkpointer and application data share {sqlite_dir}/deerflow.db."),
|
description=("Directory for the SQLite database file. Both checkpointer and application data share {sqlite_dir}/deerflow.db."),
|
||||||
)
|
)
|
||||||
postgres_url: str = Field(
|
postgres_url: str = Field(
|
||||||
default="",
|
default="",
|
||||||
description=(
|
description=(
|
||||||
"PostgreSQL connection URL, shared by checkpointer and app. "
|
"PostgreSQL connection URL, shared by checkpointer and app. "
|
||||||
"Use $DATABASE_URL in config.yaml to reference .env. "
|
"Use $DATABASE_URL in config.yaml to reference .env. "
|
||||||
"Example: postgresql://user:pass@host:5432/deerflow "
|
"Example: postgresql://user:pass@host:5432/deerflow "
|
||||||
"(the +asyncpg driver suffix is added automatically where needed)."
|
"(the +asyncpg driver suffix is added automatically where needed)."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
gauss_url: str = Field(
|
||||||
|
default="",
|
||||||
|
description=(
|
||||||
|
"openGauss/GaussDB connection URL for the application ORM. "
|
||||||
|
"Use $GAUSS_DATABASE_URL in config.yaml to reference .env. "
|
||||||
|
"Example: opengauss+asyncpg://user:pass@host:26000/deerflow. "
|
||||||
|
"Plain opengauss:// or PostgreSQL-style URLs are normalized to the "
|
||||||
|
"official asynchronous openGauss SQLAlchemy dialect."
|
||||||
|
),
|
||||||
|
)
|
||||||
mysql_url: str = Field(
|
mysql_url: str = Field(
|
||||||
default="",
|
default="",
|
||||||
description=(
|
description=(
|
||||||
@@ -75,26 +96,26 @@ class DatabaseConfig(BaseModel):
|
|||||||
)
|
)
|
||||||
pool_size: int = Field(
|
pool_size: int = Field(
|
||||||
default=5,
|
default=5,
|
||||||
description="Connection pool size for the app ORM engine (postgres/mysql only).",
|
description="Connection pool size for the app ORM engine (postgres/gauss/mysql only).",
|
||||||
)
|
)
|
||||||
max_overflow: int = Field(
|
max_overflow: int = Field(
|
||||||
default=10,
|
default=10,
|
||||||
description=(
|
description=(
|
||||||
"Extra connections allowed beyond pool_size under load (postgres/mysql only). "
|
"Extra connections allowed beyond pool_size under load (postgres/gauss/mysql only). "
|
||||||
"The real connection ceiling is pool_size + max_overflow. SQLAlchemy default is 10."
|
"The real connection ceiling is pool_size + max_overflow. SQLAlchemy default is 10."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
pool_timeout: int = Field(
|
pool_timeout: int = Field(
|
||||||
default=30,
|
default=30,
|
||||||
description=(
|
description=(
|
||||||
"Seconds a caller waits for a free pooled connection before erroring (postgres/mysql only). "
|
"Seconds a caller waits for a free pooled connection before erroring (postgres/gauss/mysql only). "
|
||||||
"SQLAlchemy default is 30."
|
"SQLAlchemy default is 30."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
pool_recycle: int = Field(
|
pool_recycle: int = Field(
|
||||||
default=1800,
|
default=1800,
|
||||||
description=(
|
description=(
|
||||||
"Recycle pooled connections after this many seconds (postgres/mysql only). "
|
"Recycle pooled connections after this many seconds (postgres/gauss/mysql only). "
|
||||||
"Prevents stale-connection errors when a firewall, pgbouncer, or MySQL "
|
"Prevents stale-connection errors when a firewall, pgbouncer, or MySQL "
|
||||||
"wait_timeout silently drops idle connections. pool_pre_ping catches most "
|
"wait_timeout silently drops idle connections. pool_pre_ping catches most "
|
||||||
"cases, but pool_recycle is the belt-and-suspenders safety net. "
|
"cases, but pool_recycle is the belt-and-suspenders safety net. "
|
||||||
@@ -133,12 +154,38 @@ class DatabaseConfig(BaseModel):
|
|||||||
"""SQLAlchemy async URL for the application ORM engine."""
|
"""SQLAlchemy async URL for the application ORM engine."""
|
||||||
if self.backend == "sqlite":
|
if self.backend == "sqlite":
|
||||||
return f"sqlite+aiosqlite:///{self.sqlite_path}"
|
return f"sqlite+aiosqlite:///{self.sqlite_path}"
|
||||||
if self.backend == "postgres":
|
if self.backend == "postgres":
|
||||||
url = self.postgres_url
|
url = self.postgres_url
|
||||||
if url.startswith("postgresql://"):
|
if url.startswith("postgresql://"):
|
||||||
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||||
return url
|
return url
|
||||||
if self.backend == "mysql":
|
if self.backend == "gauss":
|
||||||
|
url = self.gauss_url.strip()
|
||||||
|
if not url:
|
||||||
|
raise ValueError("database.gauss_url is required for backend='gauss'")
|
||||||
|
|
||||||
|
# GaussDB speaks the PostgreSQL wire protocol, but its server
|
||||||
|
# version string and DDL capabilities are not identical to
|
||||||
|
# PostgreSQL. Always select the official openGauss dialect instead
|
||||||
|
# of silently sending the URL through SQLAlchemy's generic PG
|
||||||
|
# dialect (the latter commonly connects and then fails while
|
||||||
|
# SQLAlchemy initializes/creates the schema).
|
||||||
|
for prefix in (
|
||||||
|
"opengauss://",
|
||||||
|
"gaussdb://",
|
||||||
|
"postgres://",
|
||||||
|
"postgresql://",
|
||||||
|
"postgresql+asyncpg://",
|
||||||
|
):
|
||||||
|
if url.startswith(prefix):
|
||||||
|
return "opengauss+asyncpg://" + url[len(prefix) :]
|
||||||
|
if url.startswith("opengauss+asyncpg://"):
|
||||||
|
return url
|
||||||
|
raise ValueError(
|
||||||
|
"database.gauss_url must use opengauss://, "
|
||||||
|
"opengauss+asyncpg://, or a PostgreSQL-style URL"
|
||||||
|
)
|
||||||
|
if self.backend == "mysql":
|
||||||
url = self.mysql_url
|
url = self.mysql_url
|
||||||
if not url:
|
if not url:
|
||||||
raise ValueError("database.mysql_url is required for backend='mysql'")
|
raise ValueError("database.mysql_url is required for backend='mysql'")
|
||||||
|
|||||||
@@ -73,7 +73,7 @@ class WeKnoraDirectGraphConfig(BaseModel):
|
|||||||
class LocalWikiEmbeddingConfig(BaseModel):
|
class LocalWikiEmbeddingConfig(BaseModel):
|
||||||
enabled: bool = True
|
enabled: bool = True
|
||||||
provider: str = "openai"
|
provider: str = "openai"
|
||||||
model: str = "bge-m3"
|
model: str = "bge-embedding-m3"
|
||||||
base_url: str = ""
|
base_url: str = ""
|
||||||
api_key: str = Field(default="", repr=False)
|
api_key: str = Field(default="", repr=False)
|
||||||
dimensions: int | None = Field(
|
dimensions: int | None = Field(
|
||||||
|
|||||||
+1
-1
@@ -494,7 +494,7 @@ class WeKnoraClient:
|
|||||||
await self._request(
|
await self._request(
|
||||||
"GET",
|
"GET",
|
||||||
f"/api/v1/knowledgebase/{quote(knowledge_base_id, safe='')}/wiki/graph",
|
f"/api/v1/knowledgebase/{quote(knowledge_base_id, safe='')}/wiki/graph",
|
||||||
params={"mode": "overview", "limit": max(1, min(limit, 500))},
|
params={"mode": "overview", "limit": max(1, min(limit, 10000))},
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
if not isinstance(value, dict):
|
if not isinstance(value, dict):
|
||||||
|
|||||||
+34
-4
@@ -1,7 +1,37 @@
|
|||||||
"""DeerFlow-local mirror and Wiki-only retrieval for WeKnora Wiki pages."""
|
"""DeerFlow-local mirror and Wiki-only retrieval for WeKnora Wiki pages."""
|
||||||
|
|
||||||
from deerflow.integrations.weknora.local_index.retrieval import WikiRetrievalService, is_fully_vectorized
|
from __future__ import annotations
|
||||||
from deerflow.integrations.weknora.local_index.search import WikiIndexError, WikiVectorSearchService
|
|
||||||
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
|
||||||
|
|
||||||
__all__ = ["WikiIndexError", "WikiRetrievalService", "WikiVectorSearchService", "WikiSyncService", "is_fully_vectorized"]
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from deerflow.integrations.weknora.local_index.retrieval import (
|
||||||
|
WikiRetrievalService,
|
||||||
|
is_fully_vectorized,
|
||||||
|
)
|
||||||
|
from deerflow.integrations.weknora.local_index.search import WikiIndexError, WikiVectorSearchService
|
||||||
|
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"WikiIndexError",
|
||||||
|
"WikiRetrievalService",
|
||||||
|
"WikiVectorSearchService",
|
||||||
|
"WikiSyncService",
|
||||||
|
"is_fully_vectorized",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def __getattr__(name: str) -> Any:
|
||||||
|
if name in {"WikiRetrievalService", "is_fully_vectorized"}:
|
||||||
|
from deerflow.integrations.weknora.local_index import retrieval
|
||||||
|
|
||||||
|
return getattr(retrieval, name)
|
||||||
|
if name in {"WikiIndexError", "WikiVectorSearchService"}:
|
||||||
|
from deerflow.integrations.weknora.local_index import search
|
||||||
|
|
||||||
|
return getattr(search, name)
|
||||||
|
if name == "WikiSyncService":
|
||||||
|
from deerflow.integrations.weknora.local_index.sync import WikiSyncService
|
||||||
|
|
||||||
|
return WikiSyncService
|
||||||
|
raise AttributeError(name)
|
||||||
+34
-3
@@ -1,10 +1,10 @@
|
|||||||
"""ORM model for the single global DeerFlow assistant knowledge base."""
|
"""ORM model for DeerFlow-native assistant knowledge bases."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import Float, Integer, String, UniqueConstraint
|
from sqlalchemy import Float, Index, Integer, LargeBinary, String, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
@@ -19,7 +19,12 @@ class AssistantKnowledgeBaseRow(Base):
|
|||||||
__tablename__ = "assistant_knowledge_base"
|
__tablename__ = "assistant_knowledge_base"
|
||||||
id: Mapped[str] = mapped_column(String(36), primary_key=True)
|
id: Mapped[str] = mapped_column(String(36), primary_key=True)
|
||||||
name: Mapped[str] = mapped_column(String(255), default="知识梳理")
|
name: Mapped[str] = mapped_column(String(255), default="知识梳理")
|
||||||
status: Mapped[str] = mapped_column(String(24), default="active", unique=True)
|
description: Mapped[str | None] = mapped_column(String(2048), nullable=True)
|
||||||
|
# ``global`` is the full-library aggregation base. ``custom`` bases are
|
||||||
|
# user-created assistant knowledge bases fed from WeKnora export packages.
|
||||||
|
# Nullable keeps startup column reconciliation safe for existing databases.
|
||||||
|
base_kind: Mapped[str | None] = mapped_column(String(24), nullable=True, index=True)
|
||||||
|
status: Mapped[str] = mapped_column(String(64), default="active")
|
||||||
created_by: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
created_by: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||||
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
|
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
|
||||||
updated_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now, onupdate=_now)
|
updated_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now, onupdate=_now)
|
||||||
@@ -91,6 +96,32 @@ class AssistantKnowledgeWikiRevisionRow(Base):
|
|||||||
published_at: Mapped[datetime | None] = mapped_column(BeijingDateTime(), nullable=True)
|
published_at: Mapped[datetime | None] = mapped_column(BeijingDateTime(), nullable=True)
|
||||||
|
|
||||||
|
|
||||||
|
class AssistantKnowledgeVectorRow(Base):
|
||||||
|
__tablename__ = "assistant_knowledge_vectors"
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_assistant_knowledge_vector_base_fingerprint", "base_id", "embedding_fingerprint"),
|
||||||
|
Index("ix_assistant_knowledge_vector_page", "page_id"),
|
||||||
|
Index("ix_assistant_knowledge_vector_source", "source_id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(String(36), primary_key=True)
|
||||||
|
base_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||||
|
page_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||||
|
source_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||||
|
import_job_id: Mapped[str] = mapped_column(String(36), index=True)
|
||||||
|
source_vector_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||||
|
source_chunk_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||||
|
section_index: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
heading: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||||
|
section_content: Mapped[str] = mapped_column(PortableLongText(), nullable=False, default="")
|
||||||
|
content_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||||
|
embedding_model: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||||
|
embedding_fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
||||||
|
embedding_dimensions: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
vector_blob: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), default=_now)
|
||||||
|
|
||||||
|
|
||||||
class AssistantKnowledgeEntityRow(Base):
|
class AssistantKnowledgeEntityRow(Base):
|
||||||
__tablename__ = "assistant_knowledge_entities"
|
__tablename__ = "assistant_knowledge_entities"
|
||||||
__table_args__ = (UniqueConstraint("base_id", "entity_type", "normalized_name", name="uq_assistant_entity"),)
|
__table_args__ = (UniqueConstraint("base_id", "entity_type", "normalized_name", name="uq_assistant_entity"),)
|
||||||
|
|||||||
+807
-30
@@ -1,15 +1,21 @@
|
|||||||
"""Repository for the single global assistant knowledge base."""
|
"""Repository for DeerFlow-native assistant knowledge bases."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import binascii
|
||||||
import difflib
|
import difflib
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
from sqlalchemy import delete, func, or_, select
|
from sqlalchemy import delete, func, or_, select
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||||
|
|
||||||
|
from deerflow.integrations.weknora.local_index.vector_codec import decode_vector, encode_vector, normalize_vector
|
||||||
from deerflow.persistence.assistant_knowledge.model import (
|
from deerflow.persistence.assistant_knowledge.model import (
|
||||||
AssistantKnowledgeBaseRow,
|
AssistantKnowledgeBaseRow,
|
||||||
AssistantKnowledgeEntityContributionRow,
|
AssistantKnowledgeEntityContributionRow,
|
||||||
@@ -18,11 +24,24 @@ from deerflow.persistence.assistant_knowledge.model import (
|
|||||||
AssistantKnowledgeRelationContributionRow,
|
AssistantKnowledgeRelationContributionRow,
|
||||||
AssistantKnowledgeRelationRow,
|
AssistantKnowledgeRelationRow,
|
||||||
AssistantKnowledgeSourceRow,
|
AssistantKnowledgeSourceRow,
|
||||||
|
AssistantKnowledgeVectorRow,
|
||||||
AssistantKnowledgeWikiPageRow,
|
AssistantKnowledgeWikiPageRow,
|
||||||
AssistantKnowledgeWikiRevisionRow,
|
AssistantKnowledgeWikiRevisionRow,
|
||||||
)
|
)
|
||||||
from deerflow.skill_knowledge.merge import normalize_name, normalize_predicate
|
from deerflow.skill_knowledge.merge import normalize_name, normalize_predicate
|
||||||
|
|
||||||
|
GLOBAL_BASE_NAME = "知识梳理总库"
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
VECTOR_FIELD_NAMES = (
|
||||||
|
"embedding",
|
||||||
|
"vector",
|
||||||
|
"embedding_vector",
|
||||||
|
"content_vector",
|
||||||
|
"dense_vector",
|
||||||
|
"dense",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _now() -> datetime:
|
def _now() -> datetime:
|
||||||
return datetime.now(UTC)
|
return datetime.now(UTC)
|
||||||
@@ -35,31 +54,246 @@ def _dict(row: Any) -> dict[str, Any]:
|
|||||||
data[key] = value.isoformat()
|
data[key] = value.isoformat()
|
||||||
elif key.endswith("_json"):
|
elif key.endswith("_json"):
|
||||||
data[key[:-5]] = data.pop(key) or ([] if "aliases" in key or "evidence" in key else {})
|
data[key[:-5]] = data.pop(key) or ([] if "aliases" in key or "evidence" in key else {})
|
||||||
|
if isinstance(row, AssistantKnowledgeBaseRow):
|
||||||
|
raw_status = str(data.get("status") or "")
|
||||||
|
data["status"] = "active" if raw_status.startswith("active") else raw_status
|
||||||
|
data["base_kind"] = data.get("base_kind") or "custom"
|
||||||
|
data["is_global"] = data["base_kind"] == "global"
|
||||||
|
data["description"] = data.get("description") or ""
|
||||||
return data
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def _stable_slug_part(value: Any) -> str:
|
||||||
|
raw = str(value or "").strip()
|
||||||
|
if not raw:
|
||||||
|
raw = str(uuid4())
|
||||||
|
return hashlib.sha256(raw.encode("utf-8")).hexdigest()[:24]
|
||||||
|
|
||||||
|
|
||||||
|
def _content_hash(value: str) -> str:
|
||||||
|
return hashlib.sha256(value.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _first_vector_value(raw: dict[str, Any]) -> Any:
|
||||||
|
for key in VECTOR_FIELD_NAMES:
|
||||||
|
value = raw.get(key)
|
||||||
|
if value is not None:
|
||||||
|
return value
|
||||||
|
vectors = raw.get("vectors")
|
||||||
|
if isinstance(vectors, dict):
|
||||||
|
for key in VECTOR_FIELD_NAMES:
|
||||||
|
value = vectors.get(key)
|
||||||
|
if value is not None:
|
||||||
|
return value
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _vector_blob_from_payload(raw: dict[str, Any]) -> tuple[bytes, int] | None:
|
||||||
|
blob64 = raw.get("vector_blob_base64") or raw.get("embedding_blob_base64")
|
||||||
|
dimensions = raw.get("embedding_dimensions") or raw.get("dimensions")
|
||||||
|
if blob64 and dimensions:
|
||||||
|
try:
|
||||||
|
decoded = base64.b64decode(str(blob64), validate=True)
|
||||||
|
vector = decode_vector(decoded, int(dimensions))
|
||||||
|
return encode_vector(vector)
|
||||||
|
except (ValueError, TypeError, binascii.Error):
|
||||||
|
return None
|
||||||
|
value = _first_vector_value(raw)
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
if isinstance(value, dict):
|
||||||
|
value = _first_vector_value(value)
|
||||||
|
if not isinstance(value, (list, tuple)):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return encode_vector(value, dimensions=int(dimensions) if dimensions else None)
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
class AssistantKnowledgeRepository:
|
class AssistantKnowledgeRepository:
|
||||||
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||||
self._sf = session_factory
|
self._sf = session_factory
|
||||||
|
|
||||||
async def get_base(self) -> dict[str, Any] | None:
|
async def _base_with_stats(
|
||||||
stmt = select(AssistantKnowledgeBaseRow).where(AssistantKnowledgeBaseRow.status == "active").limit(1)
|
self,
|
||||||
async with self._sf() as session:
|
session: AsyncSession,
|
||||||
row = (await session.execute(stmt)).scalar_one_or_none()
|
row: AssistantKnowledgeBaseRow,
|
||||||
return _dict(row) if row else None
|
) -> dict[str, Any]:
|
||||||
|
data = _dict(row)
|
||||||
|
data["source_count"] = int(
|
||||||
|
(
|
||||||
|
await session.execute(
|
||||||
|
select(func.count(AssistantKnowledgeSourceRow.id)).where(
|
||||||
|
AssistantKnowledgeSourceRow.base_id == row.id,
|
||||||
|
AssistantKnowledgeSourceRow.status != "deleted",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
data["page_count"] = int(
|
||||||
|
(
|
||||||
|
await session.execute(
|
||||||
|
select(func.count(AssistantKnowledgeWikiPageRow.id)).where(
|
||||||
|
AssistantKnowledgeWikiPageRow.base_id == row.id,
|
||||||
|
AssistantKnowledgeWikiPageRow.status == "active",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
data["entity_count"] = int(
|
||||||
|
(
|
||||||
|
await session.execute(
|
||||||
|
select(func.count(AssistantKnowledgeEntityRow.id)).where(
|
||||||
|
AssistantKnowledgeEntityRow.base_id == row.id,
|
||||||
|
AssistantKnowledgeEntityRow.status == "active",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
data["relation_count"] = int(
|
||||||
|
(
|
||||||
|
await session.execute(
|
||||||
|
select(func.count(AssistantKnowledgeRelationRow.id)).where(
|
||||||
|
AssistantKnowledgeRelationRow.base_id == row.id,
|
||||||
|
AssistantKnowledgeRelationRow.status == "active",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
return data
|
||||||
|
|
||||||
async def initialize(self, *, created_by: str | None, name: str = "知识梳理") -> dict[str, Any]:
|
async def get_base(self, base_id: str | None = None) -> dict[str, Any] | None:
|
||||||
|
async with self._sf() as session:
|
||||||
|
if base_id:
|
||||||
|
row = (
|
||||||
|
await session.execute(
|
||||||
|
select(AssistantKnowledgeBaseRow).where(
|
||||||
|
AssistantKnowledgeBaseRow.id == base_id,
|
||||||
|
AssistantKnowledgeBaseRow.status != "deleted",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
return await self._base_with_stats(session, row) if row else None
|
||||||
|
|
||||||
|
row = (
|
||||||
|
await session.execute(
|
||||||
|
select(AssistantKnowledgeBaseRow)
|
||||||
|
.where(
|
||||||
|
AssistantKnowledgeBaseRow.status != "deleted",
|
||||||
|
AssistantKnowledgeBaseRow.base_kind == "global",
|
||||||
|
)
|
||||||
|
.order_by(AssistantKnowledgeBaseRow.created_at.asc())
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is None:
|
||||||
|
# Backward compatibility: earlier builds had a single active row
|
||||||
|
# and no ``base_kind``. Promote the first surviving base into the
|
||||||
|
# global full-library aggregation base.
|
||||||
|
row = (
|
||||||
|
await session.execute(
|
||||||
|
select(AssistantKnowledgeBaseRow)
|
||||||
|
.where(AssistantKnowledgeBaseRow.status != "deleted")
|
||||||
|
.order_by(AssistantKnowledgeBaseRow.created_at.asc())
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is not None:
|
||||||
|
row.base_kind = "global"
|
||||||
|
if not row.description:
|
||||||
|
row.description = "默认全库沉淀库,汇聚所有 WeKnora 导出的助手知识数据。"
|
||||||
|
row.updated_at = _now()
|
||||||
|
await session.flush()
|
||||||
|
await session.commit()
|
||||||
|
return await self._base_with_stats(session, row) if row else None
|
||||||
|
|
||||||
|
async def list_bases(self) -> list[dict[str, Any]]:
|
||||||
|
await self.get_base()
|
||||||
|
async with self._sf() as session:
|
||||||
|
rows = (
|
||||||
|
await session.execute(
|
||||||
|
select(AssistantKnowledgeBaseRow)
|
||||||
|
.where(AssistantKnowledgeBaseRow.status != "deleted")
|
||||||
|
.order_by(
|
||||||
|
AssistantKnowledgeBaseRow.base_kind.desc(),
|
||||||
|
AssistantKnowledgeBaseRow.updated_at.desc(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
values = [await self._base_with_stats(session, row) for row in rows]
|
||||||
|
global_values = [item for item in values if item.get("is_global")]
|
||||||
|
custom_values = sorted(
|
||||||
|
[item for item in values if not item.get("is_global")],
|
||||||
|
key=lambda item: item.get("updated_at") or "",
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
return global_values + custom_values
|
||||||
|
|
||||||
|
async def initialize(self, *, created_by: str | None, name: str = GLOBAL_BASE_NAME) -> dict[str, Any]:
|
||||||
existing = await self.get_base()
|
existing = await self.get_base()
|
||||||
if existing:
|
if existing:
|
||||||
return existing
|
return existing
|
||||||
row = AssistantKnowledgeBaseRow(id=str(uuid4()), name=name.strip() or "知识梳理", status="active", created_by=created_by, created_at=_now(), updated_at=_now())
|
row = AssistantKnowledgeBaseRow(
|
||||||
|
id=str(uuid4()),
|
||||||
|
name=name.strip() or GLOBAL_BASE_NAME,
|
||||||
|
description="默认全库沉淀库,汇聚所有 WeKnora 导出的助手知识数据。",
|
||||||
|
base_kind="global",
|
||||||
|
status="active",
|
||||||
|
created_by=created_by,
|
||||||
|
created_at=_now(),
|
||||||
|
updated_at=_now(),
|
||||||
|
)
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
session.add(row)
|
session.add(row)
|
||||||
await session.flush()
|
await session.flush()
|
||||||
data = _dict(row)
|
data = await self._base_with_stats(session, row)
|
||||||
await session.commit()
|
await session.commit()
|
||||||
return data
|
return data
|
||||||
|
|
||||||
|
async def create_base(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
created_by: str | None,
|
||||||
|
name: str,
|
||||||
|
description: str = "",
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
row_id = str(uuid4())
|
||||||
|
row = AssistantKnowledgeBaseRow(
|
||||||
|
id=row_id,
|
||||||
|
name=name.strip(),
|
||||||
|
description=description.strip(),
|
||||||
|
base_kind="custom",
|
||||||
|
# Some upgraded SQLite/MySQL databases may still carry the old
|
||||||
|
# unique(status) constraint from the single-base implementation and
|
||||||
|
# its original VARCHAR(24) length. Use a compact per-row active
|
||||||
|
# marker internally and normalize it in API responses.
|
||||||
|
status=f"active:{row_id[:17]}",
|
||||||
|
created_by=created_by,
|
||||||
|
created_at=_now(),
|
||||||
|
updated_at=_now(),
|
||||||
|
)
|
||||||
|
async with self._sf() as session:
|
||||||
|
session.add(row)
|
||||||
|
await session.flush()
|
||||||
|
data = await self._base_with_stats(session, row)
|
||||||
|
await session.commit()
|
||||||
|
return data
|
||||||
|
|
||||||
|
async def _ensure_base(self, base_id: str | None, *, created_by: str | None) -> dict[str, Any]:
|
||||||
|
if base_id == "global":
|
||||||
|
return await self.initialize(created_by=created_by)
|
||||||
|
if base_id:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
raise ValueError("助手知识库不存在")
|
||||||
|
return base
|
||||||
|
return await self.initialize(created_by=created_by)
|
||||||
|
|
||||||
async def _upsert_source(
|
async def _upsert_source(
|
||||||
self,
|
self,
|
||||||
session: AsyncSession,
|
session: AsyncSession,
|
||||||
@@ -181,8 +415,56 @@ class AssistantKnowledgeRepository:
|
|||||||
skill_name: str | None = None,
|
skill_name: str | None = None,
|
||||||
mapping_id: str | None = None,
|
mapping_id: str | None = None,
|
||||||
upload_batch_id: str | None = None,
|
upload_batch_id: str | None = None,
|
||||||
|
base_id: str | None = None,
|
||||||
|
mirror_to_global: bool = True,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
base = await self.initialize(created_by=created_by)
|
base = await self._ensure_base(base_id, created_by=created_by)
|
||||||
|
result = await self._import_package_into_base(
|
||||||
|
base=base,
|
||||||
|
package=package,
|
||||||
|
source_type=source_type,
|
||||||
|
source_key=source_key,
|
||||||
|
source_name=source_name,
|
||||||
|
trigger=trigger,
|
||||||
|
created_by=created_by,
|
||||||
|
skill_name=skill_name,
|
||||||
|
mapping_id=mapping_id,
|
||||||
|
upload_batch_id=upload_batch_id,
|
||||||
|
)
|
||||||
|
if mirror_to_global and not base.get("is_global"):
|
||||||
|
global_base = await self.initialize(created_by=created_by)
|
||||||
|
mirror = await self._import_package_into_base(
|
||||||
|
base=global_base,
|
||||||
|
package=package,
|
||||||
|
source_type=source_type,
|
||||||
|
source_key=source_key,
|
||||||
|
source_name=source_name,
|
||||||
|
trigger=f"{trigger}:global_mirror"[:48],
|
||||||
|
created_by=created_by,
|
||||||
|
skill_name=skill_name,
|
||||||
|
mapping_id=mapping_id,
|
||||||
|
upload_batch_id=upload_batch_id,
|
||||||
|
)
|
||||||
|
result["mirrored_global_base_id"] = global_base["id"]
|
||||||
|
result["mirrored_global_job_id"] = mirror["id"]
|
||||||
|
return result
|
||||||
|
|
||||||
|
async def create_queued_import_job(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
source_type: str,
|
||||||
|
source_key: str,
|
||||||
|
source_name: str,
|
||||||
|
trigger: str,
|
||||||
|
created_by: str | None,
|
||||||
|
metadata: dict[str, Any] | None = None,
|
||||||
|
skill_name: str | None = None,
|
||||||
|
mapping_id: str | None = None,
|
||||||
|
upload_batch_id: str | None = None,
|
||||||
|
base_id: str | None = None,
|
||||||
|
phase: str = "queued",
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
base = await self._ensure_base(base_id, created_by=created_by)
|
||||||
now = _now()
|
now = _now()
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
source = await self._upsert_source(
|
source = await self._upsert_source(
|
||||||
@@ -191,22 +473,18 @@ class AssistantKnowledgeRepository:
|
|||||||
source_type=source_type,
|
source_type=source_type,
|
||||||
source_key=source_key,
|
source_key=source_key,
|
||||||
source_name=source_name,
|
source_name=source_name,
|
||||||
metadata=package.get("source") or {},
|
metadata=metadata or {},
|
||||||
skill_name=skill_name,
|
skill_name=skill_name,
|
||||||
mapping_id=mapping_id,
|
mapping_id=mapping_id,
|
||||||
upload_batch_id=upload_batch_id,
|
upload_batch_id=upload_batch_id,
|
||||||
)
|
)
|
||||||
# A re-import replaces this source's prior contributions while retaining
|
|
||||||
# canonical entities/relations that are still supported by other sources.
|
|
||||||
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source.id))
|
|
||||||
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source.id))
|
|
||||||
job = AssistantKnowledgeImportJobRow(
|
job = AssistantKnowledgeImportJobRow(
|
||||||
id=str(uuid4()),
|
id=str(uuid4()),
|
||||||
base_id=base["id"],
|
base_id=base["id"],
|
||||||
source_id=source.id,
|
source_id=source.id,
|
||||||
trigger=trigger,
|
trigger=trigger,
|
||||||
status="running",
|
status="queued",
|
||||||
phase="aligning_entities",
|
phase=phase,
|
||||||
counts_json={},
|
counts_json={},
|
||||||
created_by=created_by,
|
created_by=created_by,
|
||||||
created_at=now,
|
created_at=now,
|
||||||
@@ -214,6 +492,166 @@ class AssistantKnowledgeRepository:
|
|||||||
session.add(job)
|
session.add(job)
|
||||||
await session.flush()
|
await session.flush()
|
||||||
source.last_import_job_id = job.id
|
source.last_import_job_id = job.id
|
||||||
|
result = _dict(job)
|
||||||
|
await session.commit()
|
||||||
|
return result
|
||||||
|
|
||||||
|
async def update_import_job(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
status: str | None = None,
|
||||||
|
phase: str | None = None,
|
||||||
|
counts: dict[str, Any] | None = None,
|
||||||
|
error: str | None = None,
|
||||||
|
completed: bool = False,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
async with self._sf() as session:
|
||||||
|
job = await session.get(AssistantKnowledgeImportJobRow, job_id)
|
||||||
|
if job is None:
|
||||||
|
return None
|
||||||
|
if status is not None:
|
||||||
|
job.status = status
|
||||||
|
if phase is not None:
|
||||||
|
job.phase = phase
|
||||||
|
if counts is not None:
|
||||||
|
job.counts_json = counts
|
||||||
|
if error is not None:
|
||||||
|
job.last_error_message = error[:2048]
|
||||||
|
elif status not in {None, "failed"}:
|
||||||
|
job.last_error_message = None
|
||||||
|
if completed:
|
||||||
|
job.completed_at = _now()
|
||||||
|
await session.flush()
|
||||||
|
result = _dict(job)
|
||||||
|
await session.commit()
|
||||||
|
return result
|
||||||
|
|
||||||
|
async def import_package_for_job(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
package: dict[str, Any],
|
||||||
|
created_by: str | None,
|
||||||
|
mirror_to_global: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
async with self._sf() as session:
|
||||||
|
job = await session.get(AssistantKnowledgeImportJobRow, job_id)
|
||||||
|
if job is None:
|
||||||
|
raise ValueError("导入任务不存在")
|
||||||
|
source = await session.get(AssistantKnowledgeSourceRow, job.source_id)
|
||||||
|
base_row = await session.get(AssistantKnowledgeBaseRow, job.base_id)
|
||||||
|
if source is None or base_row is None:
|
||||||
|
raise ValueError("导入任务来源不存在")
|
||||||
|
base = _dict(base_row)
|
||||||
|
source_type = source.source_type
|
||||||
|
source_key = source.source_key
|
||||||
|
source_name = source.source_name
|
||||||
|
source_id = source.id
|
||||||
|
trigger = job.trigger
|
||||||
|
skill_name = source.skill_name
|
||||||
|
mapping_id = source.knowledge_base_mapping_id
|
||||||
|
upload_batch_id = source.upload_batch_id
|
||||||
|
|
||||||
|
result = await self._import_package_into_base(
|
||||||
|
base=base,
|
||||||
|
package=package,
|
||||||
|
source_type=source_type,
|
||||||
|
source_key=source_key,
|
||||||
|
source_name=source_name,
|
||||||
|
trigger=trigger,
|
||||||
|
created_by=created_by,
|
||||||
|
skill_name=skill_name,
|
||||||
|
mapping_id=mapping_id,
|
||||||
|
upload_batch_id=upload_batch_id,
|
||||||
|
existing_source_id=source_id,
|
||||||
|
existing_job_id=job_id,
|
||||||
|
)
|
||||||
|
if mirror_to_global and not base.get("is_global"):
|
||||||
|
global_base = await self.initialize(created_by=created_by)
|
||||||
|
mirror = await self._import_package_into_base(
|
||||||
|
base=global_base,
|
||||||
|
package=package,
|
||||||
|
source_type=source_type,
|
||||||
|
source_key=source_key,
|
||||||
|
source_name=source_name,
|
||||||
|
trigger=f"{trigger}:global_mirror"[:48],
|
||||||
|
created_by=created_by,
|
||||||
|
skill_name=skill_name,
|
||||||
|
mapping_id=mapping_id,
|
||||||
|
upload_batch_id=upload_batch_id,
|
||||||
|
)
|
||||||
|
result["mirrored_global_base_id"] = global_base["id"]
|
||||||
|
result["mirrored_global_job_id"] = mirror["id"]
|
||||||
|
return result
|
||||||
|
|
||||||
|
async def _import_package_into_base(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
base: dict[str, Any],
|
||||||
|
package: dict[str, Any],
|
||||||
|
source_type: str,
|
||||||
|
source_key: str,
|
||||||
|
source_name: str,
|
||||||
|
trigger: str,
|
||||||
|
created_by: str | None,
|
||||||
|
skill_name: str | None = None,
|
||||||
|
mapping_id: str | None = None,
|
||||||
|
upload_batch_id: str | None = None,
|
||||||
|
existing_source_id: str | None = None,
|
||||||
|
existing_job_id: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
now = _now()
|
||||||
|
async with self._sf() as session:
|
||||||
|
if existing_source_id:
|
||||||
|
source = await session.get(AssistantKnowledgeSourceRow, existing_source_id)
|
||||||
|
if source is None or source.base_id != base["id"]:
|
||||||
|
raise ValueError("导入任务来源不存在")
|
||||||
|
source.source_name = source_name
|
||||||
|
source.status = "active"
|
||||||
|
source.metadata_json = package.get("source") or {}
|
||||||
|
source.updated_at = now
|
||||||
|
else:
|
||||||
|
source = await self._upsert_source(
|
||||||
|
session,
|
||||||
|
base_id=base["id"],
|
||||||
|
source_type=source_type,
|
||||||
|
source_key=source_key,
|
||||||
|
source_name=source_name,
|
||||||
|
metadata=package.get("source") or {},
|
||||||
|
skill_name=skill_name,
|
||||||
|
mapping_id=mapping_id,
|
||||||
|
upload_batch_id=upload_batch_id,
|
||||||
|
)
|
||||||
|
# A re-import replaces this source's prior contributions while retaining
|
||||||
|
# canonical entities/relations that are still supported by other sources.
|
||||||
|
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source.id))
|
||||||
|
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source.id))
|
||||||
|
await session.execute(delete(AssistantKnowledgeVectorRow).where(AssistantKnowledgeVectorRow.source_id == source.id))
|
||||||
|
if existing_job_id:
|
||||||
|
job = await session.get(AssistantKnowledgeImportJobRow, existing_job_id)
|
||||||
|
if job is None or job.base_id != base["id"]:
|
||||||
|
raise ValueError("导入任务不存在")
|
||||||
|
job.source_id = source.id
|
||||||
|
job.status = "running"
|
||||||
|
job.phase = "aligning_entities"
|
||||||
|
job.last_error_message = None
|
||||||
|
job.completed_at = None
|
||||||
|
else:
|
||||||
|
job = AssistantKnowledgeImportJobRow(
|
||||||
|
id=str(uuid4()),
|
||||||
|
base_id=base["id"],
|
||||||
|
source_id=source.id,
|
||||||
|
trigger=trigger,
|
||||||
|
status="running",
|
||||||
|
phase="aligning_entities",
|
||||||
|
counts_json={},
|
||||||
|
created_by=created_by,
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
session.add(job)
|
||||||
|
await session.flush()
|
||||||
|
source.last_import_job_id = job.id
|
||||||
|
|
||||||
entity_map: dict[str, str] = {}
|
entity_map: dict[str, str] = {}
|
||||||
entity_rows: dict[str, tuple[AssistantKnowledgeEntityRow, dict[str, Any]]] = {}
|
entity_rows: dict[str, tuple[AssistantKnowledgeEntityRow, dict[str, Any]]] = {}
|
||||||
@@ -363,6 +801,7 @@ class AssistantKnowledgeRepository:
|
|||||||
wiki = package["wiki"]
|
wiki = package["wiki"]
|
||||||
page = wiki.get("skill_page") if isinstance(wiki, dict) else None
|
page = wiki.get("skill_page") if isinstance(wiki, dict) else None
|
||||||
pages = [page] if page else []
|
pages = [page] if page else []
|
||||||
|
page_ids_by_slug: dict[str, str] = {}
|
||||||
for raw in pages:
|
for raw in pages:
|
||||||
if not isinstance(raw, dict):
|
if not isinstance(raw, dict):
|
||||||
continue
|
continue
|
||||||
@@ -404,16 +843,139 @@ class AssistantKnowledgeRepository:
|
|||||||
)
|
)
|
||||||
session.add(revision)
|
session.add(revision)
|
||||||
page.title = title
|
page.title = title
|
||||||
|
page.page_type = str(raw.get("page_type") or "source")
|
||||||
page.content_markdown = content
|
page.content_markdown = content
|
||||||
page.summary = summary
|
page.summary = summary
|
||||||
page.search_text = "\n".join([title, summary, content, " ".join(raw.get("aliases") or [])])
|
page.search_text = "\n".join([title, summary, content, " ".join(raw.get("aliases") or [])])
|
||||||
page.current_revision_id = revision.id
|
page.current_revision_id = revision.id
|
||||||
|
page.status = "active"
|
||||||
page.updated_at = now
|
page.updated_at = now
|
||||||
|
page_ids_by_slug[slug] = page.id
|
||||||
|
|
||||||
|
chunks = list(package.get("chunks") or [])
|
||||||
|
chunk_pages = 0
|
||||||
|
imported_vectors = 0
|
||||||
|
for index, raw in enumerate(chunks):
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
continue
|
||||||
|
content = str(
|
||||||
|
raw.get("content")
|
||||||
|
or raw.get("section_content")
|
||||||
|
or raw.get("text")
|
||||||
|
or raw.get("chunk_content")
|
||||||
|
or ""
|
||||||
|
).strip()
|
||||||
|
if not content:
|
||||||
|
continue
|
||||||
|
raw_chunk_id = str(raw.get("id") or raw.get("chunk_id") or raw.get("remote_id") or f"{source.id}:{index}")
|
||||||
|
document_id = str(raw.get("source_document_id") or raw.get("knowledge_id") or raw.get("document_id") or "")
|
||||||
|
title = str(
|
||||||
|
raw.get("title")
|
||||||
|
or raw.get("knowledge_title")
|
||||||
|
or raw.get("filename")
|
||||||
|
or raw.get("knowledge_filename")
|
||||||
|
or f"{source_name} 片段 {index + 1}"
|
||||||
|
)
|
||||||
|
heading = str(raw.get("heading") or raw.get("section_title") or "")
|
||||||
|
summary = content[:240]
|
||||||
|
page_content = "\n".join(
|
||||||
|
item
|
||||||
|
for item in [
|
||||||
|
f"# {title}",
|
||||||
|
"",
|
||||||
|
f"- 来源:{source_name}",
|
||||||
|
f"- 文档ID:{document_id}" if document_id else "",
|
||||||
|
f"- 片段ID:{raw_chunk_id}" if raw_chunk_id else "",
|
||||||
|
f"- 标题层级:{heading}" if heading else "",
|
||||||
|
"",
|
||||||
|
content,
|
||||||
|
]
|
||||||
|
if item != ""
|
||||||
|
)
|
||||||
|
chunk_page = await self._publish_page(
|
||||||
|
session,
|
||||||
|
base_id=base["id"],
|
||||||
|
slug=f"chunks/{source.id}/{_stable_slug_part(raw_chunk_id)}",
|
||||||
|
title=title,
|
||||||
|
page_type="chunk",
|
||||||
|
content=page_content,
|
||||||
|
summary=summary,
|
||||||
|
search_terms=[raw_chunk_id, document_id, heading],
|
||||||
|
source_id=source.id,
|
||||||
|
import_job_id=job.id,
|
||||||
|
created_by=created_by,
|
||||||
|
change_type="chunk_import",
|
||||||
|
)
|
||||||
|
page_ids_by_slug[chunk_page.slug] = chunk_page.id
|
||||||
|
vector_blob = _vector_blob_from_payload(raw)
|
||||||
|
if vector_blob is not None:
|
||||||
|
blob, dimensions = vector_blob
|
||||||
|
session.add(
|
||||||
|
AssistantKnowledgeVectorRow(
|
||||||
|
id=str(uuid4()),
|
||||||
|
base_id=base["id"],
|
||||||
|
page_id=chunk_page.id,
|
||||||
|
source_id=source.id,
|
||||||
|
import_job_id=job.id,
|
||||||
|
source_vector_id=str(raw.get("vector_id") or raw.get("embedding_id") or raw_chunk_id),
|
||||||
|
source_chunk_id=raw_chunk_id,
|
||||||
|
section_index=int(raw.get("section_index") or raw.get("chunk_index") or index),
|
||||||
|
heading=heading or None,
|
||||||
|
section_content=content,
|
||||||
|
content_hash=str(raw.get("content_hash") or _content_hash(content)),
|
||||||
|
embedding_model=str(raw.get("embedding_model") or raw.get("model") or ""),
|
||||||
|
embedding_fingerprint=str(raw.get("embedding_fingerprint") or ""),
|
||||||
|
embedding_dimensions=dimensions,
|
||||||
|
vector_blob=blob,
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
imported_vectors += 1
|
||||||
|
chunk_pages += 1
|
||||||
|
|
||||||
|
for index, raw in enumerate(list(package.get("vectors") or [])):
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
continue
|
||||||
|
slug = str(raw.get("wiki_slug") or raw.get("page_slug") or raw.get("slug") or "").strip("/")
|
||||||
|
page_id = page_ids_by_slug.get(slug)
|
||||||
|
if not page_id:
|
||||||
|
continue
|
||||||
|
content = str(raw.get("section_content") or raw.get("content") or raw.get("text") or "").strip()
|
||||||
|
if not content:
|
||||||
|
continue
|
||||||
|
vector_blob = _vector_blob_from_payload(raw)
|
||||||
|
if vector_blob is None:
|
||||||
|
continue
|
||||||
|
blob, dimensions = vector_blob
|
||||||
|
raw_chunk_id = str(raw.get("source_chunk_id") or raw.get("chunk_id") or "")
|
||||||
|
session.add(
|
||||||
|
AssistantKnowledgeVectorRow(
|
||||||
|
id=str(uuid4()),
|
||||||
|
base_id=base["id"],
|
||||||
|
page_id=page_id,
|
||||||
|
source_id=source.id,
|
||||||
|
import_job_id=job.id,
|
||||||
|
source_vector_id=str(raw.get("id") or raw.get("vector_id") or f"{slug}:{index}"),
|
||||||
|
source_chunk_id=raw_chunk_id or None,
|
||||||
|
section_index=int(raw.get("section_index") or index),
|
||||||
|
heading=str(raw.get("heading") or "") or None,
|
||||||
|
section_content=content,
|
||||||
|
content_hash=str(raw.get("content_hash") or _content_hash(content)),
|
||||||
|
embedding_model=str(raw.get("embedding_model") or raw.get("model") or ""),
|
||||||
|
embedding_fingerprint=str(raw.get("embedding_fingerprint") or ""),
|
||||||
|
embedding_dimensions=dimensions,
|
||||||
|
vector_blob=blob,
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
imported_vectors += 1
|
||||||
|
|
||||||
counts = {
|
counts = {
|
||||||
"wiki_pages": len(pages) + len(entity_rows) + len(relation_rows),
|
"wiki_pages": len(pages) + len(entity_rows) + len(relation_rows),
|
||||||
"documents": len(package.get("documents") or []),
|
"documents": len(package.get("documents") or []),
|
||||||
"chunks": len(package.get("chunks") or []),
|
"chunks": len(chunks),
|
||||||
|
"chunk_pages": chunk_pages,
|
||||||
|
"vectors": imported_vectors,
|
||||||
"entities": len(raw_entities),
|
"entities": len(raw_entities),
|
||||||
"relations": len(raw_relations),
|
"relations": len(raw_relations),
|
||||||
"graph_nodes": len((package.get("graph") or {}).get("nodes") or []),
|
"graph_nodes": len((package.get("graph") or {}).get("nodes") or []),
|
||||||
@@ -439,8 +1001,24 @@ class AssistantKnowledgeRepository:
|
|||||||
contribution = (await session.execute(select(AssistantKnowledgeRelationContributionRow.id).where(AssistantKnowledgeRelationContributionRow.relation_id == relation.id).limit(1))).scalar_one_or_none()
|
contribution = (await session.execute(select(AssistantKnowledgeRelationContributionRow.id).where(AssistantKnowledgeRelationContributionRow.relation_id == relation.id).limit(1))).scalar_one_or_none()
|
||||||
relation.status = "active" if contribution else "deleted"
|
relation.status = "active" if contribution else "deleted"
|
||||||
|
|
||||||
async def list_pages(self, *, query: str | None = None, page_type: str | None = None, limit: int = 100) -> list[dict[str, Any]]:
|
async def list_pages(
|
||||||
stmt = select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.status == "active")
|
self,
|
||||||
|
*,
|
||||||
|
base_id: str | None = None,
|
||||||
|
query: str | None = None,
|
||||||
|
page_type: str | None = None,
|
||||||
|
limit: int = 100,
|
||||||
|
include_chunks: bool = False,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
return []
|
||||||
|
stmt = select(AssistantKnowledgeWikiPageRow).where(
|
||||||
|
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||||
|
AssistantKnowledgeWikiPageRow.status == "active",
|
||||||
|
)
|
||||||
|
if not include_chunks and not page_type:
|
||||||
|
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
|
||||||
if query:
|
if query:
|
||||||
needle = f"%{query.strip()}%"
|
needle = f"%{query.strip()}%"
|
||||||
stmt = stmt.where(or_(AssistantKnowledgeWikiPageRow.title.ilike(needle), AssistantKnowledgeWikiPageRow.search_text.ilike(needle)))
|
stmt = stmt.where(or_(AssistantKnowledgeWikiPageRow.title.ilike(needle), AssistantKnowledgeWikiPageRow.search_text.ilike(needle)))
|
||||||
@@ -450,15 +1028,178 @@ class AssistantKnowledgeRepository:
|
|||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||||
|
|
||||||
async def get_page(self, slug: str) -> dict[str, Any] | None:
|
async def search_pages(
|
||||||
stmt = select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/"), AssistantKnowledgeWikiPageRow.status == "active")
|
self,
|
||||||
|
*,
|
||||||
|
base_ids: list[str],
|
||||||
|
query: str,
|
||||||
|
limit: int = 8,
|
||||||
|
include_chunks: bool = True,
|
||||||
|
embedding_client: Any | None = None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
resolved_base_ids: list[str] = []
|
||||||
|
for base_id in list(dict.fromkeys(base_ids)):
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is not None:
|
||||||
|
resolved_base_ids.append(str(base["id"]))
|
||||||
|
if not resolved_base_ids:
|
||||||
|
return []
|
||||||
|
if embedding_client is not None and query.strip():
|
||||||
|
try:
|
||||||
|
vector_results = await self._search_pages_by_vector(
|
||||||
|
base_ids=resolved_base_ids,
|
||||||
|
query=query,
|
||||||
|
limit=limit,
|
||||||
|
include_chunks=include_chunks,
|
||||||
|
embedding_client=embedding_client,
|
||||||
|
)
|
||||||
|
if vector_results:
|
||||||
|
return vector_results
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
logger.warning("Assistant knowledge vector search failed; falling back to keyword search", exc_info=True)
|
||||||
|
tokens = [token for token in query.strip().lower().split() if token]
|
||||||
|
needle = f"%{query.strip()}%" if query.strip() else "%"
|
||||||
|
stmt = select(AssistantKnowledgeWikiPageRow).where(
|
||||||
|
AssistantKnowledgeWikiPageRow.base_id.in_(resolved_base_ids),
|
||||||
|
AssistantKnowledgeWikiPageRow.status == "active",
|
||||||
|
)
|
||||||
|
if not include_chunks:
|
||||||
|
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
|
||||||
|
if query.strip():
|
||||||
|
stmt = stmt.where(
|
||||||
|
or_(
|
||||||
|
AssistantKnowledgeWikiPageRow.title.ilike(needle),
|
||||||
|
AssistantKnowledgeWikiPageRow.search_text.ilike(needle),
|
||||||
|
*[
|
||||||
|
AssistantKnowledgeWikiPageRow.search_text.ilike(f"%{token}%")
|
||||||
|
for token in tokens[:8]
|
||||||
|
],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
stmt = stmt.order_by(AssistantKnowledgeWikiPageRow.updated_at.desc()).limit(max(limit * 8, limit))
|
||||||
|
async with self._sf() as session:
|
||||||
|
rows = [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||||
|
|
||||||
|
def score(row: dict[str, Any]) -> float:
|
||||||
|
text = f"{row.get('title') or ''}\n{row.get('summary') or ''}\n{row.get('search_text') or row.get('content_markdown') or ''}".lower()
|
||||||
|
if not tokens:
|
||||||
|
return 0.0
|
||||||
|
value = 0.0
|
||||||
|
title = str(row.get("title") or "").lower()
|
||||||
|
for token in tokens:
|
||||||
|
if token in title:
|
||||||
|
value += 3.0
|
||||||
|
value += min(text.count(token), 8) * 0.25
|
||||||
|
if row.get("page_type") == "chunk":
|
||||||
|
value += 0.5
|
||||||
|
return value
|
||||||
|
|
||||||
|
rows.sort(key=score, reverse=True)
|
||||||
|
results = rows[:limit]
|
||||||
|
for row in results:
|
||||||
|
row["score"] = score(row)
|
||||||
|
return results
|
||||||
|
|
||||||
|
async def _search_pages_by_vector(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
base_ids: list[str],
|
||||||
|
query: str,
|
||||||
|
limit: int,
|
||||||
|
include_chunks: bool,
|
||||||
|
embedding_client: Any,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
raw_query_vector = await embedding_client.embed_query(query)
|
||||||
|
configured_dimensions = getattr(embedding_client, "dimensions", None)
|
||||||
|
query_vector = normalize_vector(raw_query_vector, dimensions=configured_dimensions)
|
||||||
|
dimensions = int(query_vector.size)
|
||||||
|
stmt = (
|
||||||
|
select(AssistantKnowledgeVectorRow, AssistantKnowledgeWikiPageRow)
|
||||||
|
.join(
|
||||||
|
AssistantKnowledgeWikiPageRow,
|
||||||
|
AssistantKnowledgeWikiPageRow.id == AssistantKnowledgeVectorRow.page_id,
|
||||||
|
)
|
||||||
|
.where(
|
||||||
|
AssistantKnowledgeVectorRow.base_id.in_(base_ids),
|
||||||
|
AssistantKnowledgeVectorRow.embedding_dimensions == dimensions,
|
||||||
|
AssistantKnowledgeWikiPageRow.status == "active",
|
||||||
|
)
|
||||||
|
.order_by(AssistantKnowledgeVectorRow.created_at.desc())
|
||||||
|
)
|
||||||
|
if not include_chunks:
|
||||||
|
stmt = stmt.where(AssistantKnowledgeWikiPageRow.page_type != "chunk")
|
||||||
|
async with self._sf() as session:
|
||||||
|
rows = list((await session.execute(stmt)).all())
|
||||||
|
if not rows:
|
||||||
|
return []
|
||||||
|
tokens = [token for token in query.strip().lower().split() if token]
|
||||||
|
ranked: list[tuple[float, AssistantKnowledgeVectorRow, AssistantKnowledgeWikiPageRow]] = []
|
||||||
|
for vector, page in rows:
|
||||||
|
try:
|
||||||
|
stored = decode_vector(vector.vector_blob, int(vector.embedding_dimensions))
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
score = float(np.dot(query_vector, stored))
|
||||||
|
if tokens:
|
||||||
|
haystack = f"{page.title}\n{page.summary}\n{vector.heading or ''}\n{vector.section_content}".lower()
|
||||||
|
score += min(sum(1 for token in tokens if token in haystack), 4) * 0.02
|
||||||
|
ranked.append((score, vector, page))
|
||||||
|
ranked.sort(key=lambda item: item[0], reverse=True)
|
||||||
|
|
||||||
|
results: list[dict[str, Any]] = []
|
||||||
|
seen_pages: set[str] = set()
|
||||||
|
for score, vector, page in ranked:
|
||||||
|
if len(results) >= limit:
|
||||||
|
break
|
||||||
|
page_data = _dict(page)
|
||||||
|
page_data["score"] = score
|
||||||
|
page_data["content_markdown"] = vector.section_content or page_data.get("content_markdown") or ""
|
||||||
|
page_data["summary"] = page_data.get("summary") or (vector.section_content[:240] if vector.section_content else "")
|
||||||
|
page_data["retrieval_mode"] = "vector"
|
||||||
|
page_data["matched_section"] = {
|
||||||
|
"id": vector.id,
|
||||||
|
"section_index": int(vector.section_index),
|
||||||
|
"heading": vector.heading,
|
||||||
|
"content": vector.section_content,
|
||||||
|
"score": score,
|
||||||
|
"source_chunk_id": vector.source_chunk_id,
|
||||||
|
"embedding_model": vector.embedding_model,
|
||||||
|
"embedding_fingerprint": vector.embedding_fingerprint,
|
||||||
|
}
|
||||||
|
dedupe_key = f"{page.id}:{vector.section_index}:{vector.content_hash}"
|
||||||
|
if page.id in seen_pages and page.page_type != "chunk":
|
||||||
|
continue
|
||||||
|
seen_pages.add(page.id)
|
||||||
|
seen_pages.add(dedupe_key)
|
||||||
|
results.append(page_data)
|
||||||
|
return results
|
||||||
|
|
||||||
|
async def get_page(self, slug: str, *, base_id: str | None = None) -> dict[str, Any] | None:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
return None
|
||||||
|
stmt = select(AssistantKnowledgeWikiPageRow).where(
|
||||||
|
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||||
|
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
|
||||||
|
AssistantKnowledgeWikiPageRow.status == "active",
|
||||||
|
)
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = (await session.execute(stmt)).scalar_one_or_none()
|
row = (await session.execute(stmt)).scalar_one_or_none()
|
||||||
return _dict(row) if row else None
|
return _dict(row) if row else None
|
||||||
|
|
||||||
async def list_revisions(self, slug: str) -> list[dict[str, Any]]:
|
async def list_revisions(self, slug: str, *, base_id: str | None = None) -> list[dict[str, Any]]:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
return []
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
page = (await session.execute(select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/")))).scalar_one_or_none()
|
page = (
|
||||||
|
await session.execute(
|
||||||
|
select(AssistantKnowledgeWikiPageRow).where(
|
||||||
|
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||||
|
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
if page is None:
|
if page is None:
|
||||||
return []
|
return []
|
||||||
stmt = select(AssistantKnowledgeWikiRevisionRow).where(AssistantKnowledgeWikiRevisionRow.page_id == page.id).order_by(AssistantKnowledgeWikiRevisionRow.revision_no.desc())
|
stmt = select(AssistantKnowledgeWikiRevisionRow).where(AssistantKnowledgeWikiRevisionRow.page_id == page.id).order_by(AssistantKnowledgeWikiRevisionRow.revision_no.desc())
|
||||||
@@ -484,10 +1225,27 @@ class AssistantKnowledgeRepository:
|
|||||||
)
|
)
|
||||||
return {"left": left, "right": right, "diff": diff}
|
return {"left": left, "right": right, "diff": diff}
|
||||||
|
|
||||||
async def rollback(self, *, slug: str, revision_id: str, actor: str | None) -> dict[str, Any] | None:
|
async def rollback(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
slug: str,
|
||||||
|
revision_id: str,
|
||||||
|
actor: str | None,
|
||||||
|
base_id: str | None = None,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
return None
|
||||||
now = _now()
|
now = _now()
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
page = (await session.execute(select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.slug == slug.strip("/")))).scalar_one_or_none()
|
page = (
|
||||||
|
await session.execute(
|
||||||
|
select(AssistantKnowledgeWikiPageRow).where(
|
||||||
|
AssistantKnowledgeWikiPageRow.base_id == base["id"],
|
||||||
|
AssistantKnowledgeWikiPageRow.slug == slug.strip("/"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
source = await session.get(AssistantKnowledgeWikiRevisionRow, revision_id)
|
source = await session.get(AssistantKnowledgeWikiRevisionRow, revision_id)
|
||||||
if page is None or source is None or source.page_id != page.id:
|
if page is None or source is None or source.page_id != page.id:
|
||||||
return None
|
return None
|
||||||
@@ -517,9 +1275,19 @@ class AssistantKnowledgeRepository:
|
|||||||
await session.commit()
|
await session.commit()
|
||||||
return data
|
return data
|
||||||
|
|
||||||
async def list_sources(self) -> list[dict[str, Any]]:
|
async def list_sources(self, *, base_id: str | None = None) -> list[dict[str, Any]]:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
return []
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
stmt = select(AssistantKnowledgeSourceRow).where(AssistantKnowledgeSourceRow.status != "deleted").order_by(AssistantKnowledgeSourceRow.updated_at.desc())
|
stmt = (
|
||||||
|
select(AssistantKnowledgeSourceRow)
|
||||||
|
.where(
|
||||||
|
AssistantKnowledgeSourceRow.base_id == base["id"],
|
||||||
|
AssistantKnowledgeSourceRow.status != "deleted",
|
||||||
|
)
|
||||||
|
.order_by(AssistantKnowledgeSourceRow.updated_at.desc())
|
||||||
|
)
|
||||||
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||||
|
|
||||||
async def get_source(self, source_id: str) -> dict[str, Any] | None:
|
async def get_source(self, source_id: str) -> dict[str, Any] | None:
|
||||||
@@ -547,9 +1315,17 @@ class AssistantKnowledgeRepository:
|
|||||||
result["contributions"] = [_dict(value) for value in contributions]
|
result["contributions"] = [_dict(value) for value in contributions]
|
||||||
return result
|
return result
|
||||||
|
|
||||||
async def list_jobs(self, limit: int = 100) -> list[dict[str, Any]]:
|
async def list_jobs(self, limit: int = 100, *, base_id: str | None = None) -> list[dict[str, Any]]:
|
||||||
|
base = await self.get_base(base_id)
|
||||||
|
if base is None:
|
||||||
|
return []
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
stmt = select(AssistantKnowledgeImportJobRow).order_by(AssistantKnowledgeImportJobRow.created_at.desc()).limit(limit)
|
stmt = (
|
||||||
|
select(AssistantKnowledgeImportJobRow)
|
||||||
|
.where(AssistantKnowledgeImportJobRow.base_id == base["id"])
|
||||||
|
.order_by(AssistantKnowledgeImportJobRow.created_at.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
return [_dict(row) for row in (await session.execute(stmt)).scalars()]
|
||||||
|
|
||||||
async def get_job(self, job_id: str) -> dict[str, Any] | None:
|
async def get_job(self, job_id: str) -> dict[str, Any] | None:
|
||||||
@@ -565,6 +1341,7 @@ class AssistantKnowledgeRepository:
|
|||||||
source.status = "deleted"
|
source.status = "deleted"
|
||||||
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source_id))
|
await session.execute(delete(AssistantKnowledgeEntityContributionRow).where(AssistantKnowledgeEntityContributionRow.source_id == source_id))
|
||||||
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source_id))
|
await session.execute(delete(AssistantKnowledgeRelationContributionRow).where(AssistantKnowledgeRelationContributionRow.source_id == source_id))
|
||||||
|
await session.execute(delete(AssistantKnowledgeVectorRow).where(AssistantKnowledgeVectorRow.source_id == source_id))
|
||||||
pages = (
|
pages = (
|
||||||
await session.execute(
|
await session.execute(
|
||||||
select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.current_revision_id.in_(select(AssistantKnowledgeWikiRevisionRow.id).where(AssistantKnowledgeWikiRevisionRow.source_id == source_id)))
|
select(AssistantKnowledgeWikiPageRow).where(AssistantKnowledgeWikiPageRow.current_revision_id.in_(select(AssistantKnowledgeWikiRevisionRow.id).where(AssistantKnowledgeWikiRevisionRow.source_id == source_id)))
|
||||||
|
|||||||
@@ -162,7 +162,7 @@ async def _auto_create_postgres_db(url: str) -> None:
|
|||||||
try:
|
try:
|
||||||
async with maint_engine.connect() as conn:
|
async with maint_engine.connect() as conn:
|
||||||
await conn.execute(text(f'CREATE DATABASE "{db_name}"'))
|
await conn.execute(text(f'CREATE DATABASE "{db_name}"'))
|
||||||
logger.info("Auto-created PostgreSQL database: %s", db_name)
|
logger.info("Auto-created PostgreSQL-compatible database: %s", db_name)
|
||||||
finally:
|
finally:
|
||||||
await maint_engine.dispose()
|
await maint_engine.dispose()
|
||||||
|
|
||||||
@@ -216,10 +216,10 @@ def _ensure_orm_columns_sync(sync_conn) -> None:
|
|||||||
added column are created as well. Table-level ``UniqueConstraint`` objects
|
added column are created as well. Table-level ``UniqueConstraint`` objects
|
||||||
that are declared on the model but missing from the table are also created
|
that are declared on the model but missing from the table are also created
|
||||||
(as a ``CREATE UNIQUE INDEX`` of the same name, which is what the migrations
|
(as a ``CREATE UNIQUE INDEX`` of the same name, which is what the migrations
|
||||||
emit too, and works on SQLite/MySQL/PostgreSQL alike) — without this a newly
|
emit too, and works on SQLite/MySQL/PostgreSQL/openGauss alike) — without this a newly
|
||||||
declared uniqueness rule (e.g. ``active_dedupe_key``) would silently be absent
|
declared uniqueness rule (e.g. ``active_dedupe_key``) would silently be absent
|
||||||
on an existing database, weakening dedupe.
|
on an existing database, weakening dedupe.
|
||||||
Works on sqlite/mysql/postgres — the column DDL is compiled through the
|
Works on sqlite/mysql/postgres/gauss — the column DDL is compiled through the
|
||||||
active dialect, so portable types (``PortableJSON``/``PortableLongText``/
|
active dialect, so portable types (``PortableJSON``/``PortableLongText``/
|
||||||
``BeijingDateTime``) resolve to the right backend type. Everything is
|
``BeijingDateTime``) resolve to the right backend type. Everything is
|
||||||
per-column best-effort: a failure is logged and startup continues.
|
per-column best-effort: a failure is logged and startup continues.
|
||||||
@@ -341,13 +341,13 @@ async def init_engine(
|
|||||||
"""Create the async engine and session factory, then auto-create tables.
|
"""Create the async engine and session factory, then auto-create tables.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
backend: "memory", "sqlite", "postgres", or "mysql".
|
backend: "memory", "sqlite", "postgres", "gauss", or "mysql".
|
||||||
url: SQLAlchemy async URL (for sqlite/postgres/mysql).
|
url: SQLAlchemy async URL (for sqlite/postgres/gauss/mysql).
|
||||||
echo: Echo SQL to log.
|
echo: Echo SQL to log.
|
||||||
pool_size: Base connection pool size (postgres/mysql).
|
pool_size: Base connection pool size (postgres/gauss/mysql).
|
||||||
max_overflow: Extra connections allowed beyond pool_size under load (postgres/mysql).
|
max_overflow: Extra connections allowed beyond pool_size under load (postgres/gauss/mysql).
|
||||||
pool_timeout: Seconds to wait for a free pooled connection before erroring (postgres/mysql).
|
pool_timeout: Seconds to wait for a free pooled connection before erroring (postgres/gauss/mysql).
|
||||||
pool_recycle: Recycle connections after N seconds (postgres/mysql). Prevents stale
|
pool_recycle: Recycle connections after N seconds (postgres/gauss/mysql). Prevents stale
|
||||||
connections dropped by firewalls / MySQL wait_timeout / pgbouncer.
|
connections dropped by firewalls / MySQL wait_timeout / pgbouncer.
|
||||||
sqlite_dir: Directory to create for SQLite (ensured to exist).
|
sqlite_dir: Directory to create for SQLite (ensured to exist).
|
||||||
"""
|
"""
|
||||||
@@ -365,11 +365,21 @@ async def init_engine(
|
|||||||
except RuntimeError:
|
except RuntimeError:
|
||||||
_engine_loop = None
|
_engine_loop = None
|
||||||
|
|
||||||
if backend == "postgres":
|
if backend in {"postgres", "gauss"}:
|
||||||
try:
|
try:
|
||||||
import asyncpg # noqa: F401
|
import asyncpg # noqa: F401
|
||||||
except ImportError:
|
except ImportError:
|
||||||
raise ImportError("database.backend is set to 'postgres' but asyncpg is not installed.\nInstall it with:\n uv sync --extra postgres\nOr switch to backend: sqlite in config.yaml for single-node deployment.") from None
|
extra = "gauss" if backend == "gauss" else "postgres"
|
||||||
|
raise ImportError(
|
||||||
|
f"database.backend is set to {backend!r} but asyncpg is not installed.\n"
|
||||||
|
f"Install it with:\n uv sync --extra {extra}\n"
|
||||||
|
"Or switch to backend: sqlite in config.yaml for single-node deployment."
|
||||||
|
) from None
|
||||||
|
if backend == "gauss" and find_spec("opengauss_sqlalchemy") is None:
|
||||||
|
raise ImportError(
|
||||||
|
"database.backend is set to 'gauss' but the openGauss SQLAlchemy dialect "
|
||||||
|
"is not installed.\nInstall it with:\n uv sync --extra gauss"
|
||||||
|
)
|
||||||
if backend == "mysql":
|
if backend == "mysql":
|
||||||
_add_runtime_vendor_path()
|
_add_runtime_vendor_path()
|
||||||
driver = _mysql_driver_name(url)
|
driver = _mysql_driver_name(url)
|
||||||
@@ -409,7 +419,7 @@ async def init_engine(
|
|||||||
cursor.execute("PRAGMA foreign_keys=ON;")
|
cursor.execute("PRAGMA foreign_keys=ON;")
|
||||||
finally:
|
finally:
|
||||||
cursor.close()
|
cursor.close()
|
||||||
elif backend == "postgres":
|
elif backend in {"postgres", "gauss"}:
|
||||||
_engine = create_async_engine(
|
_engine = create_async_engine(
|
||||||
url,
|
url,
|
||||||
echo=echo,
|
echo=echo,
|
||||||
@@ -454,7 +464,7 @@ async def init_engine(
|
|||||||
async with _engine.begin() as conn:
|
async with _engine.begin() as conn:
|
||||||
await conn.run_sync(Base.metadata.create_all)
|
await conn.run_sync(Base.metadata.create_all)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
if backend == "postgres" and "does not exist" in str(exc):
|
if backend in {"postgres", "gauss"} and "does not exist" in str(exc).lower():
|
||||||
# Database not yet created — attempt to auto-create it, then retry.
|
# Database not yet created — attempt to auto-create it, then retry.
|
||||||
await _auto_create_postgres_db(url)
|
await _auto_create_postgres_db(url)
|
||||||
# Rebuild engine against the now-existing database
|
# Rebuild engine against the now-existing database
|
||||||
@@ -497,7 +507,7 @@ async def init_engine(
|
|||||||
# cannot do on its own. Failures don't abort startup — see the helper.
|
# cannot do on its own. Failures don't abort startup — see the helper.
|
||||||
# await _run_alembic_upgrade(url)
|
# await _run_alembic_upgrade(url)
|
||||||
|
|
||||||
logger.info("Persistence engine initialized: backend=%s", backend)
|
logger.info("Persistence engine initialized: backend=%s dialect=%s", backend, _engine.dialect.name)
|
||||||
|
|
||||||
|
|
||||||
async def init_engine_from_config(config) -> None:
|
async def init_engine_from_config(config) -> None:
|
||||||
|
|||||||
+18
@@ -3,6 +3,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
@@ -35,6 +36,23 @@ class LlmWikiIndexStore(ABC):
|
|||||||
@abstractmethod
|
@abstractmethod
|
||||||
async def load_vector_snapshot(self, mapping_id: str, fingerprint: str) -> dict[str, Any]: ...
|
async def load_vector_snapshot(self, mapping_id: str, fingerprint: str) -> dict[str, Any]: ...
|
||||||
|
|
||||||
|
async def iter_vector_snapshot_rows(
|
||||||
|
self,
|
||||||
|
mapping_id: str,
|
||||||
|
fingerprint: str,
|
||||||
|
*,
|
||||||
|
batch_size: int = 1000,
|
||||||
|
) -> AsyncIterator[dict[str, Any]]:
|
||||||
|
"""Iterate vector rows for streaming export.
|
||||||
|
|
||||||
|
Implementations with a database backend should override this method so
|
||||||
|
very large vector packages do not need to be materialized in memory.
|
||||||
|
"""
|
||||||
|
snapshot = await self.load_vector_snapshot(mapping_id, fingerprint)
|
||||||
|
for row in snapshot.get("rows") or []:
|
||||||
|
if isinstance(row, dict):
|
||||||
|
yield row
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]: ...
|
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]: ...
|
||||||
|
|
||||||
|
|||||||
+36
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
@@ -181,6 +182,41 @@ class SqlLlmWikiIndexStore(LlmWikiIndexStore):
|
|||||||
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
|
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
|
||||||
return {"revision": int(state.index_revision if state else 0), "fingerprint": fingerprint, "rows": rows}
|
return {"revision": int(state.index_revision if state else 0), "fingerprint": fingerprint, "rows": rows}
|
||||||
|
|
||||||
|
async def iter_vector_snapshot_rows(
|
||||||
|
self,
|
||||||
|
mapping_id: str,
|
||||||
|
fingerprint: str,
|
||||||
|
*,
|
||||||
|
batch_size: int = 1000,
|
||||||
|
) -> AsyncIterator[dict[str, Any]]:
|
||||||
|
batch = max(1, min(int(batch_size), 5000))
|
||||||
|
offset = 0
|
||||||
|
while True:
|
||||||
|
stmt = (
|
||||||
|
select(LlmWikiWikiVectorRow, LlmWikiWikiPageRow)
|
||||||
|
.join(LlmWikiWikiPageRow, LlmWikiWikiPageRow.id == LlmWikiWikiVectorRow.page_id)
|
||||||
|
.where(
|
||||||
|
LlmWikiWikiVectorRow.knowledge_base_mapping_id == mapping_id,
|
||||||
|
LlmWikiWikiVectorRow.embedding_fingerprint == fingerprint,
|
||||||
|
LlmWikiWikiPageRow.index_status == "ready",
|
||||||
|
LlmWikiWikiPageRow.is_remote_deleted.is_(False),
|
||||||
|
LlmWikiWikiPageRow.status.in_(["published", "draft"]),
|
||||||
|
)
|
||||||
|
.order_by(LlmWikiWikiVectorRow.page_id, LlmWikiWikiVectorRow.section_index)
|
||||||
|
.limit(batch)
|
||||||
|
.offset(offset)
|
||||||
|
)
|
||||||
|
async with self._sf() as session:
|
||||||
|
pairs = list((await session.execute(stmt)).all())
|
||||||
|
rows = [{**_dict(vector), "page": _dict(page)} for vector, page in pairs]
|
||||||
|
if not rows:
|
||||||
|
break
|
||||||
|
for row in rows:
|
||||||
|
yield row
|
||||||
|
if len(rows) < batch:
|
||||||
|
break
|
||||||
|
offset += len(rows)
|
||||||
|
|
||||||
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]:
|
async def get_index_revision(self, mapping_id: str) -> tuple[int, str | None, str]:
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
state = await session.get(LlmWikiWikiSyncStateRow, mapping_id)
|
state = await session.get(LlmWikiWikiSyncStateRow, mapping_id)
|
||||||
|
|||||||
+25
@@ -0,0 +1,25 @@
|
|||||||
|
"""Add assistant knowledge vector table.
|
||||||
|
|
||||||
|
Revision ID: 20260902_01
|
||||||
|
Revises: 20260901_01
|
||||||
|
Create Date: 2026-09-02
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
from deerflow.persistence.assistant_knowledge.model import AssistantKnowledgeVectorRow
|
||||||
|
|
||||||
|
revision = "20260902_01"
|
||||||
|
down_revision = "20260901_01"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
AssistantKnowledgeVectorRow.__table__.create(op.get_bind(), checkfirst=True)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
AssistantKnowledgeVectorRow.__table__.drop(op.get_bind(), checkfirst=True)
|
||||||
+1
@@ -43,6 +43,7 @@ from deerflow.persistence.assistant_knowledge.model import (
|
|||||||
AssistantKnowledgeRelationContributionRow,
|
AssistantKnowledgeRelationContributionRow,
|
||||||
AssistantKnowledgeRelationRow,
|
AssistantKnowledgeRelationRow,
|
||||||
AssistantKnowledgeSourceRow,
|
AssistantKnowledgeSourceRow,
|
||||||
|
AssistantKnowledgeVectorRow,
|
||||||
AssistantKnowledgeWikiPageRow,
|
AssistantKnowledgeWikiPageRow,
|
||||||
AssistantKnowledgeWikiRevisionRow,
|
AssistantKnowledgeWikiRevisionRow,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import UTC, datetime, timedelta, timezone
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import JSON, DateTime, Text
|
from sqlalchemy import JSON, DateTime, Text
|
||||||
@@ -14,6 +14,12 @@ from sqlalchemy.types import TypeDecorator
|
|||||||
# fixed offset is exact and avoids depending on the `tzdata` package (which the
|
# fixed offset is exact and avoids depending on the `tzdata` package (which the
|
||||||
# stdlib `zoneinfo` needs on Windows / minimal containers).
|
# stdlib `zoneinfo` needs on Windows / minimal containers).
|
||||||
BEIJING_TZ = timezone(timedelta(hours=8))
|
BEIJING_TZ = timezone(timedelta(hours=8))
|
||||||
|
_POSTGRES_FAMILY_DIALECTS = frozenset({"postgresql", "opengauss"})
|
||||||
|
|
||||||
|
|
||||||
|
def _is_postgres_family(dialect) -> bool:
|
||||||
|
"""Return whether *dialect* preserves timezone-aware timestamps natively."""
|
||||||
|
return dialect.name in _POSTGRES_FAMILY_DIALECTS
|
||||||
|
|
||||||
|
|
||||||
class PortableJSON(TypeDecorator):
|
class PortableJSON(TypeDecorator):
|
||||||
@@ -87,17 +93,17 @@ class BeijingDateTime(TypeDecorator):
|
|||||||
def process_bind_param(self, value: Any, dialect) -> Any:
|
def process_bind_param(self, value: Any, dialect) -> Any:
|
||||||
if not isinstance(value, datetime):
|
if not isinstance(value, datetime):
|
||||||
return value
|
return value
|
||||||
if dialect.name == "postgresql":
|
if _is_postgres_family(dialect):
|
||||||
return value
|
return value
|
||||||
if value.tzinfo is None:
|
if value.tzinfo is None:
|
||||||
# Naive values follow the codebase convention of being UTC.
|
# Naive values follow the codebase convention of being UTC.
|
||||||
value = value.replace(tzinfo=timezone.utc)
|
value = value.replace(tzinfo=UTC)
|
||||||
return value.astimezone(BEIJING_TZ).replace(tzinfo=None)
|
return value.astimezone(BEIJING_TZ).replace(tzinfo=None)
|
||||||
|
|
||||||
def process_result_value(self, value: Any, dialect) -> Any:
|
def process_result_value(self, value: Any, dialect) -> Any:
|
||||||
if not isinstance(value, datetime):
|
if not isinstance(value, datetime):
|
||||||
return value
|
return value
|
||||||
if dialect.name == "postgresql":
|
if _is_postgres_family(dialect):
|
||||||
return value
|
return value
|
||||||
if value.tzinfo is None:
|
if value.tzinfo is None:
|
||||||
# Stored as naive Beijing wall-clock; re-attach the zone.
|
# Stored as naive Beijing wall-clock; re-attach the zone.
|
||||||
|
|||||||
@@ -52,8 +52,12 @@ postgres = [
|
|||||||
"psycopg[binary]>=3.3.3",
|
"psycopg[binary]>=3.3.3",
|
||||||
"psycopg-pool>=3.3.0",
|
"psycopg-pool>=3.3.0",
|
||||||
]
|
]
|
||||||
mysql = ["asyncmy>=0.2.10"]
|
mysql = ["asyncmy>=0.2.10"]
|
||||||
pymupdf = ["pymupdf4llm>=0.0.17"]
|
gauss = [
|
||||||
|
"asyncpg>=0.29",
|
||||||
|
"opengauss-sqlalchemy>=2.4,<3",
|
||||||
|
]
|
||||||
|
pymupdf = ["pymupdf4llm>=0.0.17"]
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
requires = ["hatchling"]
|
requires = ["hatchling"]
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ dependencies = [
|
|||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
postgres = ["deerflow-harness[postgres]"]
|
postgres = ["deerflow-harness[postgres]"]
|
||||||
mysql = ["deerflow-harness[mysql]"]
|
mysql = ["deerflow-harness[mysql]"]
|
||||||
|
gauss = ["deerflow-harness[gauss]"]
|
||||||
|
|
||||||
[dependency-groups]
|
[dependency-groups]
|
||||||
dev = [
|
dev = [
|
||||||
|
|||||||
@@ -22,6 +22,18 @@ class _FakeThreadStore:
|
|||||||
return self.rows[offset : offset + limit]
|
return self.rows[offset : offset + limit]
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeAgentStore:
|
||||||
|
async def list_by_ids(self, agent_ids: list[str], user_id: str) -> list[dict]:
|
||||||
|
assert user_id == "owner"
|
||||||
|
agents = {
|
||||||
|
"analysis-agent": {
|
||||||
|
"id": "analysis-agent",
|
||||||
|
"name": "舆情分析智能体",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return [agents[agent_id] for agent_id in agent_ids if agent_id in agents]
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: Path, monkeypatch):
|
async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: Path, monkeypatch):
|
||||||
thread_one = tmp_path / "thread-one" / "outputs"
|
thread_one = tmp_path / "thread-one" / "outputs"
|
||||||
@@ -79,6 +91,7 @@ async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: P
|
|||||||
|
|
||||||
assert [item["name"] for item in items] == ["证据.zip", "结论.md", "数据汇总.xlsx", "研判报告.docx"]
|
assert [item["name"] for item in items] == ["证据.zip", "结论.md", "数据汇总.xlsx", "研判报告.docx"]
|
||||||
assert [item["file_kind"] for item in items] == ["archive", "text_data", "spreadsheet", "document"]
|
assert [item["file_kind"] for item in items] == ["archive", "text_data", "spreadsheet", "document"]
|
||||||
|
assert [item["source_type"] for item in items] == ["general_qa", "general_qa", "agent", "agent"]
|
||||||
assert items[-1]["thread_title"] == "战场环境综合研判"
|
assert items[-1]["thread_title"] == "战场环境综合研判"
|
||||||
assert items[-1]["agent_id"] == "analysis-agent"
|
assert items[-1]["agent_id"] == "analysis-agent"
|
||||||
assert all(item["name"] not in {"配图.png", "录音.mp3"} for item in items)
|
assert all(item["name"] not in {"配图.png", "录音.mp3"} for item in items)
|
||||||
@@ -88,9 +101,12 @@ async def test_artifact_library_aggregates_generated_non_media_files(tmp_path: P
|
|||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Path, monkeypatch):
|
async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Path, monkeypatch):
|
||||||
output_dir = tmp_path / "thread-one" / "outputs"
|
output_dir = tmp_path / "thread-one" / "outputs"
|
||||||
|
agent_output_dir = tmp_path / "thread-agent" / "outputs"
|
||||||
output_dir.mkdir(parents=True)
|
output_dir.mkdir(parents=True)
|
||||||
|
agent_output_dir.mkdir(parents=True)
|
||||||
(output_dir / "任务报告.md").write_text("report", encoding="utf-8")
|
(output_dir / "任务报告.md").write_text("report", encoding="utf-8")
|
||||||
(output_dir / "任务数据.csv").write_text("a,b", encoding="utf-8")
|
(output_dir / "任务数据.csv").write_text("a,b", encoding="utf-8")
|
||||||
|
(agent_output_dir / "智能体结论.md").write_text("agent report", encoding="utf-8")
|
||||||
|
|
||||||
store = _FakeThreadStore(
|
store = _FakeThreadStore(
|
||||||
[
|
[
|
||||||
@@ -98,7 +114,12 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
|||||||
"thread_id": "thread-one",
|
"thread_id": "thread-one",
|
||||||
"display_name": "任务研判对话",
|
"display_name": "任务研判对话",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
}
|
},
|
||||||
|
{
|
||||||
|
"thread_id": "thread-agent",
|
||||||
|
"display_name": "智能体研判对话",
|
||||||
|
"metadata": {"agent_id": "analysis-agent"},
|
||||||
|
},
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -113,6 +134,7 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
|||||||
monkeypatch.setattr(artifacts, "aresolve_path_user_id", _resolve_owner)
|
monkeypatch.setattr(artifacts, "aresolve_path_user_id", _resolve_owner)
|
||||||
monkeypatch.setattr(artifacts, "get_effective_user_id", lambda: "owner")
|
monkeypatch.setattr(artifacts, "get_effective_user_id", lambda: "owner")
|
||||||
monkeypatch.setattr(deps, "get_thread_store", lambda _request: store)
|
monkeypatch.setattr(deps, "get_thread_store", lambda _request: store)
|
||||||
|
monkeypatch.setattr(deps, "get_agent_store", lambda _request: _FakeAgentStore())
|
||||||
artifacts._artifact_library_cache.clear()
|
artifacts._artifact_library_cache.clear()
|
||||||
|
|
||||||
raw_endpoint = artifacts.list_artifact_library.__wrapped__
|
raw_endpoint = artifacts.list_artifact_library.__wrapped__
|
||||||
@@ -120,6 +142,7 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
|||||||
request=SimpleNamespace(),
|
request=SimpleNamespace(),
|
||||||
query="研判对话",
|
query="研判对话",
|
||||||
file_kind="text_data",
|
file_kind="text_data",
|
||||||
|
source_type="general_qa",
|
||||||
offset=0,
|
offset=0,
|
||||||
limit=1,
|
limit=1,
|
||||||
refresh=True,
|
refresh=True,
|
||||||
@@ -134,9 +157,24 @@ async def test_artifact_library_endpoint_filters_searches_and_pages(tmp_path: Pa
|
|||||||
request=SimpleNamespace(),
|
request=SimpleNamespace(),
|
||||||
query="任务数据",
|
query="任务数据",
|
||||||
file_kind="spreadsheet",
|
file_kind="spreadsheet",
|
||||||
|
source_type="all",
|
||||||
offset=0,
|
offset=0,
|
||||||
limit=50,
|
limit=50,
|
||||||
refresh=False,
|
refresh=False,
|
||||||
)
|
)
|
||||||
assert spreadsheet_response.total == 1
|
assert spreadsheet_response.total == 1
|
||||||
assert spreadsheet_response.items[0].name == "任务数据.csv"
|
assert spreadsheet_response.items[0].name == "任务数据.csv"
|
||||||
|
|
||||||
|
agent_response = await raw_endpoint(
|
||||||
|
request=SimpleNamespace(),
|
||||||
|
query=None,
|
||||||
|
file_kind="all",
|
||||||
|
source_type="agent",
|
||||||
|
offset=0,
|
||||||
|
limit=50,
|
||||||
|
refresh=False,
|
||||||
|
)
|
||||||
|
assert agent_response.total == 1
|
||||||
|
assert agent_response.items[0].source_type == "agent"
|
||||||
|
assert agent_response.items[0].agent_name == "舆情分析智能体"
|
||||||
|
assert agent_response.items[0].name == "智能体结论.md"
|
||||||
@@ -11,6 +11,13 @@ from deerflow.persistence.assistant_knowledge.model import (
|
|||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class FakeEmbeddingClient:
|
||||||
|
dimensions = 3
|
||||||
|
|
||||||
|
async def embed_query(self, _query: str) -> list[float]:
|
||||||
|
return [1.0, 0.0, 0.0]
|
||||||
|
|
||||||
|
|
||||||
@pytest_asyncio.fixture
|
@pytest_asyncio.fixture
|
||||||
async def repository():
|
async def repository():
|
||||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||||
@@ -94,3 +101,93 @@ async def test_delete_source_keeps_entity_until_last_contribution_is_removed(
|
|||||||
async with repository._sf() as session:
|
async with repository._sf() as session:
|
||||||
entity = (await session.execute(select(AssistantKnowledgeEntityRow))).scalar_one()
|
entity = (await session.execute(select(AssistantKnowledgeEntityRow))).scalar_one()
|
||||||
assert entity.status == "deleted"
|
assert entity.status == "deleted"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_custom_base_import_is_also_mirrored_to_global_base(
|
||||||
|
repository: AssistantKnowledgeRepository,
|
||||||
|
) -> None:
|
||||||
|
global_base = await repository.initialize(created_by="admin")
|
||||||
|
custom_base = await repository.create_base(
|
||||||
|
created_by="admin",
|
||||||
|
name="专题助手库",
|
||||||
|
description="只承接某个专题的 WeKnora 导出数据",
|
||||||
|
)
|
||||||
|
package = {
|
||||||
|
"source": {"type": "weknora"},
|
||||||
|
"wiki_pages": [{"slug": "wiki/topic", "title": "专题 Wiki", "content": "王五负责统筹。"}],
|
||||||
|
"entities": [{"local_id": "e1", "type": "person", "name": "王五"}],
|
||||||
|
"relations": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
job = await repository.import_package(
|
||||||
|
package=package,
|
||||||
|
source_type="weknora",
|
||||||
|
source_key="kb-topic",
|
||||||
|
source_name="专题普通库",
|
||||||
|
trigger="manual_from_weknora",
|
||||||
|
created_by="admin",
|
||||||
|
mapping_id="kb-topic",
|
||||||
|
base_id=custom_base["id"],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert job["base_id"] == custom_base["id"]
|
||||||
|
assert job["mirrored_global_base_id"] == global_base["id"]
|
||||||
|
assert len(await repository.list_pages(base_id=custom_base["id"], query="王五")) == 2
|
||||||
|
assert len(await repository.list_pages(base_id=global_base["id"], query="王五")) == 2
|
||||||
|
bases = await repository.list_bases()
|
||||||
|
assert [base["id"] for base in bases[:2]] == [global_base["id"], custom_base["id"]]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_imported_vectors_are_reused_for_assistant_search(
|
||||||
|
repository: AssistantKnowledgeRepository,
|
||||||
|
) -> None:
|
||||||
|
base = await repository.initialize(created_by="admin")
|
||||||
|
job = await repository.import_package(
|
||||||
|
package={
|
||||||
|
"source": {"type": "weknora"},
|
||||||
|
"wiki_pages": [{"slug": "wiki/vector", "title": "向量页", "content": "页面正文"}],
|
||||||
|
"vectors": [
|
||||||
|
{
|
||||||
|
"wiki_slug": "wiki/vector",
|
||||||
|
"section_index": 0,
|
||||||
|
"section_content": "张三负责向量化检索。",
|
||||||
|
"embedding_model": "bge-embedding-m3",
|
||||||
|
"embedding_dimensions": 3,
|
||||||
|
"vector": [1.0, 0.0, 0.0],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"wiki_slug": "wiki/vector",
|
||||||
|
"section_index": 1,
|
||||||
|
"section_content": "无关片段。",
|
||||||
|
"embedding_model": "bge-embedding-m3",
|
||||||
|
"embedding_dimensions": 3,
|
||||||
|
"vector": [0.0, 1.0, 0.0],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
source_type="weknora",
|
||||||
|
source_key="kb-vector",
|
||||||
|
source_name="向量普通库",
|
||||||
|
trigger="manual_package_upload",
|
||||||
|
created_by="admin",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert job["counts"]["vectors"] == 2
|
||||||
|
results = await repository.search_pages(
|
||||||
|
base_ids=[base["id"]],
|
||||||
|
query="这个查询不靠关键词",
|
||||||
|
embedding_client=FakeEmbeddingClient(),
|
||||||
|
)
|
||||||
|
assert results[0]["retrieval_mode"] == "vector"
|
||||||
|
assert results[0]["matched_section"]["embedding_model"] == "bge-embedding-m3"
|
||||||
|
assert results[0]["content_markdown"] == "张三负责向量化检索。"
|
||||||
|
|
||||||
|
sources = await repository.list_sources()
|
||||||
|
assert await repository.delete_source(sources[0]["id"])
|
||||||
|
assert await repository.search_pages(
|
||||||
|
base_ids=[base["id"]],
|
||||||
|
query="这个查询不靠关键词",
|
||||||
|
embedding_client=FakeEmbeddingClient(),
|
||||||
|
) == []
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
"""Regression coverage for the openGauss/GaussDB persistence backend."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from pydantic import ValidationError
|
||||||
|
from sqlalchemy.engine import make_url
|
||||||
|
from sqlalchemy.schema import CreateIndex, CreateTable
|
||||||
|
|
||||||
|
from deerflow.config.database_config import DatabaseConfig
|
||||||
|
from deerflow.persistence.types import BeijingDateTime
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("configured", "expected"),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
"opengauss://user:pass@127.0.0.1:26000/deerflow",
|
||||||
|
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"postgresql://user:pass@127.0.0.1:26000/deerflow?ssl=require",
|
||||||
|
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow?ssl=require",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
|
||||||
|
"opengauss+asyncpg://user:pass@127.0.0.1:26000/deerflow",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_gauss_urls_always_select_the_opengauss_async_dialect(
|
||||||
|
configured: str,
|
||||||
|
expected: str,
|
||||||
|
) -> None:
|
||||||
|
config = DatabaseConfig(backend="gauss", gauss_url=configured)
|
||||||
|
assert config.app_sqlalchemy_url == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_gauss_backend_requires_a_supported_nonempty_url() -> None:
|
||||||
|
with pytest.raises(ValueError, match="gauss_url is required"):
|
||||||
|
_ = DatabaseConfig(backend="gauss").app_sqlalchemy_url
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="must use opengauss"):
|
||||||
|
_ = DatabaseConfig(
|
||||||
|
backend="gauss",
|
||||||
|
gauss_url="mysql://user:pass@127.0.0.1/deerflow",
|
||||||
|
).app_sqlalchemy_url
|
||||||
|
|
||||||
|
with pytest.raises(ValidationError):
|
||||||
|
DatabaseConfig(backend="unknown")
|
||||||
|
|
||||||
|
|
||||||
|
def test_gauss_timestamps_keep_timezone_information() -> None:
|
||||||
|
dialect = type("GaussDialect", (), {"name": "opengauss"})()
|
||||||
|
value = datetime(2026, 9, 2, 8, 30, tzinfo=UTC)
|
||||||
|
column_type = BeijingDateTime()
|
||||||
|
|
||||||
|
assert column_type.process_bind_param(value, dialect) is value
|
||||||
|
assert column_type.process_result_value(value, dialect) is value
|
||||||
|
|
||||||
|
|
||||||
|
def test_official_async_dialect_compiles_the_complete_orm_schema() -> None:
|
||||||
|
"""Every registered table/index must compile before startup touches GaussDB."""
|
||||||
|
pytest.importorskip("opengauss_sqlalchemy")
|
||||||
|
|
||||||
|
import deerflow.persistence.models # noqa: F401 -- register ORM metadata
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
dialect_cls = make_url("opengauss+asyncpg://user:pass@localhost/db").get_dialect(
|
||||||
|
_is_async=True
|
||||||
|
)
|
||||||
|
dialect = dialect_cls()
|
||||||
|
|
||||||
|
assert dialect.__class__.__module__.startswith("opengauss_sqlalchemy")
|
||||||
|
assert Base.metadata.sorted_tables
|
||||||
|
for table in Base.metadata.sorted_tables:
|
||||||
|
assert str(CreateTable(table).compile(dialect=dialect)).startswith("\nCREATE TABLE")
|
||||||
|
for index in table.indexes:
|
||||||
|
assert str(CreateIndex(index).compile(dialect=dialect)).startswith("CREATE")
|
||||||
@@ -66,6 +66,9 @@ async def test_sql_store_atomically_replaces_vectors_and_marks_missing_deleted(t
|
|||||||
assert store.write_batch_size == 1
|
assert store.write_batch_size == 1
|
||||||
assert len(snapshot["rows"]) == 3
|
assert len(snapshot["rows"]) == 3
|
||||||
assert decode_vector(snapshot["rows"][0]["vector_blob"], 2).tolist() == pytest.approx([0.6, 0.8])
|
assert decode_vector(snapshot["rows"][0]["vector_blob"], 2).tolist() == pytest.approx([0.6, 0.8])
|
||||||
|
streamed = [row async for row in store.iter_vector_snapshot_rows("mapping-1", "fp", batch_size=2)]
|
||||||
|
assert [row["section_index"] for row in streamed] == [0, 1, 2]
|
||||||
|
assert streamed[0]["page"]["slug"] == "people/alice"
|
||||||
page_statuses = await store.list_page_index_status("mapping-1")
|
page_statuses = await store.list_page_index_status("mapping-1")
|
||||||
assert len(page_statuses) == 1
|
assert len(page_statuses) == 1
|
||||||
assert page_statuses[0] == {
|
assert page_statuses[0] == {
|
||||||
|
|||||||
Reference in new issue
Block a user