容错显示

This commit is contained in:
董宏凯 committed 2026-05-18 17:41:51 +08:00
1 parent 836dd73446
commit 5be5531722
17 files changed
+433 -76

No files matched your search

+188
View File
@@ -0,0 +1,188 @@
# 素材专家检索 —— 内网接口对接说明
本文档说明 AI 智能写作中「素材收集专家」的检索机制、内网知识库接口的对接逻辑、
涉及的代码文件,以及常见错误的排查方式。
---
## 1. 检索来源切换
素材专家支持多种检索来源,通过**后端配置文件**统一切换,**无需改动任何代码**。
配置位置:`offline-backend-20260512/backend/config.yaml`
```yaml
ai_writing:
# 检索来源开关:internet=互联网检索(默认)/ intranet=内网知识库 / tavily / demo
search_provider: internet
# 以下两项仅在 search_provider=intranet 时生效
intranet_search_url: http://10.106.57.180:7863/knowledge_base/search_docs_es
intranet_knowledge_base: samples
```
| `search_provider` 取值 | 含义 |
|---|---|
| `internet` | 互联网检索(DuckDuckGo),失败时回退到 demo 演示素材。**当前默认值** |
| `intranet` | 内网知识库检索(调用 `search_docs_es` 接口) |
| `tavily` | Tavily 互联网检索 |
| `demo` | 内置演示素材,仅用于本地调试 |
> 内网接口联通后,把 `search_provider` 改为 `intranet` 即可切换。
> 配置文件支持热加载(`get_app_config()` 按 mtime 自动重载),改完一般无需重启后端。
---
## 2. 内网接口规格
- **地址**:`http://10.106.57.180:7863/knowledge_base/search_docs_es`
- **方法**:`POST`
- **请求体**:
```json
{
"query": "检索关键词",
"knowledge_base_name": "samples",
"ts": "yes"
}
```
- **响应体**:一个数组,每个元素结构如下:
```json
[
{
"page_content": "正文片段",
"metadata": {
"m_title": "标题",
"content": "内容",
"recUuid": "原文id",
"m_publish":"发布时间",
"source1": "来源"
}
}
]
```
我们需要的字段都在 `metadata` 里。
### 字段映射
后端把内网响应映射为统一的 `SearchResult` 结构:
| 内网响应字段 | SearchResult 字段 | 说明 |
|--------------------------|--------------------|------|
| `metadata.m_title` | `title` | 标题,缺省为「(无标题)」 |
| `metadata.content` | `content` | 内容,截断到 500 字 |
| `metadata.source1` | `source` | 来源,缺省为「内网知识库」 |
| `metadata.m_publish` | `published_at` | 发布时间 |
| `metadata.recUuid` | `recUuid` | 原文 id,用于前端拼接「查看原文」链接 |
| — | `url` | 固定为空,原文地址由前端依据 `recUuid` 拼接 |
| — | `relevance_score` | 接口不返回分值,按返回顺序赋递减分以保留 ES 排序 |
| — | `source_type` | 固定为 `"intranet"` |
---
## 3. 对接逻辑与涉及文件
### 后端
| 文件 | 作用 |
|------|------|
| `packages/harness/deerflow/agents/ai_writing/search/intranet_stub.py` | **内网检索适配器** `IntranetSearchProvider`。发起 POST 请求、解析响应、映射为 `SearchResult` |
| `packages/harness/deerflow/agents/ai_writing/search/__init__.py` | `get_search_provider()` 工厂函数,依据 `search_provider` 配置返回对应 provider |
| `packages/harness/deerflow/agents/ai_writing/search/base.py` | `SearchResult` 数据结构 + `BaseSearchProvider` 抽象基类 |
| `packages/harness/deerflow/agents/ai_writing/nodes/researcher.py` | 素材收集专家节点,调用 provider 检索并把结果合并、排序为素材包 |
| `app/gateway/routers/ai_writing.py` | `_ai_writing_config()` 读取 `config.yaml` 的 `ai_writing` 段,经 `_make_config()` 注入运行时 |
| `config.yaml` | `ai_writing` 段,检索来源开关与内网接口地址 |
调用链:
```
ai_writing.py(_make_config 注入 ai_writing_config)
→ researcher_node
→ get_search_provider(ai_writing_config) # 依据 search_provider 选择
→ IntranetSearchProvider.search() # POST search_docs_es
→ SearchResult[] # 字段映射
```
### 前端
| 文件 | 作用 |
|------|------|
| `src/core/config/index.ts` | `INTRANET_SOURCE_URL_BASE` 原文详情页地址前缀 + `buildIntranetSourceUrl()` 构造函数 |
| `src/open-canvas/components/ai-writing/MaterialItem.tsx` | 素材卡片,`recUuid` 非空时显示「查看原文」按钮 |
| `src/open-canvas/components/ai-writing/CompletedInterventionCard.tsx` | 历史回显的素材卡,「原文」链接同样基于 `recUuid` |
| `src/open-canvas/api/ai-writing.ts` | `normalizeMaterial()` 把后端 snake_case 字段转为前端 camelCase(含 `recUuid`) |
「查看原文」链接拼接规则:
```
INTRANET_SOURCE_URL_BASE + recUuid
= https://chlib.b.uat.4.cn/knowlibrary/#/knowledge/mid_detail?recUuid={recUuid}
```
- 仅当素材的 `recUuid` **非空**时才显示「查看原文」按钮。
- 点击直接 `window.open(...)` 新标签页打开,不做站内跳转。
---
## 4. 错误处理与排查
### 4.1 内网检索返回空 / 调不通
**现象**:切到 `intranet` 后,素材专家时间线出现琥珀色告警「ES 波动导致检索异常」,
下方显示真实报错原因;若完全无结果,再提示「未检索到任何素材」。
`IntranetSearchProvider.search()` 对所有异常(连接失败、超时、非 2xx、JSON 解析失败)
都做了兜底:**捕获后记录 warning 日志、把原因收集到 `provider.errors`,并返回空列表**,
不会让整个写作流程崩溃。`researcher_node` 在检索结束后取出 `provider.errors`,
以非致命的 `notice` 事件推送到前端时间线(标题「ES 波动导致检索异常」+ 真实原因),
**接口报错不影响写作流程继续**。
> 容错相关代码:
> - 检索异常收集:`intranet_stub.py` / `duckduckgo_search.py` 的 `errors` 列表
> - 告警事件发出:`researcher.py` 的 `_notice()` 与 `progress_events`
> - 模型调用异常:`researcher.py`(检索词/排序)、`writer_outline.py` / `writer_draft.py` /
> `editor.py` 均会发出「模型错误」事件(致命错误用 `error`,可兜底的用 `notice`)
> - 前端展示:`TimelineStep.tsx` 渲染 `notice`(琥珀)/ `error`(红色)的标题 + 真实原因
排查步骤:
1. **确认网络可达**:在后端所在机器执行
`curl -X POST http://10.106.57.180:7863/knowledge_base/search_docs_es -H "Content-Type: application/json" -d "{\"query\":\"测试\",\"knowledge_base_name\":\"samples\",\"ts\":\"yes\"}"`
2. **查看后端日志**:搜索关键字 `intranet search failed`,会打印具体异常(连接超时 / DNS / 401 等)。
3. **确认配置**:`config.yaml` 的 `intranet_search_url`、`intranet_knowledge_base` 是否正确。
空结果时前端已有兜底:素材确认卡会弹出「未检索到任何素材,是否在无素材的情况下继续写作」,
用户可以选择「重新搜索」或「无数据继续写作」。
### 4.2 字段缺失 / 响应结构变化
`search()` 对每个字段都用 `or` 兜底(如 `m_title` 缺失时回退为「(无标题)」),
单条数据字段缺失不会导致整批失败。
若内网接口改为「数组外包一层 `data`/`results`」的结构,适配器已兼容;
若改成其它结构,需修改 `intranet_stub.py` 中解析 `items` 的逻辑。
### 4.3 「查看原文」按钮不显示
**原因**:该素材的 `recUuid` 为空。这是预期行为 —— 只有内网知识库素材才带 `recUuid`,
互联网检索(`internet`/`tavily`/`demo`)的素材没有 `recUuid`,不显示「查看原文」。
如需让互联网素材也能跳转,需另行调整 `MaterialItem.tsx` 的显示条件。
### 4.4 原文链接打不开 / 地址不对
「查看原文」地址前缀写在 `src/core/config/index.ts` 的 `INTRANET_SOURCE_URL_BASE`。
若内网原文详情页地址有变,只改这一个常量即可。
---
## 5. 联调上线步骤
1. 确认后端机器能访问 `10.106.57.180:7863`。
2. 把 `config.yaml` 的 `ai_writing.search_provider` 改为 `intranet`。
3. 后端配置热加载生效(或重启后端)。
4. 进入 AI 智能写作页面,发起一次写作,观察素材收集专家是否返回内网素材。
5. 校验素材卡片的标题/来源/时间显示正确,「查看原文」能跳到内网详情页。
6. 如检索为空,按 [4.1](#41-内网检索返回空--调不通) 排查。
+13
View File
@@ -1,5 +1,18 @@
import { env } from "@/env";
/**
* 内网知识库「查看原文」详情页地址前缀。
* 素材的 recUuid 直接拼接在其后即为完整原文链接。
*/
export const INTRANET_SOURCE_URL_BASE =
"https://chlib.b.uat.4.cn/knowlibrary/#/knowledge/mid_detail?recUuid=";
/** 由 recUuid 构造内网原文详情页地址;recUuid 为空时返回空串。 */
export function buildIntranetSourceUrl(recUuid?: string | null): string {
if (!recUuid) return "";
return `${INTRANET_SOURCE_URL_BASE}${encodeURIComponent(recUuid)}`;
}
function getBaseOrigin() {
if (typeof window !== "undefined") {
return window.location.origin;
@@ -33,7 +33,7 @@ const SSE_EVENT_TYPES: ProgressEventType[] = [
'materials_ready', 'outline_chunk', 'outline_ready',
'draft_chunk', 'draft_ready',
'review_chunk', 'review_item_done', 'review_ready',
'await_user', 'resumed', 'done', 'error',
'await_user', 'resumed', 'notice', 'done', 'error',
];
// ── 后端返回 snake_case,前端类型是 camelCase,统一在此转换 ──────────────────
@@ -18,6 +18,7 @@ const DISPLAYED_TYPES = new Set([
'review_ready',
'await_user',
'resumed',
'notice',
'done',
'error',
]);
@@ -4,6 +4,7 @@ import {
ExternalLink, ListTree, FileText,
} from 'lucide-react';
import { cn } from '@/lib/utils';
import { buildIntranetSourceUrl } from '@/core/config';
import type { CompletedIntervention } from '../../types/ai-writing';
import { OutlineCard } from './OutlineCard';
import { ReviewCard } from './ReviewCard';
@@ -82,11 +83,12 @@ function MaterialCompleted({ record, expanded, onToggle }: { record: CompletedIn
</div>
<span className={cn('shrink-0 text-[10px]', barColor.replace('bg-', 'text-'))}>{pct}%</span>
</div>
{m.url && (
<a href={m.url} target="_blank" rel="noopener noreferrer"
{m.recUuid && (
<button type="button"
onClick={() => window.open(buildIntranetSourceUrl(m.recUuid), '_blank', 'noopener,noreferrer')}
className="shrink-0 text-primary hover:underline inline-flex items-center gap-0.5">
原文 <ExternalLink className="size-2.5" />
</a>
</button>
)}
</div>
</div>
@@ -1,6 +1,7 @@
import { useState } from 'react';
import { ChevronRight, ChevronDown, ExternalLink } from 'lucide-react';
import { cn } from '@/lib/utils';
import { buildIntranetSourceUrl } from '@/core/config';
import type { AIMaterial } from '../../types/ai-writing';
interface MaterialItemProps {
@@ -88,16 +89,17 @@ export function MaterialItem({ material, checked, onToggle }: MaterialItemProps)
{material.content.slice(0, 300)}
{material.content.length > 300 && '…'}
</p>
{material.url && (
<a
href={material.url}
target="_blank"
rel="noopener noreferrer"
{material.recUuid && (
<button
type="button"
onClick={() =>
window.open(buildIntranetSourceUrl(material.recUuid), '_blank', 'noopener,noreferrer')
}
className="inline-flex items-center gap-1 text-primary hover:underline"
>
查看原文
<ExternalLink className="size-2.5" />
</a>
</button>
)}
</div>
)}
@@ -1,8 +1,8 @@
import { CheckCircle2, Circle, Loader2, PauseCircle } from 'lucide-react';
import { AlertCircle, AlertTriangle, CheckCircle2, Circle, Loader2, PauseCircle } from 'lucide-react';
import { cn } from '@/lib/utils';
import type { ProgressEvent } from '../../types/ai-writing';
type StepStatus = 'done' | 'running' | 'paused' | 'waiting';
type StepStatus = 'done' | 'running' | 'paused' | 'waiting' | 'error' | 'warning';
interface TimelineStepProps {
event: ProgressEvent;
@@ -11,6 +11,8 @@ interface TimelineStepProps {
}
function statusOf(event: ProgressEvent, isLast: boolean): StepStatus {
if (event.type === 'error') return 'error';
if (event.type === 'notice') return 'warning';
if (event.type === 'await_user') return 'paused';
// step_start 只有在列表末尾(后续事件还未到达)时才显示转圈,否则已有后续事件说明该步骤已完成
if (event.type === 'step_start') return isLast ? 'running' : 'done';
@@ -26,6 +28,9 @@ const AGENT_COLOR: Record<string, string> = {
export function TimelineStep({ event, isLast, children }: TimelineStepProps) {
const status = statusOf(event, isLast ?? true);
// notice=非致命告警(如 ES 波动、模型兜底),error=致命错误,两者都展示「标题 + 真实原因」
const isAlert = event.type === 'error' || event.type === 'notice';
const isError = event.type === 'error';
return (
<div className="flex gap-3">
@@ -37,19 +42,46 @@ export function TimelineStep({ event, isLast, children }: TimelineStepProps) {
{/* 内容 */}
<div className={cn('pb-4 flex-1 min-w-0', isLast && 'pb-0')}>
<div className="flex items-center gap-1.5 mb-0.5">
{event.agentName && (
<span
{event.agentName && (
<span
className={cn(
'text-xs font-semibold',
AGENT_COLOR[event.agentName] ?? 'text-muted-foreground',
)}
>
{event.agentName}
</span>
)}
{isAlert ? (
<div
className={cn(
'mt-1 rounded-md border px-2.5 py-1.5',
isError
? 'border-red-200 bg-red-50/60 dark:border-red-900/40 dark:bg-red-950/20'
: 'border-amber-200 bg-amber-50/60 dark:border-amber-900/40 dark:bg-amber-950/20',
)}
>
<p
className={cn(
'text-xs font-semibold',
AGENT_COLOR[event.agentName] ?? 'text-muted-foreground',
'text-xs font-medium',
isError
? 'text-red-600 dark:text-red-400'
: 'text-amber-600 dark:text-amber-500',
)}
>
{event.agentName}
</span>
)}
<span className="text-xs text-muted-foreground">{event.message}</span>
</div>
{event.title || event.message || (isError ? '出错了' : '提示')}
</p>
{event.detail && (
<p className="mt-0.5 text-[11px] leading-relaxed text-muted-foreground break-words">
{event.detail}
</p>
)}
</div>
) : (
<span className="ml-1.5 text-xs text-muted-foreground">{event.message}</span>
)}
{children}
</div>
</div>
@@ -64,6 +96,10 @@ function StatusIcon({ status }: { status: StepStatus }) {
return <Loader2 className="size-4 text-blue-500 shrink-0 mt-0.5 animate-spin" />;
case 'paused':
return <PauseCircle className="size-4 text-yellow-500 shrink-0 mt-0.5" />;
case 'error':
return <AlertCircle className="size-4 text-red-500 shrink-0 mt-0.5" />;
case 'warning':
return <AlertTriangle className="size-4 text-amber-500 shrink-0 mt-0.5" />;
default:
return <Circle className="size-4 text-muted-foreground/40 shrink-0 mt-0.5" />;
}
@@ -223,8 +223,17 @@ export function useAIWriting() {
case 'done':
return { ...base, status: 'done', currentPause: null };
case 'error':
return { ...base, status: 'error', error: event.message ?? '未知错误' };
case 'error': {
// error 事件携带 title(如「模型错误」)+ detail(真实原因),拼成底部状态栏文案
const msg = event.detail
? `${event.title ?? '出错了'}:${event.detail}`
: (event.message ?? '未知错误');
return { ...base, status: 'error', error: msg };
}
// notice:非致命告警(ES 波动 / 模型兜底),仅追加到时间线,不改变写作状态
case 'notice':
return base;
default:
return base;
@@ -78,6 +78,7 @@ export type ProgressEventType =
| 'review_ready'
| 'await_user'
| 'resumed'
| 'notice'
| 'done'
| 'error';
@@ -86,6 +87,9 @@ export interface ProgressEvent {
sessionId: string;
agentName?: string;
message: string;
// notice(非致命告警)/ error:标题 + 真实原因
title?: string;
detail?: string;
// materials_ready
materials?: AIMaterial[];
keywords?: string[];
@@ -149,12 +149,25 @@ def _sse(event: str, data: dict) -> str:
return f"event: {event}\ndata: {json.dumps(data, ensure_ascii=False)}\n\n"
def _ai_writing_config() -> dict:
"""读取 config.yaml 的 ai_writing 段(含素材检索 provider 配置)。"""
try:
from deerflow.config import get_app_config
aw = getattr(get_app_config(), "ai_writing", None)
if isinstance(aw, dict):
return aw
return dict(aw) if aw else {}
except Exception:
return {}
def _make_config(session_id: str, model_name: str = "", valid_article_types: list[dict] | None = None) -> dict:
return {
"configurable": {
"thread_id": session_id,
"model_name": model_name or _session_model_names.get(session_id, ""),
"ai_writing_config": {},
"ai_writing_config": _ai_writing_config(),
# 把 SSE 队列注入,供草稿/大纲节点直接推流式 chunk
"sse_queue": _event_queues.get(session_id),
"sse_session_id": session_id,
@@ -91,6 +91,9 @@ async def editor_node(state: AIWritingState, config: RunnableConfig) -> dict:
model = get_model(config)
results: dict[str, dict] = {}
review_errors: list[str] = [] # 分项审查的模型异常,供完成后非致命提示
_CATEGORY_LABEL = {"fact": "事实性", "logic": "逻辑性", "language": "语言文字"}
for category, user_prompt in REVIEW_TASKS:
fallback = {"score": 80, "issues": []}
@@ -100,6 +103,7 @@ async def editor_node(state: AIWritingState, config: RunnableConfig) -> dict:
except Exception as e:
logger.warning("editor review [%s] failed: %s", category, e)
res = fallback
review_errors.append(f"「{_CATEGORY_LABEL.get(category, category)}审查」{type(e).__name__}: {e}")
results[category] = res
@@ -140,10 +144,21 @@ async def editor_node(state: AIWritingState, config: RunnableConfig) -> dict:
revision_notes=all_issues,
)
# 分项审查出现模型异常时,以非致命 notice 提示用户,审核结论仍照常产出
notices: list[dict] = []
if review_errors:
notices.append({
"type": "notice",
"agent_name": "编辑",
"title": "模型错误",
"detail": "部分审查项调用失败,已按默认分兜底。原因:" + ";".join(review_errors[:3]),
})
return {
"review_result": review,
"progress_events": [
{"type": "step_start", "agent_name": "编辑", "message": "开始审核"},
*notices,
{
"type": "review_ready",
"agent_name": "编辑",
@@ -21,6 +21,11 @@ logger = logging.getLogger(__name__)
_MAX_MATERIALS = 12
def _notice(title: str, detail: str) -> dict:
"""构造一个非致命告警事件,前端在时间线显示「标题 + 真实原因」,不打断写作流程。"""
return {"type": "notice", "agent_name": "素材收集专家", "title": title, "detail": detail}
async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict:
intent = state["user_intent"]
article_type = state.get("article_type", "analysis")
@@ -93,9 +98,11 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
seen_urls.add(m.get("url", ""))
raw_materials.append(m)
search_errors: list[str] = []
for results in all_results_nested:
if isinstance(results, Exception):
logger.warning("search failed: %s", results)
search_errors.append(f"{type(results).__name__}: {results}")
continue
for r in results:
if r.url and r.url in seen_urls:
@@ -113,8 +120,12 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
recUuid=r.recUuid,
))
# provider 内部吞掉的检索异常(如内网 ES 连接失败),一并取出用于提示用户
search_errors.extend(getattr(provider, "errors", []))
# Step 3: 相关度排序(如素材较多,让 LLM 排序)
await _emit("检索完成,正在筛选排序素材…")
rank_error = ""
if len(raw_materials) > _MAX_MATERIALS:
materials_list = "\n".join(
f"- ID: {m['id']} 标题: {m['title']} 摘要: {m['content'][:100]}"
@@ -126,6 +137,7 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
fallback={"ranked_ids": [m["id"] for m in raw_materials[:_MAX_MATERIALS]], "summary": f"共搜索到{len(raw_materials)}条素材"},
config=config,
)
rank_error = rank_result.get("_llm_error", "")
ranked_ids: list[str] = rank_result.get("ranked_ids", [m["id"] for m in raw_materials])
id_to_mat = {m["id"]: m for m in raw_materials}
sorted_materials = [id_to_mat[mid] for mid in ranked_ids if mid in id_to_mat]
@@ -146,11 +158,22 @@ async def researcher_node(state: AIWritingState, config: RunnableConfig) -> dict
pkg = MaterialPackage(keywords=keywords, materials=materials, summary=summary)
# 容错告警:检索接口 / 模型调用出错时以非致命 notice 提示用户,不打断写作流程
notices: list[dict] = []
if kw_result.get("_llm_error"):
notices.append(_notice("模型错误", f"检索词生成失败,已用默认关键词兜底。原因:{kw_result['_llm_error']}"))
if rank_error:
notices.append(_notice("模型错误", f"素材排序失败,已按相关度兜底排序。原因:{rank_error}"))
if search_errors:
uniq = list(dict.fromkeys(search_errors))
notices.append(_notice("ES 波动导致检索异常", ";".join(uniq[:3])))
return {
"material_package": pkg,
"status": "awaiting_material_confirm",
"progress_events": [
{"type": "step_start", "agent_name": "素材收集专家", "message": f"搜索关键词:{', '.join(keywords[:3])}"},
*notices,
{"type": "materials_ready", "agent_name": "素材收集专家", "message": summary},
],
}
@@ -76,6 +76,7 @@ async def writer_draft_node(state: AIWritingState, config: RunnableConfig) -> di
model = get_model(config)
section_titles: list[str] = []
section_bufs: list[str] = [] # 各章节正文(不含 ## 标题),供扩写兜底逐节替换
section_errors: list[str] = [] # 分节写作的模型异常,供完成后非致命提示
for i, section in enumerate(sections):
sec_title = _section_title(i, section)
@@ -141,6 +142,7 @@ async def writer_draft_node(state: AIWritingState, config: RunnableConfig) -> di
logger.error("draft section [%s] failed: %s", sec_title, e, exc_info=True)
section_titles.append(sec_title)
section_bufs.append("(内容生成失败,请重试)")
section_errors.append(f"「{sec_title}」{type(e).__name__}: {e}")
def _assemble() -> str:
body = "\n\n".join(f"## {t}\n\n{b}" for t, b in zip(section_titles, section_bufs))
@@ -191,12 +193,23 @@ async def writer_draft_node(state: AIWritingState, config: RunnableConfig) -> di
revision_count=revision_count,
)
# 分节写作出现模型异常时,以非致命 notice 提示用户,草稿仍照常产出
notices: list[dict] = []
if section_errors:
notices.append({
"type": "notice",
"agent_name": "作家",
"title": "模型错误",
"detail": "部分章节生成失败,对应段落已占位。原因:" + ";".join(section_errors[:3]),
})
return {
"current_draft": draft,
"drafts": [draft],
"status": "awaiting_draft_confirm",
"progress_events": [
{"type": "step_start", "agent_name": "作家", "message": f"开始写作(第{revision_count+1}稿)"},
*notices,
{
"type": "draft_ready",
"agent_name": "作家",
@@ -87,7 +87,13 @@ async def writer_outline_node(state: AIWritingState, config: RunnableConfig) ->
logger.error("outline streaming failed: %s", e, exc_info=True)
return {
"status": "error",
"progress_events": [{"type": "error", "agent_name": "作家", "message": f"大纲生成失败:{e}"}],
"progress_events": [{
"type": "error",
"agent_name": "作家",
"title": "模型错误",
"detail": str(e),
"message": "大纲生成失败",
}],
}
fallback = {
@@ -10,11 +10,8 @@ def get_search_provider(ai_writing_config: dict) -> BaseSearchProvider:
config.yaml 中配置:
ai_writing:
search_provider: internet # internet(默认,DDG) / intranet / tavily / demo
intranet_search_url: http://...
intranet_search_api_key: ""
intranet_field_map:
content: "abstract"
source: "system_name"
intranet_search_url: http://10.106.57.180:7863/knowledge_base/search_docs_es
intranet_knowledge_base: samples
ddg_region: "cn-zh" # DuckDuckGo 区域,默认 cn-zh
internet 模式:DuckDuckGo 为主搜索(免 API Key),失败时 fallback 到 demo 演示素材。
@@ -25,8 +22,7 @@ def get_search_provider(ai_writing_config: dict) -> BaseSearchProvider:
if provider == "intranet":
return IntranetSearchProvider(
base_url=cfg.get("intranet_search_url", ""),
api_key=cfg.get("intranet_search_api_key", ""),
field_map=cfg.get("intranet_field_map", {}),
knowledge_base_name=cfg.get("intranet_knowledge_base", "samples"),
)
if provider == "tavily":
return _FallbackSearchProvider(InternetSearchProvider(), DemoSearchProvider())
@@ -45,6 +41,11 @@ class _FallbackSearchProvider(BaseSearchProvider):
self._primary = primary
self._fallback = fallback
@property
def errors(self) -> list[str]:
"""聚合主/兜底 provider 的检索异常,供 researcher 节点取用。"""
return list(getattr(self._primary, "errors", [])) + list(getattr(self._fallback, "errors", []))
async def search(self, query: str, max_results: int = 5, **kwargs):
results = await self._primary.search(query, max_results=max_results, **kwargs)
if not results:
@@ -16,6 +16,8 @@ class DuckDuckGoSearchProvider(BaseSearchProvider):
def __init__(self, region: str = "cn-zh", max_timeout: int = 30):
self.region = region
self.max_timeout = max_timeout
# 收集本次运行的检索异常,供 researcher 节点取出并提示用户
self.errors: list[str] = []
async def search(
self,
@@ -31,6 +33,7 @@ class DuckDuckGoSearchProvider(BaseSearchProvider):
return results
except Exception as e:
logger.warning("DuckDuckGo search failed for query=%r: %s", query, e)
self.errors.append(f"{type(e).__name__}: {e}")
return []
def _sync_search(self, query: str, max_results: int) -> List[SearchResult]:
@@ -38,6 +41,7 @@ class DuckDuckGoSearchProvider(BaseSearchProvider):
from ddgs import DDGS
except ImportError:
logger.error("ddgs not installed. Run: uv add ddgs")
self.errors.append("ImportError: ddgs 未安装")
return []
ddgs = DDGS(timeout=self.max_timeout)
@@ -51,6 +55,7 @@ class DuckDuckGoSearchProvider(BaseSearchProvider):
items = list(raw) if raw else []
except Exception as e:
logger.warning("DDGS.text failed: %s", e)
self.errors.append(f"{type(e).__name__}: {e}")
return []
results = []
@@ -1,57 +1,83 @@
"""内网知识库搜索适配器 —— 对接 ES 检索接口 search_docs_es。"""
from __future__ import annotations
import logging
from typing import List
from .base import BaseSearchProvider, SearchResult
logger = logging.getLogger(__name__)
# 内网知识库默认检索接口与知识库名(可经 config.yaml 的 ai_writing 段覆盖)
_DEFAULT_ENDPOINT = "http://10.106.57.180:7863/knowledge_base/search_docs_es"
_DEFAULT_KB = "samples"
class IntranetSearchProvider(BaseSearchProvider):
"""内网搜索接口适配器。
"""内网知识库搜索接口适配器。
内网系统需提供 HTTP 接口,推荐格式:
POST /api/search
请求体:{"query": "...", "page_size": 5}
响应体:{"total": 100, "results": [...]}
对接步骤:
1. 取消下方注释代码,调整字段映射
2. 在 config.yaml 的 intranet_field_map 中配置字段映射
3. 将 search_provider 改为 intranet
接口:POST {endpoint}
请求体:{"query": "...", "knowledge_base_name": "samples", "ts": "yes"}
响应体:[{"page_content": "...", "metadata": {m_title, content, recUuid, m_publish, source1}}]
"""
def __init__(self, base_url: str, api_key: str = "", timeout: int = 10, field_map: dict = None):
self.base_url = base_url
self.api_key = api_key
def __init__(
self,
base_url: str = "",
knowledge_base_name: str = _DEFAULT_KB,
timeout: int = 15,
**_ignored,
):
self.endpoint = base_url or _DEFAULT_ENDPOINT
self.knowledge_base_name = knowledge_base_name or _DEFAULT_KB
self.timeout = timeout
self.field_map = field_map or {}
def _map(self, item: dict, field: str) -> str:
key = self.field_map.get(field, field)
return str(item.get(key, item.get(field, "")))
# 收集本次运行的检索异常,供 researcher 节点取出并提示用户(ES 波动等)
self.errors: list[str] = []
async def search(self, query: str, max_results: int = 5, **kwargs) -> List[SearchResult]:
# ── 对接内网时,取消以下注释 ──────────────────────────────────────
# import httpx
# headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {}
# async with httpx.AsyncClient(timeout=self.timeout) as client:
# resp = await client.post(
# f"{self.base_url}/api/search",
# json={"query": query, "page_size": max_results},
# headers=headers,
# )
# resp.raise_for_status()
# items = resp.json().get("results", [])
# return [
# SearchResult(
# title=self._map(it, "title"),
# content=self._map(it, "content")[:500],
# source=self._map(it, "source"),
# published_at=self._map(it, "published_at"),
# relevance_score=float(it.get(self.field_map.get("relevance_score", "relevance_score"), 0)),
# recUuid=self._map(it, "recUuid"),
# source_type="intranet",
# )
# for it in items
# ]
return []
import httpx
payload = {
"query": query,
"knowledge_base_name": self.knowledge_base_name,
"ts": "yes",
}
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
resp = await client.post(self.endpoint, json=payload)
resp.raise_for_status()
data = resp.json()
except Exception as e:
logger.warning("intranet search failed for query=%r: %s", query, e)
self.errors.append(f"{type(e).__name__}: {e}")
return []
# 接口直接返回数组;兼容个别情况下包了一层 data/results 的情况
if isinstance(data, dict):
items = data.get("data") or data.get("results") or []
else:
items = data or []
results: List[SearchResult] = []
for i, item in enumerate(items[:max_results]):
md = (item or {}).get("metadata") or {}
content = md.get("content") or (item or {}).get("page_content") or ""
results.append(SearchResult(
title=str(md.get("m_title") or "").strip() or "(无标题)",
content=str(content)[:500],
source=str(md.get("source1") or "").strip() or "内网知识库",
published_at=str(md.get("m_publish") or "").strip(),
url="", # 内网素材的原文地址由前端依据 recUuid 拼接
# 接口未返回相关度分值,按 ES 返回顺序赋递减分以保留排序
relevance_score=max(0.5, 0.95 - i * 0.03),
source_type="intranet",
recUuid=str(md.get("recUuid") or "").strip(),
))
return results
async def health_check(self) -> bool:
return False
try:
await self.search("测试", max_results=1)
return True
except Exception:
return False