识图与文件上传/预览:可配置视觉模型 + 修复上传与回显
后端: - 新增可配置识图模型(config: vision.model_name/prompt/max_tokens)。配置后 view_image 委托该模型识别并返回文字结果,文本主模型也能“看图”;未配置时回退到主模型自带视觉能力。新增 deerflow/models/vision.py 与 VisionConfig。 - 修复 Windows 上传全部报 unsafe:open_upload_file_no_symlink 不再强制要求 POSIX 专有的 O_NOFOLLOW,缺失时降级(仍用 lstat/fstat 防符号链接),Windows 追加 O_BINARY。 - 修复 normalize_input 丢弃 human 消息 additional_kwargs:保留 files/client_ts 等,使上传文件能回传到 UI 并被 UploadsMiddleware 识别。 - 约束 agent 行为:上传图片时引导只用 view_image,禁止用 read_file/grep/bash/python 解析图片字节;view_image 工具说明同步加固。 - 启用文档转换(uploads.auto_convert_documents),PDF/Word/Excel/PPT 自动转 Markdown 供 read_file/grep 读取。 - 订正若干模型错误的 supports_vision 标记。 前端: - 修复上传文件不回显:parseUploadedFiles 不再因 "(empty)" 提前返回。 - 修复 artifact 401 破图:改用带 Authorization 头的 apiFetch 取 blob 显示(新增 useAuthedObjectUrl)。 - 新增上传文件预览弹窗:点击卡片可查看图片/文本/Markdown、Office(读转换后的 .md)、PDF;文件卡片改为可点击。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
43cbcd86be
commit
5aa1fdca04
17 files changed
+706
-74
No files matched your search
@@ -84,13 +84,23 @@ def normalize_input(raw_input: dict[str, Any] | None) -> dict[str, Any]:
|
||||
converted = []
|
||||
for msg in messages:
|
||||
if isinstance(msg, dict):
|
||||
role = msg.get("role", msg.get("type", "user"))
|
||||
content = msg.get("content", "")
|
||||
if role in ("user", "human"):
|
||||
converted.append(HumanMessage(content=content))
|
||||
else:
|
||||
# TODO: handle other message types (system, ai, tool)
|
||||
converted.append(HumanMessage(content=content))
|
||||
# Preserve additional_kwargs (e.g. uploaded `files`, `client_ts`,
|
||||
# `prompt_prefix`) and optional id/name so they round-trip through
|
||||
# the checkpoint to the UI and to UploadsMiddleware. Dropping them
|
||||
# here previously made uploaded files invisible in the chat and
|
||||
# broke the per-message ask-time timestamp.
|
||||
msg_kwargs: dict[str, Any] = {"content": content}
|
||||
extra = msg.get("additional_kwargs")
|
||||
if isinstance(extra, dict) and extra:
|
||||
msg_kwargs["additional_kwargs"] = extra
|
||||
if msg.get("id"):
|
||||
msg_kwargs["id"] = msg["id"]
|
||||
if msg.get("name"):
|
||||
msg_kwargs["name"] = msg["name"]
|
||||
# TODO: handle other message types (system, ai, tool); for now all
|
||||
# inbound dict messages are treated as human turns.
|
||||
converted.append(HumanMessage(**msg_kwargs))
|
||||
else:
|
||||
converted.append(msg)
|
||||
return {**raw_input, "messages": converted}
|
||||
|
||||
Reference in new issue
Block a user