识图与文件上传/预览:可配置视觉模型 + 修复上传与回显
后端: - 新增可配置识图模型(config: vision.model_name/prompt/max_tokens)。配置后 view_image 委托该模型识别并返回文字结果,文本主模型也能“看图”;未配置时回退到主模型自带视觉能力。新增 deerflow/models/vision.py 与 VisionConfig。 - 修复 Windows 上传全部报 unsafe:open_upload_file_no_symlink 不再强制要求 POSIX 专有的 O_NOFOLLOW,缺失时降级(仍用 lstat/fstat 防符号链接),Windows 追加 O_BINARY。 - 修复 normalize_input 丢弃 human 消息 additional_kwargs:保留 files/client_ts 等,使上传文件能回传到 UI 并被 UploadsMiddleware 识别。 - 约束 agent 行为:上传图片时引导只用 view_image,禁止用 read_file/grep/bash/python 解析图片字节;view_image 工具说明同步加固。 - 启用文档转换(uploads.auto_convert_documents),PDF/Word/Excel/PPT 自动转 Markdown 供 read_file/grep 读取。 - 订正若干模型错误的 supports_vision 标记。 前端: - 修复上传文件不回显:parseUploadedFiles 不再因 "(empty)" 提前返回。 - 修复 artifact 401 破图:改用带 Authorization 头的 apiFetch 取 blob 显示(新增 useAuthedObjectUrl)。 - 新增上传文件预览弹窗:点击卡片可查看图片/文本/Markdown、Office(读转换后的 .md)、PDF;文件卡片改为可点击。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
43cbcd86be
commit
5aa1fdca04
17 files changed
+706
-74
No files matched your search
@@ -32,6 +32,7 @@ from deerflow.config.title_config import TitleConfig, load_title_config_from_dic
|
||||
from deerflow.config.token_usage_config import TokenUsageConfig
|
||||
from deerflow.config.tool_config import ToolConfig, ToolGroupConfig
|
||||
from deerflow.config.tool_search_config import ToolSearchConfig, load_tool_search_config_from_dict
|
||||
from deerflow.config.vision_config import VisionConfig
|
||||
|
||||
load_dotenv()
|
||||
|
||||
@@ -104,6 +105,7 @@ class AppConfig(BaseModel):
|
||||
log_level: str = Field(default="info", description="Logging level for deerflow and app modules (debug/info/warning/error); third-party libraries are not affected")
|
||||
token_usage: TokenUsageConfig = Field(default_factory=TokenUsageConfig, description="Token usage tracking configuration")
|
||||
models: list[ModelConfig] = Field(default_factory=list, description="Available models")
|
||||
vision: VisionConfig = Field(default_factory=VisionConfig, description="Image recognition (识图) configuration")
|
||||
sandbox: SandboxConfig = Field(description="Sandbox configuration")
|
||||
tools: list[ToolConfig] = Field(default_factory=list, description="Available tools")
|
||||
tool_groups: list[ToolGroupConfig] = Field(default_factory=list, description="Available tool groups")
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
# Default instruction sent to the vision model when the agent does not provide a
|
||||
# more specific question. Kept in Chinese to match the product's primary locale.
|
||||
DEFAULT_VISION_PROMPT = "请仔细观察这张图片,尽可能详细、准确地描述其中的全部内容,包括文字、图表、物体、场景与布局等关键信息。"
|
||||
|
||||
|
||||
class VisionConfig(BaseModel):
|
||||
"""Configuration for image recognition (识图).
|
||||
|
||||
When ``model_name`` names a model from ``models[]``, the lead agent's
|
||||
``view_image`` tool delegates image understanding to that dedicated model
|
||||
and returns a text description — so a text-only chat model can still "see"
|
||||
uploaded images. When ``model_name`` is unset, vision falls back to the main
|
||||
chat model, which only works when that model has ``supports_vision: true``.
|
||||
"""
|
||||
|
||||
model_name: str | None = Field(
|
||||
default=None,
|
||||
description="Name of the model (from models[]) used for image recognition. When unset, falls back to the main chat model if it supports vision.",
|
||||
)
|
||||
prompt: str = Field(
|
||||
default=DEFAULT_VISION_PROMPT,
|
||||
description="Default instruction for the vision model when no per-call question is given.",
|
||||
)
|
||||
max_tokens: int | None = Field(
|
||||
default=None,
|
||||
description="Optional override for the vision model's max output tokens; falls back to the model's own config when unset.",
|
||||
)
|
||||
Reference in new issue
Block a user