feat: 浏览器扩展支持交互式页面操作 (browser_act)

在原有只读抓取 (browser_fetch_page) 基础上,新增 observe→act 循环,
让 Agent 能在用户的 Chrome 中实际操作网页:点击/输入/滚动/等待/页内跳转。

后端:
- 新工具 browser_act,按 session 复用标签页,返回带编号的可交互元素清单+正文
- browser_context 任务队列扩展 kind/session_id/action 与 observation 结果
- /api/browser-context 路由下发 act 任务、接收 observation
- BrowserContextConfig 增加 actions_enabled 等开关,tools.py 两级门控
- 接线 app.py 路由挂载、app_config.py 配置字段、config.yaml 工具注册

前端:
- ChatPage 处理「发送当前页」回跳的 browser_context query 参数并自动提交

扩展 (browser-extension/deer-web-clipper):
- shared.js 自包含注入函数 deerObservePage / deerPerformAction
- background.js 按 kind 分发任务,session→tab 映射存 chrome.storage.session

测试:tests/test_browser_act.py(队列往返 + 工具校验 + 模拟扩展成功路径,8 passed)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
AdministratorandClaude Opus 4.8 committed 2026-06-08 13:47:00 +08:00
1 parent 0001f86826
commit d4a71ce384
24 files changed
+4499 -162

No files matched your search

+69 -69
View File
@@ -1,76 +1,76 @@
config_version: 8
log_level: info
models:
- name: deepseek-chat
display_name: DeepSeek Chat
# - name: deepseek-chat
# display_name: DeepSeek Chat
# use: langchain_openai:ChatOpenAI
# model: deepseek-chat
# api_key: sk-9808fb2375534112ac275541aec14b9b
# base_url: https://api.deepseek.com
# request_timeout: 600.0
# max_retries: 2
# max_tokens: 4096
# temperature: 1.0
# supports_vision: false
# supports_thinking: true
# - name: zai-org/GLM-5-FP8
# display_name: zai-org/GLM-5-FP8
# use: langchain_openai:ChatOpenAI
# model: zai-org/GLM-5-FP8
# api_key: modalresearch_gWu46FDpVLYZWPVEA3b_2fq8Jw2UJiKMgEm_Rxmn1SA
# base_url: https://api.us-west-2.modal.direct/v1
# request_timeout: 600.0
# max_retries: 2
# max_tokens: 4096
# temperature: 1.0
# supports_vision: true
# supports_thinking: true
# - name: deepseek-v4-pro
# display_name: DeepSeek V4 Pro
# use: langchain_openai:ChatOpenAI
# model: deepseek-v4-pro
# api_key: sk-e1c22b7931dc446f8b67e57878edd105
# base_url: https://api.deepseek.com
# request_timeout: 600.0
# max_retries: 2
# max_tokens: 4096
# temperature: 1.0
# supports_vision: false
# supports_thinking: true
# - name: minimax-m2.7
# display_name: MiniMax M2.7
# use: langchain_openai:ChatOpenAI
# model: MiniMax-M2.7
# api_key: sk-cp-mrsAGtzDpUl0TpOBPVoiRG_pWF1kDDN9_mDAnONBs6HIPcNUexSKtSbx2QOxfKBvJT6Xw6hDTV6hdkJYFWfNA8UtHQ08TiGJiosdRoW6q1sGb6Cb7D85NHk
# base_url: https://api.minimaxi.com
# max_tokens: 4096
# temperature: 1.0
# supports_vision: true
# supports_thinking: true
# - name: minimax-m2.5
# display_name: MiniMax M2.5
# use: langchain_openai:ChatOpenAI
# model: MiniMax-M2.5
# api_key: sk-cp-mrsAGtzDpUl0TpOBPVoiRG_pWF1kDDN9_mDAnONBs6HIPcNUexSKtSbx2QOxfKBvJT6Xw6hDTV6hdkJYFWfNA8UtHQ08TiGJiosdRoW6q1sGb6Cb7D85NHk
# base_url: https://api.minimaxi.com
# max_tokens: 4096
# temperature: 1.0
# supports_vision: true
# supports_thinking: true
# - name: deepseek-ai/DeepSeek-OCR
# display_name: deepseek-ai/DeepSeek-OCR
# use: langchain_openai:ChatOpenAI
# model: deepseek-ai/DeepSeek-OCR
# api_key: sk-beshfxfxfrllsczpqfrgjbugfbpzxgqgedbehceytpcvvtyb
# base_url: https://api.siliconflow.cn/v1/
# max_tokens: 4096
# temperature: 1.0
# supports_vision: true
# supports_thinking: true
- name: deepseek-ai/DeepSeek-V4-Flash
display_name: deepseek-ai/DeepSeek-V4-Flash
use: langchain_openai:ChatOpenAI
model: deepseek-chat
api_key: sk-9808fb2375534112ac275541aec14b9b
base_url: https://api.deepseek.com
request_timeout: 600.0
max_retries: 2
max_tokens: 4096
temperature: 1.0
supports_vision: false
supports_thinking: true
- name: zai-org/GLM-5-FP8
display_name: zai-org/GLM-5-FP8
use: langchain_openai:ChatOpenAI
model: zai-org/GLM-5-FP8
api_key: modalresearch_gWu46FDpVLYZWPVEA3b_2fq8Jw2UJiKMgEm_Rxmn1SA
base_url: https://api.us-west-2.modal.direct/v1
request_timeout: 600.0
max_retries: 2
max_tokens: 4096
temperature: 1.0
supports_vision: true
supports_thinking: true
- name: deepseek-v4-pro
display_name: DeepSeek V4 Pro
use: langchain_openai:ChatOpenAI
model: deepseek-v4-pro
api_key: sk-e1c22b7931dc446f8b67e57878edd105
base_url: https://api.deepseek.com
request_timeout: 600.0
max_retries: 2
max_tokens: 4096
temperature: 1.0
supports_vision: false
supports_thinking: true
- name: minimax-m2.7
display_name: MiniMax M2.7
use: langchain_openai:ChatOpenAI
model: MiniMax-M2.7
api_key: sk-cp-mrsAGtzDpUl0TpOBPVoiRG_pWF1kDDN9_mDAnONBs6HIPcNUexSKtSbx2QOxfKBvJT6Xw6hDTV6hdkJYFWfNA8UtHQ08TiGJiosdRoW6q1sGb6Cb7D85NHk
base_url: https://api.minimaxi.com
max_tokens: 4096
temperature: 1.0
supports_vision: true
supports_thinking: true
- name: minimax-m2.5
display_name: MiniMax M2.5
use: langchain_openai:ChatOpenAI
model: MiniMax-M2.5
api_key: sk-cp-mrsAGtzDpUl0TpOBPVoiRG_pWF1kDDN9_mDAnONBs6HIPcNUexSKtSbx2QOxfKBvJT6Xw6hDTV6hdkJYFWfNA8UtHQ08TiGJiosdRoW6q1sGb6Cb7D85NHk
base_url: https://api.minimaxi.com
max_tokens: 4096
temperature: 1.0
supports_vision: true
supports_thinking: true
- name: deepseek-ai/DeepSeek-OCR
display_name: deepseek-ai/DeepSeek-OCR
use: langchain_openai:ChatOpenAI
model: deepseek-ai/DeepSeek-OCR
api_key: sk-beshfxfxfrllsczpqfrgjbugfbpzxgqgedbehceytpcvvtyb
base_url: https://api.siliconflow.cn/v1/
max_tokens: 4096
temperature: 1.0
supports_vision: true
supports_thinking: true
- name: Qwen/Qwen3.6-27B
display_name: Qwen/Qwen3.6-27B
use: langchain_openai:ChatOpenAI
model: Qwen/Qwen3.6-27B
model: deepseek-ai/DeepSeek-V4-Flash
api_key: sk-qwwwvjompnzxqhndctlwmmomvqdgaqqqgxvibfxchsikvgod
base_url: https://api.siliconflow.cn/v1/
max_tokens: 4096