feat: 浏览器扩展支持交互式页面操作 (browser_act)

在原有只读抓取 (browser_fetch_page) 基础上,新增 observe→act 循环,
让 Agent 能在用户的 Chrome 中实际操作网页:点击/输入/滚动/等待/页内跳转。

后端:
- 新工具 browser_act,按 session 复用标签页,返回带编号的可交互元素清单+正文
- browser_context 任务队列扩展 kind/session_id/action 与 observation 结果
- /api/browser-context 路由下发 act 任务、接收 observation
- BrowserContextConfig 增加 actions_enabled 等开关,tools.py 两级门控
- 接线 app.py 路由挂载、app_config.py 配置字段、config.yaml 工具注册

前端:
- ChatPage 处理「发送当前页」回跳的 browser_context query 参数并自动提交

扩展 (browser-extension/deer-web-clipper):
- shared.js 自包含注入函数 deerObservePage / deerPerformAction
- background.js 按 kind 分发任务,session→tab 映射存 chrome.storage.session

测试:tests/test_browser_act.py(队列往返 + 工具校验 + 模拟扩展成功路径,8 passed)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
AdministratorandClaude Opus 4.8 committed 2026-06-08 13:47:00 +08:00
1 parent 0001f86826
commit d4a71ce384
24 files changed
+4499 -162

No files matched your search

@@ -29,9 +29,10 @@ from app.gateway.routers import (
ai_writing,
admin_users,
agents,
artifacts,
assistants_compat,
auth,
artifacts,
assistants_compat,
auth,
browser_context,
channels,
checkpoint_migration,
feedback,
@@ -1197,11 +1198,14 @@ This gateway provides custom endpoints for models, MCP configuration, skills, an
# Artifacts API is mounted at /api/threads/{thread_id}/artifacts
app.include_router(artifacts.router)
# Uploads API is mounted at /api/threads/{thread_id}/uploads
app.include_router(uploads.router)
# Thread cleanup API is mounted at /api/threads/{thread_id}
app.include_router(threads.router)
# Uploads API is mounted at /api/threads/{thread_id}/uploads
app.include_router(uploads.router)
# Optional Chrome-extension page capture API is mounted at /api/browser-context
app.include_router(browser_context.router)
# Thread cleanup API is mounted at /api/threads/{thread_id}
app.include_router(threads.router)
# Agents API is mounted at /api/agents
app.include_router(agents.router)