feat: 浏览器扩展支持交互式页面操作 (browser_act)
在原有只读抓取 (browser_fetch_page) 基础上,新增 observe→act 循环, 让 Agent 能在用户的 Chrome 中实际操作网页:点击/输入/滚动/等待/页内跳转。 后端: - 新工具 browser_act,按 session 复用标签页,返回带编号的可交互元素清单+正文 - browser_context 任务队列扩展 kind/session_id/action 与 observation 结果 - /api/browser-context 路由下发 act 任务、接收 observation - BrowserContextConfig 增加 actions_enabled 等开关,tools.py 两级门控 - 接线 app.py 路由挂载、app_config.py 配置字段、config.yaml 工具注册 前端: - ChatPage 处理「发送当前页」回跳的 browser_context query 参数并自动提交 扩展 (browser-extension/deer-web-clipper): - shared.js 自包含注入函数 deerObservePage / deerPerformAction - background.js 按 kind 分发任务,session→tab 映射存 chrome.storage.session 测试:tests/test_browser_act.py(队列往返 + 工具校验 + 模拟扩展成功路径,8 passed) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
0001f86826
commit
d4a71ce384
24 files changed
+4499
-162
No files matched your search
@@ -11,6 +11,11 @@ from deerflow.tools.builtins.tool_search import reset_deferred_registry
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
BROWSER_CONTEXT_TOOL_USE = "deerflow.tools.builtins.browser_context_tool:browser_fetch_page_tool"
|
||||
BROWSER_ACT_TOOL_USE = "deerflow.tools.builtins.browser_act_tool:browser_act_tool"
|
||||
BROWSER_CONTEXT_TOOL_USES = {BROWSER_CONTEXT_TOOL_USE, BROWSER_ACT_TOOL_USE}
|
||||
BROWSER_CONTEXT_TOOL_NAMES = {"browser_fetch_page", "browser_act"}
|
||||
|
||||
BUILTIN_TOOLS = [
|
||||
present_file_tool,
|
||||
ask_clarification_tool,
|
||||
@@ -34,6 +39,14 @@ def _is_host_bash_tool(tool: object) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _is_browser_context_tool(tool: object) -> bool:
|
||||
"""Return True if the tool config represents the browser extension bridge."""
|
||||
|
||||
use = getattr(tool, "use", None)
|
||||
name = getattr(tool, "name", None)
|
||||
return use in BROWSER_CONTEXT_TOOL_USES or name in BROWSER_CONTEXT_TOOL_NAMES
|
||||
|
||||
|
||||
def get_available_tools(
|
||||
groups: list[str] | None = None,
|
||||
include_mcp: bool = True,
|
||||
@@ -64,6 +77,18 @@ def get_available_tools(
|
||||
config = app_config or get_app_config()
|
||||
tool_configs = [tool for tool in config.tools if groups is None or tool.group in groups]
|
||||
|
||||
browser_context_config = getattr(config, "browser_context", None)
|
||||
if not getattr(browser_context_config, "enabled", False):
|
||||
tool_configs = [tool for tool in tool_configs if not _is_browser_context_tool(tool)]
|
||||
elif not getattr(browser_context_config, "actions_enabled", True):
|
||||
# browser_context is on, but interactive page actions are disabled — keep
|
||||
# the read-only browser_fetch_page tool, drop only browser_act.
|
||||
tool_configs = [
|
||||
tool
|
||||
for tool in tool_configs
|
||||
if getattr(tool, "use", None) != BROWSER_ACT_TOOL_USE and getattr(tool, "name", None) != "browser_act"
|
||||
]
|
||||
|
||||
# Do not expose host bash by default when LocalSandboxProvider is active.
|
||||
if not is_host_bash_allowed(config):
|
||||
tool_configs = [tool for tool in tool_configs if not _is_host_bash_tool(tool)]
|
||||
|
||||
Reference in new issue
Block a user