feat(workflow): 增加会话级多智能体任务规划运行能力
提交范围聚焦于工作流编排新增:新增 workflow_planning 路由与前后端运行入口(workflow_planning.py、workflow_proposal_planner.py、workflow_deep_research_adapter.py),支持基于会话的提案-规划-执行流程,满足任务驱动式多智能体协作。 完成网关层运行时适配:扩展 workflow_resource_checks、workflow_executor、workflow_runs、workflow_agent_runner 与依赖注入,新增/更新配置(config.yaml、workflow_config),并补充 lead_agent 路径与节点实现,确保 Planner 与普通工作流运行态衔接一致。 补齐持久化与建模:新增 workflow_planning 持久化模块与 MySQL migration(20260831_01_workflow_planning_sessions),注册 SQL 与模型、内存 store,并扩展错误/表达式/validator/节点语义以承载新会话态输入输出。 增强文档与测试:新增会话式多智能体编排中文说明、更新开发文档与 AGENTS 指南、补充 workflow_engine/workflow_runs 回归测试,提高新路径的行为可验证性;同时留存 seed 资产(workflow-planner)便于一体化部署。
This commit is contained in:
1 parent
ac92f29db1
commit
1706a24182
39 files changed
+4896
-81
No files matched your search
@@ -201,7 +201,7 @@ The backend is split into two layers with a strict dependency direction:
|
||||
|
||||
### Workflow Studio
|
||||
|
||||
Coze Studio is canvas-only; ChatDev is reference-only. Plan + status per phase: `docs/WORKFLOW_STUDIO_BACKEND_DEV_ZH.md`; decisions: `docs/adr/0001-workflow-studio-runtime.md`; contracts: `docs/WORKFLOW_SCHEMA_V1_ZH.md` + `docs/WORKFLOW_SSE_V1_ZH.md` + `docs/WORKFLOW_OPENAPI_DRAFT_V1.yaml`. Master switch + all limits: `config.yaml` → `workflows` (`deerflow.config.workflow_config.WorkflowConfig`).
|
||||
Coze Studio is canvas-only; ChatDev is reference-only. Plan + status per phase: `docs/WORKFLOW_STUDIO_BACKEND_DEV_ZH.md`; decisions: `docs/adr/0001-workflow-studio-runtime.md`; contracts: `docs/WORKFLOW_SCHEMA_V1_ZH.md` + `docs/WORKFLOW_SSE_V1_ZH.md` + `docs/WORKFLOW_OPENAPI_DRAFT_V1.yaml`. Master switch + all limits: `config.yaml` → `workflows` (`deerflow.config.workflow_config.WorkflowConfig`). `workflows.agent_recursion_limit` defaults to 250: LangGraph counts each model→tool research exchange as about two super-steps, so workflow agent/skill nodes must not fall back to a chat-sized 60-step limit. In the local Studio proxy, keep Rsbuild `server.compress: false`; dev gzip buffers small SSE frames and makes live planning/run output arrive only at completion despite the backend's `no-transform` / `X-Accel-Buffering: no` headers.
|
||||
|
||||
**Execution kernel is a custom DAG scheduler, not a compiled LangGraph.** LangGraph stays the kernel *inside* an `agent`/`skill` node. The reason is single-source-of-truth recovery: a re-leased or resumed run replays completed node outputs from `workflow_node_runs` plus the `workflow_run_events` log, so a second (checkpoint) copy of workflow-level state would have no defensible tie-breaker. See the ADR's phase-2 amendment.
|
||||
|
||||
@@ -209,15 +209,15 @@ Coze Studio is canvas-only; ChatDev is reference-only. Plan + status per phase:
|
||||
- `schemas` / `events` / `errors` / `ports` / `validator` / `samples` — frozen v1 contracts.
|
||||
- `expressions.py` — `{{ inputs.x }}` / `{{ nodes.n.data.y }}` templates and conditions resolved by path walking + a small comparator set. **No `eval`.** Roots are restricted to `inputs` / `nodes` / `run` / `loop` / `env`.
|
||||
- `runtime/engine.py` — the scheduler. Every edge is `pending` → `active` (source chose this port) or `pruned`; a node is ready when its non-back incoming edges are all resolved and at least one is `active`, and is skipped when all are `pruned` (skips propagate). That one rule covers fan-out, condition branching, and merge joins with no topological pre-pass. The `loop` node's `continue` back edge is the only permitted cycle; re-entry clears the body's results and invalidates its node runs.
|
||||
- `runtime/{sink,sql_runner,code_runner,context}.py`, `nodes/*` (13 executors), `security/{http_policy,sql_policy,secret_box}.py`.
|
||||
- `runtime/{sink,sql_runner,code_runner,context}.py`, `nodes/*` (15 executors, including deterministic `evidence_normalizer` and durable `deep_research_write`), `security/{http_policy,sql_policy,secret_box}.py`.
|
||||
|
||||
**Persistence**: `workflow_definitions` / `workflow_versions` (migration `20260828_01`), `workflow_runs` / `workflow_node_runs` / `workflow_run_events` / `workflow_run_artifacts` (`20260828_02`), `workflow_data_sources` (`20260828_03`). Stores under `deerflow.persistence.workflow_{runs,events,data_sources}` with SQL + in-memory implementations.
|
||||
**Persistence**: `workflow_definitions` / `workflow_versions` (migration `20260828_01`), `workflow_runs` / `workflow_node_runs` / `workflow_run_events` / `workflow_run_artifacts` (`20260828_02`), `workflow_data_sources` (`20260828_03`), and short-lived-but-durable `workflow_planning_sessions` / `workflow_planning_proposals` (`20260831_01`). Stores under `deerflow.persistence.workflow_{runs,events,data_sources,planning}` with SQL + in-memory implementations. Because planning proposal rows have a scalar FK rather than an ORM relationship, `SqlWorkflowPlanningStore.create_session` must flush the parent session before adding its children; otherwise SQLite can insert the child first and return a 500. A planning proposal is distinct from a run: its graph can be edited with optimistic `graph_revision` checks, but it becomes executable only once selected and atomically bound to one run.
|
||||
|
||||
**App layer**: `workflow_dispatcher.py` (lease scan → atomic claim; an expired lease is how a dead worker's run is reclaimed, so every worker can run the loop), `workflow_executor.py` (one lease's lifecycle: replay → engine under a run timeout with heartbeat + cancel watcher → exactly one terminal transition; also runs subworkflows inline, depth ≤ 3), `workflow_retention.py` (hourly sweep of `workflows.retention.event_retention_days` / `run_retention_days`), `workflow_live_hub.py` (in-process SSE fan-out), `workflow_agent_runner.py` (agent/skill nodes → lead agent).
|
||||
**App layer**: `workflow_dispatcher.py` (lease scan → atomic claim; an expired lease is how a dead worker's run is reclaimed, so every worker can run the loop), `workflow_executor.py` (one lease's lifecycle: replay → engine under a run timeout with heartbeat + cancel watcher → exactly one terminal transition; also runs subworkflows inline, depth ≤ 3), `workflow_retention.py` (hourly sweep of `workflows.retention.event_retention_days` / `run_retention_days`), `workflow_live_hub.py` (in-process SSE fan-out), `workflow_agent_runner.py` (agent/skill nodes → lead agent; `disable_tools=True` means an empty bound-tool set for control-plane callers), `workflow_deep_research_adapter.py` (Evidence Pack → selected Deep Research sources/job/event projection; no internal HTTP call), and `workflow_proposal_planner.py` (calls the dedicated `workflow-planner` controller over visible-agent metadata, then constructs safe candidate graphs server-side). `workflow-planner` is a separate Workflow Studio control-plane agent; never reuse or alias `roundtable-coordinator` for workflow planning, and never let it dispatch a run, call research tools, or return arbitrary graph JSON. It always runs with tools and model thinking disabled.
|
||||
|
||||
**Gateway routes** (register static prefixes before `workflows.router`'s `/{workflow_id}` catch-alls): `/api/workflows` CRUD/draft/validate/publish/copy, `/api/workflows/node-types` + `/resources/*`, `/api/workflows/{id}/runs` + `/api/workflows/runs/{run_id}{,/events,/stream,/cancel,/resume,/retry,/artifacts}`, `/api/workflows/data-sources` (+ `/{id}/introspect`, `/{id}/validate-query`), `/api/workflows/embed/{tickets,verify}`, `/api/workflow_api/*` Coze compat.
|
||||
**Gateway routes** (register static prefixes before `workflows.router`'s `/{workflow_id}` catch-alls): `/api/workflows` CRUD/draft/validate/publish/copy, `/api/workflows/node-types` + `/resources/*`, `/api/workflows/{id}/planning-sessions{/stream}` + `/api/workflows/planning-sessions/{session_id}{,/proposals/{proposal_id},/proposals/{proposal_id}/graph}`, `/api/workflows/{id}/runs` + `/api/workflows/runs/{run_id}{,/events,/stream,/cancel,/resume,/retry,/feedback,/artifacts}`, `/api/workflows/data-sources` (+ `/{id}/introspect`, `/{id}/validate-query`), `/api/workflows/embed/{tickets,verify}`, `/api/workflow_api/*` Coze compat. The planning `/stream` endpoint sends product-safe real controller milestones (`catalog`, `controller`, `selection`, `assembly`, `validation`, `complete`) then a final `result` session; never stream agent catalog descriptions or raw planner JSON. Formal runs accept the selected `planningSessionId` and `proposalId`; creating the same pair again returns the already-bound run rather than launching another execution.
|
||||
|
||||
**Guarantees worth not breaking**: events are persisted *then* published, and the DB is authoritative for `seq` ordering (live frames at or below the replay cursor are dropped and re-read on the next poll, so a reconnect sees no gap; `Last-Event-ID` wins over `?after=`); a run reaching a terminal status always gets a terminal event, synthesised if necessary, or SSE clients hang; `resumeToken` appears only in the `run.awaiting_input` event, never in the run view; `POST /retry` on a `failed` run copies completed node outputs onto a new run marked `retry_of_run_id` so the engine resumes from the failed node; data-source secrets are write-only (encrypted via `WORKFLOW_SECRET_KEY`, decrypted only inside the executor, reads expose `masked_target`); the `http` node denies private/loopback/link-local targets by default (SSRF), `sql_read` allows one read-only statement with bound parameters, and the `code` node is **off by default** because its runner is a hardened subprocess rather than a container.
|
||||
**Guarantees worth not breaking**: events are persisted *then* published, and the DB is authoritative for `seq` ordering (live frames at or below the replay cursor are dropped and re-read on the next poll, so a reconnect sees no gap; `Last-Event-ID` wins over `?after=`); a run reaching a terminal status always gets a terminal event, synthesised if necessary, or SSE clients hang; `resumeToken` appears only in the `run.awaiting_input` event, never in the run view; `POST /retry` on a `failed` run copies completed node outputs onto a new run marked `retry_of_run_id` so the engine resumes from the failed node; `POST /feedback` never mutates its source run—on a terminal acyclic run it may target only a completed `agent`/`skill`, reuses completed nodes outside that node's downstream closure, records those `reusedNodeIds` alongside the feedback summary, and puts the bounded feedback only in `RunContext.env.nodeFeedback[target]` before creating a revision run; data-source secrets are write-only (encrypted via `WORKFLOW_SECRET_KEY`, decrypted only inside the executor, reads expose `masked_target`); the `http` node denies private/loopback/link-local targets by default (SSRF), `sql_read` allows one read-only statement with bound parameters, and the `code` node is **off by default** because its runner is a hardened subprocess rather than a container.
|
||||
|
||||
HTTP interface resources persist a non-secret base_url plus an allowed_methods allowlist (migration
|
||||
20260829_01); request headers stay only in the encrypted secret blob. The resource catalog may return
|
||||
@@ -230,6 +230,22 @@ and a `createdBy` display label. Resolve human labels from the immutable owner i
|
||||
fall back to that id if lookup fails; built-in records must return `系统内置`. Do not create a parallel
|
||||
creator table or accept creator data from the browser.
|
||||
|
||||
`evidence_normalizer` is a deterministic data node, not an LLM: it consumes only configured/upstream
|
||||
node results and emits a bounded `evidencePack` of claims with node provenance, role labels and gaps.
|
||||
The planner inserts it between parallel research roles and the synthesis Agent. It must not treat
|
||||
source text as verified fact or fetch any external resource, and its explicit `sources[].nodeId` values
|
||||
must stay upstream-only under `validate_workflow_graph`.
|
||||
|
||||
`deep_research_write` is a durable report node, not an Agent prompt shortcut. It accepts only an
|
||||
Evidence Pack, derives a session identity from the workflow run/node plus the topic, evidence snapshot and writing config, writes
|
||||
the pack as selected `other` source rows with upstream-node provenance, and queues/reuses the existing
|
||||
Deep Research `regenerate` job. Its live `report_delta` and persisted `report_chunk` frames are
|
||||
deduplicated before becoming workflow `node.output.delta` events; only report prose is forwarded (never
|
||||
planning/reasoning deltas). Workflow cancellation requests cancellation on that same job. Do not make it
|
||||
self-call `/api/deep-research/*`, use Deep Research `multi_agent` inside this node, or treat Evidence
|
||||
Pack claims as verified facts. The parallel-research candidate grants the node an explicit 1800-second
|
||||
timeout; deployments that override `workflows.node_timeout_seconds` must keep it at least that high.
|
||||
|
||||
Tests: `tests/test_workflow_schemas.py`, `tests/test_workflow_definitions.py`, `tests/test_workflow_engine.py`, `tests/test_workflow_runs.py`.
|
||||
|
||||
**Import conventions**:
|
||||
|
||||
Reference in new issue
Block a user