diff --git a/frontend-web/docs/microsoft-agent-framework-多智能体会商-实施计划.md b/frontend-web/docs/microsoft-agent-framework-多智能体会商-实施计划.md index 4cf5681e..19776e9a 100644 --- a/frontend-web/docs/microsoft-agent-framework-多智能体会商-实施计划.md +++ b/frontend-web/docs/microsoft-agent-framework-多智能体会商-实施计划.md @@ -643,3 +643,38 @@ maf-roundtable/conversation/ - The final plan is Pydantic-validated and allowlisted before it is checkpointed. Invalid JSON, illegal intent or seat, missing convergence, manager failure, and timeout all finish on the clarification branch; they do not create expert threads or start expert/tool work. - A conversation initially persists and binds the manager only. After a valid decision, only `selected_seat_agent_ids` are added to the participant store and receive internal threads. - Every non-clarification route, including `group_chat`, starts at the same manager dispatch node. Group speakers are limited to the verified selected seats; static candidate configuration cannot bypass the plan. + +--- + +## 阶段 4 实施记录（当前代码基线） + +本轮完成阶段 4（R3-Q：席位任务合同、总控验收、打回/改派/受控缺口、阶段质量闸门）的服务端实现。范围仅限 V2 structured 派活（`dispatch_contract_enabled` 且 `quality_gate_enabled`）；旧 DeerFlow、旧 roundtable、AI 写作、group_chat 与 quality 关闭的 V2 路径行为不变。 + +### 纯策略层（`app/maf_roundtable/quality_contracts.py`，无 I/O、无 SDK） + +- `MafTaskContract`：objective / expected_deliverable / acceptance_criteria / required / max_rework_attempts（默认 2）/ fallback_policy（`reassign_then_gap` / `controlled_gap` / `ask_user`），由校验后的派发 assignment 服务端冻结。 +- 确定性检查 `check_delivery_deterministic`：empty_content、schema_invalid、missing_conclusion、below_min_length、missing_evidence、duplicate_content；任一失败即不可 accept（总控模型自评不是验收证据）。 +- 总控验收 schema `MafManagerReviewOutput`（decision ∈ accept / return_for_rework / reassign / ask_user / controlled_gap + reason + revision_requirements），`finalize_review_decision` 服务端终审：确定性失败永不 accept；返工受合同上限约束；改派受 `max_reassignments` 约束；耗尽后只能 `controlled_gap`（带风险标记）或 `ask_user` 暂停，绝不伪装成功；总控输出缺失/非法时确定性降级。 + +### 持久化 + +- 新表 `maf_roundtable_seat_tasks`（`model.py` + `seat_tasks.py` + store facade 注册）：task_key 稳定（`NN:agent_id`），改派只变 `agent_id` 不新建任务；`(run_id, task_key)` 唯一；状态机 CHECK（assigned/delivering/submitted/manager_review/accepted/rework/reassigned/exhausted/controlled_gap）。 +- `maf_roundtable_seat_runs.execution_stage` 闭合枚举新增 `rework`、`manager_review`（seat_runs 常量 + ORM CHECK + 迁移脚本同步）。 +- 迁移脚本 `scripts/migrate_maf_roundtable_stage4.py`（建表 + SQLite 重建 seat_runs 以放宽 CHECK，PG/MySQL drop+add，幂等，`validate_schema`）。 + +### 执行器与接线 + +- `QualitySeatTasksExecutor`（id=`quality_seat_tasks`）替换 V2 structured 的席位 fan-out：有界质量轮（初稿 + 最多 2 次返工 + 每次改派 1 轮），每轮「交付（复用 SeatExecutor，静默 ctx 防消息重入图）→ 确定性检查 → 批处理总控验收 → 服务端终审落库」，`ask_user` 置 `waiting_input`；改派席位须为服务端候选且未占用其他任务（一个席位同时只承担一个任务，避免四元组冲突）。 +- `QualityGateExecutor`（id=`quality_gate`）：所有 required 任务达到 accepted / controlled_gap 终态才放行（`passed` / `passed_with_gaps`）；非终态 required 任务确定性阻塞（硬失败安全网）；`waiting_input` 路由到 `WaitingInputTerminalExecutor`（不跑 cross_review/finalizer）。 +- 最终报告：受控缺口注入综合 prompt 的「风险/待补信息」章节并逐条要求不得写成已验证事实；`final_result.quality_decision` 有缺口即为 `controlled_gap`，`final_result.controlled_gaps` 可追溯。 +- 配置：`quality_gate_enabled`（默认 True，V2 会话可用 `maf_v2.quality_gate_enabled` 显式关闭）、`max_rework_attempts`、`max_reassignments`、`min_delivery_chars`、`min_delivery_evidence`、`default_fallback_policy`。 +- 事件：`seat.delivery.submitted`、`manager.review.decision`（带 task_key/attempt/source）、`seat.rework.assigned`、`seat.reassigned`、`stage.quality_gate`；conversation 投影白名单与安全键已扩展。 + +### 测试 + +- `tests/test_maf_quality_contracts.py`（34 例纯策略：合同冻结、确定性检查、验收解析、终审规则矩阵）。 +- `tests/test_maf_phase4_quality_gate.py`（6 例 Runner 级端到端：好交付接受、短答→2 返工→1 改派→受控缺口（含 DB/事件/最终报告断言）、ask_user→waiting_input、质量闸门阻塞、缺失验收的确定性降级、事件落库）。 +- `tests/test_maf_migration_stage4.py`（3 例迁移：建表+CHECK 放宽、幂等、旧数据保留、新阶段可写入）。 +- 回归：全量 MAF 测试 459 通过；`ruff check` 全绿。 + +仍按阶段 5/8 前置条件保持阻塞：真实 UAT 写入系统、最小权限凭据、外部核验 API、补偿协议和生产 `MAF_ACTION_SECRET_KEY`。前端对话页的质量反馈（派活卡显示尝试次数、「总控审核中」、打回/改派/缺口卡片、席位条状态区分）属阶段 7 的前端工作，尚未实施。 diff --git a/offline-backend-20260512/backend/app/maf_roundtable/conversation_service.py b/offline-backend-20260512/backend/app/maf_roundtable/conversation_service.py index e5102d12..cd367afd 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/conversation_service.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/conversation_service.py @@ -430,6 +430,7 @@ class MafConversationService: _ALLOWED_TYPES = frozenset({ "agent.message.start", "agent.message.delta", "agent.message.complete", "seat.delivery.submitted", "manager.dispatch", "manager.review.decision", + "seat.rework.assigned", "seat.reassigned", "stage.quality_gate", "tool.requested", "tool.started", "tool.progress", "tool.completed", "tool.failed", "skill.selected", "skill.started", "skill.completed", "skill.failed", "artifact.created", "approval.required", "approval.decided", "clarification.required", @@ -437,6 +438,11 @@ class MafConversationService: "run.cancelled", "run.completed", }) + # Execution stages whose ``agent.message.completed`` is a genuine seat + # delivery. Manager / dispatch / review / cross-review / finalizer / + # format-repair messages must NEVER be projected as seat.delivery.submitted. + _SEAT_DELIVERY_STAGES = frozenset({"expert", "rework", "legacy", "group_chat"}) + _SAFE_KEYS: dict[str, dict[str, bool]] = { "agent.message.start": {"thread_id": False}, "agent.message.delta": {"text": True}, @@ -463,10 +469,27 @@ class MafConversationService: "reason": True, "source": True, "clarification_prompt": True, }, "clarification.required": {"prompt": True, "reason": True}, - "manager.review.decision": {"decision": True, "reason": True, "quality": True}, + "manager.review.decision": { + "decision": True, "reason": True, "quality": True, + "task_key": True, "attempt": True, "source": True, + }, "fact.confirmed": {"fact_id": True, "summary": True}, "review.completed": {"summary": True, "decision": True}, - "seat.delivery.submitted": {"content": True, "schema_status": True}, + "seat.delivery.submitted": { + "content": True, "schema_status": True, + "task_key": True, "attempt": True, + "deterministic_passed": True, "failure_codes": True, + }, + "seat.rework.assigned": { + "task_key": True, "rework_count": True, "revision_requirements": True, + }, + "seat.reassigned": { + "task_key": True, "from_agent_id": True, "reassign_count": True, + }, + "stage.quality_gate": { + "decision": True, "total": True, "accepted": True, + "controlled_gap": True, "blocked_required": True, + }, } def _build_projection_items( @@ -501,19 +524,26 @@ class MafConversationService: "source_event_id": source_evt_id, }] + # Only a genuine seat delivery turn may be projected as + # seat.delivery.submitted. Manager / dispatch / manager_review / + # cross-review / finalizer / format-repair messages are ordinary + # agent messages and must never be mislabelled as seat deliveries + # (Stage-4 runs emit their own explicit seat.delivery.submitted event). if source_type == "agent.message.completed": - delivery_data: dict[str, Any] = { - key: payload[key] - for key in ("content", "schema_status") - if key in payload - } - if event.get("agent_id"): - delivery_data["agent_id"] = event["agent_id"] - items.append({ - "event_type": "seat.delivery.submitted", - "event_data": delivery_data, - "source_event_id": f"{source_evt_id}:delivery" if source_evt_id else None, - }) + stage = str(payload.get("execution_stage") or "") + if stage in self._SEAT_DELIVERY_STAGES: + delivery_data: dict[str, Any] = { + key: payload[key] + for key in ("content", "schema_status") + if key in payload + } + if event.get("agent_id"): + delivery_data["agent_id"] = event["agent_id"] + items.append({ + "event_type": "seat.delivery.submitted", + "event_data": delivery_data, + "source_event_id": f"{source_evt_id}:delivery" if source_evt_id else None, + }) return items diff --git a/offline-backend-20260512/backend/app/maf_roundtable/deerflow_agent_adapter.py b/offline-backend-20260512/backend/app/maf_roundtable/deerflow_agent_adapter.py index 40dd8260..b35bf1cc 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/deerflow_agent_adapter.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/deerflow_agent_adapter.py @@ -275,6 +275,7 @@ class DeerFlowAgentAdapter: content=response.content, usage=response.usage, metadata=response.metadata, + execution_stage=execution_stage, )) return response diff --git a/offline-backend-20260512/backend/app/maf_roundtable/event_normalizer.py b/offline-backend-20260512/backend/app/maf_roundtable/event_normalizer.py index 7c753e6f..b7d364d0 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/event_normalizer.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/event_normalizer.py @@ -75,12 +75,17 @@ def make_turn_completed( content: str, usage: dict[str, Any] | None = None, metadata: dict[str, Any] | None = None, + execution_stage: str | None = None, ) -> dict[str, Any]: payload: dict[str, Any] = {"content": content} if usage: payload["usage"] = usage if metadata: payload["metadata"] = metadata + # The conversation projection must be able to tell a real seat delivery + # apart from manager / review / finalizer messages — carry the stage. + if execution_stage: + payload["execution_stage"] = execution_stage return _make("agent.message.completed", agent_id, turn_index, payload) @@ -154,6 +159,99 @@ def make_workflow_checkpointed(agent_id: str | None, turn_index: int, *, checkpo return _make("workflow.checkpointed", agent_id or "", turn_index, {"checkpoint_id": checkpoint_id, "reason": reason}) +# ── Stage 4 (R3-Q) quality gate events ────────────────────────────────── + + +def make_seat_delivery_submitted( + agent_id: str, + turn_index: int, + *, + task_key: str, + attempt: int, + deterministic_passed: bool, + failure_codes: list[str] | None = None, +) -> dict[str, Any]: + """A seat task submitted its delivery and passed/failed deterministic checks.""" + return _make("seat.delivery.submitted", agent_id, turn_index, { + "task_key": task_key, + "attempt": attempt, + "deterministic_passed": deterministic_passed, + "failure_codes": list(failure_codes or []), + }) + + +def make_manager_review_decision( + agent_id: str, + turn_index: int, + *, + task_key: str, + decision: str, + reason: str, + source: str, + attempt: int, +) -> dict[str, Any]: + """Server-finalized manager review decision for one seat task.""" + return _make("manager.review.decision", agent_id, turn_index, { + "task_key": task_key, + "decision": decision, + "reason": _truncate(reason, 500), + "source": source, + "attempt": attempt, + }) + + +def make_seat_rework_assigned( + agent_id: str, + turn_index: int, + *, + task_key: str, + rework_count: int, + requirements: list[str] | None = None, +) -> dict[str, Any]: + """A seat task is sent back for rework with explicit revision requirements.""" + return _make("seat.rework.assigned", agent_id, turn_index, { + "task_key": task_key, + "rework_count": rework_count, + "revision_requirements": [str(r) for r in (requirements or [])][:16], + }) + + +def make_seat_reassigned( + agent_id: str, + turn_index: int, + *, + task_key: str, + from_agent_id: str, + reassign_count: int, +) -> dict[str, Any]: + """A seat task is reassigned to a backup seat (agent_id = new owner).""" + return _make("seat.reassigned", agent_id, turn_index, { + "task_key": task_key, + "from_agent_id": from_agent_id, + "reassign_count": reassign_count, + }) + + +def make_stage_quality_gate( + agent_id: str | None, + turn_index: int, + *, + decision: str, + total: int, + accepted: int, + controlled_gap: int, + blocked_required: list[str] | None = None, +) -> dict[str, Any]: + """Stage quality gate outcome: passed / blocked / passed_with_gaps.""" + return _make("stage.quality_gate", agent_id or "", turn_index, { + "decision": decision, + "total": total, + "accepted": accepted, + "controlled_gap": controlled_gap, + "blocked_required": list(blocked_required or []), + }) + + # ── internal helpers ────────────────────────────────────────────────────────── diff --git a/offline-backend-20260512/backend/app/maf_roundtable/quality_contracts.py b/offline-backend-20260512/backend/app/maf_roundtable/quality_contracts.py new file mode 100644 index 00000000..811ba009 --- /dev/null +++ b/offline-backend-20260512/backend/app/maf_roundtable/quality_contracts.py @@ -0,0 +1,568 @@ +"""Stage 4 (R3-Q): seat task contracts, deterministic quality checks, manager +review schema, and server-owned decision finalization. + +Pure policy module: no I/O and no MAF SDK imports, so the whole acceptance +policy is unit-testable with a weak/fake model before any workflow runs. + +The manager model only ever *proposes* review decisions. This module is the +final validator and never allows: + +- accepting a delivery that failed the deterministic quality checks (the + manager's self-assessment is NOT acceptance evidence); +- reworking a seat beyond its frozen ``max_rework_attempts``; +- more reassignments than the server-configured cap; +- an exhausted task being disguised as accepted — it becomes a + ``controlled_gap`` carrying its risk, or an explicit ``ask_user`` pause. + +Task status state machine (persisted per seat task):: + + assigned → delivering → submitted → manager_review → + accepted + | rework (→ delivering, with explicit revision requirements) + | reassigned (→ delivering, backup seat takes over) + | exhausted (rework/reassign caps reached) + → controlled_gap (risk recorded, carried into the final report) +""" + +from __future__ import annotations + +import hashlib +import json +import re +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field + +# ── task status state machine ─────────────────────────────────────────────── + +TASK_ASSIGNED = "assigned" +TASK_DELIVERING = "delivering" +TASK_SUBMITTED = "submitted" +TASK_MANAGER_REVIEW = "manager_review" +TASK_ACCEPTED = "accepted" +TASK_REWORK = "rework" +TASK_REASSIGNED = "reassigned" +TASK_EXHAUSTED = "exhausted" +TASK_CONTROLLED_GAP = "controlled_gap" + +TASK_TERMINAL_STATUSES = frozenset({TASK_ACCEPTED, TASK_CONTROLLED_GAP}) +TASK_ALL_STATUSES = frozenset({ + TASK_ASSIGNED, + TASK_DELIVERING, + TASK_SUBMITTED, + TASK_MANAGER_REVIEW, + TASK_ACCEPTED, + TASK_REWORK, + TASK_REASSIGNED, + TASK_EXHAUSTED, + TASK_CONTROLLED_GAP, +}) + +MafReviewDecision = Literal[ + "accept", "return_for_rework", "reassign", "ask_user", "controlled_gap" +] +MafFallbackPolicy = Literal["reassign_then_gap", "controlled_gap", "ask_user"] + +# Final (server-owned) outcomes of the decision finalizer. +OUTCOME_ACCEPTED = "accepted" +OUTCOME_REWORK = "rework" +OUTCOME_REASSIGNED = "reassigned" +OUTCOME_CONTROLLED_GAP = "controlled_gap" +OUTCOME_ASK_USER = "ask_user" + + +class InvalidTaskContractError(ValueError): + """Raised when a task contract cannot be frozen from a dispatch plan.""" + + +# ── task contract ─────────────────────────────────────────────────────────── + + +class MafTaskContract(BaseModel): + """Frozen acceptance contract for one seat task. + + Created server-side from the validated dispatch assignment before the seat + starts working. The manager model can never widen these terms afterwards. + """ + + model_config = ConfigDict(extra="forbid") + + objective: str = Field(min_length=1, max_length=2000) + expected_deliverable: str = Field(min_length=1, max_length=2000) + acceptance_criteria: list[str] = Field(default_factory=list, max_length=32) + required: bool = True + max_rework_attempts: int = Field(default=2, ge=0, le=8) + fallback_policy: MafFallbackPolicy = "reassign_then_gap" + + +DEFAULT_EXPECTED_DELIVERABLE = ( + "结构化 JSON 交付：结论(summary)、依据/证据(findings/evidence_refs)、" + "未解决项(open_questions)、自检(self_check)" +) + + +def build_task_contract( + *, + task_goal: str, + delivery_requirements: list[str] | None = None, + required: bool = True, + max_rework_attempts: int = 2, + fallback_policy: MafFallbackPolicy = "reassign_then_gap", +) -> MafTaskContract: + """Freeze a task contract from one validated dispatch assignment.""" + objective = str(task_goal or "").strip() + if not objective: + raise InvalidTaskContractError("task contract objective must be non-empty") + criteria = [ + str(item).strip() + for item in (delivery_requirements or []) + if str(item or "").strip() + ] + return MafTaskContract( + objective=objective, + expected_deliverable=DEFAULT_EXPECTED_DELIVERABLE, + acceptance_criteria=criteria, + required=required, + max_rework_attempts=max_rework_attempts, + fallback_policy=fallback_policy, + ) + + +# ── seat delivery ─────────────────────────────────────────────────────────── + + +class MafSeatDelivery(BaseModel): + """Structured delivery extracted from one seat turn output.""" + + model_config = ConfigDict(extra="forbid") + + conclusion: str = "" + evidence: list[str] = Field(default_factory=list) + # P1: only ``evidence_refs`` can satisfy the evidence requirement. Seat + # ``findings`` are candidate claims — they are kept in ``evidence`` for + # review display but never count as verified evidence. + evidence_refs: list[str] = Field(default_factory=list) + unresolved_items: list[str] = Field(default_factory=list) + self_check: str = "" + + +def _flat_text(item: Any, *keys: str) -> str: + if isinstance(item, dict): + for key in keys: + value = item.get(key) + if isinstance(value, str) and value.strip(): + return value.strip() + return "" + return str(item or "").strip() + + +def delivery_from_seat_output( + *, structured_output: dict[str, Any] | None, raw_content: str +) -> MafSeatDelivery: + """Map a validated SeatTurnOutput dict onto the delivery contract. + + ``structured_output`` is ``None`` when the seat output failed the schema — + the resulting delivery is (almost) empty so the deterministic checks flag + it instead of trusting the raw text. + """ + out = structured_output if isinstance(structured_output, dict) else {} + conclusion = str(out.get("summary") or "").strip() + + evidence: list[str] = [] + for item in out.get("findings") or []: + text = _flat_text(item, "text") + if text: + evidence.append(text) + evidence_refs: list[str] = [] + for ref in out.get("evidence_refs") or []: + text = _flat_text(ref, "ref", "text") + if text: + evidence_refs.append(text) + evidence.append(text) + + unresolved = [ + str(q).strip() for q in (out.get("open_questions") or []) if str(q or "").strip() + ] + self_check = str(out.get("self_check") or "").strip() + return MafSeatDelivery( + conclusion=conclusion, + evidence=evidence, + evidence_refs=evidence_refs, + unresolved_items=unresolved, + self_check=self_check, + ) + + +# ── deterministic quality checks ──────────────────────────────────────────── + +FAIL_EMPTY_CONTENT = "empty_content" +FAIL_SCHEMA_INVALID = "schema_invalid" +FAIL_MISSING_CONCLUSION = "missing_conclusion" +FAIL_BELOW_MIN_LENGTH = "below_min_length" +FAIL_MISSING_EVIDENCE = "missing_evidence" +FAIL_DUPLICATE_CONTENT = "duplicate_content" +FAIL_UNVERIFIED_EVIDENCE_REF = "unverified_evidence_ref" + +_REPAIR_HINTS: dict[str, str] = { + FAIL_EMPTY_CONTENT: "交付内容为空，请给出完整的分析与结论", + FAIL_SCHEMA_INVALID: "输出不是合法的结构化 JSON，请按 SeatTurnOutput 字段重新输出", + FAIL_MISSING_CONCLUSION: "缺少明确结论（summary），请补充", + FAIL_BELOW_MIN_LENGTH: "结论过短，请展开具体分析、依据与建议", + FAIL_MISSING_EVIDENCE: ( + "缺少可信证据（evidence_refs）。findings 只是候选主张，不能充当证据；" + "请引用事实账本/成功工具结果中真实存在的证据引用" + ), + FAIL_DUPLICATE_CONTENT: "内容与上一次交付重复，请针对退回意见提供新的分析", + FAIL_UNVERIFIED_EVIDENCE_REF: ( + "引用了事实账本中不存在的证据（自造引用不可接受）；" + "请只引用已记录的可信证据或成功工具结果" + ), +} + + +def conclusion_digest(conclusion: str) -> str: + """SHA-256 digest of the normalized conclusion. + + Duplicate detection persists these digests (never the conclusion bodies) + in the task history, so the check survives restarts without storing the + duplicated text again. + """ + normalized = " ".join(str(conclusion or "").split()) + return hashlib.sha256(normalized.encode("utf-8")).hexdigest() + + +def check_delivery_deterministic( + *, + delivery: MafSeatDelivery, + raw_content: str, + schema_status: str | None, + min_length: int = 40, + min_evidence: int = 1, + prior_conclusion_digests: set[str] | list[str] | None = None, + verified_evidence_refs: set[str] | None = None, +) -> dict[str, Any]: + """Deterministic acceptance gate run BEFORE the manager review. + + A delivery that fails any check can never be accepted, regardless of what + the manager model claims. Returns ``{"passed": bool, "failure_codes": [...]}``. + + Evidence rules (P1): + - seat ``findings`` are candidate claims only — they can NEVER satisfy the + evidence requirement; only ``evidence_refs`` count; + - when ``verified_evidence_refs`` is supplied (the trusted refs the fact + ledger actually backs: recorded evidence + successful tool results), + every cited ref must resolve — an invented ref fails with + ``unverified_evidence_ref``, and only verified refs count toward + ``min_evidence``; + - duplicate detection compares SHA-256 digests of normalized conclusions, + so it survives restarts via the persisted task history. + """ + codes: list[str] = [] + text = str(raw_content or "").strip() + if not text: + codes.append(FAIL_EMPTY_CONTENT) + if schema_status == "schema_invalid": + codes.append(FAIL_SCHEMA_INVALID) + if not delivery.conclusion: + codes.append(FAIL_MISSING_CONCLUSION) + elif len(delivery.conclusion) < max(0, int(min_length)): + codes.append(FAIL_BELOW_MIN_LENGTH) + + cited_refs = [str(r) for r in (delivery.evidence_refs or []) if str(r or "").strip()] + if verified_evidence_refs is not None: + trusted = {str(r) for r in verified_evidence_refs} + verified = [r for r in cited_refs if r in trusted] + if any(r not in trusted for r in cited_refs): + codes.append(FAIL_UNVERIFIED_EVIDENCE_REF) + else: + verified = cited_refs + if len(verified) < max(0, int(min_evidence)): + codes.append(FAIL_MISSING_EVIDENCE) + + if ( + delivery.conclusion + and prior_conclusion_digests + and conclusion_digest(delivery.conclusion) in set(prior_conclusion_digests) + ): + codes.append(FAIL_DUPLICATE_CONTENT) + return {"passed": not codes, "failure_codes": codes} + + +def repair_requirements_for_codes(failure_codes: list[str]) -> list[str]: + """Deterministic, targeted revision requirements for failed checks.""" + hints = [_REPAIR_HINTS[code] for code in failure_codes if code in _REPAIR_HINTS] + return hints or ["请按照任务合同补充缺失内容后重新交付"] + + +# ── manager review schema ─────────────────────────────────────────────────── + + +class MafManagerReviewItem(BaseModel): + """One per-task review decision proposed by the manager model.""" + + model_config = ConfigDict(extra="forbid") + + agent_id: str = Field(min_length=1, max_length=64) + decision: MafReviewDecision + reason: str = Field(min_length=1, max_length=1000) + revision_requirements: list[str] = Field(default_factory=list, max_length=16) + + +class MafManagerReviewOutput(BaseModel): + """Strict model-facing manager review schema (one entry per reviewed seat).""" + + model_config = ConfigDict(extra="forbid") + + decisions: list[MafManagerReviewItem] = Field(default_factory=list, max_length=64) + + +_JSON_FENCE_RE = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE) + + +def _extract_json_object(text: str) -> str | None: + raw = (text or "").strip() + if not raw: + return None + m = _JSON_FENCE_RE.search(raw) + if m: + raw = m.group(1).strip() + if raw.startswith("{") and raw.endswith("}"): + return raw + start = raw.find("{") + end = raw.rfind("}") + if start >= 0 and end > start: + return raw[start : end + 1] + return None + + +def parse_manager_review(text: str) -> tuple[MafManagerReviewOutput | None, list[str]]: + """Parse the manager's review JSON. Returns ``(model, readable_errors)``.""" + blob = _extract_json_object(text) + if blob is None: + return None, ["review output is not a JSON object; emit MafManagerReviewOutput JSON"] + try: + data = json.loads(blob) + except json.JSONDecodeError as exc: + return None, [f"invalid JSON: {exc.msg}"] + if not isinstance(data, dict): + return None, ["JSON root must be an object"] + try: + return MafManagerReviewOutput.model_validate(data), [] + except Exception as exc: # pydantic ValidationError + errors = [] + detail = getattr(exc, "errors", None) + if callable(detail): + for err in detail(): + loc = ".".join(str(p) for p in err.get("loc", ())) + errors.append(f"{loc}: {err.get('msg', 'invalid')}") + return None, errors or ["manager review schema validation failed"] + + +def build_manager_review_prompt( + *, + task_text: str, + review_items: list[dict[str, Any]], +) -> str: + """Build the strict manager review prompt. + + Each review item: ``{agent_id, attempt, contract, delivery, check}``. The + manager may only propose decisions; the server finalizes them. + """ + contract_shape = { + "decisions": [ + { + "agent_id": "被审查席位 id", + "decision": "accept|return_for_rework|reassign|ask_user|controlled_gap", + "reason": "必填：针对性理由", + "revision_requirements": ["打回时必填：缺什么、怎样补、哪些原结论不可用"], + } + ] + } + lines = [ + "你是 MAF 总控，负责验收下列席位交付。只输出一个 JSON 对象（不要 Markdown），" + "对每个席位给出一条 decision。验收依据是任务合同与确定性检查结果；" + "确定性检查未通过的交付不得 accept。打回时必须写明缺什么、怎样补。" + "不要输出任何席位/工具/线程 id 之外的字段。", + f"原始任务：\n{task_text[:2000]}", + "待审查交付：", + json.dumps(review_items, ensure_ascii=False)[:12000], + f"必须返回的 JSON 结构：{json.dumps(contract_shape, ensure_ascii=False)}", + ] + return "\n\n".join(lines) + + +# ── server-owned decision finalization ────────────────────────────────────── + + +def finalize_review_decision( + *, + contract: MafTaskContract, + check_result: dict[str, Any], + review_item: MafManagerReviewItem | None, + rework_count: int, + reassign_count: int, + has_backup_seat: bool, + max_reassignments: int = 1, +) -> dict[str, Any]: + """Finalize one task's review decision server-side. + + Returns ``{"decision", "reason", "revision_requirements", "source"}`` where + ``decision`` is one of the final outcomes: ``accepted`` / ``rework`` / + ``reassigned`` / ``controlled_gap`` / ``ask_user``. + + Rules enforced here (never by the model alone): + + 1. A deterministic check failure can NEVER end in ``accepted``. + 2. Rework is only granted while ``rework_count < contract.max_rework_attempts``. + 3. Reassignment is only granted while a backup seat exists and + ``reassign_count < max_reassignments``. + 4. Once every cap is exhausted the task ends as ``controlled_gap`` (or an + explicit ``ask_user`` pause when the contract's fallback policy says + so) — never as a disguised success. + 5. A missing/invalid manager review (timeout, exception, empty output, + invalid JSON, or a missing per-seat decision) is NEVER an acceptance, + even when the delivery passed every deterministic check: while the + rework cap allows it, the server forces a system rework; once the caps + are exhausted the contract's fallback policy decides (reassign / + ``controlled_gap`` / ``ask_user``). + """ + codes = list(check_result.get("failure_codes") or []) + deterministic_passed = bool(check_result.get("passed")) + can_rework = rework_count < contract.max_rework_attempts + can_reassign = has_backup_seat and reassign_count < max_reassignments + + def _outcome(decision: str, reason: str, source: str, requirements: list[str] | None = None) -> dict[str, Any]: + return { + "decision": decision, + "reason": str(reason or "")[:1000], + "revision_requirements": [str(r) for r in (requirements or [])][:16], + "source": source, + } + + def _exhaustion_outcome(reason: str, source: str) -> dict[str, Any]: + if contract.fallback_policy == "ask_user": + return _outcome(OUTCOME_ASK_USER, reason, source) + if contract.fallback_policy == "reassign_then_gap" and can_reassign: + return _outcome(OUTCOME_REASSIGNED, reason, source) + return _outcome(OUTCOME_CONTROLLED_GAP, reason, source) + + # 1) Deterministic failure — never accepted. + if not deterministic_passed: + base_reason = f"deterministic_quality_check_failed: {', '.join(codes)}" + if can_rework: + requirements = ( + list(review_item.revision_requirements) + if review_item is not None + and review_item.decision == "return_for_rework" + and review_item.revision_requirements + else repair_requirements_for_codes(codes) + ) + return _outcome(OUTCOME_REWORK, base_reason, "deterministic", requirements) + return _exhaustion_outcome(base_reason, "deterministic") + + # 2) Deterministic checks passed — but acceptance still requires a valid + # manager review. A missing/invalid review (timeout, exception, empty + # output, invalid JSON, or a missing per-seat decision) is NEVER an + # acceptance: passing length/format checks alone does not make a delivery + # verified. The server forces a system rework while the cap allows it, and + # falls back to the contract's exhaustion policy afterwards. + if review_item is None: + reason = "manager_review_missing_or_invalid" + if can_rework: + return _outcome( + OUTCOME_REWORK, + reason, + "system", + ["总控未获得有效验收结论；请重新提交完整、可核验的交付。"], + ) + return _exhaustion_outcome(reason, "system") + + decision = review_item.decision + if decision == "accept": + return _outcome(OUTCOME_ACCEPTED, review_item.reason, "manager") + if decision == "return_for_rework": + if can_rework: + requirements = list(review_item.revision_requirements) or repair_requirements_for_codes(codes) + return _outcome(OUTCOME_REWORK, review_item.reason, "manager", requirements) + return _exhaustion_outcome( + f"rework_cap_reached: {review_item.reason}", "manager" + ) + if decision == "reassign": + if can_reassign: + return _outcome(OUTCOME_REASSIGNED, review_item.reason, "manager") + return _exhaustion_outcome( + f"reassign_unavailable: {review_item.reason}", "manager" + ) + if decision == "ask_user": + return _outcome(OUTCOME_ASK_USER, review_item.reason, "manager") + # decision == "controlled_gap" + return _outcome(OUTCOME_CONTROLLED_GAP, review_item.reason, "manager") + + +def make_controlled_gap( + *, + task_key: str, + agent_id: str, + objective: str, + reason: str, + failure_codes: list[str] | None = None, + attempts: int = 0, +) -> dict[str, Any]: + """Build the persisted, traceable controlled-gap record. + + A controlled gap is NEVER a verified fact: the final report must carry it + in its risk / pending-information section. + """ + return { + "task_key": task_key, + "agent_id": agent_id, + "objective": str(objective or "")[:2000], + "reason": str(reason or "")[:1000], + "failure_codes": list(failure_codes or []), + "attempts": int(attempts), + "risk": "该问题未获得合格席位交付，不得作为已验证事实写入最终结论", + } + + +__all__ = [ + "DEFAULT_EXPECTED_DELIVERABLE", + "FAIL_BELOW_MIN_LENGTH", + "FAIL_DUPLICATE_CONTENT", + "FAIL_EMPTY_CONTENT", + "FAIL_MISSING_CONCLUSION", + "FAIL_MISSING_EVIDENCE", + "FAIL_SCHEMA_INVALID", + "FAIL_UNVERIFIED_EVIDENCE_REF", + "InvalidTaskContractError", + "MafFallbackPolicy", + "MafManagerReviewItem", + "MafManagerReviewOutput", + "MafReviewDecision", + "MafSeatDelivery", + "MafTaskContract", + "OUTCOME_ACCEPTED", + "OUTCOME_ASK_USER", + "OUTCOME_CONTROLLED_GAP", + "OUTCOME_REASSIGNED", + "OUTCOME_REWORK", + "TASK_ACCEPTED", + "TASK_ALL_STATUSES", + "TASK_ASSIGNED", + "TASK_CONTROLLED_GAP", + "TASK_DELIVERING", + "TASK_EXHAUSTED", + "TASK_MANAGER_REVIEW", + "TASK_REASSIGNED", + "TASK_REWORK", + "TASK_SUBMITTED", + "TASK_TERMINAL_STATUSES", + "build_manager_review_prompt", + "build_task_contract", + "check_delivery_deterministic", + "conclusion_digest", + "delivery_from_seat_output", + "finalize_review_decision", + "make_controlled_gap", + "parse_manager_review", + "repair_requirements_for_codes", +] diff --git a/offline-backend-20260512/backend/app/maf_roundtable/seat_output.py b/offline-backend-20260512/backend/app/maf_roundtable/seat_output.py index ba6dc47b..dc327e7c 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/seat_output.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/seat_output.py @@ -24,6 +24,9 @@ class SeatTurnOutput(BaseModel): executed_actions: list[str | dict[str, Any]] = Field(default_factory=list) open_questions: list[str] = Field(default_factory=list) conflicts: list[str] = Field(default_factory=list) + # Stage 4 (R3-Q): optional self-check against the task contract. The + # manager review treats this as advisory — it is never acceptance evidence. + self_check: str | None = Field(default=None, max_length=2000) class ConfirmedCandidate(BaseModel): @@ -134,7 +137,8 @@ def seat_output_instruction() -> str: """Stage instruction fragment telling the model to emit SeatTurnOutput JSON.""" return ( "请以 JSON 对象回复，字段：summary(string,必填)、findings(string[])、" - "evidence_refs、proposed_actions、executed_actions、open_questions、conflicts。" + "evidence_refs、proposed_actions、executed_actions、open_questions、conflicts、" + "self_check(string,可选：对照任务合同自检：已覆盖哪些验收标准、哪些仍缺失)。" "proposed_actions 只表示建议；不要把未真正由系统执行的外部写入标为已执行。" ) diff --git a/offline-backend-20260512/backend/app/maf_roundtable/workflow_executors.py b/offline-backend-20260512/backend/app/maf_roundtable/workflow_executors.py index ee319ab7..e844024d 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/workflow_executors.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/workflow_executors.py @@ -32,14 +32,45 @@ from app.maf_roundtable.dispatch_contracts import ( MafDispatchCatalog, MafDispatchPlan, ) +from app.maf_roundtable.event_normalizer import ( + make_manager_review_decision, + make_seat_delivery_submitted, + make_seat_reassigned, + make_seat_rework_assigned, + make_stage_quality_gate, +) from app.maf_roundtable.fact_ledger import ( confirm_candidates, empty_ledger, + known_evidence_refs, merge_seat_output, normalize_ledger, record_action_transition, record_tool_result, ) +from app.maf_roundtable.quality_contracts import ( + OUTCOME_ACCEPTED, + OUTCOME_ASK_USER, + OUTCOME_CONTROLLED_GAP, + OUTCOME_REASSIGNED, + OUTCOME_REWORK, + TASK_ACCEPTED, + TASK_CONTROLLED_GAP, + TASK_EXHAUSTED, + TASK_REASSIGNED, + TASK_REWORK, + TASK_SUBMITTED, + TASK_TERMINAL_STATUSES, + MafTaskContract, + build_manager_review_prompt, + build_task_contract, + check_delivery_deterministic, + conclusion_digest, + delivery_from_seat_output, + finalize_review_decision, + make_controlled_gap, + parse_manager_review, +) from app.maf_roundtable.seat_output import ( build_repair_prompt, parse_cross_review_output, @@ -85,6 +116,7 @@ class RoundtableState: dispatch_plan: dict[str, Any] | None = None, candidate_seat_agent_ids: list[str] | None = None, dispatch_contract_enabled: bool | None = None, + quality_gate_enabled: bool | None = None, ) -> None: self.session_id = session_id self.run_id = run_id @@ -110,6 +142,14 @@ class RoundtableState: if dispatch_contract_enabled is None else bool(dispatch_contract_enabled) ) + # Stage 4 (R3-Q): quality gate for V2 structured dispatch. Defaults to + # the dispatch-contract value when not explicitly resolved, so + # checkpoints written before Stage 4 keep the new behavior. + self.quality_gate_enabled = ( + bool(self.dispatch_contract_enabled) + if quality_gate_enabled is None + else bool(quality_gate_enabled) + ) plan_seats = list((dispatch_plan or {}).get("seat_agent_ids") or []) self.candidate_seat_agent_ids = list( (dispatch_plan or {}).get("candidate_seat_agent_ids") @@ -132,6 +172,19 @@ class RoundtableState: self.stage: str = "planning" self.partial_failure: bool = False self.cross_review_result: dict[str, Any] | None = None + # Stage 4 (R3-Q): per-task quality state machine summary. + # task_key → {status, agent_id, original_agent_id, attempt_count, + # rework_count, reassign_count, required} + self.seat_task_statuses: dict[str, Any] = {} + # Controlled-gap records to be carried into the final report's risk + # section. Each entry is a make_controlled_gap(...) dict. + self.controlled_gaps: list[dict[str, Any]] = [] + # Stage 4 (R3-Q): the ONLY seat outputs that cross-review and the + # final synthesis may consume when the quality gate is enabled. An + # entry is appended exactly when a task's final decision is + # ``accepted`` — reworked / reassigned / gap deliveries never land + # here, so unaccepted content cannot leak into the final report. + self.accepted_task_results: list[dict[str, Any]] = [] def mark_completed(self, node_id: str) -> None: if node_id not in self.completed_nodes: @@ -159,6 +212,7 @@ class RoundtableState: "intent_type": self.intent_type, "dispatch_plan": self.dispatch_plan, "dispatch_contract_enabled": self.dispatch_contract_enabled, + "quality_gate_enabled": self.quality_gate_enabled, "candidate_seat_agent_ids": list(self.candidate_seat_agent_ids), "selected_seat_agent_ids": self.selected_seat_agent_ids, "current_round": self.current_round, @@ -173,6 +227,9 @@ class RoundtableState: "stage": self.stage, "partial_failure": self.partial_failure, "cross_review_result": self.cross_review_result, + "seat_task_statuses": dict(self.seat_task_statuses), + "controlled_gaps": list(self.controlled_gaps), + "accepted_task_results": list(self.accepted_task_results), } @classmethod @@ -196,6 +253,7 @@ class RoundtableState: dispatch_plan=d.get("dispatch_plan"), candidate_seat_agent_ids=d.get("candidate_seat_agent_ids"), dispatch_contract_enabled=d.get("dispatch_contract_enabled"), + quality_gate_enabled=d.get("quality_gate_enabled"), ) state.current_round = d.get("current_round", 0) state.turn_results = d.get("turn_results", []) @@ -212,6 +270,9 @@ class RoundtableState: state.stage = d.get("stage") or "planning" state.partial_failure = bool(d.get("partial_failure", False)) state.cross_review_result = d.get("cross_review_result") + state.seat_task_statuses = dict(d.get("seat_task_statuses") or {}) + state.controlled_gaps = list(d.get("controlled_gaps") or []) + state.accepted_task_results = list(d.get("accepted_task_results") or []) state.candidate_seat_agent_ids = list( d.get("candidate_seat_agent_ids") or state.candidate_seat_agent_ids ) @@ -863,12 +924,24 @@ class SeatExecutor(Executor): agent_id: str, *, action_coordinator: ActionCoordinator | None = None, + execution_stage: str | None = None, + defer_fact_ledger_merge: bool = False, ) -> None: super().__init__(id=f"seat_{agent_id}") self._adapter = adapter self._state = state self._agent_id = agent_id self._action_coordinator = action_coordinator + # Stage 4 (R3-Q): optional override for the execution_stage passed to + # run_seat_turn. When None, falls back to the mode-based default + # ("expert" for structured, "group_chat" for group_chat). + self._execution_stage = execution_stage + # Stage 4 (R3-Q): when True the structured output is validated, tool + # results are audited and events are recorded as usual, but the output + # is NOT merged into the fact ledger. The quality loop merges it only + # if the manager's final decision accepts the delivery — reworked, + # reassigned or gapped content must never become candidate facts. + self._defer_fact_ledger_merge = defer_fact_ledger_merge @handler(input=Any, output=dict) async def handle(self, message: Any, ctx: WorkflowContext) -> None: @@ -918,7 +991,7 @@ class SeatExecutor(Executor): run_id=self._state.run_id, agent_id=self._agent_id, turn_index=self._state.current_round, - execution_stage="group_chat" if self._state.mode == "group_chat" else "expert", + execution_stage=self._execution_stage or ("group_chat" if self._state.mode == "group_chat" else "expert"), message=self._state.task_text, thread_id=thread_id, model_name=caps.get("model_name") or self._state.model_name, @@ -956,7 +1029,7 @@ class SeatExecutor(Executor): schema_result = await self._validate_output(raw_content) content = schema_result.get("raw_content") or raw_content - if schema_result.get("output"): + if schema_result.get("output") and not self._defer_fact_ledger_merge: merge_seat_output(self._state.fact_ledger, schema_result["output"], agent_id=self._agent_id) await self._capture_pending_approvals(response.tool_calls or []) @@ -981,7 +1054,7 @@ class SeatExecutor(Executor): # Phase-3 P0-1: record which execution stage produced this turn so # recovery can distinguish an expert's original output from its # format_repair / cross_review / finalizer re-runs. - "execution_stage": "group_chat" if self._state.mode == "group_chat" else "expert", + "execution_stage": self._execution_stage or ("group_chat" if self._state.mode == "group_chat" else "expert"), "content": content, "tool_calls": response.tool_calls, "error": response.error, @@ -1160,8 +1233,21 @@ class CrossReviewExecutor(Executor): self._state.stage = "review" - # Barrier: ensure every seat has an outcome (fan-in should guarantee this). - review_seats = self._state.selected_seat_agent_ids or self._state.seat_agent_ids + # P0-3: with the Stage-4 quality gate the cross-review may only see + # ACCEPTED seat results — reworked/reassigned/gapped deliveries never + # enter the review prompt (their content is already absent from the + # fact ledger because the merge was deferred until acceptance). + if self._state.quality_gate_enabled: + accepted_results = list(self._state.accepted_task_results) + review_seats = sorted({ + str(r.get("agent_id") or "") + for r in accepted_results + if r.get("agent_id") + }) + else: + accepted_results = None + # Barrier: ensure every seat has an outcome (fan-in should guarantee this). + review_seats = self._state.selected_seat_agent_ids or self._state.seat_agent_ids missing = [a for a in review_seats if a not in self._state.seat_outcomes] for aid in missing: self._state.seat_outcomes[aid] = {"status": "missing", "error": "no_outcome"} @@ -1193,9 +1279,16 @@ class CrossReviewExecutor(Executor): thread_id = self._state.thread_map.get(manager_id or "", "") if manager_id and thread_id: try: + if accepted_results is not None: + seat_results_for_review: list[dict[str, Any]] = accepted_results + else: + seat_results_for_review = [ + r for r in self._state.turn_results + if r.get("agent_id") in review_seats + ] prompt = build_manager_synthesis_input( task_text=self._state.task_text, - seat_results=[r for r in self._state.turn_results if r.get("agent_id") in review_seats], + seat_results=seat_results_for_review, fact_ledger=self._state.fact_ledger, ) # Surface the candidate ids + the evidence refs the ledger has @@ -1257,6 +1350,9 @@ class CrossReviewExecutor(Executor): or review.get("conflicts") or review.get("tool_failures") or review.get("schema_status") == "schema_invalid" + # Stage 4: a controlled gap is authoritative — the cross-review + # decision must reflect it even when everything else looks clean. + or (self._state.quality_gate_enabled and self._state.controlled_gaps) ) review["decision"] = "controlled_gap" if has_quality_gap else "accept" review["reason"] = ( @@ -1549,16 +1645,24 @@ async def generate_final_report(adapter: Any, state: RoundtableState) -> dict[st manager_id = state.manager_agent_id or "coordinator" thread_id = state.thread_map.get(manager_id, "") - usable = [ - r for r in state.turn_results - if not r.get("error") or r.get("content") - ] - if state.partial_failure: - usable = [r for r in usable if (state.seat_outcomes.get(r.get("agent_id")) or {}).get("status") == "completed"] + if state.quality_gate_enabled: + # P0-3: Stage 4 synthesizes ONLY accepted task results. There is + # deliberately NO fallback to ``turn_results`` — when no delivery was + # accepted, the report must be built from the controlled-gap risk + # section alone instead of smuggling rejected content back in. + seat_results_for_synthesis: list[dict[str, Any]] = list(state.accepted_task_results) + else: + usable = [ + r for r in state.turn_results + if not r.get("error") or r.get("content") + ] + if state.partial_failure: + usable = [r for r in usable if (state.seat_outcomes.get(r.get("agent_id")) or {}).get("status") == "completed"] + seat_results_for_synthesis = usable or state.turn_results synthesis_input = build_manager_synthesis_input( task_text=state.task_text, - seat_results=usable or state.turn_results, + seat_results=seat_results_for_synthesis, fact_ledger=state.fact_ledger, ) if state.partial_failure: @@ -1573,6 +1677,32 @@ async def generate_final_report(adapter: Any, state: RoundtableState) -> dict[st if state.cross_review_result: synthesis_input += f"\n\n【交叉审查摘要】\n{state.cross_review_result.get('summary', '')[:2000]}" + # Stage 4 (R3-Q): controlled gaps must reach the final report traceably and + # must NEVER be presented as verified facts. They are injected into the + # synthesis prompt as an explicit risk/pending-info section. + if state.controlled_gaps: + gap_lines = [ + f"- 任务 {gap.get('task_key')}（席位 {gap.get('agent_id')}，" + f"尝试 {gap.get('attempts', 0)} 次）：{gap.get('reason')}" + for gap in state.controlled_gaps + ] + synthesis_input += ( + "\n\n【受控缺口（尚未获得合格席位交付，严禁写成已验证事实）】\n" + + "\n".join(gap_lines) + + "\n要求：把上述每一条缺口完整列入最终结论的『风险/待补信息』章节，" + "逐条说明其对结论的影响；不得将缺口内容陈述为已验证结论。" + ) + + has_gaps = bool(state.controlled_gaps) + quality_decision = ( + "controlled_gap" + if has_gaps + else ( + (state.cross_review_result or {}).get("decision") + or ("controlled_gap" if state.partial_failure else "accept") + ) + ) + if thread_id: response = await adapter.run_seat_turn( session_id=state.session_id, @@ -1595,11 +1725,9 @@ async def generate_final_report(adapter: Any, state: RoundtableState) -> dict[st "content": response.content, "agent_id": manager_id, "partial_failure": state.partial_failure, - "degraded": state.partial_failure, - "quality_decision": ( - (state.cross_review_result or {}).get("decision") - or ("controlled_gap" if state.partial_failure else "accept") - ), + "degraded": state.partial_failure or has_gaps, + "quality_decision": quality_decision, + "controlled_gaps": list(state.controlled_gaps), } else: state.final_result = { @@ -1607,11 +1735,9 @@ async def generate_final_report(adapter: Any, state: RoundtableState) -> dict[st "agent_id": manager_id, "note": "no_manager_thread", "partial_failure": state.partial_failure, - "degraded": state.partial_failure, - "quality_decision": ( - (state.cross_review_result or {}).get("decision") - or ("controlled_gap" if state.partial_failure else "accept") - ), + "degraded": state.partial_failure or has_gaps, + "quality_decision": quality_decision, + "controlled_gaps": list(state.controlled_gaps), } await adapter._emit_critical(state.run_id, make_run_completed(manager_id, state.current_round + 1, { @@ -1659,6 +1785,617 @@ class WaitingTerminalExecutor(Executor): await ctx.yield_output({"status": "waiting_approval"}) +class _SilentWorkflowCtx: + """WorkflowContext stand-in for SeatExecutor invoked INSIDE the quality + loop. The embedded SeatExecutor reuses the outer executor's ctx when + called directly, which would otherwise publish its messages with the + outer node's source id and re-enter the graph. A silent ctx discards + those intermediate messages — the quality loop reads the delivery state + from ``turn_results`` / ``seat_outcomes`` instead.""" + + async def send_message(self, msg: Any) -> None: + return None + + async def yield_output(self, msg: Any) -> None: + return None + + +class WaitingInputTerminalExecutor(Executor): + """Terminal executor for the Stage-4 ask_user / waiting_input path. + + Reached when a seat task's fallback policy requests user clarification + (``ask_user``) and the manager cannot resolve the gap. The run ends in + ``waiting_input`` — never as a disguised success. + """ + + def __init__(self) -> None: + super().__init__(id="waiting_input_terminal") + + @handler(input=Any, workflow_output=dict) + async def handle(self, message: Any, ctx: WorkflowContext) -> None: + await ctx.yield_output({"status": "waiting_input"}) + + +class QualitySeatTasksExecutor(Executor): + """Stage 4 (R3-Q): run every seat task through the acceptance state machine. + + Replaces the plain SeatExecutor fan-out for V2 structured dispatch. One + executor drives all tasks in bounded quality rounds:: + + delivering → submitted → manager_review → + accepted | rework(→delivering) | reassigned(→delivering) | + controlled_gap | ask_user(→ waiting_input) + + Every delivery runs a deterministic quality check first; the manager model + then proposes one structured decision per task, and the server finalizes it + via :func:`quality_contracts.finalize_review_decision` — a delivery that + failed the deterministic checks can NEVER be accepted, reworks are capped + by the frozen contract, and an exhausted task ends as ``controlled_gap`` + (or an explicit ``ask_user`` pause) instead of being disguised as success. + + Task rows persist in ``maf_roundtable_seat_tasks``; reassignment only + changes the task's ``agent_id`` (a server-validated backup candidate) and + never creates a new task. + """ + + def __init__( + self, + adapter: Any, + state: RoundtableState, + *, + action_coordinator: ActionCoordinator | None = None, + max_rework_attempts: int = 2, + max_reassignments: int = 1, + min_delivery_chars: int = 40, + min_delivery_evidence: int = 1, + fallback_policy: str = "reassign_then_gap", + ) -> None: + super().__init__(id="quality_seat_tasks") + self._adapter = adapter + self._state = state + self._action_coordinator = action_coordinator + self._max_rework_attempts = max_rework_attempts + self._max_reassignments = max_reassignments + self._min_delivery_chars = min_delivery_chars + self._min_delivery_evidence = min_delivery_evidence + self._fallback_policy = fallback_policy + self._tasks: list[dict[str, Any]] = [] + + @property + def _store(self) -> Any: + return self._adapter._service.store + + @handler(input=Any, output=dict) + async def handle(self, message: Any, ctx: WorkflowContext) -> None: + if self._state.is_completed("quality_seat_tasks"): + await ctx.send_message({ + "tasks": dict(self._state.seat_task_statuses), + "replayed": True, + }) + return + + self._state.stage = "experts" + tasks = await self._load_or_create_tasks() + self._tasks = tasks + base_round = max(self._state.current_round, 1) + # P1: duplicate detection is seeded from the persisted task history + # (SHA-256 digests of prior conclusions), so a restart cannot re-submit + # an already-rejected conclusion and pass it off as new content. + prior_conclusions: dict[str, set[str]] = {} + for task in tasks: + digests = { + str(h.get("conclusion_sha256")) + for h in (task.get("history") or []) + if isinstance(h, dict) and h.get("conclusion_sha256") + } + prior_conclusions[str(task.get("task_key") or "")] = digests + + # Bounded quality loop: initial delivery + max_rework_attempts reworks + # per task budget, plus one delivery round per possible reassignment. + max_rounds = (self._max_rework_attempts + 1) * (self._max_reassignments + 1) + 1 + for round_offset in range(max_rounds): + self._state.current_round = base_round + round_offset + active = [t for t in tasks if t["status"] not in TASK_TERMINAL_STATUSES] + if not active or self._state.status == "waiting_input": + break + + submitted: list[dict[str, Any]] = [] + for task in active: + outcome = await self._deliver(task, prior_conclusions) + if outcome is not None: + submitted.append(outcome) + + if not submitted: + continue + + reviews = await self._manager_review(submitted) + for task in submitted: + final_outcome = await self._apply_decision(task, reviews.get(task["agent_id"])) + if final_outcome == OUTCOME_ASK_USER: + self._state.status = "waiting_input" + self._state.stage = "waiting_input" + + self._state.seat_task_statuses = { + t["task_key"]: { + "task_key": t["task_key"], + "status": t["status"], + "agent_id": t["agent_id"], + "original_agent_id": t["original_agent_id"], + "attempt_count": t["attempt_count"], + "rework_count": t["rework_count"], + "reassign_count": t["reassign_count"], + "required": bool(t["required"]), + } + for t in tasks + } + self._state.mark_completed("quality_seat_tasks") + await ctx.send_message({"tasks": dict(self._state.seat_task_statuses)}) + + # ── task init (atomic, recovery-safe) ─────────────────────────────────── + + def _expected_task_specs(self) -> list[dict[str, Any]]: + """The server-computed expected task set from the dispatch assignments. + + ``task_key`` is stable per original assignment, so recovery always + derives the identical key set — existing rows are kept, rows that a + crashed initialization never created are filled back in. + """ + plan = self._state.dispatch_plan or {} + assignments = plan.get("assignments") or [] + specs: list[dict[str, Any]] = [] + for index, assignment in enumerate(assignments, start=1): + agent_id = str(assignment.get("agent_id") or "").strip() + if not agent_id: + continue + contract = build_task_contract( + task_goal=str(assignment.get("task_goal") or ""), + delivery_requirements=list(assignment.get("delivery_requirements") or []), + required=True, + max_rework_attempts=self._max_rework_attempts, + fallback_policy=self._fallback_policy, # type: ignore[arg-type] + ) + specs.append({ + "task_key": f"{index:02d}:{agent_id}", + "agent_id": agent_id, + "contract": contract.model_dump(), + "required": True, + }) + return specs + + async def _load_or_create_tasks(self) -> list[dict[str, Any]]: + """Atomic get-or-create over the FULL expected task set (P0-4). + + Never short-circuits on "some tasks already exist": a crash mid-init + must not permanently drop the remaining required seats. Existing rows + keep their frozen contracts; concurrent recoveries converge without + unique-key errors. + """ + return await self._store.seat_tasks.get_or_create_tasks( + session_id=self._state.session_id, + run_id=self._state.run_id, + expected=self._expected_task_specs(), + ) + + # ── delivery step ─────────────────────────────────────────────────────── + + async def _deliver( + self, + task: dict[str, Any], + prior_conclusions: dict[str, set[str]], + ) -> dict[str, Any] | None: + """Run one delivery via SeatExecutor (approvals / ledger / outcomes + reuse the existing seat path), then submit it with its deterministic + check result. Returns the submitted task dict or ``None`` if the seat + run hard-failed without any output.""" + agent_id = task["agent_id"] + + # Re-delivery must not replay the completed node. + node_id = f"seat_{agent_id}" + if node_id in self._state.completed_nodes: + self._state.completed_nodes.remove(node_id) + + # Reassigned tasks: add the validated backup seat to the selected set + # and ensure its participant row + thread before it delivers. + if task["status"] == TASK_REASSIGNED: + if self._state.selected_seat_agent_ids is not None and agent_id not in self._state.selected_seat_agent_ids: + self._state.selected_seat_agent_ids.append(agent_id) + await _ensure_selected_seat_threads(self._adapter, self._state) + + is_rework = task["attempt_count"] > 0 + seat_exec = SeatExecutor( + self._adapter, + self._state, + agent_id, + action_coordinator=self._action_coordinator, + execution_stage="rework" if is_rework else None, + # P0-3: the delivery must not touch the fact ledger until the + # manager's final decision accepts it. + defer_fact_ledger_merge=True, + ) + await seat_exec.handle({}, _SilentWorkflowCtx()) + + turn_row = next( + ( + r for r in reversed(self._state.turn_results) + if r.get("agent_id") == agent_id + and r.get("turn_index") == self._state.current_round + ), + None, + ) + raw_content = str((turn_row or {}).get("content") or "") + structured_output = (turn_row or {}).get("structured_output") + schema_status = str((turn_row or {}).get("schema_status") or "ok") + delivery = delivery_from_seat_output( + structured_output=structured_output, raw_content=raw_content + ) + prior = prior_conclusions.get(task["task_key"]) + if prior is None: + prior = set() + prior_conclusions[task["task_key"]] = prior + # P1: evidence must be traceable to the fact ledger (recorded evidence + # or successful tool results). Self-made findings / invented refs fail + # the deterministic gate. + verified_refs = known_evidence_refs(self._state.fact_ledger) + check = check_delivery_deterministic( + delivery=delivery, + raw_content=raw_content, + schema_status=schema_status, + min_length=self._min_delivery_chars, + min_evidence=self._min_delivery_evidence, + prior_conclusion_digests=prior, + verified_evidence_refs=verified_refs, + ) + digest = conclusion_digest(delivery.conclusion) if delivery.conclusion else None + if digest: + prior.add(digest) + + task["attempt_count"] = int(task.get("attempt_count") or 0) + 1 + history = list(task.get("history") or []) + history.append({ + "attempt": task["attempt_count"], + "agent_id": agent_id, + "round": self._state.current_round, + "schema_status": schema_status, + "failure_codes": list(check["failure_codes"]), + # P1: persist the digest only — never the conclusion body — so + # duplicate detection survives restarts. + "conclusion_sha256": digest, + }) + # P0-3: persisted delivery envelope. ``contract_delivery`` is the + # acceptance-facing view; ``validated_seat_output`` is the schema- + # validated raw SeatTurnOutput kept back until acceptance merges it + # into the fact ledger. Runtime objects / tool-call arguments are + # never written here. + task["delivery"] = { + "contract_delivery": delivery.model_dump(), + "validated_seat_output": structured_output + if isinstance(structured_output, dict) + else None, + } + task["deterministic_check"] = check + task["history"] = history + # In-memory only (never persisted): the turn row of this attempt, so + # an acceptance can merge exactly what was submitted. + task["_turn_row"] = turn_row + await self._store.seat_tasks.update_task(task["id"], { + "status": TASK_SUBMITTED, + "attempt_count": task["attempt_count"], + "delivery": task["delivery"], + "deterministic_check": check, + "history": history, + }) + task["status"] = TASK_SUBMITTED + + await self._adapter._emit_critical(self._state.run_id, make_seat_delivery_submitted( + agent_id, + self._state.current_round, + task_key=task["task_key"], + attempt=task["attempt_count"], + deterministic_passed=bool(check["passed"]), + failure_codes=list(check["failure_codes"]), + )) + return task + + # ── manager review step ───────────────────────────────────────────────── + + async def _manager_review(self, tasks: list[dict[str, Any]]) -> dict[str, Any]: + """One batched manager review call for the round's submissions. + + Returns ``{agent_id: MafManagerReviewItem}``. Any failure (timeout, + runtime error, empty output, invalid JSON, schema failure) degrades to + an empty dict — every task then finalizes as an INVALID review + (``source=system``): a system rework while the cap allows it, or the + contract's exhaustion outcome afterwards. A delivery is never accepted + without a valid manager review, and a seat missing from a partially + valid review is treated the same way. + """ + manager_id = self._state.manager_agent_id + thread_id = self._state.thread_map.get(manager_id or "", "") + if not (manager_id and thread_id): + return {} + + review_items: list[dict[str, Any]] = [] + for task in tasks: + contract = task.get("contract") or {} + envelope = task.get("delivery") or {} + # The manager reviews the contract-facing delivery view; the raw + # validated seat output stays server-side until acceptance. + contract_delivery = envelope.get("contract_delivery") or envelope + review_items.append({ + "agent_id": task["agent_id"], + "task_key": task["task_key"], + "attempt": task["attempt_count"], + "contract": { + "objective": contract.get("objective"), + "expected_deliverable": contract.get("expected_deliverable"), + "acceptance_criteria": list(contract.get("acceptance_criteria") or []), + "required": bool(contract.get("required", True)), + }, + "delivery": contract_delivery, + "check": task.get("deterministic_check") or {}, + }) + prompt = build_manager_review_prompt( + task_text=self._state.task_text, review_items=review_items + ) + try: + response = await asyncio.wait_for( + self._adapter.run_seat_turn( + session_id=self._state.session_id, + run_id=self._state.run_id, + agent_id=manager_id, + turn_index=self._state.current_round, + execution_stage="manager_review", + message=prompt, + thread_id=thread_id, + model_name=self._state.model_name, + reasoning_effort=self._state.reasoning_effort, + task_text=prompt, + stage_instruction="返回严格的总控验收 JSON（每席位一条 decision）", + fact_ledger=self._state.fact_ledger, + capability_snapshot=self._state.capability_snapshot, + write_policy=self._state.write_policy, + wait_timeout_seconds=self._state.seat_timeout_seconds, + ), + timeout=self._state.seat_timeout_seconds, + ) + except Exception: # noqa: BLE001 - degraded to deterministic finalization + return {} + if response.error: + return {} + model, _errors = parse_manager_review(response.content or "") + if model is None: + return {} + return {item.agent_id: item for item in model.decisions} + + # ── decision application ──────────────────────────────────────────────── + + async def _apply_decision( + self, + task: dict[str, Any], + review_item: Any | None, + ) -> str: + """Finalize and persist one task's review decision; returns the outcome.""" + contract = MafTaskContract.model_validate(task.get("contract") or {}) + check = task.get("deterministic_check") or {"passed": False, "failure_codes": []} + final = finalize_review_decision( + contract=contract, + check_result=check, + review_item=review_item, + rework_count=int(task.get("rework_count") or 0), + reassign_count=int(task.get("reassign_count") or 0), + has_backup_seat=bool(self._backup_agents(task)), + max_reassignments=self._max_reassignments, + ) + outcome = final["decision"] + + await self._adapter._emit_critical(self._state.run_id, make_manager_review_decision( + task["agent_id"], + self._state.current_round, + task_key=task["task_key"], + decision=outcome, + reason=final["reason"], + source=final["source"], + attempt=task["attempt_count"], + )) + + if outcome == OUTCOME_ACCEPTED: + task["status"] = TASK_ACCEPTED + # P0-3: only an accepted delivery may enter the fact ledger and + # the accepted result set consumed by cross-review / the final + # synthesis. Reworked / reassigned / gapped attempts never do. + envelope = task.get("delivery") or {} + validated = envelope.get("validated_seat_output") + if isinstance(validated, dict): + merge_seat_output( + self._state.fact_ledger, validated, agent_id=task["agent_id"] + ) + turn_row = task.get("_turn_row") or {} + self._state.accepted_task_results.append({ + "task_key": task["task_key"], + "agent_id": task["agent_id"], + "attempt": task["attempt_count"], + "accepted": True, + "source": final["source"], + "turn_index": turn_row.get("turn_index", self._state.current_round), + "content": str(turn_row.get("content") or ""), + "execution_stage": turn_row.get("execution_stage"), + "schema_status": turn_row.get("schema_status"), + "structured_output": validated if isinstance(validated, dict) else None, + }) + await self._store.seat_tasks.update_task(task["id"], { + "status": TASK_ACCEPTED, + "review_decision": final, + }) + + elif outcome == OUTCOME_REWORK: + task["rework_count"] = int(task.get("rework_count") or 0) + 1 + task["status"] = TASK_REWORK + await self._store.seat_tasks.update_task(task["id"], { + "status": TASK_REWORK, + "rework_count": task["rework_count"], + "review_decision": final, + }) + await self._adapter._emit_critical(self._state.run_id, make_seat_rework_assigned( + task["agent_id"], + self._state.current_round, + task_key=task["task_key"], + rework_count=task["rework_count"], + requirements=list(final.get("revision_requirements") or []), + )) + + elif outcome == OUTCOME_REASSIGNED: + from_agent_id = task["agent_id"] + backup = self._pick_backup(task) + task["reassign_count"] = int(task.get("reassign_count") or 0) + 1 + task["agent_id"] = backup + task["status"] = TASK_REASSIGNED + await self._store.seat_tasks.update_task(task["id"], { + "agent_id": backup, + "status": TASK_REASSIGNED, + "reassign_count": task["reassign_count"], + "review_decision": final, + }) + await self._adapter._emit_critical(self._state.run_id, make_seat_reassigned( + backup, + self._state.current_round, + task_key=task["task_key"], + from_agent_id=from_agent_id, + reassign_count=task["reassign_count"], + )) + + elif outcome == OUTCOME_CONTROLLED_GAP: + gap = make_controlled_gap( + task_key=task["task_key"], + agent_id=task["agent_id"], + objective=contract.objective, + reason=final["reason"], + failure_codes=list(check.get("failure_codes") or []), + attempts=task["attempt_count"], + ) + task["gap"] = gap + task["status"] = TASK_CONTROLLED_GAP + self._state.controlled_gaps.append(gap) + await self._store.seat_tasks.update_task(task["id"], { + "status": TASK_CONTROLLED_GAP, + "gap": gap, + "review_decision": final, + }) + + else: # OUTCOME_ASK_USER + task["status"] = TASK_EXHAUSTED + await self._store.seat_tasks.update_task(task["id"], { + "status": TASK_EXHAUSTED, + "review_decision": final, + }) + + return outcome + + # ── backup selection (server-validated) ───────────────────────────────── + + def _backup_agents(self, task: dict[str, Any]) -> list[str]: + """Ordered backup candidates for a task. + + Candidates = server candidate seats minus every agent this task has + already used (original + rework owners) minus every agent that is the + current owner of ANY other task. A seat owns at most one task at a + time — a seat whose own task was accepted must not be double-booked as + a backup, because it would then hold two tasks and its deliveries + would collide on the (run, agent, turn, stage) idempotency key. + """ + candidates = list(self._state.candidate_seat_agent_ids or []) + used: set[str] = { + str(item.get("agent_id") or "") + for item in (task.get("history") or []) + if str(item.get("agent_id") or "") + } + used.add(str(task.get("agent_id") or "")) + for other in self._tasks: + if other.get("task_key") == task.get("task_key"): + continue + used.add(str(other.get("agent_id") or "")) + return [agent_id for agent_id in candidates if agent_id not in used] + + def _pick_backup(self, task: dict[str, Any]) -> str: + backups = self._backup_agents(task) + if not backups: + raise RuntimeError( + f"MAF quality loop requested reassignment for task " + f"{task.get('task_key')} but no validated backup seat exists" + ) + return backups[0] + + +class QualityGateExecutor(Executor): + """Stage 4 (R3-Q): phase quality gate between seat tasks and cross-review. + + All required seat tasks must have reached a terminal state (``accepted`` + or ``controlled_gap``) before the phase may proceed. A required task left + non-terminal blocks the phase deterministically (hard run failure — the + quality loop guarantees terminal states, so a blocked gate is a safety + net, never a normal outcome). Controlled gaps do NOT block the phase: + they proceed as ``passed_with_gaps`` and are carried into the final + report's risk section. + """ + + def __init__(self, adapter: Any, state: RoundtableState) -> None: + super().__init__(id="quality_gate") + self._adapter = adapter + self._state = state + + @handler(input=Any, output=dict) + async def handle(self, message: Any, ctx: WorkflowContext) -> None: + if self._state.is_completed("quality_gate"): + await ctx.send_message({"decision": "passed", "replayed": True}) + return + + # ask_user path: route to the waiting_input terminal, never to + # cross-review / finalizer. + if self._state.status == "waiting_input": + self._state.mark_completed("quality_gate") + await ctx.send_message({"decision": "waiting_input"}) + return + + statuses = self._state.seat_task_statuses + required = {k: v for k, v in statuses.items() if v.get("required")} + non_terminal = [ + k for k, v in required.items() if v.get("status") not in TASK_TERMINAL_STATUSES + ] + # P0-4 safety net: every dispatch assignment must be represented by a + # task row. A missing required task blocks the gate exactly like a + # non-terminal one — the gate must never pass on a dropped seat. + expected_keys: list[str] = [] + plan = self._state.dispatch_plan or {} + for index, assignment in enumerate(plan.get("assignments") or [], start=1): + agent_id = str(assignment.get("agent_id") or "").strip() + if agent_id: + expected_keys.append(f"{index:02d}:{agent_id}") + missing = [k for k in expected_keys if k not in statuses] + blocked = sorted({*non_terminal, *missing}) + accepted = sum(1 for v in statuses.values() if v.get("status") == TASK_ACCEPTED) + gap_count = sum(1 for v in statuses.values() if v.get("status") == TASK_CONTROLLED_GAP) + decision = "blocked" if blocked else ("passed_with_gaps" if gap_count else "passed") + + await self._adapter._emit_critical(self._state.run_id, make_stage_quality_gate( + self._state.manager_agent_id, + self._state.current_round, + decision=decision, + total=len(statuses), + accepted=accepted, + controlled_gap=gap_count, + blocked_required=blocked, + )) + + if blocked: + raise RuntimeError( + f"MAF quality gate blocked: required seat tasks not terminal or missing: " + f"{', '.join(blocked)}" + ) + + self._state.mark_completed("quality_gate") + self._state.stage = "review" + await ctx.send_message({"decision": decision, "tasks": dict(statuses)}) + + # ── helpers ───────────────────────────────────────────────────────────────── diff --git a/offline-backend-20260512/backend/app/maf_roundtable/workflow_factory.py b/offline-backend-20260512/backend/app/maf_roundtable/workflow_factory.py index 6a9f2e6b..2863b870 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/workflow_factory.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/workflow_factory.py @@ -55,11 +55,19 @@ def build_structured_workflow( dispatch_seat_entry: Any | None = None, manager_answer: Any | None = None, clarification_terminal: Any | None = None, + quality_gate: Any | None = None, + waiting_input_terminal: Any | None = None, ) -> Any: """Build a structured (fixed-topology) MAF Workflow. - Topology: planner → parallel seat executors → cross_review → approval gate - → finalizer | waiting_terminal. + Topology (Stage 3): planner → parallel seat executors → cross_review → + approval gate → finalizer | waiting_terminal. + + Topology (Stage 4 quality gate): quality seat-tasks loop → quality gate → + (waiting_input | cross_review) → approval gate → finalizer | waiting_terminal. + When ``quality_gate`` is provided, the seat fan-out is replaced by the + single quality loop executor and ``waiting_input_terminal`` becomes an + additional workflow output for the ``ask_user`` pause path. """ from agent_framework import Case, Default, WorkflowBuilder @@ -71,6 +79,8 @@ def build_structured_workflow( outputs.append(manager_answer) if clarification_terminal is not None: outputs.append(clarification_terminal) + if waiting_input_terminal is not None: + outputs.append(waiting_input_terminal) builder = WorkflowBuilder( start_executor=start, name="maf-structured-roundtable", @@ -97,7 +107,20 @@ def build_structured_workflow( builder.add_switch_case_edge_group(dispatch_router, cases) seat_start = dispatch_seat_entry - if execution_mode == "chain" and seat_executors: + if quality_gate is not None: + # Stage 4: single quality-loop node replaces the seat fan-out; the + # quality gate routes ask_user pauses to waiting_input_terminal and + # everything else to cross-review. + builder.add_edge(seat_start, quality_gate) + gate_cases = [] + if waiting_input_terminal is not None: + gate_cases.append(Case( + condition=lambda out: isinstance(out, dict) and out.get("decision") == "waiting_input", + target=waiting_input_terminal, + )) + gate_cases.append(Default(target=cross_review)) + builder.add_switch_case_edge_group(quality_gate, gate_cases) + elif execution_mode == "chain" and seat_executors: builder.add_edge(seat_start, seat_executors[0]) for previous, current in zip(seat_executors, seat_executors[1:]): builder.add_edge(previous, current) @@ -249,6 +272,8 @@ def create_workflow( dispatch_router: Any | None = None, dispatch_seat_entry: Any | None = None, clarification_terminal: Any | None = None, + quality_gate: Any | None = None, + waiting_input_terminal: Any | None = None, ) -> Any: """Factory: create the appropriate workflow for the given mode.""" if mode == "manager_only": @@ -290,4 +315,6 @@ def create_workflow( dispatch_seat_entry=dispatch_seat_entry, manager_answer=manager_answer, clarification_terminal=clarification_terminal, + quality_gate=quality_gate, + waiting_input_terminal=waiting_input_terminal, ) diff --git a/offline-backend-20260512/backend/app/maf_roundtable/workflow_runner.py b/offline-backend-20260512/backend/app/maf_roundtable/workflow_runner.py index c93f5c41..e39f8c31 100644 --- a/offline-backend-20260512/backend/app/maf_roundtable/workflow_runner.py +++ b/offline-backend-20260512/backend/app/maf_roundtable/workflow_runner.py @@ -36,8 +36,11 @@ from app.maf_roundtable.workflow_executors import ( ManagerAnswerExecutor, ManagerDispatchExecutor, PlannerExecutor, + QualityGateExecutor, + QualitySeatTasksExecutor, RoundtableState, SeatExecutor, + WaitingInputTerminalExecutor, WaitingTerminalExecutor, _agent_capabilities_from_snapshot, generate_final_report, @@ -165,6 +168,7 @@ class WorkflowRunner: or candidate_set or maf_v2_payload.get("dispatch_plan") is not None ), + quality_gate_enabled=self._resolve_quality_gate_enabled(maf_v2_payload), ) if state.dispatch_contract_enabled: @@ -220,6 +224,54 @@ class WorkflowRunner: except Exception: # noqa: BLE001 return 600.0, True + def _resolve_quality_gate_enabled(self, maf_v2_payload: dict[str, Any]) -> bool: + """Resolve the per-run quality gate flag. + + A boolean frozen in the session's ``maf_v2`` payload wins (sessions + created before Stage 4 can opt out explicitly); otherwise the service + config default applies. + """ + payload_flag = maf_v2_payload.get("quality_gate_enabled") + if isinstance(payload_flag, bool): + return payload_flag + return bool(self._quality_settings()["enabled"]) + + @staticmethod + def _cfg_int(cfg: Any, name: str, default: int) -> int: + """Config int where an explicit 0 is honoured (only None falls back). + + ``int(getattr(...) or default)`` would silently turn a deliberate + ``max_rework_attempts: 0`` back into the default — zero must mean zero. + """ + value = getattr(cfg, name, None) + return default if value is None else int(value) + + def _quality_settings(self) -> dict[str, Any]: + try: + from deerflow.config import get_app_config + + cfg = get_app_config().maf_roundtable + return { + "enabled": bool(getattr(cfg, "quality_gate_enabled", True)), + "max_rework_attempts": self._cfg_int(cfg, "max_rework_attempts", 2), + "max_reassignments": self._cfg_int(cfg, "max_reassignments", 1), + "min_delivery_chars": self._cfg_int(cfg, "min_delivery_chars", 40), + "min_delivery_evidence": self._cfg_int(cfg, "min_delivery_evidence", 1), + "fallback_policy": str( + getattr(cfg, "default_fallback_policy", None) + or "reassign_then_gap" + ), + } + except Exception: # noqa: BLE001 + return { + "enabled": True, + "max_rework_attempts": 2, + "max_reassignments": 1, + "min_delivery_chars": 40, + "min_delivery_evidence": 1, + "fallback_policy": "reassign_then_gap", + } + def _participants_requiring_threads( self, participants: list[dict[str, Any]], @@ -344,6 +396,37 @@ class WorkflowRunner: state.current_round = max(state.current_round, 1) cross_review = CrossReviewExecutor(self._adapter, state) + + # Stage 4 (R3-Q): the quality seat-task loop + phase gate replace + # the plain seat fan-out for V2 structured dispatch. + if state.quality_gate_enabled: + q = self._quality_settings() + quality_seats = QualitySeatTasksExecutor( + self._adapter, + state, + action_coordinator=self._action_coordinator, + max_rework_attempts=q["max_rework_attempts"], + max_reassignments=q["max_reassignments"], + min_delivery_chars=q["min_delivery_chars"], + min_delivery_evidence=q["min_delivery_evidence"], + fallback_policy=q["fallback_policy"], + ) + quality_gate = QualityGateExecutor(self._adapter, state) + waiting_input_terminal = WaitingInputTerminalExecutor() + return create_workflow( + mode="structured", + planner=quality_seats, + cross_review=cross_review, + seat_executors=[], + approval_gate=approval_gate, + finalizer=finalizer, + waiting_terminal=waiting_terminal, + seat_agent_ids=[], + execution_mode=execution_mode, + quality_gate=quality_gate, + waiting_input_terminal=waiting_input_terminal, + ) + return create_workflow( mode="structured", planner=DispatchSeatEntryExecutor(state), diff --git a/offline-backend-20260512/backend/packages/harness/deerflow/config/maf_roundtable_config.py b/offline-backend-20260512/backend/packages/harness/deerflow/config/maf_roundtable_config.py index dad6a619..07344342 100644 --- a/offline-backend-20260512/backend/packages/harness/deerflow/config/maf_roundtable_config.py +++ b/offline-backend-20260512/backend/packages/harness/deerflow/config/maf_roundtable_config.py @@ -220,6 +220,39 @@ class MafRoundtableConfig(BaseModel): le=60, description="后台自动 reconcile 扫描间隔（秒）；补偿投影失败的审计事件。", ) + # Stage 4 (R3-Q): seat task quality contracts + manager acceptance gate. + quality_gate_enabled: bool = Field( + default=True, + description="V2 结构化派活是否启用席位任务合同与总控验收闸门；关闭时退化为 Stage 3 行为。", + ) + max_rework_attempts: int = Field( + default=2, + ge=0, + le=8, + description="同一席位任务最多返工次数（初稿之外）；到达上限后按 fallback_policy 升级。", + ) + max_reassignments: int = Field( + default=1, + ge=0, + le=4, + description="同一席位任务最多改派次数（备用席位接管）；到达上限后受控降级为 controlled_gap。", + ) + min_delivery_chars: int = Field( + default=40, + ge=0, + le=10000, + description="席位交付结论(summary)的最小有效长度；低于此值触发 below_min_length 确定性失败。", + ) + min_delivery_evidence: int = Field( + default=1, + ge=0, + le=16, + description="席位交付必须提供的最少证据/依据条目；低于此值触发 missing_evidence 确定性失败。", + ) + default_fallback_policy: Literal["reassign_then_gap", "controlled_gap", "ask_user"] = Field( + default="reassign_then_gap", + description="到达返工/改派上限后的默认降级策略：改派后受控缺口 / 直接受控缺口 / 请求用户澄清。", + ) @field_validator("default_manager_agent_id", mode="before") diff --git a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/model.py b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/model.py index 00df93f8..c9521a72 100644 --- a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/model.py +++ b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/model.py @@ -31,6 +31,7 @@ from __future__ import annotations from datetime import UTC, datetime from sqlalchemy import ( + Boolean, CheckConstraint, ForeignKey, Integer, @@ -368,7 +369,8 @@ class MafRoundtableSeatRunRow(Base): # migration / manual SQL honest. Keep in sync with VALID_EXECUTION_STAGES. CheckConstraint( "execution_stage IN ('legacy','planner','expert','format_repair'," - "'cross_review','cross_review_repair','finalizer','group_chat')", + "'cross_review','cross_review_repair','finalizer','group_chat'," + "'rework','manager_review')", name="ck_maf_seat_runs_execution_stage", ), ) @@ -406,6 +408,64 @@ class MafRoundtableSeatRunRow(Base): ) +# ── 8b. seat tasks (Stage 4 / R3-Q quality state machine) ───────────────── + + +class MafRoundtableSeatTaskRow(Base): + """One seat task with its frozen contract and quality lifecycle. + + Stage 4 (R3-Q): each manager dispatch assignment becomes one seat task. + The row persists the whole acceptance state machine + (``assigned → delivering → submitted → manager_review → + accepted / rework / reassigned / exhausted / controlled_gap``) together + with the frozen ``contract_json``, the latest ``delivery_json``, + ``deterministic_check_json``, ``review_decision_json`` and a per-attempt + ``history_json``. When every rework/reassign cap is exhausted the task + ends as ``controlled_gap`` with a ``gap_json`` risk record that the final + report must carry — it is never disguised as an accepted delivery. + + ``task_key`` is stable per (run, original assignment); reassignment only + changes ``agent_id`` and never creates a new task. + """ + + __tablename__ = "maf_roundtable_seat_tasks" + __table_args__ = ( + UniqueConstraint("run_id", "task_key", name="uq_maf_seat_tasks_run_task"), + CheckConstraint( + "status IN ('assigned','delivering','submitted','manager_review'," + "'accepted','rework','reassigned','exhausted','controlled_gap')", + name="ck_maf_seat_tasks_status", + ), + ) + + id: Mapped[str] = mapped_column(String(64), primary_key=True) + session_id: Mapped[str] = mapped_column( + String(64), ForeignKey("maf_roundtable_sessions.id", ondelete="CASCADE"), nullable=False, index=True + ) + run_id: Mapped[str] = mapped_column( + String(64), ForeignKey("maf_roundtable_runs.id", ondelete="CASCADE"), nullable=False, index=True + ) + task_key: Mapped[str] = mapped_column(String(128), nullable=False) + # Current owning seat (changes on reassignment). + agent_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True) + original_agent_id: Mapped[str] = mapped_column(String(64), nullable=False) + status: Mapped[str] = mapped_column(String(32), default="assigned", index=True) + required: Mapped[bool] = mapped_column(Boolean, default=True) + attempt_count: Mapped[int] = mapped_column(Integer, default=0) + rework_count: Mapped[int] = mapped_column(Integer, default=0) + reassign_count: Mapped[int] = mapped_column(Integer, default=0) + contract_json: Mapped[dict | None] = mapped_column(PortableJSON(), nullable=True) + delivery_json: Mapped[dict | None] = mapped_column(PortableJSON(), nullable=True) + deterministic_check_json: Mapped[dict | None] = mapped_column(PortableJSON(), nullable=True) + review_decision_json: Mapped[dict | None] = mapped_column(PortableJSON(), nullable=True) + history_json: Mapped[list | None] = mapped_column(PortableJSON(), nullable=True) + gap_json: Mapped[dict | None] = mapped_column(PortableJSON(), nullable=True) + created_at: Mapped[datetime] = mapped_column(BeijingDateTime(), nullable=False, default=_now) + updated_at: Mapped[datetime] = mapped_column( + BeijingDateTime(), nullable=False, default=_now, onupdate=_now + ) + + # ── 9. action secrets (encrypted actual execution params, server-only) ──── @@ -576,5 +636,6 @@ __all__ = [ "MafRoundtableParticipantRow", "MafRoundtableRunRow", "MafRoundtableSeatRunRow", + "MafRoundtableSeatTaskRow", "MafRoundtableSessionRow", ] diff --git a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_runs.py b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_runs.py index 8bfa1faa..394f957e 100644 --- a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_runs.py +++ b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_runs.py @@ -36,6 +36,11 @@ CROSS_REVIEW = "cross_review" CROSS_REVIEW_REPAIR = "cross_review_repair" FINALIZER = "finalizer" GROUP_CHAT = "group_chat" +# Stage 4 (R3-Q) quality-loop stages. Rework deliveries run at an advanced +# business round (one round per quality iteration), so the four-tuple +# (run, agent, turn, stage) stays unique across attempts. +REWORK = "rework" +MANAGER_REVIEW = "manager_review" VALID_EXECUTION_STAGES: frozenset[str] = frozenset({ LEGACY, @@ -46,6 +51,8 @@ VALID_EXECUTION_STAGES: frozenset[str] = frozenset({ CROSS_REVIEW_REPAIR, FINALIZER, GROUP_CHAT, + REWORK, + MANAGER_REVIEW, }) diff --git a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_tasks.py b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_tasks.py new file mode 100644 index 00000000..595dc36e --- /dev/null +++ b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/seat_tasks.py @@ -0,0 +1,232 @@ +"""Seat task repository — Stage 4 (R3-Q) quality state machine persistence. + +Each manager dispatch assignment becomes one seat task row. The row carries the +frozen ``contract_json`` and the full acceptance lifecycle (delivery, deterministic +check, manager review decision, rework / reassign counts, controlled-gap record). + +``task_key`` is stable per (run, original assignment); reassignment only mutates +``agent_id`` and never creates a new row, so the (run_id, task_key) unique +constraint keeps a task single even across backup-seat handovers. +""" + +from __future__ import annotations + +import logging +from datetime import UTC, datetime +from typing import Any +from uuid import uuid4 + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker + +from deerflow.persistence.maf_roundtable.model import MafRoundtableSeatTaskRow + +logger = logging.getLogger(__name__) + + +def _now() -> datetime: + return datetime.now(UTC) + + +def _to_dict(row: MafRoundtableSeatTaskRow) -> dict[str, Any]: + return { + "id": row.id, + "session_id": row.session_id, + "run_id": row.run_id, + "task_key": row.task_key, + "agent_id": row.agent_id, + "original_agent_id": row.original_agent_id, + "status": row.status, + "required": bool(row.required), + "attempt_count": row.attempt_count, + "rework_count": row.rework_count, + "reassign_count": row.reassign_count, + "contract": row.contract_json, + "delivery": row.delivery_json, + "deterministic_check": row.deterministic_check_json, + "review_decision": row.review_decision_json, + "history": row.history_json or [], + "gap": row.gap_json, + "created_at": row.created_at.isoformat() if isinstance(row.created_at, datetime) else row.created_at, + "updated_at": row.updated_at.isoformat() if isinstance(row.updated_at, datetime) else row.updated_at, + } + + +# Column map for partial updates (public field name → ORM column). +_FIELD_MAP = { + "agent_id": "agent_id", + "status": "status", + "required": "required", + "attempt_count": "attempt_count", + "rework_count": "rework_count", + "reassign_count": "reassign_count", + "contract": "contract_json", + "delivery": "delivery_json", + "deterministic_check": "deterministic_check_json", + "review_decision": "review_decision_json", + "history": "history_json", + "gap": "gap_json", +} + + +class MafSeatTaskRepository: + def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None: + self._sf = session_factory + + async def create_task( + self, + *, + session_id: str, + run_id: str, + task_key: str, + agent_id: str, + contract: dict[str, Any], + required: bool = True, + ) -> dict[str, Any]: + """Create a single seat task in ``assigned`` status with its frozen + contract. Prefer :meth:`get_or_create_tasks` for run initialization — + it is atomic across the whole dispatch assignment set.""" + row = self._new_row( + session_id=session_id, + run_id=run_id, + task_key=task_key, + agent_id=agent_id, + contract=contract, + required=required, + ) + async with self._sf() as session: + session.add(row) + await session.commit() + return _to_dict(row) + + def _new_row( + self, + *, + session_id: str, + run_id: str, + task_key: str, + agent_id: str, + contract: dict[str, Any], + required: bool, + ) -> MafRoundtableSeatTaskRow: + now = _now() + return MafRoundtableSeatTaskRow( + id=uuid4().hex, + session_id=session_id, + run_id=run_id, + task_key=task_key, + agent_id=agent_id, + original_agent_id=agent_id, + status="assigned", + required=required, + attempt_count=0, + rework_count=0, + reassign_count=0, + contract_json=contract, + history_json=[], + created_at=now, + updated_at=now, + ) + + async def get_or_create_tasks( + self, + *, + session_id: str, + run_id: str, + expected: list[dict[str, Any]], + ) -> list[dict[str, Any]]: + """Return the COMPLETE task set for a run (P0-4 atomic init/recovery). + + ``expected`` items: ``{task_key, agent_id, contract, required}`` — the + server-computed task set derived from the dispatch assignments. + + Semantics: + - existing rows are preserved untouched (their frozen contract, counts + and status are NEVER overwritten); + - missing ``task_key`` rows are created; + - the full set is returned, ordered by ``task_key``; + - concurrent recovery calls never surface a unique-key violation: on a + (run_id, task_key) race the loser retries and reads the winner's rows. + """ + expected_items = [ + item for item in (expected or []) + if str(item.get("task_key") or "") and str(item.get("agent_id") or "") + ] + for _attempt in range(3): + try: + async with self._sf() as session: + async with session.begin(): + rows = ( + await session.execute( + select(MafRoundtableSeatTaskRow).where( + MafRoundtableSeatTaskRow.run_id == run_id + ) + ) + ).scalars().all() + existing_keys = {r.task_key for r in rows} + for item in expected_items: + if item["task_key"] in existing_keys: + continue + session.add(self._new_row( + session_id=session_id, + run_id=run_id, + task_key=str(item["task_key"]), + agent_id=str(item["agent_id"]), + contract=dict(item.get("contract") or {}), + required=bool(item.get("required", True)), + )) + break # commit succeeded + except IntegrityError: + # A concurrent recovery created the same task(s) first. Retry: + # the re-read finds them and skips creation. + logger.info( + "seat task init race for run %s — retrying (attempt %d)", + run_id, _attempt + 1, + ) + continue + return await self.list_by_run(run_id) + + async def get_task(self, task_id: str) -> dict[str, Any] | None: + async with self._sf() as session: + stmt = select(MafRoundtableSeatTaskRow).where( + MafRoundtableSeatTaskRow.id == task_id + ).limit(1) + row = (await session.execute(stmt)).scalars().first() + return _to_dict(row) if row is not None else None + + async def get_by_run_task(self, run_id: str, task_key: str) -> dict[str, Any] | None: + async with self._sf() as session: + stmt = select(MafRoundtableSeatTaskRow).where( + MafRoundtableSeatTaskRow.run_id == run_id, + MafRoundtableSeatTaskRow.task_key == task_key, + ).limit(1) + row = (await session.execute(stmt)).scalars().first() + return _to_dict(row) if row is not None else None + + async def list_by_run(self, run_id: str) -> list[dict[str, Any]]: + async with self._sf() as session: + stmt = ( + select(MafRoundtableSeatTaskRow) + .where(MafRoundtableSeatTaskRow.run_id == run_id) + .order_by(MafRoundtableSeatTaskRow.task_key.asc()) + ) + rows = (await session.execute(stmt)).scalars().all() + return [_to_dict(r) for r in rows] + + async def update_task(self, task_id: str, fields: dict[str, Any]) -> dict[str, Any] | None: + """Partial update using the public field names (see ``_FIELD_MAP``).""" + async with self._sf() as session: + stmt = select(MafRoundtableSeatTaskRow).where( + MafRoundtableSeatTaskRow.id == task_id + ).limit(1) + row = (await session.execute(stmt)).scalars().first() + if row is None: + return None + for key, value in fields.items(): + col = _FIELD_MAP.get(key) + if col is not None: + setattr(row, col, value) + row.updated_at = _now() + await session.commit() + return _to_dict(row) diff --git a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/store.py b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/store.py index 973ff3ab..feb9a768 100644 --- a/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/store.py +++ b/offline-backend-20260512/backend/packages/harness/deerflow/persistence/maf_roundtable/store.py @@ -49,6 +49,7 @@ from deerflow.persistence.maf_roundtable.model import ( from deerflow.persistence.maf_roundtable.participants import MafParticipantRepository from deerflow.persistence.maf_roundtable.runs import MafRunRepository, run_to_dict from deerflow.persistence.maf_roundtable.seat_runs import MafSeatRunRepository +from deerflow.persistence.maf_roundtable.seat_tasks import MafSeatTaskRepository from deerflow.persistence.maf_roundtable.sessions import MafSessionRepository # Session statuses that mean a run is in flight — the CAS claim refuses these. @@ -80,6 +81,7 @@ class MafRoundtableStore: self.actions = MafActionRepository(session_factory) self.approvals = MafApprovalRepository(session_factory) self.seat_runs = MafSeatRunRepository(session_factory) + self.seat_tasks = MafSeatTaskRepository(session_factory) self.action_secrets = MafActionSecretRepository(session_factory) # V2 conversation aggregation (user-visible layer). self.conversations = MafConversationRepository(session_factory) diff --git a/offline-backend-20260512/backend/scripts/migrate_maf_roundtable_stage4.py b/offline-backend-20260512/backend/scripts/migrate_maf_roundtable_stage4.py new file mode 100644 index 00000000..afd9ef34 --- /dev/null +++ b/offline-backend-20260512/backend/scripts/migrate_maf_roundtable_stage4.py @@ -0,0 +1,460 @@ +"""Stage 4 migration for the MAF roundtable experiment (R3-Q quality gate). + +Idempotent migration that brings a Stage-3 deployment up to Stage-4 schema: + +1. Create the ``maf_roundtable_seat_tasks`` table (seat task contracts + + quality state machine rows) with its unique (run_id, task_key) constraint + and closed status CHECK constraint. +2. Extend the ``maf_roundtable_seat_runs.execution_stage`` CHECK constraint to + allow the two new Stage-4 stages ``rework`` and ``manager_review``. + +Dialect handling +---------------- +- **PostgreSQL / MySQL**: ``CREATE TABLE IF NOT EXISTS`` + ``ALTER TABLE`` to + swap the execution_stage CHECK. Each step is guarded by introspection so + re-runs are no-ops. +- **SQLite**: a CHECK constraint cannot be dropped in place, so the seat_runs + table is rebuilt (same recipe as the Phase-3 migration): a temp table with + the widened CHECK is created, data is copied as-is, the old table is + dropped and the temp renamed. Fully idempotent — the rebuild only runs when + the widened CHECK is absent. + +**Deployment order**: run AFTER the Phase-3 migration and BEFORE enabling the +Stage-4 code path. + +Usage (from ``offline-backend-20260512/backend/``):: + + # Preview (no writes) + PYTHONPATH=. uv run python scripts/migrate_maf_roundtable_stage4.py --dry-run + + # Execute + PYTHONPATH=. uv run python scripts/migrate_maf_roundtable_stage4.py +""" + +from __future__ import annotations + +import argparse +import asyncio +import logging +import sys +from typing import Any + +logging.basicConfig(level=logging.INFO, format="%(message)s") +logger = logging.getLogger("migrate_maf_roundtable_stage4") + +SEAT_TASKS_TABLE = "maf_roundtable_seat_tasks" +SEAT_RUNS_TABLE = "maf_roundtable_seat_runs" +EXECUTION_STAGE_CHECK_NAME = "ck_maf_seat_runs_execution_stage" +SEAT_TASKS_STATUS_CHECK_NAME = "ck_maf_seat_tasks_status" +STAGE4_STAGES = "'legacy','planner','expert','format_repair'," \ + "'cross_review','cross_review_repair','finalizer','group_chat'," \ + "'rework','manager_review'" +# The two stages Stage 4 adds on top of the Stage-3 enum. A constraint that +# does not contain BOTH of them is still a Stage-3 constraint, even when it +# carries the same constraint name. +STAGE4_NEW_STAGES = ("rework", "manager_review") +SEAT_TASKS_STATUSES = ( + "assigned", "delivering", "submitted", "manager_review", + "accepted", "rework", "reassigned", "exhausted", "controlled_gap", +) + + +def _dialect_name(conn_or_dialect: Any) -> str: + return str(getattr(getattr(conn_or_dialect, "dialect", conn_or_dialect), "name", "sqlite")).lower() + + +def _timestamp_sql(dialect: str) -> str: + if dialect == "postgresql": + return "TIMESTAMP WITH TIME ZONE" + if dialect in {"mysql", "mariadb"}: + return "DATETIME(6)" + return "DATETIME" + + +def _json_sql(dialect: str) -> str: + if dialect == "postgresql": + return "JSONB" + if dialect in {"mysql", "mariadb"}: + return "JSON" + return "TEXT" + + +def _boolean_default_sql(dialect: str) -> str: + """Dialect-correct literal for a BOOLEAN column default. + + PostgreSQL has a real BOOLEAN type whose literal is ``TRUE`` — ``DEFAULT 1`` + is rejected there. SQLite/MySQL store booleans as integers. + """ + if dialect == "postgresql": + return "TRUE" + return "1" + + +def _seat_tasks_ddl(dialect: str) -> str: + ts = _timestamp_sql(dialect) + js = _json_sql(dialect) + bool_default = _boolean_default_sql(dialect) + return f""" + CREATE TABLE {SEAT_TASKS_TABLE} ( + id VARCHAR(64) NOT NULL PRIMARY KEY, + session_id VARCHAR(64) NOT NULL, + run_id VARCHAR(64) NOT NULL, + task_key VARCHAR(128) NOT NULL, + agent_id VARCHAR(64) NOT NULL, + original_agent_id VARCHAR(64) NOT NULL, + status VARCHAR(32) NOT NULL DEFAULT 'assigned', + required BOOLEAN NOT NULL DEFAULT {bool_default}, + attempt_count INTEGER NOT NULL DEFAULT 0, + rework_count INTEGER NOT NULL DEFAULT 0, + reassign_count INTEGER NOT NULL DEFAULT 0, + contract_json {js}, + delivery_json {js}, + deterministic_check_json {js}, + review_decision_json {js}, + history_json {js}, + gap_json {js}, + created_at {ts} NOT NULL, + updated_at {ts} NOT NULL, + CONSTRAINT uq_maf_seat_tasks_run_task UNIQUE (run_id, task_key), + CONSTRAINT ck_maf_seat_tasks_status CHECK ( + status IN ('assigned','delivering','submitted','manager_review', + 'accepted','rework','reassigned','exhausted','controlled_gap') + ), + FOREIGN KEY (session_id) REFERENCES maf_roundtable_sessions (id) ON DELETE CASCADE, + FOREIGN KEY (run_id) REFERENCES maf_roundtable_runs (id) ON DELETE CASCADE + ) + """ + + +def _seat_tasks_indexes_ddl(dialect: str) -> list[str]: + return [ + f"CREATE INDEX ix_maf_seat_tasks_session ON {SEAT_TASKS_TABLE} (session_id)", + f"CREATE INDEX ix_maf_seat_tasks_run ON {SEAT_TASKS_TABLE} (run_id)", + f"CREATE INDEX ix_maf_seat_tasks_agent ON {SEAT_TASKS_TABLE} (agent_id)", + f"CREATE INDEX ix_maf_seat_tasks_status ON {SEAT_TASKS_TABLE} (status)", + ] + + +def _ddl_add_execution_stage_check(dialect: str) -> str: + if dialect in {"mysql", "mariadb"}: + return ( + f"ALTER TABLE {SEAT_RUNS_TABLE} " + f"ADD CONSTRAINT ck_maf_seat_runs_execution_stage " + f"CHECK (execution_stage IN ({STAGE4_STAGES}))" + ) + return ( + f"ALTER TABLE {SEAT_RUNS_TABLE} " + f"ADD CONSTRAINT ck_maf_seat_runs_execution_stage " + f"CHECK (execution_stage IN ({STAGE4_STAGES}))" + ) + + +def _ddl_drop_execution_stage_check(dialect: str) -> str: + if dialect in {"mysql", "mariadb"}: + return f"ALTER TABLE {SEAT_RUNS_TABLE} DROP CHECK ck_maf_seat_runs_execution_stage" + return f"ALTER TABLE {SEAT_RUNS_TABLE} DROP CONSTRAINT ck_maf_seat_runs_execution_stage" + + +def _check_constraint_sql(insp: Any, table: str, name: str) -> str | None: + """Return the SQL text of a CHECK constraint, or ``None`` when absent. + + The Stage-3 migration created a constraint with the SAME name + (``ck_maf_seat_runs_execution_stage``) but the old stage list, so the name + alone can never prove a table is already on Stage 4 — only the SQL text can. + """ + getter = getattr(insp, "get_check_constraints", None) + if getter is None: + return None + try: + for ck in getter(table): + if ck.get("name") == name: + return str(ck.get("sqltext") or "") + except Exception: # noqa: BLE001 + return None + return None + + +def _execution_stage_check_sql(insp: Any) -> str | None: + """SQL text of the seat_runs ``execution_stage`` CHECK, or ``None``.""" + return _check_constraint_sql(insp, SEAT_RUNS_TABLE, EXECUTION_STAGE_CHECK_NAME) + + +def _has_stage4_execution_stage_check(insp: Any) -> bool: + """True only when the constraint exists AND already allows BOTH Stage-4 stages. + + A same-named Stage-3 constraint (no ``rework`` / ``manager_review``) is + treated as the old constraint and must be widened — never reported as + "already widened". + """ + sql = _execution_stage_check_sql(insp) + if sql is None: + return False + return all(stage in sql for stage in STAGE4_NEW_STAGES) + + +def _widen_check_ddls(dialect: str, *, check_exists: bool, is_stage4: bool) -> list[str]: + """Pure decision function: DDL needed to widen the execution_stage CHECK. + + - already Stage 4 → no-op; + - SQLite → rebuild the table (a CHECK cannot be dropped in place), whether + the old constraint exists or not; + - PostgreSQL / MySQL → drop the old constraint first when it exists, then + add the Stage-4 constraint; when it does not exist, just add it. + + Kept free of I/O so the PostgreSQL / MySQL branches are unit-testable + without a live database. + """ + if is_stage4: + return [] + if dialect == "sqlite": + return _sqlite_rebuild_seat_runs_with_stage4() + ddls: list[str] = [] + if check_exists: + ddls.append(_ddl_drop_execution_stage_check(dialect)) + ddls.append(_ddl_add_execution_stage_check(dialect)) + return ddls + + +def _sqlite_rebuild_seat_runs_with_stage4() -> list[str]: + """Rebuild seat_runs with the widened execution_stage CHECK (SQLite only). + + Same recipe as the Phase-3 migration: no column shape changes, only the + CHECK constraint is widened, so the data copy is a straight ``SELECT *``. + """ + return [ + f""" + CREATE TABLE {SEAT_RUNS_TABLE}__stage4 ( + id VARCHAR(64) NOT NULL PRIMARY KEY, + maf_run_id VARCHAR(64) NOT NULL, + agent_id VARCHAR(64) NOT NULL, + turn_index INTEGER NOT NULL, + execution_stage VARCHAR(32) NOT NULL DEFAULT 'legacy', + deerflow_run_id VARCHAR(64), + deerflow_thread_id VARCHAR(64), + sandbox_id VARCHAR(64), + workflow_checkpoint_id VARCHAR(128), + status VARCHAR(32) DEFAULT 'pending' NOT NULL, + result_snapshot TEXT, + started_at DATETIME, + finished_at DATETIME, + error TEXT, + created_at DATETIME NOT NULL, + updated_at DATETIME NOT NULL, + FOREIGN KEY (maf_run_id) REFERENCES maf_roundtable_runs (id) ON DELETE CASCADE, + CONSTRAINT uq_maf_seat_runs_run_agent_turn_stage + UNIQUE (maf_run_id, agent_id, turn_index, execution_stage), + CONSTRAINT ck_maf_seat_runs_execution_stage CHECK (execution_stage IN ({STAGE4_STAGES})) + ) + """, + f""" + INSERT INTO {SEAT_RUNS_TABLE}__stage4 ( + id, maf_run_id, agent_id, turn_index, execution_stage, + deerflow_run_id, deerflow_thread_id, sandbox_id, workflow_checkpoint_id, + status, result_snapshot, started_at, finished_at, error, created_at, updated_at + ) + SELECT + id, maf_run_id, agent_id, turn_index, execution_stage, + deerflow_run_id, deerflow_thread_id, sandbox_id, workflow_checkpoint_id, + status, result_snapshot, started_at, finished_at, error, created_at, updated_at + FROM {SEAT_RUNS_TABLE} + """, + f"DROP TABLE {SEAT_RUNS_TABLE}", + f"ALTER TABLE {SEAT_RUNS_TABLE}__stage4 RENAME TO {SEAT_RUNS_TABLE}", + f"CREATE INDEX ix_maf_seat_runs_maf_run_id ON {SEAT_RUNS_TABLE} (maf_run_id)", + f"CREATE INDEX ix_maf_seat_runs_agent_id ON {SEAT_RUNS_TABLE} (agent_id)", + f"CREATE INDEX ix_maf_seat_runs_deerflow_run_id ON {SEAT_RUNS_TABLE} (deerflow_run_id)", + f"CREATE INDEX ix_maf_seat_runs_status ON {SEAT_RUNS_TABLE} (status)", + ] + + +def validate_schema(inspector: Any) -> list[str]: + """Return contract violations found by SQLAlchemy inspector. + + Validates constraint CONTENT, not just names: a same-named Stage-3 CHECK is + reported as a violation until it really contains the Stage-4 stages. + """ + issues: list[str] = [] + tables = set(inspector.get_table_names()) + + if SEAT_TASKS_TABLE not in tables: + issues.append(f"missing table: {SEAT_TASKS_TABLE}") + else: + cols = {c["name"] for c in inspector.get_columns(SEAT_TASKS_TABLE)} + for required in ( + "id", "session_id", "run_id", "task_key", "agent_id", + "original_agent_id", "status", "required", "attempt_count", + "rework_count", "reassign_count", "created_at", "updated_at", + ): + if required not in cols: + issues.append(f"{SEAT_TASKS_TABLE}: missing column {required}") + uq_cols = [ + set(u.get("column_names", [])) + for u in inspector.get_unique_constraints(SEAT_TASKS_TABLE) + ] + idx_cols = [ + set(idx.get("column_names", [])) + for idx in inspector.get_indexes(SEAT_TASKS_TABLE) + if idx.get("unique") + ] + if {"run_id", "task_key"} not in (uq_cols + idx_cols): + issues.append(f"{SEAT_TASKS_TABLE}: missing UNIQUE(run_id, task_key)") + + status_sql = _check_constraint_sql( + inspector, SEAT_TASKS_TABLE, SEAT_TASKS_STATUS_CHECK_NAME + ) + if status_sql is None: + issues.append(f"{SEAT_TASKS_TABLE}: missing status CHECK constraint") + else: + missing = [s for s in SEAT_TASKS_STATUSES if s not in status_sql] + if missing: + issues.append( + f"{SEAT_TASKS_TABLE}: status CHECK missing statuses: {', '.join(missing)}" + ) + + fk_targets: dict[str, str] = {} + try: + for fk in inspector.get_foreign_keys(SEAT_TASKS_TABLE): + for local in fk.get("constrained_columns") or []: + fk_targets[local] = fk.get("referred_table") or "" + except Exception: # noqa: BLE001 + pass + if fk_targets.get("session_id") != "maf_roundtable_sessions": + issues.append( + f"{SEAT_TASKS_TABLE}: session_id must reference maf_roundtable_sessions" + ) + if fk_targets.get("run_id") != "maf_roundtable_runs": + issues.append( + f"{SEAT_TASKS_TABLE}: run_id must reference maf_roundtable_runs" + ) + + if SEAT_RUNS_TABLE in tables: + if not _has_stage4_execution_stage_check(inspector): + issues.append( + f"{SEAT_RUNS_TABLE}: execution_stage CHECK missing or not widened " + f"to include {', '.join(STAGE4_NEW_STAGES)}" + ) + return issues + + +def _do_migrate_sync( + sync_conn: Any, + *, + dry_run: bool, + logger: logging.Logger, + changes_ref: list[int], +) -> None: + """Run all migration steps on a sync connection (called via run_sync). + + Mirrors the Phase-3 migration's sync entry point so tests can drive the + migration against an arbitrary engine. + """ + from sqlalchemy import inspect, text + + insp = inspect(sync_conn) + dialect = _dialect_name(sync_conn) + + def _exec(sql: str) -> None: + if dry_run: + logger.info(" [dry-run] would execute: %s", sql.strip()[:300]) + return + sync_conn.execute(text(sql)) + changes_ref[0] += 1 + + existing = set(insp.get_table_names()) + + if SEAT_TASKS_TABLE not in existing: + logger.info("Creating %s", SEAT_TASKS_TABLE) + _exec(_seat_tasks_ddl(dialect)) + for index_sql in _seat_tasks_indexes_ddl(dialect): + _exec(index_sql) + else: + logger.info("Already exists: %s", SEAT_TASKS_TABLE) + + if SEAT_RUNS_TABLE in existing: + # The Stage-3 constraint carries the SAME name — only the SQL text + # (must contain both new stages) proves the table is already Stage 4. + check_sql = _execution_stage_check_sql(insp) + is_stage4 = _has_stage4_execution_stage_check(insp) + ddls = _widen_check_ddls( + dialect, check_exists=check_sql is not None, is_stage4=is_stage4 + ) + if not ddls: + logger.info("execution_stage CHECK already widened — no-op.") + elif dialect == "sqlite": + logger.info("SQLite: rebuilding %s to widen execution_stage CHECK", SEAT_RUNS_TABLE) + for stmt in ddls: + _exec(stmt) + else: + if check_sql is not None: + logger.info("Dropping stale execution_stage CHECK on %s", SEAT_RUNS_TABLE) + logger.info("Adding Stage-4 execution_stage CHECK on %s", SEAT_RUNS_TABLE) + for stmt in ddls: + _exec(stmt) + + logger.info("Migration steps: %d applied.", changes_ref[0]) + + +async def run_migration(database_url: str, *, dry_run: bool = False) -> None: + from sqlalchemy import inspect + from sqlalchemy.ext.asyncio import create_async_engine + from sqlalchemy.pool import NullPool + + engine = create_async_engine(database_url, poolclass=NullPool) + try: + async with engine.begin() as conn: + logger.info("Dialect: %s", _dialect_name(conn)) + # Single code path: the sync entry point owns all detection / DDL + # decisions (including the Stage-3 same-name constraint case). + changes_ref = [0] + await conn.run_sync( + _do_migrate_sync, + dry_run=dry_run, + logger=logger, + changes_ref=changes_ref, + ) + + if dry_run: + logger.info("[DRY RUN] No statements were executed.") + return + + def _validate(sync_conn: Any) -> list[str]: + return validate_schema(inspect(sync_conn)) + + issues = await conn.run_sync(_validate) + if issues: + logger.error("Schema validation FAILED:") + for issue in issues: + logger.error(" - %s", issue) + sys.exit(1) + logger.info("Schema validation passed.") + finally: + await engine.dispose() + + +def main() -> None: + parser = argparse.ArgumentParser(description="MAF roundtable Stage 4 migration") + parser.add_argument("--database-url", default=None, help="Async database URL (default: from config)") + parser.add_argument("--dry-run", action="store_true", help="Print DDL without executing") + args = parser.parse_args() + + database_url = args.database_url + if database_url is None: + from pathlib import Path + + import yaml + + config_path = Path(__file__).resolve().parent.parent / "config.yaml" + if config_path.exists(): + with open(config_path) as f: + cfg = yaml.safe_load(f) or {} + db_cfg = cfg.get("database", {}) + database_url = db_cfg.get("url", "sqlite+aiosqlite:///./data/deerflow.db") + else: + database_url = "sqlite+aiosqlite:///./data/deerflow.db" + + logger.info("Database: %s", database_url.split("?")[0]) + asyncio.run(run_migration(database_url, dry_run=args.dry_run)) + + +if __name__ == "__main__": + main() diff --git a/offline-backend-20260512/backend/tests/test_maf_conversation_stream.py b/offline-backend-20260512/backend/tests/test_maf_conversation_stream.py index 81142707..0dd58011 100644 --- a/offline-backend-20260512/backend/tests/test_maf_conversation_stream.py +++ b/offline-backend-20260512/backend/tests/test_maf_conversation_stream.py @@ -875,8 +875,9 @@ async def test_sse_generator_realtime_terminal_stops(svc, store, hub): @pytest.mark.asyncio async def test_p0_1_atomic_projection_both_events_or_neither(svc, store): - """agent.message.completed must atomically project both - agent.message.complete AND seat.delivery.submitted.""" + """A genuine seat delivery (agent.message.completed with a seat delivery + execution_stage) must atomically project both agent.message.complete AND + seat.delivery.submitted.""" import uuid from sqlalchemy import select @@ -914,7 +915,11 @@ async def test_p0_1_atomic_projection_both_events_or_neither(svc, store): "event_type": "agent.message.completed", "agent_id": "seat_a", "turn_index": 1, - "payload": {"content": "result", "schema_status": "valid"}, + "payload": { + "content": "result", + "schema_status": "valid", + "execution_stage": "expert", + }, } written = await svc.bridge_workflow_event( @@ -1020,7 +1025,7 @@ async def test_p0_1_projection_failure_does_not_duplicate_on_retry(svc, store): id="audit_retry_1", session_id=session.id, run_id=run.id, seq=1, event_type="agent.message.completed", agent_id="seat_a", turn_index=0, - payload_json={"content": "done"}, + payload_json={"content": "done", "execution_stage": "expert"}, )) await svc.bridge_workflow_event( @@ -1029,7 +1034,7 @@ async def test_p0_1_projection_failure_does_not_duplicate_on_retry(svc, store): "event_type": "agent.message.completed", "agent_id": "seat_a", "turn_index": 0, - "payload": {"content": "done"}, + "payload": {"content": "done", "execution_stage": "expert"}, }, source_audit_event_id="audit_retry_1", ) diff --git a/offline-backend-20260512/backend/tests/test_maf_migration_stage4.py b/offline-backend-20260512/backend/tests/test_maf_migration_stage4.py new file mode 100644 index 00000000..4e1726af --- /dev/null +++ b/offline-backend-20260512/backend/tests/test_maf_migration_stage4.py @@ -0,0 +1,436 @@ +"""Acceptance tests for the Stage-4 migration script (R3-Q quality gate). + +P0-1: the tests start from a REAL Stage-3 schema — the seat_runs table is +created by hand with the Stage-3 ``execution_stage`` CHECK (same constraint +NAME as Stage 4, but WITHOUT ``rework`` / ``manager_review``). ``Base.metadata +.create_all()`` is never used to fake the Stage-3 seat_runs table, because the +current ORM metadata already carries the widened Stage-4 CHECK. + +Verified behaviour: + +1. A same-named Stage-3 constraint is detected as NOT widened (sqltext must + contain both new stages — the name alone never proves Stage 4). +2. SQLite: the incomplete constraint triggers the table rebuild; the original + seat-run rows survive; indexes are recreated; a second run is a no-op. +3. After migration, inserting ``rework`` and ``manager_review`` succeeds while + an unknown stage is still rejected by the CHECK. +4. Two consecutive runs produce no errors and no duplicate tables/indexes. +5. ``validate_schema`` checks constraint CONTENT plus the seat_tasks status + CHECK, unique key and foreign keys. +6. PostgreSQL / MySQL DDL generation is asserted without a live database + (drop+recreate on an existing old constraint, plain add when absent, no-op + when already Stage 4; PostgreSQL boolean default is ``TRUE`` not ``1``). +""" + +from __future__ import annotations + +import importlib.util +from datetime import datetime +from pathlib import Path + +import pytest +from sqlalchemy import inspect as sa_inspect +from sqlalchemy import text +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from deerflow.persistence.maf_roundtable.model import ( + Base, + MafRoundtableRunRow, + MafRoundtableSeatRunRow, + MafRoundtableSessionRow, +) + +STAGE3_STAGES = ( + "'legacy','planner','expert','format_repair'," + "'cross_review','cross_review_repair','finalizer','group_chat'" +) + +# Real Stage-3 seat_runs DDL — identical shape to the Phase-3 output, with the +# Stage-3 (unwidened) execution_stage CHECK. +STAGE3_SEAT_RUNS_DDL = f""" +CREATE TABLE maf_roundtable_seat_runs ( + id VARCHAR(64) NOT NULL PRIMARY KEY, + maf_run_id VARCHAR(64) NOT NULL, + agent_id VARCHAR(64) NOT NULL, + turn_index INTEGER NOT NULL, + execution_stage VARCHAR(32) NOT NULL DEFAULT 'legacy', + deerflow_run_id VARCHAR(64), + deerflow_thread_id VARCHAR(64), + sandbox_id VARCHAR(64), + workflow_checkpoint_id VARCHAR(128), + status VARCHAR(32) DEFAULT 'pending' NOT NULL, + result_snapshot TEXT, + started_at DATETIME, + finished_at DATETIME, + error TEXT, + created_at DATETIME NOT NULL, + updated_at DATETIME NOT NULL, + FOREIGN KEY (maf_run_id) REFERENCES maf_roundtable_runs (id) ON DELETE CASCADE, + CONSTRAINT uq_maf_seat_runs_run_agent_turn_stage + UNIQUE (maf_run_id, agent_id, turn_index, execution_stage), + CONSTRAINT ck_maf_seat_runs_execution_stage + CHECK (execution_stage IN ({STAGE3_STAGES})) +) +""" + +STAGE3_SEAT_RUNS_INDEXES = [ + "CREATE INDEX ix_maf_seat_runs_maf_run_id ON maf_roundtable_seat_runs (maf_run_id)", + "CREATE INDEX ix_maf_seat_runs_agent_id ON maf_roundtable_seat_runs (agent_id)", + "CREATE INDEX ix_maf_seat_runs_deerflow_run_id ON maf_roundtable_seat_runs (deerflow_run_id)", + "CREATE INDEX ix_maf_seat_runs_status ON maf_roundtable_seat_runs (status)", +] + + +def _load_migration_module(): + """Load scripts/migrate_maf_roundtable_stage4.py as a module.""" + here = Path(__file__).resolve().parent.parent + script = here / "scripts" / "migrate_maf_roundtable_stage4.py" + spec = importlib.util.spec_from_file_location("migrate_maf_roundtable_stage4", script) + assert spec and spec.loader, "could not load migration script" + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +async def _create_stage3_schema(engine) -> None: + """Create a genuine Stage-3 schema. + + sessions / runs come from the ORM metadata (they did not change in Stage 4); + seat_runs is created BY HAND with the Stage-3 CHECK; seat_tasks does not + exist yet. + """ + async with engine.begin() as conn: + await conn.run_sync( + lambda sync_conn: Base.metadata.create_all( + sync_conn, + tables=[ + MafRoundtableSessionRow.__table__, + MafRoundtableRunRow.__table__, + ], + ) + ) + await conn.execute(text(STAGE3_SEAT_RUNS_DDL)) + for ddl in STAGE3_SEAT_RUNS_INDEXES: + await conn.execute(text(ddl)) + + +async def _seed_stage3_data(engine) -> None: + sf = async_sessionmaker(engine, expire_on_commit=False) + now = datetime(2026, 1, 1) + async with sf() as session: + session.add(MafRoundtableSessionRow( + id="s1", user_id="u1", title="t", task_text="t", + mode="structured", status="draft", write_policy="confirm_high_risk", + max_rounds=8, created_at=now, updated_at=now, + )) + session.add(MafRoundtableRunRow( + id="r1", session_id="s1", status="completed", + created_at=now, updated_at=now, + )) + session.add(MafRoundtableSeatRunRow( + id="sr1", maf_run_id="r1", agent_id="seat-a", turn_index=1, + execution_stage="expert", status="completed", + created_at=now, updated_at=now, + )) + session.add(MafRoundtableSeatRunRow( + id="sr2", maf_run_id="r1", agent_id="manager-1", turn_index=1, + execution_stage="cross_review", status="completed", + created_at=now, updated_at=now, + )) + await session.commit() + + +async def _run_migrate(engine, mod) -> int: + changes_ref = [0] + async with engine.begin() as conn: + await conn.run_sync( + mod._do_migrate_sync, dry_run=False, logger=mod.logger, changes_ref=changes_ref + ) + return changes_ref[0] + + +def _seat_runs_check_sql(insp) -> str | None: + for ck in insp.get_check_constraints("maf_roundtable_seat_runs"): + if ck.get("name") == "ck_maf_seat_runs_execution_stage": + return str(ck.get("sqltext") or "") + return None + + +# ── detection: same-named Stage-3 constraint is NOT Stage 4 ──────────────── + + +@pytest.mark.asyncio +async def test_stage3_same_name_constraint_detected_as_not_stage4(): + mod = _load_migration_module() + engine = create_async_engine("sqlite+aiosqlite://", echo=False) + try: + await _create_stage3_schema(engine) + + def _detect(sync_conn): + insp = sa_inspect(sync_conn) + return ( + mod._execution_stage_check_sql(insp), + mod._has_stage4_execution_stage_check(insp), + ) + + async with engine.begin() as conn: + sql, is_stage4 = await conn.run_sync(_detect) + + # The Stage-3 constraint exists under the same name ... + assert sql is not None + # ... but it does NOT allow the Stage-4 stages. + assert "rework" not in sql + assert "manager_review" not in sql + assert is_stage4 is False + finally: + await engine.dispose() + + +# ── SQLite: real Stage-3 → Stage-4, idempotent, data preserved ───────────── + + +@pytest.mark.asyncio +async def test_migration_widens_real_stage3_and_is_idempotent(): + mod = _load_migration_module() + engine = create_async_engine("sqlite+aiosqlite://", echo=False) + try: + await _create_stage3_schema(engine) + await _seed_stage3_data(engine) + + # Sanity: before migration, 'rework' is rejected by the Stage-3 CHECK. + sf = async_sessionmaker(engine, expire_on_commit=False) + with pytest.raises(Exception): + async with sf() as session: + session.add(MafRoundtableSeatRunRow( + id="sr-pre", maf_run_id="r1", agent_id="seat-x", turn_index=9, + execution_stage="rework", status="pending", + created_at=datetime(2026, 1, 1), updated_at=datetime(2026, 1, 1), + )) + await session.commit() + + # Run the migration twice — the second run must be a no-op. + first_changes = await _run_migrate(engine, mod) + assert first_changes > 0, "first run must apply changes on a Stage-3 schema" + + def _index_names(sync_conn): + return sorted( + idx["name"] + for idx in sa_inspect(sync_conn).get_indexes("maf_roundtable_seat_runs") + ) + + async with engine.begin() as conn: + indexes_after_first = await conn.run_sync(_index_names) + + second_changes = await _run_migrate(engine, mod) + assert second_changes == 0, "second run must be a no-op" + + async with engine.begin() as conn: + indexes_after_second = await conn.run_sync(_index_names) + assert indexes_after_first == indexes_after_second, "no duplicate indexes" + + def _check(sync_conn): + insp = sa_inspect(sync_conn) + tables = set(insp.get_table_names()) + # No leftover rebuild artefacts, exactly one seat_runs table. + assert "maf_roundtable_seat_runs__stage4" not in tables + assert "maf_roundtable_seat_tasks" in tables + + # The widened CHECK contains BOTH new stages. + sql = _seat_runs_check_sql(insp) + assert sql is not None + assert "rework" in sql + assert "manager_review" in sql + + # seat_tasks contract: columns, unique key, status CHECK, FKs. + cols = {c["name"] for c in insp.get_columns("maf_roundtable_seat_tasks")} + for required in ( + "id", "session_id", "run_id", "task_key", "agent_id", + "original_agent_id", "status", "required", "contract_json", + "delivery_json", "history_json", "gap_json", + ): + assert required in cols, f"missing column {required}" + uq = [ + set(u.get("column_names", [])) + for u in insp.get_unique_constraints("maf_roundtable_seat_tasks") + ] + assert {"run_id", "task_key"} in uq + + # Original Stage-3 rows survived the rebuild untouched. + rows = sync_conn.execute(text( + "SELECT id, execution_stage, status FROM maf_roundtable_seat_runs ORDER BY id" + )).fetchall() + assert rows == [ + ("sr1", "expert", "completed"), + ("sr2", "cross_review", "completed"), + ] + + async with engine.begin() as conn: + await conn.run_sync(_check) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_migration_allows_stage4_stages_and_rejects_unknown(): + mod = _load_migration_module() + engine = create_async_engine("sqlite+aiosqlite://", echo=False) + try: + await _create_stage3_schema(engine) + await _seed_stage3_data(engine) + await _run_migrate(engine, mod) + + sf = async_sessionmaker(engine, expire_on_commit=False) + now = datetime(2026, 1, 2) + async with sf() as session: + session.add(MafRoundtableSeatRunRow( + id="sr-rework", maf_run_id="r1", agent_id="seat-a", turn_index=2, + execution_stage="rework", status="completed", + created_at=now, updated_at=now, + )) + session.add(MafRoundtableSeatRunRow( + id="sr-review", maf_run_id="r1", agent_id="manager-1", turn_index=2, + execution_stage="manager_review", status="completed", + created_at=now, updated_at=now, + )) + await session.commit() + + async with sf() as session: + stages = ( + (await session.execute(text( + "SELECT execution_stage FROM maf_roundtable_seat_runs ORDER BY id" + ))).scalars().all() + ) + assert "rework" in stages + assert "manager_review" in stages + + # The CHECK is still enforced — an unknown stage must be rejected. + with pytest.raises(Exception): + async with sf() as session: + session.add(MafRoundtableSeatRunRow( + id="sr-bogus", maf_run_id="r1", agent_id="seat-a", turn_index=3, + execution_stage="bogus_stage", status="pending", + created_at=now, updated_at=now, + )) + await session.commit() + finally: + await engine.dispose() + + +# ── validate_schema: content checks, not just names ──────────────────────── + + +@pytest.mark.asyncio +async def test_validate_schema_flags_stage3_then_passes_stage4(): + mod = _load_migration_module() + engine = create_async_engine("sqlite+aiosqlite://", echo=False) + try: + await _create_stage3_schema(engine) + await _seed_stage3_data(engine) + + def _validate(sync_conn): + return mod.validate_schema(sa_inspect(sync_conn)) + + # Before migration: seat_tasks missing AND the CHECK not widened. + async with engine.begin() as conn: + issues = await conn.run_sync(_validate) + assert any("missing table" in i for i in issues) + assert any("not widened" in i for i in issues) + + await _run_migrate(engine, mod) + async with engine.begin() as conn: + issues = await conn.run_sync(_validate) + assert issues == [] + + # Dropping the status CHECK must be reported (content validation). + # SQLite cannot DROP CONSTRAINT, so simulate by checking a tampered + # inspector view: rebuild without the constraint is out of scope here; + # instead verify the FK/unique violations are reported when the table + # is dropped entirely. + async with engine.begin() as conn: + await conn.execute(text("DROP TABLE maf_roundtable_seat_tasks")) + async with engine.begin() as conn: + issues = await conn.run_sync(_validate) + assert any("missing table" in i for i in issues) + finally: + await engine.dispose() + + +# ── PostgreSQL / MySQL DDL generation (no live database) ─────────────────── + + +def test_widen_check_ddls_postgresql(): + mod = _load_migration_module() + # Old same-named constraint present → drop then re-add. + ddls = mod._widen_check_ddls("postgresql", check_exists=True, is_stage4=False) + assert len(ddls) == 2 + assert "DROP CONSTRAINT ck_maf_seat_runs_execution_stage" in ddls[0] + assert "ADD CONSTRAINT ck_maf_seat_runs_execution_stage" in ddls[1] + for stage in ("rework", "manager_review"): + assert stage in ddls[1] + # No constraint at all → plain add. + ddls = mod._widen_check_ddls("postgresql", check_exists=False, is_stage4=False) + assert len(ddls) == 1 + assert ddls[0].startswith("ALTER TABLE") + assert "DROP" not in ddls[0] + # Already Stage 4 → no-op. + assert mod._widen_check_ddls("postgresql", check_exists=True, is_stage4=True) == [] + + +def test_widen_check_ddls_mysql(): + mod = _load_migration_module() + ddls = mod._widen_check_ddls("mysql", check_exists=True, is_stage4=False) + assert len(ddls) == 2 + assert "DROP CHECK ck_maf_seat_runs_execution_stage" in ddls[0] + assert "ADD CONSTRAINT ck_maf_seat_runs_execution_stage" in ddls[1] + assert mod._widen_check_ddls("mysql", check_exists=True, is_stage4=True) == [] + + +def test_widen_check_ddls_sqlite_rebuilds(): + mod = _load_migration_module() + ddls = mod._widen_check_ddls("sqlite", check_exists=True, is_stage4=False) + joined = ";\n".join(ddls) + assert "CREATE TABLE maf_roundtable_seat_runs__stage4" in joined + assert "INSERT INTO maf_roundtable_seat_runs__stage4" in joined + assert "DROP TABLE maf_roundtable_seat_runs" in joined + assert "RENAME TO maf_roundtable_seat_runs" in joined + assert "CREATE INDEX" in joined + assert mod._widen_check_ddls("sqlite", check_exists=True, is_stage4=True) == [] + + +def test_seat_tasks_boolean_default_per_dialect(): + mod = _load_migration_module() + pg = mod._seat_tasks_ddl("postgresql") + assert "required BOOLEAN NOT NULL DEFAULT TRUE" in pg + assert "DEFAULT 1" not in pg + sqlite = mod._seat_tasks_ddl("sqlite") + assert "required BOOLEAN NOT NULL DEFAULT 1" in sqlite + mysql = mod._seat_tasks_ddl("mysql") + assert "required BOOLEAN NOT NULL DEFAULT 1" in mysql + + +def test_fake_inspector_detection_matrix(): + mod = _load_migration_module() + + class FakeInspector: + def __init__(self, checks): + self._checks = checks + + def get_check_constraints(self, table): + return self._checks.get(table, []) + + old = FakeInspector({ + "maf_roundtable_seat_runs": [ + {"name": "ck_maf_seat_runs_execution_stage", "sqltext": f"execution_stage IN ({STAGE3_STAGES})"}, + ], + }) + new = FakeInspector({ + "maf_roundtable_seat_runs": [ + {"name": "ck_maf_seat_runs_execution_stage", "sqltext": f"execution_stage IN ({mod.STAGE4_STAGES})"}, + ], + }) + none = FakeInspector({}) + + assert mod._has_stage4_execution_stage_check(old) is False + assert mod._has_stage4_execution_stage_check(new) is True + assert mod._has_stage4_execution_stage_check(none) is False + assert mod._execution_stage_check_sql(none) is None diff --git a/offline-backend-20260512/backend/tests/test_maf_phase3_conversation_dispatch.py b/offline-backend-20260512/backend/tests/test_maf_phase3_conversation_dispatch.py index 40c88a30..6a937abd 100644 --- a/offline-backend-20260512/backend/tests/test_maf_phase3_conversation_dispatch.py +++ b/offline-backend-20260512/backend/tests/test_maf_phase3_conversation_dispatch.py @@ -250,6 +250,9 @@ async def test_workflow_projection_whitelists_user_visible_event_data(store): "payload": { "content": "answer", "metadata": {"thread_id": "thread-secret", "deerflow_run_id": "df-secret"}, + # A genuine seat delivery stage — only these may project a + # seat.delivery.submitted companion event. + "execution_stage": "expert", }, }, ) diff --git a/offline-backend-20260512/backend/tests/test_maf_phase3_manager_workflow.py b/offline-backend-20260512/backend/tests/test_maf_phase3_manager_workflow.py index 431f4c26..b7ecdaee 100644 --- a/offline-backend-20260512/backend/tests/test_maf_phase3_manager_workflow.py +++ b/offline-backend-20260512/backend/tests/test_maf_phase3_manager_workflow.py @@ -291,6 +291,9 @@ async def v2_svc(tmp_path): task_payload_json={ "maf_v2": { "dispatch_contract_enabled": True, + # Stage-3 topology tests freeze the pre-Stage-4 behavior: + # plain seat fan-out, no quality gate. + "quality_gate_enabled": False, "candidate_set": { "manager_agent_id": MANAGER, "candidate_seat_agent_ids": ["seat-a", "seat-b", "seat-c"], diff --git a/offline-backend-20260512/backend/tests/test_maf_phase4_conversation_projection.py b/offline-backend-20260512/backend/tests/test_maf_phase4_conversation_projection.py new file mode 100644 index 00000000..44f24df8 --- /dev/null +++ b/offline-backend-20260512/backend/tests/test_maf_phase4_conversation_projection.py @@ -0,0 +1,264 @@ +"""Stage 4 (R3-Q) P1: real conversation SSE projection tests. + +The Stage-4 quality events must reach the USER-VISIBLE conversation event +stream (``maf_roundtable_conversation_events``) — not just the per-run audit +table — and they must reach it safely: + +1. all five Stage-4 event types are projected into the conversation table; +2. each source audit event is projected exactly ONCE (idempotent re-bridge); +3. only whitelisted fields are exposed in ``data``; +4. thread ids, run ids, internal action ids and tool parameters never leak; +5. ordinary manager / finalizer / review messages are never mislabelled as + ``seat.delivery.submitted`` (only genuine seat delivery stages are). +""" + +from __future__ import annotations + +from datetime import datetime + +import pytest +import pytest_asyncio +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.maf_roundtable.conversation_service import MafConversationService +from app.maf_roundtable.event_normalizer import ( + make_manager_review_decision, + make_seat_delivery_submitted, + make_seat_reassigned, + make_seat_rework_assigned, + make_stage_quality_gate, + make_turn_completed, +) +from deerflow.persistence.base import Base +from deerflow.persistence.maf_roundtable import make_maf_roundtable_store +from deerflow.persistence.maf_roundtable.model import ( + MafRoundtableConversationRow, + MafRoundtableRunRow, + MafRoundtableSessionRow, +) + + +class FakeMafConfig: + enabled = True + default_mode = "structured" + default_model_name = "test-model" + default_reasoning_effort = None + default_manager_agent_id = "mgr-agent" + default_max_rounds = 8 + default_write_policy = "confirm_high_risk" + admin_diagnostic_access = True + + +@pytest_asyncio.fixture +async def proj_env(tmp_path): + """Store + conversation service + one conversation bound to one run.""" + engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'proj.db'}") + try: + async with engine.begin() as conn: + import deerflow.persistence.maf_roundtable.model # noqa: F401 + + await conn.run_sync(Base.metadata.create_all) + sf = async_sessionmaker(engine, expire_on_commit=False) + store = make_maf_roundtable_store(sf) + svc = MafConversationService(store, maf_config=FakeMafConfig(), event_hub=None) + + conv = await store.conversations.create_conversation({ + "owner_user_id": "u1", "title": "projection", "task_context": None, + }) + now = datetime(2026, 1, 1) + async with sf() as session: + session.add(MafRoundtableSessionRow( + id="proj-sess", user_id="u1", title="t", task_text="t", + mode="structured", status="running", write_policy="auto", + manager_agent_id="manager-1", max_rounds=8, + created_at=now, updated_at=now, + )) + session.add(MafRoundtableRunRow( + id="proj-run", session_id="proj-sess", status="running", + created_at=now, updated_at=now, + )) + row = await session.get(MafRoundtableConversationRow, conv["id"]) + row.internal_session_id = "proj-sess" + await session.commit() + + yield svc, store, conv["id"] + finally: + await engine.dispose() + + +def _stage4_events() -> list[tuple[str, dict]]: + """(source_audit_event_id, event) for the five Stage-4 quality events. + + Payloads are deliberately seeded with sensitive fields that must be + stripped by the projection whitelist. + """ + sensitive = { + "thread_id": "thread-secret", + "run_id": "run-secret", + "session_id": "sess-secret", + "action_id": "action-secret", + "approval_id": "approval-secret", + "target_key": "hmac-target-key", + "idempotency_key": "idem-secret", + "tool_arguments": {"url": "http://internal", "token": "abc"}, + } + + def _with_sensitive(event: dict) -> dict: + event = dict(event) + event["payload"] = {**event.get("payload", {}), **sensitive} + return event + + return [ + ("audit-delivery", _with_sensitive(make_seat_delivery_submitted( + "seat-a", 1, task_key="01:seat-a", attempt=1, + deterministic_passed=False, failure_codes=["below_min_length"], + ))), + ("audit-review", _with_sensitive(make_manager_review_decision( + "seat-a", 1, task_key="01:seat-a", decision="rework", + reason="结论过短", source="manager", attempt=1, + ))), + ("audit-rework", _with_sensitive(make_seat_rework_assigned( + "seat-a", 1, task_key="01:seat-a", rework_count=1, + requirements=["请补充证据"], + ))), + ("audit-reassigned", _with_sensitive(make_seat_reassigned( + "seat-c", 1, task_key="01:seat-a", from_agent_id="seat-a", + reassign_count=1, + ))), + ("audit-gate", _with_sensitive(make_stage_quality_gate( + "manager-1", 1, decision="passed_with_gaps", total=2, + accepted=1, controlled_gap=1, blocked_required=[], + ))), + ] + + +_FORBIDDEN_KEYS = { + "thread_id", "run_id", "session_id", "action_id", "approval_id", + "target_key", "idempotency_key", "tool_arguments", "params", "arguments", + "capability_snapshot", "secret", "token", +} + + +@pytest.mark.asyncio +async def test_stage4_events_projected_into_conversation_stream(proj_env): + svc, store, conv_id = proj_env + + for source_id, event in _stage4_events(): + written = await svc.bridge_workflow_event( + run_id="proj-run", event=event, source_audit_event_id=source_id, + ) + assert written is True, f"{event['event_type']} was not projected" + + events = await store.conversation_events.list_events(conv_id, after_seq=0) + types = [e["type"] for e in events] + assert "seat.delivery.submitted" in types + assert "manager.review.decision" in types + assert "seat.rework.assigned" in types + assert "seat.reassigned" in types + assert "stage.quality_gate" in types + + # Whitelisted fields survive; the projection carries the quality payload. + delivery = next(e for e in events if e["type"] == "seat.delivery.submitted") + assert delivery["data"].get("task_key") == "01:seat-a" + assert delivery["data"].get("deterministic_passed") is False + assert delivery["data"].get("failure_codes") == ["below_min_length"] + review = next(e for e in events if e["type"] == "manager.review.decision") + assert review["data"].get("decision") == "rework" + assert review["data"].get("source") == "manager" + gate = next(e for e in events if e["type"] == "stage.quality_gate") + assert gate["data"].get("decision") == "passed_with_gaps" + + +@pytest.mark.asyncio +async def test_stage4_projection_is_idempotent_per_source_event(proj_env): + svc, store, conv_id = proj_env + + # Bridge every event, then bridge the SAME audit events again (retry / + # reconcile after restart). Each source event projects exactly once. + for _round in range(2): + for source_id, event in _stage4_events(): + await svc.bridge_workflow_event( + run_id="proj-run", event=event, source_audit_event_id=source_id, + ) + + events = await store.conversation_events.list_events(conv_id, after_seq=0) + projected = [ + e for e in events + if e["type"] in { + "seat.delivery.submitted", "manager.review.decision", + "seat.rework.assigned", "seat.reassigned", "stage.quality_gate", + } + ] + assert len(projected) == 5, f"expected exactly 5 projections, got {len(projected)}" + + +@pytest.mark.asyncio +async def test_stage4_projection_exposes_only_whitelist_fields(proj_env): + svc, store, conv_id = proj_env + + for source_id, event in _stage4_events(): + await svc.bridge_workflow_event( + run_id="proj-run", event=event, source_audit_event_id=source_id, + ) + + events = await store.conversation_events.list_events(conv_id, after_seq=0) + assert events + for e in events: + data = e.get("data") or {} + leaked = _FORBIDDEN_KEYS & set(data.keys()) + assert not leaked, f"{e['type']} leaks forbidden keys: {leaked}" + # Defense in depth: forbidden VALUES never appear anywhere either. + blob = str(data) + assert "thread-secret" not in blob + assert "run-secret" not in blob + assert "hmac-target-key" not in blob + assert "idem-secret" not in blob + assert "action-secret" not in blob + + +@pytest.mark.asyncio +async def test_manager_and_finalizer_messages_not_seat_deliveries(proj_env): + svc, store, conv_id = proj_env + + # Ordinary manager-side messages: none of them may become a seat delivery. + non_delivery_stages = [ + "manager_dispatch", "manager_review", "cross_review", + "cross_review_repair", "finalizer", "format_repair", "planner", + ] + for i, stage in enumerate(non_delivery_stages): + await svc.bridge_workflow_event( + run_id="proj-run", + event=make_turn_completed( + "manager-1", i + 1, content=f"{stage} message", + execution_stage=stage, + ), + source_audit_event_id=f"audit-msg-{stage}", + ) + # A completed message WITHOUT a stage (legacy audit row) is fail-closed: + # it must not be guessed to be a seat delivery either. + await svc.bridge_workflow_event( + run_id="proj-run", + event=make_turn_completed("manager-1", 99, content="unknown stage"), + source_audit_event_id="audit-msg-nostage", + ) + + # A genuine seat delivery stage still projects both events. + await svc.bridge_workflow_event( + run_id="proj-run", + event=make_turn_completed( + "seat-a", 1, content='{"summary": "交付"}', execution_stage="expert", + ), + source_audit_event_id="audit-msg-expert", + ) + + events = await store.conversation_events.list_events(conv_id, after_seq=0) + deliveries = [e for e in events if e["type"] == "seat.delivery.submitted"] + assert len(deliveries) == 1, ( + "only the expert seat turn may project seat.delivery.submitted, got " + f"{len(deliveries)}" + ) + assert deliveries[0]["data"].get("agent_id") == "seat-a" + + # All ordinary messages still appear as agent messages. + completions = [e for e in events if e["type"] == "agent.message.complete"] + assert len(completions) == len(non_delivery_stages) + 2 # + nostage + expert diff --git a/offline-backend-20260512/backend/tests/test_maf_phase4_quality_gate.py b/offline-backend-20260512/backend/tests/test_maf_phase4_quality_gate.py new file mode 100644 index 00000000..64ef4ef9 --- /dev/null +++ b/offline-backend-20260512/backend/tests/test_maf_phase4_quality_gate.py @@ -0,0 +1,748 @@ +"""Stage 4 (R3-Q) runner-level tests: seat task quality loop + phase gate. + +Drives the REAL workflow (QualitySeatTasksExecutor → QualityGateExecutor → +CrossReviewExecutor → ApprovalGateExecutor → FinalizerExecutor) through +WorkflowRunner.execute() with a fake adapter, proving the acceptance +criteria end-to-end against the real database: + +- a good delivery is accepted and the run completes with quality_decision=accept; +- a short / evidence-free delivery is sent back, reworked up to the frozen cap, + reassigned to a backup seat once, and finally ends as a traceable + controlled_gap that is carried into the final report (never as a fact); +- an ``ask_user`` fallback pauses the run in waiting_input; +- a required task left non-terminal blocks the phase quality gate; +- a missing/invalid manager review NEVER auto-accepts: the server forces + system reworks and exhausts into a controlled gap (source=system events). +""" + +from __future__ import annotations + +import json +from datetime import datetime +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +import pytest_asyncio +from maf_test_helpers import make_maf_service +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.maf_roundtable.deerflow_agent_adapter import SeatResponse +from app.maf_roundtable.workflow_executors import QualityGateExecutor, RoundtableState +from deerflow.persistence.maf_roundtable import make_maf_roundtable_store +from deerflow.persistence.maf_roundtable.model import ( + Base, + MafRoundtableParticipantRow, + MafRoundtableRunRow, + MafRoundtableSessionRow, +) + +MANAGER = "manager-1" +SEATS = ["seat-a", "seat-b", "seat-c"] + +# ``@TOOL_REF@`` is replaced per-turn with ``tool:kb_search:`` — the +# trusted evidence ref created by the seat's (fake) successful tool call, so a +# good delivery cites ONLY evidence the fact ledger actually backs (P1). +GOOD_DELIVERY = json.dumps({ + "summary": "结论：该市场存在明确增长机会，建议优先进入华东区域并配套本地化运营资源投入，同时预留华南试点预算", + "findings": ["依据一：华东区销售数据连续三个月增长", "依据二：竞品在华东投放低于均值"], + "evidence_refs": ["@TOOL_REF@"], + "open_questions": [], + "self_check": "已覆盖全部验收标准", +}) +SHORT_DELIVERY = json.dumps({"summary": "好", "findings": []}) + +DISPATCH_PARALLEL = { + "intent_type": "parallel_consultation", + "selected_seat_agent_ids": ["seat-a", "seat-b"], + "assignments": [ + {"agent_id": "seat-a", "task_goal": "市场增长机会分析", "delivery_requirements": ["证据", "优先级"], "order": 1}, + {"agent_id": "seat-b", "task_goal": "竞品定价分析", "delivery_requirements": ["证据", "建议"], "order": 2}, + ], + "execution_mode": "parallel", + "convergence_path": ["manager synthesis"], + "reason": "需要两位专家", +} + + +@pytest_asyncio.fixture +async def q_svc(tmp_path): + """Service with a V2 session: manager + 3 candidates, quality gate ON.""" + engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'q4.db'}", echo=False) + try: + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + sf = async_sessionmaker(engine, expire_on_commit=False) + store = make_maf_roundtable_store(sf) + service = make_maf_service(store) + now = datetime(2026, 1, 1) + async with sf() as session: + session.add(MafRoundtableSessionRow( + id="q-sess", user_id="u1", title="q", task_text="分析市场趋势", + mode="structured", status="pending", write_policy="auto", + manager_agent_id=MANAGER, max_rounds=8, created_at=now, updated_at=now, + task_payload_json={ + "maf_v2": { + "dispatch_contract_enabled": True, + "quality_gate_enabled": True, + "candidate_set": { + "manager_agent_id": MANAGER, + "candidate_seat_agent_ids": list(SEATS), + "mode": "structured", + "max_rounds": 8, + }, + }, + }, + )) + session.add(MafRoundtableRunRow( + id="q-run", session_id="q-sess", status="pending", + created_at=now, updated_at=now, + )) + for i, agent_id in enumerate([MANAGER, *SEATS]): + session.add(MafRoundtableParticipantRow( + id=f"qp-{i}", session_id="q-sess", agent_id=agent_id, + role="manager" if agent_id == MANAGER else "seat", + sort_order=i, created_at=now, updated_at=now, + )) + await session.commit() + yield service + finally: + await engine.dispose() + + +def _make_q_runner( + q_svc, + monkeypatch, + *, + deliveries: dict[str, list[str]], + review_plan: list[dict[str, dict[str, object]]], + fallback_policy: str = "reassign_then_gap", + cross_review_ok: bool = True, + emit_real: bool = False, + config_overrides: dict | None = None, +): + """Runner with a fake adapter driven by explicit delivery/review plans. + + ``deliveries``: agent_id → list of JSON payloads, one per delivery attempt. + ``review_plan``: one entry per quality round → {agent_id: {"decision", ...}}. + """ + from app.maf_roundtable.workflow_runner import WorkflowRunner + + app = SimpleNamespace(state=SimpleNamespace(maf_roundtable_service=q_svc, checkpointer=None)) + runner = WorkflowRunner(app, service=q_svc, user_id="u1") + + calls: list[tuple[str, str, int]] = [] + stage_messages: dict[str, list[str]] = {} + delivery_idx: dict[str, int] = {} + review_round: list[int] = [0] + + cfg_values = { + "seat_timeout_seconds": 5, + "schema_repair_enabled": True, + "quality_gate_enabled": True, + "max_rework_attempts": 2, + "max_reassignments": 1, + "min_delivery_chars": 40, + "min_delivery_evidence": 1, + "default_fallback_policy": fallback_policy, + } + cfg_values.update(config_overrides or {}) + fake_cfg = SimpleNamespace(maf_roundtable=SimpleNamespace(**cfg_values)) + monkeypatch.setattr("deerflow.config.get_app_config", lambda: fake_cfg) + + async def _run_seat(**kwargs): + stage = kwargs.get("execution_stage", "") + agent_id = kwargs["agent_id"] + turn_index = kwargs["turn_index"] + calls.append((stage, agent_id, turn_index)) + stage_messages.setdefault(stage, []).append(str(kwargs.get("message") or "")) + if stage == "manager_dispatch": + return SeatResponse(agent_id=agent_id, turn_index=0, content=json.dumps(DISPATCH_PARALLEL), tool_calls=[], usage={"input_tokens": 1, "output_tokens": 1}) + if stage in ("expert", "rework"): + seq = deliveries.get(agent_id) or [GOOD_DELIVERY] + idx = delivery_idx.get(agent_id, 0) + delivery_idx[agent_id] = idx + 1 + content = seq[min(idx, len(seq) - 1)] + # The seat ran a real (fake) tool this turn: its successful result + # becomes the trusted evidence ref the delivery may cite. + content = content.replace("@TOOL_REF@", f"tool:kb_search:{turn_index}") + return SeatResponse( + agent_id=agent_id, + turn_index=turn_index, + content=content, + tool_calls=[{"id": f"tc-{agent_id}-{turn_index}", "name": "kb_search"}], + usage={"input_tokens": 1, "output_tokens": 1}, + ) + if stage == "manager_review": + rnd = review_round[0] + review_round[0] = rnd + 1 + plan = review_plan[rnd] if rnd < len(review_plan) else {} + decisions = [ + { + "agent_id": agent, + "decision": d.get("decision", "accept"), + "reason": str(d.get("reason") or "r")[:1000], + "revision_requirements": list(d.get("requirements") or []), + } + for agent, d in plan.items() + ] + return SeatResponse(agent_id=agent_id, turn_index=turn_index, content=json.dumps({"decisions": decisions}), tool_calls=[], usage={"input_tokens": 1, "output_tokens": 1}) + if stage == "cross_review": + content = ( + json.dumps({"summary": "ok", "evidence_ok": True, "confirmed_candidates": []}) + if cross_review_ok + else json.dumps({"summary": "有缺口", "evidence_ok": False, "gaps": ["证据不足"], "confirmed_candidates": []}) + ) + return SeatResponse(agent_id=agent_id, turn_index=turn_index, content=content, tool_calls=[], usage={"input_tokens": 1, "output_tokens": 1}) + if stage == "finalizer": + return SeatResponse(agent_id=agent_id, turn_index=turn_index, content="final report", tool_calls=[], usage={"input_tokens": 1, "output_tokens": 1}) + return SeatResponse(agent_id=agent_id, turn_index=turn_index, content=GOOD_DELIVERY, tool_calls=[], usage={"input_tokens": 1, "output_tokens": 1}) + + ensure_threads_calls: list[list[str]] = [] + + async def _ensure_threads(session_id, run_id, participants): + agent_ids = [p["agent_id"] for p in participants] + ensure_threads_calls.append(agent_ids) + result = {} + for participant in participants: + agent_id = participant["agent_id"] + thread_id = f"thread-{agent_id}" + await q_svc.store.participants.bind_thread(session_id, agent_id, thread_id=thread_id) + result[agent_id] = thread_id + return result + + monkeypatch.setattr(runner._adapter, "run_seat_turn", AsyncMock(side_effect=_run_seat)) + monkeypatch.setattr(runner._adapter, "ensure_threads", AsyncMock(side_effect=_ensure_threads)) + if emit_real: + real_emit_critical = runner._adapter._emit_critical + monkeypatch.setattr(runner._adapter, "_emit_critical", AsyncMock(side_effect=real_emit_critical)) + else: + monkeypatch.setattr(runner._adapter, "_emit_critical", AsyncMock()) + monkeypatch.setattr(runner._adapter, "_emit", AsyncMock()) + runner.stage_messages = stage_messages + return runner, calls, ensure_threads_calls + + +async def _run(q_svc, monkeypatch, **kwargs): + runner, calls, ensure_threads_calls = _make_q_runner(q_svc, monkeypatch, **kwargs) + session = await q_svc.store.sessions.get_session("q-sess", user_id="u1") + run = await q_svc.store.runs.get_run("q-run") + await runner.execute(session, run) + final_run = await q_svc.store.runs.get_run("q-run") + final_session = await q_svc.store.sessions.get_session("q-sess", user_id="u1") + return runner, calls, ensure_threads_calls, final_run, final_session + + +@pytest.mark.asyncio +async def test_runner_quality_accepts_good_deliveries(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + _, calls, _, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={"seat-a": [GOOD_DELIVERY], "seat-b": [GOOD_DELIVERY]}, + review_plan=[{"seat-a": {"decision": "accept"}, "seat-b": {"decision": "accept"}}], + ) + + assert final_run["status"] == "completed" + assert final_session["status"] == "completed" + + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + by_key = {t["task_key"]: t for t in tasks} + assert set(by_key) == {"01:seat-a", "02:seat-b"} + assert all(t["status"] == "accepted" for t in tasks) + assert all(t["attempt_count"] == 1 for t in tasks) + assert all(t["rework_count"] == 0 for t in tasks) + + checkpoint = final_run.get("maf_checkpoint") or {} + assert checkpoint.get("seat_task_statuses", {}).get("01:seat-a", {}).get("status") == "accepted" + assert checkpoint.get("controlled_gaps") == [] + + result = final_session.get("final_result") or {} + assert result.get("quality_decision") == "accept" + assert result.get("controlled_gaps") == [] + + review_stages = [s for s, a, t in calls if s == "manager_review"] + assert len(review_stages) == 1, "one batched manager review round" + + +# ── 2. short answer → reworks → reassign → controlled gap ─────────────────── + + +@pytest.mark.asyncio +async def test_runner_quality_rework_reassign_controlled_gap(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + # seat-a stays short across every attempt; seat-b is always good. + # Manager keeps returning return_for_rework for seat-a (cap 2), then seat-a + # is reassigned to seat-c whose delivery also fails → controlled gap. + _, calls, ensure_threads_calls, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={ + "seat-a": [SHORT_DELIVERY, SHORT_DELIVERY, SHORT_DELIVERY, SHORT_DELIVERY], + "seat-b": [GOOD_DELIVERY], + "seat-c": [SHORT_DELIVERY], + }, + review_plan=[ + {"seat-a": {"decision": "return_for_rework", "reason": "太短", "requirements": ["请补充证据"]}, "seat-b": {"decision": "accept"}}, + {"seat-a": {"decision": "return_for_rework", "reason": "仍太短", "requirements": ["请展开分析"]}}, + {"seat-a": {"decision": "return_for_rework", "reason": "第三次不合格"}}, + {"seat-c": {"decision": "return_for_rework", "reason": "新席位也短"}}, + ], + ) + + # Run completes via the gap path (controlled gaps do NOT block the phase). + assert final_run["status"] == "completed" + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + by_key = {t["task_key"]: t for t in tasks} + + task_a = by_key["01:seat-a"] + assert task_a["status"] == "controlled_gap" + assert task_a["attempt_count"] == 4 # seat-a 3 + seat-c 1 + assert task_a["rework_count"] == 2 # capped at max_rework_attempts + assert task_a["reassign_count"] == 1 # exactly one reassignment + assert task_a["agent_id"] == "seat-c" # reassigned to backup + assert task_a["original_agent_id"] == "seat-a" + assert task_a["gap"] is not None + assert "不得作为已验证事实" in task_a["gap"]["risk"] + + task_b = by_key["02:seat-b"] + assert task_b["status"] == "accepted" + assert task_b["attempt_count"] == 1 + + # seat-c received a thread (backup handover) and delivered exactly once. + assert "seat-c" in [aid for call in ensure_threads_calls for aid in call] + seat_c_turns = [(s, a, t) for s, a, t in calls if a == "seat-c" and s in ("expert", "rework")] + assert len(seat_c_turns) == 1 + + # The quality gate passed with gaps and the final report carries them. + checkpoint = final_run.get("maf_checkpoint") or {} + gaps = checkpoint.get("controlled_gaps") or [] + assert len(gaps) == 1 + assert gaps[0]["task_key"] == "01:seat-a" + + result = final_session.get("final_result") or {} + assert result.get("quality_decision") == "controlled_gap" + assert len(result.get("controlled_gaps") or []) == 1 + assert result.get("degraded") is True + + # Manager review was batched once per quality round. + review_stages = [t for s, a, t in calls if s == "manager_review"] + assert len(review_stages) == 4 + + +# ── 3. ask_user fallback pauses in waiting_input ──────────────────────────── + + +@pytest.mark.asyncio +async def test_runner_quality_ask_user_waits_for_input(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + _, calls, _, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={"seat-a": [SHORT_DELIVERY, SHORT_DELIVERY, SHORT_DELIVERY], "seat-b": [GOOD_DELIVERY]}, + review_plan=[ + {"seat-a": {"decision": "return_for_rework", "reason": "太短"}, "seat-b": {"decision": "accept"}}, + {"seat-a": {"decision": "return_for_rework", "reason": "仍太短"}}, + {"seat-a": {"decision": "return_for_rework", "reason": "第三次不合格"}}, + ], + fallback_policy="ask_user", + ) + + assert final_run["status"] == "waiting_input" + assert final_session["status"] == "waiting_input" + + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + by_key = {t["task_key"]: t for t in tasks} + assert by_key["01:seat-a"]["status"] == "exhausted" + assert by_key["02:seat-b"]["status"] == "accepted" + + # The finalizer must never run on the ask_user path. + finalizer_stages = [s for s, a, t in calls if s == "finalizer"] + assert finalizer_stages == [] + # No finished_at for a non-terminal pause. + assert final_run.get("finished_at") is None + + +# ── 4. phase gate blocks on non-terminal required task ────────────────────── + + +@pytest.mark.asyncio +async def test_quality_gate_blocks_non_terminal_required_task(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + state = RoundtableState( + session_id="q-sess", run_id="q-run", task_text="t", + mode="structured", model_name=None, reasoning_effort=None, + max_rounds=8, write_policy="auto", manager_agent_id=MANAGER, + seat_agent_ids=["seat-a"], thread_map={}, capability_snapshot=None, + seat_timeout_seconds=5, schema_repair_enabled=True, + intent_type="parallel_consultation", + dispatch_plan={"execution_mode": "parallel"}, + dispatch_contract_enabled=True, quality_gate_enabled=True, + ) + state.seat_task_statuses = { + "01:seat-a": { + "task_key": "01:seat-a", "status": "submitted", + "agent_id": "seat-a", "required": True, + }, + } + adapter = SimpleNamespace( + _service=SimpleNamespace(store=SimpleNamespace(seat_tasks=SimpleNamespace())), + _emit_critical=AsyncMock(), + ) + gate = QualityGateExecutor(adapter, state) + ctx = SimpleNamespace(send_message=AsyncMock(), yield_output=AsyncMock()) + + with pytest.raises(RuntimeError, match="quality gate blocked"): + await gate.handle({}, ctx) + + # The blocked gate must not masquerade as passed. + assert state.status != "completed" + + +# ── 5. invalid/missing manager review NEVER auto-accepts ─────────────────── + + +def _good_variant(n: int) -> str: + """A deterministic-pass delivery whose conclusion differs per attempt, so + duplicate detection never interferes with the missing-review behaviour.""" + return json.dumps({ + "summary": ( + f"结论{n}：该市场存在明确增长机会，建议优先进入华东区域并配套本地化运营资源投入，" + f"同时预留华南试点预算，第{n}次补充细化执行路径与里程碑安排" + ), + "findings": [f"依据{n}：华东区销售数据连续三个月增长", "依据二：竞品在华东投放低于均值"], + "evidence_refs": ["@TOOL_REF@"], + "open_questions": [], + "self_check": "已覆盖全部验收标准", + }) + + +@pytest.mark.asyncio +async def test_runner_quality_missing_review_never_accepts(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + # The manager review returns garbage (empty decisions) every round. Even + # though every delivery passes the deterministic length/format/evidence + # checks, NO task may be accepted without a valid manager review: the + # server forces system reworks, then exhausts into controlled gaps. + variants_a = [_good_variant(i) for i in range(1, 7)] + variants_b = [_good_variant(i + 10) for i in range(1, 7)] + _, calls, _, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={ + "seat-a": variants_a + [_good_variant(90)], + "seat-b": variants_b + [_good_variant(91)], + "seat-c": [_good_variant(80)], + }, + review_plan=[{}], + emit_real=True, + ) + + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + assert tasks, "seat tasks must exist" + for task in tasks: + assert task["status"] != "accepted", ( + f"task {task['task_key']} was accepted without a valid manager review" + ) + assert task["status"] == "controlled_gap" + assert task["rework_count"] == 2 + assert task["reassign_count"] == 1 + decision = task.get("review_decision") or {} + assert decision.get("source") == "system" + assert decision.get("reason") == "manager_review_missing_or_invalid" + + # The run still terminates (gaps do not block the phase) — but only as a + # degraded controlled-gap result, never as a clean accept. + assert final_run["status"] == "completed" + result = final_session.get("final_result") or {} + assert result.get("quality_decision") == "controlled_gap" + assert len(result.get("controlled_gaps") or []) == len(tasks) + + # The persisted review events carry source=system + the safe reason. + events = await q_svc.store.events.list_events(run_id="q-run") + review_events = [e for e in events if e["event_type"] == "manager.review.decision"] + assert review_events + system_events = [e for e in review_events if e["payload"].get("source") == "system"] + assert system_events + assert all( + e["payload"].get("reason") == "manager_review_missing_or_invalid" + for e in system_events + ) + + +# ── 6. unaccepted deliveries never reach facts / review / final report ───── + + +PLAUSIBLE_GAP_DELIVERY = json.dumps({ + # Fails below_min_length on purpose, but carries "plausible" findings and + # an invented evidence ref — none of it may survive into the ledger, the + # cross-review prompt or the final synthesis. + "summary": "好", + "findings": ["PLAUSIBLE_FINDING: 已核实华东区增长数据并确认竞品全面撤离"], + "evidence_refs": ["plausible-ref-001"], + "open_questions": [], + "self_check": "ok", +}) + + +@pytest.mark.asyncio +async def test_runner_quality_gap_content_isolated_from_facts_and_report(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + runner, calls, _, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={ + "seat-a": [PLAUSIBLE_GAP_DELIVERY], + "seat-b": [GOOD_DELIVERY], + "seat-c": [PLAUSIBLE_GAP_DELIVERY], + }, + review_plan=[ + {"seat-a": {"decision": "return_for_rework", "reason": "太短"}, "seat-b": {"decision": "accept"}}, + {"seat-a": {"decision": "return_for_rework", "reason": "仍太短"}}, + {"seat-a": {"decision": "return_for_rework", "reason": "第三次不合格"}}, + {"seat-c": {"decision": "return_for_rework", "reason": "新席位也不合格"}}, + ], + ) + + assert final_run["status"] == "completed" + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + by_key = {t["task_key"]: t for t in tasks} + assert by_key["01:seat-a"]["status"] == "controlled_gap" + assert by_key["02:seat-b"]["status"] == "accepted" + + # 1) Fact ledger: the gapped seat's claims/refs are absent; the accepted + # seat's findings ARE present (merge happens on acceptance). + ledger = final_session.get("fact_ledger") or {} + candidate_texts = [ + str(c.get("text") or "") for c in ledger.get("candidate_facts") or [] + ] + confirmed_texts = [ + str(f.get("text") or "") for f in ledger.get("confirmed_facts") or [] + ] + evidence_texts = [ + str(e.get("ref") or e.get("text") or "") for e in ledger.get("evidence") or [] + ] + assert not any("PLAUSIBLE_FINDING" in t for t in candidate_texts) + assert not any("PLAUSIBLE_FINDING" in t for t in confirmed_texts) + assert not any("plausible-ref-001" in t for t in evidence_texts) + assert any("华东区销售数据连续三个月增长" in t for t in candidate_texts) + + # 2) Cross-review prompt: accepted seat only, never the gapped content. + cross_review_msgs = runner.stage_messages.get("cross_review") or [] + assert cross_review_msgs + assert all("PLAUSIBLE_FINDING" not in m for m in cross_review_msgs) + assert all("plausible-ref-001" not in m for m in cross_review_msgs) + assert any("华东区销售数据连续三个月增长" in m for m in cross_review_msgs) + + # 3) Final synthesis: accepted seat only + traceable gap risk section. + finalizer_msgs = runner.stage_messages.get("finalizer") or [] + assert finalizer_msgs + assert all("PLAUSIBLE_FINDING" not in m for m in finalizer_msgs) + assert all("plausible-ref-001" not in m for m in finalizer_msgs) + assert any("01:seat-a" in m for m in finalizer_msgs), "gap must be traceable" + assert any("受控缺口" in m for m in finalizer_msgs) + + # 4) Checkpoint: exactly one accepted result (the accepted seat). + checkpoint = final_run.get("maf_checkpoint") or {} + accepted_results = checkpoint.get("accepted_task_results") or [] + assert len(accepted_results) == 1 + assert accepted_results[0]["task_key"] == "02:seat-b" + assert accepted_results[0]["accepted"] is True + assert "PLAUSIBLE_FINDING" not in str(accepted_results) + + # 5) Final result carries the gap as risk, never as a verified fact. + result = final_session.get("final_result") or {} + assert result.get("quality_decision") == "controlled_gap" + gaps = result.get("controlled_gaps") or [] + assert len(gaps) == 1 + assert gaps[0]["task_key"] == "01:seat-a" + assert "不得作为已验证事实" in gaps[0]["risk"] + + +@pytest.mark.asyncio +async def test_runner_quality_events_persisted_and_projected(q_svc, monkeypatch): + """The Stage-4 audit events persist to maf_roundtable_events.""" + pytest.importorskip("agent_framework") + + _, calls, _, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={"seat-a": [SHORT_DELIVERY], "seat-b": [GOOD_DELIVERY]}, + review_plan=[{"seat-a": {"decision": "return_for_rework", "requirements": ["补证据"]}, "seat-b": {"decision": "accept"}}], + emit_real=True, + ) + + events = await q_svc.store.events.list_events(run_id="q-run") + types = {e["event_type"] for e in events} + assert "seat.delivery.submitted" in types + assert "manager.review.decision" in types + assert "seat.rework.assigned" in types + assert "stage.quality_gate" in types + + # The short delivery was flagged deterministically in the persisted event. + deliveries = [e for e in events if e["event_type"] == "seat.delivery.submitted"] + seat_a = [e for e in deliveries if e.get("agent_id") == "seat-a"] + assert seat_a and seat_a[0]["payload"].get("deterministic_passed") is False + + +# ── 7. P1: zero config values are honoured, not coerced to defaults ──────── + + +def test_quality_settings_honour_zero_config_values(monkeypatch): + from app.maf_roundtable.workflow_runner import WorkflowRunner + + fake_cfg = SimpleNamespace(maf_roundtable=SimpleNamespace( + quality_gate_enabled=True, + max_rework_attempts=0, + max_reassignments=0, + min_delivery_chars=0, + min_delivery_evidence=0, + default_fallback_policy="controlled_gap", + )) + monkeypatch.setattr("deerflow.config.get_app_config", lambda: fake_cfg) + + runner = WorkflowRunner.__new__(WorkflowRunner) # no __init__ side effects + settings = runner._quality_settings() + assert settings["max_rework_attempts"] == 0 + assert settings["max_reassignments"] == 0 + assert settings["min_delivery_chars"] == 0 + assert settings["min_delivery_evidence"] == 0 + assert settings["fallback_policy"] == "controlled_gap" + + +def test_quality_settings_none_falls_back_to_defaults(monkeypatch): + from app.maf_roundtable.workflow_runner import WorkflowRunner + + # Attributes absent entirely (getattr → None) must fall back to defaults. + fake_cfg = SimpleNamespace(maf_roundtable=SimpleNamespace( + quality_gate_enabled=True, + )) + monkeypatch.setattr("deerflow.config.get_app_config", lambda: fake_cfg) + + runner = WorkflowRunner.__new__(WorkflowRunner) + settings = runner._quality_settings() + assert settings["max_rework_attempts"] == 2 + assert settings["max_reassignments"] == 1 + assert settings["min_delivery_chars"] == 40 + assert settings["min_delivery_evidence"] == 1 + assert settings["fallback_policy"] == "reassign_then_gap" + + +@pytest.mark.asyncio +async def test_runner_quality_zero_rework_cap_no_rework_granted(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + # max_rework_attempts=0 must be honoured: a failing delivery is NOT sent + # back for rework — it exhausts straight into the fallback (controlled gap + # here, no backup reassignment budget either). + _, calls, _, final_run, final_session = await _run( + q_svc, monkeypatch, + deliveries={"seat-a": [SHORT_DELIVERY], "seat-b": [GOOD_DELIVERY]}, + review_plan=[ + {"seat-a": {"decision": "return_for_rework", "reason": "太短"}, "seat-b": {"decision": "accept"}}, + ], + config_overrides={"max_rework_attempts": 0, "max_reassignments": 0}, + ) + + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + by_key = {t["task_key"]: t for t in tasks} + task_a = by_key["01:seat-a"] + assert task_a["status"] == "controlled_gap" + assert task_a["attempt_count"] == 1, "no rework delivery may happen with a 0 cap" + assert task_a["rework_count"] == 0 + assert task_a["reassign_count"] == 0 + # Only the initial delivery + the (ignored) rework request — no rework turn. + expert_turns_a = [s for s, a, _t in calls if a == "seat-a" and s in ("expert", "rework")] + assert len(expert_turns_a) == 1 + + result = final_session.get("final_result") or {} + assert result.get("quality_decision") == "controlled_gap" + + +@pytest.mark.asyncio +async def test_runner_quality_zero_min_evidence_accepts_refless_delivery(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + # min_delivery_evidence=0 must be honoured: a conclusion-only delivery with + # no evidence refs passes the deterministic evidence gate. + no_ref_delivery = json.dumps({ + "summary": "结论：该市场存在明确增长机会，建议优先进入华东区域并配套本地化运营资源投入，同时预留华南试点预算", + "findings": [], + "evidence_refs": [], + "open_questions": [], + "self_check": "ok", + }) + _, _calls, _, final_run, _final_session = await _run( + q_svc, monkeypatch, + deliveries={"seat-a": [no_ref_delivery], "seat-b": [no_ref_delivery]}, + review_plan=[{"seat-a": {"decision": "accept"}, "seat-b": {"decision": "accept"}}], + config_overrides={"min_delivery_evidence": 0}, + ) + + assert final_run["status"] == "completed" + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + assert all(t["status"] == "accepted" for t in tasks) + + +# ── 8. P1: restart recovery re-submission hits duplicate via digest ──────── + + +@pytest.mark.asyncio +async def test_runner_quality_restart_resubmission_hits_duplicate_digest(q_svc, monkeypatch): + pytest.importorskip("agent_framework") + + from app.maf_roundtable.quality_contracts import build_task_contract, conclusion_digest + + good = json.loads(GOOD_DELIVERY.replace("@TOOL_REF@", "tool:kb_search:1")) + digest = conclusion_digest(good["summary"]) + + # Simulate the pre-restart state: seat-a's first delivery was rejected and + # its conclusion digest was persisted to the task history; then the process + # restarted. The task row (with history) survives in the DB. + contract = build_task_contract( + task_goal="市场增长机会分析", delivery_requirements=["证据", "优先级"], + max_rework_attempts=2, fallback_policy="reassign_then_gap", + ) + seeded = await q_svc.store.seat_tasks.create_task( + session_id="q-sess", run_id="q-run", task_key="01:seat-a", + agent_id="seat-a", contract=contract.model_dump(), required=True, + ) + await q_svc.store.seat_tasks.update_task(seeded["id"], { + "status": "rework", + "attempt_count": 1, + "rework_count": 1, + "history": [{ + "attempt": 1, "agent_id": "seat-a", "round": 1, + "schema_status": "ok", "failure_codes": ["below_min_length"], + "conclusion_sha256": digest, + }], + }) + + # After the restart the seat re-submits the SAME conclusion. It must be + # flagged duplicate_content from the persisted digest — even though the + # in-memory prior_conclusions starts empty on a fresh process. + _, _calls, _, final_run, _final_session = await _run( + q_svc, monkeypatch, + deliveries={"seat-a": [GOOD_DELIVERY], "seat-b": [GOOD_DELIVERY]}, + review_plan=[ + {"seat-a": {"decision": "accept"}, "seat-b": {"decision": "accept"}}, + {"seat-a": {"decision": "accept"}}, + {"seat-a": {"decision": "accept"}}, + {"seat-a": {"decision": "accept"}}, + ], + ) + + tasks = await q_svc.store.seat_tasks.list_by_run("q-run") + by_key = {t["task_key"]: t for t in tasks} + task_a = by_key["01:seat-a"] + # The re-submission was deterministically flagged as a duplicate. + check = task_a.get("deterministic_check") or {} + history_codes = [ + code + for h in (task_a.get("history") or []) + for code in (h.get("failure_codes") or []) + ] + assert "duplicate_content" in (check.get("failure_codes") or []) or "duplicate_content" in history_codes + # And a duplicate can never be accepted, even with the manager saying yes. + assert task_a["status"] != "accepted" + assert final_run["status"] == "completed" diff --git a/offline-backend-20260512/backend/tests/test_maf_phase4_task_init.py b/offline-backend-20260512/backend/tests/test_maf_phase4_task_init.py new file mode 100644 index 00000000..e89aa5af --- /dev/null +++ b/offline-backend-20260512/backend/tests/test_maf_phase4_task_init.py @@ -0,0 +1,256 @@ +"""Stage 4 (R3-Q) P0-4: atomic seat-task initialization and safe recovery. + +The old initializer created tasks one by one and short-circuited with +``if existing: return existing`` — a crash after the first create permanently +dropped the remaining required seats on recovery, and the quality gate could +then pass on an incomplete task set. + +These tests prove: + +- ``get_or_create_tasks`` creates the FULL expected set in one transaction; +- after a simulated partial initialization (only the first task exists), + recovery fills the missing tasks and never touches existing rows; +- a frozen contract / lifecycle progress is NEVER overwritten; +- two concurrent recovery calls converge on one complete set without raising + unique-key violations; +- ``QualitySeatTasksExecutor._load_or_create_tasks`` derives the expected set + from the dispatch assignments and fills gaps; +- the quality gate blocks when an expected required task is missing. +""" + +from __future__ import annotations + +import asyncio +from datetime import datetime +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +import pytest_asyncio +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.maf_roundtable.workflow_executors import ( + QualityGateExecutor, + QualitySeatTasksExecutor, + RoundtableState, +) +from deerflow.persistence.maf_roundtable import make_maf_roundtable_store +from deerflow.persistence.maf_roundtable.model import ( + Base, + MafRoundtableRunRow, + MafRoundtableSessionRow, +) + +CONTRACT_A = { + "objective": "原始合同目标 A", + "expected_deliverable": "结论+证据", + "acceptance_criteria": ["证据"], + "required": True, + "max_rework_attempts": 2, + "fallback_policy": "reassign_then_gap", +} +CONTRACT_B = { + "objective": "试图覆盖的合同 B", + "expected_deliverable": "别的", + "acceptance_criteria": [], + "required": True, + "max_rework_attempts": 0, + "fallback_policy": "controlled_gap", +} + + +@pytest_asyncio.fixture +async def task_store(tmp_path): + engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'tasks.db'}", echo=False) + try: + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + sf = async_sessionmaker(engine, expire_on_commit=False) + store = make_maf_roundtable_store(sf) + now = datetime(2026, 1, 1) + async with sf() as session: + session.add(MafRoundtableSessionRow( + id="t-sess", user_id="u1", title="t", task_text="t", + mode="structured", status="pending", write_policy="auto", + manager_agent_id="manager-1", max_rounds=8, + created_at=now, updated_at=now, + )) + session.add(MafRoundtableRunRow( + id="t-run", session_id="t-sess", status="pending", + created_at=now, updated_at=now, + )) + await session.commit() + yield store + finally: + await engine.dispose() + + +def _expected_specs() -> list[dict]: + return [ + {"task_key": "01:seat-a", "agent_id": "seat-a", "contract": dict(CONTRACT_A), "required": True}, + {"task_key": "02:seat-b", "agent_id": "seat-b", "contract": dict(CONTRACT_A), "required": True}, + {"task_key": "03:seat-c", "agent_id": "seat-c", "contract": dict(CONTRACT_A), "required": True}, + ] + + +# ── repository: atomic full-set creation ──────────────────────────────────── + + +@pytest.mark.asyncio +async def test_get_or_create_tasks_creates_full_set(task_store): + tasks = await task_store.seat_tasks.get_or_create_tasks( + session_id="t-sess", run_id="t-run", expected=_expected_specs(), + ) + assert [t["task_key"] for t in tasks] == ["01:seat-a", "02:seat-b", "03:seat-c"] + assert all(t["status"] == "assigned" for t in tasks) + assert all(t["contract"]["objective"] == "原始合同目标 A" for t in tasks) + + +@pytest.mark.asyncio +async def test_get_or_create_tasks_recovers_partial_initialization(task_store): + # Simulate the old crash mode: only the FIRST task survived initialization. + await task_store.seat_tasks.create_task( + session_id="t-sess", run_id="t-run", task_key="01:seat-a", + agent_id="seat-a", contract=dict(CONTRACT_A), required=True, + ) + + tasks = await task_store.seat_tasks.get_or_create_tasks( + session_id="t-sess", run_id="t-run", expected=_expected_specs(), + ) + # The complete set comes back — the missing seats are filled in. + assert [t["task_key"] for t in tasks] == ["01:seat-a", "02:seat-b", "03:seat-c"] + by_key = {t["task_key"]: t for t in tasks} + # The pre-existing row kept its identity and frozen contract. + assert by_key["01:seat-a"]["contract"]["objective"] == "原始合同目标 A" + assert by_key["01:seat-a"]["status"] == "assigned" + + +@pytest.mark.asyncio +async def test_get_or_create_tasks_never_overwrites_frozen_contract(task_store): + existing = await task_store.seat_tasks.create_task( + session_id="t-sess", run_id="t-run", task_key="01:seat-a", + agent_id="seat-c", contract=dict(CONTRACT_A), required=True, + ) + # Lifecycle progressed (e.g. before a restart): reassignment + attempts. + await task_store.seat_tasks.update_task(existing["id"], { + "status": "accepted", + "attempt_count": 3, + "rework_count": 2, + "reassign_count": 1, + }) + + specs = _expected_specs() + specs[0]["contract"] = dict(CONTRACT_B) # recovery must NOT apply this + tasks = await task_store.seat_tasks.get_or_create_tasks( + session_id="t-sess", run_id="t-run", expected=specs, + ) + by_key = {t["task_key"]: t for t in tasks} + kept = by_key["01:seat-a"] + assert kept["id"] == existing["id"] + assert kept["contract"]["objective"] == "原始合同目标 A" + assert kept["agent_id"] == "seat-c" + assert kept["status"] == "accepted" + assert kept["attempt_count"] == 3 + assert kept["rework_count"] == 2 + assert kept["reassign_count"] == 1 + + +@pytest.mark.asyncio +async def test_get_or_create_tasks_concurrent_recovery(task_store): + # Two recoveries race on the same empty run: both must return the complete + # set, neither may raise a unique-key violation, and exactly one row per + # task_key may exist afterwards. + results = await asyncio.gather( + task_store.seat_tasks.get_or_create_tasks( + session_id="t-sess", run_id="t-run", expected=_expected_specs(), + ), + task_store.seat_tasks.get_or_create_tasks( + session_id="t-sess", run_id="t-run", expected=_expected_specs(), + ), + ) + for tasks in results: + assert [t["task_key"] for t in tasks] == ["01:seat-a", "02:seat-b", "03:seat-c"] + + final = await task_store.seat_tasks.list_by_run("t-run") + keys = [t["task_key"] for t in final] + assert sorted(keys) == ["01:seat-a", "02:seat-b", "03:seat-c"] + assert len(keys) == len(set(keys)), "no duplicate tasks" + + +# ── executor: expected set from dispatch assignments ──────────────────────── + + +def _make_state(**overrides) -> RoundtableState: + kwargs = { + "session_id": "t-sess", + "run_id": "t-run", + "task_text": "分析市场", + "mode": "structured", + "model_name": None, + "reasoning_effort": None, + "max_rounds": 8, + "write_policy": "auto", + "manager_agent_id": "manager-1", + "seat_agent_ids": ["seat-a", "seat-b"], + "thread_map": {}, + "capability_snapshot": None, + "seat_timeout_seconds": 5, + "schema_repair_enabled": True, + "intent_type": "parallel_consultation", + "dispatch_plan": { + "execution_mode": "parallel", + "assignments": [ + {"agent_id": "seat-a", "task_goal": "目标一", "delivery_requirements": ["证据"]}, + {"agent_id": "seat-b", "task_goal": "目标二", "delivery_requirements": ["建议"]}, + ], + }, + "dispatch_contract_enabled": True, + "quality_gate_enabled": True, + } + kwargs.update(overrides) + return RoundtableState(**kwargs) + + +@pytest.mark.asyncio +async def test_executor_load_or_create_fills_missing_tasks(task_store): + # Partial init: only the first assignment's task exists. + await task_store.seat_tasks.create_task( + session_id="t-sess", run_id="t-run", task_key="01:seat-a", + agent_id="seat-a", contract=dict(CONTRACT_A), required=True, + ) + state = _make_state() + adapter = SimpleNamespace(_service=SimpleNamespace(store=task_store)) + executor = QualitySeatTasksExecutor(adapter, state) + + tasks = await executor._load_or_create_tasks() + assert [t["task_key"] for t in tasks] == ["01:seat-a", "02:seat-b"] + by_key = {t["task_key"]: t for t in tasks} + # Existing contract untouched; missing task created from the assignment. + assert by_key["01:seat-a"]["contract"]["objective"] == "原始合同目标 A" + assert by_key["02:seat-b"]["contract"]["objective"] == "目标二" + assert by_key["02:seat-b"]["status"] == "assigned" + + # Idempotent second recovery. + again = await executor._load_or_create_tasks() + assert [t["id"] for t in again] == [t["id"] for t in tasks] + + +@pytest.mark.asyncio +async def test_quality_gate_blocks_missing_required_task(task_store): + state = _make_state() + # Only ONE of the two expected required tasks is represented. + state.seat_task_statuses = { + "01:seat-a": { + "task_key": "01:seat-a", "status": "accepted", + "agent_id": "seat-a", "required": True, + }, + } + adapter = SimpleNamespace( + _service=SimpleNamespace(store=SimpleNamespace(seat_tasks=SimpleNamespace())), + _emit_critical=AsyncMock(), + ) + gate = QualityGateExecutor(adapter, state) + ctx = SimpleNamespace(send_message=AsyncMock(), yield_output=AsyncMock()) + with pytest.raises(RuntimeError, match="quality gate blocked"): + await gate.handle({}, ctx) + assert state.status != "completed" diff --git a/offline-backend-20260512/backend/tests/test_maf_quality_contracts.py b/offline-backend-20260512/backend/tests/test_maf_quality_contracts.py new file mode 100644 index 00000000..3279f7d7 --- /dev/null +++ b/offline-backend-20260512/backend/tests/test_maf_quality_contracts.py @@ -0,0 +1,565 @@ +"""Stage 4 (R3-Q) pure-policy tests: task contracts, deterministic quality +checks, manager review schema, and server-owned decision finalization. + +These tests exercise ``quality_contracts`` directly (no MAF SDK, no I/O) and +prove the weak-model protections: + +- a delivery that failed deterministic checks can never be accepted, no matter + what the manager model claims; +- rework is bounded by the frozen contract's ``max_rework_attempts``; +- reassignment is bounded by the server-configured cap; +- an exhausted task ends as ``controlled_gap`` / ``ask_user``, never as a + disguised success. +""" + +from __future__ import annotations + +from app.maf_roundtable.quality_contracts import ( + OUTCOME_ACCEPTED, + OUTCOME_ASK_USER, + OUTCOME_CONTROLLED_GAP, + OUTCOME_REASSIGNED, + OUTCOME_REWORK, + InvalidTaskContractError, + MafManagerReviewItem, + MafManagerReviewOutput, + MafSeatDelivery, + MafTaskContract, + build_manager_review_prompt, + build_task_contract, + check_delivery_deterministic, + conclusion_digest, + delivery_from_seat_output, + finalize_review_decision, + make_controlled_gap, + parse_manager_review, + repair_requirements_for_codes, +) + +# ── task contract freezing ────────────────────────────────────────────────── + + +def test_build_task_contract_freezes_from_dispatch_assignment(): + contract = build_task_contract( + task_goal="评估竞品定价策略", + delivery_requirements=["风险清单", "依据", "优先级", "建议"], + required=True, + max_rework_attempts=2, + fallback_policy="reassign_then_gap", + ) + assert contract.objective == "评估竞品定价策略" + assert contract.expected_deliverable + assert contract.acceptance_criteria == ["风险清单", "依据", "优先级", "建议"] + assert contract.required is True + assert contract.max_rework_attempts == 2 + assert contract.fallback_policy == "reassign_then_gap" + + +def test_build_task_contract_rejects_empty_objective(): + try: + build_task_contract(task_goal=" ", delivery_requirements=[]) + except InvalidTaskContractError: + return + raise AssertionError("empty objective must raise InvalidTaskContractError") + + +def test_build_task_contract_strips_blank_criteria(): + contract = build_task_contract(task_goal="t", delivery_requirements=["a", " ", "b"]) + assert contract.acceptance_criteria == ["a", "b"] + + +# ── delivery mapping ──────────────────────────────────────────────────────── + + +def test_delivery_from_seat_output_maps_all_fields(): + delivery = delivery_from_seat_output( + structured_output={ + "summary": "结论：竞品定价偏高", + "findings": ["find-1", {"text": "find-2"}], + "evidence_refs": [{"ref": "ref-a"}, "ref-b"], + "open_questions": ["q1"], + "self_check": "已覆盖优先级标准", + }, + raw_content='{"summary": "结论：竞品定价偏高"}', + ) + assert delivery.conclusion == "结论：竞品定价偏高" + assert delivery.evidence == ["find-1", "find-2", "ref-a", "ref-b"] + # P1: only evidence_refs can satisfy the evidence requirement. + assert delivery.evidence_refs == ["ref-a", "ref-b"] + assert delivery.unresolved_items == ["q1"] + assert delivery.self_check == "已覆盖优先级标准" + + +def test_delivery_from_seat_output_schema_invalid_is_empty(): + delivery = delivery_from_seat_output(structured_output=None, raw_content="not json") + assert delivery.conclusion == "" + assert delivery.evidence == [] + assert delivery.evidence_refs == [] + + +# ── deterministic checks ──────────────────────────────────────────────────── + + +def _good_delivery() -> MafSeatDelivery: + return MafSeatDelivery( + conclusion="结论：该市场存在明确增长机会，建议优先进入华东区域并配套本地化运营资源投入，同时预留华南试点预算", + evidence=["ref-1", "ref-2"], + evidence_refs=["ref-1", "ref-2"], + unresolved_items=[], + self_check="ok", + ) + + +def test_check_delivery_deterministic_passes_well_formed(): + delivery = _good_delivery() + result = check_delivery_deterministic( + delivery=delivery, raw_content='{"summary":"x"}', schema_status="ok", + ) + assert result["passed"] is True + assert result["failure_codes"] == [] + + +def test_check_delivery_deterministic_empty_content(): + result = check_delivery_deterministic( + delivery=_good_delivery(), raw_content=" ", schema_status="ok", + ) + assert result["passed"] is False + assert "empty_content" in result["failure_codes"] + + +def test_check_delivery_deterministic_schema_invalid(): + result = check_delivery_deterministic( + delivery=_good_delivery(), raw_content="not json", schema_status="schema_invalid", + ) + assert result["passed"] is False + assert "schema_invalid" in result["failure_codes"] + + +def test_check_delivery_deterministic_missing_conclusion(): + delivery = MafSeatDelivery(conclusion="", evidence=["e"], self_check="") + result = check_delivery_deterministic( + delivery=delivery, raw_content='{"summary":""}', schema_status="ok", + ) + assert "missing_conclusion" in result["failure_codes"] + + +def test_check_delivery_deterministic_below_min_length(): + delivery = MafSeatDelivery(conclusion="好", evidence=["e"], self_check="") + result = check_delivery_deterministic( + delivery=delivery, raw_content='{"summary":"好"}', schema_status="ok", min_length=40, + ) + assert "below_min_length" in result["failure_codes"] + + +def test_check_delivery_deterministic_missing_evidence(): + delivery = MafSeatDelivery(conclusion="这是一个足够长的结论用于通过最小长度校验", evidence=[], self_check="") + result = check_delivery_deterministic( + delivery=delivery, raw_content='{"summary":"x"}', schema_status="ok", min_evidence=1, + ) + assert "missing_evidence" in result["failure_codes"] + + +def test_check_delivery_findings_alone_never_satisfy_evidence(): + # P1: findings are candidate claims only — a delivery full of findings but + # without evidence_refs still fails the evidence requirement. + delivery = MafSeatDelivery( + conclusion="这是一个足够长的结论用于通过最小长度校验，附带大量候选主张作为干扰项", + evidence=["我查到 A", "我查到 B", "find-3"], + evidence_refs=[], + self_check="", + ) + result = check_delivery_deterministic( + delivery=delivery, raw_content='{"summary":"x"}', schema_status="ok", min_evidence=1, + ) + assert result["passed"] is False + assert "missing_evidence" in result["failure_codes"] + + +def test_check_delivery_invented_evidence_ref_fails_unverified(): + # P1: a cited ref that the fact ledger does not back is a hallucination. + delivery = MafSeatDelivery( + conclusion="这是一个足够长的结论用于通过最小长度校验，引用了不存在的证据", + evidence=["fake-ref-001"], + evidence_refs=["fake-ref-001"], + self_check="", + ) + result = check_delivery_deterministic( + delivery=delivery, + raw_content='{"summary":"x"}', + schema_status="ok", + min_evidence=1, + verified_evidence_refs={"tool:kb_search:1", "ref-real"}, + ) + assert result["passed"] is False + assert "unverified_evidence_ref" in result["failure_codes"] + # The invented ref also cannot count toward the evidence minimum. + assert "missing_evidence" in result["failure_codes"] + + +def test_check_delivery_verified_evidence_ref_passes(): + delivery = MafSeatDelivery( + conclusion="这是一个足够长的结论用于通过最小长度校验，引用了事实账本中真实存在的证据引用，满足最小证据数量要求", + evidence=["tool:kb_search:1"], + evidence_refs=["tool:kb_search:1"], + self_check="", + ) + result = check_delivery_deterministic( + delivery=delivery, + raw_content='{"summary":"x"}', + schema_status="ok", + min_evidence=1, + verified_evidence_refs={"tool:kb_search:1"}, + ) + assert result["passed"] is True + assert result["failure_codes"] == [] + + +def test_check_delivery_deterministic_duplicate_content_by_digest(): + delivery = MafSeatDelivery( + conclusion="结论：与上次完全相同的内容", evidence=["e"], evidence_refs=["e"], self_check="", + ) + result = check_delivery_deterministic( + delivery=delivery, + raw_content="x", + schema_status="ok", + prior_conclusion_digests={conclusion_digest("结论：与上次完全相同的内容")}, + ) + assert "duplicate_content" in result["failure_codes"] + + +def test_conclusion_digest_normalizes_whitespace(): + assert conclusion_digest("结论：内容 相同") == conclusion_digest("结论：内容 相同\n") + assert conclusion_digest("a") != conclusion_digest("b") + + +def test_repair_requirements_cover_failure_codes(): + reqs = repair_requirements_for_codes(["missing_evidence", "below_min_length"]) + assert len(reqs) == 2 + assert any("证据" in r for r in reqs) + + +# ── manager review parsing ────────────────────────────────────────────────── + + +def test_parse_manager_review_valid(): + model, errors = parse_manager_review("""{"decisions": [ + {"agent_id": "seat-a", "decision": "return_for_rework", + "reason": "缺依据", "revision_requirements": ["补证据"]} + ]}""") + assert errors == [] + assert model is not None + assert model.decisions[0].agent_id == "seat-a" + assert model.decisions[0].decision == "return_for_rework" + + +def test_parse_manager_review_fenced_json(): + model, errors = parse_manager_review("```json\n{\"decisions\": []}\n```") + assert errors == [] + assert model is not None + assert model.decisions == [] + + +def test_parse_manager_review_invalid_decision_value(): + model, errors = parse_manager_review("""{"decisions": [ + {"agent_id": "seat-a", "decision": "fake_decision", "reason": "x"} + ]}""") + assert model is None + assert errors + + +def test_parse_manager_review_missing_reason(): + model, errors = parse_manager_review("""{"decisions": [ + {"agent_id": "seat-a", "decision": "accept"} + ]}""") + assert model is None + assert errors + + +def test_parse_manager_review_not_json(): + model, errors = parse_manager_review("definitely not json") + assert model is None + assert errors + + +def test_build_manager_review_prompt_contains_contract_and_check(): + prompt = build_manager_review_prompt( + task_text="分析市场", + review_items=[{ + "agent_id": "seat-a", + "task_key": "01:seat-a", + "attempt": 1, + "contract": {"objective": "评估竞品"}, + "delivery": {"conclusion": "结论"}, + "check": {"passed": False, "failure_codes": ["missing_evidence"]}, + }], + ) + assert "评估竞品" in prompt + assert "missing_evidence" in prompt + assert "seat-a" in prompt + + +# ── server-owned decision finalization ────────────────────────────────────── + + +def _contract(**overrides) -> MafTaskContract: + base = MafTaskContract( + objective="评估竞品", + expected_deliverable="结论+证据", + acceptance_criteria=["证据"], + required=True, + max_rework_attempts=2, + fallback_policy="reassign_then_gap", + ) + return base.model_copy(update=overrides) + + +def _item(decision: str, reason: str = "r", requirements: list[str] | None = None) -> MafManagerReviewItem: + return MafManagerReviewItem( + agent_id="seat-a", decision=decision, reason=reason, + revision_requirements=list(requirements or []), + ) + + +def _check(passed: bool, codes: list[str] | None = None) -> dict: + return {"passed": passed, "failure_codes": list(codes or [])} + + +def test_finalize_deterministic_fail_never_accepted_even_if_manager_accepts(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(False, ["below_min_length"]), + review_item=_item("accept", "看起来可以"), + rework_count=0, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REWORK + assert final["source"] == "deterministic" + assert final["revision_requirements"] + + +def test_finalize_deterministic_fail_uses_manager_rework_requirements(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(False, ["below_min_length"]), + review_item=_item("return_for_rework", "太短", ["请补充具体依据"]), + rework_count=0, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REWORK + assert final["revision_requirements"] == ["请补充具体依据"] + + +def test_finalize_deterministic_fail_at_cap_reassigns(): + final = finalize_review_decision( + contract=_contract(max_rework_attempts=2), + check_result=_check(False, ["below_min_length"]), + review_item=None, + rework_count=2, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REASSIGNED + + +def test_finalize_deterministic_fail_at_cap_no_backup_controlled_gap(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(False, ["below_min_length"]), + review_item=None, + rework_count=2, + reassign_count=0, + has_backup_seat=False, + ) + assert final["decision"] == OUTCOME_CONTROLLED_GAP + assert final["source"] == "deterministic" + + +def test_finalize_deterministic_fail_at_cap_ask_user_fallback(): + final = finalize_review_decision( + contract=_contract(fallback_policy="ask_user"), + check_result=_check(False, ["below_min_length"]), + review_item=None, + rework_count=2, + reassign_count=1, + has_backup_seat=False, + ) + assert final["decision"] == OUTCOME_ASK_USER + + +def test_finalize_pass_manager_accept_accepted(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("accept", "达标"), + rework_count=0, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_ACCEPTED + assert final["source"] == "manager" + + +def test_finalize_pass_no_valid_review_never_accepts_forces_rework(): + # Deterministic checks passed, but the manager review is missing/invalid: + # under the rework cap the server forces a system rework — never accepts. + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=None, + rework_count=0, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REWORK + assert final["source"] == "system" + assert final["reason"] == "manager_review_missing_or_invalid" + assert final["revision_requirements"] + + +def test_finalize_pass_no_valid_review_at_cap_follows_fallback(): + # Rework cap exhausted: the contract fallback policy decides — reassign + # first, then controlled_gap, then ask_user. Never accepted. + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=None, + rework_count=2, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REASSIGNED + assert final["source"] == "system" + assert final["reason"] == "manager_review_missing_or_invalid" + + final_gap = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=None, + rework_count=2, + reassign_count=1, + has_backup_seat=False, + ) + assert final_gap["decision"] == OUTCOME_CONTROLLED_GAP + assert final_gap["source"] == "system" + + final_ask = finalize_review_decision( + contract=_contract(fallback_policy="ask_user"), + check_result=_check(True), + review_item=None, + rework_count=2, + reassign_count=1, + has_backup_seat=False, + ) + assert final_ask["decision"] == OUTCOME_ASK_USER + assert final_ask["source"] == "system" + + +def test_finalize_pass_manager_rework_under_cap(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("return_for_rework", "再补充"), + rework_count=1, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REWORK + + +def test_finalize_pass_manager_rework_at_cap_reassigns_then_gap(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("return_for_rework", "仍不足"), + rework_count=2, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REASSIGNED + + final2 = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("return_for_rework", "仍不足"), + rework_count=2, + reassign_count=1, + has_backup_seat=False, + ) + assert final2["decision"] == OUTCOME_CONTROLLED_GAP + + +def test_finalize_manager_reassign_without_backup_degrades(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("reassign", "换人"), + rework_count=2, + reassign_count=0, + has_backup_seat=False, + ) + assert final["decision"] == OUTCOME_CONTROLLED_GAP + + +def test_finalize_manager_reassign_with_backup(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("reassign", "换 seat-c"), + rework_count=2, + reassign_count=0, + has_backup_seat=True, + ) + assert final["decision"] == OUTCOME_REASSIGNED + + +def test_finalize_manager_ask_user_passthrough(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("ask_user", "需要用户补充"), + rework_count=2, + reassign_count=1, + has_backup_seat=False, + ) + assert final["decision"] == OUTCOME_ASK_USER + + +def test_finalize_manager_controlled_gap_passthrough(): + final = finalize_review_decision( + contract=_contract(), + check_result=_check(True), + review_item=_item("controlled_gap", "无法完成"), + rework_count=2, + reassign_count=1, + has_backup_seat=False, + ) + assert final["decision"] == OUTCOME_CONTROLLED_GAP + assert final["source"] == "manager" + + +# ── controlled gap records ────────────────────────────────────────────────── + + +def test_make_controlled_gap_marks_risk(): + gap = make_controlled_gap( + task_key="01:seat-a", + agent_id="seat-a", + objective="评估竞品", + reason="返工上限仍无合格交付", + failure_codes=["below_min_length"], + attempts=3, + ) + assert gap["task_key"] == "01:seat-a" + assert gap["agent_id"] == "seat-a" + assert gap["attempts"] == 3 + assert "不得作为已验证事实" in gap["risk"] + + +def test_mamager_review_output_model_roundtrip(): + output = MafManagerReviewOutput(decisions=[ + {"agent_id": "a", "decision": "accept", "reason": "ok"}, + ]) + dumped = output.model_dump() + assert dumped["decisions"][0]["agent_id"] == "a" diff --git a/offline-backend-20260512/backend/tests/test_maf_roundtable_isolation.py b/offline-backend-20260512/backend/tests/test_maf_roundtable_isolation.py index e6b37b1c..9629af3a 100644 --- a/offline-backend-20260512/backend/tests/test_maf_roundtable_isolation.py +++ b/offline-backend-20260512/backend/tests/test_maf_roundtable_isolation.py @@ -169,6 +169,7 @@ async def test_maf_tables_isolated_from_legacy_roundtable(store): "maf_roundtable_action_leases", "maf_roundtable_approvals", "maf_roundtable_seat_runs", + "maf_roundtable_seat_tasks", "maf_roundtable_action_secrets", "maf_roundtable_conversations", "maf_roundtable_messages",