fix(定时任务): require_html 选内置模板时用真实模板出页,保留全部子页面
- 模板是 JS 驱动多页 SPA,子页面靠脚本从 DATA 渲染;之前用 style_reference 抽 CSS 摘要 让模型自由渲染,结果只剩主页、子页面全丢 - 改为:_run_html_page_attempt 见 tpl:<id> 参考即提前分支到 _run_html_template_reference, 用真实模板文件出页——可转换模板=检索数据→转换→fill_template;只展示模板=原样输出; 转换失败兜底默认数据,保证页面与子页面始终可用 - 抽出 _convert_to_template_data / _finalize_template_html 复用,删除已废弃的 style_reference - 测试覆盖:真模板保留 const DATA/script/id=tabs;只展示原样;转换失败回退 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
ec60b77a71
commit
ba885ca201
3 files changed
+242
-152
No files matched your search
@@ -300,7 +300,7 @@ Two on-demand/automatic refresh paths layer on top:
|
||||
|
||||
**Unified output switches (`require_html` + `require_markdown`).** The task form no longer splits "???? / HTML ??" into separate kinds. `execution_context` now carries two **independent** booleans that may both be on: `require_html` (???? HTML) and `require_markdown` (???? Markdown). `_task_requires_html()` returns true for `require_html` **or** legacy `task_kind == "html_page"` (so old tasks need no data migration), and `_run_task_attempt()` dispatches to `_run_html_page_attempt` when it is true, else the regular agent attempt. When **both** switches are on, `_gather_html_data()` returns `(text, files)` ? the gather agent run is additionally told to save a `.md` file ? and `_run_html_page_attempt` validates it (`MarkdownRequirementError` on failure) and surfaces it alongside the generated page, so the run delivers both artifacts. The single main `prompt` doubles as the HTML page requirement (`html_config.page_prompt`); there is no separate page-prompt field. Tests: `tests/test_scheduled_task_require_html.py` + the markdown-coexistence cases in `tests/test_html_page_tasks.py`.
|
||||
|
||||
**Template-fill mode (JSON → 固定 HTML 模板填充).** A third execution mode, distinct from the free-form `require_html` render pipeline: convert a user-supplied JSON into a **fixed** HTML template's data structure and fill it. Triggered when `execution_context.template_config.template_id` is set (chosen in the task form after selecting the built-in **「模板数据转换器」** agent `template-json-builder`). `_task_template_id()` reads it and `_run_task_attempt()` dispatches to `_run_template_fill_attempt` **before** the `require_html` check. Pipeline: take the source (the task `prompt`, pasted or uploaded .json) — which may be **pure JSON, plain text, or text-with-embedded-JSON** (`template_fill.extract_embedded_json` pre-extracts any embedded `{…}`/`[…]` with a string-aware balanced scan and surfaces it as a labeled block while keeping the full original text; plain text → no block, the model reads it semantically) → load the selected agent's `SOUL.md`+model and run a **direct** `_invoke_conversion_model` call (system=SOUL, human=conversion prompt carrying the template's compact **schema skeleton** + the recognized-JSON block + source) → `extract_json_object` → `template_fill.validate_data` (top-level keys + container types) → **on failure feed the structure errors + `field_coverage_gaps` back and retry up to `_MAX_TEMPLATE_FILL_RETRIES`=2** → `template_fill.fill_template` replaces the template's `const DATA={…}` line (and refreshes `MANIFEST.系统.更新时间`) → archive `index.html` + persist a `scheduled_html_pages` row → return `format:"html"` (reusing the existing HTML preview at `/runs/{id}/html`). Any terminal failure raises `TemplateFillError` → run marked failed (no retry, message shown). **Pure engine** `deerflow/runtime/scheduler/template_fill.py` (harness, no `app.*`): the template registry (`TEMPLATES`: id=`no1` name=`no.1` 台湾滨海防卫大屏, and id=`no2` name=`no.2` 台湾飞行部队平台 with **`convertible=False`** — a 「只展示」 template that renders its existing HTML **as-is**, no model/validation/fill, until conversion is wired in later; `_run_template_fill_attempt` short-circuits to `read_template_text` for it, and `list_templates`/`is_convertible` expose the flag so the form hides the JSON input), `list_templates`/`reference_data`/`schema_skeleton_json`/`validate_data`/`field_coverage_gaps`/`fill_template` — **the template's own embedded `DATA` is the schema source of truth** (parsed at load, 1-sample-per-array skeleton fed to the LLM, top-level structure as the validation basis); template HTML files live under `runtime/scheduler/templates/<id>/`, add a `TemplateSpec` to register a no.2. The agent is seeded like the AI-writing ones: `app/gateway/routers/_template_builder_seed.py` + `_template_builder_seed_assets/template-json-builder/{config.yaml,SOUL.md}`, called from the lifespan before `_sync_legacy_agents` (which upserts it into the `agents` table as built-in/published so it shows in `/api/agents` → the task form's 执行智能体 dropdown). Template list API: `GET /api/scheduled-tasks/html-templates` → `{templates:[{id,name,description,convertible}]}` (open). **Debug-render API** `POST /api/scheduled-tasks/template-fill/render` `{template_id, text?|data?}` → `{ok, display_only?, errors, gaps, html}` (open): no model call — `extract_embedded_json(text)` (or `data`) → `validate_data` → `fill_template`, returning the filled HTML **plus** the validation errors/field-gaps (fills even when invalid so you can see the effect). Powers the chat-page debug button below; non-convertible templates return their raw HTML. **Templates as `require_html` style references**: the 参考收藏页面 list in the task form also offers the templates (value `tpl:<id>`, e.g. `tpl:no1`); when one is picked, `_run_html_page_attempt` calls `template_fill.style_reference(id)` (extracts the `<style>` CSS + script-stripped body skeleton, ~13–28KB — the giant DATA/scripts are dropped) and feeds it as `reference_style`/`reference_layout` so the free-form HTML render mimics the template's look. `template_fill.reference_template_id("tpl:no1")→"no1"` distinguishes a template marker from a real favorite id. Frontend: when the agent dropdown = `template-json-builder` the form hides the HTML/Markdown switches and shows a 「HTML模板」 select (`useHtmlTemplates`) + a .json upload (reads file text into the prompt box). Tests: `tests/test_template_fill.py`, `tests/test_template_fill_attempt.py`, `tests/test_template_builder_seed.py`.
|
||||
**Template-fill mode (JSON → 固定 HTML 模板填充).** A third execution mode, distinct from the free-form `require_html` render pipeline: convert a user-supplied JSON into a **fixed** HTML template's data structure and fill it. Triggered when `execution_context.template_config.template_id` is set (chosen in the task form after selecting the built-in **「模板数据转换器」** agent `template-json-builder`). `_task_template_id()` reads it and `_run_task_attempt()` dispatches to `_run_template_fill_attempt` **before** the `require_html` check. Pipeline: take the source (the task `prompt`, pasted or uploaded .json) — which may be **pure JSON, plain text, or text-with-embedded-JSON** (`template_fill.extract_embedded_json` pre-extracts any embedded `{…}`/`[…]` with a string-aware balanced scan and surfaces it as a labeled block while keeping the full original text; plain text → no block, the model reads it semantically) → load the selected agent's `SOUL.md`+model and run a **direct** `_invoke_conversion_model` call (system=SOUL, human=conversion prompt carrying the template's compact **schema skeleton** + the recognized-JSON block + source) → `extract_json_object` → `template_fill.validate_data` (top-level keys + container types) → **on failure feed the structure errors + `field_coverage_gaps` back and retry up to `_MAX_TEMPLATE_FILL_RETRIES`=2** → `template_fill.fill_template` replaces the template's `const DATA={…}` line (and refreshes `MANIFEST.系统.更新时间`) → archive `index.html` + persist a `scheduled_html_pages` row → return `format:"html"` (reusing the existing HTML preview at `/runs/{id}/html`). Any terminal failure raises `TemplateFillError` → run marked failed (no retry, message shown). **Pure engine** `deerflow/runtime/scheduler/template_fill.py` (harness, no `app.*`): the template registry (`TEMPLATES`: id=`no1` name=`no.1` 台湾滨海防卫大屏, and id=`no2` name=`no.2` 台湾飞行部队平台 with **`convertible=False`** — a 「只展示」 template that renders its existing HTML **as-is**, no model/validation/fill, until conversion is wired in later; `_run_template_fill_attempt` short-circuits to `read_template_text` for it, and `list_templates`/`is_convertible` expose the flag so the form hides the JSON input), `list_templates`/`reference_data`/`schema_skeleton_json`/`validate_data`/`field_coverage_gaps`/`fill_template` — **the template's own embedded `DATA` is the schema source of truth** (parsed at load, 1-sample-per-array skeleton fed to the LLM, top-level structure as the validation basis); template HTML files live under `runtime/scheduler/templates/<id>/`, add a `TemplateSpec` to register a no.2. The agent is seeded like the AI-writing ones: `app/gateway/routers/_template_builder_seed.py` + `_template_builder_seed_assets/template-json-builder/{config.yaml,SOUL.md}`, called from the lifespan before `_sync_legacy_agents` (which upserts it into the `agents` table as built-in/published so it shows in `/api/agents` → the task form's 执行智能体 dropdown). Template list API: `GET /api/scheduled-tasks/html-templates` → `{templates:[{id,name,description,convertible}]}` (open). **Debug-render API** `POST /api/scheduled-tasks/template-fill/render` `{template_id, text?|data?}` → `{ok, display_only?, errors, gaps, html}` (open): no model call — `extract_embedded_json(text)` (or `data`) → `validate_data` → `fill_template`, returning the filled HTML **plus** the validation errors/field-gaps (fills even when invalid so you can see the effect). Powers the chat-page debug button below; non-convertible templates return their raw HTML. **Templates as `require_html` 参考收藏页面 (use the REAL template, all sub-pages)**: the 参考收藏页面 list in the task form also offers the templates (value `tpl:<id>`, e.g. `tpl:no1`). These templates are **JS-driven multi-page SPAs** (sub-pages/tabs are rendered by script from `DATA`), so a free-form model render would only ever produce the main page and **drop every sub-page**. Therefore, when `_run_html_page_attempt` sees a `tpl:<id>` reference (`template_fill.reference_template_id` parses it; distinguishes a template marker from a real favorite id), it short-circuits **before** the free-form pipeline to `_run_html_template_reference`, which renders the **actual template file** (all sub-pages intact): convertible templates gather real data (`_gather_html_data`) → convert to the template `DATA` (`_convert_to_template_data`, shared with template-fill mode) → `fill_template`; non-convertible templates (no.2) output the raw template as-is; a conversion failure falls back to the template's default data so the page + sub-pages always render. Archive/persist/result are shared via `_finalize_template_html`. (The earlier `style_reference` CSS-extraction approach was removed — it lost the sub-pages.) Frontend: when the agent dropdown = `template-json-builder` the form hides the HTML/Markdown switchLine truncated
|
||||
|
||||
### Per-user Prompts & UI Preferences
|
||||
|
||||
|
||||
+238
-123
@@ -1198,6 +1198,18 @@ class ScheduledTaskService:
|
||||
negative_prompt = str(cfg.get("negative_prompt") or "").strip()
|
||||
reference_favorite_id = cfg.get("reference_favorite_id") or None
|
||||
|
||||
# 选了内置模板作参考(``tpl:<id>``) → 用**真实模板**出页,保留全部子页面;
|
||||
# 绝不走下面的自由渲染(模型重画会丢掉 JS 驱动的多页/子页面)。
|
||||
tpl_ref_id = (
|
||||
template_fill.reference_template_id(str(reference_favorite_id))
|
||||
if reference_favorite_id
|
||||
else None
|
||||
)
|
||||
if tpl_ref_id is not None:
|
||||
return await self._run_html_template_reference(
|
||||
task, tpl_id=tpl_ref_id, runtime_thread_id=runtime_thread_id, scheduled_for=scheduled_for, run_id=run_id
|
||||
)
|
||||
|
||||
# 1) Gather source data (real, via the agent's search tools). When the
|
||||
# Markdown switch is also on, the gather pass additionally saves the .md
|
||||
# file, returned here so we can validate + surface it next to the page.
|
||||
@@ -1210,30 +1222,20 @@ class ScheduledTaskService:
|
||||
f"未满足输出要求:需生成 Markdown(.md) 文件,且文件正文不少于 {_MARKDOWN_MIN_CHARS} 字"
|
||||
)
|
||||
|
||||
# 2) Load reference style/layout. 参考来源有两种:
|
||||
# - ``tpl:<id>``:内置 HTML 模板(no.1/no.2),抽其 CSS+结构骨架当风格参考;
|
||||
# - 普通 id:用户在执行记录里收藏的页面(html store)。
|
||||
# 2) Load reference style/layout from a favorite if one was selected.
|
||||
# (模板参考 ``tpl:<id>`` 已在上面分支提前 return,这里只剩用户收藏的页面。)
|
||||
reference_style = None
|
||||
reference_layout = None
|
||||
if reference_favorite_id:
|
||||
tpl_id = template_fill.reference_template_id(str(reference_favorite_id))
|
||||
if tpl_id is not None:
|
||||
store = self._html_store
|
||||
if store is not None:
|
||||
try:
|
||||
ref = template_fill.style_reference(tpl_id)
|
||||
reference_style = ref.get("style") or None
|
||||
reference_layout = ref.get("layout") or None
|
||||
favorite = await store.get_favorite(str(reference_favorite_id))
|
||||
except Exception:
|
||||
logger.warning("加载模板样式参考失败:%s", reference_favorite_id, exc_info=True)
|
||||
else:
|
||||
store = self._html_store
|
||||
if store is not None:
|
||||
try:
|
||||
favorite = await store.get_favorite(str(reference_favorite_id))
|
||||
except Exception:
|
||||
favorite = None
|
||||
if favorite is not None:
|
||||
reference_style = favorite.get("reuse_prompt") or favorite.get("style_summary")
|
||||
reference_layout = favorite.get("layout_summary")
|
||||
favorite = None
|
||||
if favorite is not None:
|
||||
reference_style = favorite.get("reuse_prompt") or favorite.get("style_summary")
|
||||
reference_layout = favorite.get("layout_summary")
|
||||
|
||||
# 3) Render with the chosen model, injecting any enabled HTML skills.
|
||||
skill_instructions = self._collect_html_skill_instructions()
|
||||
@@ -1443,6 +1445,186 @@ class ScheduledTaskService:
|
||||
return "\n".join(p for p in parts if p)
|
||||
return str(content)
|
||||
|
||||
async def _convert_to_template_data(
|
||||
self,
|
||||
template_id: str,
|
||||
source: str,
|
||||
*,
|
||||
model_name: str | None,
|
||||
soul: str | None,
|
||||
run_id: str,
|
||||
) -> dict[str, Any]:
|
||||
"""把源数据(JSON/文本/含JSON文本)转换成模板的 DATA 结构,校验+回喂重试。
|
||||
|
||||
成功返回校验通过的对象;全部尝试失败抛 :class:`TemplateFillError`。
|
||||
"""
|
||||
skeleton = template_fill.schema_skeleton_json(template_id)
|
||||
# 源数据可能是纯 JSON / 纯文本 / 含 JSON 的文本:先尽力抽出内嵌 JSON 单独标注。
|
||||
embedded_json = template_fill.extract_embedded_json(source)
|
||||
base_prompt = self._build_conversion_prompt(skeleton, source, embedded_json)
|
||||
last_errors: list[str] = []
|
||||
prompt = base_prompt
|
||||
for attempt in range(_MAX_TEMPLATE_FILL_RETRIES + 1):
|
||||
raw = await self._invoke_conversion_model(model_name, soul, prompt)
|
||||
candidate = html_page_utils.extract_json_object(raw)
|
||||
if candidate is None:
|
||||
last_errors = ["模型未返回可解析的 JSON 对象"]
|
||||
else:
|
||||
errors = template_fill.validate_data(template_id, candidate)
|
||||
if not errors:
|
||||
return candidate
|
||||
last_errors = errors
|
||||
gaps = template_fill.field_coverage_gaps(template_id, candidate)
|
||||
prompt = self._build_retry_prompt(base_prompt, errors, gaps)
|
||||
logger.info(
|
||||
"模板填充[run=%s]第 %d 次校验未通过:%s",
|
||||
run_id,
|
||||
attempt + 1,
|
||||
";".join(last_errors[:5]),
|
||||
)
|
||||
raise TemplateFillError("转换结果未通过结构校验:" + ";".join(last_errors[:8]))
|
||||
|
||||
async def _finalize_template_html(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
*,
|
||||
spec: Any,
|
||||
template_id: str,
|
||||
html: str,
|
||||
run_id: str,
|
||||
scheduled_for: datetime,
|
||||
content: str,
|
||||
source_chars: int,
|
||||
used_default: bool,
|
||||
) -> dict[str, Any]:
|
||||
"""归档 index.html + 持久化 scheduled_html_pages + 构造 format=html 结果。"""
|
||||
task_tz = ZoneInfo(task.get("timezone") or "UTC")
|
||||
local_time = scheduled_for.astimezone(task_tz)
|
||||
archive_path = self._archive_html_index(run_id, html)
|
||||
title = str(task.get("name") or spec.name).strip()
|
||||
files: list[dict[str, Any]] = []
|
||||
if archive_path is not None:
|
||||
files.append(
|
||||
{
|
||||
"name": "index.html",
|
||||
"path": str(archive_path),
|
||||
"virtual_path": "/mnt/user-data/outputs/index.html",
|
||||
"mime_type": "text/html",
|
||||
"size": archive_path.stat().st_size if archive_path.exists() else None,
|
||||
"kind": "html",
|
||||
}
|
||||
)
|
||||
data_snapshot = {"template_id": template_id, "source_chars": source_chars, "used_default": used_default}
|
||||
prompt_snapshot = {"template_id": template_id, "used_default": used_default}
|
||||
store = self._html_store
|
||||
if store is not None:
|
||||
try:
|
||||
await store.create_page(
|
||||
{
|
||||
"id": str(uuid.uuid4()),
|
||||
"task_id": task["task_id"],
|
||||
"run_id": run_id,
|
||||
"user_id": task["user_id"],
|
||||
"title": title,
|
||||
"html_content": html,
|
||||
"html_file_path": str(archive_path) if archive_path else None,
|
||||
"data_snapshot": data_snapshot,
|
||||
"prompt_snapshot": prompt_snapshot,
|
||||
"reference_favorite_id": None,
|
||||
"style_summary": None,
|
||||
"layout_summary": None,
|
||||
}
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist scheduled_html_pages row for template run %s", run_id, exc_info=True)
|
||||
inline_html = html if len(html) <= _MAX_INLINE_RESULT_HTML else None
|
||||
return {
|
||||
"title": f"{title} - {local_time.strftime('%Y-%m-%d %H:%M')}",
|
||||
"content": content,
|
||||
"format": "html",
|
||||
"html": inline_html,
|
||||
"data_snapshot": data_snapshot,
|
||||
"prompt_snapshot": prompt_snapshot,
|
||||
"files": files,
|
||||
"created_at": datetime.now(UTC).isoformat(),
|
||||
}
|
||||
|
||||
async def _run_html_template_reference(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
*,
|
||||
tpl_id: str,
|
||||
runtime_thread_id: str,
|
||||
scheduled_for: datetime,
|
||||
run_id: str,
|
||||
) -> dict[str, Any]:
|
||||
"""require_html 模式选了内置模板(``tpl:<id>``):用**真实模板**出页,保留全部子页面。
|
||||
|
||||
模板是 JS 驱动的多页应用,子页面靠脚本从 DATA 渲染——绝不能让模型重画(会丢子
|
||||
页面)。故:可转换模板=先用智能体检索真实数据→转换成模板 DATA→填进真模板;只展示
|
||||
模板=原样输出。转换失败兜底为模板默认数据,至少保证页面与子页面可用。
|
||||
"""
|
||||
spec = template_fill.get_template(tpl_id)
|
||||
if spec is None:
|
||||
raise HtmlGenerationError(f"未知的 HTML 模板:{tpl_id}")
|
||||
cfg = self._html_config(task)
|
||||
model_name = str(cfg.get("model_name") or "").strip() or None
|
||||
|
||||
# 只展示模板:原样输出(含全部子页面)。
|
||||
if not spec.convertible:
|
||||
html = template_fill.read_template_text(tpl_id)
|
||||
return await self._finalize_template_html(
|
||||
task,
|
||||
spec=spec,
|
||||
template_id=tpl_id,
|
||||
html=html,
|
||||
run_id=run_id,
|
||||
scheduled_for=scheduled_for,
|
||||
content=f"已套用模板「{spec.name}」(含全部子页面)",
|
||||
source_chars=0,
|
||||
used_default=True,
|
||||
)
|
||||
|
||||
# 可转换模板:检索真实数据(复用 gather) → 转换成模板 DATA → 填进真模板。
|
||||
data_text, _files = await self._gather_html_data(
|
||||
task, runtime_thread_id=runtime_thread_id, scheduled_for=scheduled_for, run_id=run_id
|
||||
)
|
||||
source = (data_text or "").strip()
|
||||
used_default = False
|
||||
if not source:
|
||||
data_obj = template_fill.reference_data(tpl_id)
|
||||
content = f"未获取到数据,已用模板「{spec.name}」默认数据生成(含全部子页面)"
|
||||
used_default = True
|
||||
else:
|
||||
try:
|
||||
data_obj = await self._convert_to_template_data(
|
||||
tpl_id, source, model_name=model_name, soul=None, run_id=run_id
|
||||
)
|
||||
content = f"已检索数据并套用模板「{spec.name}」(含全部子页面)"
|
||||
except TemplateFillError:
|
||||
logger.warning("模板参考转换失败,回退默认数据 run=%s tpl=%s", run_id, tpl_id, exc_info=True)
|
||||
data_obj = template_fill.reference_data(tpl_id)
|
||||
content = f"数据转换未通过,已回退模板「{spec.name}」默认数据(含全部子页面)"
|
||||
used_default = True
|
||||
|
||||
task_tz = ZoneInfo(task.get("timezone") or "UTC")
|
||||
local_time = scheduled_for.astimezone(task_tz)
|
||||
try:
|
||||
html = template_fill.fill_template(tpl_id, data_obj, update_date=local_time.date())
|
||||
except template_fill.TemplateError as exc:
|
||||
raise HtmlGenerationError(f"模板填充失败:{exc}") from exc
|
||||
return await self._finalize_template_html(
|
||||
task,
|
||||
spec=spec,
|
||||
template_id=tpl_id,
|
||||
html=html,
|
||||
run_id=run_id,
|
||||
scheduled_for=scheduled_for,
|
||||
content=content,
|
||||
source_chars=len(source),
|
||||
used_default=used_default,
|
||||
)
|
||||
|
||||
async def _run_template_fill_attempt(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
@@ -1465,23 +1647,33 @@ class ScheduledTaskService:
|
||||
raise TemplateFillError(f"未知的 HTML 模板:{template_id!r}")
|
||||
|
||||
source = str(task.get("prompt") or "").strip()
|
||||
used_default = False
|
||||
data_obj: dict[str, Any] | None = None
|
||||
|
||||
# 「只展示」模板(convertible=False):暂不转换,原样输出模板已有的 HTML。
|
||||
# 「只展示」模板:原样输出已有 HTML(含全部子页面)。
|
||||
if not spec.convertible:
|
||||
html = template_fill.read_template_text(template_id)
|
||||
used_default = True
|
||||
elif not source:
|
||||
# 未提供源数据 → 直接展示模板自带的默认数据(不调模型、不校验)。
|
||||
return await self._finalize_template_html(
|
||||
task,
|
||||
spec=spec,
|
||||
template_id=template_id,
|
||||
html=html,
|
||||
run_id=run_id,
|
||||
scheduled_for=scheduled_for,
|
||||
content=f"已展示模板「{spec.name}」的现有数据",
|
||||
source_chars=len(source),
|
||||
used_default=True,
|
||||
)
|
||||
|
||||
if not source:
|
||||
# 未提供源数据 → 用模板自带默认数据(不调模型、不校验)。
|
||||
data_obj = template_fill.reference_data(template_id)
|
||||
content = f"未提供源数据,已用模板「{spec.name}」的默认数据生成页面"
|
||||
used_default = True
|
||||
else:
|
||||
ctx = task.get("execution_context") or {}
|
||||
tcfg = ctx.get("template_config") if isinstance(ctx, dict) else {}
|
||||
tcfg = tcfg if isinstance(tcfg, dict) else {}
|
||||
|
||||
# 解析选中的内置 agent 的 SOUL + 模型(template_config.model_name 优先,其次 agent.model)。
|
||||
# 选中的内置 agent 的 SOUL + 模型(template_config.model_name 优先,其次 agent.model)。
|
||||
agent_ref = task.get("execution_agent_name")
|
||||
agent_id = await self._resolve_agent_id(agent_ref, task["user_id"]) if agent_ref else None
|
||||
soul = load_agent_soul(agent_id) if agent_id else None
|
||||
@@ -1493,106 +1685,29 @@ class ScheduledTaskService:
|
||||
except Exception:
|
||||
model_name = None
|
||||
|
||||
skeleton = template_fill.schema_skeleton_json(template_id)
|
||||
# 源数据可能是纯 JSON / 纯文本 / 含 JSON 的文本:先尽力抽出内嵌 JSON
|
||||
# 单独标注,原文一并保留(纯文本则无内嵌 JSON,让模型按语义理解)。
|
||||
embedded_json = template_fill.extract_embedded_json(source)
|
||||
base_prompt = self._build_conversion_prompt(skeleton, source, embedded_json)
|
||||
data_obj = await self._convert_to_template_data(
|
||||
template_id, source, model_name=model_name, soul=soul, run_id=run_id
|
||||
)
|
||||
content = f"已根据源数据填充模板「{spec.name}」生成页面"
|
||||
used_default = False
|
||||
|
||||
# 转换 → 校验 → 失败回喂重试。
|
||||
last_errors: list[str] = []
|
||||
prompt = base_prompt
|
||||
for attempt in range(_MAX_TEMPLATE_FILL_RETRIES + 1):
|
||||
raw = await self._invoke_conversion_model(model_name, soul, prompt)
|
||||
candidate = html_page_utils.extract_json_object(raw)
|
||||
if candidate is None:
|
||||
last_errors = ["模型未返回可解析的 JSON 对象"]
|
||||
else:
|
||||
errors = template_fill.validate_data(template_id, candidate)
|
||||
if not errors:
|
||||
data_obj = candidate
|
||||
break
|
||||
last_errors = errors
|
||||
gaps = template_fill.field_coverage_gaps(template_id, candidate)
|
||||
prompt = self._build_retry_prompt(base_prompt, errors, gaps)
|
||||
logger.info(
|
||||
"模板填充[run=%s]第 %d 次校验未通过:%s",
|
||||
run_id,
|
||||
attempt + 1,
|
||||
";".join(last_errors[:5]),
|
||||
)
|
||||
if data_obj is None:
|
||||
raise TemplateFillError("转换结果未通过结构校验:" + ";".join(last_errors[:8]))
|
||||
|
||||
# 填充模板(顺便把 MANIFEST 更新时间刷成本次计划日期)。「只展示」模板上面已
|
||||
# 把原始 HTML 放进 html,这里跳过填充。
|
||||
task_tz = ZoneInfo(task.get("timezone") or "UTC")
|
||||
local_time = scheduled_for.astimezone(task_tz)
|
||||
if data_obj is not None:
|
||||
try:
|
||||
html = template_fill.fill_template(template_id, data_obj, update_date=local_time.date())
|
||||
except template_fill.TemplateError as exc:
|
||||
raise TemplateFillError(f"模板填充失败:{exc}") from exc
|
||||
|
||||
# 归档 index.html。
|
||||
archive_path = self._archive_html_index(run_id, html)
|
||||
title = str(task.get("name") or spec.name).strip()
|
||||
files: list[dict[str, Any]] = []
|
||||
if archive_path is not None:
|
||||
files.append(
|
||||
{
|
||||
"name": "index.html",
|
||||
"path": str(archive_path),
|
||||
"virtual_path": "/mnt/user-data/outputs/index.html",
|
||||
"mime_type": "text/html",
|
||||
"size": archive_path.stat().st_size if archive_path.exists() else None,
|
||||
"kind": "html",
|
||||
}
|
||||
)
|
||||
|
||||
data_snapshot = {"template_id": template_id, "source_chars": len(source), "used_default": used_default}
|
||||
prompt_snapshot = {"template_id": template_id, "used_default": used_default}
|
||||
|
||||
# 持久化页面资产(best-effort,失败不影响 run 成功)。
|
||||
store = self._html_store
|
||||
if store is not None:
|
||||
try:
|
||||
await store.create_page(
|
||||
{
|
||||
"id": str(uuid.uuid4()),
|
||||
"task_id": task["task_id"],
|
||||
"run_id": run_id,
|
||||
"user_id": task["user_id"],
|
||||
"title": title,
|
||||
"html_content": html,
|
||||
"html_file_path": str(archive_path) if archive_path else None,
|
||||
"data_snapshot": data_snapshot,
|
||||
"prompt_snapshot": prompt_snapshot,
|
||||
"reference_favorite_id": None,
|
||||
"style_summary": None,
|
||||
"layout_summary": None,
|
||||
}
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("Failed to persist scheduled_html_pages row for template-fill run %s", run_id, exc_info=True)
|
||||
|
||||
inline_html = html if len(html) <= _MAX_INLINE_RESULT_HTML else None
|
||||
return {
|
||||
"title": f"{title} - {local_time.strftime('%Y-%m-%d %H:%M')}",
|
||||
"content": (
|
||||
f"已展示模板「{spec.name}」的现有数据"
|
||||
if not spec.convertible
|
||||
else f"未提供源数据,已用模板「{spec.name}」的默认数据生成页面"
|
||||
if used_default
|
||||
else f"已根据源数据填充模板「{spec.name}」生成页面"
|
||||
),
|
||||
"format": "html",
|
||||
"html": inline_html,
|
||||
"data_snapshot": data_snapshot,
|
||||
"prompt_snapshot": prompt_snapshot,
|
||||
"files": files,
|
||||
"created_at": datetime.now(UTC).isoformat(),
|
||||
}
|
||||
try:
|
||||
html = template_fill.fill_template(template_id, data_obj, update_date=local_time.date())
|
||||
except template_fill.TemplateError as exc:
|
||||
raise TemplateFillError(f"模板填充失败:{exc}") from exc
|
||||
return await self._finalize_template_html(
|
||||
task,
|
||||
spec=spec,
|
||||
template_id=template_id,
|
||||
html=html,
|
||||
run_id=run_id,
|
||||
scheduled_for=scheduled_for,
|
||||
content=content,
|
||||
source_chars=len(source),
|
||||
used_default=used_default,
|
||||
)
|
||||
|
||||
async def _run_agent_attempt(
|
||||
self,
|
||||
|
||||
+3
-28
@@ -356,39 +356,14 @@ def _deep_merge(base: dict[str, Any], overrides: dict[str, Any]) -> None:
|
||||
base[k] = v
|
||||
|
||||
|
||||
# ── 作为 require_html 渲染的「风格参考」───────────────────────────────────────────
|
||||
# 「参考收藏页面」下拉里把模板选项的值存成 ``tpl:<id>``,与用户收藏(普通 id)区分。
|
||||
# ── require_html 模式选「内置模板」作参考 ─────────────────────────────────────────
|
||||
# 「参考收藏页面」下拉里模板选项的值存成 ``tpl:<id>``,与用户收藏(普通 id)区分。
|
||||
# 选了模板 → 调度器用**真实模板**出页(保留全部子页面),不抽风格摘要(那会丢子页面)。
|
||||
REFERENCE_PREFIX = "tpl:"
|
||||
|
||||
_STYLE_RE = re.compile(r"<style[^>]*>(.*?)</style>", re.DOTALL | re.IGNORECASE)
|
||||
_SCRIPT_BLOCK_RE = re.compile(r"<script[^>]*>.*?</script>", re.DOTALL | re.IGNORECASE)
|
||||
_BODY_RE = re.compile(r"<body[^>]*>(.*?)</body>", re.DOTALL | re.IGNORECASE)
|
||||
|
||||
|
||||
def reference_template_id(reference: str | None) -> str | None:
|
||||
"""``tpl:no1`` → ``no1``;非模板参考(普通收藏 id / 空)返回 None。"""
|
||||
if isinstance(reference, str) and reference.startswith(REFERENCE_PREFIX):
|
||||
return reference[len(REFERENCE_PREFIX) :] or None
|
||||
return None
|
||||
|
||||
|
||||
def style_reference(template_id: str, *, max_style: int = 24000, max_layout: int = 6000) -> dict[str, str]:
|
||||
"""抽取模板的紧凑「风格/版式参考」(``<style>`` CSS + 去脚本的静态结构骨架)。
|
||||
|
||||
模板内嵌的巨大 DATA/脚本会被剥掉,只留视觉相关的 CSS 与 body 结构,供
|
||||
require_html 的渲染模型**模仿其风格**,而不会把整份(几百 KB~MB 的)模板塞进 prompt。
|
||||
"""
|
||||
text = read_template_text(template_id)
|
||||
styles = _STYLE_RE.findall(text)
|
||||
css = "\n".join(s.strip() for s in styles).strip()
|
||||
if len(css) > max_style:
|
||||
css = css[:max_style] + "\n/* …(CSS 截断) */"
|
||||
layout = ""
|
||||
body = _BODY_RE.search(text)
|
||||
if body is not None:
|
||||
skeleton = _SCRIPT_BLOCK_RE.sub("", body.group(1))
|
||||
skeleton = re.sub(r"\n[ \t]*\n+", "\n", skeleton).strip()
|
||||
if len(skeleton) > max_layout:
|
||||
skeleton = skeleton[:max_layout] + "\n<!-- …(结构截断) -->"
|
||||
layout = skeleton
|
||||
return {"style": css, "layout": layout}
|
||||
Reference in new issue
Block a user