sync update from LAN repo (source master @ ca48c4ec)

This commit is contained in:
gozeros committed 2026-11-05 10:53:54 +08:00
1 parent 8bc8b0e082
commit ea9dcff9c0
873 files changed
+107866 -14011

No files matched your search

+311 -8
View File
@@ -1,5 +1,232 @@
# CLAUDE.md
## Cold agent construction (2026-10-30)
`runtime/runs/worker.py` dispatches opted-in synchronous factories through
`build_agent_with_timing`, backed by `utils/background_init.py`'s
`run_background_init` and one shared `FileIOPool(1, 1, "agent-init")`.
`make_lead_agent._deerflow_background_build = True` opts the built-in Lead Agent
into background construction; unmarked custom factories retain their caller-loop
contract. Serial admission prevents concurrent SDK cold imports and keeps waiting
builds out of authentication's executor. The synchronous timing helper remains
available. Model execution, checkpoint/store attachment and streaming stay in the
original run task; no graph is shared or cached across runs/users.
Every async Lead Agent entry point uses this builder, including non-streaming
and streaming open-chat endpoints and Workflow agent nodes. Do not call the
synchronous factory directly on their event loop: it can block SDK import locks
or deadlock while another background build awaits MCP discovery on that loop.
Async tool consumers (AI writing skill agents and `task_tool`) use
`get_available_tools_async`; deep-research capabilities and job execution also
construct their material providers through `run_background_init`. Keep the
common helper in `utils` with a lazy MCP import to avoid runtime/agent/tools
import cycles. Synchronous client/provider APIs remain available.
Copy the caller's ContextVars into construction and restore the successfully built
context before streaming, including a deferred-tool registry reset to `None`.
The `mcp_initialization_loop` context manager forwards lazy MCP discovery from the
worker to the owner event loop; exit it before copying back context. MCP cache
invalidation and legacy synchronous callers retain their existing semantics.
Cancellation drains the actual worker before releasing capacity, never starts the
graph afterwards, and preserves terminal status/SSE cleanup. Check `abort_event`
after construction as well as honoring task cancellation.
Regression: `packages/harness/tests/test_agent_build_responsiveness.py`,
`test_agent_build_timing.py`, `test_mcp_background_build.py` and the run-finalization
tests; `tests/test_async_agent_entrypoints.py` and
`tests/test_async_tool_initialization.py` exercise overlapping HTTP/workflow/tool
initializers. The HTTP/workflow cases run in subprocesses so an event-loop
deadlock fails with a bounded timeout. Cold profiling found ~4.7 s in model SDK imports/client setup, compared with
~14 ms in skill loading; moving only disk reads would not address this stall.
Local live cold acceptance (2026-10-30, official MiniMax-M2.7, tools enabled):
the first build took 4.28 s while a second request received stream metadata in
402 ms. During the conservative interior of the build window, 43 health/auth
probes completed successfully; across all three conversations, health/auth
P95 latency was 28/73 ms, with no event-loop-stall warning. Every answer ended
normally and all synthetic threads were removed. Initialization/model latency
still contributes to first-answer time; this change prevents blocking the server
loop, rather than promising immediate model output.
A separate live read-only tool check uploaded a random marker, allowed only
`read_file`, and confirmed the tool result and final answer reproduced that marker
without placing it in the prompt; the stream ended normally and the thread was removed.
Worker compatibility: no HTTP/SSE, task payload, auth or database contract changes.
Running workers without reload retain old construction behavior; the new factory
marker is harmless to old executors. `worker_pool/revision.py` hashes only the
adapter directory and does not identify this harness patch. This change does not
alter the controller's existing snapshot/ImageID-based drain-and-replace policy
or `on_disconnect=cancel` behavior: restarting a production controller can still
rotate idle workers or disconnect proxied streams. Do not infer uninterrupted
worker operation from these process-local initialization changes. The live local
acceptance instance has no enabled worker controller and does not touch remote
workers. Existing worker routing/admission/storage/discovery/diagnostics/launcher
compatibility regressions: 68 passed.
Final combined regression (2026-10-30): all 58 newly added responsiveness,
cancellation, MCP-entrypoint and provider-lifecycle cases plus the 68 existing
worker compatibility cases passed together (126 passed, no skips).
The live recheck uploaded and converted 20 DOCX files while another conversation
produced 43 visible text increments; all 172 concurrent authenticated probes
succeeded (P95 105 ms, max 436 ms). A fresh-process cold build took 10.92 s while
617 heartbeat samples continued (P95 scheduling delay 25 ms, max 146 ms).
The existing backend and child workers were not restarted for this recheck;
later entrypoint changes were verified through isolated HTTP/workflow tests.
The full versioned backend/harness run completed with 3140 passed, 39 failed,
4 skipped and 1 deselected. Against baseline `47cfed02`, 30 failures also failed
on HEAD and the other 9 passed in isolated reruns on both versions; no current-only
stable failure remained. The deselected legacy
`test_cancelled_history_closes_iterator` hangs on both versions: its fake saver
implements `alist`, while `limit=1` takes `aget_tuple`, so its unbounded entry
event wait never completes. This is not a claim that the repository's entire
test suite is green. Changed Python files introduced no Ruff diagnostics.
## Upload responsiveness (2026-10-30)
`app/gateway/routers/uploads.py` runs each staged upload transaction in a dedicated
`FileIOPool(2, 2, "uploads")`, including 256 KiB spool copies, signature validation,
optional synchronous conversion, sandbox acquisition/sync, commit, rollback and
cleanup. Resolve the thread owner asynchronously before dispatch and propagate
contextvars into workers. Waiting batches consume no worker from authentication,
artifacts or event streaming. Per-event-loop directory locks serialize same-thread
batches and listing/deletion through rollback/cleanup; keep them held until a
cancelled worker stops. Listing/deletion use the artifact file pool off the loop.
Cancellation signals a cooperative event, checks between file chunks and around
commit/sync, and waits for real completion before multipart handles may close.
The response contract and upload-policy enforcement remain unchanged.
`sandbox/sandbox_provider.py` serializes initial singleton construction with
reset/replacement/shutdown. Concurrent cold uploads must not create competing
providers with independent sandbox reconciliation and idle-cleanup threads.
Regression: `packages/harness/tests/test_sandbox_provider_concurrency.py`.
`file_conversion.py` also isolates its asynchronous conversion/extraction callers
in a separate two-thread pool. PDF/Office parsing runs in a lazily initialized,
two-process pool using `spawn` on Windows and Linux, so Python parser CPU work
does not contend for the gateway GIL. Pass only file paths and converter choices;
bound returned text inside the child and write sidecars in the owning parent
worker. The global parser semaphore bounds sync/async callers together. A broken
process pool fails the current conversion and is rebuilt on the next call.
`FileIOPool.run_until_complete` provides non-abandoning cancellation for workers
that own temporary input files; cancellation cannot close/delete an input still
being parsed by a child process.
Regression: `tests/test_upload_responsiveness.py`,
`tests/test_conversion_responsiveness.py`, `tests/test_uploads_policy_enforcement.py`.
Local acceptance (Windows, Python 3.12, synthetic files via ASGI; no delay injected):
20 x 900 KiB TXT uploads reduced first concurrent probe delay from 664/682 ms to
3/4 ms. With real MarkItDown conversion, 20 x 800-paragraph DOCX uploads reduced
maximum probe scheduling lag from 286/285 ms to 41/14 ms; all 20 sidecars matched
the source text. The first spawned reader took 13.7 s to initialize/convert in
this environment, while concurrent probe lag stayed below 16 ms. These are local
responsiveness measurements, not production throughput guarantees. Upload,
policy, conversion, large-file and runtime-contention regressions: 87 passed.
Full-suite collection is separately blocked by the pre-existing missing
`deerflow.models.global_prompt` import in `tests/test_global_prompt.py`.
## Thread metadata query performance and ORM deadlines (2026-10-22)
`ThreadMetaRepository._query_conditions` is shared by search/count: common
SQLite scalar predicates and PG/MySQL string predicates plus system-thread
exclusion run before SQL pagination/counting. Preserve AUTO owner resolution and
the nonempty taskId owner bypass while applying agent_id/actionId together.
`json_compat.JsonPredicate` binds values, preserves local missing/null and numeric
equality on SQLite, and uses binary MySQL id equality and Unicode-safe blank
seat detection. Nested/unusual-key filters and PG/MySQL numeric filters remain exact
residual filters; openGauss retains its original path. Never drop an unsupported
predicate or replace this with upstream's unadapted SQLite/PG-only compiler.
PG's omitted pool_recycle defaults to 300s (other backends remain 1800s);
database.command_timeout is finite positive or null, default 30s. The shared
pooled-engine kwargs are used on initial creation and auto-create retry. Only
asyncpg receives command_timeout; this does not configure psycopg checkpointers.
Regression entries: packages/harness/tests/test_thread_metadata_sql_filters.py,
test_database_pool_deadlines.py and the existing test_thread_operation_timing.py.
Master integration (2026-10-23) retains the maintained task/agent/action index
columns as additional SQL narrowing before exact JSON predicates. Keep dedicated
task deep-link card synchronization on every existing metadata mutation. Hydrated
history still uses slim_display to bound oversized tool payloads without editing
checkpoint contents; frontend retrieval-progress scopes remain isolated per turn.
Thread POST defaults to metadata-only creation; only an explicit
initialize_checkpoint=true writes an empty checkpoint. Never enqueue an empty
checkpoint in the background: it could become the head after the real first run.
Authorized metadata-only threads have empty state/history. Metadata GET defaults
to the SQL row (running maps to busy); include_state=true and checkpoint-only
legacy rows retain state reads. Owner decorators run before unscoped row reads.
History explicitly uses root checkpoint_ns and closes alist before serialization;
only the latest head carries full channel values.
The frontend uses BackgroundHistory as the SDK's external history adapter. Its
post-stream mutate immediately captures streamed state with no checkpoint id;
explicit end triggers background hydration. EOF/error still requires existing
run confirmation/rejoin. Every async write is fenced by thread/run/epoch/request;
new turns, navigation and unmount invalidate prior requests even if abort is
ignored. Reject history missing the newest streamed turn, bound each request to
15 seconds and retry completion sync once. Failure keeps the streamed snapshot.
Normal explicit-end callbacks use the completed streamed snapshot before the
next send can change page refs; background hydration never repeats them. Recovery
may have missed final SSE frames, so its callback waits for guarded hydration or
the retained snapshot fallback. New submits use checkpoint:null to follow the
latest server head.
Auth's SQLiteUserRepository is a historical name: its injected session factory
is the shared business ORM. With database.backend=mysql, login/auth/users and SQL
business repositories use MySQL, independently of checkpoint configuration.
LangGraph Store still follows checkpointer config and startup awaits runtime
setup/migrations; do not claim total service/startup isolation from PG or migrate
existing stores implicitly. See docs/THREAD_LATENCY_OPTIMIZATION_ZH.md for the audit.
Additional regression entries: test_thread_lazy_checkpoint.py,
tests/test_account_authentication.py and frontend pnpm test:thread-history.
## Multi-user contention controls (2026-10-22)
- `utils/file_io.py` separates bounded artifact IO (4 workers / 8 admitted) and
event disk IO (2 / 4) from the default executor used by authentication.
Contextvars propagate; cancellation cannot release admission until disk work
actually stops. Heavy artifact listing, ZIP previews, sniffing and writes use it.
- `runtime/runs/manager.py` sweeps completed live records after 300 seconds,
200 candidates per pass. It requires the actual task to be done and verifies
status/error/metadata/kwargs in the backing RunStore, updating a final snapshot
when necessary. Failed persistence retains the record for later retry. It never
deletes RunStore rows, checkpoints, events or stream buffers. `watch_completion`
starts grace after finalization, and lifespan closes the sweeper. The live
superseded fence survives a newer record's retirement; finalization never adds
a business-database history read to its completion fence.
- Run routes authorize the thread before unscoped persisted lookups, and verify
that the loaded run belongs to that exact thread. Preserve private-owner 404s,
taskId/system-thread sharing and existing action/agent partitions. Completed
rejoin without a live bridge buffer emits end immediately; Last-Event-ID replay
during the existing replay window remains untouched.
- `run_events.backend=memory` is a compatibility alias for
`SpoolingRunEventStore`, stored under `runtime_home()/run-events/events.sqlite3`.
Per-thread sequence assignment and each batch commit are transactional; paged
queries do not retain completed events in RAM. No automatic event/message
deletion or truncation is introduced. The pure `MemoryRunEventStore` remains
available for unit tests. Explicit db/JSONL formats are unchanged; JSONL disk
operations now use the event executor and striped per-thread locks. Keep the
spool and its WAL on the persistent local runtime volume. Do not reclassify it
as a PG checkpoint or silently change explicit db configuration.
- `runtime/fair_queue.py` provides bounded round-robin admission by user for the
subagent scheduler (3 total / 2 per user) and background roundtables (4 / 2).
Queued jobs consume no worker; subagent deadlines include queue time. Per-user
pending capacity also prevents one user from exhausting the entire backlog.
Actual coroutine completion, including cancellation before its first step,
acknowledges slot release. Parent context/identity propagation stays intact.
`RuntimeContentionConfig` defines optional budgets; limits apply per process
and require restart. The older roundtable per-user environment override remains.
- Both internal roundtable creation helpers only create metadata. Init does not
append a synthetic seat message via loopback; `_leader_run` injects authoritative
IDs on each real input, deriving them from the thread mapping for older clients
without seat_meta. Background first-leader injection remains. Never schedule a
late empty checkpoint write: it could overwrite real history.
Regression entries: `test_runtime_contention.py`, `test_completed_run_retention.py`,
`test_thread_lazy_checkpoint.py`, `test_run_completion_timeout.py`,
`tests/test_position_roundtable_sessions.py`, roundtable role/model/seat tests,
artifact/auth/model concurrency tests and frontend `pnpm test:thread-history`.
See `docs/MULTI_USER_CONTENTION_AUDIT_ZH.md` for measured scope and remaining
model/SQL/checkpointer contention. Process-local run control still requires the
current single-worker deployment; these queues do not enable multi-worker routing.
## Ordinary knowledge smart retrieval dual surface (2026-10-16)
`SmartReasoningRetrievalMiddleware` treats knowledge retrieval as an ordinary
@@ -120,7 +347,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
### Ordinary Q&A map visualization
The ordinary lead-agent prompt includes `_ORDINARY_QA_VISUALIZATION_SECTION` only for normal, non-bootstrap, non-writing, non-notebook, non-scheduled runs and defines the highest-priority output-carrier gate. Only an explicit current-request ask for HTML, a webpage/page, dashboard, large-screen view, or `.html` file permits reading `page-report-generation`, calling `page_report_template`, or writing HTML. “Draw a flowchart/chart/map/mind map” and “make a visualization” never imply HTML: ordinary flowcharts must be inline Mermaid, ordinary charts keep the established message-inline path, and requested maps emit strict JSON in a `map-json` fence. A mind map is emitted only for an explicit current-request ask for a mind map/knowledge map (or `analysis-mindmap`); ordinary analyses, summaries, outlines, flowcharts, and generic JSON must never auto-upgrade into one. Its strict `mindmap-json` contract is a bounded rooted tree (`id`, `parentId`, `label`, semantic `kind`, optional detail/source refs), targeting 12–24 nodes with a hard cap of 36, at most four layers including the center topic, at most six top-level dimensions and three children beneath every later node. Models must consolidate related facts into `detail`/`sourceRefs`, not make one node per report bullet, datum, paragraph, or reference. An unfinished dedicated `mindmap-json` fence (and compatible historical `mindmap` fence) remains an ordinary streaming code block; only the completed, valid object lazy-renders as a compact left-to-right message card. Clicking a parent node, or its +/− control at the outgoing connection point, expands/collapses its child tree and the canvas uses an explicit visible edge stroke. The full dialog keeps its close and view tools in the top-right and a layout-derived all-map preview in the bottom-right; the message preview also permits pan, wheel zoom, and zoom/reset controls. The message card's icon-only download control opens PNG/SVG/JSON choices. Generic JSON never invokes this renderer. Ambiguity defaults to inline output. Once the HTML gate passes, delivery is mandatory as a sandbox-generated `/mnt/user-data/outputs/*.html` artifact opened with `present_files`; a full inline `html` fence is forbidden as a substitute. The shared frontend `MarkdownContent` keeps its exact legacy branch when no map-shaped fence exists; dedicated `map-json` fences and complete generic `json` fences that contain map fields and pass the full map schema are validated and lazy-rendered from `components/workspace/visualizations/`. Map/loading/error cards use the full message-content width so their edges align with the step panel. Non-map and incomplete generic JSON retain the ordinary code-block path.
The ordinary lead-agent prompt includes `_ORDINARY_QA_VISUALIZATION_SECTION` only for normal, non-bootstrap, non-writing, non-notebook, non-scheduled runs and defines the highest-priority output-carrier gate. Only an explicit current-request ask for HTML, a webpage/page, dashboard, large-screen view, or `.html` file permits reading `page-report-generation`, calling `page_report_template`, or writing HTML. “Draw a flowchart/chart/map/mind map” and “make a visualization” never imply HTML: ordinary flowcharts must be inline Mermaid, ordinary charts keep the established message-inline path, and requested maps emit strict JSON in a `map-json` fence (markers use `name`+`coordinate` plus optional `color`; lines are included only when the user explicitly asks to connect places, must use `coordinates` with at least two `[lng,lat]` points, not `from`/`to`/`coords`, and default to `arrow`/`flow`). The frontend recovers those common aliases, assigns distinct marker colors, draws requested lines as looping SVG arrows, and drops unusable lines so markers still render. A mind map is emitted only for an explicit current-request ask for a mind map/knowledge map (or `analysis-mindmap`); ordinary analyses, summaries, outlines, flowcharts, and generic JSON must never auto-upgrade into one. Its strict `mindmap-json` contract is a bounded rooted tree (`id`, `parentId`, `label`, semantic `kind`, optional detail/source refs), targeting 12–24 nodes with a hard cap of 36, at most four layers including the center topic, at most six top-level dimensions and three children beneath every later node. Models must consolidate related facts into `detail`/`sourceRefs`, not make one node per report bullet, datum, paragraph, or reference. An unfinished dedicated `mindmap-json` fence (and compatible historical `mindmap` fence) remains an ordinary streaming code block; only the completed, valid object lazy-renders as a compact left-to-right message card. Clicking a parent node, or its +/− control at the outgoing connection point, expands/collapses its child tree and the canvas uses an explicit visible edge stroke. The full dialog keeps its close and view tools in the top-right and a layout-derived all-map preview in the bottom-right; the message preview also permits pan, wheel zoom, and zoom/reset controls. The message card's icon-only download control opens PNG/SVG/JSON choices. Generic JSON never invokes this renderer. Ambiguity defaults to inline output. Once the HTML gate passes, delivery is mandatory as a sandbox-generated `/mnt/user-data/outputs/*.html` artifact opened with `present_files`; a full inline `html` fence is forbidden as a substitute. The shared frontend `MarkdownContent` keeps its exact legacy branch when no map-shaped fence exists; dedicated `map-json` fences and complete generic `json` fences that contain map fields and pass the full map schema are validated and lazy-rendered from `components/workspace/visualizations/`. Map/loading/error cards use the full message-content width so their edges align with the step panel. Non-map and incomplete generic JSON retain the ordinary code-block path.
HTML artifacts are single-page by default. Omitted `preferred_page_count` is normalized to `1` during direct validation. A count greater than one is allowed only when the current user directly asks for multiple pages or gives a page count; content length and report chapter count never imply multi-page intent. Explicit multi-page intent without a count defaults to a three-page lightweight portal. It is one artifact HTML, not a folder of linked pages: `<main class="pages">` contains logical `.page` sections with unique `id`/`data-page`, and `href="#page-..."` navigation works in the sandboxed Blob preview through the injected anchor handler. Its validator requires the local Bootstrap/Alpine assets, `.pages`, multiple `data-page` sections and real anchor navigation: every actual `<a href="#page-...">` and every script-set `location.hash` must match a unique logical-page id, and each page must be reachable; comments or `#overview`/`#stations` data-page values cannot spoof this check. `MultiPageHtmlMiddleware` repeats that structural inspection at the `present_files` boundary, so a model that skips validate or writes custom tabs cannot expose a nonconforming portal. A user-uploaded background goes only on the requested logical `.page` in this mode (home page when unspecified), not on every page. When a template is explicitly requested, prepare strictly excludes profiles from the wrong page-count mode, persists `requested_page_count` and `page_mode`, and validate reloads them. Validation rejects multiple detected page views (`data-page`, tabpanels, `.report-page`, or `.view`) in single-page mode, or a single view when multi-page was explicitly requested.
@@ -130,7 +357,7 @@ For an explicit multi-page HTML request without an explicit template request, `M
### Generated HTML artifact visualization runtimes
The same normal lead-agent prompt includes `_HTML_ARTIFACT_VISUALIZATION_SECTION`, but its contents are executable only after the explicit HTML-intent gate above passes; visualization content alone never implies this intent. New generated HTML explicitly loads browser distributions from Gateway `/api/public/html-runtime/`: `bootstrap.min.css`, `alpine.min.js`, `bootstrap.bundle.min.js`, `g6.min.js`, `echarts.min.js`, and then `deerflow-runtime.js`; it can also load the packaged country/region map catalog: `china-provinces`, `us-states`, `japan-prefectures`, and `taiwan-counties`. Bootstrap is the responsive layout/modal layer and Alpine is the small state/list-rendering layer; jQuery, React, Vue, Babel and public CDNs are not part of this artifact contract. The files are package resources under `packages/harness/deerflow/html_runtime/assets/` and ship with the harness wheel, so HTML execution does not depend on frontend Vite/node_modules assets or a public CDN. `system_settings.json -> html_map.engine` (`GET|PUT /api/system-settings/html-map`, admin write) chooses `echarts` (default, local GeoJSON `data-df-geo`) or `leaflet` (地图网页, `data-df-map`). `config.yaml -> html_runtime.map_tile_url` is injected into `deerflow-runtime.js` as `mapTileUrl` and is used only by the Leaflet engine. Both initializers stay in the runtime so already generated pages keep rendering after the switch. The next ordinary Q&A prompt and `page_report_template` validation follow the selected engine. `config.yaml -> html_runtime.base_url` is the browser-reachable absolute runtime base written into generated source (localhost default; deployments must replace it with the real Gateway origin). Flowcharts use paired `data-df-flow` JSON blocks and AntV G6, not Mermaid; charts use `data-df-chart`; maps use ECharts Canvas-backed `data-df-geo` with the local GeoJSON catalog. `data-df-chart` must be strict JSON: no JavaScript callbacks or function-valued colors, with literal colors/data-item styles and string tooltip formatters instead. Dark HTML must set `data-df-theme="dark"` and explicitly give ECharts global/title/legend/axis/visible-series labels readable light text, including `nameTextStyle` for named axes. Map `description` values are plain text because the runtime escapes markup; click-to-filter or page-switch behavior listens to `deerflow:geo-map-click` and uses a valid `#page-...` anchor. `deerflow-runtime.js` waits until a hidden logical page has a real layout before creating ECharts and tracks later container changes with `ResizeObserver`, so map/chart canvases never depend on a browser-window resize for their first visible render. It initializes all three contracts, opens a detail popover for described regions/markers, and exposes catalog URLs through its public dynamic response. Map large screens use a centered map stage with narrow side rails and bottom summaries, rather than cards layered across a full-page map background. Frontend runtime replacement is bypassed whenever this explicit Gateway runtime is present. This remains separate from ordinary message rendering; ordinary `map-json` keeps its existing frontend tile renderer and `VITE_MAP_TILE_URL` configuration, and ordinary `flow-json`/`chart-json` must not be reintroduced.
The same normal lead-agent prompt includes `_HTML_ARTIFACT_VISUALIZATION_SECTION`, but its contents are executable only after the explicit HTML-intent gate above passes; visualization content alone never implies this intent. New generated HTML explicitly loads browser distributions from Gateway `/api/public/html-runtime/`: `bootstrap.min.css`, `alpine.min.js`, `bootstrap.bundle.min.js`, `g6.min.js`, `echarts.min.js`, and then `deerflow-runtime.js`; it can also load the packaged country/region map catalog: `china-provinces`, `us-states`, `japan-prefectures`, and `taiwan-counties`. Bootstrap is the responsive layout/modal layer and Alpine is the small state/list-rendering layer; jQuery, React, Vue, Babel and public CDNs are not part of this artifact contract. The files are package resources under `packages/harness/deerflow/html_runtime/assets/` and ship with the harness wheel, so HTML execution does not depend on frontend Vite/node_modules assets or a public CDN. `system_settings.json -> html_map.engine` (`GET|PUT /api/system-settings/html-map`, admin write) chooses `echarts` (default, local GeoJSON `data-df-geo`) or `leaflet` (地图网页, `data-df-map`). The map tile URL saved in the settings dialog (`system_settings.json -> html_runtime.map_tile_url`, `GET|PUT /api/system-settings/html-runtime`) is injected into `deerflow-runtime.js` as `mapTileUrl` and is used only by the Leaflet engine. Both initializers stay in the runtime so already generated pages keep rendering after the switch. The next ordinary Q&A prompt and `page_report_template` validation follow the selected engine. The runtime base URL saved in the same settings block is the browser-reachable absolute address written into generated source. An empty value falls back to an older `config.yaml` entry and then `http://localhost:8001/api/public/html-runtime`. Flowcharts use paired `data-df-flow` JSON blocks and AntV G6, not Mermaid; charts use `data-df-chart`; maps use ECharts Canvas-backed `data-df-geo` with the local GeoJSON catalog. `data-df-chart` must be strict JSON: no JavaScript callbacks or function-valued colors, with literal colors/data-item styles and string tooltip formatters instead. Dark HTML must set `data-df-theme="dark"` and explicitly give ECharts global/title/legend/axis/visible-series labels readable light text, including `nameTextStyle` for named axes. Map `description` values are plain text because the runtime escapes markup; click-to-filter or page-switch behavior listens to `deerflow:geo-map-click` and uses a valid `#page-...` anchor. `deerflow-runtime.js` waits until a hidden logical page has a real layout before creating ECharts and tracks later container changes with `ResizeObserver`, so map/chart canvases never depend on a browser-window resize for their first visible render. It initializes all three contracts, opens a detail popover for described regions/markers, and exposes catalog URLs through its public dynamic response. Map large screens use a centered map stage with narrow side rails and bottom summaries, rather than cards layered across a full-page map background. Frontend runtime replacement is bypassed whenever this explicit Gateway runtime is present. This remains separate from ordinary message rendering; ordinary `map-json` keeps its existing frontend tile renderer and `VITE_MAP_TILE_URL` configuration, and ordinary `flow-json`/`chart-json` must not be reintroduced.
User-uploaded raster images can be used as generated-HTML backgrounds through signed, unauthenticated `GET /api/public/html-images/{token}` URLs. `UploadsMiddleware` mints one URL per uploaded `.jpg/.jpeg/.png/.webp/.gif/.bmp` and injects it into `<uploaded_files>` as `Public URL`; the HTML contract requires that exact absolute URL in `background-image`, never a sandbox path, artifact API path, relative path, or base64 data URI. Single-page artifacts place it on `html/body`; the multi-page portal places it only on the requested logical page. Tokens are HMAC-signed, do not expire (the URL is written into durable HTML), and only grant read access to one file under the thread `uploads/` or `outputs/` directory. Engine: `deerflow.html_runtime.public_images`; router: `app/gateway/routers/html_images.py`; tests: `tests/test_html_images.py`.
@@ -303,6 +530,28 @@ commands.py` and `tests/test_position_roundtable_concurrency.py` cover the
state transitions; `tests/test_roundtable_postgres_concurrency.py` is the real
PostgreSQL acceptance suite (set `DEERFLOW_POSTGRES_CONCURRENCY_TEST_URL`).
Step-2 roundtable rooms share one deployment-wide cap,
``system_settings.json -> roundtable_room_limit`` (default enabled, 3 rooms).
``GET/PUT /api/system-settings/roundtable-room-limit`` is the admin switch.
Occupancy is summed across **every account**: durable foreground leases keyed
by a hidden per-live-execution ``roomSessionId``, in-flight LangGraph runs on
threads stamped with the corresponding ``roundtable_room_key``, and executing
background jobs (``queued`` / ``running`` / ``cancel_requested`` with a live
worker lease or a recent update). Different pages running the same saved draft
have different session ids and therefore take different rooms; all seats and
Step 3 calls within one live execution share its key. Run-manager inspection
is reconciliation-only: when a durable lease/job already owns the same
draft+user/task scope, stale or overlapping run keys in that scope are not
counted again. Merely opening the page,
viewing saved history, paused ``awaiting_input`` jobs, stale historical rows,
and Step-1 intent threads do not occupy a slot. Foreground-to-background
handoff replaces the foreground lease with the durable job occupancy. There is
no per-user pool. Admission is serialized by a database gate row, and the cap
is enforced at acquire, init, every model-producing run, and background-job
start. Full pools return HTTP 409; an enabled gate with missing storage or a
missing session id fails closed. The switch applies on the next admission
without a restart. Position and eight-department roundtables are not counted.
Roundtable backend jobs use a database unique active-dedupe key plus a leased
dispatcher, never process-local start deduplication. Cancellation is two
phase (`cancel_requested` then `cancelled`): the lease owner’s watcher normally
@@ -449,6 +698,8 @@ from deerflow.config import get_app_config
### Agent System
**Gallery chat link.** An agent's `config.yaml` may store `chat_link` (`url`, `open_mode` of `blank`/`self`, and light-app-shaped `route_params`). The gallery 「对话」 button opens that resolved URL; an empty url keeps the in-app chat. The list endpoint reads the mirrored `agent_extras.chat_link` column (nullable JSON, added by the startup column sweep) instead of parsing every config file. Create/update and directory backfill write the mirror; a skills-only `set_extras` leaves an existing link in place.
**Lead Agent** (`packages/harness/deerflow/agents/lead_agent/agent.py`):
- Entry point: `make_lead_agent(config: RunnableConfig)` registered in `langgraph.json`
- Dynamic model selection via `create_chat_model()` with thinking/vision support
@@ -463,6 +714,8 @@ from deerflow.config import get_app_config
**Strategy-selection functional agents.** `app/gateway/routers/_strategy_selection_seed.py` seeds nine stable folder-backed agents from `_strategy_selection_seed_assets/` before `_sync_legacy_agents`: intent, governance/diplomacy/economy/security/public-opinion/law-policy angles, judge, and solution editor. Seeding is missing-directory-only and must never overwrite administrator changes. The global `strategy_agent_roles` registry classifies them independently of projects: intent/judge/solution are fixed, unique roles that may be configured but never added, disabled, reordered, or deleted; angle roles are a shared, ordered catalog that administrators may add, edit, disable (soft delete), restore, and reorder through `/api/strategy-agents`. Public `GET /api/strategy-agents` returns enabled identity/description metadata only; `include_disabled=true` or `include_config=true` is admin-only. Admin writes are `POST /api/strategy-agents/angles`, `PUT /api/strategy-agents/{id}`, and `DELETE /api/strategy-agents/{id}`; DELETE retains the folder-backed agent and role row to preserve historical references. Dynamic angles remain built-in folder-backed agents under the reserved `strategy-selection-angle-` id namespace, with `web` + `file:read`; a blank SOUL on create receives a safe role template. All strategy skill writes validate that names exist, are active, and are globally visible; generic agent CRUD cannot create or hard-delete reserved strategy IDs, and parallel execution rejects disabled roles. Angle configs reuse `policy-knowledge`; intent reuses `intent-clarification-policy`; the judge has no retrieval skill; the solution editor gets only `file:write`. Keep the SOUL contracts neutral and evidence-aware, with explicit de-escalation/civilian-protection framing and hard bans on operational combat guidance and targeted political manipulation. Tests: `tests/test_strategy_selection_seed.py`, `tests/test_strategy_agent_management.py`.
Only `strategy-selection-intent` has a clarification reply surface in the strategy workbench. `parallel_agents` must exclude `ask_clarification` for every other `strategy-selection-*` role and inject a direct-delivery guard: angle, judge, and solution stages use conservative assumptions, lower confidence/score where needed, list gaps under data limitations, and still finish in one pass. The judge brief repeats this rule at runtime because existing folder-backed agent SOUL files may be administrator-customized and are intentionally never overwritten. If a legacy backend nevertheless emits a judge clarification, the frontend treats it as a bounded repair attempt rather than asking the user to repeat the same manual run. Malformed or incomplete evaluation JSON receives the same bounded repair path; prior attempts are replaced in the live panel, never concatenated into duplicate final content.
`POST /api/parallel-agents/run/stream` must remain observable during silent skill calls. Each running seat emits an `agent_progress` heartbeat every five seconds with the latest translated tool/reasoning/body phase and monotonic elapsed seconds; an upstream iterator that produces no frame for 180 seconds is closed and that seat emits `agent_error`, while the other seats continue. Set the terminal flag before queuing `agent_done` / clarification so a late heartbeat cannot reopen a completed card. The strategy workbench renders the intent Q&A history, readable skill execution steps and live heartbeat on the affected agent card; judge and synthesis traces use the same step translation. Tests: `tests/test_roundtable_model_fallback.py` and frontend `src/strategy-selection/lib.test.ts`.
**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
@@ -593,7 +846,7 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
| **Answer feedback** (`/api/threads/{id}/messages/{mid}/feedback`) | Message-scoped `PUT`/`DELETE`, `GET /feedback-context`, and batched `POST /api/threads/{id}/feedback-contexts`; persists excellent/poor ratings, six stable poor-reason codes and optional detail idempotently per user/thread/message/agent. Legacy run-scoped feedback routes remain compatible. Admin-only `GET /api/agents/{agent_id}/feedback-score` uses the project-wide `system_role == "admin"` rule (and preserves auth-disabled mode) and returns a Bayesian user-feedback answer-quality score, raw task-success rate, weighted response-quality dimensions and reason counts; it is explicitly not a complete agent-capability evaluation. |
| **Runs** (`/api/runs`) | `POST /stream` - stateless run + SSE; `POST /wait` - stateless run + block; `GET /{rid}/messages` - paginated messages by run_id `{data, has_more}` (cursor: `after_seq`/`before_seq`); `GET /{rid}/feedback` - list feedback by run_id |
| **Open Chat** (`/api/open/chat`) | **Unauthenticated** open Q&A entry for external systems (the `/api/open/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES` + CSRF-exempt in `csrf_middleware.should_check_csrf`). `POST /chat` `{message \| messages[], model_name?, agent_name?, agent_id?, thinking_enabled=false, strip_think=true, thread_id?, recursion_limit=100}` → resolves the agent by **Chinese display name** (`agent_name`, matched against each `.deer-flow/agents/*/config.yaml` `name` via `_resolve_agent_id`; explicit `agent_id` wins if given), builds the lead agent via `make_lead_agent` (so the chosen **agent's** SOUL/tools/skills all apply), `ainvoke`s it once (stateless — no checkpointer, no thread persistence; runs under the `"default"` user bucket), and returns **only the final** `{answer, model_name, agent_name, agent_id, thinking_enabled, thread_id}` (no intermediate tool-call/process). **`POST /chat/stream`** is the **SSE streaming variant** (same request body) for **long tasks** — the blocking `/chat` produces no downstream bytes for the whole run, so a front nginx `proxy_read_timeout` (default 60s) cuts it off with a **504**; the stream keeps the connection warm (a `: keepalive` SSE comment every `_HEARTBEAT_SECONDS=15`s via a background producer-task + queue, plus `event: delta` AI-text increments and `event: progress`), and ends with **`event: result`** carrying the same final `{answer, …}` JSON (or `event: error`). Clients only need to read `event: result`; intermediate events are ignorable. Both endpoints share `_build_run_context` (config/agent/thread). `GET /agents` lists `{name, id}` so callers know what `agent_name` to send. **Thinking off is reliable**: when `thinking_enabled=false` the route also sets `thinking_force_disabled=true` so the lead agent calls `create_chat_model(force_disable_thinking=True)`, which injects **three** OpenAI-compatible "thinking off" shapes into `extra_body` — top-level `enable_thinking=false` (**SiliconFlow / DashScope hybrid models** like `deepseek-ai/DeepSeek-V4-*` on `api.siliconflow.cn` — the knob the nested `chat_template_kwargs` shape does NOT reach), `chat_template_kwargs.enable_thinking=false` (vLLM/Qwen native), and `thinking.type="disabled"` (Zhipu GLM) — even for models that only declare `supports_thinking: true`. For the **web-UI path** (which sends `thinking_enabled=false` but NOT `force_disable`), SiliconFlow models need `when_thinking_disabled: {extra_body: {enable_thinking: false}}` in their `config.yaml` entry (added to `deepseek-ai/DeepSeek-V4-Flash`) for thinking-off to take effect. Sets `is_scheduled_run=true` internally to skip the auto-title LLM call. **Generated-file return (`include_files=true`, default on)**: an agent that produces its report as a file (writes to `/mnt/user-data/outputs/` + `present_files`) returns only a short summary in `answer`; to recover the real deliverable both endpoints now also return `files: [{name, virtual_path, encoding(text|base64), content}]` — `_collect_output_files` scans the run's `state['thread_data']['outputs_path']` (the open thread is stateless so artifacts can't be fetched by thread, but the endpoint runs server-side and the sandbox already wrote the files to disk), text decoded as utf-8 else base64, capped by `_MAX_OUTPUT_FILES=20` / `_MAX_OUTPUT_FILE_BYTES=5MB`. `/chat` puts `files` on the response; `/chat/stream` puts it in the `event: result` payload. A run with no answer **and** no files now 502s ("…文本答案或文件"). Router `app/gateway/routers/open_chat.py`; standalone caller `test_open_chat_api.py` (repo root, streaming by default); tests `tests/test_open_chat.py`. |
| **Open 3qfx Ask** (`/api/open/3qfx/ask`) | **Unauthenticated** open entry that, given just a **任务id**, directly kicks off the 3qfx(三情分析)「第一次问答」as a **real persisted conversation** (sibling of `/api/open/chat`; same `/api/open/` auth-whitelist + CSRF-exempt). `POST /ask` `{task_id, message?, model_name?, thinking_enabled=false}` → (1) resolves the `3Q` agent from `app.state.business_mapping_store.get_mapping("3Q")` (the **same** agent the frontend deep-link `goPath=3qfx` uses; 503 if unconfigured); (2) **fetches the task detail server-side** (consumer `cop-task-detail`) and builds the opening text **identical to the frontend** `buildOpeningText` 3qfx branch — `任务名称:…\n任务内容:…\n任务id(task_id)为{task_id}` (an explicit `message` overrides and **skips** the fetch); (3) creates a fresh thread tagged `metadata.{taskId, agent_id}` (the exact filter `useTaskThreads` uses, and `metadata.taskId` makes it task-deeplink-shared / creator-path-resolved) and **starts a background persisted run** via `start_run(..., enforce_access=False)` (writes the checkpointer, runs under the `"default"` bucket); (4) **returns immediately** `{thread_id, run_id, agent_id, status, opening_text}` — the caller just needs "started", then watches the run in the 3qfx 任务工作区 conversation list. `start_run` gained a keyword-only `enforce_access` (default `True`); the open entry passes `False` so the admin-configured agent isn't rejected by the per-(missing)-login visibility check. **Consumer fetch config** lives in `config.yaml → task_deeplink`: `cop_task_detail_url` (full URL, appends `?taskId=`), `cop_task_detail_authorization` (default `Bearer admin`), `cop_task_detail_test` + `mock.cop_task_detail` (offline fake when the upstream is unreachable — on in this deployment). Router `app/gateway/routers/open_3qfx.py`; tests `tests/test_open_3qfx.py`. |
| **Open 3qfx Ask** (`/api/open/3qfx/ask`) | **Unauthenticated** open entry that, given just a **任务id**, directly kicks off the 3qfx(三情分析)「第一次问答」as a **real persisted conversation** (sibling of `/api/open/chat`; same `/api/open/` auth-whitelist + CSRF-exempt). `POST /ask` `{task_id, message?, model_name?, thinking_enabled=false}` → (1) resolves the `3Q` agent from `app.state.business_mapping_store.get_mapping("3Q")` (the **same** agent the frontend deep-link `goPath=3qfx` uses; 503 if unconfigured); (2) **fetches the task detail server-side** (consumer `cop-task-detail`) and builds the opening text **identical to the frontend** `buildOpeningText` 3qfx branch — `任务名称:…\n任务内容:…\n任务类型:…\n任务id(task_id)为{task_id}`(`taskType` 1/2/3 → 打击/塑势/防御,无法识别则省略) (an explicit `message` overrides and **skips** the fetch); (3) creates a fresh thread tagged `metadata.{taskId, agent_id}` (the exact filter `useTaskThreads` uses, and `metadata.taskId` makes it task-deeplink-shared / creator-path-resolved) and **starts a background persisted run** via `start_run(..., enforce_access=False)` (writes the checkpointer, runs under the `"default"` bucket); (4) **returns immediately** `{thread_id, run_id, agent_id, status, opening_text}` — the caller just needs "started", then watches the run in the 3qfx 任务工作区 conversation list. `start_run` gained a keyword-only `enforce_access` (default `True`); the open entry passes `False` so the admin-configured agent isn't rejected by the per-(missing)-login visibility check. **Consumer fetch config** lives in `config.yaml → task_deeplink`: `cop_task_detail_url` (full URL, appends `?taskId=`), `cop_task_detail_authorization` (default `Bearer admin`), `cop_task_detail_test` + `mock.cop_task_detail` (offline fake when the upstream is unreachable — on in this deployment). Router `app/gateway/routers/open_3qfx.py`; tests `tests/test_open_3qfx.py`. |
| **Thread Shares** (`/api/thread-shares`) | Per-conversation sharing with two modes (`thread_shares.mode`). **`import`**: `POST /` `{thread_id, mode}` - create (or reuse) a revocable share code; `GET /{code}/preview` + `POST /{code}/import` - copy that conversation into the caller's account (fresh `thread_id` via `app/gateway/thread_copy.py` + new `threads_meta` row + `user-data` copy; dedup on `metadata.shared_origin`). **`view`**: a public, read-only static page ? see Public Shares below. Both modes: `GET /` - list my codes; `DELETE /{code}` - revoke. |
| **Public Shares** (`/api/public/shares`) | **Unauthenticated** (the `/api/public/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES`). `GET /{code}` - read-only conversation snapshot (title + serialized messages + artifacts) for a `view`-mode share; messages are read straight from the latest checkpoint. `GET /{code}/files/{path}` - serves a file referenced by that conversation (images, Markdown, text, PDF, ?) so the shared page's links are openable; path resolution is confined to the owner's thread directory. Active content (`text/html`, `application/xhtml+xml`, `image/svg+xml`) is forced to download (`_public_share_disposition`); everything else is served inline. Missing/revoked/non-`view` codes all return 404. |
| **Roundtable Drafts** (`/api/roundtable-drafts`) | **Strictly per-user** persistence for the roundtable-planning page (replaces the old browser `localStorage`). `GET /` - list drafts (lightweight meta — incl. `chain_title`, the business-chain name used this session, cheaply regex-extracted from the in-memory `step2` blob in `_draft_to_meta`/`_extract_chain_title`; null = no chain / 自由模式; surfaced on the 第一步「分析记录」history card + sidebar list instead of the old「第 X 步」); `POST /` - create; `GET/PUT/DELETE /{id}` - full read / partial update / delete (all owner-scoped). A draft = one full session (Step 1 intent + Step 2 roundtable), step snapshots stored as JSON in `PortableLongText`. Recommendation history bound to a draft: `GET /{id}/recommendations` - list newest-first; `POST /{id}/recommendations` - append one run (objective/status/rationale/picks/candidates). taskId 深链会商记录 are **not** here — they live in the separate `/api/roundtable-task-drafts` store (see below) so personal history never mixes with task records. Backed by `deerflow.persistence.roundtable_drafts` (tables `roundtable_drafts`, `roundtable_recommend_history`); store wired in `deps.py` as `app.state.roundtable_draft_store`. Tests: `tests/test_roundtable_task_drafts.py`. |
@@ -660,6 +913,8 @@ The intranet client uses `trust_env=False` and defaults `intranet_verify_ssl=fal
The admin ????? is served from **precomputed per-day snapshots** in `admin_leaderboard_daily_stats` (keyed by `(stat_date, settings_hash)`), not by aggregating raw rows on every request. A background scheduler inside the Gateway (`_leaderboard_snapshot_scheduler_loop` in `app/gateway/app.py`, singleton file-locked) ticks every `ADMIN_LEADERBOARD_BACKGROUND_INTERVAL_SECONDS` (default 300s): it prewarms recent days, then builds any `queued`/`error`/stale-`running` day. `GET /leaderboard` merges the per-day snapshots for the requested range. **Today** is rebuilt whenever its snapshot is older than `ADMIN_LEADERBOARD_TODAY_TTL_SECONDS` (default 43200s = half a day, so the leaderboard refreshes ~twice daily); a past day, once `ready`, is treated as fresh forever.
**Recent-only admin audit policy.** The server enforces a rolling **30-calendar-day** window for administrator leaderboard, user analytics, user Q&A/thread audit, LLM/tool metric list/summary/export, and the user-list Q&A counter; URL parameters cannot widen it. The Q&A fallback that reads checkpoints is additionally capped, while indexed run summaries remain the primary audit source. Gateway startup also launches the singleton `admin_audit_retention` sweep: it deletes audit metric rows, terminal `runs` summaries (which contain compact question/answer audit fields and were also last updated before the cutoff), and leaderboard snapshots older than **90 days**. Cleanup is deliberately low-priority: after the default 5-minute startup delay, it runs every 15 minutes using 100-row committed transactions, round-robins each store (at most four batches per store), yields between commits, and stops starting new batches after an 8-second sweep budget; operators may tune these bounds with `ADMIN_AUDIT_RETENTION_*` environment variables. It deliberately does **not** delete conversation threads, checkpoints, or message history; those are user-content records governed by their own lifecycle.
Two on-demand/automatic refresh paths layer on top:
- **Manual "????"** ? `POST /leaderboard/refresh-today` force-rebuilds *today's* snapshot **synchronously** (bounded by the same semaphore/timeout the scheduler uses), then returns the merged range. Lets an admin pull the latest real-time records into the stats without waiting for the TTL. Frontend: a "????" button on `AdminUserLeaderboardPage` ? `useRefreshTodayLeaderboard` (`core/admin/hooks.ts`), which primes + invalidates the `["admin-activity","leaderboard"]` query.
- **Daily finalize of the previous day** ? once per day, on/after `ADMIN_LEADERBOARD_FINALIZE_HOUR` (local Beijing, default `1` = ?? 1 ?), the scheduler tick force-recomputes *yesterday* a single time so runs/tool-calls that only landed near midnight are fully captured. The decision is the pure `deerflow.persistence.admin_stats.finalize.should_finalize_previous_day` (status/generated_at/now/hour) ? the rebuild stamps `generated_at` to today, which self-debounces it for the rest of the day (no extra state file). Tests: `tests/test_leaderboard_finalize.py`.
@@ -768,7 +1023,7 @@ The fixed-deep-blue wx full-chat route (`#/page/workspace/wx/chats/new`) hides t
### Business-mapping authorization
`/api/business-mapping` remains globally readable. Seeded rows include the main codes (3Q / 6BF / 7BF / 8BF / AFX) plus independent slots `3q-0…3q-6`, `6-1…6-4`, and `7-1…7-4` (empty agent/chain until configured on the 配置管理 page). Updates to 3Q, 6BF, 7BF, and those independent slots require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.
`/api/business-mapping` remains globally readable. Seeded rows include the main codes (3Q / 6BF / 7BF / 8BF / AFX) plus independent slots `3q-0…3q-6`, `6-1…6-4`, and `7-1…7-4` (empty agent/chain until configured on the 配置管理 page). `3q-1…3q-6` seed as `is_custom=true` / `mapping_type=自定义类型` so `goPath=3q-1` resolves as a custom deep link; the configuration page hides those six rows and defaults the type filter to 自定义类型. Updates to 3Q, 6BF, 7BF, and those independent slots require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.
### Position-collaboration foundation
@@ -1357,10 +1612,11 @@ scheduler runs once per deployment. Per run:
会商第二步(leader/seat)+ 第三步(summary/report/dashboard + 结构化抽取)单轮执行时,**某模型出问题就自动切到下一个模型继续**,并告知用户「『X』模型异常 → 已切换『Y』」。前台实时(`multi_agent.py`)与后台挂起(`engine.py` + `InProcessRoundtableGateway`)两条路径都覆盖。
- **可检测的模型失败**:`LLMErrorHandlingMiddleware` 在同一模型重试 3 次后**不抛异常**,而是返回兜底 `AIMessage` —— 现在给它打 `additional_kwargs[LLM_ERROR_MARKER="deerflow_llm_error"]=reason`(model_dump 保留,经 checkpointer 序列化仍在)。`is_llm_error_message(msg)` 据标记(回退按已知兜底文案正文)识别,返回 reason。
- **候选模型链**:`app/gateway/roundtable_model_fallback.py::fallback_model_chain(current)` 按 `config.yaml models[]` 顺序给出 `[当前模型, …其余]`(去重保序)。`reason_text(reason)` 翻译成简短中文。总开关 env `ROUNDTABLE_MODEL_FALLBACK`(默认开,`0/false/off` 关 → 单模型旧行为);`ROUNDTABLE_MODEL_FALLBACK_MAX`(0=试完全部)。
- **触发换模型的情形**:① 单轮 run 抛异常(start_run 失败 / idle 卡死 → `timeout`,其余 `exception`);② 兜底错误文案(`is_llm_error_message`);③ 交付无效(仅 seat/leader;leader 还含「既无正文又无派活」)。席位**只认剥掉 `<think>` 后的可见正文**——思考-only / 空串 / 断流都不算已交付(`deerflow.agents.roundtable_orchestrator.delivery`:`visible_seat_text` / `classify_seat_delivery`)。前台 `_special_run` 对此 yield `status:"invalid_delivery"` 且不广播;`GET /last-ai-message` 回 `{delivered, content, length, invalid_reason}`(思考-only 标 `thinking_only`,总控席位清单标 ⚠️)。后台引擎只把可见正文写入 `deliveries`,续轮用 `build_continue_prompt`:有无效席位时强制重派,禁止声称「已完成交付」。Step3 单例(report/summary/dashboard/ingest/结构化)的产物是落盘文件,**不**把「正文为空」当失败,只对真模型错误换模型。
- **候选模型链**:保留真正模型故障后的自动换模,但“仅有思考 / 暂时无正文 / 空交付”不再判作模型故障,绝不据此切换用户选择的模型。`app/gateway/roundtable_model_fallback.py::fallback_model_chain(current)` 按 `config.yaml models[]` 顺序给出 `[当前模型, …其余]`(去重保序)。`reason_text(reason)` 翻译成简短中文。总开关 env `ROUNDTABLE_MODEL_FALLBACK`(默认开,`0/false/off` 关);`ROUNDTABLE_MODEL_FALLBACK_MAX`(0=试完全部)。
- **触发换模型的情形**:① 单轮 run 抛异常(start_run 失败 / idle 卡死 → `timeout`,其余 `exception`);② 兜底错误文案(`is_llm_error_message`)。交付无效、仅有思考、空串以及普通总控轮既无正文又无派活**均不触发换模**;唯一例外是无工具的最终收口请求已经完整结束仍无正文,此时结果客观不可交付,才继续自动换下一个模型。席位仍只认剥掉 `<think>` 后的可见正文(`deerflow.agents.roundtable_orchestrator.delivery`:`visible_seat_text` / `classify_seat_delivery`):前台 `_special_run` 对无效交付 yield `status:"invalid_delivery"` 且不广播;`GET /last-ai-message` 回 `{delivered, content, length, invalid_reason}`(思考-only 标 `thinking_only`,总控席位清单标 ⚠️)。后台引擎只把可见正文写入 `deliveries`,无效席位继续走既有未交付处理,不切换模型。Step3 单例(report/summary/dashboard/ingest/结构化)的产物是落盘文件,同样只对真模型错误换模型。
- **Step2 总控工具与可配置收口轮数**:普通总控使用真正白名单,只允许 `agent_orchestration` / `ask_clarification`;`write_file`、`present_files`、技能目录与文件/命令工具均不可见。`system_settings.json -> roundtable_room_limit.max_cycles`(默认 8,范围 2..100;管理员在设置弹窗「会商并发与收口」中修改)控制 recommend 模式到第几轮切换为显式 `finalize_only`:空工具白名单、注入已有有效席位全文、要求直接输出最终总结(失败席位如实标缺口),不能表现为“已停止”。一次会商启动时锁定当时值,运行中修改从下一次研讨生效;前台实时会商与后台作业读取同一设置。同一页的「会商文件流式兼容」(`write_file_compatibility_enabled`,默认关) 只作用于会商总控、席位、第三步和后台作业:全局文件流式兼容开着时会商一并使用;全局关着而本开关开着时,只有这些会商 run 改走 `buffered_gateway`。普通对话不读这个开关。chain / dag 的最终总结仍在既定席位完成后走同一纯正文路径;LangGraph 单轮 recursion limit 仍为 1000。
- **后台**:`InProcessRoundtableGateway._run_turn_with_fallback(...)` 包住 `_run_agent_turn`,逐候选模型试;换模型落 `model_switch` 诊断、全失败落 `model_fallback_exhausted`。`run_leader/run_seat/run_report/run_summary/run_dashboard_data` 全走它(report/summary/dashboard 的产物自愈重试保留,并记住已切到的可用模型用于后续续写轮)。各 Turn/Result 带回 `switches: list[ModelSwitch]`,引擎 `_emit_switches` → `JobProgressWriter.model_switch(...)` 往研讨时间线插一条 `role="system"` 系统提示气泡。
- **前台**:`_special_run`/`_leader_run` stream 收尾后检测失败 → yield `{status:"model_switch", role, agent_name, failed_model, next_model, reason}` 帧并用下一个模型重新流式;终态帧带 `model_used`。结构化抽取 `flow_extract.py` 的 round1 `model.astream` 也包了同款换模型循环(emit `event:"model_switch"` 帧)。
- **前台**:`_special_run`/`_leader_run` stream 收尾后仅检测明确的模型错误标记;命中才 yield `{status:"model_switch", role, agent_name, failed_model, next_model, reason}` 帧并用下一个模型重新流式,思考-only/空正文不进入该分支;终态帧带 `model_used`。结构化抽取 `flow_extract.py` 的 round1 `model.astream` 也包了同款换模型循环(emit `event:"model_switch"` 帧)。
- **Step2 单席位隔离**:`engine._run_recommend/_run_chain/_run_dag` 的每个席位执行包 try/except —— 一个席位彻底失败(落 `seat_failed` 诊断 + 置空交付)不拖垮整批/整链/整个作业。
- **前端**:`multi-agent.ts` 解析 `model_switch` 帧 → `onModelSwitch` 回调(清空失败尝试的半截气泡 + 累计的 tool_call delta,phase 退回 connected);`useStep2Orchestration` 弹 toast + 时间线插「系统提示」气泡;`useStep3Summary/Report/Dashboard` + `flow-extract.ts` 重置对应气泡并在步骤卡留可见提示;后台路径经 `job-live.ts::toStep2Dialogue` 把 `role="system"` 渲染成 amber「系统提示」气泡。
- Tests: `tests/test_roundtable_model_fallback.py`。
@@ -1502,6 +1758,29 @@ from the selected space names and hides generic recommendations while these
questions are present. Design and remaining graph/profile phases are in
`docs/KNOWLEDGE_SPACE_RETRIEVAL_AND_WIKI_EXPANSION_ZH.md`.
**Knowledge-base research bookshelf:** `/page/workspace/knowledge/:id/research`
is ordinary Lead Agent Q&A constrained to one existing knowledge base; it does
not invoke or reuse the legacy Deep Research workflow. Any account with read
access may create a private session/draft, while only the existing KB writer
may publish or overwrite its Markdown Wiki article. Published snapshots follow
the KB's read permission, but the session/thread/draft and the continuation
link remain creator-only (administrators do not inherit another user's private
chat). Every run resolves the immutable thread binding from
`knowledge_research_sessions` (registered on `deerflow.persistence.models` so
startup `create_all` builds the table; `get_by_thread` treats a missing table
as no binding so ordinary `/runs/stream` is not aborted), reauthorizes KB read access, accepts exactly one
text-only human turn, forces classic single-KB retrieval and disables tools,
memory, prompt prefixes, planning and subagents; fixed-answer shortcuts are
bypassed. Saving manually upserts a stable
`knowledge-research/{publication_id}` Wiki slug under the reserved root
「深度研究」 without enabling automatic Wiki generation, uses a saved hash for
optimistic overwrite checks and idempotent retries, and keeps the editable
publication title separate from the Q&A title. REST endpoints live in
`app/gateway/routers/llmwiki.py`, runtime enforcement in
`app/gateway/knowledge_research.py`, persistence/migration in
`deerflow.persistence.knowledge_research` / `20261022_01`, and regression
coverage in `tests/test_knowledge_research.py`.
**Embedding ownership:** WeKnora's per-knowledge-base `embedding_model_id`
indexes uploaded source documents; `llmwiki.local_wiki_index.embedding` indexes
the generated Wiki-page mirror inside DeerFlow. Do not couple these two model
@@ -1742,7 +2021,10 @@ Automatic conversation summarization when approaching token limits:
`conversation_context.recent_turn_window_enabled` is a deployment-wide,
default-off alternative to summarization. Administrators manage it in
「设置 → 上下文管理」 through
`GET/PUT /api/system-settings/conversation-context`. When enabled, each lead
`GET/PUT /api/system-settings/conversation-context`. The same block stores
`max_conversation_rounds` (default 20, range 1–50), which `ChatPage` uses as
the maximum number of real user turns in one ordinary chat; missing values in
older settings files retain the 20-turn default. When enabled, each lead
agent model request retains only the current turn and the two preceding real
user turns, together with their assistant/tool continuations and any hidden
context attached to the retained boundary. The operation is request-only:
@@ -2641,3 +2923,24 @@ calls. The active sandbox skill mirror must be synchronized with the canonical c
Gateway detail writes emitted by `fs-collect` and `fs-detail-submit` log a token-masked request and
the corresponding response so an interrupted chain can be distinguished from a failed persistence
call without exposing the gateway bearer token.
### Task-detail custom skill password authentication (2026-10-26)
`skills/custom/task-context` and `skills/custom/task-detail-update` authenticate detail reads/writes
with a short-lived token minted by `POST /api/v1/auth/login/local`, using the runtime-only
`TASK_DETAIL_AGENT_USERNAME` / `TASK_DETAIL_AGENT_PASSWORD` pair and `TASK_DETAIL_GATEWAY_URL`.
They must never use general `DEERFLOW_AUTH_TOKEN` or `TOOLS_TOKEN`, nor emit the password or JWT in
an input, trace, or model-visible output. `/api/parallel-agents/auth/token` is only a legacy fallback:
it follows `sso_enabled` and returns 403 when external SSO is disabled. Password-login failures other
than the policy 403 must fail closed; do not fall back to identity-only token minting. Keep the
canonical custom directories and `.deer-flow/.sandbox-online-skills` mirror synchronized through the
custom-skill reconciliation flow. `scripts/setup_biz_detail_assistant.py` logs in through the same
formal endpoint and accepts `DEERFLOW_SETUP_BASE`, `DEERFLOW_SETUP_USERNAME`,
`DEERFLOW_SETUP_PASSWORD` for deployment overrides; it may fall back to the legacy username route
only for the formal login policy 403, never for a credential or rate-limit failure.
`biz-detail-assistant` must retain the native `ask_clarification` tool. Do not special-case it by
adding that tool to `requested_excluded_tools`, and do not mount `DetailConfirmMiddleware` to forge
confirmation cards from a plain model response: those cards are not genuine model tool calls and break
the standard pause/resume interaction. The final `ClarificationMiddleware` is the single authority for
all missing-information, scope, approach and risk-confirmation questions.