sync update from LAN repo (source dhk-1 @ 3b1ec5b4)

This commit is contained in:
gozeros committed 2026-11-05 10:56:34 +08:00
1 parent da6521b6cb
commit 264e2200d6
995 files changed
+148105 -13399

No files matched your search

+538 -28
View File
@@ -1,12 +1,353 @@
# CLAUDE.md
## Cold agent construction (2026-10-30)
`runtime/runs/worker.py` dispatches opted-in synchronous factories through
`build_agent_with_timing`, backed by `utils/background_init.py`'s
`run_background_init` and one shared `FileIOPool(1, 1, "agent-init")`.
`make_lead_agent._deerflow_background_build = True` opts the built-in Lead Agent
into background construction; unmarked custom factories retain their caller-loop
contract. Serial admission prevents concurrent SDK cold imports and keeps waiting
builds out of authentication's executor. The synchronous timing helper remains
available. Model execution, checkpoint/store attachment and streaming stay in the
original run task; no graph is shared or cached across runs/users.
Every async Lead Agent entry point uses this builder, including non-streaming
and streaming open-chat endpoints and Workflow agent nodes. Do not call the
synchronous factory directly on their event loop: it can block SDK import locks
or deadlock while another background build awaits MCP discovery on that loop.
Async tool consumers (AI writing skill agents and `task_tool`) use
`get_available_tools_async`; deep-research capabilities and job execution also
construct their material providers through `run_background_init`. Keep the
common helper in `utils` with a lazy MCP import to avoid runtime/agent/tools
import cycles. Synchronous client/provider APIs remain available.
Copy the caller's ContextVars into construction and restore the successfully built
context before streaming, including a deferred-tool registry reset to `None`.
The `mcp_initialization_loop` context manager forwards lazy MCP discovery from the
worker to the owner event loop; exit it before copying back context. MCP cache
invalidation and legacy synchronous callers retain their existing semantics.
Cancellation drains the actual worker before releasing capacity, never starts the
graph afterwards, and preserves terminal status/SSE cleanup. Check `abort_event`
after construction as well as honoring task cancellation.
Regression: `packages/harness/tests/test_agent_build_responsiveness.py`,
`test_agent_build_timing.py`, `test_mcp_background_build.py` and the run-finalization
tests; `tests/test_async_agent_entrypoints.py` and
`tests/test_async_tool_initialization.py` exercise overlapping HTTP/workflow/tool
initializers. The HTTP/workflow cases run in subprocesses so an event-loop
deadlock fails with a bounded timeout. Cold profiling found ~4.7 s in model SDK imports/client setup, compared with
~14 ms in skill loading; moving only disk reads would not address this stall.
Local live cold acceptance (2026-10-30, official MiniMax-M2.7, tools enabled):
the first build took 4.28 s while a second request received stream metadata in
402 ms. During the conservative interior of the build window, 43 health/auth
probes completed successfully; across all three conversations, health/auth
P95 latency was 28/73 ms, with no event-loop-stall warning. Every answer ended
normally and all synthetic threads were removed. Initialization/model latency
still contributes to first-answer time; this change prevents blocking the server
loop, rather than promising immediate model output.
A separate live read-only tool check uploaded a random marker, allowed only
`read_file`, and confirmed the tool result and final answer reproduced that marker
without placing it in the prompt; the stream ended normally and the thread was removed.
Worker compatibility: no HTTP/SSE, task payload, auth or database contract changes.
Running workers without reload retain old construction behavior; the new factory
marker is harmless to old executors. `worker_pool/revision.py` hashes only the
adapter directory and does not identify this harness patch. This change does not
alter the controller's existing snapshot/ImageID-based drain-and-replace policy
or `on_disconnect=cancel` behavior: restarting a production controller can still
rotate idle workers or disconnect proxied streams. Do not infer uninterrupted
worker operation from these process-local initialization changes. The live local
acceptance instance has no enabled worker controller and does not touch remote
workers. Existing worker routing/admission/storage/discovery/diagnostics/launcher
compatibility regressions: 68 passed.
Final combined regression (2026-10-30): all 58 newly added responsiveness,
cancellation, MCP-entrypoint and provider-lifecycle cases plus the 68 existing
worker compatibility cases passed together (126 passed, no skips).
The live recheck uploaded and converted 20 DOCX files while another conversation
produced 43 visible text increments; all 172 concurrent authenticated probes
succeeded (P95 105 ms, max 436 ms). A fresh-process cold build took 10.92 s while
617 heartbeat samples continued (P95 scheduling delay 25 ms, max 146 ms).
The existing backend and child workers were not restarted for this recheck;
later entrypoint changes were verified through isolated HTTP/workflow tests.
The full versioned backend/harness run completed with 3140 passed, 39 failed,
4 skipped and 1 deselected. Against baseline `47cfed02`, 30 failures also failed
on HEAD and the other 9 passed in isolated reruns on both versions; no current-only
stable failure remained. The deselected legacy
`test_cancelled_history_closes_iterator` hangs on both versions: its fake saver
implements `alist`, while `limit=1` takes `aget_tuple`, so its unbounded entry
event wait never completes. This is not a claim that the repository's entire
test suite is green. Changed Python files introduced no Ruff diagnostics.
## Upload responsiveness (2026-10-30)
`app/gateway/routers/uploads.py` runs each staged upload transaction in a dedicated
`FileIOPool(2, 2, "uploads")`, including 256 KiB spool copies, signature validation,
optional synchronous conversion, sandbox acquisition/sync, commit, rollback and
cleanup. Resolve the thread owner asynchronously before dispatch and propagate
contextvars into workers. Waiting batches consume no worker from authentication,
artifacts or event streaming. Per-event-loop directory locks serialize same-thread
batches and listing/deletion through rollback/cleanup; keep them held until a
cancelled worker stops. Listing/deletion use the artifact file pool off the loop.
Cancellation signals a cooperative event, checks between file chunks and around
commit/sync, and waits for real completion before multipart handles may close.
The response contract and upload-policy enforcement remain unchanged.
`sandbox/sandbox_provider.py` serializes initial singleton construction with
reset/replacement/shutdown. Concurrent cold uploads must not create competing
providers with independent sandbox reconciliation and idle-cleanup threads.
Regression: `packages/harness/tests/test_sandbox_provider_concurrency.py`.
`file_conversion.py` also isolates its asynchronous conversion/extraction callers
in a separate two-thread pool. PDF/Office parsing runs in a lazily initialized,
two-process pool using `spawn` on Windows and Linux, so Python parser CPU work
does not contend for the gateway GIL. Pass only file paths and converter choices;
bound returned text inside the child and write sidecars in the owning parent
worker. The global parser semaphore bounds sync/async callers together. A broken
process pool fails the current conversion and is rebuilt on the next call.
`FileIOPool.run_until_complete` provides non-abandoning cancellation for workers
that own temporary input files; cancellation cannot close/delete an input still
being parsed by a child process.
Regression: `tests/test_upload_responsiveness.py`,
`tests/test_conversion_responsiveness.py`, `tests/test_uploads_policy_enforcement.py`.
Local acceptance (Windows, Python 3.12, synthetic files via ASGI; no delay injected):
20 x 900 KiB TXT uploads reduced first concurrent probe delay from 664/682 ms to
3/4 ms. With real MarkItDown conversion, 20 x 800-paragraph DOCX uploads reduced
maximum probe scheduling lag from 286/285 ms to 41/14 ms; all 20 sidecars matched
the source text. The first spawned reader took 13.7 s to initialize/convert in
this environment, while concurrent probe lag stayed below 16 ms. These are local
responsiveness measurements, not production throughput guarantees. Upload,
policy, conversion, large-file and runtime-contention regressions: 87 passed.
Full-suite collection is separately blocked by the pre-existing missing
`deerflow.models.global_prompt` import in `tests/test_global_prompt.py`.
## Thread metadata query performance and ORM deadlines (2026-10-22)
`ThreadMetaRepository._query_conditions` is shared by search/count: common
SQLite scalar predicates and PG/MySQL string predicates plus system-thread
exclusion run before SQL pagination/counting. Preserve AUTO owner resolution and
the nonempty taskId owner bypass while applying agent_id/actionId together.
`json_compat.JsonPredicate` binds values, preserves local missing/null and numeric
equality on SQLite, and uses binary MySQL id equality and Unicode-safe blank
seat detection. Nested/unusual-key filters and PG/MySQL numeric filters remain exact
residual filters; openGauss retains its original path. Never drop an unsupported
predicate or replace this with upstream's unadapted SQLite/PG-only compiler.
PG's omitted pool_recycle defaults to 300s (other backends remain 1800s);
database.command_timeout is finite positive or null, default 30s. The shared
pooled-engine kwargs are used on initial creation and auto-create retry. Only
asyncpg receives command_timeout; this does not configure psycopg checkpointers.
Regression entries: packages/harness/tests/test_thread_metadata_sql_filters.py,
test_database_pool_deadlines.py and the existing test_thread_operation_timing.py.
Master integration (2026-10-23) retains the maintained task/agent/action index
columns as additional SQL narrowing before exact JSON predicates. Keep dedicated
task deep-link card synchronization on every existing metadata mutation. Hydrated
history still uses slim_display to bound oversized tool payloads without editing
checkpoint contents; frontend retrieval-progress scopes remain isolated per turn.
Thread POST defaults to metadata-only creation; only an explicit
initialize_checkpoint=true writes an empty checkpoint. Never enqueue an empty
checkpoint in the background: it could become the head after the real first run.
Authorized metadata-only threads have empty state/history. Metadata GET defaults
to the SQL row (running maps to busy); include_state=true and checkpoint-only
legacy rows retain state reads. Owner decorators run before unscoped row reads.
History explicitly uses root checkpoint_ns and closes alist before serialization;
only the latest head carries full channel values.
The frontend uses BackgroundHistory as the SDK's external history adapter. Its
post-stream mutate immediately captures streamed state with no checkpoint id;
explicit end triggers background hydration. EOF/error still requires existing
run confirmation/rejoin. Every async write is fenced by thread/run/epoch/request;
new turns, navigation and unmount invalidate prior requests even if abort is
ignored. Reject history missing the newest streamed turn, bound each request to
15 seconds and retry completion sync once. Failure keeps the streamed snapshot.
Normal explicit-end callbacks use the completed streamed snapshot before the
next send can change page refs; background hydration never repeats them. Recovery
may have missed final SSE frames, so its callback waits for guarded hydration or
the retained snapshot fallback. New submits use checkpoint:null to follow the
latest server head.
Auth's SQLiteUserRepository is a historical name: its injected session factory
is the shared business ORM. With database.backend=mysql, login/auth/users and SQL
business repositories use MySQL, independently of checkpoint configuration.
LangGraph Store still follows checkpointer config and startup awaits runtime
setup/migrations; do not claim total service/startup isolation from PG or migrate
existing stores implicitly. See docs/THREAD_LATENCY_OPTIMIZATION_ZH.md for the audit.
Additional regression entries: test_thread_lazy_checkpoint.py,
tests/test_account_authentication.py and frontend pnpm test:thread-history.
## Multi-user contention controls (2026-10-22)
- `utils/file_io.py` separates bounded artifact IO (4 workers / 8 admitted) and
event disk IO (2 / 4) from the default executor used by authentication.
Contextvars propagate; cancellation cannot release admission until disk work
actually stops. Heavy artifact listing, ZIP previews, sniffing and writes use it.
- `runtime/runs/manager.py` sweeps completed live records after 300 seconds,
200 candidates per pass. It requires the actual task to be done and verifies
status/error/metadata/kwargs in the backing RunStore, updating a final snapshot
when necessary. Failed persistence retains the record for later retry. It never
deletes RunStore rows, checkpoints, events or stream buffers. `watch_completion`
starts grace after finalization, and lifespan closes the sweeper. The live
superseded fence survives a newer record's retirement; finalization never adds
a business-database history read to its completion fence.
- Run routes authorize the thread before unscoped persisted lookups, and verify
that the loaded run belongs to that exact thread. Preserve private-owner 404s,
taskId/system-thread sharing and existing action/agent partitions. Completed
rejoin without a live bridge buffer emits end immediately; Last-Event-ID replay
during the existing replay window remains untouched.
- `run_events.backend=memory` is a compatibility alias for
`SpoolingRunEventStore`, stored under `runtime_home()/run-events/events.sqlite3`.
Per-thread sequence assignment and each batch commit are transactional; paged
queries do not retain completed events in RAM. No automatic event/message
deletion or truncation is introduced. The pure `MemoryRunEventStore` remains
available for unit tests. Explicit db/JSONL formats are unchanged; JSONL disk
operations now use the event executor and striped per-thread locks. Keep the
spool and its WAL on the persistent local runtime volume. Do not reclassify it
as a PG checkpoint or silently change explicit db configuration.
- `runtime/fair_queue.py` provides bounded round-robin admission by user for the
subagent scheduler (3 total / 2 per user) and background roundtables (4 / 2).
Queued jobs consume no worker; subagent deadlines include queue time. Per-user
pending capacity also prevents one user from exhausting the entire backlog.
Actual coroutine completion, including cancellation before its first step,
acknowledges slot release. Parent context/identity propagation stays intact.
`RuntimeContentionConfig` defines optional budgets; limits apply per process
and require restart. The older roundtable per-user environment override remains.
- Both internal roundtable creation helpers only create metadata. Init does not
append a synthetic seat message via loopback; `_leader_run` injects authoritative
IDs on each real input, deriving them from the thread mapping for older clients
without seat_meta. Background first-leader injection remains. Never schedule a
late empty checkpoint write: it could overwrite real history.
Regression entries: `test_runtime_contention.py`, `test_completed_run_retention.py`,
`test_thread_lazy_checkpoint.py`, `test_run_completion_timeout.py`,
`tests/test_position_roundtable_sessions.py`, roundtable role/model/seat tests,
artifact/auth/model concurrency tests and frontend `pnpm test:thread-history`.
See `docs/MULTI_USER_CONTENTION_AUDIT_ZH.md` for measured scope and remaining
model/SQL/checkpointer contention. Process-local run control still requires the
current single-worker deployment; these queues do not enable multi-worker routing.
## Ordinary knowledge smart retrieval dual surface (2026-10-16)
`SmartReasoningRetrievalMiddleware` treats knowledge retrieval as an ordinary
model-selected information tool inside the reasoning loop. A direct answer or
skill-only turn is never followed by a forced search. Once the model decides it
needs selected-library evidence, the first retrieval action is
`ordinary_knowledge_search`; after reading those raw hits, the model decides
whether Wiki search/read, regex verification, document reads, or another raw
query is needed for a real evidence gap. Each model iteration executes at most
one smart-retrieval call, and any content emitted beside that call is removed so
the final answer can only follow the completed tool loop. The middleware
deduplicates current-turn raw queries, regexes, Wiki
patterns, `page_ref+cursor` reads and document pages; a new continuation cursor
is a distinct action, while a repeated/non-advancing cursor terminates that
page. If deduplication removes an empty duplicate-only response, the middleware
makes one tool-free synthesis retry over existing evidence instead of returning
a generic non-answer. The user only asks the question; no retrieval recipe is
required in the message. Original document chunks win when Wiki and raw
evidence conflict, while stable general knowledge may explain background if it
is clearly distinguished from library facts and inference. Smart-mode answers
do not require `[1]`/`[2]` markers or a numbered reference list. Classic
retrieval does not mount this behavior.
Default WeKnora clients are cached per running event loop because they own an
async authentication lock and a cached directory task; never restore a
process-wide client cache. Smart-tool mapping authorization reads are routed to
the SQL engine's owning loop before provider I/O. These two boundaries prevent
healthy selected knowledge bases from being reported as `PROVIDER_UNAVAILABLE`
when Gateway and LangGraph work run on different loops.
Wiki creation resolves its KnowledgeQA model only from WeKnora's database model
records. It prefers an active explicit default and calls WeKnora's model debug
endpoint before binding that model to `summary_model_id` and
`wiki_config.synthesis_model_id`; `active` alone is not proof that the upstream
provider still serves the model. A candidate returning `ok=false` may be skipped,
but authentication, timeout, server, and response-contract failures remain service
errors. Existing documents that already reached Wiki dead letters require an
explicit reprocess after both model fields are repaired. Directory and focus
instructions remain persisted in the knowledge base's database `wiki_config`.
`llmwiki_reasoning_tools._wiki_search_impl` applies the safe POSIX expression
to normalized Wiki pages, but obtains bounded candidates from both the local
database keyword index and the newest-page snapshot. A completed local mirror
with no regex match probes the live WeKnora provider, which prevents newly
generated Wiki pages from being hidden by a stale mirror when periodic sync is
off. A 0/0 mirror is never considered complete. Provider `content`,
`content_md`, and `markdown` bodies are normalized for both search and page
read. Keep the live-provider zero-match fallback when changing mirror policy.
## Gongwen bounded delivery (2026-10-07)
Gongwen seed v3 ships `scripts/delivery.py` through the restricted
`run_gongwen.py deliver` / `verify-file` commands. Deliver stages a new DOCX,
runs draft or repair with one output verification, injects only the explicit
elements JSON, then checks package/XML/body/element presence without treating
the red header or imprint as body text. It publishes without replacing an
existing output and returns ready/review_required/failed JSON plus the review.
No renderer or external model is invoked; visual pagination and factual review
are not claimed. Do not add automatic repair/check loops or ad-hoc Python to
the workflow/SOUL prompts. P1/P2 warnings are not delivery blockers.
The vendored optimize command reuses its input check and returns full output
review details; failed verification exits nonzero. Draft propagates its resolved
document type into optimization. Gateway generate/repair reuse this invocation's
review, with one check fallback for older custom skill versions. Do not reuse
arbitrary client-supplied review files as trusted verification.
Regression coverage: `tests/test_gongwen_delivery.py` and existing gongwen tests.
## Retrieved skill files (2026-10-07)
`citation_display.retrieved_files_enabled` (default true) controls retrieved-file
cards globally. It is read and patched through `/api/system-settings/citation-display`;
updates require `_require_admin` and preserve other citation settings.
Chat file cards reuse the existing artifact and knowledge original-file readers.
`GET /api/llmwiki/sources/resolve` accepts either a local mapping id or a WeKnora
provider id and returns only an authorized local mapping id. A provider lookup
must be followed by the existing `get_authorized(..., write=False)` check before
returning any identifier; missing and unreadable sources both return 404.
The frontend uses this resolved id for both preview and original-file download.
## Qwen3.8 Flash Next stream integrity (2026-10-07)
`deerflow.models.chat_stream_state.ChatStreamState` belongs to one provider invocation.
`PatchedChatOpenAI` enables it only for the normalized `qwen3.8-flash-next` id;
the model factory also checks the configured name for deployments using a generic
provider id. The local-only `strict_stream_integrity` override is never sent upstream.
Keep unrelated Qwen versions and other models on their existing stream path by default.
Only explicit `choice.message` string snapshots are converted into unseen suffixes.
True `delta` text wins and is appended verbatim, including repeated content/reasoning;
structured provider payloads stay opaque. Repair a null terminal delta for this model.
EOF without a first-choice `finish_reason`, or a contradictory snapshot, raises the
content-free `IncompleteChatStreamError`. The existing LLM middleware retries within
its normal cap before any failed result reaches the checkpoint or tool executor.
Cancellation propagates without retry; a declared finish reason (including `length`)
retains provider semantics. Do not guess completion from answer punctuation.
Non-stream exchange capture is opened and closed in `_generate` / `_agenerate` so
executor-based conversion never resets a ContextVar token from another context.
Reuse the safe response dump for conversion and preserve typed `parsed` / `refusal`
metadata without serializing arbitrary parsed objects a second time.
Regression entry: `packages/harness/tests/test_patched_openai_text_integrity.py`.
Coverage includes sync/async provider replay, model/alias scoping, failed-attempt
checkpoint isolation, same-thread follow-ups, tool safety, cancellation and 20
overlapping streams that reuse the same provider completion id. These fixtures
exercise code-level compatibility gaps; they do not establish the deployed server's
actual trigger without its response evidence.
??
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
### Personal follow-up question preference
`GET/PUT /api/user-preferences` exposes `followup_suggestions_enabled`, defaulting to `false` for both new accounts and older JSON rows without the key. Partial updates merge into the current account's `user_preferences.preferences` JSON alongside `hide_admin_prompt_prefix`; no database migration is needed. Frontend recommendation generation and display must both require this explicit opt-in. Regression coverage: `tests/test_user_preferences.py` (default-off behavior, persisted opt-in/opt-out, account isolation and partial-update preservation).
### Ordinary Q&A map visualization
The ordinary lead-agent prompt includes `_ORDINARY_QA_VISUALIZATION_SECTION` only for normal, non-bootstrap, non-writing, non-notebook, non-scheduled runs and defines the highest-priority output-carrier gate. Only an explicit current-request ask for HTML, a webpage/page, dashboard, large-screen view, or `.html` file permits reading `page-report-generation`, calling `page_report_template`, or writing HTML. “Draw a flowchart/chart/map/mind map” and “make a visualization” never imply HTML: ordinary flowcharts must be inline Mermaid, ordinary charts keep the established message-inline path, and requested maps emit strict JSON in a `map-json` fence. A mind map is emitted only for an explicit current-request ask for a mind map/knowledge map (or `analysis-mindmap`); ordinary analyses, summaries, outlines, flowcharts, and generic JSON must never auto-upgrade into one. Its strict `mindmap-json` contract is a bounded rooted tree (`id`, `parentId`, `label`, semantic `kind`, optional detail/source refs), targeting 12–24 nodes with a hard cap of 36, at most four layers including the center topic, at most six top-level dimensions and three children beneath every later node. Models must consolidate related facts into `detail`/`sourceRefs`, not make one node per report bullet, datum, paragraph, or reference. An unfinished dedicated `mindmap-json` fence (and compatible historical `mindmap` fence) remains an ordinary streaming code block; only the completed, valid object lazy-renders as a compact left-to-right message card. Clicking a parent node, or its +/− control at the outgoing connection point, expands/collapses its child tree and the canvas uses an explicit visible edge stroke. The full dialog keeps its close and view tools in the top-right and a layout-derived all-map preview in the bottom-right; the message preview also permits pan, wheel zoom, and zoom/reset controls. The message card's icon-only download control opens PNG/SVG/JSON choices. Generic JSON never invokes this renderer. Ambiguity defaults to inline output. Once the HTML gate passes, delivery is mandatory as a sandbox-generated `/mnt/user-data/outputs/*.html` artifact opened with `present_files`; a full inline `html` fence is forbidden as a substitute. The shared frontend `MarkdownContent` keeps its exact legacy branch when no map-shaped fence exists; dedicated `map-json` fences and complete generic `json` fences that contain map fields and pass the full map schema are validated and lazy-rendered from `components/workspace/visualizations/`. Map/loading/error cards use the full message-content width so their edges align with the step panel. Non-map and incomplete generic JSON retain the ordinary code-block path.
The ordinary lead-agent prompt includes `_ORDINARY_QA_VISUALIZATION_SECTION` only for normal, non-bootstrap, non-writing, non-notebook, non-scheduled runs and defines the highest-priority output-carrier gate. Only an explicit current-request ask for HTML, a webpage/page, dashboard, large-screen view, or `.html` file permits reading `page-report-generation`, calling `page_report_template`, or writing HTML. “Draw a flowchart/chart/map/mind map” and “make a visualization” never imply HTML: ordinary flowcharts must be inline Mermaid, ordinary charts keep the established message-inline path, and requested maps emit strict JSON in a `map-json` fence (markers use `name`+`coordinate` plus optional `color`; lines are included only when the user explicitly asks to connect places, must use `coordinates` with at least two `[lng,lat]` points, not `from`/`to`/`coords`, and default to `arrow`/`flow`). The frontend recovers those common aliases, assigns distinct marker colors, draws requested lines as looping SVG arrows, and drops unusable lines so markers still render. A mind map is emitted only for an explicit current-request ask for a mind map/knowledge map (or `analysis-mindmap`); ordinary analyses, summaries, outlines, flowcharts, and generic JSON must never auto-upgrade into one. Its strict `mindmap-json` contract is a bounded rooted tree (`id`, `parentId`, `label`, semantic `kind`, optional detail/source refs), targeting 12–24 nodes with a hard cap of 36, at most four layers including the center topic, at most six top-level dimensions and three children beneath every later node. Models must consolidate related facts into `detail`/`sourceRefs`, not make one node per report bullet, datum, paragraph, or reference. An unfinished dedicated `mindmap-json` fence (and compatible historical `mindmap` fence) remains an ordinary streaming code block; only the completed, valid object lazy-renders as a compact left-to-right message card. Clicking a parent node, or its +/− control at the outgoing connection point, expands/collapses its child tree and the canvas uses an explicit visible edge stroke. The full dialog keeps its close and view tools in the top-right and a layout-derived all-map preview in the bottom-right; the message preview also permits pan, wheel zoom, and zoom/reset controls. The message card's icon-only download control opens PNG/SVG/JSON choices. Generic JSON never invokes this renderer. Ambiguity defaults to inline output. Once the HTML gate passes, delivery is mandatory as a sandbox-generated `/mnt/user-data/outputs/*.html` artifact opened with `present_files`; a full inline `html` fence is forbidden as a substitute. The shared frontend `MarkdownContent` keeps its exact legacy branch when no map-shaped fence exists; dedicated `map-json` fences and complete generic `json` fences that contain map fields and pass the full map schema are validated and lazy-rendered from `components/workspace/visualizations/`. Map/loading/error cards use the full message-content width so their edges align with the step panel. Non-map and incomplete generic JSON retain the ordinary code-block path.
HTML artifacts are single-page by default. Omitted `preferred_page_count` is normalized to `1` during direct validation. A count greater than one is allowed only when the current user directly asks for multiple pages or gives a page count; content length and report chapter count never imply multi-page intent. Explicit multi-page intent without a count defaults to a three-page lightweight portal. It is one artifact HTML, not a folder of linked pages: `<main class="pages">` contains logical `.page` sections with unique `id`/`data-page`, and `href="#page-..."` navigation works in the sandboxed Blob preview through the injected anchor handler. Its validator requires the local Bootstrap/Alpine assets, `.pages`, multiple `data-page` sections and real anchor navigation: every actual `<a href="#page-...">` and every script-set `location.hash` must match a unique logical-page id, and each page must be reachable; comments or `#overview`/`#stations` data-page values cannot spoof this check. `MultiPageHtmlMiddleware` repeats that structural inspection at the `present_files` boundary, so a model that skips validate or writes custom tabs cannot expose a nonconforming portal. A user-uploaded background goes only on the requested logical `.page` in this mode (home page when unspecified), not on every page. When a template is explicitly requested, prepare strictly excludes profiles from the wrong page-count mode, persists `requested_page_count` and `page_mode`, and validate reloads them. Validation rejects multiple detected page views (`data-page`, tabpanels, `.report-page`, or `.view`) in single-page mode, or a single view when multi-page was explicitly requested.
@@ -16,13 +357,13 @@ For an explicit multi-page HTML request without an explicit template request, `M
### Generated HTML artifact visualization runtimes
The same normal lead-agent prompt includes `_HTML_ARTIFACT_VISUALIZATION_SECTION`, but its contents are executable only after the explicit HTML-intent gate above passes; visualization content alone never implies this intent. New generated HTML explicitly loads browser distributions from Gateway `/api/public/html-runtime/`: `bootstrap.min.css`, `alpine.min.js`, `bootstrap.bundle.min.js`, `g6.min.js`, `echarts.min.js`, and then `deerflow-runtime.js`; it can also load the packaged country/region map catalog: `china-provinces`, `us-states`, `japan-prefectures`, and `taiwan-counties`. Bootstrap is the responsive layout/modal layer and Alpine is the small state/list-rendering layer; jQuery, React, Vue, Babel and public CDNs are not part of this artifact contract. The files are package resources under `packages/harness/deerflow/html_runtime/assets/` and ship with the harness wheel, so HTML execution does not depend on frontend Vite/node_modules assets, Leaflet, map tiles, or a public CDN. `config.yaml -> html_runtime.base_url` is the browser-reachable absolute runtime base written into generated source (localhost default; deployments must replace it with the real Gateway origin). Flowcharts use paired `data-df-flow` JSON blocks and AntV G6, not Mermaid; charts use `data-df-chart`; maps use ECharts Canvas-backed `data-df-geo` with the local GeoJSON catalog. `data-df-chart` must be strict JSON: no JavaScript callbacks or function-valued colors, with literal colors/data-item styles and string tooltip formatters instead. Dark HTML must set `data-df-theme="dark"` and explicitly give ECharts global/title/legend/axis/visible-series labels readable light text, including `nameTextStyle` for named axes. Map `description` values are plain text because the runtime escapes markup; click-to-filter or page-switch behavior listens to `deerflow:geo-map-click` and uses a valid `#page-...` anchor. `deerflow-runtime.js` waits until a hidden logical page has a real layout before creating ECharts and tracks later container changes with `ResizeObserver`, so map/chart canvases never depend on a browser-window resize for their first visible render. It initializes all three contracts, opens a detail popover for described regions/markers, and exposes catalog URLs through its public dynamic response. Map large screens use a centered map stage with narrow side rails and bottom summaries, rather than cards layered across a full-page map background. Frontend runtime replacement is bypassed whenever this explicit Gateway runtime is present. This remains separate from ordinary message rendering; ordinary `map-json` keeps its existing frontend tile renderer and `VITE_MAP_TILE_URL` configuration, and ordinary `flow-json`/`chart-json` must not be reintroduced.
The same normal lead-agent prompt includes `_HTML_ARTIFACT_VISUALIZATION_SECTION`, but its contents are executable only after the explicit HTML-intent gate above passes; visualization content alone never implies this intent. New generated HTML explicitly loads browser distributions from Gateway `/api/public/html-runtime/`: `bootstrap.min.css`, `alpine.min.js`, `bootstrap.bundle.min.js`, `g6.min.js`, `echarts.min.js`, and then `deerflow-runtime.js`; it can also load the packaged country/region map catalog: `china-provinces`, `us-states`, `japan-prefectures`, and `taiwan-counties`. Bootstrap is the responsive layout/modal layer and Alpine is the small state/list-rendering layer; jQuery, React, Vue, Babel and public CDNs are not part of this artifact contract. The files are package resources under `packages/harness/deerflow/html_runtime/assets/` and ship with the harness wheel, so HTML execution does not depend on frontend Vite/node_modules assets or a public CDN. `system_settings.json -> html_map.engine` (`GET|PUT /api/system-settings/html-map`, admin write) chooses `echarts` (default, local GeoJSON `data-df-geo`) or `leaflet` (地图网页, `data-df-map`). The map tile URL saved in the settings dialog (`system_settings.json -> html_runtime.map_tile_url`, `GET|PUT /api/system-settings/html-runtime`) is injected into `deerflow-runtime.js` as `mapTileUrl` and is used only by the Leaflet engine. Both initializers stay in the runtime so already generated pages keep rendering after the switch. The next ordinary Q&A prompt and `page_report_template` validation follow the selected engine. The runtime base URL saved in the same settings block is the browser-reachable absolute address written into generated source. An empty value falls back to an older `config.yaml` entry and then `http://localhost:8001/api/public/html-runtime`. Flowcharts use paired `data-df-flow` JSON blocks and AntV G6, not Mermaid; charts use `data-df-chart`; maps use ECharts Canvas-backed `data-df-geo` with the local GeoJSON catalog. `data-df-chart` must be strict JSON: no JavaScript callbacks or function-valued colors, with literal colors/data-item styles and string tooltip formatters instead. Dark HTML must set `data-df-theme="dark"` and explicitly give ECharts global/title/legend/axis/visible-series labels readable light text, including `nameTextStyle` for named axes. Map `description` values are plain text because the runtime escapes markup; click-to-filter or page-switch behavior listens to `deerflow:geo-map-click` and uses a valid `#page-...` anchor. `deerflow-runtime.js` waits until a hidden logical page has a real layout before creating ECharts and tracks later container changes with `ResizeObserver`, so map/chart canvases never depend on a browser-window resize for their first visible render. It initializes all three contracts, opens a detail popover for described regions/markers, and exposes catalog URLs through its public dynamic response. Map large screens use a centered map stage with narrow side rails and bottom summaries, rather than cards layered across a full-page map background. Frontend runtime replacement is bypassed whenever this explicit Gateway runtime is present. This remains separate from ordinary message rendering; ordinary `map-json` keeps its existing frontend tile renderer and `VITE_MAP_TILE_URL` configuration, and ordinary `flow-json`/`chart-json` must not be reintroduced.
User-uploaded raster images can be used as generated-HTML backgrounds through signed, unauthenticated `GET /api/public/html-images/{token}` URLs. `UploadsMiddleware` mints one URL per uploaded `.jpg/.jpeg/.png/.webp/.gif/.bmp` and injects it into `<uploaded_files>` as `Public URL`; the HTML contract requires that exact absolute URL in `background-image`, never a sandbox path, artifact API path, relative path, or base64 data URI. Single-page artifacts place it on `html/body`; the multi-page portal places it only on the requested logical page. Tokens are HMAC-signed, do not expire (the URL is written into durable HTML), and only grant read access to one file under the thread `uploads/` or `outputs/` directory. Engine: `deerflow.html_runtime.public_images`; router: `app/gateway/routers/html_images.py`; tests: `tests/test_html_images.py`.
The G6 runtime must not mutate author layout by auto-switching long LR graphs to TB or by increasing the target element height after render. Layout quality is decided before delivery: keep a flow to at most eight nodes where possible; otherwise split it or declare TB with at least 600px. Each flow gets its own document row and a parent that can grow naturally, preventing canvas overlap with later sections.
Markdown-to-HTML conversion must inventory source Mermaid/flowcharts, charts, and maps before editing. Direct `page_report_template validate` derives `required_visualizations` from its `required_features` and `page_brief`; an explicit-template prepare also persists a machine-readable `visualization_contract` in the per-output brief, which validate reloads and merges with its arguments. Required flowcharts fail validation unless the HTML contains `g6.min.js`, `deerflow-runtime.js`, a `.df-flow` container, and `data-df-flow`; embedded Mermaid and inline `new G6.Graph(...)` are errors. Required charts use ECharts + `data-df-chart`; required maps use ECharts + `data-df-geo`, an `.df-geo-map` container, and one of the four local GeoJSON catalog entries. Leaflet, `data-df-map`, map tiles, and manual `echarts.init` are validation errors for generated HTML. A failed report must be repaired and revalidated before `present_files`.
Markdown-to-HTML conversion must inventory source Mermaid/flowcharts, charts, and maps before editing. Direct `page_report_template validate` derives `required_visualizations` from its `required_features` and `page_brief`; an explicit-template prepare also persists a machine-readable `visualization_contract` in the per-output brief, which validate reloads and merges with its arguments. Required flowcharts fail validation unless the HTML contains `g6.min.js`, `deerflow-runtime.js`, a `.df-flow` container, and `data-df-flow`; embedded Mermaid and inline `new G6.Graph(...)` are errors. Required charts use ECharts + `data-df-chart`. Required maps follow `html_map.engine`: ECharts uses `data-df-geo`, an `.df-geo-map` container, and one of the four local GeoJSON catalog entries; 地图网页 uses Leaflet `leaflet.css`/`leaflet.js`, `data-df-map`, and a `.df-map` container. The unselected contract, handwritten map initialization, and (for ECharts) map tile URLs are validation errors. A failed report must be repaired and revalidated before `present_files`.
G6 nodes use light semantic rounded cards with matching borders, dark high-contrast text, centered type-prefixed labels, subtle shadows, and non-animated initial rendering (avoids a faded first frame). More than seven nodes automatically switch an `LR` request to layered `TB` layout and expand a short container to 720px, preventing long workflows from shrinking into illegible shapes; drag/zoom remain enabled.
@@ -189,6 +530,28 @@ commands.py` and `tests/test_position_roundtable_concurrency.py` cover the
state transitions; `tests/test_roundtable_postgres_concurrency.py` is the real
PostgreSQL acceptance suite (set `DEERFLOW_POSTGRES_CONCURRENCY_TEST_URL`).
Step-2 roundtable rooms share one deployment-wide cap,
``system_settings.json -> roundtable_room_limit`` (default enabled, 3 rooms).
``GET/PUT /api/system-settings/roundtable-room-limit`` is the admin switch.
Occupancy is summed across **every account**: durable foreground leases keyed
by a hidden per-live-execution ``roomSessionId``, in-flight LangGraph runs on
threads stamped with the corresponding ``roundtable_room_key``, and executing
background jobs (``queued`` / ``running`` / ``cancel_requested`` with a live
worker lease or a recent update). Different pages running the same saved draft
have different session ids and therefore take different rooms; all seats and
Step 3 calls within one live execution share its key. Run-manager inspection
is reconciliation-only: when a durable lease/job already owns the same
draft+user/task scope, stale or overlapping run keys in that scope are not
counted again. Merely opening the page,
viewing saved history, paused ``awaiting_input`` jobs, stale historical rows,
and Step-1 intent threads do not occupy a slot. Foreground-to-background
handoff replaces the foreground lease with the durable job occupancy. There is
no per-user pool. Admission is serialized by a database gate row, and the cap
is enforced at acquire, init, every model-producing run, and background-job
start. Full pools return HTTP 409; an enabled gate with missing storage or a
missing session id fails closed. The switch applies on the next admission
without a restart. Position and eight-department roundtables are not counted.
Roundtable backend jobs use a database unique active-dedupe key plus a leased
dispatcher, never process-local start deduplication. Cancellation is two
phase (`cancel_requested` then `cancelled`): the lease owner’s watcher normally
@@ -335,6 +698,8 @@ from deerflow.config import get_app_config
### Agent System
**Gallery chat link.** An agent's `config.yaml` may store `chat_link` (`url`, `open_mode` of `blank`/`self`, and light-app-shaped `route_params`). The gallery 「对话」 button opens that resolved URL; an empty url keeps the in-app chat. The list endpoint reads the mirrored `agent_extras.chat_link` column (nullable JSON, added by the startup column sweep) instead of parsing every config file. Create/update and directory backfill write the mirror; a skills-only `set_extras` leaves an existing link in place.
**Lead Agent** (`packages/harness/deerflow/agents/lead_agent/agent.py`):
- Entry point: `make_lead_agent(config: RunnableConfig)` registered in `langgraph.json`
- Dynamic model selection via `create_chat_model()` with thinking/vision support
@@ -345,8 +710,12 @@ from deerflow.config import get_app_config
**Forced-research built-in agent.** `forced-research-responder` (「强制检索输出助手」) is seeded from `app/gateway/routers/_forced_research_seed_assets/` before `_sync_legacy_agents`. `ForcedResearchMiddleware` is mounted only for this stable id. On every visible user turn it requires a successful information-gathering tool result before the model may finish; `tool_choice=required` is backed by an `after_model -> model` retry for gateways that ignore tool choice, with an eight-attempt failure cap that permits only an honest failure response. Skill discovery, reading `SKILL.md`, and writing a helper script do not satisfy the gate; web/knowledge/MCP results, non-skill uploaded-file reads, or successful Python skill-script execution do. Tool filtering removes `present_files`, `skill_manage`, task/schedule/memory/clarification surfaces. The middleware permits `write_file`/`str_replace` only for `/mnt/user-data/workspace/*.py` and permits `bash` only for `.py` execution without redirection/document paths, so skills can run scripts but the agent cannot create Markdown/document artifacts or write `outputs`. Tests: `tests/test_forced_research_agent.py`.
**Elasticsearch offline operations built-in agent.** `elasticsearch-offline-ops` (「Elasticsearch 离线运维助手」) and its same-name public skill are seeded from `app/gateway/routers/_elasticsearch_offline_ops_seed_assets/` before `_sync_legacy_agents`. Seeding is missing-directory-only so administrator edits survive restarts. The agent intentionally has `file:read`, `file:write`, and `bash` but no web group: air-gapped guidance comes from the shipped version-parameterized deployment and troubleshooting references. Its SOUL requires the exact ES version/OS/architecture/topology before final commands, keeps Elasticsearch security/TLS enabled, separates the DeerFlow sandbox from the target ES hosts, and makes read-only evidence collection precede changes. `scripts/collect_diagnostics.sh` gathers Linux/service/log and authenticated cluster evidence without mutations or `--insecure`; credentials come from operator-owned netrc/header files rather than command arguments or an unrestricted curl config. `scripts/analyze_diagnostics.py` is a stdlib-only heuristic that emits matched evidence and a next read-only check, or `insufficient_evidence`; it never claims an automatic root cause. Tests: `tests/test_elasticsearch_offline_ops_seed.py`, `tests/test_seed_assets.py`.
**Strategy-selection functional agents.** `app/gateway/routers/_strategy_selection_seed.py` seeds nine stable folder-backed agents from `_strategy_selection_seed_assets/` before `_sync_legacy_agents`: intent, governance/diplomacy/economy/security/public-opinion/law-policy angles, judge, and solution editor. Seeding is missing-directory-only and must never overwrite administrator changes. The global `strategy_agent_roles` registry classifies them independently of projects: intent/judge/solution are fixed, unique roles that may be configured but never added, disabled, reordered, or deleted; angle roles are a shared, ordered catalog that administrators may add, edit, disable (soft delete), restore, and reorder through `/api/strategy-agents`. Public `GET /api/strategy-agents` returns enabled identity/description metadata only; `include_disabled=true` or `include_config=true` is admin-only. Admin writes are `POST /api/strategy-agents/angles`, `PUT /api/strategy-agents/{id}`, and `DELETE /api/strategy-agents/{id}`; DELETE retains the folder-backed agent and role row to preserve historical references. Dynamic angles remain built-in folder-backed agents under the reserved `strategy-selection-angle-` id namespace, with `web` + `file:read`; a blank SOUL on create receives a safe role template. All strategy skill writes validate that names exist, are active, and are globally visible; generic agent CRUD cannot create or hard-delete reserved strategy IDs, and parallel execution rejects disabled roles. Angle configs reuse `policy-knowledge`; intent reuses `intent-clarification-policy`; the judge has no retrieval skill; the solution editor gets only `file:write`. Keep the SOUL contracts neutral and evidence-aware, with explicit de-escalation/civilian-protection framing and hard bans on operational combat guidance and targeted political manipulation. Tests: `tests/test_strategy_selection_seed.py`, `tests/test_strategy_agent_management.py`.
Only `strategy-selection-intent` has a clarification reply surface in the strategy workbench. `parallel_agents` must exclude `ask_clarification` for every other `strategy-selection-*` role and inject a direct-delivery guard: angle, judge, and solution stages use conservative assumptions, lower confidence/score where needed, list gaps under data limitations, and still finish in one pass. The judge brief repeats this rule at runtime because existing folder-backed agent SOUL files may be administrator-customized and are intentionally never overwritten. If a legacy backend nevertheless emits a judge clarification, the frontend treats it as a bounded repair attempt rather than asking the user to repeat the same manual run. Malformed or incomplete evaluation JSON receives the same bounded repair path; prior attempts are replaced in the live panel, never concatenated into duplicate final content.
`POST /api/parallel-agents/run/stream` must remain observable during silent skill calls. Each running seat emits an `agent_progress` heartbeat every five seconds with the latest translated tool/reasoning/body phase and monotonic elapsed seconds; an upstream iterator that produces no frame for 180 seconds is closed and that seat emits `agent_error`, while the other seats continue. Set the terminal flag before queuing `agent_done` / clarification so a late heartbeat cannot reopen a completed card. The strategy workbench renders the intent Q&A history, readable skill execution steps and live heartbeat on the affected agent card; judge and synthesis traces use the same step translation. Tests: `tests/test_roundtable_model_fallback.py` and frontend `src/strategy-selection/lib.test.ts`.
**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
@@ -470,14 +839,14 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
| **Threads search** (`POST /threads/search`, LangGraph-compat) | Conversation listing. **System threads are excluded by default**: rows whose `metadata.system` is truthy (scheduler runs, roundtable workers) or whose `thread_type` is a known system type (`task-analysis` seat runs included; older seat rows are also matched by `task_analysis_seat`) never crowd the user's list; a caller whose `metadata` filter explicitly references `thread_type`/`system` (e.g. studio notebooks) opts out of the exclusion. New `query` field = case-insensitive title (`display_name`) substring search, used by the chats page's server-side pagination. `POST /threads/count` takes the same request body (limit/offset ignored) and returns `{total}` under identical filtering/exclusion semantics ? powers the chats page's ??/??? display without fetching rows (`ThreadMetaStore.count`). `ThreadMetaStore.search` gained `exclude_system`/`query` params; JSON-blob filters batch-scan until the requested `limit`/`offset` window is full (so pagination over a heavily-filtered set never under-fills), with a `thread_id` tie-breaker on the `updated_at DESC` ordering for stable OFFSET paging. Tests: `tests/test_thread_meta_search.py` |
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`, `/api/strategy-project-artifacts/{id}/download`) | `GET /{path}` streams thread artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `GET /api/threads/{id}/artifact-preview?path=` returns bounded text, using streaming statistics plus deterministic head/middle/tail samples above the database threshold. `GET /api/artifact-library` combines every current-user normal thread's authoritative `outputs/` files with completed strategy-selection syntheses projected from the owner-scoped `parallel_agent_tasks.messages` snapshot. Strategy items use `source_type=strategy`, carry `strategy_project_id` / `strategy_name` / `strategy_stale`, and explicitly distinguish `strategy_source_kind=strategy_selection` from an `analysis_project_mode=strategy` thread (`analysis_project`); virtual items also expose `download_url`, so clients never infer routing from sentinel thread/path values. Search/source filters cover both. `workflowStale=true` keeps the last completed historical synthesis visible and downloadable. The dedicated strategy download route revalidates current-user ownership and snapshot completion and serves Markdown without pretending the virtual item is a thread artifact. Uploads and image/audio/video media remain excluded. Library metadata is persisted in `artifact_library_items`; list/search/filter/count/pagination read only this user-scoped SQL index. A generation-fenced `artifact_library_sync_states` lease schedules low-concurrency filesystem/task reconciliation in the background on first/stale reads or `refresh=true`, keeps the last successful snapshot visible during refresh/failure, and reports `sync_status` / `syncing` / `indexed_at` / `sync_error` to the client. Files themselves remain authoritative on disk and are never copied into the database. |
| **Word export** (`POST /api/writing/export/docx`) | Shared Markdown → DOCX generator used by conversation, artifact, scheduled-task, Deep Research, and AI-writing download surfaces. `app/gateway/word_export.py` builds OOXML with no Python conversion dependency: A4 portrait, 3.7/3.5/2.8/2.6 cm margins, real multilevel heading numbering (`一、` / `(一)` / `1.`), two-character first-line indent on heading levels 1–3, formal body/table styles, page numbers beginning at 1 with odd pages left-aligned and even pages right-aligned, and embedded `app/gateway/assets/fonts/方正小标宋简体.ttf`. Before numbering it removes complete manual heading prefixes (including `2.1` / `2.1.1`) and drops Markdown horizontal rules from Word output. The ordinary-Q&A Lead Agent can emit the matching Markdown heading forms and avoid unordered bullet lists when the admin enables `prompt_prefix.ordinary_qa_markdown_format_enabled` in `system_settings.json`; the switch defaults off, while writing setup, writing mode, notebook, scheduled runs, and SOUL-driven custom agents keep their independent prompts. The font loader validates its internal family, SHA-256, and OpenType embedding permission (`OS/2.fsType`) before export. Tests: `tests/test_word_export.py`, `tests/test_es_query_routing_prompt.py`. |
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. When `custom_outline` is empty (the workbench default — the user did not pick a report structure), chat collection first calls `POST /sessions/{id}/intent-outline` after the collector turn ends: the chosen model plans a Markdown skeleton from the topic plus collected materials, the outline is frozen into the session snapshot and shown in the「开始撰写」form for editing. If that step is skipped or fails, `basic`/`quick` still call `generate_intent_outline` at write time, emit `plan_created` (`source: intent`), then write through the same shared writer; generation failure is a recoverable `intent_outline_fallback` warning and does not abort the report. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. The compact writer validates generated Markdown against that frozen selection: `fixed` requires every non-placeholder heading with the same level and order, while `adaptive` retries only when no core template heading survives. That correction shares the single existing retry budget (never more than two writer calls), and its deterministic fallback is outline-shaped as well. When `custom_outline` is empty (the workbench default — the user did not pick a report structure), chat collection first calls `POST /sessions/{id}/intent-outline` after the collector turn ends: the chosen model plans a Markdown skeleton from the topic plus collected materials, the outline is frozen into the session snapshot and shown in the「开始撰写」form for editing. If that step is skipped or fails, `basic`/`quick` still call `generate_intent_outline` at write time, emit `plan_created` (`source: intent`), then write through the same shared writer; generation failure is a recoverable `intent_outline_fallback` warning and does not abort the report. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
| **Gateway auth** | Owner-scoped Deep Research routes await the platform's asynchronous `get_current_user` resolver before they read or write persistence, ensuring each session and job operation receives a concrete user ID. Deep Research artifacts are always resolved from a whitelisted filename via `/mnt/user-data/outputs/<name>` before mapping to a host path, preserving the sandbox guard on native Windows and Linux hosts. |
| **Suggestions** (`/api/threads/{id}/suggestions`) | `POST /` - generate follow-up questions; rich list/block model content is normalized before JSON parsing |
| **Thread Runs** (`/api/threads/{id}/runs`) | `POST /` - create background run; `POST /stream` - create + SSE stream; `POST /wait` - create + block; `GET /` - list runs; `GET /{rid}` - run details; `POST /{rid}/cancel` - cancel; `GET /{rid}/join` - join SSE; `GET /{rid}/messages` - paginated messages `{data, has_more}`; `GET /{rid}/events` - full event stream; `GET /../messages` - thread messages with feedback; `GET /../token-usage` - aggregate tokens |
| **Thread Runs** (`/api/threads/{id}/runs`) | `POST /` - create background run; `POST /stream` - create + SSE stream; `POST /wait` - create + block; `GET /` - list runs; `GET /{rid}` - run details; `POST /{rid}/cancel` - cancel; `GET /{rid}/join` - join SSE; `GET /{rid}/messages` - paginated messages `{data, has_more}`; `GET /{rid}/events` - full event stream; `GET /../run-messages` - cross-run event-store messages with feedback. The separate `GET /api/threads/{id}/messages` contract remains the chronological latest-checkpoint transcript; `GET /../token-usage` - aggregate tokens |
| **Answer feedback** (`/api/threads/{id}/messages/{mid}/feedback`) | Message-scoped `PUT`/`DELETE`, `GET /feedback-context`, and batched `POST /api/threads/{id}/feedback-contexts`; persists excellent/poor ratings, six stable poor-reason codes and optional detail idempotently per user/thread/message/agent. Legacy run-scoped feedback routes remain compatible. Admin-only `GET /api/agents/{agent_id}/feedback-score` uses the project-wide `system_role == "admin"` rule (and preserves auth-disabled mode) and returns a Bayesian user-feedback answer-quality score, raw task-success rate, weighted response-quality dimensions and reason counts; it is explicitly not a complete agent-capability evaluation. |
| **Runs** (`/api/runs`) | `POST /stream` - stateless run + SSE; `POST /wait` - stateless run + block; `GET /{rid}/messages` - paginated messages by run_id `{data, has_more}` (cursor: `after_seq`/`before_seq`); `GET /{rid}/feedback` - list feedback by run_id |
| **Open Chat** (`/api/open/chat`) | **Unauthenticated** open Q&A entry for external systems (the `/api/open/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES` + CSRF-exempt in `csrf_middleware.should_check_csrf`). `POST /chat` `{message \| messages[], model_name?, agent_name?, agent_id?, thinking_enabled=false, strip_think=true, thread_id?, recursion_limit=100}` → resolves the agent by **Chinese display name** (`agent_name`, matched against each `.deer-flow/agents/*/config.yaml` `name` via `_resolve_agent_id`; explicit `agent_id` wins if given), builds the lead agent via `make_lead_agent` (so the chosen **agent's** SOUL/tools/skills all apply), `ainvoke`s it once (stateless — no checkpointer, no thread persistence; runs under the `"default"` user bucket), and returns **only the final** `{answer, model_name, agent_name, agent_id, thinking_enabled, thread_id}` (no intermediate tool-call/process). **`POST /chat/stream`** is the **SSE streaming variant** (same request body) for **long tasks** — the blocking `/chat` produces no downstream bytes for the whole run, so a front nginx `proxy_read_timeout` (default 60s) cuts it off with a **504**; the stream keeps the connection warm (a `: keepalive` SSE comment every `_HEARTBEAT_SECONDS=15`s via a background producer-task + queue, plus `event: delta` AI-text increments and `event: progress`), and ends with **`event: result`** carrying the same final `{answer, …}` JSON (or `event: error`). Clients only need to read `event: result`; intermediate events are ignorable. Both endpoints share `_build_run_context` (config/agent/thread). `GET /agents` lists `{name, id}` so callers know what `agent_name` to send. **Thinking off is reliable**: when `thinking_enabled=false` the route also sets `thinking_force_disabled=true` so the lead agent calls `create_chat_model(force_disable_thinking=True)`, which injects **three** OpenAI-compatible "thinking off" shapes into `extra_body` — top-level `enable_thinking=false` (**SiliconFlow / DashScope hybrid models** like `deepseek-ai/DeepSeek-V4-*` on `api.siliconflow.cn` — the knob the nested `chat_template_kwargs` shape does NOT reach), `chat_template_kwargs.enable_thinking=false` (vLLM/Qwen native), and `thinking.type="disabled"` (Zhipu GLM) — even for models that only declare `supports_thinking: true`. For the **web-UI path** (which sends `thinking_enabled=false` but NOT `force_disable`), SiliconFlow models need `when_thinking_disabled: {extra_body: {enable_thinking: false}}` in their `config.yaml` entry (added to `deepseek-ai/DeepSeek-V4-Flash`) for thinking-off to take effect. Sets `is_scheduled_run=true` internally to skip the auto-title LLM call. **Generated-file return (`include_files=true`, default on)**: an agent that produces its report as a file (writes to `/mnt/user-data/outputs/` + `present_files`) returns only a short summary in `answer`; to recover the real deliverable both endpoints now also return `files: [{name, virtual_path, encoding(text|base64), content}]` — `_collect_output_files` scans the run's `state['thread_data']['outputs_path']` (the open thread is stateless so artifacts can't be fetched by thread, but the endpoint runs server-side and the sandbox already wrote the files to disk), text decoded as utf-8 else base64, capped by `_MAX_OUTPUT_FILES=20` / `_MAX_OUTPUT_FILE_BYTES=5MB`. `/chat` puts `files` on the response; `/chat/stream` puts it in the `event: result` payload. A run with no answer **and** no files now 502s ("…文本答案或文件"). Router `app/gateway/routers/open_chat.py`; standalone caller `test_open_chat_api.py` (repo root, streaming by default); tests `tests/test_open_chat.py`. |
| **Open 3qfx Ask** (`/api/open/3qfx/ask`) | **Unauthenticated** open entry that, given just a **任务id**, directly kicks off the 3qfx(三情分析)「第一次问答」as a **real persisted conversation** (sibling of `/api/open/chat`; same `/api/open/` auth-whitelist + CSRF-exempt). `POST /ask` `{task_id, message?, model_name?, thinking_enabled=false}` → (1) resolves the `3Q` agent from `app.state.business_mapping_store.get_mapping("3Q")` (the **same** agent the frontend deep-link `goPath=3qfx` uses; 503 if unconfigured); (2) **fetches the task detail server-side** (consumer `cop-task-detail`) and builds the opening text **identical to the frontend** `buildOpeningText` 3qfx branch — `任务名称:…\n任务内容:…\n任务id(task_id)为{task_id}` (an explicit `message` overrides and **skips** the fetch); (3) creates a fresh thread tagged `metadata.{taskId, agent_id}` (the exact filter `useTaskThreads` uses, and `metadata.taskId` makes it task-deeplink-shared / creator-path-resolved) and **starts a background persisted run** via `start_run(..., enforce_access=False)` (writes the checkpointer, runs under the `"default"` bucket); (4) **returns immediately** `{thread_id, run_id, agent_id, status, opening_text}` — the caller just needs "started", then watches the run in the 3qfx 任务工作区 conversation list. `start_run` gained a keyword-only `enforce_access` (default `True`); the open entry passes `False` so the admin-configured agent isn't rejected by the per-(missing)-login visibility check. **Consumer fetch config** lives in `config.yaml → task_deeplink`: `cop_task_detail_url` (full URL, appends `?taskId=`), `cop_task_detail_authorization` (default `Bearer admin`), `cop_task_detail_test` + `mock.cop_task_detail` (offline fake when the upstream is unreachable — on in this deployment). Router `app/gateway/routers/open_3qfx.py`; tests `tests/test_open_3qfx.py`. |
| **Open 3qfx Ask** (`/api/open/3qfx/ask`) | **Unauthenticated** open entry that, given just a **任务id**, directly kicks off the 3qfx(三情分析)「第一次问答」as a **real persisted conversation** (sibling of `/api/open/chat`; same `/api/open/` auth-whitelist + CSRF-exempt). `POST /ask` `{task_id, message?, model_name?, thinking_enabled=false}` → (1) resolves the `3Q` agent from `app.state.business_mapping_store.get_mapping("3Q")` (the **same** agent the frontend deep-link `goPath=3qfx` uses; 503 if unconfigured); (2) **fetches the task detail server-side** (consumer `cop-task-detail`) and builds the opening text **identical to the frontend** `buildOpeningText` 3qfx branch — `任务名称:…\n任务内容:…\n任务类型:…\n任务id(task_id)为{task_id}`(`taskType` 1/2/3 → 打击/塑势/防御,无法识别则省略) (an explicit `message` overrides and **skips** the fetch); (3) creates a fresh thread tagged `metadata.{taskId, agent_id}` (the exact filter `useTaskThreads` uses, and `metadata.taskId` makes it task-deeplink-shared / creator-path-resolved) and **starts a background persisted run** via `start_run(..., enforce_access=False)` (writes the checkpointer, runs under the `"default"` bucket); (4) **returns immediately** `{thread_id, run_id, agent_id, status, opening_text}` — the caller just needs "started", then watches the run in the 3qfx 任务工作区 conversation list. `start_run` gained a keyword-only `enforce_access` (default `True`); the open entry passes `False` so the admin-configured agent isn't rejected by the per-(missing)-login visibility check. **Consumer fetch config** lives in `config.yaml → task_deeplink`: `cop_task_detail_url` (full URL, appends `?taskId=`), `cop_task_detail_authorization` (default `Bearer admin`), `cop_task_detail_test` + `mock.cop_task_detail` (offline fake when the upstream is unreachable — on in this deployment). Router `app/gateway/routers/open_3qfx.py`; tests `tests/test_open_3qfx.py`. |
| **Thread Shares** (`/api/thread-shares`) | Per-conversation sharing with two modes (`thread_shares.mode`). **`import`**: `POST /` `{thread_id, mode}` - create (or reuse) a revocable share code; `GET /{code}/preview` + `POST /{code}/import` - copy that conversation into the caller's account (fresh `thread_id` via `app/gateway/thread_copy.py` + new `threads_meta` row + `user-data` copy; dedup on `metadata.shared_origin`). **`view`**: a public, read-only static page ? see Public Shares below. Both modes: `GET /` - list my codes; `DELETE /{code}` - revoke. |
| **Public Shares** (`/api/public/shares`) | **Unauthenticated** (the `/api/public/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES`). `GET /{code}` - read-only conversation snapshot (title + serialized messages + artifacts) for a `view`-mode share; messages are read straight from the latest checkpoint. `GET /{code}/files/{path}` - serves a file referenced by that conversation (images, Markdown, text, PDF, ?) so the shared page's links are openable; path resolution is confined to the owner's thread directory. Active content (`text/html`, `application/xhtml+xml`, `image/svg+xml`) is forced to download (`_public_share_disposition`); everything else is served inline. Missing/revoked/non-`view` codes all return 404. |
| **Roundtable Drafts** (`/api/roundtable-drafts`) | **Strictly per-user** persistence for the roundtable-planning page (replaces the old browser `localStorage`). `GET /` - list drafts (lightweight meta — incl. `chain_title`, the business-chain name used this session, cheaply regex-extracted from the in-memory `step2` blob in `_draft_to_meta`/`_extract_chain_title`; null = no chain / 自由模式; surfaced on the 第一步「分析记录」history card + sidebar list instead of the old「第 X 步」); `POST /` - create; `GET/PUT/DELETE /{id}` - full read / partial update / delete (all owner-scoped). A draft = one full session (Step 1 intent + Step 2 roundtable), step snapshots stored as JSON in `PortableLongText`. Recommendation history bound to a draft: `GET /{id}/recommendations` - list newest-first; `POST /{id}/recommendations` - append one run (objective/status/rationale/picks/candidates). taskId 深链会商记录 are **not** here — they live in the separate `/api/roundtable-task-drafts` store (see below) so personal history never mixes with task records. Backed by `deerflow.persistence.roundtable_drafts` (tables `roundtable_drafts`, `roundtable_recommend_history`); store wired in `deps.py` as `app.state.roundtable_draft_store`. Tests: `tests/test_roundtable_task_drafts.py`. |
@@ -544,6 +913,8 @@ The intranet client uses `trust_env=False` and defaults `intranet_verify_ssl=fal
The admin ????? is served from **precomputed per-day snapshots** in `admin_leaderboard_daily_stats` (keyed by `(stat_date, settings_hash)`), not by aggregating raw rows on every request. A background scheduler inside the Gateway (`_leaderboard_snapshot_scheduler_loop` in `app/gateway/app.py`, singleton file-locked) ticks every `ADMIN_LEADERBOARD_BACKGROUND_INTERVAL_SECONDS` (default 300s): it prewarms recent days, then builds any `queued`/`error`/stale-`running` day. `GET /leaderboard` merges the per-day snapshots for the requested range. **Today** is rebuilt whenever its snapshot is older than `ADMIN_LEADERBOARD_TODAY_TTL_SECONDS` (default 43200s = half a day, so the leaderboard refreshes ~twice daily); a past day, once `ready`, is treated as fresh forever.
**Recent-only admin audit policy.** The server enforces a rolling **30-calendar-day** window for administrator leaderboard, user analytics, user Q&A/thread audit, LLM/tool metric list/summary/export, and the user-list Q&A counter; URL parameters cannot widen it. The Q&A fallback that reads checkpoints is additionally capped, while indexed run summaries remain the primary audit source. Gateway startup also launches the singleton `admin_audit_retention` sweep: it deletes audit metric rows, terminal `runs` summaries (which contain compact question/answer audit fields and were also last updated before the cutoff), and leaderboard snapshots older than **90 days**. Cleanup is deliberately low-priority: after the default 5-minute startup delay, it runs every 15 minutes using 100-row committed transactions, round-robins each store (at most four batches per store), yields between commits, and stops starting new batches after an 8-second sweep budget; operators may tune these bounds with `ADMIN_AUDIT_RETENTION_*` environment variables. It deliberately does **not** delete conversation threads, checkpoints, or message history; those are user-content records governed by their own lifecycle.
Two on-demand/automatic refresh paths layer on top:
- **Manual "????"** ? `POST /leaderboard/refresh-today` force-rebuilds *today's* snapshot **synchronously** (bounded by the same semaphore/timeout the scheduler uses), then returns the merged range. Lets an admin pull the latest real-time records into the stats without waiting for the TTL. Frontend: a "????" button on `AdminUserLeaderboardPage` ? `useRefreshTodayLeaderboard` (`core/admin/hooks.ts`), which primes + invalidates the `["admin-activity","leaderboard"]` query.
- **Daily finalize of the previous day** ? once per day, on/after `ADMIN_LEADERBOARD_FINALIZE_HOUR` (local Beijing, default `1` = ?? 1 ?), the scheduler tick force-recomputes *yesterday* a single time so runs/tool-calls that only landed near midnight are fully captured. The decision is the pure `deerflow.persistence.admin_stats.finalize.should_finalize_previous_day` (status/generated_at/now/hour) ? the rebuild stamps `generated_at` to today, which self-debounces it for the rest of the day (no extra state file). Tests: `tests/test_leaderboard_finalize.py`.
@@ -652,7 +1023,7 @@ The fixed-deep-blue wx full-chat route (`#/page/workspace/wx/chats/new`) hides t
### Business-mapping authorization
`/api/business-mapping` remains globally readable. Seeded rows include the main codes (3Q / 6BF / 7BF / 8BF / AFX) plus independent slots `3q-0…3q-6`, `6-1…6-4`, and `7-1…7-4` (empty agent/chain until configured on the 配置管理 page). Updates to 3Q, 6BF, 7BF, and those independent slots require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.
`/api/business-mapping` remains globally readable. Seeded rows include the main codes (3Q / 6BF / 7BF / 8BF / AFX) plus independent slots `3q-0…3q-6`, `6-1…6-4`, and `7-1…7-4` (empty agent/chain until configured on the 配置管理 page). `3q-1…3q-6` seed as `is_custom=true` / `mapping_type=自定义类型` so `goPath=3q-1` resolves as a custom deep link; the configuration page hides those six rows and defaults the type filter to 自定义类型. Updates to 3Q, 6BF, 7BF, and those independent slots require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.
### Position-collaboration foundation
@@ -855,7 +1226,20 @@ the catalogue; a command or read that names a skill reconciles only that skill
and any support library it declares. Skill-management file saves, deletions,
online/offline changes, skill deletion, and skill-address apply/rollback sync
that skill into the view immediately and publish the new extensions hash, so the
next command stays on the narrow path. A missing view or an extensions hash the
next command stays on the narrow path. Agent `skill_manage` create, edit, patch,
and delete do the same for the sandbox view and, when the enabled-skills prompt
cache is already warm, replace only that skill in the cache. A cold prompt cache
still reloads the full catalogue. Creating a skill stats `custom/<name>/SKILL.md`
instead of parsing every skill. If the model omits `content`, create reads a
`write_file` path under the thread workspace instead of failing the turn.
`write_file` and `remove_file` sync that skill's view and leave the prompt cache unchanged.
Creating an agent calls `skill_list` / `search_skills`; when this cache is warm those
tools reuse it instead of parsing every SKILL.md again, and `skill_view` reads only
the named skill. A cold cache still loads the catalogue, then checks the canonical
directory before a full walk.
A repeated-tool warning is appended to the assistant tool-call message so the
tool result still follows it; a user message between them makes the next model
request fail. A missing view or an extensions hash the
sync did not publish still falls back to the full rebuild. Direct edits outside
those writes stay stale until the next full rebuild. Offline skills are removed
from the view at disable time, which is what keeps a hidden shell path from
@@ -970,7 +1354,10 @@ pool changes; model timing instances also need restart for consistent behavior.
source runtime under `skills/public/cognitive-situation-map/assets/runtime/`
and a Gateway-served copy under
`packages/harness/deerflow/html_runtime/assets/cognitive-situation-map/`.
Keep their JavaScript, schema, and map atlas assets in sync. Built-in regions
Keep their JavaScript, schema, and map atlas assets in sync. `template.aspect`
locks a new page to `16:9` (1920×1080, the default when unspecified) or
`32:9` (2560×720, explicit ultra-wide only); pages without the field keep
the older viewport-adaptive layout. Built-in regions
include Taiwan Strait, South China Sea, the Philippines-centred surrounding
region (with its southern Sulu/Celebes approaches), First Island Chain, and
the U.S.--Pacific connection; every region is a four-quadrant atlas for the
@@ -983,7 +1370,7 @@ pool changes; model timing instances also need restart for consistent behavior.
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools). `name` and `description` are the minimal metadata contract; the frontend install dialog renders the shared `MINIMAL_SKILL_MD_EXAMPLE` containing only those two keys so optional fields are not mistaken for requirements. Keep `frontend-web/src/core/skill-example.test.ts` aligned with backend validation.
- **Loading**: `load_skills()` recursively scans `skills/{public,custom}` for `SKILL.md`, parses metadata, and reads enabled state from extensions_config.json. **A directory containing `SKILL.md` is a self-contained leaf** — discovery does not descend into its subdirectories, so a skill *package* that bundles its own `examples/`/`references/` sub-skills (each with a SKILL.md) surfaces as **one** skill, not a swarm of phantom entries.
- **Built-in seed source**: Product-owned defaults live under `app/gateway/routers/_*_seed_assets/skills/` and must be versioned even though runtime `skills/` trees are ignored. Seeders copy only when a target directory is absent and never overwrite administrator changes. Internal support packages such as `task-report-lib` deliberately omit `SKILL.md` and declare dependents in `.deerflow-skill-support.json`; keep `tests/test_seed_assets.py` and `tests/test_task_report_convert.py` synchronized with the shipped inventory and contracts.
- **Online/offline lifecycle**: reuse `Skill.enabled`; do not add a parallel lifecycle flag and do not substitute `published` or curator `state`. `PUT /api/skills/{name}` is a global switch: administrators may toggle every skill, while an ordinary user may toggle only a non-built-in `custom` skill whose persisted `owner_user_id` matches that user. It patches only `skills.<name>.enabled` in the raw `extensions_config.json`, preserving MCP placeholders and unrelated/unknown fields, then reloads configuration and refreshes the lead prompt cache. Writes use atomic replacement on ordinary files and an `fsync` in-place fallback for writable Docker single-file bind mounts; supported launchers instead keep the runtime file under a writable persisted config directory. Offline skills remain on disk and in the full catalogue (`load_skills(enabled_only=False)`), and existing agent/workflow references are deliberately retained so bringing the skill online restores them. Every model-facing path must use the enabled catalogue: prompt injection, `search_skills`, `skill_list`, `skill_view`, subagents, AI-writing/deep-research/roundtable discovery, and Workflow run-start resource validation. Direct sandbox access is enforced by a copied online-only view under writable `DEER_FLOW_HOME`; `/mnt/skills` in LocalSandbox and AIO/remote sandboxes must point only to that view, never to the canonical tree. AIO creation/reuse attests the view marker and fails closed when an older provisioner ignores the requested mount; explicitly declared internal support libraries (such as `task-report-lib`) enter the view only while a dependent skill is online. Path-sensitive `DEER_FLOW_*` variables and `PYTHONPATH` are removed from LocalSandbox bash while non-path settings such as `DEER_FLOW_GATEWAY_PORT` remain. LocalSandbox host bash still shares the backend UID and is not an OS isolation boundary; production requiring hostile-code isolation must use an attested AIO sandbox. Frontend controls are in the administrator 技能管理 list and, for owners, 「技能 → 技能列表 → 我的」; offline rows and retained agent references remain visible with an 已下线 badge and can be removed, while new selectors exclude them.
- **Online/offline lifecycle**: reuse `Skill.enabled`; do not add a parallel lifecycle flag and do not substitute `published` or curator `state`. `PUT /api/skills/{name}` is a global switch: administrators may toggle every skill, while an ordinary user may toggle only a non-built-in `custom` skill whose persisted `owner_user_id` matches that user. It patches only `skills.<name>.enabled` in the raw `extensions_config.json`, preserving MCP placeholders and unrelated/unknown fields, then reloads configuration and refreshes the lead prompt cache. Writes use atomic replacement on ordinary files and an `fsync` in-place fallback for writable Docker single-file bind mounts; supported launchers instead keep the runtime file under a writable persisted config directory. Offline skills remain on disk and in the full catalogue (`load_skills(enabled_only=False)`), and existing agent/workflow references are deliberately retained so bringing the skill online restores them. The per-skill incremental view sync must treat a missing destination as stale even when its target-cache entry is also absent, so re-enabling recopies the latest canonical package (including `SKILL.md` and nested helpers edited while offline) after the published copy was removed. Every model-facing path must use the enabled catalogue: prompt injection, `search_skills`, `skill_list`, `skill_view`, subagents, AI-writing/deep-research/roundtable discovery, and Workflow run-start resource validation. Direct sandbox access is enforced by a copied online-only view under writable `DEER_FLOW_HOME`; `/mnt/skills` in LocalSandbox and AIO/remote sandboxes must point only to that view, never to the canonical tree. AIO creation/reuse attests the view marker and fails closed when an older provisioner ignores the requested mount; explicitly declared internal support libraries (such as `task-report-lib`) enter the view only while a dependent skill is online. Path-sensitive `DEER_FLOW_*` variables and `PYTHONPATH` are removed from LocalSandbox bash while non-path settings such as `DEER_FLOW_GATEWAY_PORT` remain. LocalSandbox host bash still shares the backend UID and is not an OS isolation boundary; production requiring hostile-code isolation must use an attested AIO sandbox. Frontend controls are in the administrator 技能管理 list and, for owners, 「技能 → 技能列表 → 我的」; offline rows and retained agent references remain visible with an 已下线 badge and can be removed, while new selectors exclude them.
- **Discovery↔delete consistency**: a skill is keyed for discovery by its frontmatter `name`, but its on-disk directory name may differ (hand-dropped / mis-extracted package). `LocalSkillStorage._resolve_skill_dir(name, category)` resolves the *discovered* directory (canonical `<category>/<name>` fast-path, else scan-by-frontmatter-name); `custom_skill_exists`/`read_custom_skill`/`delete_custom_skill` all go through it, so **anything that gets listed can also be read and deleted** regardless of directory name (the old code assumed `custom/<name>/` and 404'd "Custom skill 'x' not found." on any non-canonical layout). Delete is path-guarded to never escape the `custom/` root. Tests: `tests/test_skill_storage_delete.py`.
- **Injection**: Enabled skills listed in agent system prompt with container paths
- **Configurable `es_query` routing prompt**: `config.yaml → skills.es_query_routing`
@@ -1225,10 +1612,11 @@ scheduler runs once per deployment. Per run:
会商第二步(leader/seat)+ 第三步(summary/report/dashboard + 结构化抽取)单轮执行时,**某模型出问题就自动切到下一个模型继续**,并告知用户「『X』模型异常 → 已切换『Y』」。前台实时(`multi_agent.py`)与后台挂起(`engine.py` + `InProcessRoundtableGateway`)两条路径都覆盖。
- **可检测的模型失败**:`LLMErrorHandlingMiddleware` 在同一模型重试 3 次后**不抛异常**,而是返回兜底 `AIMessage` —— 现在给它打 `additional_kwargs[LLM_ERROR_MARKER="deerflow_llm_error"]=reason`(model_dump 保留,经 checkpointer 序列化仍在)。`is_llm_error_message(msg)` 据标记(回退按已知兜底文案正文)识别,返回 reason。
- **候选模型链**:`app/gateway/roundtable_model_fallback.py::fallback_model_chain(current)` 按 `config.yaml models[]` 顺序给出 `[当前模型, …其余]`(去重保序)。`reason_text(reason)` 翻译成简短中文。总开关 env `ROUNDTABLE_MODEL_FALLBACK`(默认开,`0/false/off` 关 → 单模型旧行为);`ROUNDTABLE_MODEL_FALLBACK_MAX`(0=试完全部)。
- **触发换模型的情形**:① 单轮 run 抛异常(start_run 失败 / idle 卡死 → `timeout`,其余 `exception`);② 兜底错误文案(`is_llm_error_message`);③ 交付无效(仅 seat/leader;leader 还含「既无正文又无派活」)。席位**只认剥掉 `<think>` 后的可见正文**——思考-only / 空串 / 断流都不算已交付(`deerflow.agents.roundtable_orchestrator.delivery`:`visible_seat_text` / `classify_seat_delivery`)。前台 `_special_run` 对此 yield `status:"invalid_delivery"` 且不广播;`GET /last-ai-message` 回 `{delivered, content, length, invalid_reason}`(思考-only 标 `thinking_only`,总控席位清单标 ⚠️)。后台引擎只把可见正文写入 `deliveries`,续轮用 `build_continue_prompt`:有无效席位时强制重派,禁止声称「已完成交付」。Step3 单例(report/summary/dashboard/ingest/结构化)的产物是落盘文件,**不**把「正文为空」当失败,只对真模型错误换模型。
- **候选模型链**:保留真正模型故障后的自动换模,但“仅有思考 / 暂时无正文 / 空交付”不再判作模型故障,绝不据此切换用户选择的模型。`app/gateway/roundtable_model_fallback.py::fallback_model_chain(current)` 按 `config.yaml models[]` 顺序给出 `[当前模型, …其余]`(去重保序)。`reason_text(reason)` 翻译成简短中文。总开关 env `ROUNDTABLE_MODEL_FALLBACK`(默认开,`0/false/off` 关);`ROUNDTABLE_MODEL_FALLBACK_MAX`(0=试完全部)。
- **触发换模型的情形**:① 单轮 run 抛异常(start_run 失败 / idle 卡死 → `timeout`,其余 `exception`);② 兜底错误文案(`is_llm_error_message`)。交付无效、仅有思考、空串以及普通总控轮既无正文又无派活**均不触发换模**;唯一例外是无工具的最终收口请求已经完整结束仍无正文,此时结果客观不可交付,才继续自动换下一个模型。席位仍只认剥掉 `<think>` 后的可见正文(`deerflow.agents.roundtable_orchestrator.delivery`:`visible_seat_text` / `classify_seat_delivery`):前台 `_special_run` 对无效交付 yield `status:"invalid_delivery"` 且不广播;`GET /last-ai-message` 回 `{delivered, content, length, invalid_reason}`(思考-only 标 `thinking_only`,总控席位清单标 ⚠️)。后台引擎只把可见正文写入 `deliveries`,无效席位继续走既有未交付处理,不切换模型。Step3 单例(report/summary/dashboard/ingest/结构化)的产物是落盘文件,同样只对真模型错误换模型。
- **Step2 总控工具与可配置收口轮数**:普通总控使用真正白名单,只允许 `agent_orchestration` / `ask_clarification`;`write_file`、`present_files`、技能目录与文件/命令工具均不可见。`system_settings.json -> roundtable_room_limit.max_cycles`(默认 8,范围 2..100;管理员在设置弹窗「会商并发与收口」中修改)控制 recommend 模式到第几轮切换为显式 `finalize_only`:空工具白名单、注入已有有效席位全文、要求直接输出最终总结(失败席位如实标缺口),不能表现为“已停止”。一次会商启动时锁定当时值,运行中修改从下一次研讨生效;前台实时会商与后台作业读取同一设置。同一页的「会商文件流式兼容」(`write_file_compatibility_enabled`,默认关) 只作用于会商总控、席位、第三步和后台作业:全局文件流式兼容开着时会商一并使用;全局关着而本开关开着时,只有这些会商 run 改走 `buffered_gateway`。普通对话不读这个开关。chain / dag 的最终总结仍在既定席位完成后走同一纯正文路径;LangGraph 单轮 recursion limit 仍为 1000。
- **后台**:`InProcessRoundtableGateway._run_turn_with_fallback(...)` 包住 `_run_agent_turn`,逐候选模型试;换模型落 `model_switch` 诊断、全失败落 `model_fallback_exhausted`。`run_leader/run_seat/run_report/run_summary/run_dashboard_data` 全走它(report/summary/dashboard 的产物自愈重试保留,并记住已切到的可用模型用于后续续写轮)。各 Turn/Result 带回 `switches: list[ModelSwitch]`,引擎 `_emit_switches` → `JobProgressWriter.model_switch(...)` 往研讨时间线插一条 `role="system"` 系统提示气泡。
- **前台**:`_special_run`/`_leader_run` stream 收尾后检测失败 → yield `{status:"model_switch", role, agent_name, failed_model, next_model, reason}` 帧并用下一个模型重新流式;终态帧带 `model_used`。结构化抽取 `flow_extract.py` 的 round1 `model.astream` 也包了同款换模型循环(emit `event:"model_switch"` 帧)。
- **前台**:`_special_run`/`_leader_run` stream 收尾后仅检测明确的模型错误标记;命中才 yield `{status:"model_switch", role, agent_name, failed_model, next_model, reason}` 帧并用下一个模型重新流式,思考-only/空正文不进入该分支;终态帧带 `model_used`。结构化抽取 `flow_extract.py` 的 round1 `model.astream` 也包了同款换模型循环(emit `event:"model_switch"` 帧)。
- **Step2 单席位隔离**:`engine._run_recommend/_run_chain/_run_dag` 的每个席位执行包 try/except —— 一个席位彻底失败(落 `seat_failed` 诊断 + 置空交付)不拖垮整批/整链/整个作业。
- **前端**:`multi-agent.ts` 解析 `model_switch` 帧 → `onModelSwitch` 回调(清空失败尝试的半截气泡 + 累计的 tool_call delta,phase 退回 connected);`useStep2Orchestration` 弹 toast + 时间线插「系统提示」气泡;`useStep3Summary/Report/Dashboard` + `flow-extract.ts` 重置对应气泡并在步骤卡留可见提示;后台路径经 `job-live.ts::toStep2Dialogue` 把 `role="system"` 渲染成 amber「系统提示」气泡。
- Tests: `tests/test_roundtable_model_fallback.py`。
@@ -1334,7 +1722,7 @@ Per-user runtime overrides (whitelisted fields) live at `{base_dir}/users/{user_
- `subagents.enabled` - Master switch for subagent delegation
- `memory` - Memory system (enabled, storage_path, debounce_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens)
- `auth_login` - Custom login modes layered on the base auth (`deerflow/config/login_config.py`):
- `username_login.{require_password, password}` - **Shared** password gate for `POST /api/v1/auth/login/username`. When `require_password=true`, the (otherwise passwordless) username login additionally requires `?password=<shared-secret>`; empty `password` falls back to `123ewq`. Gate only ? it does not change which account logs in. **Per-account override (账号级登录口令)**: independent of this shared gate, an admin can assign **one specific account** an individual **plaintext** password via the 白名单管理 page (`users.login_password` column, migration `20260625_04`, nullable). When the target account carries a `login_password`, `/login/username` **requires** `?password=` to match it **exactly** and that personal password **takes precedence over** the shared gate (the shared secret will not unlock that account); accounts with no personal password keep the shared-gate / passwordless behavior. The check happens before auto-register via the lookup-only helper `_lookup_user_by_username` (so a brand-new username is unaffected). Surfaced/managed admin-only: `GET /api/v1/auth/users` returns `UserListItem.login_password` (plaintext — the list is already admin-gated), and `PUT /api/v1/auth/users/{user_id}/login-password` `{password: str|null}` sets (non-empty) / clears (null·empty) it without touching `token_version`. Plaintext storage is intentional (the password travels in the login URL query anyway, so it stays re-copyable). The user logs in via the existing `#/login/<username>?password=xxxx`. Tests: `tests/test_login_personal_password.py`.
- `username_login.{require_password, password}` - Shared password gate for `POST /api/v1/auth/login/username`; blank configured secret falls back to `123ewq`. An account's own password/passphrase takes precedence. **Account login, registration and SSO policy (2026-10-05):** `system_settings.json -> authentication` stores `password_login_enabled=false`, `registration_enabled=false`, `single_session_enabled=false`, `sso_enabled=true` defaults; administrator GET/PUT `/api/system-settings/authentication` edits them immediately, and anonymous `/api/v1/auth/options` exposes the switches plus `needs_setup`. At least one login method must remain enabled; registration needs password login. Missing personal credentials use the existing shared passphrase (default `123ewq`), so disabling SSO does not require an administrator to first set a personal password. `/login/local` accepts username or email; `/register` accepts `{username,password}` (or historical `{email,password}`), always creates a regular user, and returns the full bearer login response. Names containing `admin` never create or promote administrators; persisted roles remain authoritative. Passwords and URL passphrases share the same versioned hash in `users.password_hash` and `users.login_password`; legacy plaintext passphrases remain verifiable and take precedence over an old independent password hash until reset. User `/change-password` and administrator `/users/{id}/login-password` (alias `/reset-password`) update both columns and atomically increment `token_version`; stale profile writes cannot overwrite credentials or revive sessions. Password removal is rejected. `/me.has_password` denotes personal credentials; `uses_default_password` marks shared-passphrase fallback, and `requires_current_password=true` requires the current shared passphrase even for the first password change. Self changes/resets return a replacement bearer and Cookie; administrator list items return `has_login_password`, never hashes or newly written plaintext (existing legacy plaintext remains admin-only for old link-copy compatibility). New link copies use only the reset input still in frontend memory. SSO policy also protects token/OAuth and the parallel panel `/api/parallel-agents/auth/token` compatibility entry, which now shares standard password/rate-limit/approval checks. Public first-admin initialization returns a full login response and inserts an `auth_initialization` singleton claim plus the first administrator in one transaction; the ORM automatically creates that table plus `auth_usernames`, whose unique username claim prevents concurrent registrations across email domains without rewriting historical users. Username SSO rechecks the final account credential after a concurrent registration or reset. When password login is enabled, username and parallel compatibility login require a personal or shared/default passphrase regardless of the old require_password flag. All verified issuing paths explicitly set JWT auth_method=password/passphrase/sso; its default remains None for legacy/identity-only sessions. Gateway and LangGraph share session_policy_error, rejecting unmarked sessions under enabled password login and SSO sessions when SSO is disabled. DEER_FLOW_AUTH_DISABLED cannot bypass an enabled password-login or single-session policy; disabling password login preserves legacy compatibility behavior. CLI `auth.reset_admin` rotates both credential columns. Single-session login (one online location per account) is the separate single_session_enabled=false policy, not external SSO. Every successful login route, including the parallel alias, registration and initialization, calls _prepare_login_session: the SQL repository atomically increments token_version and returns that transaction's current user before minting the JWT. Ordinary competing logins advance versions independently so only the latest survives, while credential/approval changes after verification cause 409 rather than authenticating a stale password. Failed logins never advance the version; when disabled, logins keep existing sessions. Password changes still always rotate the version. Tests: `tests/test_account_authentication.py`, `tests/test_username_login_domain.py`; contracts: `docs/AUTH_LOGIN.md`.
- `token_login.{enabled, token_info_url, service_authorization, timeout_seconds, verify_ssl, ca_cert_path, mock_tokens}` - Token-exchange login (`POST /api/v1/auth/login/token`). The gateway POSTs to `token_info_url` with the caller-supplied token placed directly in the `Authorization: Bearer <token>` header (no cookie; the static `service_authorization` field is deprecated/unused, kept only for backward compat), reads `data.username` from a `{"state":"200",...}` body, then signs the matching local user in (auto-registering if absent). `token_info_url` is HTTPS ? `verify_ssl: false` skips cert verification for self-signed/internal UAT hosts, or set `ca_cert_path` to a PEM bundle (takes precedence, keeps verification on). **`mock_tokens` (offline test mapping `{token: username}`)**: when an incoming token matches a key, `_fetch_username_from_token` returns the mapped username and **skips the `token_info_url` call entirely** — for environments where the upstream `getTokenInfo` host is unreachable (e.g. external networks). Empty in prod (no effect); seeded `{"123ewq": "admin"}` in this deployment so `/login/任意?authToken=123ewq` signs in as admin. **`UsernameLoginResponse` now also returns `username`** (the raw resolved login name, pre email-normalization) on both `/login/token` and `/login/username`, so the frontend can append a faithful `?username=` on jump links (按钮管理 追加登录态) instead of the lossy email local-part. See [docs/AUTH_LOGIN.md](docs/AUTH_LOGIN.md) for the full request/response contract.
Both endpoints are whitelisted in `auth_middleware._PUBLIC_EXACT_PATHS`. The shared `_get_or_create_user_by_username()` helper (in `routers/auth.py`) backs both `/login/username` and `/login/token`.
@@ -1370,6 +1758,29 @@ from the selected space names and hides generic recommendations while these
questions are present. Design and remaining graph/profile phases are in
`docs/KNOWLEDGE_SPACE_RETRIEVAL_AND_WIKI_EXPANSION_ZH.md`.
**Knowledge-base research bookshelf:** `/page/workspace/knowledge/:id/research`
is ordinary Lead Agent Q&A constrained to one existing knowledge base; it does
not invoke or reuse the legacy Deep Research workflow. Any account with read
access may create a private session/draft, while only the existing KB writer
may publish or overwrite its Markdown Wiki article. Published snapshots follow
the KB's read permission, but the session/thread/draft and the continuation
link remain creator-only (administrators do not inherit another user's private
chat). Every run resolves the immutable thread binding from
`knowledge_research_sessions` (registered on `deerflow.persistence.models` so
startup `create_all` builds the table; `get_by_thread` treats a missing table
as no binding so ordinary `/runs/stream` is not aborted), reauthorizes KB read access, accepts exactly one
text-only human turn, forces classic single-KB retrieval and disables tools,
memory, prompt prefixes, planning and subagents; fixed-answer shortcuts are
bypassed. Saving manually upserts a stable
`knowledge-research/{publication_id}` Wiki slug under the reserved root
「深度研究」 without enabling automatic Wiki generation, uses a saved hash for
optimistic overwrite checks and idempotent retries, and keeps the editable
publication title separate from the Q&A title. REST endpoints live in
`app/gateway/routers/llmwiki.py`, runtime enforcement in
`app/gateway/knowledge_research.py`, persistence/migration in
`deerflow.persistence.knowledge_research` / `20261022_01`, and regression
coverage in `tests/test_knowledge_research.py`.
**Embedding ownership:** WeKnora's per-knowledge-base `embedding_model_id`
indexes uploaded source documents; `llmwiki.local_wiki_index.embedding` indexes
the generated Wiki-page mirror inside DeerFlow. Do not couple these two model
@@ -1379,15 +1790,33 @@ the selected active model id. When the caller omits a choice, retain WeKnora's
active-default behavior. Existing KBs cannot change this binding in place, and
cross-KB WeKnora searches should use model-compatible knowledge bases.
Wiki-enabled knowledge bases accept two bounded, user-owned prompt extensions:
`wiki_config.content_instructions` (generated-page language, structure and
emphasis) and `wiki_config.extraction_instructions` (domain entities/concepts
to prioritize). The create/edit dialogs expose both values; Gateway accepts
only those two fields and `WeKnoraClient` merges edits into the remote
`wiki_config` so `synthesis_model_id`, extraction granularity and WeKnora-owned
runtime knobs are never dropped. WeKnora's upload/reparse Wiki ingest is the
actual consumer: saving instructions affects later ingests and does not itself
rewrite existing pages.
Wiki-enabled knowledge bases accept bounded user-owned content/extraction
instructions plus two independent optional generation inputs:
`wiki_config.directory_paths` (a one-to-three-level reference structure) and
`wiki_config.focus_instructions` (a strict extraction scope). The raw values
and normalized reference tree live in `llmwiki_knowledge_bases`; the effective
prompt tells WeKnora to use the directory names and hierarchy when they fit the
source, while allowing it to add, merge, rename, or omit nodes according to the
actual evidence. Directory guidance is never an allowlist and never a fact
source. `WeKnoraClient` merges edits into the remote `wiki_config` so synthesis
models and provider runtime knobs are retained.
Creation does not pre-create the reference skeleton. Mutation-settle and
periodic reconciliation run only for the strict focus scope: all page decisions
complete before any deletion, focus-outside generated pages are removed, and
the directory layout inferred by WeKnora is preserved. An incomplete or invalid
classifier result makes no destructive change and is retried. Directory-only
configuration therefore needs no post-generation reconciliation. The current
`directory_paths` field and the legacy `wiki_parent_directories` field have the
same prompt-only reference semantics; neither may trigger folder creation,
movement, deletion, or periodic directory reconciliation. Empty inputs keep
WeKnora's automatic behavior. Schema version 3 identifies this reference-only
prompt semantics. The background coordinator recompiles managed schema-2 rows
from their database-owned raw inputs and advances them only after the WeKnora
update succeeds; failures remain on schema 2 for retry.
Regression coverage:
`tests/test_wiki_directory_constraints.py` and
`tests/test_llmwiki_parent_directory_routing.py`.
Ordinary knowledge-base cards expose an owner/admin-only name/description/Wiki-instructions edit dialog in `WeKnoraKnowledgeWorkspace.tsx`, reusing `useLlmWikiActions().update` and `PATCH /api/llmwiki/knowledge-bases/{id}`. The route trims names, rejects blank or reserved conversation-deposit names, writes to WeKnora before updating local metadata, and retains existing write authorization. Default test questions are derived from the authoritative remote display name. The editor omits `test_questions` unless the user actually changes that field, and `knowledge_profile.generated_test_questions_name` recognizes only the exact legacy four-question template as automatic; genuinely edited lists remain custom. `_mapping_view` must resolve both questions and `test_questions_customized` from a profile row containing the current remote name, so a rename immediately replaces old-name defaults without a migration. A successful remote list is authoritative for every list scope, including `personal`: absent remote IDs are hidden without deleting mappings/references. Upstream failures remain errors. The grid also filters cached `remote_status=missing` rows and no longer renders stale-record cleanup cards. Regression coverage: `tests/test_llmwiki_knowledge_management.py`, `tests/test_llmwiki_weknora.py`.
@@ -1588,6 +2017,20 @@ See [docs/plan_mode_usage.md](docs/plan_mode_usage.md) for details.
### Context Summarization
Automatic conversation summarization when approaching token limits:
- **Administrator recent-turn window:** `system_settings.json` →
`conversation_context.recent_turn_window_enabled` is a deployment-wide,
default-off alternative to summarization. Administrators manage it in
「设置 → 上下文管理」 through
`GET/PUT /api/system-settings/conversation-context`. The same block stores
`max_conversation_rounds` (default 20, range 1–50), which `ChatPage` uses as
the maximum number of real user turns in one ordinary chat; missing values in
older settings files retain the 20-turn default. When enabled, each lead
agent model request retains only the current turn and the two preceding real
user turns, together with their assistant/tool continuations and any hidden
context attached to the retained boundary. The operation is request-only:
checkpoints and the transcript shown in the UI remain complete. This mode
takes priority over the per-chat 「上下文压缩」 toggle and never invokes the
summary model.
- **On/off switch is per-request, not the config file.** The chat input box has a
「上下文压缩」 toggle (below 参考文献, **default off**) → frontend sends
`context.summarization_enabled` → whitelisted in `_CONTEXT_CONFIGURABLE_KEYS`
@@ -1622,8 +2065,9 @@ Automatic conversation summarization when approaching token limits:
errors are non-retriable so a tokenizer mismatch cannot trigger repeated
submission of the same oversized request.
Tests: `tests/test_summarization_toggle.py` (gating + whitelist),
`tests/test_summarization_middleware.py` (cache behavior).
Tests: `tests/test_prompt_prefix.py` (system-setting API/defaults) and
`tests/test_summarization_multiturn.py` (summarization plus transient recent-turn
window behavior).
See [docs/summarization.md](docs/summarization.md) for details.
### Vision Support / Image Recognition (??)
@@ -1664,6 +2108,23 @@ applies when the short-lived signed WeKnora embed cookie is valid. Keep static
path recognition in `app/gateway/weknora_embed.py` synchronized with the
proxy's static fallback paths in `app/gateway/routers/llmwiki.py`.
Static assets use a separate signed-cookie-only path: never fetch an upstream
admin session or query per-KB ACLs for public JS/CSS/fonts. HTML and data APIs
still revalidate mapping permissions. `app/gateway/weknora_static.py` coalesces
concurrent reads and bounds its LRU by bytes and entries; cache keys include
the upstream web URL, public prefix, asset path and query. Bundle rewriting,
gzip and ETag generation run in a worker thread. Fingerprinted `/assets/`
files receive immutable browser caching; `/config.js` and upstream private,
no-store, no-cache or identity-varying responses must not enter the shared
cache. Explicit browser cache bypass refreshes the cached representation.
The injected iframe guard observes root theme changes separately and ignores
its own writes. Process only changed content branches, coalesced per batch;
ordinary loaded pages must have no recurring DOM-maintenance timer. Source
detail deep links retain a bounded nine-second drawer fallback. Preserve the
inner Wiki directory, protected images and theme repair. Regression tests:
`tests/test_weknora_static.py`, `tests/test_weknora_embed_script.py`.
### Position-roundtable role directory
`position_roles` is the global shared directory for position-roundtable work
@@ -2137,7 +2598,10 @@ deletable because it owns full-library aggregation.
Assistant-knowledge reads keep the landing and article paths size-bounded.
`list_bases()` is metadata/statistics-only and must never call the lazy global
alignment pass: imports align their snapshot before commit, while content reads
and search retain `_ensure_base_aligned`. The Wiki directory calls
use the last committed snapshot without waiting for the global import/alignment
locks; search and export retain `_ensure_base_aligned` for legacy data. Directory,
article, source and import-job reads resolve their base with `include_stats=False`
so loading/polling does not recount every content table. The Wiki directory calls
`GET /api/assistant-knowledge/wiki-pages?include_content=false`; repository
`load_only` projection must exclude both `content_markdown` and `search_text`
(response-only deletion is insufficient), and the frontend loads only the
@@ -2181,8 +2645,8 @@ Both Gateway document APIs and successful content mutations through the
embedded WeKnora proxy schedule one per-mapping debounced retry sequence with
bounded 1/8/30/90/300-second backoff. Never cancel an in-flight sync to debounce
a newer mutation; wake the coordinator, update the local generation, and let
the database lease deduplicate workers. While WeKnora reports processing, poll
only the cheap knowledge-base state and retain both a post-processing scan and
the database lease deduplicate workers. While WeKnora generates Wiki content,
poll both the knowledge-base state and per-document pending subtasks, then retain a post-processing scan and
one short quiet confirmation. Before retiring missing pages or replacing graph
edges, the sync service must re-read processing state; processing or an
unverifiable state produces a retryable partial result and preserves the prior
@@ -2306,6 +2770,31 @@ Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
names as timezone-aware PostgreSQL-family engines in portable types and
dialect-specific session settings.
### openGauss Dolphin / MySQL-protocol mode
The offline deployment may select `DATABASE_TYPE=gauss_mysql`. Its renderer
keeps `database.backend=mysql` and sets `DEERFLOW_DATABASE_PRODUCT=gauss_mysql`;
this explicit product marker is the switch because SQLAlchemy cannot safely
distinguish Dolphin from MySQL before dialect initialization. Only that mode
installs `deerflow.persistence.gauss_mysql_compat` for the configured async
driver. An unset marker, or `mysql`, must retain the stock MySQL dialect and
schema-maintenance behavior unchanged.
Dolphin table existence and additive schema checks use `current_schema()` plus
the native `pg_catalog` through `gauss_schema.ensure_gauss_schema_sync`.
Never run the generic MySQL `SHOW CREATE TABLE` reflection sweep, MySQL JSON
projection backfill, MySQL session-timeout variables, or application-side
`CREATE DATABASE` retry in this mode: these paths can misread the schema or
abort the openGauss transaction with error 322. Database/user/schema bootstrap
remains the offline deployment layer's responsibility. The startup DDL block
is still best-effort under the global startup-schema invariant and always
releases its host-local cross-worker lock.
`PortableJSON` writes strict JSON for both products. Its legacy decoder is
available only on a dialect class carrying the Dolphin compatibility marker;
ordinary MySQL must continue to reject malformed/non-JSON values rather than
silently applying the Dolphin fallbacks.
## 2026-09-14 我的项目与知识库验收修复
- 简洁模式的「我的项目」列表、策略工作台和普通问答项目恢复工作区功能导航;窄屏提供展开菜单按钮。
@@ -2434,3 +2923,24 @@ calls. The active sandbox skill mirror must be synchronized with the canonical c
Gateway detail writes emitted by `fs-collect` and `fs-detail-submit` log a token-masked request and
the corresponding response so an interrupted chain can be distinguished from a failed persistence
call without exposing the gateway bearer token.
### Task-detail custom skill password authentication (2026-10-26)
`skills/custom/task-context` and `skills/custom/task-detail-update` authenticate detail reads/writes
with a short-lived token minted by `POST /api/v1/auth/login/local`, using the runtime-only
`TASK_DETAIL_AGENT_USERNAME` / `TASK_DETAIL_AGENT_PASSWORD` pair and `TASK_DETAIL_GATEWAY_URL`.
They must never use general `DEERFLOW_AUTH_TOKEN` or `TOOLS_TOKEN`, nor emit the password or JWT in
an input, trace, or model-visible output. `/api/parallel-agents/auth/token` is only a legacy fallback:
it follows `sso_enabled` and returns 403 when external SSO is disabled. Password-login failures other
than the policy 403 must fail closed; do not fall back to identity-only token minting. Keep the
canonical custom directories and `.deer-flow/.sandbox-online-skills` mirror synchronized through the
custom-skill reconciliation flow. `scripts/setup_biz_detail_assistant.py` logs in through the same
formal endpoint and accepts `DEERFLOW_SETUP_BASE`, `DEERFLOW_SETUP_USERNAME`,
`DEERFLOW_SETUP_PASSWORD` for deployment overrides; it may fall back to the legacy username route
only for the formal login policy 403, never for a credential or rate-limit failure.
`biz-detail-assistant` must retain the native `ask_clarification` tool. Do not special-case it by
adding that tool to `requested_excluded_tools`, and do not mount `DetailConfirmMiddleware` to forge
confirmation cards from a plain model response: those cards are not genuine model tool calls and break
the standard pause/resume interaction. The final `ClarificationMiddleware` is the single authority for
all missing-information, scope, approach and risk-confirmation questions.