Merge branch 'ji' into master
This commit is contained in:
commit
49d9ced04f
84 files changed
+13258
-662
No files matched your search
@@ -379,7 +379,7 @@ FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api
|
||||
| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); `GET /list` - list; `DELETE /{filename}` - delete |
|
||||
| **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
|
||||
| **Threads search** (`POST /threads/search`, LangGraph-compat) | Conversation listing. **System threads are excluded by default**: rows whose `metadata.system` is truthy (scheduler runs, roundtable workers) never crowd the user's list; a caller whose `metadata` filter explicitly references `thread_type`/`system` (e.g. studio notebooks) opts out of the exclusion. New `query` field = case-insensitive title (`display_name`) substring search, used by the chats page's server-side pagination. `POST /threads/count` takes the same request body (limit/offset ignored) and returns `{total}` under identical filtering/exclusion semantics ? powers the chats page's ??/??? display without fetching rows (`ThreadMetaStore.count`). `ThreadMetaStore.search` gained `exclude_system`/`query` params; JSON-blob filters batch-scan until the requested `limit`/`offset` window is full (so pagination over a heavily-filtered set never under-fills), with a `thread_id` tie-breaker on the `updated_at DESC` ordering for stable OFFSET paging. Tests: `tests/test_thread_meta_search.py` |
|
||||
| **Artifacts** (`/api/threads/{id}/artifacts`) | `GET /{path}` - serve artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types |
|
||||
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`) | `GET /{path}` - serve artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `GET /api/artifact-library` scans every current-user normal thread's authoritative `outputs/` directory (including files omitted from `present_files`), excludes uploads and image/audio/video media, joins the owning conversation title/agent route metadata, and exposes search/filter/pagination with a bounded 15-second per-user cache plus `refresh=true` bypass. Frontend route `/page/workspace/library` links each item back to the source chat with `?artifact=<path>` so the chat opens the existing artifact panel directly. |
|
||||
| **Word export** (`POST /api/writing/export/docx`) | Shared Markdown → DOCX generator used by conversation, artifact, scheduled-task, Deep Research, and AI-writing download surfaces. `app/gateway/word_export.py` builds OOXML with no Python conversion dependency: A4 portrait, 3.7/3.5/2.8/2.6 cm margins, real multilevel heading numbering (`一、` / `(一)` / `1.`), two-character first-line indent on heading levels 1–3, formal body/table styles, page numbers beginning at 1 with odd pages left-aligned and even pages right-aligned, and embedded `app/gateway/assets/fonts/方正小标宋简体.ttf`. Before numbering it removes complete manual heading prefixes (including `2.1` / `2.1.1`) and drops Markdown horizontal rules from Word output. The ordinary-Q&A Lead Agent can emit the matching Markdown heading forms and avoid unordered bullet lists when the admin enables `prompt_prefix.ordinary_qa_markdown_format_enabled` in `system_settings.json`; the switch defaults off, while writing setup, writing mode, notebook, scheduled runs, and SOUL-driven custom agents keep their independent prompts. The font loader validates its internal family, SHA-256, and OpenType embedding permission (`OS/2.fsType`) before export. Tests: `tests/test_word_export.py`, `tests/test_es_query_routing_prompt.py`. |
|
||||
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
|
||||
| **Gateway auth** | Owner-scoped Deep Research routes await the platform's asynchronous `get_current_user` resolver before they read or write persistence, ensuring each session and job operation receives a concrete user ID. Deep Research artifacts are always resolved from a whitelisted filename via `/mnt/user-data/outputs/<name>` before mapping to a host path, preserving the sandbox guard on native Windows and Linux hosts. |
|
||||
@@ -1611,8 +1611,10 @@ same DOM value, so a fetched-and-revoked object URL is not acceptable.
|
||||
Third-party pages embed published Wiki content through the chrome-free route
|
||||
`/#/embed/knowledge/{mapping_id}` (optional exact-page `wikiSlug`). It defaults
|
||||
to a recognized Wiki index page (title `索引`/`index`, page type `index`, or an
|
||||
`index`/`home`/`readme` slug), promotes that page to the first public-directory
|
||||
entry, then falls back to an overview/root page when no index exists. It retains
|
||||
`index`/`home`/`readme` slug) and promotes that page to the first public-directory
|
||||
entry. When upstream pages exist without an explicit index shell, the frontend
|
||||
injects a virtual `__index__` page and builds its catalogue from the page list;
|
||||
an all-empty dynamic index response must also fall back to that page list. It retains
|
||||
directory navigation and supports the shared public-embed `theme` / `bg` /
|
||||
`text` appearance inputs. WeKnora dynamically renders index page-type counts,
|
||||
nested category headings, page summaries, and `data-slug` links even though the
|
||||
@@ -1628,9 +1630,14 @@ the loaded page title, slug, or alias and rewrites them to exact embedded
|
||||
`wikiSlug` routes. Preserve WeKnora's `wiki-content-link` interaction: primary
|
||||
brand color, medium weight, dashed underline becoming solid on hover, focus
|
||||
outline, and in-reader SPA navigation rather than a page reload. `GET
|
||||
/api/public/llmwiki/knowledge-bases` is the
|
||||
anonymous discovery endpoint and returns only each published ordinary mapping's
|
||||
`id` and `name` (plus the collection total). All anonymous reader calls use that
|
||||
/api/public/llmwiki/knowledge-bases` is the anonymous discovery endpoint. It
|
||||
validates published ordinary mappings against WeKnora, drops stale mappings,
|
||||
and returns safe comprehensive metadata from the WeKnora list/detail/page APIs:
|
||||
document/chunk/processing/share counts, non-secret configuration and timestamps,
|
||||
Wiki page type/status counts and explicit/generated index entry, plus aggregate
|
||||
local-vector state/counts/timestamps. Never expose remote WeKnora ids,
|
||||
tenant/creator identity, storage credentials, API credentials, raw vectors, or
|
||||
per-page indexing errors. All anonymous reader calls use that
|
||||
public namespace and the `/knowledge-bases/{id}` Wiki-page children. Those
|
||||
endpoints must resolve with a synthetic anonymous user and `is_admin=False`,
|
||||
explicitly require `publication_status=published`, and hard-deny the system
|
||||
@@ -1709,3 +1716,44 @@ before MarkItDown extraction, avoiding concurrent profile locks. The image
|
||||
build and `OFFLINE_INSTALL.sh --office-check` both run
|
||||
`scripts/office_runtime_selfcheck.py`; that smoke test creates and re-parses
|
||||
minimal DOCX/XLSX/PPTX files and does not need the application database.
|
||||
|
||||
## Skill/assistant knowledge invariants
|
||||
|
||||
Skill distillation is implemented in `deerflow.skill_knowledge` and
|
||||
`deerflow.persistence.skill_knowledge`; native assistant bases are in
|
||||
`deerflow.assistant_knowledge` and
|
||||
`deerflow.persistence.assistant_knowledge`. Gateway composition, remote adapters,
|
||||
the lease dispatcher, and APIs remain in `app.gateway` so the harness-to-app
|
||||
import firewall is preserved.
|
||||
|
||||
Never execute a skill or read/upload/summarize its code and script files.
|
||||
Knowledge fingerprints exclude those files and Markdown code fences. All
|
||||
resyncs must stage and validate the knowledge digest before activation; the old
|
||||
snapshot remains authoritative on failure. A binding may have only one active
|
||||
item (`active_dedupe_key` plus atomic lease claims). Skill distillation targets
|
||||
only writable WeKnora Wiki knowledge bases in this product version; reject direct
|
||||
assistant-base targets, non-Wiki/FAQ/conversation/readonly targets, and direct
|
||||
Neo4j writes. Detach/delete operates on source contributions, not canonical rows,
|
||||
and shared entities/relations are retired only when their final contribution is
|
||||
gone.
|
||||
|
||||
Assistant knowledge has one global full-library base (`知识梳理总库`) and any
|
||||
number of admin-created custom bases. Ordinary users have read/search access and
|
||||
administrators own all mutations. Assistant bases only accept WeKnora export
|
||||
packages or imports triggered from a WeKnora mapping. Do not add direct skill
|
||||
imports or arbitrary local-file imports to assistant knowledge. Importing into a
|
||||
custom assistant base must also mirror the package into the global base for
|
||||
cross-source entity alignment.
|
||||
|
||||
## openGauss / GaussDB persistence invariant
|
||||
|
||||
`database.backend=gauss` is an application-ORM backend, parallel to `mysql`.
|
||||
It normalizes `database.gauss_url` to `opengauss+asyncpg://` and requires the
|
||||
`gauss` optional dependency (`asyncpg` plus `opengauss-sqlalchemy`). Never route
|
||||
a Gauss URL through SQLAlchemy's generic PostgreSQL dialect: the wire protocol
|
||||
may connect, but dialect initialization and schema DDL can fail against the
|
||||
Gauss server/version behavior. Keep the LangGraph checkpointer on the separately
|
||||
configured SQLite backend; the PostgreSQL checkpointer schema is not part of the
|
||||
Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
|
||||
names as timezone-aware PostgreSQL-family engines in portable types and
|
||||
dialect-specific session settings.
|
||||
Reference in new issue
Block a user