feat: improve assistant knowledge import and database compatibility

This commit is contained in:
吉鑫升 committed 2026-09-02 17:55:17 +08:00
1 parent fbd8dec48f
commit 6b724bd00d
51 files changed
+4393 -1021

No files matched your search

+49 -21
View File
@@ -957,26 +957,54 @@ sandbox-directory recreation cannot discard a completed rewrite.
## Skill knowledge distillation and assistant knowledge
The Gateway now exposes an administrator-only skill distillation pipeline at
`/api/skill-knowledge`. A matrix job can send up to 200 skills to multiple
WeKnora or global assistant-knowledge targets. The scanner never reads or
uploads code/script files, strips Markdown code fences, blocks credential-like
files and content, safely inspects ZIP members, and converts supported
Office/PDF/spreadsheet files off the event loop. Jobs use database claims and
renewable leases, stage snapshots before activation, retain the prior active
version on failure, support per-item retry and entity/relation review, and keep
per-binding contribution records for safe detach/delete.
`/api/skill-knowledge`. Each request can send up to 200 skills to one writable
WeKnora Wiki knowledge base; multi-base distillation is done by repeated
operations so the local binding remains `skill_name + target_id`. The scanner
never reads or uploads code/script files, strips Markdown code fences, blocks
credential-like files and content, safely inspects ZIP members, and converts
supported Office/PDF/spreadsheet files off the event loop. Jobs use database
claims and renewable leases, stage snapshots before activation, retain the
prior active version on failure, support per-item retry, and keep per-binding
contribution records for safe detach/delete. Direct assistant-base targets,
non-Wiki/FAQ/conversation/readonly targets, and direct Neo4j writes are rejected
for this product version.
Bindings may opt into debounced file watching and cron synchronization through
`/api/skill-knowledge/auto-sync-settings`. Only non-code knowledge digests can
mark a binding stale. WeKnora Wiki, document, FAQ, conversation-archive and
read-only sidecar targets have separate projections. Optional Neo4j direct
writes are disabled by default under `llmwiki.weknora.graph`; when enabled they
use only fixed parameterized templates and Neo4j's transactional HTTP endpoint.
`/api/assistant-knowledge` owns exactly one global DeerFlow-native base named
`知识梳理`. Reads and search are available to authenticated users; initialization,
local-file import, WeKnora export/import, source reimport/delete, and revision
rollback are admin-only. Imports align canonical entities and relations while
retaining source contributions, publish source/entity/relation Wiki revisions,
and remove canonical knowledge only after its final contribution disappears.
`/api/assistant-knowledge` manages DeerFlow-native assistant knowledge bases.
There is one global full-library base named `知识梳理总库`, plus admin-created
custom assistant bases. Reads and search are available to authenticated users;
global initialization, custom-base creation, WeKnora export/import, source
reimport, and source deletion are admin-only. Assistant bases only accept
WeKnora export packages or imports triggered from a WeKnora mapping; direct
skill import and arbitrary local-file import return 410. Importing into a custom
assistant base also mirrors the same package into the global base, where
canonical entities and relations are aligned across all imported sources.
The database schema is introduced by Alembic revision `20260901_01`.
## openGauss / GaussDB application database
The application ORM can use a PG-compatible openGauss/GaussDB database through
the official `opengauss-sqlalchemy` async dialect. Install the optional driver
set before enabling it:
```bash
uv sync --extra gauss
```
Configure the business database independently from the LangGraph checkpointer:
```yaml
database:
backend: gauss
gauss_url: $GAUSS_DATABASE_URL
checkpointer:
type: sqlite
connection_string: .deer-flow/data/deerflow.db
```
`GAUSS_DATABASE_URL` should normally be
`opengauss+asyncpg://user:password@host:26000/deerflow`. Plain
`opengauss://`, `gaussdb://`, and PostgreSQL-style URLs are accepted and
normalized to the openGauss dialect. Do not configure the LangGraph
PostgreSQL checkpointer against GaussDB: its migrations are PostgreSQL-specific.
For a production image, build with `--build-arg UV_EXTRAS=gauss`.