吉鑫升
c4fd9dd7da
# Conflicts: # .gitignore # frontend-web/src/components/workspace/input-box.tsx # frontend-web/src/core/threads/types.ts # frontend-web/src/pages/ChatPage.tsx
565 lines
33 KiB
Markdown
565 lines
33 KiB
Markdown
# DeerFlow Backend
|
|
|
|
DeerFlow is a LangGraph-based AI super agent with sandbox execution, persistent memory, and extensible tool integration. The backend enables AI agents to execute code, browse the web, manage files, delegate tasks to subagents, and retain context across conversations - all in isolated, per-thread environments.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌──────────────────────────────────────┐
|
|
│ Nginx (Port 2026) │
|
|
│ Unified reverse proxy │
|
|
└───────┬──────────────────┬───────────┘
|
|
│ │
|
|
/api/langgraph/* │ │ /api/* (other)
|
|
▼ ▼
|
|
┌────────────────────┐ ┌────────────────────────┐
|
|
│ LangGraph Server │ │ Gateway API (8001) │
|
|
│ (Port 2024) │ │ FastAPI REST │
|
|
│ │ │ │
|
|
│ ┌────────────────┐ │ │ Models, MCP, Skills, │
|
|
│ │ Lead Agent │ │ │ Memory, Uploads, │
|
|
│ │ ┌──────────┐ │ │ │ Artifacts │
|
|
│ │ │Middleware│ │ │ └────────────────────────┘
|
|
│ │ │ Chain │ │ │
|
|
│ │ └──────────┘ │ │
|
|
│ │ ┌──────────┐ │ │
|
|
│ │ │ Tools │ │ │
|
|
│ │ └──────────┘ │ │
|
|
│ │ ┌──────────┐ │ │
|
|
│ │ │Subagents │ │ │
|
|
│ │ └──────────┘ │ │
|
|
│ └────────────────┘ │
|
|
└────────────────────┘
|
|
```
|
|
|
|
**Request Routing** (via Nginx):
|
|
- `/api/langgraph/*` → LangGraph Server - agent interactions, threads, streaming
|
|
- `/api/*` (other) → Gateway API - models, MCP, skills, memory, artifacts, uploads, thread-local cleanup
|
|
- `/` (non-API) → Frontend - Next.js web interface
|
|
|
|
Health probes are always unauthenticated, even when API authentication is enabled: `GET|HEAD /health`, `/api/health`, and `/api/langgraph/health` all return the Gateway health status without requiring a cookie or bearer token.
|
|
|
|
---
|
|
|
|
## Core Components
|
|
|
|
### Lead Agent
|
|
|
|
The single LangGraph agent (`lead_agent`) is the runtime entry point, created via `make_lead_agent(config)`. It combines:
|
|
|
|
- **Dynamic model selection** with thinking and vision support
|
|
- **Middleware chain** for cross-cutting concerns (9 middlewares)
|
|
- **Tool system** with sandbox, MCP, community, and built-in tools
|
|
- **Subagent delegation** for parallel task execution
|
|
- **System prompt** with skills injection, memory context, and working directory guidance
|
|
|
|
### Middleware Chain
|
|
|
|
Middlewares execute in strict order, each handling a specific concern:
|
|
|
|
| # | Middleware | Purpose |
|
|
|---|-----------|---------|
|
|
| 1 | **ThreadDataMiddleware** | Creates per-thread isolated directories (workspace, uploads, outputs) |
|
|
| 2 | **UploadsMiddleware** | Injects newly uploaded files into conversation context |
|
|
| 3 | **SandboxMiddleware** | Acquires sandbox environment for code execution |
|
|
| 4 | **SummarizationMiddleware** | Reduces context when approaching token limits (optional) |
|
|
| 5 | **TodoListMiddleware** | Tracks multi-step tasks in plan mode (optional) |
|
|
| 6 | **TitleMiddleware** | Auto-generates conversation titles after first exchange |
|
|
| 7 | **MemoryMiddleware** | Queues conversations for async memory extraction |
|
|
| 8 | **ViewImageMiddleware** | Injects image data for vision-capable models (conditional) |
|
|
| 9 | **ClarificationMiddleware** | Intercepts clarification requests and interrupts execution (must be last) |
|
|
|
|
### Sandbox System
|
|
|
|
Per-thread isolated execution with virtual path translation:
|
|
|
|
- **Abstract interface**: `execute_command`, `read_file`, `write_file`, `list_dir`
|
|
- **Providers**: `LocalSandboxProvider` (filesystem) and `AioSandboxProvider` (Docker, in community/)
|
|
- **Virtual paths**: `/mnt/user-data/{workspace,uploads,outputs}` → thread-specific physical directories
|
|
- **Skills path**: `/mnt/skills` → `deer-flow/skills/` directory
|
|
- **Skills loading**: Recursively discovers nested `SKILL.md` files under `skills/{public,custom}` and preserves nested container paths
|
|
- **Conditional WeKnora LLMWiki**: a non-empty `llmwiki.weknora.api_base_url` enables the DeerFlow-native WeKnora BFF and UI; an empty address preserves the existing LLMWiki page. WeKnora remains independently deployed and owns its database, Redis, storage, parsers and models. DeerFlow keeps only the API/admin addresses, user/publication mappings, agent knowledge-base bindings and per-conversation selections; it stores and sends no WeKnora credential, so the endpoint must be restricted to a trusted network and allow DeerFlow traffic. Owners publish directly to the public LLMWiki without an approval queue for ordinary bases; the system-owned conversation-deposit base is always published, shared by all users, excluded from My Knowledge, immutable through user/admin detail operations, and recognized by both its reserved name and `system` owner. An existing same-name remote mapping is adopted in place as the system deposit so its mapping id and document links stay valid. Its user-facing knowledge-base description and generated document metadata use the `cmzs` brand; list/detail reads synchronize stale remote descriptions, while conversation-deposit search results, source previews and embedded-detail JSON rewrite legacy brand text to `cmzs`. Deposit requests must reference a thread accessible to the current account. Personal knowledge-base lists are scoped to the current DeerFlow account even for admins, while frontend LLMWiki query caches also include the active account id so switching accounts cannot reuse another account's list/detail data. Admins remain the only role that sees the WeKnora connection-config tab. Knowledge cards open a dedicated WeKnora-style detail route: Wiki-enabled bases default to Wiki, then documents and graph; standard bases hide Wiki and default to documents. Writable ordinary knowledge-base details expose a native multi-file upload action that submits files sequentially and reports aggregate success/failure; read-only public bases and the system deposit base do not. It supports file/URL/manual-Markdown imports, Wiki page browsing, document preview, real parsed chunks and Wiki graph nodes/edges. New chats use a compact WeKnora-style searchable/grouped multi-select inside the composer, while agent create/edit pages bind spaces through an inline panel. A single proxy service identity means DeerFlow users are isolated logically while their WeKnora entities share that identity's workspace. The read-only `llmwiki_search` tool reauthorizes the current DeerFlow user on every call and renders credential-free source previews; source previews prefer cleaned Wiki/LLMWiki text when a retrieved chunk maps to a Wiki page, with raw chunks as the fallback. The detail page first creates a signed iframe session through the normal DeerFlow API auth path, then points the iframe directly at the proxied WeKnora `/platform/...` route using the signed `deerflow_weknora_embed` cookie; global auth/CSRF bypass is limited to those signed iframe proxy requests, and HTTPS cookie attributes honor `X-Forwarded-Proto` behind reverse proxies. The iframe proxy keeps WeKnora's root-relative `/platform`, `/assets`, `/locales`, and `/api/v1/*` paths, but is registered last through `llmwiki.proxy_router` so it cannot intercept first-party Gateway APIs such as `POST /api/v1/auth/login/username`. See [docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md](docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md).
|
|
- **File-write safety**: `str_replace` serializes read-modify-write per `(sandbox.id, path)` so isolated sandboxes keep concurrency even when virtual paths match
|
|
- **Tools**: `bash`, `ls`, `read_file`, `write_file`, `str_replace` (`bash` is disabled by default when using `LocalSandboxProvider`; use `AioSandboxProvider` for isolated shell access)
|
|
|
|
### Subagent System
|
|
|
|
Async task delegation with concurrent execution:
|
|
|
|
- **Built-in agents**: `general-purpose` (full toolset) and `bash` (command specialist, exposed only when shell access is available)
|
|
- **Concurrency**: Max 3 subagents per turn, 15-minute timeout
|
|
- **Execution**: Background thread pools with status tracking and SSE events
|
|
- **Flow**: Agent calls `task()` tool → executor runs subagent in background → polls for completion → returns result
|
|
|
|
### Memory System
|
|
|
|
LLM-powered persistent context retention across conversations:
|
|
|
|
- **Automatic extraction**: Analyzes conversations for user context, facts, and preferences
|
|
- **Structured storage**: User context (work, personal, top-of-mind), history, and confidence-scored facts
|
|
- **Debounced updates**: Batches updates to minimize LLM calls (configurable wait time)
|
|
- **System prompt injection**: Top facts + context injected into agent prompts
|
|
- **Storage**: JSON file with mtime-based cache invalidation
|
|
|
|
### Tool Ecosystem
|
|
|
|
| Category | Tools |
|
|
|----------|-------|
|
|
| **Sandbox** | `bash`, `ls`, `read_file`, `write_file`, `str_replace` |
|
|
| **Built-in** | `present_files`, `ask_clarification`, `view_image`, `task` (subagent) |
|
|
| **Community** | Tavily (web search), Jina AI (web fetch), Firecrawl (scraping), DuckDuckGo (image search) |
|
|
| **MCP** | Any Model Context Protocol server (stdio, SSE, HTTP transports) |
|
|
| **Skills** | Domain-specific workflows injected via system prompt |
|
|
|
|
#### Configurable HTTP web search
|
|
|
|
The config-defined `web_search` tool can call a deployment-specific JSON API
|
|
instead of a public search provider. Set its `endpoint`, static `payload`, SSL
|
|
verification, timeout, result limit, and `result_url_template` in `config.yaml`.
|
|
At runtime only `payload.query` is overwritten with the model's search text.
|
|
The API response must use `{"results": [{"recUuid": "", "content": "",
|
|
"title": "", "url": ""}]}`; missing or null result fields are normalized to empty strings. When
|
|
`recUuid` is present, `{recUuid}` in `result_url_template` replaces the original
|
|
result URL; otherwise the original URL is retained. Set `enabled: false` to keep
|
|
the tool completely out of the agent's registered tool list. `verify_ssl: false`
|
|
supports trusted internal HTTPS services with self-signed certificates.
|
|
|
|
### Gateway API
|
|
|
|
FastAPI application providing REST endpoints for frontend integration:
|
|
|
|
| Route | Purpose |
|
|
|-------|---------|
|
|
| `GET /api/models` | List available LLM models |
|
|
| `GET/PUT /api/mcp/config` | Manage MCP server configurations |
|
|
| `GET/PUT /api/skills` | List and manage skills |
|
|
| `POST /api/skills/install` | Install skill from `.skill` archive |
|
|
| `GET/POST /api/agents` | List visible agents or create a user-owned agent |
|
|
| `GET/PUT/DELETE /api/agents/{id}` | Read, update, or delete an id-addressed agent |
|
|
| `GET/POST/PUT/DELETE /api/roundtable-chains` | Manage user-owned roundtable business chains; each seat may optionally carry `position_id` for the separate position-collaboration workspace, plus `human_validation` to mark a Step 2 client-side pause checkpoint after that seat/stage completes (the original roundtable has no extra validation card) |
|
|
| `GET/PUT /api/business-mapping` | Read global business mappings. Updates to 3Q/6BF/7BF require an administrator; the 8BF public business-chain mapping is maintained exclusively by the trusted `lqq` account (derived from the authenticated email prefix). |
|
|
| `GET/POST/PUT /api/position-roundtable/sessions` | Standalone position-collaboration sessions: freeze task/intent/chain snapshots, index one direct-agent node per seat, build upstream handoff context (including bounded UTF-8 excerpts of text artifacts), and make archived sessions read-only without invoking the original roundtable coordinator or jobs. Built-in summary/action-plan completion mirrors visible report text into a Markdown artifact when a model omits its final `write_file` call. `POST /sessions/{id}/nodes/{nodeKey}/reject` accepts an optional reason, marks the rejected delivery for rework, and invalidates affected downstream deliveries so they are regenerated in order. The same router is also mounted at `/api/multi-agent/position-roundtable/*` for intranet gateways that only forward the established multi-agent namespace; the compatibility mount is intentionally omitted from OpenAPI. |
|
|
| `GET /api/memory` | Retrieve memory data |
|
|
| `POST /api/memory/reload` | Force memory reload |
|
|
| `GET /api/memory/config` | Memory configuration |
|
|
| `GET /api/memory/status` | Combined config + data |
|
|
| `POST /api/threads/{id}/uploads` | Upload files (auto-converts PDF/PPT/Excel/Word to Markdown, rejects directory paths) |
|
|
| `GET /api/threads/{id}/uploads/list` | List uploaded files |
|
|
| `DELETE /api/threads/{id}` | Delete DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
|
|
| `GET /api/threads/{id}/artifacts/{path}` | Serve generated artifacts |
|
|
| `POST /api/ai-writing/sample/extract` | Extract text from Word/PDF/Markdown/TXT samples for AI-writing imitation mode |
|
|
| `POST /cop/saveSuperiorTask` | TaskCOP compatibility API (DeerFlow Bearer/session authentication required): create a situation-overview task with a sequential four-digit id (`0001`, `0002`, ...) and return the legacy `{state, msg, data}` envelope |
|
|
| `POST /api/taskcop/import/tasks` | TaskCOP compatibility API (authenticated): import tasks from a JSON array or `tasks`/`records` envelope; every row's supplied `id`/`taskId` is authoritative and a duplicate id completely replaces the stored task |
|
|
| `DELETE /cop/tasks/{task_id}` | TaskCOP compatibility API (authenticated): delete one task and its task-scoped situation-report document |
|
|
| `GET /taskAnalyseSearch/cop-task-three-list` | TaskCOP compatibility API (authenticated): paginated task list for the situation-overview sentiment page (`content`, `taskStatus`, `taskDirection`, `startTime`, `endTime`) |
|
|
| `GET/POST/PUT /api/task-reports/by-task/{task_id}` | Shared task-scoped situation-report detail. Query, first save, and full update all use the legacy-compatible `{state, msg, data:[{id, taskId, createTime, updateTime, sessionId, contentJson, categoryType}]}` shape; `contentJson` is preserved as a JSON string. Saving a non-empty report automatically moves the matching TaskCOP task to status `25` (the legacy list's “已完成” state). |
|
|
| `POST /api/task-reports/import/by-task/{task_id}` | TaskCOP detail import (authenticated): accept a `sq-report-mock.json`-compatible `data`/`records` array and atomically replace the selected task's full report. The path task id overrides task ids inside the file. |
|
|
|
|
The separately deployed TaskCOP Vue app uses `POST /api/v1/auth/login/username` directly (its `username`, `userId`, or `yUserId` URL parameter identifies the user), so this compatibility flow does not depend on the retired Consumer login service. It calls the Gateway address configured as `b1ConsumerUrl` directly; it does not use a Vite `/api` reverse proxy. An empty TaskCOP store remains empty; task data is introduced through normal creation or the JSON import API. The Gateway CORS middleware always merges the local TaskCOP Vite origins `http://localhost:8080` and `http://127.0.0.1:8080`, including when `GATEWAY_CORS_ORIGINS` is set for another frontend. `scripts/env.sh` also includes them in its startup default. For a deployment, set `GATEWAY_CORS_ORIGINS` to the exact additional frontend origins that are allowed to call the Gateway.
|
|
|
|
Position-roundtable history supports two modes. A non-empty `external_task_id` from the frontend route's `taskId` creates a row with `task_scoped = true`, so all sessions, nodes and persisted conversation snapshots for that task are shared across users; `user_id` is then creator audit metadata only. When the field is absent or empty, the original per-user personal-session behavior remains unchanged. Alembic revision `20260808_01` adds `task_scoped` with the safe default `false` for historical rows plus the task/update-time index.
|
|
|
|
AI writing supports a sample-imitation mode: the frontend can upload or paste a sample article, the Gateway extracts text for document files, and the `ai_writing` graph runs a `sample_analyzer` node before normal intent/research/outline/draft stages. The resulting style profile is injected into outline and draft prompts with explicit guardrails against copying source sentences, facts, names, or data.
|
|
|
|
### Custom Agents
|
|
|
|
Custom agents are indexed in the `agents` database table. The `id` column is the stable runtime identifier and filesystem directory name (`.deer-flow/agents/{id}/SOUL.md`); `name` is display-only, may be Chinese, and may be duplicated. Rows with `user_id = NULL` are built-in agents, user rows are private unless `published = true`, and users can only update or delete agents they created.
|
|
|
|
### AI Writing Intranet Retrieval
|
|
|
|
AI 写作的「知识库检索」会直连 `ai_writing.intranet_search_url` 内网 ES/知识库接口，并默认跳过代理环境变量和 HTTPS 证书校验，适配自签名或内部 CA 证书的内网服务。需要强校验时可在 `config.yaml` 里设置 `ai_writing.intranet_verify_ssl: true`。通用检索里的非内置技能在 `ai_writing.researcher_skill_agent: true` 时会用真实 agent 运行时执行技能：以 `ai-writing-researcher` 身份读取该技能的 `SKILL.md`，并按技能要求通过 `bash` 运行脚本，弱模型也会在本轮任务里收到明确的读文件和脚本执行指令。
|
|
|
|
### IM Channels
|
|
|
|
The IM bridge supports Feishu, Slack, and Telegram. Slack and Telegram still use the final `runs.wait()` response path, while Feishu now streams through `runs.stream(["messages-tuple", "values"])` and updates a single in-thread card in place.
|
|
|
|
For Feishu card updates, DeerFlow stores the running card's `message_id` per inbound message and patches that same card until the run finishes, preserving the existing `OK` / `DONE` reaction flow.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- Python 3.12+
|
|
- [uv](https://docs.astral.sh/uv/) package manager
|
|
- API keys for your chosen LLM provider
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
cd deer-flow
|
|
|
|
# Copy configuration files
|
|
cp config.example.yaml config.yaml
|
|
|
|
# Install backend dependencies
|
|
cd backend
|
|
make install
|
|
```
|
|
|
|
### Configuration
|
|
|
|
Edit `config.yaml` in the project root:
|
|
|
|
```yaml
|
|
models:
|
|
- name: gpt-4o
|
|
display_name: GPT-4o
|
|
use: langchain_openai:ChatOpenAI
|
|
model: gpt-4o
|
|
api_key: $OPENAI_API_KEY
|
|
supports_thinking: false
|
|
supports_vision: true
|
|
|
|
- name: gpt-5-responses
|
|
display_name: GPT-5 (Responses API)
|
|
use: langchain_openai:ChatOpenAI
|
|
model: gpt-5
|
|
api_key: $OPENAI_API_KEY
|
|
use_responses_api: true
|
|
output_version: responses/v1
|
|
supports_vision: true
|
|
```
|
|
|
|
Set your API keys:
|
|
|
|
```bash
|
|
export OPENAI_API_KEY="your-api-key-here"
|
|
```
|
|
|
|
### Running
|
|
|
|
**Full Application** (from project root):
|
|
|
|
```bash
|
|
make dev # Starts LangGraph + Gateway + Frontend + Nginx
|
|
```
|
|
|
|
Access at: http://localhost:2026
|
|
|
|
**Backend Only** (from backend directory):
|
|
|
|
```bash
|
|
# Terminal 1: LangGraph server
|
|
make dev
|
|
|
|
# Terminal 2: Gateway API
|
|
make gateway
|
|
```
|
|
|
|
Direct access: LangGraph at http://localhost:2024, Gateway at http://localhost:8001
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
backend/
|
|
├── src/
|
|
│ ├── agents/ # Agent system
|
|
│ │ ├── lead_agent/ # Main agent (factory, prompts)
|
|
│ │ ├── middlewares/ # 9 middleware components
|
|
│ │ ├── memory/ # Memory extraction & storage
|
|
│ │ └── thread_state.py # ThreadState schema
|
|
│ ├── gateway/ # FastAPI Gateway API
|
|
│ │ ├── app.py # Application setup
|
|
│ │ └── routers/ # 6 route modules
|
|
│ ├── sandbox/ # Sandbox execution
|
|
│ │ ├── local/ # Local filesystem provider
|
|
│ │ ├── sandbox.py # Abstract interface
|
|
│ │ ├── tools.py # bash, ls, read/write/str_replace
|
|
│ │ └── middleware.py # Sandbox lifecycle
|
|
│ ├── subagents/ # Subagent delegation
|
|
│ │ ├── builtins/ # general-purpose, bash agents
|
|
│ │ ├── executor.py # Background execution engine
|
|
│ │ └── registry.py # Agent registry
|
|
│ ├── tools/builtins/ # Built-in tools
|
|
│ ├── mcp/ # MCP protocol integration
|
|
│ ├── models/ # Model factory
|
|
│ ├── skills/ # Skill discovery & loading
|
|
│ ├── config/ # Configuration system
|
|
│ ├── community/ # Community tools & providers
|
|
│ ├── reflection/ # Dynamic module loading
|
|
│ └── utils/ # Utilities
|
|
├── docs/ # Documentation
|
|
├── tests/ # Test suite
|
|
├── langgraph.json # LangGraph server configuration
|
|
├── pyproject.toml # Python dependencies
|
|
├── Makefile # Development commands
|
|
└── Dockerfile # Container build
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Main Configuration (`config.yaml`)
|
|
|
|
Place in project root. Config values starting with `$` resolve as environment variables.
|
|
|
|
Key sections:
|
|
- `models` - LLM configurations with class paths, API keys, thinking/vision flags
|
|
- `tools` - Tool definitions with module paths and groups
|
|
- `tool_groups` - Logical tool groupings
|
|
- `sandbox` - Execution environment provider
|
|
- `skills` - Skills directory paths
|
|
- `title` - Auto-title generation settings
|
|
- `summarization` - Context summarization settings
|
|
- `subagents` - Subagent system (enabled/disabled)
|
|
- `memory` - Memory system settings (enabled, storage, debounce, facts limits)
|
|
|
|
Provider note:
|
|
- `models[*].use` references provider classes by module path (for example `langchain_openai:ChatOpenAI`).
|
|
- If a provider module is missing, DeerFlow now returns an actionable error with install guidance (for example `uv add langchain-google-genai`).
|
|
|
|
### Browser CORS
|
|
|
|
The Gateway permits the local TaskCOP/Vite development origins
|
|
`http://localhost:8080` and `http://127.0.0.1:8080` with credentials. For a
|
|
separate deployed frontend, set `GATEWAY_CORS_ORIGINS` to its exact origin
|
|
(including scheme and port), for example `http://47.88.25.99:7010`. Explicit
|
|
origins are used in addition to the local TaskCOP development origins.
|
|
|
|
### Extensions Configuration (`extensions_config.json`)
|
|
|
|
MCP servers and skill states in a single file:
|
|
|
|
```json
|
|
{
|
|
"mcpServers": {
|
|
"github": {
|
|
"enabled": true,
|
|
"type": "stdio",
|
|
"command": "npx",
|
|
"args": ["-y", "@modelcontextprotocol/server-github"],
|
|
"env": {"GITHUB_TOKEN": "$GITHUB_TOKEN"}
|
|
},
|
|
"secure-http": {
|
|
"enabled": true,
|
|
"type": "http",
|
|
"url": "https://api.example.com/mcp",
|
|
"oauth": {
|
|
"enabled": true,
|
|
"token_url": "https://auth.example.com/oauth/token",
|
|
"grant_type": "client_credentials",
|
|
"client_id": "$MCP_OAUTH_CLIENT_ID",
|
|
"client_secret": "$MCP_OAUTH_CLIENT_SECRET"
|
|
}
|
|
}
|
|
},
|
|
"skills": {
|
|
"pdf-processing": {"enabled": true}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Environment Variables
|
|
|
|
- `DEER_FLOW_CONFIG_PATH` - Override config.yaml location
|
|
- `DEER_FLOW_EXTENSIONS_CONFIG_PATH` - Override extensions_config.json location
|
|
- Model API keys: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `DEEPSEEK_API_KEY`, etc.
|
|
- Tool API keys: `TAVILY_API_KEY`, `GITHUB_TOKEN`, etc.
|
|
|
|
### LangSmith Tracing
|
|
|
|
DeerFlow has built-in [LangSmith](https://smith.langchain.com) integration for observability. When enabled, all LLM calls, agent runs, tool executions, and middleware processing are traced and visible in the LangSmith dashboard.
|
|
|
|
**Setup:**
|
|
|
|
1. Sign up at [smith.langchain.com](https://smith.langchain.com) and create a project.
|
|
2. Add the following to your `.env` file in the project root:
|
|
|
|
```bash
|
|
LANGSMITH_TRACING=true
|
|
LANGSMITH_ENDPOINT=https://api.smith.langchain.com
|
|
LANGSMITH_API_KEY=lsv2_pt_xxxxxxxxxxxxxxxx
|
|
LANGSMITH_PROJECT=xxx
|
|
```
|
|
|
|
**Legacy variables:** The `LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY`, `LANGCHAIN_PROJECT`, and `LANGCHAIN_ENDPOINT` variables are also supported for backward compatibility. `LANGSMITH_*` variables take precedence when both are set.
|
|
|
|
### Langfuse Tracing
|
|
|
|
DeerFlow also supports [Langfuse](https://langfuse.com) observability for LangChain-compatible runs.
|
|
|
|
Add the following to your `.env` file:
|
|
|
|
```bash
|
|
LANGFUSE_TRACING=true
|
|
LANGFUSE_PUBLIC_KEY=pk-lf-xxxxxxxxxxxxxxxx
|
|
LANGFUSE_SECRET_KEY=sk-lf-xxxxxxxxxxxxxxxx
|
|
LANGFUSE_BASE_URL=https://cloud.langfuse.com
|
|
```
|
|
|
|
If you are using a self-hosted Langfuse deployment, set `LANGFUSE_BASE_URL` to your Langfuse host.
|
|
|
|
### Dual Provider Behavior
|
|
|
|
If both LangSmith and Langfuse are enabled, DeerFlow initializes and attaches both callbacks so the same run data is reported to both systems.
|
|
|
|
If a provider is explicitly enabled but required credentials are missing, or the provider callback cannot be initialized, DeerFlow raises an error when tracing is initialized during model creation instead of silently disabling tracing.
|
|
|
|
**Docker:** In `docker-compose.yaml`, tracing is disabled by default (`LANGSMITH_TRACING=false`). Set `LANGSMITH_TRACING=true` and/or `LANGFUSE_TRACING=true` in your `.env`, together with the required credentials, to enable tracing in containerized deployments.
|
|
|
|
---
|
|
|
|
## Development
|
|
|
|
### Commands
|
|
|
|
```bash
|
|
make install # Install dependencies
|
|
make dev # Run LangGraph server (port 2024)
|
|
make gateway # Run Gateway API (port 8001)
|
|
make lint # Run linter (ruff)
|
|
make format # Format code (ruff)
|
|
```
|
|
|
|
### Code Style
|
|
|
|
- **Linter/Formatter**: `ruff`
|
|
- **Line length**: 240 characters
|
|
- **Python**: 3.12+ with type hints
|
|
- **Quotes**: Double quotes
|
|
- **Indentation**: 4 spaces
|
|
|
|
### Testing
|
|
|
|
```bash
|
|
uv run pytest
|
|
```
|
|
|
|
### Runtime appearance themes
|
|
|
|
The admin appearance settings are persisted in `.deer-flow/system_settings.json`.
|
|
Besides the built-in themes, an administrator can create up to 12 runtime
|
|
themes through `PUT /api/system-settings/appearance`. Each theme stores a
|
|
safe six-digit `primary` colour, an optional primary-button `text_color`,
|
|
`light`/`dark` mode, name, enabled flag and sort order. The frontend derives
|
|
the rest of the visual tokens; the optional `overrides` object only exposes
|
|
ten interaction colours (`button_hover`, `navigation_gradient_start`,
|
|
`navigation_gradient_end`, `selected_background`, `border`, `muted`,
|
|
`muted_text`, `destructive`, `brand_title`, `primary_foreground`) for
|
|
exceptional brand requirements. The two navigation values override the
|
|
automatically derived top-bar gradient. A custom theme id
|
|
must use the `custom-theme-` prefix. Disabled or removed custom themes cannot
|
|
remain the system default and automatically fall back to `light`.
|
|
|
|
---
|
|
|
|
## Technology Stack
|
|
|
|
- **LangGraph** (1.0.6+) - Agent framework and multi-agent orchestration
|
|
- **LangChain** (1.2.3+) - LLM abstractions and tool system
|
|
- **FastAPI** (0.115.0+) - Gateway REST API
|
|
- **langchain-mcp-adapters** - Model Context Protocol support
|
|
- **agent-sandbox** - Sandboxed code execution
|
|
- **markitdown** - Multi-format document conversion
|
|
- **tavily-python** / **firecrawl-py** - Web search and scraping
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
- [Configuration Guide](docs/CONFIGURATION.md)
|
|
- [Architecture Details](docs/ARCHITECTURE.md)
|
|
- [API Reference](docs/API.md)
|
|
- [Offline Docker wheelhouse deployment (ZH)](../README.md)
|
|
- [File Upload](docs/FILE_UPLOAD.md)
|
|
- [Path Examples](docs/PATH_EXAMPLES.md)
|
|
- [Context Summarization](docs/summarization.md)
|
|
- [Plan Mode](docs/plan_mode_usage.md)
|
|
- [Backend Run/Stop Guide (ZH)](docs/BACKEND_RUN_STOP_ZH.md)
|
|
- [Setup Guide](docs/SETUP.md)
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
See the [LICENSE](../LICENSE) file in the project root.
|
|
|
|
## Contributing
|
|
|
|
See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines.
|
|
|
|
## Position roundtable durable history
|
|
|
|
The position-roundtable workflow stores database-owned recovery snapshots for
|
|
Step-1 intent chat, every position-node chat, the summary/action-plan chats,
|
|
unsent composer drafts, and generated text deliverables. LangGraph checkpoints
|
|
remain the execution source, but SQL history continues to render after a
|
|
checkpoint or sandbox-output volume is replaced. Apply Alembic revision
|
|
`20260723_05` (or simply upgrade to `head`) before deploying this version.
|
|
|
|
Its live intent conversation still uses the shared `/api/intent/init` and
|
|
`/api/intent/stream` endpoints used by the original multi-agent roundtable.
|
|
Position roundtable identifies itself with `intent_agent_id: "position-roundtable-intent"`;
|
|
this is a separately seeded built-in agent, not an alias of `roundtable-intent`.
|
|
It preserves the same thread, SSE streaming, and file-reading lifecycle as the
|
|
generic Step-1 agent, but has its own **at-most-five user-assistance-round**
|
|
brake. A round is one AI turn plus the user reply, not one tool call: the
|
|
dedicated agent may emit several independent `ask_clarification` calls in the
|
|
same turn, which the frontend renders as several separate selection cards and
|
|
submits together. A topic-only request is not considered resolved: the agent
|
|
asks for every independent missing decision (such as scope, decision use, or
|
|
task constraints) as separate cards. For the dedicated agent, a fresh
|
|
`ask_clarification` emitted in a stream is a strict user-input pause and wins over any same-run premature
|
|
`[INTENT_READY]`; a historical clarification from the prior turn does not
|
|
block the user answer from completing. Its final
|
|
`[INTENT_READY]` JSON contains four string arrays: `coreGoals`, `riskWarnings`,
|
|
`keyPoints`, and `strategicSignificance`; it must contain completed conclusions,
|
|
not “待确认” or other unresolved placeholders. The multi-agent endpoint remains
|
|
backwards compatible when this optional field is omitted and continues to emit
|
|
`objective`, `constraints`, and `assumptions`.
|
|
`/api/intent/init` creates its empty thread directly through the in-process
|
|
thread store/checkpointer, matching multi-agent initialization and avoiding a
|
|
deployment-sensitive `127.0.0.1:<gateway-port>` loopback. If initial checkpoint
|
|
creation fails or is cancelled, bootstrap removes both the partial checkpoint and
|
|
its thread metadata row. Thread metadata creation issues no post-commit ORM
|
|
read-back on any code path: all returned fields are assigned before commit, and
|
|
skipping the extra read keeps a MySQL read/write-splitting endpoint from sending
|
|
it to a lagging replica (which fails the request with
|
|
`Could not refresh instance`). Only the
|
|
recovery snapshot layer is position-specific. Deployments must expose
|
|
either `/api/position-roundtable/*` or the compatibility mirror
|
|
`/api/multi-agent/position-roundtable/*`; updated clients probe both and only
|
|
retry when the first response is the framework/proxy-level `404 Not Found`.
|
|
|
|
## Position-roundtable role directory
|
|
|
|
The position-roundtable work views are persisted separately from organisation
|
|
positions/RBAC in `position_roles`. Gateway startup idempotently inserts the
|
|
four built-in ids (testing analysis, organisation test, solution design and
|
|
comprehensive test), but only when an id is missing: administrator changes to
|
|
the name, order, enabled state or role type are never overwritten.
|
|
|
|
`GET /api/position-roles` is consumed by the workbench and `PUT
|
|
/api/position-roles` is administrator-only. Exactly one enabled `intent` role
|
|
is required. It is the initial intent-identification workspace and owns the
|
|
summary/action-plan closing agents. Built-in rows retain stable ids so
|
|
historical chain nodes remain readable; they can be edited or disabled but not
|
|
deleted. Custom roles may be freely added or removed.
|
|
|
|
## Roundtable artifact submission status
|
|
|
|
The task-scoped roundtable workbench exposes a lightweight submission-status
|
|
API for generated deliverables:
|
|
|
|
- `GET /api/roundtable/tasks/{taskId}/artifacts?submitted=true` returns the
|
|
submitted artifact references for a task (or `items: []`).
|
|
- `PUT /api/roundtable/tasks/{taskId}/artifacts/submission` records
|
|
`submitted: true` or cancels it with `submitted: false`.
|
|
|
|
Both routes use the normal Gateway `Authorization: Bearer <token>` middleware.
|
|
The record is shared by `taskId`; the authenticated user is retained only in
|
|
audit columns. Its server-owned idempotency key is
|
|
`taskId + sourceKey + path + revision`. Repeating the same submitted payload
|
|
returns the existing record; cancellation updates its state and timestamp
|
|
without deleting the audit row. DeerFlow stores only the artifact reference
|
|
(`threadId` + sandbox `path`, plus the optional session/node/name/MIME fields),
|
|
never the Markdown body and never a task-system delivery. Continue to preview
|
|
the file through the existing thread-artifact endpoint.
|