wsk/deerflow-code

0 Fork 0

Files

T

fengxiangdebaozi cb2edeadb4 文档修改

2026-05-13 16:04:05 +08:00

23 KiB

Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

DeerFlow is a full-stack AI agent system combining:

Frontend: React/Vite web interface for interactive agent conversations
Backend: LangGraph-based agent runtime with sandbox execution, persistent memory, and subagent delegation

This is a monorepo with frontend and backend components running in parallel.

Architecture

┌────────────────────────────────────────────────────────┐
│                  DeerFlow Application                  │
├────────────────────────────────────────────────────────┤
│                    Nginx (Port 2026)                   │
│              (Unified reverse proxy)                   │
└─────────────────┬──────────────────┬──────────────────┘
                  │                  │
    /api/langgraph/* (Gateway)      / (Frontend + /api/*)
           │                              │
    ┌──────▼──────────┐        ┌─────────▼────────────┐
    │  Gateway API    │        │  Frontend Web UI     │
    │   (Port 8001)   │        │   (Port 3000)        │
    │                 │        │                      │
    │  - LangGraph    │        │ - React 19 + Vite   │
    │  - FastAPI      │        │ - TypeScript         │
    │  - Tools        │        │ - Tailwind CSS       │
    │  - Memory       │        │                      │
    └─────────────────┘        └──────────────────────┘
            │
    ┌───────▼──────────────────┐
    │  Agent Runtime (LangGraph)│
    │  ┌──────────────────────┐ │
    │  │    Lead Agent        │ │
    │  │  ┌────────────────┐  │ │
    │  │  │  Middlewares   │  │ │
    │  │  │  (10 total)    │  │ │
    │  │  └────────────────┘  │ │
    │  │  ┌────────────────┐  │ │
    │  │  │  Tools         │  │ │
    │  │  │  - Sandbox     │  │ │
    │  │  │  - MCP         │  │ │
    │  │  │  - Built-in    │  │ │
    │  │  │  - Subagents   │  │ │
    │  │  └────────────────┘  │ │
    │  └──────────────────────┘ │
    └───────────────────────────┘

Request Routing:

/api/langgraph/* → Gateway (LangGraph runtime)
/api/* (other) → Gateway (FastAPI REST endpoints)
/ → Frontend (React web interface)

Project Structure

deerflow-server-master/
├── CLAUDE.md                          # This file (root guidance)
├── .gitignore                         # Git ignore rules
│
├── frontend-web/                      # React/Vite web interface
│   ├── package.json                   # Frontend dependencies
│   ├── package-lock.json
│   ├── vite.config.ts                 # Vite bundler config
│   ├── tsconfig.json                  # TypeScript config
│   ├── tailwind.config.ts             # Tailwind CSS config
│   ├── postcss.config.js              # PostCSS config
│   ├── index.html                     # HTML entry point
│   └── src/
│       ├── App.tsx                    # Root component
│       ├── main.tsx                   # React DOM mount
│       ├── components/
│       │   ├── ai-elements/           # Agent UI components (message, artifact, plan, etc.)
│       │   ├── landing/               # Landing page components
│       │   ├── ui/                    # Base UI components (shadcn/ui styled)
│       │   ├── workspace/             # Main workspace interface
│       │   ├── query-client-provider.tsx
│       │   └── theme-provider.tsx
│       ├── hooks/                     # Custom React hooks
│       ├── lib/                       # Utilities (HTTP client, formatting)
│       ├── pages/                     # Route pages
│       └── styles/                    # Global CSS
│
└── offline-backend-20260512/backend/  # Python FastAPI + LangGraph backend
    ├── CLAUDE.md                      # Detailed backend guidance
    ├── README.md                      # Backend documentation
    ├── Makefile                       # Backend commands
    ├── pyproject.toml                 # Python dependencies + workspace
    ├── langgraph.json                 # LangGraph Studio configuration
    │
    ├── packages/harness/              # deerflow-harness: Publishable agent framework
    │   ├── pyproject.toml
    │   └── deerflow/
    │       ├── agents/                # LangGraph agent system
    │       ├── sandbox/               # Code execution isolation
    │       ├── subagents/             # Task delegation system
    │       ├── tools/                 # Tool ecosystem
    │       ├── models/                # LLM model factory
    │       ├── mcp/                   # Model Context Protocol integration
    │       ├── skills/                # Skills discovery & loading
    │       ├── config/                # Configuration system
    │       ├── community/             # Community tools (Tavily, Jina, Firecrawl)
    │       ├── reflection/            # Dynamic module loading
    │       └── client.py              # Embedded DeerFlowClient
    │
    ├── app/                           # Unpublished application code
    │   ├── gateway/                   # FastAPI Gateway API
    │   │   ├── app.py                 # FastAPI application
    │   │   └── routers/               # REST endpoints
    │   └── channels/                  # IM integrations (Feishu, Slack, Telegram, DingTalk)
    │
    ├── tests/                         # Comprehensive test suite
    ├── docs/                          # Detailed documentation
    └── .deer-flow/                    # Runtime directory (created at startup)
        ├── agents/                    # Custom agent definitions
        ├── users/                     # Per-user data
        │   └── {user_id}/
        │       ├── memory.json        # Memory system
        │       ├── threads/           # Per-thread data
        │       └── agents/            # Per-agent memory
        └── threads/                   # Thread-specific workspace (legacy)

Setup & Installation

Prerequisites

Node.js 18+ and pnpm
Python 3.12+
uv package manager
API keys for LLM providers (OpenAI, Anthropic, etc.)

Quick Start (Full Application)

cd offline-backend-20260512/backend

# Install all dependencies
make install

# Create config from template
cp config.example.yaml config.yaml

# Set your API keys
export OPENAI_API_KEY="your-key-here"
# ... other API keys as needed

# Start all services
make dev

Access at: http://localhost:2026

Development Commands

Full Application (from backend directory)

make install      # Install frontend + backend dependencies
make dev          # Start Gateway + Frontend + Nginx (port 2026)
make gateway      # Run Gateway API only (port 8001)
make test         # Run all tests
make lint         # Lint code
make format       # Format code with ruff

Frontend Only (from frontend-web directory)

pnpm install      # Install frontend dependencies
pnpm dev          # Start Vite dev server (port 3000)
pnpm build        # Build for production
pnpm preview      # Preview production build
pnpm typecheck    # TypeScript type checking

Backend Only (from backend directory)

make install      # Install dependencies
make dev          # Run Gateway API with auto-reload (port 8001)
make gateway      # Run Gateway API (no reload)
make test         # Run tests
make lint         # Check code style
make format       # Auto-format code

Architecture Highlights

Frontend (`frontend-web/`)

Framework: React 19 with TypeScript
Bundler: Vite for fast dev/build
Styling: Tailwind CSS + shadcn/ui components
State Management: React Query (TanStack Query) + local state
Routing: React Router v7
Key Libraries:
- @xyflow/react - Node/edge visualization for agent workflows
- codemirror - Code editing with language support
- sonner - Toast notifications
- motion - Animation library
- katex - Math rendering
- remark-gfm + rehype - Markdown processing

Main Components:

components/ai-elements/ - Agent-specific UI (message, artifact, plan, canvas, etc.)
components/ui/ - Reusable shadcn/ui-style components
components/workspace/ - Main workspace interface
components/landing/ - Public landing page

Backend (`offline-backend-20260512/backend/`)

The backend is split into two layers:

Harness (packages/harness/deerflow/)
- Publishable agent framework package
- Contains: agents, sandbox, tools, models, MCP, skills, config
- Import prefix: from deerflow.*
- Can be published as deerflow-harness PyPI package
App (app/)
- Unpublished application code
- Contains: FastAPI Gateway API, IM channel integrations
- Import prefix: from app.*
- Boundary rule: App imports Harness, but Harness never imports App

Key Systems:

Lead Agent - Single LangGraph agent with 10 middleware components
Sandbox - Per-thread isolated execution (local filesystem or Docker)
Subagents - Background task delegation with 3-concurrent limit
Memory - LLM-powered persistent context across conversations
Tools - Sandbox (bash, read/write), MCP, community (web search), built-in (clarification, image view)
Skills - Domain-specific workflows injected via system prompt
MCP - Model Context Protocol server integration

For detailed backend architecture, see offline-backend-20260512/backend/CLAUDE.md

Configuration

Main Configuration (`config.yaml`)

Place in project root (same level as this file). See offline-backend-20260512/backend/config.example.yaml for full template.

Key sections:

models - LLM configurations (OpenAI, Anthropic, etc.) with thinking/vision flags
tools - Tool definitions with module paths
sandbox - Execution environment provider
skills - Skills directory paths
memory - Memory system (enabled, storage, debounce)
title - Auto-title generation
summarization - Context summarization
subagents - Task delegation (enabled/disabled)

Config values starting with $ are resolved as environment variables (e.g., $OPENAI_API_KEY).

Extensions Configuration (`extensions_config.json`)

Place in project root. Configures MCP servers and skill states:

{
  "mcpServers": {
    "github": {
      "enabled": true,
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {"GITHUB_TOKEN": "$GITHUB_TOKEN"}
    }
  },
  "skills": {
    "pdf-processing": {"enabled": true}
  }
}

Environment Variables

# LLM API Keys
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

# Tool API Keys
export TAVILY_API_KEY="tvly-..."
export GITHUB_TOKEN="ghp_..."

# Configuration paths
export DEER_FLOW_CONFIG_PATH="./config.yaml"
export DEER_FLOW_EXTENSIONS_CONFIG_PATH="./extensions_config.json"

# Tracing (optional)
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="lsv2_pt_..."
export LANGSMITH_PROJECT="myproject"

Important Development Guidelines

Documentation Updates

CRITICAL: Always update relevant documentation after code changes:

Update offline-backend-20260512/backend/README.md for user-facing changes
Update offline-backend-20260512/backend/CLAUDE.md for internal changes
Keep documentation synchronized with code

Testing

Test-Driven Development (TDD) is mandatory for backend:

cd offline-backend-20260512/backend

# Run all tests
make test

# Run specific test
PYTHONPATH=. uv run pytest tests/test_<feature>.py -v

Frontend testing is lighter-weight but should be done for critical components.

Code Style

Backend: ruff (line length: 240, Python 3.12+, double quotes)
Frontend: TypeScript with eslint/prettier defaults

Architecture Boundaries

The Harness ↔ App boundary (enforced by CI tests):

# ✅ Allowed (App → Harness)
from deerflow.agents import make_lead_agent

# ❌ Forbidden (Harness → App)
from app.gateway.routers import ...  # This will fail CI tests

Key Features

1. Sandbox Execution

Per-thread isolated code execution with virtual path mapping:

Local: Filesystem-based execution (default)
Docker: Aio sandbox provider for complete isolation
Tools: bash, ls, read_file, write_file, str_replace
Paths: /mnt/user-data/{workspace,uploads,outputs} → thread-specific physical directories

2. Persistent Memory

LLM-powered context retention across conversations:

Automatic extraction of facts and preferences
Structured storage with confidence scoring
Debounced updates to minimize LLM calls
Injected into agent system prompt for context awareness

See offline-backend-20260512/backend/docs/memory-improvements.md for details.

3. Subagent Delegation

Background task execution with concurrent limits:

Built-in agents: general-purpose (full toolset) and bash (commands)
Max 3 concurrent subagents per turn
15-minute timeout per task
Real-time status updates via SSE

4. Tool Ecosystem

Modular tool system supporting multiple providers:

Sandbox tools: Code execution in isolated environments
Built-in tools: ask_clarification, view_image, present_files
MCP tools: Any Model Context Protocol server (stdio, SSE, HTTP)
Community tools: Tavily (web search), Jina (web fetch), Firecrawl (scraping)
Skills: Domain-specific workflows loaded from SKILL.md files

5. Model Context Protocol (MCP)

Integrate external tools and services:

Transports: stdio, SSE, HTTP (with OAuth support)
Configuration: extensions_config.json for server setup
Runtime updates: Changes detected and reloaded automatically

6. Plan Mode

TodoList middleware for complex multi-step tasks:

Controlled via runtime config: is_plan_mode = True
Provides write_todos tool for task tracking
Real-time task status updates

See offline-backend-20260512/backend/docs/plan_mode_usage.md.

7. Vision Support

For models with supports_vision: true:

ViewImageMiddleware processes images automatically
view_image_tool added to agent toolset
Images converted to base64 and injected into state

8. File Upload

Multi-file upload with automatic document conversion:

Supports: PDF, PPT, Excel, Word documents
Auto-conversion via markitdown
Endpoint: POST /api/threads/{thread_id}/uploads
Files stored in thread-isolated directories

See offline-backend-20260512/backend/docs/FILE_UPLOAD.md.

9. IM Channel Integration

External messaging platform bridges:

Supported: Feishu, Slack, Telegram, DingTalk
Architecture: Channels use langgraph-sdk HTTP client
Streaming: Feishu updates single card in place; Slack/Telegram use wait() response
See offline-backend-20260512/backend/app/channels/ for implementation

10. Context Summarization

Automatic conversation compression when approaching token limits:

Configurable trigger types: tokens, messages, or token fraction
Keeps recent messages while summarizing older ones
See offline-backend-20260512/backend/docs/summarization.md.

Middleware Chain (Backend)

The agent runtime executes 18 middleware components in strict order:

ThreadDataMiddleware - Per-thread isolated directories
UploadsMiddleware - Inject uploaded files
SandboxMiddleware - Acquire sandbox environment
DanglingToolCallMiddleware - Handle incomplete tool responses
LLMErrorHandlingMiddleware - Normalize provider errors
GuardrailMiddleware - Tool call authorization (optional)
SandboxAuditMiddleware - Security audit logging
ToolErrorHandlingMiddleware - Convert exceptions to error messages
SummarizationMiddleware - Context reduction (optional)
TodoListMiddleware - Task tracking (optional, plan mode)
TokenUsageMiddleware - Token tracking (optional)
TitleMiddleware - Auto-generate conversation titles
MemoryMiddleware - Queue for async memory updates
ViewImageMiddleware - Inject vision image data (conditional)
DeferredToolFilterMiddleware - Hide deferred tools (optional)
SubagentLimitMiddleware - Enforce concurrent subagent limit (optional)
LoopDetectionMiddleware - Detect and break tool-call loops
ClarificationMiddleware - Interrupt on clarification requests (must be last)

See offline-backend-20260512/backend/CLAUDE.md for detailed middleware descriptions.

Common Workflows

Adding a New Tool

Define tool in packages/harness/deerflow/tools/ or packages/harness/deerflow/community/
Add to config.yaml tools section with module path
Write unit tests in backend/tests/
Update CLAUDE.md with tool description
Test via make test

Creating a Custom Agent

Add agent metadata to database (agents table)
Create agent directory: .deer-flow/agents/{agent_id}/SOUL.md
Define agent behavior via SOUL.md frontmatter
Access via Gateway API: GET /api/agents/{id}

Configuring MCP Servers

Add server to extensions_config.json:

{
  "mcpServers": {
    "my-server": {
      "enabled": true,
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "my-mcp-package"]
    }
  }
}

Restart Gateway API - MCP tools auto-loaded via lazy initialization
Tools available immediately in agent toolset

Testing in Development

# Backend unit tests
cd offline-backend-20260512/backend
make test

# Run specific test
PYTHONPATH=. uv run pytest tests/test_memory_updater.py -v

# Frontend development
cd frontend-web
pnpm dev  # Access at http://localhost:3000

Port Mapping

Service	Port	Purpose
Frontend	3000	React development server
Gateway API	8001	FastAPI REST endpoints + LangGraph runtime
LangGraph Server	2024	Raw LangGraph protocol (used internally)
Nginx	2026	Unified reverse proxy (public entry point)
Provisioner	8002	Kubernetes sandbox provisioner (optional, Docker dev only)

Important Files & References

Backend Architecture

offline-backend-20260512/backend/CLAUDE.md - Detailed backend guidance
offline-backend-20260512/backend/README.md - User-facing backend documentation
offline-backend-20260512/backend/docs/ARCHITECTURE.md - Deep-dive architecture
offline-backend-20260512/backend/docs/API.md - Gateway API reference
offline-backend-20260512/backend/docs/CONFIGURATION.md - Config schema

Frontend Components

frontend-web/src/components/ai-elements/ - Agent UI components
frontend-web/src/components/ui/ - Reusable UI components
frontend-web/src/components/workspace/ - Main workspace

Configuration

config.example.yaml - Backend configuration template
extensions_config.json - MCP servers and skills
offline-backend-20260512/backend/langgraph.json - LangGraph Studio config

Testing

offline-backend-20260512/backend/tests/ - Test suite
Regression tests: test_docker_sandbox_mode_detection.py, test_provisioner_kubeconfig.py, test_harness_boundary.py

Troubleshooting

Port Already in Use

If a port is already in use, kill the process:

# Find process on port (e.g., 8001)
lsof -i :8001

# Kill process
kill -9 <PID>

Configuration Issues

Verify config.yaml exists in project root
Run: make config-upgrade to auto-merge missing fields
Check config_version in config.yaml matches config.example.yaml

Memory System Issues

Check memory state:

# List memory files
ls -la offline-backend-20260512/backend/.deer-flow/users/

# Reload memory
curl -X POST http://localhost:8001/api/memory/reload

Test Failures

Ensure the test dependency boundary is not violated:

# Run boundary test specifically
cd offline-backend-20260512/backend
PYTHONPATH=. uv run pytest tests/test_harness_boundary.py -v

Technology Stack

Frontend

React 19 - UI framework
TypeScript - Type safety
Vite - Fast bundler
Tailwind CSS - Utility-first styling
React Router - Client routing
TanStack Query - Data fetching
XYFlow - Graph visualization
CodeMirror - Code editor
Shiki - Syntax highlighting
Remark/Rehype - Markdown processing

Backend

LangGraph 1.0.6+ - Agent orchestration framework
LangChain 1.2.3+ - LLM abstractions and tools
FastAPI 0.115.0+ - REST API framework
Uvicorn - ASGI server
Python 3.12+ - Programming language
uv - Fast Python package manager
Ruff - Linting and formatting
Pytest - Testing framework
langchain-mcp-adapters - Model Context Protocol support
markitdown - Document conversion
Tavily/Jina/Firecrawl - Web tools

Contributing

Create a feature branch from master
Make changes and write tests
Run linting and tests:
```
make lint
make test
```
Update relevant CLAUDE.md and README.md files
Create a pull request

CI/CD

GitHub Actions runs:

Backend unit tests (including harness boundary test)
Linting checks
Type checking

All must pass before merging to master.

Support

For issues and feature requests:

Check the detailed docs in offline-backend-20260512/backend/docs/
Review existing tests for usage examples
Ask Claude Code for help with /help command

License & Attribution

See LICENSE file in project root.

For more details on specific systems, see:

Backend: offline-backend-20260512/backend/CLAUDE.md
Backend architecture: offline-backend-20260512/backend/docs/ARCHITECTURE.md
Frontend: Components in frontend-web/src/components/

23 KiB Raw Blame History

CLAUDE.md

Project Overview

Architecture

Project Structure

Setup & Installation

Prerequisites

Quick Start (Full Application)

Development Commands

Full Application (from backend directory)

Frontend Only (from frontend-web directory)

Backend Only (from backend directory)

Architecture Highlights

Frontend (frontend-web/)

Backend (offline-backend-20260512/backend/)

Configuration

Main Configuration (config.yaml)

Extensions Configuration (extensions_config.json)

Environment Variables

Important Development Guidelines

Documentation Updates

Testing

Code Style

Architecture Boundaries

Key Features

1. Sandbox Execution

2. Persistent Memory

3. Subagent Delegation

4. Tool Ecosystem

5. Model Context Protocol (MCP)

6. Plan Mode

7. Vision Support

8. File Upload

9. IM Channel Integration

10. Context Summarization

Middleware Chain (Backend)

Common Workflows

Adding a New Tool

Creating a Custom Agent

Configuring MCP Servers

Testing in Development

Port Mapping

Important Files & References

Backend Architecture

Frontend Components

Configuration

Testing

Troubleshooting

Port Already in Use

Configuration Issues

Memory System Issues

Test Failures

Technology Stack

Frontend

Backend

Contributing

CI/CD

Support

License & Attribution

23 KiB

Raw Blame History

Frontend (`frontend-web/`)

Backend (`offline-backend-20260512/backend/`)

Main Configuration (`config.yaml`)

Extensions Configuration (`extensions_config.json`)