文档修改

2026-05-13 16:04:05 +08:00
1 parent 309c864096
commit cb2edeadb4
@@ -0,0 +1,672 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+DeerFlow is a full-stack AI agent system combining:
+- **Frontend**: React/Vite web interface for interactive agent conversations
+- **Backend**: LangGraph-based agent runtime with sandbox execution, persistent memory, and subagent delegation
+
+This is a monorepo with frontend and backend components running in parallel.
+
+---
+
+## Architecture
+
+```
+┌────────────────────────────────────────────────────────┐
+│                  DeerFlow Application                  │
+├────────────────────────────────────────────────────────┤
+│                    Nginx (Port 2026)                   │
+│              (Unified reverse proxy)                   │
+└─────────────────┬──────────────────┬──────────────────┘
+                  │                  │
+    /api/langgraph/* (Gateway)      / (Frontend + /api/*)
+           │                              │
+    ┌──────▼──────────┐        ┌─────────▼────────────┐
+    │  Gateway API    │        │  Frontend Web UI     │
+    │   (Port 8001)   │        │   (Port 3000)        │
+    │                 │        │                      │
+    │  - LangGraph    │        │ - React 19 + Vite   │
+    │  - FastAPI      │        │ - TypeScript         │
+    │  - Tools        │        │ - Tailwind CSS       │
+    │  - Memory       │        │                      │
+    └─────────────────┘        └──────────────────────┘
+            │
+    ┌───────▼──────────────────┐
+    │  Agent Runtime (LangGraph)│
+    │  ┌──────────────────────┐ │
+    │  │    Lead Agent        │ │
+    │  │  ┌────────────────┐  │ │
+    │  │  │  Middlewares   │  │ │
+    │  │  │  (10 total)    │  │ │
+    │  │  └────────────────┘  │ │
+    │  │  ┌────────────────┐  │ │
+    │  │  │  Tools         │  │ │
+    │  │  │  - Sandbox     │  │ │
+    │  │  │  - MCP         │  │ │
+    │  │  │  - Built-in    │  │ │
+    │  │  │  - Subagents   │  │ │
+    │  │  └────────────────┘  │ │
+    │  └──────────────────────┘ │
+    └───────────────────────────┘
+```
+
+**Request Routing**:
+- `/api/langgraph/*` → Gateway (LangGraph runtime)
+- `/api/*` (other) → Gateway (FastAPI REST endpoints)
+- `/` → Frontend (React web interface)
+
+---
+
+## Project Structure
+
+```
+deerflow-server-master/
+├── CLAUDE.md                          # This file (root guidance)
+├── .gitignore                         # Git ignore rules
+│
+├── frontend-web/                      # React/Vite web interface
+│   ├── package.json                   # Frontend dependencies
+│   ├── package-lock.json
+│   ├── vite.config.ts                 # Vite bundler config
+│   ├── tsconfig.json                  # TypeScript config
+│   ├── tailwind.config.ts             # Tailwind CSS config
+│   ├── postcss.config.js              # PostCSS config
+│   ├── index.html                     # HTML entry point
+│   └── src/
+│       ├── App.tsx                    # Root component
+│       ├── main.tsx                   # React DOM mount
+│       ├── components/
+│       │   ├── ai-elements/           # Agent UI components (message, artifact, plan, etc.)
+│       │   ├── landing/               # Landing page components
+│       │   ├── ui/                    # Base UI components (shadcn/ui styled)
+│       │   ├── workspace/             # Main workspace interface
+│       │   ├── query-client-provider.tsx
+│       │   └── theme-provider.tsx
+│       ├── hooks/                     # Custom React hooks
+│       ├── lib/                       # Utilities (HTTP client, formatting)
+│       ├── pages/                     # Route pages
+│       └── styles/                    # Global CSS
+│
+└── offline-backend-20260512/backend/  # Python FastAPI + LangGraph backend
+    ├── CLAUDE.md                      # Detailed backend guidance
+    ├── README.md                      # Backend documentation
+    ├── Makefile                       # Backend commands
+    ├── pyproject.toml                 # Python dependencies + workspace
+    ├── langgraph.json                 # LangGraph Studio configuration
+    │
+    ├── packages/harness/              # deerflow-harness: Publishable agent framework
+    │   ├── pyproject.toml
+    │   └── deerflow/
+    │       ├── agents/                # LangGraph agent system
+    │       ├── sandbox/               # Code execution isolation
+    │       ├── subagents/             # Task delegation system
+    │       ├── tools/                 # Tool ecosystem
+    │       ├── models/                # LLM model factory
+    │       ├── mcp/                   # Model Context Protocol integration
+    │       ├── skills/                # Skills discovery & loading
+    │       ├── config/                # Configuration system
+    │       ├── community/             # Community tools (Tavily, Jina, Firecrawl)
+    │       ├── reflection/            # Dynamic module loading
+    │       └── client.py              # Embedded DeerFlowClient
+    │
+    ├── app/                           # Unpublished application code
+    │   ├── gateway/                   # FastAPI Gateway API
+    │   │   ├── app.py                 # FastAPI application
+    │   │   └── routers/               # REST endpoints
+    │   └── channels/                  # IM integrations (Feishu, Slack, Telegram, DingTalk)
+    │
+    ├── tests/                         # Comprehensive test suite
+    ├── docs/                          # Detailed documentation
+    └── .deer-flow/                    # Runtime directory (created at startup)
+        ├── agents/                    # Custom agent definitions
+        ├── users/                     # Per-user data
+        │   └── {user_id}/
+        │       ├── memory.json        # Memory system
+        │       ├── threads/           # Per-thread data
+        │       └── agents/            # Per-agent memory
+        └── threads/                   # Thread-specific workspace (legacy)
+```
+
+---
+
+## Setup & Installation
+
+### Prerequisites
+
+- Node.js 18+ and pnpm
+- Python 3.12+
+- uv package manager
+- API keys for LLM providers (OpenAI, Anthropic, etc.)
+
+### Quick Start (Full Application)
+
+```bash
+cd offline-backend-20260512/backend
+
+# Install all dependencies
+make install
+
+# Create config from template
+cp config.example.yaml config.yaml
+
+# Set your API keys
+export OPENAI_API_KEY="your-key-here"
+# ... other API keys as needed
+
+# Start all services
+make dev
+```
+
+Access at: **http://localhost:2026**
+
+---
+
+## Development Commands
+
+### Full Application (from backend directory)
+
+```bash
+make install      # Install frontend + backend dependencies
+make dev          # Start Gateway + Frontend + Nginx (port 2026)
+make gateway      # Run Gateway API only (port 8001)
+make test         # Run all tests
+make lint         # Lint code
+make format       # Format code with ruff
+```
+
+### Frontend Only (from frontend-web directory)
+
+```bash
+pnpm install      # Install frontend dependencies
+pnpm dev          # Start Vite dev server (port 3000)
+pnpm build        # Build for production
+pnpm preview      # Preview production build
+pnpm typecheck    # TypeScript type checking
+```
+
+### Backend Only (from backend directory)
+
+```bash
+make install      # Install dependencies
+make dev          # Run Gateway API with auto-reload (port 8001)
+make gateway      # Run Gateway API (no reload)
+make test         # Run tests
+make lint         # Check code style
+make format       # Auto-format code
+```
+
+---
+
+## Architecture Highlights
+
+### Frontend (`frontend-web/`)
+
+- **Framework**: React 19 with TypeScript
+- **Bundler**: Vite for fast dev/build
+- **Styling**: Tailwind CSS + shadcn/ui components
+- **State Management**: React Query (TanStack Query) + local state
+- **Routing**: React Router v7
+- **Key Libraries**:
+  - `@xyflow/react` - Node/edge visualization for agent workflows
+  - `codemirror` - Code editing with language support
+  - `sonner` - Toast notifications
+  - `motion` - Animation library
+  - `katex` - Math rendering
+  - `remark-gfm` + `rehype` - Markdown processing
+
+**Main Components**:
+- `components/ai-elements/` - Agent-specific UI (message, artifact, plan, canvas, etc.)
+- `components/ui/` - Reusable shadcn/ui-style components
+- `components/workspace/` - Main workspace interface
+- `components/landing/` - Public landing page
+
+### Backend (`offline-backend-20260512/backend/`)
+
+The backend is split into two layers:
+
+1. **Harness** (`packages/harness/deerflow/`)
+   - Publishable agent framework package
+   - Contains: agents, sandbox, tools, models, MCP, skills, config
+   - Import prefix: `from deerflow.*`
+   - Can be published as `deerflow-harness` PyPI package
+
+2. **App** (`app/`)
+   - Unpublished application code
+   - Contains: FastAPI Gateway API, IM channel integrations
+   - Import prefix: `from app.*`
+   - **Boundary rule**: App imports Harness, but Harness never imports App
+
+**Key Systems**:
+- **Lead Agent** - Single LangGraph agent with 10 middleware components
+- **Sandbox** - Per-thread isolated execution (local filesystem or Docker)
+- **Subagents** - Background task delegation with 3-concurrent limit
+- **Memory** - LLM-powered persistent context across conversations
+- **Tools** - Sandbox (bash, read/write), MCP, community (web search), built-in (clarification, image view)
+- **Skills** - Domain-specific workflows injected via system prompt
+- **MCP** - Model Context Protocol server integration
+
+**For detailed backend architecture, see `offline-backend-20260512/backend/CLAUDE.md`**
+
+---
+
+## Configuration
+
+### Main Configuration (`config.yaml`)
+
+Place in project root (same level as this file). See `offline-backend-20260512/backend/config.example.yaml` for full template.
+
+Key sections:
+- `models` - LLM configurations (OpenAI, Anthropic, etc.) with thinking/vision flags
+- `tools` - Tool definitions with module paths
+- `sandbox` - Execution environment provider
+- `skills` - Skills directory paths
+- `memory` - Memory system (enabled, storage, debounce)
+- `title` - Auto-title generation
+- `summarization` - Context summarization
+- `subagents` - Task delegation (enabled/disabled)
+
+Config values starting with `$` are resolved as environment variables (e.g., `$OPENAI_API_KEY`).
+
+### Extensions Configuration (`extensions_config.json`)
+
+Place in project root. Configures MCP servers and skill states:
+
+```json
+{
+  "mcpServers": {
+    "github": {
+      "enabled": true,
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@modelcontextprotocol/server-github"],
+      "env": {"GITHUB_TOKEN": "$GITHUB_TOKEN"}
+    }
+  },
+  "skills": {
+    "pdf-processing": {"enabled": true}
+  }
+}
+```
+
+### Environment Variables
+
+```bash
+# LLM API Keys
+export OPENAI_API_KEY="sk-..."
+export ANTHROPIC_API_KEY="sk-ant-..."
+
+# Tool API Keys
+export TAVILY_API_KEY="tvly-..."
+export GITHUB_TOKEN="ghp_..."
+
+# Configuration paths
+export DEER_FLOW_CONFIG_PATH="./config.yaml"
+export DEER_FLOW_EXTENSIONS_CONFIG_PATH="./extensions_config.json"
+
+# Tracing (optional)
+export LANGSMITH_TRACING=true
+export LANGSMITH_API_KEY="lsv2_pt_..."
+export LANGSMITH_PROJECT="myproject"
+```
+
+---
+
+## Important Development Guidelines
+
+### Documentation Updates
+
+**CRITICAL**: Always update relevant documentation after code changes:
+- Update `offline-backend-20260512/backend/README.md` for user-facing changes
+- Update `offline-backend-20260512/backend/CLAUDE.md` for internal changes
+- Keep documentation synchronized with code
+
+### Testing
+
+**Test-Driven Development (TDD) is mandatory** for backend:
+```bash
+cd offline-backend-20260512/backend
+
+# Run all tests
+make test
+
+# Run specific test
+PYTHONPATH=. uv run pytest tests/test_<feature>.py -v
+```
+
+Frontend testing is lighter-weight but should be done for critical components.
+
+### Code Style
+
+- **Backend**: `ruff` (line length: 240, Python 3.12+, double quotes)
+- **Frontend**: TypeScript with eslint/prettier defaults
+
+### Architecture Boundaries
+
+The **Harness ↔ App boundary** (enforced by CI tests):
+```python
+# ✅ Allowed (App → Harness)
+from deerflow.agents import make_lead_agent
+
+# ❌ Forbidden (Harness → App)
+from app.gateway.routers import ...  # This will fail CI tests
+```
+
+---
+
+## Key Features
+
+### 1. Sandbox Execution
+
+Per-thread isolated code execution with virtual path mapping:
+- **Local**: Filesystem-based execution (default)
+- **Docker**: Aio sandbox provider for complete isolation
+- **Tools**: `bash`, `ls`, `read_file`, `write_file`, `str_replace`
+- **Paths**: `/mnt/user-data/{workspace,uploads,outputs}` → thread-specific physical directories
+
+### 2. Persistent Memory
+
+LLM-powered context retention across conversations:
+- Automatic extraction of facts and preferences
+- Structured storage with confidence scoring
+- Debounced updates to minimize LLM calls
+- Injected into agent system prompt for context awareness
+
+See `offline-backend-20260512/backend/docs/memory-improvements.md` for details.
+
+### 3. Subagent Delegation
+
+Background task execution with concurrent limits:
+- Built-in agents: `general-purpose` (full toolset) and `bash` (commands)
+- Max 3 concurrent subagents per turn
+- 15-minute timeout per task
+- Real-time status updates via SSE
+
+### 4. Tool Ecosystem
+
+Modular tool system supporting multiple providers:
+- **Sandbox tools**: Code execution in isolated environments
+- **Built-in tools**: `ask_clarification`, `view_image`, `present_files`
+- **MCP tools**: Any Model Context Protocol server (stdio, SSE, HTTP)
+- **Community tools**: Tavily (web search), Jina (web fetch), Firecrawl (scraping)
+- **Skills**: Domain-specific workflows loaded from SKILL.md files
+
+### 5. Model Context Protocol (MCP)
+
+Integrate external tools and services:
+- **Transports**: stdio, SSE, HTTP (with OAuth support)
+- **Configuration**: `extensions_config.json` for server setup
+- **Runtime updates**: Changes detected and reloaded automatically
+
+### 6. Plan Mode
+
+TodoList middleware for complex multi-step tasks:
+- Controlled via runtime config: `is_plan_mode = True`
+- Provides `write_todos` tool for task tracking
+- Real-time task status updates
+
+See `offline-backend-20260512/backend/docs/plan_mode_usage.md`.
+
+### 7. Vision Support
+
+For models with `supports_vision: true`:
+- `ViewImageMiddleware` processes images automatically
+- `view_image_tool` added to agent toolset
+- Images converted to base64 and injected into state
+
+### 8. File Upload
+
+Multi-file upload with automatic document conversion:
+- Supports: PDF, PPT, Excel, Word documents
+- Auto-conversion via `markitdown`
+- Endpoint: `POST /api/threads/{thread_id}/uploads`
+- Files stored in thread-isolated directories
+
+See `offline-backend-20260512/backend/docs/FILE_UPLOAD.md`.
+
+### 9. IM Channel Integration
+
+External messaging platform bridges:
+- **Supported**: Feishu, Slack, Telegram, DingTalk
+- **Architecture**: Channels use `langgraph-sdk` HTTP client
+- **Streaming**: Feishu updates single card in place; Slack/Telegram use `wait()` response
+- See `offline-backend-20260512/backend/app/channels/` for implementation
+
+### 10. Context Summarization
+
+Automatic conversation compression when approaching token limits:
+- Configurable trigger types: tokens, messages, or token fraction
+- Keeps recent messages while summarizing older ones
+- See `offline-backend-20260512/backend/docs/summarization.md`.
+
+---
+
+## Middleware Chain (Backend)
+
+The agent runtime executes 18 middleware components in strict order:
+
+1. **ThreadDataMiddleware** - Per-thread isolated directories
+2. **UploadsMiddleware** - Inject uploaded files
+3. **SandboxMiddleware** - Acquire sandbox environment
+4. **DanglingToolCallMiddleware** - Handle incomplete tool responses
+5. **LLMErrorHandlingMiddleware** - Normalize provider errors
+6. **GuardrailMiddleware** - Tool call authorization (optional)
+7. **SandboxAuditMiddleware** - Security audit logging
+8. **ToolErrorHandlingMiddleware** - Convert exceptions to error messages
+9. **SummarizationMiddleware** - Context reduction (optional)
+10. **TodoListMiddleware** - Task tracking (optional, plan mode)
+11. **TokenUsageMiddleware** - Token tracking (optional)
+12. **TitleMiddleware** - Auto-generate conversation titles
+13. **MemoryMiddleware** - Queue for async memory updates
+14. **ViewImageMiddleware** - Inject vision image data (conditional)
+15. **DeferredToolFilterMiddleware** - Hide deferred tools (optional)
+16. **SubagentLimitMiddleware** - Enforce concurrent subagent limit (optional)
+17. **LoopDetectionMiddleware** - Detect and break tool-call loops
+18. **ClarificationMiddleware** - Interrupt on clarification requests (must be last)
+
+See `offline-backend-20260512/backend/CLAUDE.md` for detailed middleware descriptions.
+
+---
+
+## Common Workflows
+
+### Adding a New Tool
+
+1. Define tool in `packages/harness/deerflow/tools/` or `packages/harness/deerflow/community/`
+2. Add to `config.yaml` `tools` section with module path
+3. Write unit tests in `backend/tests/`
+4. Update `CLAUDE.md` with tool description
+5. Test via `make test`
+
+### Creating a Custom Agent
+
+1. Add agent metadata to database (`agents` table)
+2. Create agent directory: `.deer-flow/agents/{agent_id}/SOUL.md`
+3. Define agent behavior via SOUL.md frontmatter
+4. Access via Gateway API: `GET /api/agents/{id}`
+
+### Configuring MCP Servers
+
+1. Add server to `extensions_config.json`:
+```json
+{
+  "mcpServers": {
+    "my-server": {
+      "enabled": true,
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "my-mcp-package"]
+    }
+  }
+}
+```
+
+2. Restart Gateway API - MCP tools auto-loaded via lazy initialization
+3. Tools available immediately in agent toolset
+
+### Testing in Development
+
+```bash
+# Backend unit tests
+cd offline-backend-20260512/backend
+make test
+
+# Run specific test
+PYTHONPATH=. uv run pytest tests/test_memory_updater.py -v
+
+# Frontend development
+cd frontend-web
+pnpm dev  # Access at http://localhost:3000
+```
+
+---
+
+## Port Mapping
+
+| Service | Port | Purpose |
+|---------|------|---------|
+| **Frontend** | 3000 | React development server |
+| **Gateway API** | 8001 | FastAPI REST endpoints + LangGraph runtime |
+| **LangGraph Server** | 2024 | Raw LangGraph protocol (used internally) |
+| **Nginx** | 2026 | Unified reverse proxy (public entry point) |
+| **Provisioner** | 8002 | Kubernetes sandbox provisioner (optional, Docker dev only) |
+
+---
+
+## Important Files & References
+
+### Backend Architecture
+- `offline-backend-20260512/backend/CLAUDE.md` - Detailed backend guidance
+- `offline-backend-20260512/backend/README.md` - User-facing backend documentation
+- `offline-backend-20260512/backend/docs/ARCHITECTURE.md` - Deep-dive architecture
+- `offline-backend-20260512/backend/docs/API.md` - Gateway API reference
+- `offline-backend-20260512/backend/docs/CONFIGURATION.md` - Config schema
+
+### Frontend Components
+- `frontend-web/src/components/ai-elements/` - Agent UI components
+- `frontend-web/src/components/ui/` - Reusable UI components
+- `frontend-web/src/components/workspace/` - Main workspace
+
+### Configuration
+- `config.example.yaml` - Backend configuration template
+- `extensions_config.json` - MCP servers and skills
+- `offline-backend-20260512/backend/langgraph.json` - LangGraph Studio config
+
+### Testing
+- `offline-backend-20260512/backend/tests/` - Test suite
+- Regression tests: `test_docker_sandbox_mode_detection.py`, `test_provisioner_kubeconfig.py`, `test_harness_boundary.py`
+
+---
+
+## Troubleshooting
+
+### Port Already in Use
+
+If a port is already in use, kill the process:
+```bash
+# Find process on port (e.g., 8001)
+lsof -i :8001
+
+# Kill process
+kill -9 <PID>
+```
+
+### Configuration Issues
+
+1. Verify `config.yaml` exists in project root
+2. Run: `make config-upgrade` to auto-merge missing fields
+3. Check `config_version` in `config.yaml` matches `config.example.yaml`
+
+### Memory System Issues
+
+Check memory state:
+```bash
+# List memory files
+ls -la offline-backend-20260512/backend/.deer-flow/users/
+
+# Reload memory
+curl -X POST http://localhost:8001/api/memory/reload
+```
+
+### Test Failures
+
+Ensure the test dependency boundary is not violated:
+```bash
+# Run boundary test specifically
+cd offline-backend-20260512/backend
+PYTHONPATH=. uv run pytest tests/test_harness_boundary.py -v
+```
+
+---
+
+## Technology Stack
+
+### Frontend
+- **React** 19 - UI framework
+- **TypeScript** - Type safety
+- **Vite** - Fast bundler
+- **Tailwind CSS** - Utility-first styling
+- **React Router** - Client routing
+- **TanStack Query** - Data fetching
+- **XYFlow** - Graph visualization
+- **CodeMirror** - Code editor
+- **Shiki** - Syntax highlighting
+- **Remark/Rehype** - Markdown processing
+
+### Backend
+- **LangGraph** 1.0.6+ - Agent orchestration framework
+- **LangChain** 1.2.3+ - LLM abstractions and tools
+- **FastAPI** 0.115.0+ - REST API framework
+- **Uvicorn** - ASGI server
+- **Python 3.12+** - Programming language
+- **uv** - Fast Python package manager
+- **Ruff** - Linting and formatting
+- **Pytest** - Testing framework
+- **langchain-mcp-adapters** - Model Context Protocol support
+- **markitdown** - Document conversion
+- **Tavily/Jina/Firecrawl** - Web tools
+
+---
+
+## Contributing
+
+1. Create a feature branch from `master`
+2. Make changes and write tests
+3. Run linting and tests:
+   ```bash
+   make lint
+   make test
+   ```
+4. Update relevant CLAUDE.md and README.md files
+5. Create a pull request
+
+### CI/CD
+
+GitHub Actions runs:
+- Backend unit tests (including harness boundary test)
+- Linting checks
+- Type checking
+
+All must pass before merging to `master`.
+
+---
+
+## Support
+
+For issues and feature requests:
+- Check the detailed docs in `offline-backend-20260512/backend/docs/`
+- Review existing tests for usage examples
+- Ask Claude Code for help with `/help` command
+
+---
+
+## License & Attribution
+
+See LICENSE file in project root.
+
+For more details on specific systems, see:
+- Backend: `offline-backend-20260512/backend/CLAUDE.md`
+- Backend architecture: `offline-backend-20260512/backend/docs/ARCHITECTURE.md`
+- Frontend: Components in `frontend-web/src/components/`