refactor(memory): Memory V2 P3 - 舍弃后台抽取 + Gateway 端点收口

P3 在 P1/P2 基础上完成 Memory V2 收尾:

舍弃后台抽取:
- 删除 updater/queue/storage/prompt/summarization_hook 五个模块
- 移除 background_extraction 配置段、扁平字段兼容、storage 字段
- 理由:Hindsight auto_retain 已自动留存每轮对话,后台抽取冗余

Hindsight 召回注入改用 wrap_model_call:
- 由 before_model 改为 wrap_model_call —— 召回只注入单次模型请求、
  不落持久 state,从根上消除 <memory-context> 泄漏给前端的风险
- builtin frozen snapshot 仍走系统提示词

Gateway 端点收口为 /api/memory/v2*:
- 删除旧 facts/history 端点 (/api/memory、/memory/facts 等)
- 新增 entries 增删改 (POST/PUT/DELETE /memory/v2/entries)
- 新增 POST /memory/v2/search —— Hindsight recall/reflect 检索代理
- client.py memory 方法重写为 V2 等价方法

文档同步更新 CLAUDE.md 与 MEMORY_V2_DESIGN_ZH.md。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

offline-backend-20260512/backend/CLAUDE.md

@@ -31,7 +31,7 @@ deer-flow/
 │   │           ├── agents/            # LangGraph agent system
 │   │           │   ├── lead_agent/    # Main agent (factory + system prompt)
 │   │           │   ├── middlewares/   # 10 middleware components
-│   │           │   ├── memory/        # Memory extraction, queue, prompts
+│   │           │   ├── memory/        # Memory V2: provider/manager/builtin/hindsight
 │   │           │   └── thread_state.py # ThreadState schema
 │   │           ├── sandbox/           # Sandbox execution system
 │   │           │   ├── local/         # Local filesystem provider
@@ -167,7 +167,7 @@ Lead-agent middlewares are assembled in strict append order across `packages/har
 10. **TodoListMiddleware** - Task tracking with `write_todos` tool (optional, if plan_mode)
 11. **TokenUsageMiddleware** - Records token usage metrics when token tracking is enabled (optional)
 12. **TitleMiddleware** - Auto-generates thread title after first complete exchange and normalizes structured message content before prompting the title model
-13. **MemoryMiddleware** - Queues conversations for async memory update (filters to user + final AI responses)
+13. **MemoryMiddleware** - Memory V2: `wrap_model_call` injects Hindsight recall transiently; `after_agent` auto-retains the turn to Hindsight
 14. **ViewImageMiddleware** - Injects base64 image data before LLM call (conditional on vision support)
 15. **DeferredToolFilterMiddleware** - Hides deferred tool schemas from the bound model until tool search is enabled (optional)
 16. **SubagentLimitMiddleware** - Truncates excess `task` tool calls from model response to enforce `MAX_CONCURRENT_SUBAGENTS` limit (optional, if `subagent_enabled`)
@@ -214,7 +214,7 @@ FastAPI application on port 8001 with health check at `GET /health`. Set `GATEWA
 | **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details |
 | **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) |
 | **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
-| **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data |
+| **Memory V2** (`/api/memory/v2`) | `GET /` - builtin data (USER.md/MEMORY.md); `POST/PUT/DELETE /entries` - add/replace/remove entry; `GET/PATCH/DELETE /config` - effective config + per-user overrides; `POST /search` - Hindsight recall/reflect proxy |
 | **Agents** (`/api/agents`) | `GET /` - list built-in, owned, and published agents; `POST /` - create user-owned agent; `GET/PUT/DELETE /{id}` - read/update/delete by stable agent id; `GET /check` - validate non-empty display name (duplicates allowed) |
 | **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); `GET /list` - list; `DELETE /{filename}` - delete |
 | **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
@@ -348,43 +348,46 @@ Bridges external messaging platforms (Feishu, Slack, Telegram, DingTalk) to the
 - Per-channel configs: `feishu` (app_id, app_secret), `slack` (bot_token, app_token), `telegram` (bot_token), `dingtalk` (client_id, client_secret, optional `card_template_id` for AI Card streaming)
 
 
-### Memory System (`packages/harness/deerflow/agents/memory/`)
+### Memory System — Memory V2 (`packages/harness/deerflow/agents/memory/`)
+
+Pluggable memory architecture (Hermes-style). See `docs/MEMORY_V2_DESIGN_ZH.md`
+and `docs/HINDSIGHT_SETUP_ZH.md` for the full design.
 
 **Components**:
-- `updater.py` - LLM-based memory updates with fact extraction, whitespace-normalized fact deduplication (trims leading/trailing whitespace before comparing), and atomic file I/O
-- `queue.py` - Debounced update queue (per-thread deduplication, configurable wait time); captures `user_id` at enqueue time so it survives the `threading.Timer` boundary
-- `prompt.py` - Prompt templates for memory updates
-- `storage.py` - File-based storage with per-user isolation; cache keyed by `(user_id, agent_name)` tuple
+- `provider.py` - `MemoryProvider` abstract base (12 lifecycle/hook methods)
+- `manager.py` - `MemoryManager` orchestrates builtin + ≤1 external provider; `build_memory_manager()` is the construction entry point
+- `providers/builtin.py` - `BuiltinFileProvider`: local USER.md / MEMORY.md, § delimiter, char quotas, file locking, atomic writes, frozen snapshot
+- `providers/hindsight.py` - `HindsightProvider`: HTTP client to a self-hosted Hindsight Docker instance, dual-bank routing
+- `security.py` - injection/exfiltration scan + invisible-unicode block for memory content
+- `message_processing.py` - `filter_messages_for_memory` (last-turn extraction for Hindsight retain)
 
-**Per-User Isolation**:
-- Memory is stored per-user at `{base_dir}/users/{user_id}/memory.json`
-- Per-agent per-user memory at `{base_dir}/users/{user_id}/agents/{agent_name}/memory.json`
-- `user_id` is resolved via `get_effective_user_id()` from `deerflow.runtime.user_context`
-- In no-auth mode, `user_id` defaults to `"default"` (constant `DEFAULT_USER_ID`)
-- Absolute `storage_path` in config opts out of per-user isolation
-- **Migration**: Run `PYTHONPATH=. python scripts/migrate_user_isolation.py` to move legacy `memory.json` and `threads/` into per-user layout; supports `--dry-run`
+**Two stores (binary classification)**:
+- `USER.md` at `{base_dir}/users/{user_id}/USER.md` — user profile, shared across agents
+- `MEMORY.md` at `{base_dir}/users/{user_id}/agents/{agent_id}/MEMORY.md` — agent-private working notes
+- `user_id` resolved via `get_effective_user_id()`; `"default"` in no-auth mode
 
-**Data Structure** (stored in `{base_dir}/users/{user_id}/memory.json`):
-- **User Context**: `workContext`, `personalContext`, `topOfMind` (1-3 sentence summaries)
-- **History**: `recentMonths`, `earlierContext`, `longTermBackground`
-- **Facts**: Discrete facts with `id`, `content`, `category` (preference/knowledge/context/behavior/goal), `confidence` (0-1), `createdAt`, `source`
+**Write paths**:
+- LLM actively calls the `memory` tool (`tools/builtins/memory_tool.py`) → writes USER.md/MEMORY.md, mirrors to Hindsight
+- LLM calls `hindsight_recall`/`reflect`/`retain` tools (`tools/builtins/hindsight_tools.py`) — only in `tools`/`hybrid` memory_mode
+- Hindsight `auto_retain` — every turn auto-stored to the work bank (via `MemoryMiddleware.after_agent`)
 
-**Workflow**:
-1. `MemoryMiddleware` filters messages (user inputs + final AI responses), captures `user_id` via `get_effective_user_id()`, and queues conversation with the captured `user_id`
-2. Queue debounces (30s default), batches updates, deduplicates per-thread
-3. Background thread invokes LLM to extract context updates and facts, using the stored `user_id` (not the contextvar, which is unavailable on timer threads)
-4. Applies updates atomically (temp file + rename) with cache invalidation, skipping duplicate fact content before append
-5. Next interaction injects top 15 facts + context into `<memory>` tags in system prompt
+**Injection**:
+- builtin frozen snapshot → injected into the system prompt via `lead_agent/prompt.py` `_get_memory_context`
+- Hindsight dynamic recall → injected per model call via `MemoryMiddleware.wrap_model_call` (transient — not persisted to state, so it never leaks to the frontend)
 
-Focused regression coverage for the updater lives in `backend/tests/test_memory_updater.py`.
+**Hindsight dual-bank isolation** (B-scheme):
+- global user bank `deerflow-user-{user_id}` — cross-agent
+- agent work bank `deerflow-work-{user_id}-{agent_id}` — per-agent
+- `hindsight-client` is an optional dependency; when missing, `HindsightProvider.is_available()` returns False and the system falls back to builtin-only
 
-**Configuration** (`config.yaml` → `memory`):
-- `enabled` / `injection_enabled` - Master switches
-- `storage_path` - Path to memory.json (absolute path opts out of per-user isolation)
-- `debounce_seconds` - Wait time before processing (default: 30)
-- `model_name` - LLM for updates (null = default model)
-- `max_facts` / `fact_confidence_threshold` - Fact storage limits (100 / 0.7)
-- `max_injection_tokens` - Token limit for prompt injection (2000)
+**Migration**: `PYTHONPATH=. python scripts/migrate_memory_to_v2.py [--all|--user-id X] [--dry-run]` converts legacy `memory.json` into USER.md/MEMORY.md.
+
+**Configuration** (`config.yaml` → `memory`, nested): `enabled`, `provider` (builtin/hindsight),
+`builtin.{memory_char_limit,user_char_limit,...}`, `hindsight.{mode,api_url,api_key,bank_id_template,work_bank_id_template,memory_mode,...}`,
+`injection.{enabled,max_tokens,include_builtin,include_hindsight,context_tag}`, `security.{scan_content,block_invisible_unicode}`.
+Per-user runtime overrides (whitelisted fields) live at `{base_dir}/users/{user_id}/memory_config.json` — see `get_effective_memory_config()`.
+
+**Gateway API** — see the Memory router below (`/api/memory/v2*`).
 
 ### Reflection System (`packages/harness/deerflow/reflection/`)