feat: 定时任务管理 + 会话只读分享
定时任务: - 管理员可查看/编辑/删除/触发全部用户的定时任务(新增 /api/scheduled-tasks/admin/* 接口与管理页) - 卡死任务恢复:单次执行超 30 分钟超时重试(最多 3 次),耗尽后标记 failed;看门狗回收服务重启遗留的「执行中」任务 会话分享: - 新增「只读链接」分享模式,生成公开免登录的只读对话快照页(/share/:code) - 公开接口 /api/public/shares/* 提供对话快照与图片附件 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
4761677676
commit
e0c1cb6581
45 files changed
+3235
-434
No files matched your search
@@ -214,7 +214,7 @@ FastAPI application on port 8001 with health check at `GET /health`. Set `GATEWA
|
||||
|--------|-----------|
|
||||
| **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details |
|
||||
| **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) |
|
||||
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + `tag_id` filters; each item carries `tags` + owner-authored `detail`); `GET /{name}` - details; `PUT /{name}` - update enabled; `PUT /{name}/detail` - set the long-form `detail` text (custom skill: owner or admin; built-in/public: admin only — materializes a `skills` row on demand); `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
|
||||
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + `tag_id` filters; each item carries `tags` + owner-authored `detail`); `GET /{name}` - details; `PUT /{name}` - update enabled; `PUT /{name}/detail` - set the long-form `detail` text (custom skill: owner or admin; built-in/public: admin only — materializes a `skills` row on demand); `GET /{name}/content` - raw SKILL.md source, restricted to an admin or the publisher (custom-skill owner); other users get 403 and see only description + detail; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
|
||||
| **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data |
|
||||
| **Agents** (`/api/agents`) | `GET /` - list built-in, owned, and published agents (optional `search` + `tag_id` filters; each item carries `tags`); `POST /` - create user-owned agent; `GET/PUT/DELETE /{id}` - read/update/delete by stable agent id; `GET /check` - validate non-empty display name (duplicates allowed) |
|
||||
| **Tags** (`/api/tags`) | `GET /` - list tags (optional `tag_type` filter; open to all); `POST /` - create tag (admin); `PUT /{id}` / `DELETE /{id}` - update/delete tag (admin); `GET /assignments/{target_type}/{target_id}` - tags on a resource; `PUT /assignments/{target_type}/{target_id}` - replace a resource's tags (admin). Tags are typed `agent`/`skill`/`scheduled_task` and may only tag resources of their own type. |
|
||||
@@ -225,9 +225,18 @@ FastAPI application on port 8001 with health check at `GET /health`. Set `GATEWA
|
||||
| **Thread Runs** (`/api/threads/{id}/runs`) | `POST /` - create background run; `POST /stream` - create + SSE stream; `POST /wait` - create + block; `GET /` - list runs; `GET /{rid}` - run details; `POST /{rid}/cancel` - cancel; `GET /{rid}/join` - join SSE; `GET /{rid}/messages` - paginated messages `{data, has_more}`; `GET /{rid}/events` - full event stream; `GET /../messages` - thread messages with feedback; `GET /../token-usage` - aggregate tokens |
|
||||
| **Feedback** (`/api/threads/{id}/runs/{rid}/feedback`) | `PUT /` - upsert feedback; `DELETE /` - delete user feedback; `POST /` - create feedback; `GET /` - list feedback; `GET /stats` - aggregate stats; `DELETE /{fid}` - delete specific |
|
||||
| **Runs** (`/api/runs`) | `POST /stream` - stateless run + SSE; `POST /wait` - stateless run + block; `GET /{rid}/messages` - paginated messages by run_id `{data, has_more}` (cursor: `after_seq`/`before_seq`); `GET /{rid}/feedback` - list feedback by run_id |
|
||||
| **Thread Shares** (`/api/thread-shares`) | Per-conversation sharing with two modes (`thread_shares.mode`). **`import`**: `POST /` `{thread_id, mode}` - create (or reuse) a revocable share code; `GET /{code}/preview` + `POST /{code}/import` - copy that conversation into the caller's account (fresh `thread_id` via `app/gateway/thread_copy.py` + new `threads_meta` row + `user-data` copy; dedup on `metadata.shared_origin`). **`view`**: a public, read-only static page — see Public Shares below. Both modes: `GET /` - list my codes; `DELETE /{code}` - revoke. |
|
||||
| **Public Shares** (`/api/public/shares`) | **Unauthenticated** (the `/api/public/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES`). `GET /{code}` - read-only conversation snapshot (title + serialized messages + artifacts) for a `view`-mode share; messages are read straight from the latest checkpoint. `GET /{code}/files/{path}` - serves an **image** referenced by that conversation (images only — non-image paths 404; SVG forced to download). Missing/revoked/non-`view` codes all return 404. |
|
||||
| **Scheduled Tasks** (`/api/scheduled-tasks`) | Per-user cron-like agent tasks. `GET/POST /` - list/create owned tasks; `PATCH/DELETE /{id}` - update/delete (owner); `POST /{id}/{pause,resume,trigger}`; `GET /{id}/runs`, `GET/DELETE /runs/{rid}`, run-file preview/edit/download; publish/subscribe under `/{id}/{publish,unpublish,subscribe}` and `GET /published`. **Admin (cross-user):** `GET /admin/all` - every task; `PATCH/DELETE /admin/{id}` - edit/delete any task; `POST /admin/{id}/trigger`; `GET /admin/{id}/runs`. Admin routes require `system_role == "admin"`. |
|
||||
|
||||
Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` → Gateway.
|
||||
|
||||
### Scheduled Task Runtime
|
||||
|
||||
`ScheduledTaskService` (`deerflow/runtime/scheduler/service.py`) polls due tasks every `poll_interval_seconds` and executes each in its own throwaway thread, serialized by a shared `_agent_run_lock`.
|
||||
|
||||
**Stuck-run recovery**: each execution attempt is bounded by `_RUN_ATTEMPT_TIMEOUT_SECONDS` (30 min) via `asyncio.wait_for`. On timeout the run is retried up to `_MAX_RUN_RETRIES` (3) times, then marked `failed`. A watchdog (`_reconcile_stuck_runs`, run every poll) recovers runs left in `running` with no in-process owner — e.g. orphaned by a crash or restart — re-driving them through the same retry budget so a task can never stay perpetually "executing". The `scheduled_task_runs.retry_count` column tracks consumed retries.
|
||||
|
||||
### Custom Agent Persistence
|
||||
|
||||
Custom agent metadata lives in the `agents` ORM table (`deerflow.persistence.agents`). `agents.id` is the stable runtime id and the filesystem directory name under `.deer-flow/agents/{id}/`; `agents.name` is display-only, can contain Chinese characters, and is not unique. `user_id IS NULL` marks built-in agents, user-owned agents are visible only to their owner unless `published = true`, and update/delete operations require ownership. The runtime accepts `agent_id` in context/configurable, and non-default `assistant_id` values are treated as agent ids after visibility checks.
|
||||
|
||||
@@ -41,6 +41,7 @@ from app.gateway.routers import (
|
||||
system_settings,
|
||||
tags,
|
||||
thread_runs,
|
||||
thread_shares,
|
||||
threads,
|
||||
uploads,
|
||||
)
|
||||
@@ -701,6 +702,10 @@ This gateway provides custom endpoints for models, MCP configuration, skills, an
|
||||
# Thread Runs API (LangGraph Platform-compatible runs lifecycle)
|
||||
app.include_router(thread_runs.router)
|
||||
|
||||
app.include_router(thread_shares.router)
|
||||
# Public, unauthenticated read-only share snapshots (/api/public/shares/*).
|
||||
app.include_router(thread_shares.public_router)
|
||||
|
||||
# Stateless Runs API (stream/wait without a pre-existing thread)
|
||||
app.include_router(runs.router)
|
||||
|
||||
|
||||
@@ -23,11 +23,16 @@ from app.gateway.internal_auth import INTERNAL_AUTH_HEADER_NAME, get_internal_us
|
||||
from deerflow.runtime.user_context import reset_current_user, set_current_user
|
||||
|
||||
# Paths that never require authentication.
|
||||
#
|
||||
# ``/api/public/`` is a namespace reserved for intentionally world-readable,
|
||||
# read-only endpoints (e.g. public conversation share snapshots). Only mount
|
||||
# safe read-only routes under it — everything there bypasses auth entirely.
|
||||
_PUBLIC_PATH_PREFIXES: tuple[str, ...] = (
|
||||
"/health",
|
||||
"/docs",
|
||||
"/redoc",
|
||||
"/openapi.json",
|
||||
"/api/public/",
|
||||
)
|
||||
|
||||
# Exact auth paths that are public (login/register/status check).
|
||||
|
||||
@@ -122,6 +122,10 @@ async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
|
||||
|
||||
app.state.tag_store = make_tag_store(sf)
|
||||
|
||||
from deerflow.persistence.thread_shares import make_thread_share_store
|
||||
|
||||
app.state.thread_share_store = make_thread_share_store(sf)
|
||||
|
||||
# Run event store (has its own factory with config-driven backend selection)
|
||||
run_events_config = getattr(config, "run_events", None)
|
||||
app.state.run_event_store = make_run_event_store(run_events_config)
|
||||
@@ -222,6 +226,14 @@ def get_tag_store(request: Request) -> TagStore:
|
||||
return val
|
||||
|
||||
|
||||
def get_thread_share_store(request: Request):
|
||||
"""Return the conversation share-link store."""
|
||||
val = getattr(request.app.state, "thread_share_store", None)
|
||||
if val is None:
|
||||
raise HTTPException(status_code=503, detail="Thread share store not available")
|
||||
return val
|
||||
|
||||
|
||||
def get_run_context(request: Request) -> RunContext:
|
||||
"""Build a :class:`RunContext` from ``app.state`` singletons.
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from fastapi import APIRouter, HTTPException, Query, Request
|
||||
from fastapi.responses import FileResponse, Response, StreamingResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from app.gateway.deps import get_current_user, get_local_provider, get_tag_store
|
||||
from app.gateway.deps import get_current_user, get_local_provider, get_optional_user_from_request, get_tag_store
|
||||
from deerflow.runtime.scheduler import get_scheduled_task_service
|
||||
|
||||
router = APIRouter(prefix="/api/scheduled-tasks", tags=["scheduled-tasks"])
|
||||
@@ -61,6 +61,26 @@ async def _user_id(request: Request) -> str:
|
||||
return user_id
|
||||
|
||||
|
||||
async def _is_admin(request: Request) -> bool:
|
||||
"""True when the caller is an admin (or auth is disabled entirely)."""
|
||||
user = await get_optional_user_from_request(request)
|
||||
if user is None:
|
||||
return True
|
||||
return getattr(user, "system_role", None) == "admin"
|
||||
|
||||
|
||||
async def _load_run(service, request: Request, run_id: str, user_id: str) -> dict[str, Any] | None:
|
||||
"""Owner/subscriber run lookup, with an admin fallback to any run.
|
||||
|
||||
Lets admins open run results for tasks they do not own (the admin
|
||||
management view links into the shared run-detail page).
|
||||
"""
|
||||
run = await _load_run(service, request, run_id, user_id)
|
||||
if run is None and await _is_admin(request):
|
||||
run = await service.admin_get_run(run_id)
|
||||
return run
|
||||
|
||||
|
||||
async def _enrich_with_creator(tasks: list[dict[str, Any]] | dict[str, Any] | None) -> Any:
|
||||
"""Attach ``creator_email`` / ``creator_user_id`` to one or many task dicts.
|
||||
|
||||
@@ -208,7 +228,7 @@ async def list_scheduled_task_runs(task_id: str, request: Request, limit: int =
|
||||
async def get_scheduled_task_run(run_id: str, request: Request) -> dict:
|
||||
service = get_scheduled_task_service()
|
||||
user_id = await _user_id(request)
|
||||
run = await service.get_run(user_id, run_id)
|
||||
run = await _load_run(service, request, run_id, user_id)
|
||||
if run is None:
|
||||
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
|
||||
return run
|
||||
@@ -231,7 +251,7 @@ async def delete_scheduled_task_run(run_id: str, request: Request) -> dict:
|
||||
async def get_scheduled_task_run_file(run_id: str, file_index: int, request: Request) -> dict:
|
||||
service = get_scheduled_task_service()
|
||||
user_id = await _user_id(request)
|
||||
run = await service.get_run(user_id, run_id)
|
||||
run = await _load_run(service, request, run_id, user_id)
|
||||
if run is None:
|
||||
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
|
||||
item, path = _run_file_path(run, file_index)
|
||||
@@ -279,7 +299,7 @@ async def update_scheduled_task_run_file(
|
||||
async def download_scheduled_task_run(run_id: str, request: Request):
|
||||
service = get_scheduled_task_service()
|
||||
user_id = await _user_id(request)
|
||||
run = await service.get_run(user_id, run_id)
|
||||
run = await _load_run(service, request, run_id, user_id)
|
||||
if run is None:
|
||||
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
|
||||
|
||||
@@ -328,7 +348,7 @@ async def download_scheduled_task_run(run_id: str, request: Request):
|
||||
async def download_scheduled_task_run_file(run_id: str, file_index: int, request: Request):
|
||||
service = get_scheduled_task_service()
|
||||
user_id = await _user_id(request)
|
||||
run = await service.get_run(user_id, run_id)
|
||||
run = await _load_run(service, request, run_id, user_id)
|
||||
if run is None:
|
||||
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
|
||||
item, path = _run_file_path(run, file_index)
|
||||
@@ -497,3 +517,82 @@ async def delete_scheduled_task_subscription(task_id: str, request: Request) ->
|
||||
if not deleted:
|
||||
raise HTTPException(status_code=404, detail="Subscription not found")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
# --- Admin (cross-user management) ---
|
||||
|
||||
|
||||
async def _require_admin(request: Request) -> None:
|
||||
"""Reject non-admins. Lets the call through when auth is disabled."""
|
||||
user = await get_optional_user_from_request(request)
|
||||
if user is None:
|
||||
return
|
||||
if getattr(user, "system_role", None) != "admin":
|
||||
raise HTTPException(status_code=403, detail="定时任务管理仅限管理员")
|
||||
|
||||
|
||||
@router.get("/admin/all")
|
||||
async def admin_list_scheduled_tasks(
|
||||
request: Request,
|
||||
search: str | None = Query(default=None, description="Case-insensitive fuzzy match on task name"),
|
||||
) -> list[dict]:
|
||||
"""List every scheduled task across all users. Admin-only."""
|
||||
await _require_admin(request)
|
||||
service = get_scheduled_task_service()
|
||||
tasks = await service.admin_list_tasks()
|
||||
await _enrich_with_creator(tasks)
|
||||
|
||||
tag_store = get_tag_store(request)
|
||||
assignments = await tag_store.list_assignments("scheduled_task", [str(t.get("task_id")) for t in tasks])
|
||||
for task in tasks:
|
||||
task["tags"] = assignments.get(str(task.get("task_id")), [])
|
||||
if search:
|
||||
needle = search.strip().lower()
|
||||
tasks = [t for t in tasks if needle in str(t.get("name", "")).lower()]
|
||||
return tasks
|
||||
|
||||
|
||||
@router.get("/admin/{task_id}/runs")
|
||||
async def admin_list_scheduled_task_runs(task_id: str, request: Request, limit: int = 50) -> list[dict]:
|
||||
"""List runs for any task regardless of owner. Admin-only."""
|
||||
await _require_admin(request)
|
||||
service = get_scheduled_task_service()
|
||||
return await service.admin_list_runs(task_id, limit=max(1, min(limit, 200)))
|
||||
|
||||
|
||||
@router.patch("/admin/{task_id}")
|
||||
async def admin_update_scheduled_task(task_id: str, body: ScheduledTaskUpdateRequest, request: Request) -> dict:
|
||||
"""Update any task regardless of owner. Admin-only."""
|
||||
await _require_admin(request)
|
||||
service = get_scheduled_task_service()
|
||||
task = await service.admin_update_task(task_id, body.model_dump(exclude_unset=True))
|
||||
if task is None:
|
||||
raise HTTPException(status_code=404, detail=f"Task {task_id} not found")
|
||||
await _enrich_with_creator(task)
|
||||
return task
|
||||
|
||||
|
||||
@router.delete("/admin/{task_id}")
|
||||
async def admin_delete_scheduled_task(task_id: str, request: Request) -> dict:
|
||||
"""Delete any task regardless of owner. Admin-only."""
|
||||
await _require_admin(request)
|
||||
service = get_scheduled_task_service()
|
||||
deleted = await service.admin_delete_task(task_id)
|
||||
if not deleted:
|
||||
raise HTTPException(status_code=404, detail=f"Task {task_id} not found")
|
||||
try:
|
||||
await get_tag_store(request).unassign_all("scheduled_task", task_id)
|
||||
except Exception: # noqa: BLE001 — tag cleanup must not block deletion
|
||||
pass
|
||||
return {"success": True}
|
||||
|
||||
|
||||
@router.post("/admin/{task_id}/trigger")
|
||||
async def admin_trigger_scheduled_task(task_id: str, request: Request) -> dict:
|
||||
"""Trigger an immediate run of any task regardless of owner. Admin-only."""
|
||||
await _require_admin(request)
|
||||
service = get_scheduled_task_service()
|
||||
task = await service.admin_trigger_task(task_id)
|
||||
if task is None:
|
||||
raise HTTPException(status_code=404, detail=f"Task {task_id} not found")
|
||||
return {"success": True, "task": task}
|
||||
@@ -45,6 +45,24 @@ def _is_admin_user(request: Request) -> bool:
|
||||
return getattr(user, "system_role", None) == "admin"
|
||||
|
||||
|
||||
def _can_view_skill_source(
|
||||
category: SkillCategory,
|
||||
owner_user_id: str | None,
|
||||
user_id: str,
|
||||
*,
|
||||
is_admin: bool,
|
||||
) -> bool:
|
||||
"""Whether the raw SKILL.md source is visible to a user.
|
||||
|
||||
Admins may view any skill's source; otherwise only the publisher (the
|
||||
owner of a custom skill) qualifies. Built-in / public skills have no human
|
||||
owner, so for non-admins they are never source-visible.
|
||||
"""
|
||||
if is_admin:
|
||||
return True
|
||||
return category == SkillCategory.CUSTOM and owner_user_id is not None and owner_user_id == user_id
|
||||
|
||||
|
||||
async def _get_owned_skill_or_raise(store: SkillStore, name: str, user_id: str) -> dict[str, Any]:
|
||||
owned = await store.get_owned(name, user_id)
|
||||
if owned is not None:
|
||||
@@ -870,15 +888,27 @@ class SkillContentResponse(BaseModel):
|
||||
"/skills/{skill_name}/content",
|
||||
response_model=SkillContentResponse,
|
||||
summary="Get Skill Markdown Content",
|
||||
description="Retrieve the raw SKILL.md content of a specific skill (public or custom).",
|
||||
description="Retrieve the raw SKILL.md content of a skill. Only an admin or the skill's publisher (owner of a custom skill) may read it; other users see only the description and detail.",
|
||||
)
|
||||
async def get_skill_content(skill_name: str, config: AppConfig = Depends(get_config)) -> SkillContentResponse:
|
||||
async def get_skill_content(
|
||||
skill_name: str,
|
||||
request: Request,
|
||||
config: AppConfig = Depends(get_config),
|
||||
skill_store: SkillStore = Depends(get_skill_store),
|
||||
) -> SkillContentResponse:
|
||||
try:
|
||||
skill_name = skill_name.replace("\r\n", "").replace("\n", "")
|
||||
skills = get_or_new_skill_storage(app_config=config).load_skills(enabled_only=False)
|
||||
skill = next((s for s in skills if s.name == skill_name), None)
|
||||
if skill is None:
|
||||
raise HTTPException(status_code=404, detail=f"Skill '{skill_name}' not found")
|
||||
# The raw SKILL.md is implementation detail — only an admin or the
|
||||
# publisher (owner of a custom skill) may view it. Everyone else is
|
||||
# limited to the description + detail surfaced by the list/detail APIs.
|
||||
record = await skill_store.get_any(skill_name)
|
||||
owner_user_id = record.get("owner_user_id") if record else None
|
||||
if not _can_view_skill_source(skill.category, owner_user_id, _current_user_id(request), is_admin=_is_admin_user(request)):
|
||||
raise HTTPException(status_code=403, detail="Only the skill publisher or an admin can view the SKILL.md source")
|
||||
content = skill.skill_file.read_text(encoding="utf-8")
|
||||
return SkillContentResponse(name=skill.name, category=skill.category, content=content)
|
||||
except HTTPException:
|
||||
|
||||
@@ -0,0 +1,340 @@
|
||||
"""Conversation sharing API.
|
||||
|
||||
A user creates a share code for *one* of their conversations; redeeming the
|
||||
code copies that conversation into the redeemer's account as an independent
|
||||
copy they fully own and can keep chatting in. See ``app/gateway/thread_copy.py``
|
||||
for the checkpoint duplication.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import mimetypes
|
||||
import shutil
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from app.gateway.deps import get_checkpointer, get_thread_share_store, get_thread_store
|
||||
from app.gateway.thread_copy import copy_thread_state
|
||||
from deerflow.config.paths import get_paths
|
||||
from deerflow.persistence.thread_meta.base import ThreadMetaStore
|
||||
from deerflow.persistence.thread_shares.base import ThreadShareStore
|
||||
from deerflow.runtime import serialize_channel_values
|
||||
from deerflow.runtime.user_context import get_effective_user_id
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/thread-shares", tags=["thread-shares"])
|
||||
|
||||
# Public, unauthenticated router. Its prefix is whitelisted in
|
||||
# ``auth_middleware._PUBLIC_PATH_PREFIXES`` — only mount read-only endpoints
|
||||
# that are intentionally world-readable here.
|
||||
public_router = APIRouter(prefix="/api/public/shares", tags=["thread-shares-public"])
|
||||
|
||||
_UNTITLED = "未命名对话"
|
||||
|
||||
# Valid share modes. "import" copies the conversation into the recipient's
|
||||
# account; "view" is a public read-only static page anyone can open.
|
||||
_SHARE_MODES = ("import", "view")
|
||||
|
||||
|
||||
def _current_user_id(request: Request) -> str:
|
||||
user = getattr(request.state, "user", None)
|
||||
if user is not None:
|
||||
return str(user.id)
|
||||
return get_effective_user_id()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request / response models
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class CreateShareRequest(BaseModel):
|
||||
thread_id: str = Field(..., description="The conversation to share")
|
||||
mode: str = Field(
|
||||
default="import",
|
||||
description='"import" — recipient copies the conversation; "view" — public read-only page',
|
||||
)
|
||||
|
||||
|
||||
class ShareResponse(BaseModel):
|
||||
share_code: str = Field(..., description="Short code that recipients enter to import")
|
||||
owner_user_id: str = Field(..., description="User who created the share")
|
||||
thread_id: str = Field(..., description="The shared conversation")
|
||||
revoked: bool = Field(default=False, description="Whether the code has been revoked")
|
||||
mode: str = Field(default="import", description='"import" or "view"')
|
||||
created_at: str | None = Field(default=None, description="Creation timestamp (ISO-8601)")
|
||||
|
||||
|
||||
class PublicShareSnapshotResponse(BaseModel):
|
||||
"""Read-only conversation snapshot served to anonymous viewers."""
|
||||
|
||||
share_code: str
|
||||
title: str = Field(..., description="Conversation title")
|
||||
mode: str = Field(default="view")
|
||||
messages: list[dict] = Field(default_factory=list, description="Serialized conversation messages")
|
||||
artifacts: list[str] = Field(default_factory=list, description="Artifact virtual paths produced in the conversation")
|
||||
created_at: str | None = Field(default=None, description="When the share was created (ISO-8601)")
|
||||
|
||||
|
||||
class SharesListResponse(BaseModel):
|
||||
shares: list[ShareResponse]
|
||||
|
||||
|
||||
class SharePreviewResponse(BaseModel):
|
||||
share_code: str
|
||||
shared_by: str = Field(..., description="Owner user id of the shared conversation")
|
||||
title: str = Field(..., description="Title of the shared conversation")
|
||||
|
||||
|
||||
class ImportResultResponse(BaseModel):
|
||||
imported: int = Field(..., description="1 when the conversation was copied, else 0")
|
||||
skipped: int = Field(..., description="1 when it was already imported, else 0")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Owner side — create / list / revoke
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("", response_model=ShareResponse, summary="Create a share code for one conversation")
|
||||
async def create_share(
|
||||
body: CreateShareRequest,
|
||||
request: Request,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
thread_store: ThreadMetaStore = Depends(get_thread_store),
|
||||
) -> ShareResponse:
|
||||
me = _current_user_id(request)
|
||||
mode = body.mode if body.mode in _SHARE_MODES else "import"
|
||||
# Only the conversation's owner may share it.
|
||||
meta = await thread_store.get(body.thread_id, user_id=me)
|
||||
if meta is None:
|
||||
raise HTTPException(status_code=404, detail="Conversation not found")
|
||||
# Reuse an existing active code (of the same mode) so re-sharing is idempotent.
|
||||
existing = await share_store.get_active_for_thread(me, body.thread_id, mode=mode)
|
||||
if existing is not None:
|
||||
return ShareResponse(**existing)
|
||||
record = await share_store.create(me, body.thread_id, mode=mode)
|
||||
return ShareResponse(**record)
|
||||
|
||||
|
||||
@router.get("", response_model=SharesListResponse, summary="List my conversation share codes")
|
||||
async def list_shares(
|
||||
request: Request,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
) -> SharesListResponse:
|
||||
records = await share_store.list_for_owner(_current_user_id(request))
|
||||
return SharesListResponse(shares=[ShareResponse(**r) for r in records])
|
||||
|
||||
|
||||
@router.delete("/{share_code}", summary="Revoke a conversation share code")
|
||||
async def revoke_share(
|
||||
share_code: str,
|
||||
request: Request,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
) -> dict[str, bool]:
|
||||
ok = await share_store.revoke(share_code, _current_user_id(request))
|
||||
if not ok:
|
||||
raise HTTPException(status_code=404, detail="Share code not found")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Recipient side — preview / import
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _resolve_active_share(share_code: str, share_store: ThreadShareStore) -> dict:
|
||||
share = await share_store.get(share_code)
|
||||
if share is None:
|
||||
raise HTTPException(status_code=404, detail="Share code not found")
|
||||
if share.get("revoked"):
|
||||
raise HTTPException(status_code=410, detail="This share code has been revoked")
|
||||
return share
|
||||
|
||||
|
||||
@router.get("/{share_code}/preview", response_model=SharePreviewResponse, summary="Preview a shared conversation")
|
||||
async def preview_share(
|
||||
share_code: str,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
thread_store: ThreadMetaStore = Depends(get_thread_store),
|
||||
) -> SharePreviewResponse:
|
||||
share = await _resolve_active_share(share_code, share_store)
|
||||
owner = share["owner_user_id"]
|
||||
meta = await thread_store.get(share["thread_id"], user_id=owner)
|
||||
if meta is None:
|
||||
raise HTTPException(status_code=404, detail="The shared conversation no longer exists")
|
||||
return SharePreviewResponse(
|
||||
share_code=share_code,
|
||||
shared_by=owner,
|
||||
title=meta.get("display_name") or _UNTITLED,
|
||||
)
|
||||
|
||||
|
||||
def _copy_thread_files(origin_thread_id: str, owner_user_id: str, new_thread_id: str, recipient_user_id: str) -> None:
|
||||
"""Copy a thread's user-data directory (uploads + outputs/artifacts)."""
|
||||
paths = get_paths()
|
||||
src_data = paths.thread_dir(origin_thread_id, user_id=owner_user_id) / "user-data"
|
||||
if not src_data.is_dir():
|
||||
return
|
||||
dst_data = paths.thread_dir(new_thread_id, user_id=recipient_user_id) / "user-data"
|
||||
shutil.copytree(src_data, dst_data, dirs_exist_ok=True)
|
||||
|
||||
|
||||
@router.post("/{share_code}/import", response_model=ImportResultResponse, summary="Import a shared conversation")
|
||||
async def import_share(
|
||||
share_code: str,
|
||||
request: Request,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
thread_store: ThreadMetaStore = Depends(get_thread_store),
|
||||
) -> ImportResultResponse:
|
||||
share = await _resolve_active_share(share_code, share_store)
|
||||
if share.get("mode") == "view":
|
||||
raise HTTPException(status_code=400, detail="This is a view-only share — open its link to read it instead of importing")
|
||||
owner = share["owner_user_id"]
|
||||
origin_id = share["thread_id"]
|
||||
recipient = _current_user_id(request)
|
||||
if owner == recipient:
|
||||
raise HTTPException(status_code=400, detail="You cannot import your own share code")
|
||||
|
||||
src = await thread_store.get(origin_id, user_id=owner)
|
||||
if src is None:
|
||||
raise HTTPException(status_code=404, detail="The shared conversation no longer exists")
|
||||
|
||||
# Dedup: skip if this user already imported this conversation.
|
||||
mine = await thread_store.search(user_id=recipient, limit=2000)
|
||||
already = {(t.get("metadata") or {}).get("shared_origin") for t in mine}
|
||||
if origin_id in already:
|
||||
return ImportResultResponse(imported=0, skipped=1)
|
||||
|
||||
new_id = str(uuid.uuid4())
|
||||
checkpointer = get_checkpointer(request)
|
||||
await copy_thread_state(checkpointer, origin_id, new_id)
|
||||
|
||||
metadata = dict(src.get("metadata") or {})
|
||||
metadata.update({"shared_from_code": share_code, "shared_origin": origin_id, "shared_by": owner})
|
||||
await thread_store.create(
|
||||
new_id,
|
||||
assistant_id=src.get("assistant_id"),
|
||||
user_id=recipient,
|
||||
display_name=src.get("display_name"),
|
||||
metadata=metadata,
|
||||
)
|
||||
try:
|
||||
_copy_thread_files(origin_id, owner, new_id, recipient)
|
||||
except Exception as exc: # noqa: BLE001 — file copy failure must not lose the conversation
|
||||
logger.warning("Failed to copy files for thread %s -> %s: %s", origin_id, new_id, exc)
|
||||
|
||||
logger.info("Share %s imported by %s as thread %s", share_code, recipient, new_id)
|
||||
return ImportResultResponse(imported=1, skipped=0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public read-only view — no authentication required
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _load_thread_snapshot(checkpointer, thread_id: str) -> tuple[list[dict], list[str]]:
|
||||
"""Return (messages, artifacts) from a thread's latest checkpoint.
|
||||
|
||||
Reads straight from the checkpointer — the same source the live chat
|
||||
streams from — so the static snapshot matches the conversation exactly.
|
||||
"""
|
||||
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
|
||||
checkpoint_tuple = await checkpointer.aget_tuple(config)
|
||||
if checkpoint_tuple is None:
|
||||
return [], []
|
||||
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {}
|
||||
values = serialize_channel_values(checkpoint.get("channel_values", {}) or {})
|
||||
messages = values.get("messages")
|
||||
artifacts = values.get("artifacts")
|
||||
return (
|
||||
[m for m in messages if isinstance(m, dict)] if isinstance(messages, list) else [],
|
||||
[a for a in artifacts if isinstance(a, str)] if isinstance(artifacts, list) else [],
|
||||
)
|
||||
|
||||
|
||||
@public_router.get(
|
||||
"/{share_code}",
|
||||
response_model=PublicShareSnapshotResponse,
|
||||
summary="Public read-only snapshot of a view-shared conversation",
|
||||
)
|
||||
async def get_public_share_snapshot(
|
||||
share_code: str,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
thread_store: ThreadMetaStore = Depends(get_thread_store),
|
||||
checkpointer=Depends(get_checkpointer),
|
||||
) -> PublicShareSnapshotResponse:
|
||||
"""Serve a frozen, read-only conversation. Anyone with the link can read it.
|
||||
|
||||
Only ``mode="view"`` shares are exposed; missing, revoked, or import-mode
|
||||
codes all return 404 so the endpoint never reveals which case applies.
|
||||
"""
|
||||
share = await share_store.get(share_code)
|
||||
if share is None or share.get("revoked") or share.get("mode") != "view":
|
||||
raise HTTPException(status_code=404, detail="Share not found")
|
||||
|
||||
meta = await thread_store.get(share["thread_id"], user_id=share["owner_user_id"])
|
||||
if meta is None:
|
||||
raise HTTPException(status_code=404, detail="The shared conversation no longer exists")
|
||||
|
||||
messages, artifacts = await _load_thread_snapshot(checkpointer, share["thread_id"])
|
||||
return PublicShareSnapshotResponse(
|
||||
share_code=share_code,
|
||||
title=meta.get("display_name") or _UNTITLED,
|
||||
mode="view",
|
||||
messages=messages,
|
||||
artifacts=artifacts,
|
||||
created_at=share.get("created_at"),
|
||||
)
|
||||
|
||||
|
||||
@public_router.get(
|
||||
"/{share_code}/files/{path:path}",
|
||||
summary="Public image attachment from a view-shared conversation",
|
||||
)
|
||||
async def get_public_share_file(
|
||||
share_code: str,
|
||||
path: str,
|
||||
share_store: ThreadShareStore = Depends(get_thread_share_store),
|
||||
) -> FileResponse:
|
||||
"""Serve an image referenced by a view-shared conversation.
|
||||
|
||||
Restricted to images only — the public share page renders inline images,
|
||||
but the endpoint must not become a way to read arbitrary thread files.
|
||||
Non-image paths return 404 just like a missing file.
|
||||
"""
|
||||
share = await share_store.get(share_code)
|
||||
if share is None or share.get("revoked") or share.get("mode") != "view":
|
||||
raise HTTPException(status_code=404, detail="Share not found")
|
||||
|
||||
# Resolve the virtual path inside the *owner's* thread directory.
|
||||
try:
|
||||
actual = get_paths().resolve_virtual_path(share["thread_id"], path, user_id=share["owner_user_id"])
|
||||
except ValueError as exc:
|
||||
status = 403 if "traversal" in str(exc) else 400
|
||||
raise HTTPException(status_code=status, detail=str(exc))
|
||||
|
||||
actual_path = Path(actual)
|
||||
if not actual_path.exists() or not actual_path.is_file():
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
|
||||
mime_type, _ = mimetypes.guess_type(actual_path.name)
|
||||
if not mime_type or not mime_type.startswith("image/"):
|
||||
# Only images are public; everything else is treated as not found.
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
|
||||
# SVG can carry scripts — serve it as a download so it can never execute
|
||||
# as a top-level document in the app origin. Raster images render inline.
|
||||
is_svg = mime_type == "image/svg+xml"
|
||||
return FileResponse(
|
||||
actual_path,
|
||||
media_type=mime_type,
|
||||
content_disposition_type="attachment" if is_svg else "inline",
|
||||
headers={"Cache-Control": "public, max-age=300"},
|
||||
)
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Copy a conversation's LangGraph checkpoint state from one thread to another.
|
||||
|
||||
Used by the conversation-sharing feature: redeeming a share link duplicates the
|
||||
sharer's threads into the redeemer's account. Each duplicated thread needs its
|
||||
full checkpoint history copied under a fresh ``thread_id``.
|
||||
|
||||
The copy goes through the checkpointer's public API (``alist`` / ``aput`` /
|
||||
``aput_writes``) rather than raw SQL, so it is independent of the checkpoint
|
||||
table schema and works for every backend (SQLite, Postgres, in-memory).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def copy_thread_state(checkpointer, src_thread_id: str, dst_thread_id: str) -> bool:
|
||||
"""Duplicate every checkpoint of ``src_thread_id`` under ``dst_thread_id``.
|
||||
|
||||
Returns True when at least one checkpoint was copied, False when the source
|
||||
thread has no checkpoints (e.g. a thread that was never run).
|
||||
|
||||
The destination keeps the source's checkpoint ids — they only need to be
|
||||
unique within a ``thread_id``, and reusing them preserves the parent chain.
|
||||
"""
|
||||
src_config = {"configurable": {"thread_id": src_thread_id}}
|
||||
# alist yields newest-first; reverse so parents are written before children.
|
||||
tuples = [t async for t in checkpointer.alist(src_config)]
|
||||
if not tuples:
|
||||
return False
|
||||
|
||||
copied = 0
|
||||
for tup in reversed(tuples):
|
||||
configurable = (tup.config or {}).get("configurable", {})
|
||||
checkpoint_ns = configurable.get("checkpoint_ns", "") or ""
|
||||
parent_id = ((tup.parent_config or {}).get("configurable", {}) or {}).get("checkpoint_id")
|
||||
|
||||
put_config: dict = {"configurable": {"thread_id": dst_thread_id, "checkpoint_ns": checkpoint_ns}}
|
||||
if parent_id:
|
||||
# aput reads the parent checkpoint id from the incoming config.
|
||||
put_config["configurable"]["checkpoint_id"] = parent_id
|
||||
|
||||
new_versions = tup.checkpoint.get("channel_versions", {}) or {}
|
||||
await checkpointer.aput(put_config, tup.checkpoint, tup.metadata or {}, new_versions)
|
||||
|
||||
# pending_writes is a list of (task_id, channel, value); aput_writes
|
||||
# wants them grouped per task as (channel, value) pairs.
|
||||
if tup.pending_writes:
|
||||
by_task: dict[str, list] = {}
|
||||
for task_id, channel, value in tup.pending_writes:
|
||||
by_task.setdefault(task_id, []).append((channel, value))
|
||||
checkpoint_id = tup.checkpoint.get("id")
|
||||
for task_id, writes in by_task.items():
|
||||
writes_config = {
|
||||
"configurable": {
|
||||
"thread_id": dst_thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint_id,
|
||||
}
|
||||
}
|
||||
await checkpointer.aput_writes(writes_config, writes, task_id)
|
||||
copied += 1
|
||||
|
||||
logger.info("Copied %d checkpoint(s) from thread %s to %s", copied, src_thread_id, dst_thread_id)
|
||||
return copied > 0
|
||||
+53
@@ -0,0 +1,53 @@
|
||||
"""Create thread_shares table (per-conversation share links).
|
||||
|
||||
Revision ID: 20260518_02
|
||||
Revises: 20260518_01
|
||||
Create Date: 2026-05-18
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import inspect
|
||||
|
||||
|
||||
revision = "20260518_02"
|
||||
down_revision = "20260518_01"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _has_table(table_name: str) -> bool:
|
||||
inspector = inspect(op.get_bind())
|
||||
return table_name in inspector.get_table_names()
|
||||
|
||||
|
||||
def _has_column(table_name: str, column_name: str) -> bool:
|
||||
if not _has_table(table_name):
|
||||
return False
|
||||
inspector = inspect(op.get_bind())
|
||||
return any(col["name"] == column_name for col in inspector.get_columns(table_name))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
if not _has_table("thread_shares"):
|
||||
op.create_table(
|
||||
"thread_shares",
|
||||
sa.Column("share_code", sa.String(32), primary_key=True),
|
||||
sa.Column("owner_user_id", sa.String(64), nullable=False),
|
||||
sa.Column("thread_id", sa.String(64), nullable=False),
|
||||
sa.Column("revoked", sa.Boolean(), nullable=False, server_default=sa.false()),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
)
|
||||
op.create_index("ix_thread_shares_owner_user_id", "thread_shares", ["owner_user_id"])
|
||||
op.create_index("ix_thread_shares_thread_id", "thread_shares", ["thread_id"])
|
||||
elif not _has_column("thread_shares", "thread_id"):
|
||||
# An earlier all-conversations variant of this table — add the column.
|
||||
op.add_column("thread_shares", sa.Column("thread_id", sa.String(64), nullable=False, server_default=""))
|
||||
op.create_index("ix_thread_shares_thread_id", "thread_shares", ["thread_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
if _has_table("thread_shares"):
|
||||
op.drop_table("thread_shares")
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
"""Ensure thread_shares.thread_id exists (per-conversation share links).
|
||||
|
||||
Standalone migration so databases that already applied an earlier
|
||||
all-conversations variant of ``20260518_02`` (which had no ``thread_id``
|
||||
column) get the column added. Idempotent — a no-op when the column is
|
||||
already present.
|
||||
|
||||
Revision ID: 20260518_03
|
||||
Revises: 20260518_02
|
||||
Create Date: 2026-05-18
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import inspect
|
||||
|
||||
|
||||
revision = "20260518_03"
|
||||
down_revision = "20260518_02"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _has_table(table_name: str) -> bool:
|
||||
inspector = inspect(op.get_bind())
|
||||
return table_name in inspector.get_table_names()
|
||||
|
||||
|
||||
def _has_column(table_name: str, column_name: str) -> bool:
|
||||
if not _has_table(table_name):
|
||||
return False
|
||||
inspector = inspect(op.get_bind())
|
||||
return any(col["name"] == column_name for col in inspector.get_columns(table_name))
|
||||
|
||||
|
||||
def _has_index(table_name: str, index_name: str) -> bool:
|
||||
if not _has_table(table_name):
|
||||
return False
|
||||
inspector = inspect(op.get_bind())
|
||||
return any(ix["name"] == index_name for ix in inspector.get_indexes(table_name))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
if not _has_table("thread_shares"):
|
||||
# Fresh database — 20260518_02 already created the table with thread_id.
|
||||
return
|
||||
if not _has_column("thread_shares", "thread_id"):
|
||||
op.add_column("thread_shares", sa.Column("thread_id", sa.String(64), nullable=False, server_default=""))
|
||||
if not _has_index("thread_shares", "ix_thread_shares_thread_id"):
|
||||
op.create_index("ix_thread_shares_thread_id", "thread_shares", ["thread_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Leave the column in place — dropping it would break the share feature.
|
||||
pass
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
"""Add retry_count to scheduled_task_runs.
|
||||
|
||||
Tracks how many recovery attempts a run has consumed so the scheduler can
|
||||
cap retries on stuck/timed-out runs.
|
||||
|
||||
Revision ID: 20260518_04
|
||||
Revises: 20260518_03
|
||||
Create Date: 2026-05-18
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import inspect
|
||||
|
||||
|
||||
revision = "20260518_04"
|
||||
down_revision = "20260518_03"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _has_column(table_name: str, column_name: str) -> bool:
|
||||
inspector = inspect(op.get_bind())
|
||||
return any(column["name"] == column_name for column in inspector.get_columns(table_name))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
if not _has_column("scheduled_task_runs", "retry_count"):
|
||||
op.add_column(
|
||||
"scheduled_task_runs",
|
||||
sa.Column("retry_count", sa.Integer(), nullable=False, server_default="0"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
if _has_column("scheduled_task_runs", "retry_count"):
|
||||
op.drop_column("scheduled_task_runs", "retry_count")
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
"""Add mode to thread_shares.
|
||||
|
||||
Distinguishes "import" shares (recipient copies the conversation into their
|
||||
account) from "view" shares (a public, read-only static page).
|
||||
|
||||
Revision ID: 20260518_05
|
||||
Revises: 20260518_04
|
||||
Create Date: 2026-05-18
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import inspect
|
||||
|
||||
|
||||
revision = "20260518_05"
|
||||
down_revision = "20260518_04"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _has_column(table_name: str, column_name: str) -> bool:
|
||||
inspector = inspect(op.get_bind())
|
||||
if table_name not in inspector.get_table_names():
|
||||
return True # table missing entirely — skip (created with the column)
|
||||
return any(column["name"] == column_name for column in inspector.get_columns(table_name))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
if not _has_column("thread_shares", "mode"):
|
||||
op.add_column(
|
||||
"thread_shares",
|
||||
sa.Column("mode", sa.String(16), nullable=False, server_default="import"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
inspector = inspect(op.get_bind())
|
||||
if "thread_shares" in inspector.get_table_names() and any(
|
||||
column["name"] == "mode" for column in inspector.get_columns("thread_shares")
|
||||
):
|
||||
op.drop_column("thread_shares", "mode")
|
||||
+2
@@ -32,6 +32,7 @@ from deerflow.persistence.scheduled_tasks.model import (
|
||||
from deerflow.persistence.skills.model import SkillRow
|
||||
from deerflow.persistence.tags.model import TagAssignmentRow, TagRow
|
||||
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
||||
from deerflow.persistence.thread_shares.model import ThreadShareRow
|
||||
from deerflow.persistence.user.model import UserRow
|
||||
|
||||
__all__ = [
|
||||
@@ -51,5 +52,6 @@ __all__ = [
|
||||
"TagAssignmentRow",
|
||||
"TagRow",
|
||||
"ThreadMetaRow",
|
||||
"ThreadShareRow",
|
||||
"UserRow",
|
||||
]
|
||||
+29
@@ -36,6 +36,35 @@ class ScheduledTaskStore(ABC):
|
||||
async def due_tasks(self, now: datetime, *, limit: int = 20) -> list[dict[str, Any]]:
|
||||
raise NotImplementedError
|
||||
|
||||
# --- Admin (cross-user) ---
|
||||
|
||||
@abstractmethod
|
||||
async def list_all_tasks(self) -> list[dict[str, Any]]:
|
||||
"""Return every task across all users. Admin-only callers."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def update_task_any(self, task_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Update a task by id without restricting by owner. Admin-only callers."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def delete_task_any(self, task_id: str) -> bool:
|
||||
"""Delete a task by id without restricting by owner. Admin-only callers."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def list_runs_any(self, task_id: str, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
"""Return runs for a task without restricting by owner. Admin-only callers."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def list_stuck_runs(self, cutoff: datetime, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
"""Return runs still in a non-terminal state whose ``started_at`` is older
|
||||
than ``cutoff``. Used by the scheduler watchdog to recover runs that hung
|
||||
or were orphaned by a service restart."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def create_run_once(self, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
raise NotImplementedError
|
||||
|
||||
+31
@@ -59,6 +59,37 @@ class MemoryScheduledTaskStore(ScheduledTaskStore):
|
||||
due.sort(key=lambda task: task["next_run_at"])
|
||||
return due[:limit]
|
||||
|
||||
# --- Admin (cross-user) ---
|
||||
|
||||
async def list_all_tasks(self) -> list[dict[str, Any]]:
|
||||
return [dict(task) for task in self._tasks.values()]
|
||||
|
||||
async def update_task_any(self, task_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
task = self._tasks.get(task_id)
|
||||
if task is None:
|
||||
return None
|
||||
task.update(data)
|
||||
return dict(task)
|
||||
|
||||
async def delete_task_any(self, task_id: str) -> bool:
|
||||
return self._tasks.pop(task_id, None) is not None
|
||||
|
||||
async def list_runs_any(self, task_id: str, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
runs = [dict(run) for run in self._runs.values() if run["task_id"] == task_id]
|
||||
runs.sort(key=lambda run: run.get("scheduled_for"), reverse=True)
|
||||
return runs[:limit]
|
||||
|
||||
async def list_stuck_runs(self, cutoff: datetime, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
stuck = [
|
||||
dict(run)
|
||||
for run in self._runs.values()
|
||||
if run.get("status") in ("running", "pending")
|
||||
and run.get("started_at") is not None
|
||||
and run["started_at"] <= cutoff
|
||||
]
|
||||
stuck.sort(key=lambda run: run["started_at"])
|
||||
return stuck[:limit]
|
||||
|
||||
async def create_run_once(self, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
key = (data["task_id"], data["scheduled_for"].isoformat())
|
||||
if key in self._runs_by_key:
|
||||
|
||||
+5
-1
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import JSON, Boolean, DateTime, String, Text, UniqueConstraint
|
||||
from sqlalchemy import JSON, Boolean, DateTime, Integer, String, Text, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from deerflow.persistence.base import Base
|
||||
@@ -72,6 +72,10 @@ class ScheduledTaskRunRow(Base):
|
||||
status: Mapped[str] = mapped_column(String(32), default="pending")
|
||||
error: Mapped[str | None] = mapped_column(Text)
|
||||
result_json: Mapped[dict | None] = mapped_column(JSON, default=dict)
|
||||
# Number of recovery attempts made after the first try. A run is retried
|
||||
# when it overruns the per-attempt timeout or is found orphaned in the
|
||||
# "running" state (e.g. after a service restart). See ScheduledTaskService.
|
||||
retry_count: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC))
|
||||
|
||||
|
||||
+54
@@ -124,6 +124,60 @@ class ScheduledTaskRepository(ScheduledTaskStore):
|
||||
)
|
||||
return [_to_dict(row) for row in result.scalars()]
|
||||
|
||||
# --- Admin (cross-user) ---
|
||||
|
||||
async def list_all_tasks(self) -> list[dict[str, Any]]:
|
||||
async with self._sf() as session:
|
||||
result = await session.execute(select(ScheduledTaskRow).order_by(ScheduledTaskRow.created_at.desc()))
|
||||
return [_to_dict(row) for row in result.scalars()]
|
||||
|
||||
async def update_task_any(self, task_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
async with self._sf() as session:
|
||||
row = await session.get(ScheduledTaskRow, task_id)
|
||||
if row is None:
|
||||
return None
|
||||
for key, value in data.items():
|
||||
attr = "execution_context_json" if key == "execution_context" else key
|
||||
if hasattr(row, attr):
|
||||
setattr(row, attr, value)
|
||||
row.updated_at = datetime.now(UTC)
|
||||
await session.commit()
|
||||
await session.refresh(row)
|
||||
return _to_dict(row)
|
||||
|
||||
async def delete_task_any(self, task_id: str) -> bool:
|
||||
async with self._sf() as session:
|
||||
row = await session.get(ScheduledTaskRow, task_id)
|
||||
if row is None:
|
||||
return False
|
||||
await session.delete(row)
|
||||
await session.commit()
|
||||
return True
|
||||
|
||||
async def list_runs_any(self, task_id: str, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
async with self._sf() as session:
|
||||
result = await session.execute(
|
||||
select(ScheduledTaskRunRow)
|
||||
.where(ScheduledTaskRunRow.task_id == task_id)
|
||||
.order_by(ScheduledTaskRunRow.scheduled_for.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
return [_to_dict(row) for row in result.scalars()]
|
||||
|
||||
async def list_stuck_runs(self, cutoff: datetime, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
async with self._sf() as session:
|
||||
result = await session.execute(
|
||||
select(ScheduledTaskRunRow)
|
||||
.where(
|
||||
ScheduledTaskRunRow.status.in_(("running", "pending")),
|
||||
ScheduledTaskRunRow.started_at.is_not(None),
|
||||
ScheduledTaskRunRow.started_at <= cutoff,
|
||||
)
|
||||
.order_by(ScheduledTaskRunRow.started_at.asc())
|
||||
.limit(limit)
|
||||
)
|
||||
return [_to_dict(row) for row in result.scalars()]
|
||||
|
||||
async def create_run_once(self, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
row = ScheduledTaskRunRow(
|
||||
id=data["id"],
|
||||
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
"""Persistence layer for conversation-history share links."""
|
||||
|
||||
from deerflow.persistence.thread_shares.base import ThreadShareStore, generate_share_code
|
||||
from deerflow.persistence.thread_shares.memory import MemoryThreadShareStore
|
||||
from deerflow.persistence.thread_shares.sql import ThreadShareRepository
|
||||
|
||||
__all__ = [
|
||||
"ThreadShareStore",
|
||||
"MemoryThreadShareStore",
|
||||
"ThreadShareRepository",
|
||||
"generate_share_code",
|
||||
"make_thread_share_store",
|
||||
]
|
||||
|
||||
|
||||
def make_thread_share_store(session_factory):
|
||||
if session_factory is None:
|
||||
return MemoryThreadShareStore()
|
||||
return ThreadShareRepository(session_factory)
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
"""Abstract persistence API for per-conversation share links."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Any
|
||||
|
||||
# Unambiguous alphabet — no 0/O/1/I/L so codes are easy to read and type.
|
||||
_CODE_ALPHABET = "ABCDEFGHJKMNPQRSTUVWXYZ23456789"
|
||||
|
||||
|
||||
def generate_share_code(length: int = 8) -> str:
|
||||
"""Return a random, unguessable, human-readable share code."""
|
||||
return "".join(secrets.choice(_CODE_ALPHABET) for _ in range(length))
|
||||
|
||||
|
||||
class ThreadShareStore(ABC):
|
||||
@abstractmethod
|
||||
async def create(self, owner_user_id: str, thread_id: str, *, mode: str = "import") -> dict[str, Any]:
|
||||
"""Create a new share link for one conversation and return its record.
|
||||
|
||||
``mode`` is ``"import"`` (recipient copies the conversation) or
|
||||
``"view"`` (a public read-only static page).
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def get(self, share_code: str) -> dict[str, Any] | None:
|
||||
"""Look up a share by code regardless of owner (None if missing)."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def get_active_for_thread(self, owner_user_id: str, thread_id: str, *, mode: str = "import") -> dict[str, Any] | None:
|
||||
"""Return the owner's non-revoked share of ``mode`` for ``thread_id``, if any.
|
||||
|
||||
Used so re-sharing the same conversation in the same mode reuses its
|
||||
existing code instead of minting a new one each time.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def list_for_owner(self, owner_user_id: str) -> list[dict[str, Any]]:
|
||||
"""List all share links created by ``owner_user_id`` (newest first)."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
async def revoke(self, share_code: str, owner_user_id: str) -> bool:
|
||||
"""Revoke a share link. Returns False if missing or not owned."""
|
||||
raise NotImplementedError
|
||||
+59
@@ -0,0 +1,59 @@
|
||||
"""In-memory ``ThreadShareStore`` implementation for tests / memory backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from deerflow.persistence.thread_shares.base import ThreadShareStore, generate_share_code
|
||||
|
||||
|
||||
class MemoryThreadShareStore(ThreadShareStore):
|
||||
def __init__(self) -> None:
|
||||
self._shares: dict[str, dict[str, Any]] = {}
|
||||
|
||||
async def create(self, owner_user_id: str, thread_id: str, *, mode: str = "import") -> dict[str, Any]:
|
||||
code = generate_share_code()
|
||||
while code in self._shares:
|
||||
code = generate_share_code()
|
||||
record = {
|
||||
"share_code": code,
|
||||
"owner_user_id": owner_user_id,
|
||||
"thread_id": thread_id,
|
||||
"revoked": False,
|
||||
"mode": mode,
|
||||
"created_at": datetime.now(UTC).isoformat(),
|
||||
}
|
||||
self._shares[code] = record
|
||||
return copy.deepcopy(record)
|
||||
|
||||
async def get(self, share_code: str) -> dict[str, Any] | None:
|
||||
record = self._shares.get(share_code)
|
||||
return copy.deepcopy(record) if record is not None else None
|
||||
|
||||
async def get_active_for_thread(self, owner_user_id: str, thread_id: str, *, mode: str = "import") -> dict[str, Any] | None:
|
||||
matches = [
|
||||
r
|
||||
for r in self._shares.values()
|
||||
if r["owner_user_id"] == owner_user_id
|
||||
and r["thread_id"] == thread_id
|
||||
and r.get("mode", "import") == mode
|
||||
and not r["revoked"]
|
||||
]
|
||||
if not matches:
|
||||
return None
|
||||
matches.sort(key=lambda r: r["created_at"], reverse=True)
|
||||
return copy.deepcopy(matches[0])
|
||||
|
||||
async def list_for_owner(self, owner_user_id: str) -> list[dict[str, Any]]:
|
||||
rows = [r for r in self._shares.values() if r["owner_user_id"] == owner_user_id]
|
||||
rows.sort(key=lambda r: r["created_at"], reverse=True)
|
||||
return [copy.deepcopy(r) for r in rows]
|
||||
|
||||
async def revoke(self, share_code: str, owner_user_id: str) -> bool:
|
||||
record = self._shares.get(share_code)
|
||||
if record is None or record["owner_user_id"] != owner_user_id:
|
||||
return False
|
||||
record["revoked"] = True
|
||||
return True
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
"""ORM model for conversation share links.
|
||||
|
||||
A share link is a short, revocable code created by a user for *one* of their
|
||||
conversations. Redeeming it copies that single conversation into the redeemer's
|
||||
account. The table only stores the link itself — the conversation is copied
|
||||
lazily at import time.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from deerflow.persistence.base import Base
|
||||
|
||||
|
||||
class ThreadShareRow(Base):
|
||||
__tablename__ = "thread_shares"
|
||||
|
||||
# Short, URL-safe, unguessable code embedded in the share link.
|
||||
share_code: Mapped[str] = mapped_column(String(32), primary_key=True)
|
||||
# The user who created the share.
|
||||
owner_user_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||
# The single conversation (thread) this link shares.
|
||||
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||
# Soft-disable: a revoked link can no longer be previewed or imported.
|
||||
revoked: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
# Share mode:
|
||||
# "import" — recipient (a logged-in user) copies the conversation into
|
||||
# their own account and can keep chatting in the copy.
|
||||
# "view" — a public, read-only static page anyone can open without
|
||||
# logging in; viewers cannot send new messages.
|
||||
mode: Mapped[str] = mapped_column(String(16), nullable=False, default="import", server_default="import")
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(UTC))
|
||||
+80
@@ -0,0 +1,80 @@
|
||||
"""SQLAlchemy-backed per-conversation share-link repository."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from deerflow.persistence.thread_shares.base import ThreadShareStore, generate_share_code
|
||||
from deerflow.persistence.thread_shares.model import ThreadShareRow
|
||||
|
||||
|
||||
def _row_to_dict(row: ThreadShareRow) -> dict[str, Any]:
|
||||
data = row.to_dict()
|
||||
created = data.get("created_at")
|
||||
if isinstance(created, datetime):
|
||||
data["created_at"] = created.isoformat()
|
||||
return data
|
||||
|
||||
|
||||
class ThreadShareRepository(ThreadShareStore):
|
||||
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||
self._sf = session_factory
|
||||
|
||||
async def create(self, owner_user_id: str, thread_id: str, *, mode: str = "import") -> dict[str, Any]:
|
||||
# Retry on the (astronomically unlikely) code collision.
|
||||
async with self._sf() as session:
|
||||
for _ in range(8):
|
||||
code = generate_share_code()
|
||||
if await session.get(ThreadShareRow, code) is not None:
|
||||
continue
|
||||
row = ThreadShareRow(
|
||||
share_code=code,
|
||||
owner_user_id=owner_user_id,
|
||||
thread_id=thread_id,
|
||||
revoked=False,
|
||||
mode=mode,
|
||||
)
|
||||
session.add(row)
|
||||
await session.commit()
|
||||
await session.refresh(row)
|
||||
return _row_to_dict(row)
|
||||
raise RuntimeError("Failed to allocate a unique share code")
|
||||
|
||||
async def get(self, share_code: str) -> dict[str, Any] | None:
|
||||
async with self._sf() as session:
|
||||
row = await session.get(ThreadShareRow, share_code)
|
||||
return _row_to_dict(row) if row is not None else None
|
||||
|
||||
async def get_active_for_thread(self, owner_user_id: str, thread_id: str, *, mode: str = "import") -> dict[str, Any] | None:
|
||||
stmt = (
|
||||
select(ThreadShareRow)
|
||||
.where(
|
||||
ThreadShareRow.owner_user_id == owner_user_id,
|
||||
ThreadShareRow.thread_id == thread_id,
|
||||
ThreadShareRow.mode == mode,
|
||||
ThreadShareRow.revoked.is_(False),
|
||||
)
|
||||
.order_by(ThreadShareRow.created_at.desc())
|
||||
)
|
||||
async with self._sf() as session:
|
||||
row = (await session.execute(stmt)).scalars().first()
|
||||
return _row_to_dict(row) if row is not None else None
|
||||
|
||||
async def list_for_owner(self, owner_user_id: str) -> list[dict[str, Any]]:
|
||||
stmt = select(ThreadShareRow).where(ThreadShareRow.owner_user_id == owner_user_id).order_by(ThreadShareRow.created_at.desc())
|
||||
async with self._sf() as session:
|
||||
result = await session.execute(stmt)
|
||||
return [_row_to_dict(row) for row in result.scalars()]
|
||||
|
||||
async def revoke(self, share_code: str, owner_user_id: str) -> bool:
|
||||
async with self._sf() as session:
|
||||
row = await session.get(ThreadShareRow, share_code)
|
||||
if row is None or row.owner_user_id != owner_user_id:
|
||||
return False
|
||||
row.revoked = True
|
||||
await session.commit()
|
||||
return True
|
||||
+257
-50
@@ -9,7 +9,7 @@ import re
|
||||
import shutil
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
@@ -27,6 +27,14 @@ from deerflow.utils.time import now_iso
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# A single execution attempt may not exceed this wall-clock budget. When it
|
||||
# does, the run is treated as stuck (the agent loop hung, a tool blocked, etc.)
|
||||
# and retried. The watchdog uses the same window to recover runs orphaned by a
|
||||
# crash or restart — those stay "running" in the DB with no in-process owner.
|
||||
_RUN_ATTEMPT_TIMEOUT_SECONDS = 30 * 60 # 30 minutes
|
||||
# Retries allowed after the first attempt before the run is marked "failed".
|
||||
_MAX_RUN_RETRIES = 3
|
||||
|
||||
|
||||
@dataclass
|
||||
class _RuntimeUser:
|
||||
@@ -398,7 +406,13 @@ class ScheduledTaskService:
|
||||
)
|
||||
return task
|
||||
|
||||
async def update_task(self, user_id: str, task_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
@staticmethod
|
||||
def _build_task_update(data: dict[str, Any], existing: dict[str, Any] | None) -> dict[str, Any]:
|
||||
"""Translate an API update payload into a storage update dict.
|
||||
|
||||
Shared by owner-scoped :meth:`update_task` and admin :meth:`admin_update_task`
|
||||
so both apply identical schedule/timezone normalization.
|
||||
"""
|
||||
update_data: dict[str, Any] = {}
|
||||
for key in ("name", "prompt", "enabled", "execution_agent_name"):
|
||||
if key in data and data[key] is not None:
|
||||
@@ -409,20 +423,74 @@ class ScheduledTaskService:
|
||||
if data.get("schedule") is not None:
|
||||
schedule = str(data["schedule"])
|
||||
cron_expr = _schedule_to_cron(schedule)
|
||||
tz = update_data.get("timezone")
|
||||
if tz is None:
|
||||
existing = await self.store.get_task(task_id, user_id)
|
||||
tz = (existing or {}).get("timezone", "UTC")
|
||||
tz = update_data.get("timezone") or (existing or {}).get("timezone") or "UTC"
|
||||
update_data["schedule_text"] = schedule
|
||||
update_data["cron_expr"] = cron_expr
|
||||
update_data["next_run_at"] = _next_run_at(cron_expr, tz)
|
||||
if data.get("execution_context") is not None:
|
||||
update_data["execution_context"] = data["execution_context"]
|
||||
# Re-enabling a task must also schedule it forward from now, otherwise a
|
||||
# paused task with a stale/empty next_run_at would never fire again.
|
||||
if update_data.get("enabled") is True and "next_run_at" not in update_data:
|
||||
cron = (existing or {}).get("cron_expr")
|
||||
if cron:
|
||||
tz = update_data.get("timezone") or (existing or {}).get("timezone") or "UTC"
|
||||
update_data["next_run_at"] = _next_run_at(cron, tz)
|
||||
return update_data
|
||||
|
||||
async def update_task(self, user_id: str, task_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
existing = await self.store.get_task(task_id, user_id)
|
||||
if existing is None:
|
||||
return None
|
||||
update_data = self._build_task_update(data, existing)
|
||||
return await self.store.update_task(task_id, user_id, update_data)
|
||||
|
||||
async def delete_task(self, user_id: str, task_id: str) -> bool:
|
||||
return await self.store.delete_task(task_id, user_id)
|
||||
|
||||
# --- Admin (cross-user management) ---
|
||||
|
||||
async def admin_list_tasks(self) -> list[dict[str, Any]]:
|
||||
"""List every scheduled task across all users. Admin-only callers."""
|
||||
tasks = await self.store.list_all_tasks()
|
||||
for task in tasks:
|
||||
task["role"] = "owner"
|
||||
return tasks
|
||||
|
||||
async def admin_update_task(self, task_id: str, data: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Update any task regardless of owner. Admin-only callers."""
|
||||
existing = await self.store.get_task_any(task_id)
|
||||
if existing is None:
|
||||
return None
|
||||
update_data = self._build_task_update(data, existing)
|
||||
return await self.store.update_task_any(task_id, update_data)
|
||||
|
||||
async def admin_delete_task(self, task_id: str) -> bool:
|
||||
"""Delete any task regardless of owner. Admin-only callers."""
|
||||
return await self.store.delete_task_any(task_id)
|
||||
|
||||
async def admin_list_runs(self, task_id: str, *, limit: int = 50) -> list[dict[str, Any]]:
|
||||
"""List runs for any task regardless of owner. Admin-only callers."""
|
||||
runs = await self.store.list_runs_any(task_id, limit=limit)
|
||||
for run in runs:
|
||||
run["can_edit"] = True
|
||||
return runs
|
||||
|
||||
async def admin_get_run(self, run_id: str) -> dict[str, Any] | None:
|
||||
"""Fetch a single run regardless of owner. Admin-only callers."""
|
||||
run = await self.store.get_run_any(run_id)
|
||||
if run is not None:
|
||||
run["can_edit"] = True
|
||||
return run
|
||||
|
||||
async def admin_trigger_task(self, task_id: str) -> dict[str, Any] | None:
|
||||
"""Trigger an immediate run of any task regardless of owner."""
|
||||
task = await self.store.get_task_any(task_id)
|
||||
if task is None:
|
||||
return None
|
||||
asyncio.create_task(self._execute_tracked(task, scheduled_for=datetime.now(UTC), manual=True))
|
||||
return task
|
||||
|
||||
async def pause_task(self, user_id: str, task_id: str) -> dict[str, Any] | None:
|
||||
return await self.store.update_task(task_id, user_id, {"enabled": False})
|
||||
|
||||
@@ -437,11 +505,19 @@ class ScheduledTaskService:
|
||||
task = await self.store.get_task(task_id, user_id)
|
||||
if task is None:
|
||||
return None
|
||||
asyncio.create_task(self._execute_task(task, scheduled_for=datetime.now(UTC), manual=True))
|
||||
asyncio.create_task(self._execute_tracked(task, scheduled_for=datetime.now(UTC), manual=True))
|
||||
return task
|
||||
|
||||
async def _run_loop(self) -> None:
|
||||
while not self._stopping.is_set():
|
||||
# Recover hung / orphaned runs first so a single stuck task cannot
|
||||
# silently wedge the scheduler indefinitely.
|
||||
try:
|
||||
await self._reconcile_stuck_runs()
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception:
|
||||
logger.exception("Scheduled task watchdog scan failed")
|
||||
try:
|
||||
due = await self.store.due_tasks(datetime.now(UTC), limit=20)
|
||||
for task in due:
|
||||
@@ -466,64 +542,195 @@ class ScheduledTaskService:
|
||||
finally:
|
||||
self._running_task_ids.discard(task["task_id"])
|
||||
|
||||
async def _execute_task(self, task: dict[str, Any], *, scheduled_for: datetime, manual: bool) -> None:
|
||||
async def _execute_tracked(self, task: dict[str, Any], *, scheduled_for: datetime, manual: bool) -> None:
|
||||
"""Run a task while marking it busy so the watchdog won't double-drive it."""
|
||||
self._running_task_ids.add(task["task_id"])
|
||||
try:
|
||||
await self._execute_task(task, scheduled_for=scheduled_for, manual=manual)
|
||||
finally:
|
||||
self._running_task_ids.discard(task["task_id"])
|
||||
|
||||
async def _reconcile_stuck_runs(self) -> None:
|
||||
"""Recover runs left in the ``running`` state with no in-process owner.
|
||||
|
||||
A run becomes stuck when the agent loop hangs past the per-attempt
|
||||
timeout, or when the process crashes / restarts mid-run (the original
|
||||
asyncio task is gone but the DB row still says ``running``). For each
|
||||
such run we retry up to ``_MAX_RUN_RETRIES`` times, then mark it
|
||||
``failed`` so it no longer shows as perpetually executing.
|
||||
"""
|
||||
cutoff = datetime.now(UTC) - timedelta(seconds=_RUN_ATTEMPT_TIMEOUT_SECONDS)
|
||||
stuck = await self.store.list_stuck_runs(cutoff, limit=20)
|
||||
for run in stuck:
|
||||
task_id = run["task_id"]
|
||||
# A run still owned by a live in-process execution is handled by
|
||||
# that execution's own timeout/retry loop — don't touch it here.
|
||||
if task_id in self._running_task_ids:
|
||||
continue
|
||||
self._running_task_ids.add(task_id)
|
||||
asyncio.create_task(self._recover_and_untrack(run))
|
||||
|
||||
async def _recover_and_untrack(self, run: dict[str, Any]) -> None:
|
||||
try:
|
||||
await self._recover_stuck_run(run)
|
||||
except Exception:
|
||||
logger.exception("Failed to recover stuck scheduled task run %s", run.get("id"))
|
||||
finally:
|
||||
self._running_task_ids.discard(run["task_id"])
|
||||
|
||||
async def _recover_stuck_run(self, run: dict[str, Any]) -> None:
|
||||
run_id = run["id"]
|
||||
retry_count = int(run.get("retry_count") or 0)
|
||||
task = await self.store.get_task_any(run["task_id"])
|
||||
scheduled_for = _coerce_datetime(run.get("scheduled_for")) or datetime.now(UTC)
|
||||
if task is None:
|
||||
# Parent task was deleted — just close out the orphaned run.
|
||||
await self.store.update_run(
|
||||
run_id,
|
||||
{"status": "failed", "finished_at": datetime.now(UTC), "error": "执行超时:所属任务已被删除"},
|
||||
)
|
||||
return
|
||||
if retry_count >= _MAX_RUN_RETRIES:
|
||||
await self._fail_run(
|
||||
task,
|
||||
run_id,
|
||||
scheduled_for,
|
||||
error=f"执行超时:超过 {_RUN_ATTEMPT_TIMEOUT_SECONDS // 60} 分钟未完成,重试 {_MAX_RUN_RETRIES} 次后仍失败",
|
||||
retry_count=retry_count,
|
||||
)
|
||||
return
|
||||
logger.warning("Recovering stuck scheduled task run %s (retry budget %s/%s)", run_id, retry_count, _MAX_RUN_RETRIES)
|
||||
await self._execute_task(task, scheduled_for=scheduled_for, manual=False, run_row=run)
|
||||
|
||||
async def _fail_run(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
run_id: str,
|
||||
scheduled_for: datetime,
|
||||
*,
|
||||
error: str,
|
||||
retry_count: int = 0,
|
||||
) -> None:
|
||||
"""Mark a run failed and reflect the outcome on the parent task."""
|
||||
result = _build_result(task, scheduled_for=scheduled_for, content=f"Scheduled task failed: {error}")
|
||||
await self.store.update_run(
|
||||
run_id,
|
||||
{"status": "failed", "finished_at": datetime.now(UTC), "error": error, "result": result, "retry_count": retry_count},
|
||||
)
|
||||
await self.store.update_task(task["task_id"], task["user_id"], {"last_run_at": datetime.now(UTC), "last_status": "failed"})
|
||||
|
||||
async def _execute_task(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
*,
|
||||
scheduled_for: datetime,
|
||||
manual: bool,
|
||||
run_row: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
# Each run uses its own throwaway thread so message history cannot accumulate
|
||||
# across runs and blow past the model's input window. The per-user
|
||||
# task["scheduler_thread_id"] is no longer used to drive the LLM call —
|
||||
# we keep it on the task row only for backwards compatibility.
|
||||
runtime_thread_id = str(uuid.uuid4())
|
||||
run_row = await self.store.create_run_once(
|
||||
{
|
||||
"id": str(uuid.uuid4()),
|
||||
"task_id": task["task_id"],
|
||||
"user_id": task["user_id"],
|
||||
"scheduler_thread_id": runtime_thread_id,
|
||||
"scheduled_for": scheduled_for,
|
||||
"status": "running",
|
||||
"started_at": datetime.now(UTC),
|
||||
}
|
||||
)
|
||||
is_recovery = run_row is not None
|
||||
if run_row is None:
|
||||
return
|
||||
run_row = await self.store.create_run_once(
|
||||
{
|
||||
"id": str(uuid.uuid4()),
|
||||
"task_id": task["task_id"],
|
||||
"user_id": task["user_id"],
|
||||
"scheduler_thread_id": str(uuid.uuid4()),
|
||||
"scheduled_for": scheduled_for,
|
||||
"status": "running",
|
||||
"started_at": datetime.now(UTC),
|
||||
}
|
||||
)
|
||||
if run_row is None:
|
||||
return
|
||||
else:
|
||||
# Recovery path: re-claim an existing (orphaned) run row.
|
||||
await self.store.update_run(run_row["id"], {"status": "running", "started_at": datetime.now(UTC)})
|
||||
|
||||
if not manual:
|
||||
# Advancing the schedule only happens once, when the run is first
|
||||
# created — recovery retries reuse the original scheduled_for.
|
||||
if not manual and not is_recovery:
|
||||
next_run_at = _next_run_at(task["cron_expr"], task.get("timezone") or "UTC", base=scheduled_for)
|
||||
await self.store.update_task(task["task_id"], task["user_id"], {"next_run_at": next_run_at, "last_status": "running"})
|
||||
|
||||
try:
|
||||
await self._ensure_thread_meta(task["user_id"], runtime_thread_id)
|
||||
async with self._agent_run_lock:
|
||||
output_snapshot = self._snapshot_result_files(runtime_thread_id, task["user_id"])
|
||||
record = await self._start_agent_run(task, scheduled_for=scheduled_for, runtime_thread_id=runtime_thread_id)
|
||||
await self.store.update_run(run_row["id"], {"agent_run_id": record.run_id})
|
||||
if record.task is not None:
|
||||
await record.task
|
||||
result_text = await self._extract_latest_ai_text(runtime_thread_id)
|
||||
result_files = await self._extract_result_files(
|
||||
runtime_thread_id,
|
||||
task["user_id"],
|
||||
run_id=run_row["id"],
|
||||
output_snapshot=output_snapshot,
|
||||
run_id = run_row["id"]
|
||||
retry_count = int(run_row.get("retry_count") or 0)
|
||||
|
||||
while True:
|
||||
runtime_thread_id = str(uuid.uuid4())
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
self._run_agent_attempt(task, runtime_thread_id=runtime_thread_id, scheduled_for=scheduled_for, run_id=run_id),
|
||||
timeout=_RUN_ATTEMPT_TIMEOUT_SECONDS,
|
||||
)
|
||||
result = _build_result(task, scheduled_for=scheduled_for, content=result_text, files=result_files)
|
||||
except TimeoutError:
|
||||
# The attempt overran the 30-minute cap. Retry a bounded number
|
||||
# of times, then give up and mark the run failed.
|
||||
if retry_count >= _MAX_RUN_RETRIES:
|
||||
await self._fail_run(
|
||||
task,
|
||||
run_id,
|
||||
scheduled_for,
|
||||
error=f"执行超时:单次执行超过 {_RUN_ATTEMPT_TIMEOUT_SECONDS // 60} 分钟,重试 {_MAX_RUN_RETRIES} 次后仍失败",
|
||||
retry_count=retry_count,
|
||||
)
|
||||
return
|
||||
retry_count += 1
|
||||
logger.warning("Scheduled task %s timed out; retry %s/%s", task["task_id"], retry_count, _MAX_RUN_RETRIES)
|
||||
await self.store.update_run(
|
||||
run_id,
|
||||
{
|
||||
"retry_count": retry_count,
|
||||
"status": "running",
|
||||
"started_at": datetime.now(UTC),
|
||||
"error": f"执行超时,正在重试(第 {retry_count}/{_MAX_RUN_RETRIES} 次)",
|
||||
},
|
||||
)
|
||||
continue
|
||||
except Exception as exc:
|
||||
logger.exception("Scheduled task %s failed", task["task_id"])
|
||||
await self._fail_run(task, run_id, scheduled_for, error=str(exc), retry_count=retry_count)
|
||||
return
|
||||
|
||||
await self.store.update_run(
|
||||
run_row["id"],
|
||||
{"status": "succeeded", "finished_at": datetime.now(UTC), "result": result},
|
||||
run_id,
|
||||
{"status": "succeeded", "finished_at": datetime.now(UTC), "result": result, "retry_count": retry_count, "error": None},
|
||||
)
|
||||
await self.store.update_task(task["task_id"], task["user_id"], {"last_run_at": datetime.now(UTC), "last_status": "succeeded"})
|
||||
await self._deliver_element_result(task, result["content"])
|
||||
except Exception as exc:
|
||||
logger.exception("Scheduled task %s failed", task["task_id"])
|
||||
result = _build_result(
|
||||
task,
|
||||
scheduled_for=scheduled_for,
|
||||
content=f"Scheduled task failed: {exc}",
|
||||
return
|
||||
|
||||
async def _run_agent_attempt(
|
||||
self,
|
||||
task: dict[str, Any],
|
||||
*,
|
||||
runtime_thread_id: str,
|
||||
scheduled_for: datetime,
|
||||
run_id: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Run the agent once and build the result payload.
|
||||
|
||||
Wrapped in :func:`asyncio.wait_for` by the caller so a hung agent loop
|
||||
cannot hold the shared run lock — and the whole scheduler — forever.
|
||||
"""
|
||||
await self._ensure_thread_meta(task["user_id"], runtime_thread_id)
|
||||
async with self._agent_run_lock:
|
||||
output_snapshot = self._snapshot_result_files(runtime_thread_id, task["user_id"])
|
||||
record = await self._start_agent_run(task, scheduled_for=scheduled_for, runtime_thread_id=runtime_thread_id)
|
||||
await self.store.update_run(run_id, {"agent_run_id": record.run_id})
|
||||
if record.task is not None:
|
||||
await record.task
|
||||
result_text = await self._extract_latest_ai_text(runtime_thread_id)
|
||||
result_files = await self._extract_result_files(
|
||||
runtime_thread_id,
|
||||
task["user_id"],
|
||||
run_id=run_id,
|
||||
output_snapshot=output_snapshot,
|
||||
)
|
||||
await self.store.update_run(
|
||||
run_row["id"],
|
||||
{"status": "failed", "finished_at": datetime.now(UTC), "error": str(exc), "result": result},
|
||||
)
|
||||
await self.store.update_task(task["task_id"], task["user_id"], {"last_run_at": datetime.now(UTC), "last_status": "failed"})
|
||||
return _build_result(task, scheduled_for=scheduled_for, content=result_text, files=result_files)
|
||||
|
||||
async def _ensure_thread_meta(self, user_id: str, thread_id: str) -> None:
|
||||
token = set_current_user(_RuntimeUser(id=user_id))
|
||||
|
||||
@@ -17,6 +17,7 @@ import pytest
|
||||
from app.gateway.routers.skills import (
|
||||
SkillDetailRequest,
|
||||
SkillResponse,
|
||||
_can_view_skill_source,
|
||||
_skill_to_response,
|
||||
)
|
||||
from deerflow.config import paths as paths_module
|
||||
@@ -160,3 +161,29 @@ def test_skill_to_response_detail_none_without_ownership(isolated_paths) -> None
|
||||
resp = _skill_to_response(skill, None)
|
||||
assert resp.detail is None
|
||||
assert resp.builtin is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SKILL.md source visibility — admin / publisher only
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_admin_can_view_any_skill_source() -> None:
|
||||
# Admin sees a built-in skill's source (no owner).
|
||||
assert _can_view_skill_source(SkillCategory.PUBLIC, None, "user-a", is_admin=True) is True
|
||||
# ...and another user's custom skill.
|
||||
assert _can_view_skill_source(SkillCategory.CUSTOM, "user-b", "user-a", is_admin=True) is True
|
||||
|
||||
|
||||
def test_publisher_can_view_own_custom_skill_source() -> None:
|
||||
assert _can_view_skill_source(SkillCategory.CUSTOM, "user-a", "user-a", is_admin=False) is True
|
||||
|
||||
|
||||
def test_non_owner_cannot_view_custom_skill_source() -> None:
|
||||
# A different user — even if the skill is published to them — cannot read source.
|
||||
assert _can_view_skill_source(SkillCategory.CUSTOM, "user-b", "user-a", is_admin=False) is False
|
||||
|
||||
|
||||
def test_non_admin_cannot_view_builtin_skill_source() -> None:
|
||||
# Built-in / public skills have no publisher, so non-admins never see source.
|
||||
assert _can_view_skill_source(SkillCategory.PUBLIC, None, "user-a", is_admin=False) is False
|
||||
Reference in new issue
Block a user