diff --git a/.gitignore b/.gitignore index e13542d..8771397 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ .superpowers/ +.worktrees/ AGENTS.md agents.md node_modules/ diff --git a/docs/superpowers/plans/2026-07-16-llm-architecture-observability-tab.md b/docs/superpowers/plans/2026-07-16-llm-architecture-observability-tab.md new file mode 100644 index 0000000..1e930f9 --- /dev/null +++ b/docs/superpowers/plans/2026-07-16-llm-architecture-observability-tab.md @@ -0,0 +1,1888 @@ +# LLM 后台架构观测标签页实施计划 + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 在首页增加只读的 `后台架构` 标签,以真实后端事件驱动固定架构图,并让授权用户查看当前或最近一次任务的完整 LLM 请求、完整响应和校验结果。 + +**Architecture:** `src/lib/llm/client.ts` 产生包含真实 SDK 对象的内部追踪事件,独立追踪收集器把索引写入 D1/SQLite、把完整正文写入现有私有 R2/本地数据目录,并向现有 NDJSON 流发送不含正文的状态事件。前端共享当前任务事件,刷新后通过受保护的只读 API 恢复追踪清单,并按需加载请求或响应正文。 + +**Tech Stack:** Next.js 16、React 19、TypeScript、Vitest、Playwright、Zod、OpenAI-compatible SDK、Cloudflare Workers、D1、R2、better-sqlite3。 + +--- + +## 实施前文件结构 + +### 新建文件 + +- `migrations/0004_llm_trace_observability.sql`:生产 D1 追踪表和索引。 +- `src/lib/llm/trace-types.ts`:Provider、任务、运行、调用、内部事件、公开流事件和 API 类型。 +- `src/lib/llm/trace-repository.ts`:追踪索引仓储接口和运行时选择。 +- `src/lib/llm/sqlite-trace-repository.ts`:本地 SQLite 追踪索引实现。 +- `src/lib/llm/d1-trace-repository.ts`:Cloudflare D1 追踪索引实现。 +- `src/lib/llm/trace-payload-store.ts`:本地目录与 R2 的完整 JSON 正文存储。 +- `src/lib/llm/trace-recorder.ts`:内部 LLM 事件、工作流事件、持久化、公开事件和保留策略协调器。 +- `src/lib/llm/trace-http.ts`:受保护追踪 API 的无缓存 JSON 响应和正文读取助手。 +- `src/lib/llm/__tests__/trace-repository.test.ts`:SQLite 追踪仓储测试。 +- `src/lib/llm/__tests__/d1-trace-repository.test.ts`:D1 追踪仓储测试。 +- `src/lib/llm/__tests__/trace-payload-store.test.ts`:本地和 R2 正文存储测试。 +- `src/lib/llm/__tests__/trace-recorder.test.ts`:事件顺序、业务结果、失败隔离和保留测试。 +- `src/app/api/llm-traces/latest/route.ts`:当前或最近终态追踪入口。 +- `src/app/api/jobs/[jobId]/llm-trace/route.ts`:任务追踪清单入口。 +- `src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts`:完整请求入口。 +- `src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts`:完整响应入口。 +- `src/app/api/__tests__/llm-traces.test.ts`:追踪 API 鉴权、正文和缓存测试。 +- `src/components/architecture/api-client.ts`:浏览器端追踪 API 客户端。 +- `src/components/architecture/trace-state.ts`:实时事件归并和固定架构节点派生。 +- `src/components/architecture/architecture-flow.tsx`:固定拓扑架构图。 +- `src/components/architecture/llm-call-detail.tsx`:请求、响应、校验详情。 +- `src/components/architecture/architecture-observer-panel.tsx`:只读观测标签页容器。 +- `src/components/architecture/__tests__/api-client.test.ts`:按需正文请求测试。 +- `src/components/architecture/__tests__/trace-state.test.ts`:状态归并和业务/技术失败区分测试。 +- `tests/e2e/architecture-observer.spec.ts`:标签切换、调用选择和技术详情端到端测试。 + +### 修改文件 + +- `src/lib/db/schema.ts`:本地 SQLite 创建追踪表。 +- `src/lib/llm/audit.ts`:从共享类型导入 Provider 和任务名。 +- `src/lib/llm/client.ts`:记录真实请求、响应、解析、校验和安全错误。 +- `src/lib/llm/__tests__/client.test.ts`:请求/响应深度相等和事件顺序测试。 +- `src/lib/workflow/fact-extractor.ts`:传递事实提取追踪上下文。 +- `src/lib/workflow/article-optimizer.ts`:传递生成草稿追踪上下文。 +- `src/lib/workflow/quality-inspector.ts`:传递 QA 和复检轮次追踪上下文。 +- `src/lib/workflow/targeted-rewriter.ts`:传递定向修复轮次追踪上下文。 +- `src/lib/workflow/streaming-optimizer.ts`:向追踪器转发工作流事件和轮次。 +- `src/lib/workflow/stream-events.ts`:公开 LLM 状态事件加入 NDJSON 联合类型。 +- `src/lib/workflow/__tests__/stream-events.test.ts`:新事件编码和分块解析测试。 +- `src/app/api/jobs/optimize-stream/route.ts`:创建、连接和结束追踪器。 +- `src/app/api/__tests__/jobs.test.ts`:真实调用状态事件和失败现场测试。 +- `src/app/page.tsx`:第三标签、实时追踪状态和只读组件接入。 +- `src/app/globals.css`:架构图、调用轨迹、详情和窄屏布局。 +- `tests/e2e/mvp.spec.ts`:把旧的“分析—确认—优化”流程断言更新为当前一键优化界面。 +- `tests/e2e/sample-flow/page-flow.ts`:真实 Provider 样例核对追踪接口,不保存客户正文。 + +### 明确不修改 + +- `wrangler.jsonc`:复用现有私有 `EXPORT_BUCKET`,不增加绑定。 +- `README.md`:本次规格已经完整记录功能,不扩大文档范围。 +- 用户当前未提交的样例、导出目录和本地数据。 + +## Task 1:定义追踪契约和公开流事件 + +**Files:** +- Create: `src/lib/llm/trace-types.ts` +- Modify: `src/lib/llm/audit.ts` +- Modify: `src/lib/llm/client.ts` +- Modify: `src/lib/workflow/stream-events.ts` +- Test: `src/lib/workflow/__tests__/stream-events.test.ts` + +- [ ] **Step 1:先写公开追踪事件的失败测试** + +在 `src/lib/workflow/__tests__/stream-events.test.ts` 增加: + +~~~ts +it("encodes LLM call metadata without raw request or response bodies", () => { + const event: OptimizationStreamEvent = { + type: "llm_call_started", + job_id: "job_123", + call_id: "llmcall_1", + sequence: 1, + task: "fact_extractor", + workflow_stage: "fact_card", + rewrite_round: null, + provider: "deepseek", + model: "deepseek-v4-pro", + started_at: "2026-07-16T00:00:00.000Z", + request_available: true, + }; + + const encoded = encodeOptimizationStreamEvent(event); + expect(JSON.parse(encoded)).toEqual(event); + expect(encoded).not.toContain("messages"); + expect(encoded).not.toContain("Authorization"); +}); +~~~ + +- [ ] **Step 2:运行测试并确认类型或断言失败** + +Run: `npm test -- src/lib/workflow/__tests__/stream-events.test.ts` + +Expected: FAIL,因为 `OptimizationStreamEvent` 尚不接受 `llm_call_started`。 + +- [ ] **Step 3:创建共享类型并接入现有类型引用** + +`src/lib/llm/trace-types.ts` 必须定义以下完整公共边界: + +~~~ts +export type LlmProviderName = "deepseek" | "openai"; + +export type LlmTaskName = + | "unknown" + | "fact_extractor" + | "article_optimizer" + | "quality_inspector" + | "targeted_rewriter" + | "renwei_copy_optimizer"; + +export type LlmTraceWorkflowStage = + | "unknown" + | "input" + | "fact_card" + | "draft" + | "qa" + | "rewrite" + | "final"; + +export type LlmTraceRunStatus = + | "running" + | "completed" + | "failed" + | "interrupted"; + +export type LlmTraceCallStatus = + | "started" + | "responded" + | "validated" + | "failed"; + +export type LlmTraceCompleteness = "complete" | "incomplete"; +export type LlmBusinessStatus = "pass" | "warn" | "fail"; +export type LlmTraceErrorType = + | "provider" + | "json_parse" + | "schema_validation"; + +export interface LlmTraceContext { + workflow_stage: LlmTraceWorkflowStage; + rewrite_round?: number; + schema_name?: string; +} + +export interface LlmTraceRun { + job_id: string; + case_id: string | null; + status: LlmTraceRunStatus; + current_stage: LlmTraceWorkflowStage; + trace_completeness: LlmTraceCompleteness; + error_stage: string | null; + error_summary: string | null; + started_at: string; + finished_at: string | null; + updated_at: string; +} + +export interface LlmTraceCall { + call_id: string; + job_id: string; + sequence: number; + task: LlmTaskName; + workflow_stage: LlmTraceWorkflowStage; + rewrite_round: number | null; + provider: LlmProviderName; + model: string; + status: LlmTraceCallStatus; + request_object_key: string | null; + response_object_key: string | null; + token_usage: Record | null; + schema_name: string | null; + schema_valid: boolean | null; + validation_issues: string[]; + business_status: LlmBusinessStatus | null; + duration_ms: number | null; + started_at: string; + responded_at: string | null; + validated_at: string | null; + failed_at: string | null; + error_type: LlmTraceErrorType | null; + error_summary: string | null; +} + +export type LlmTraceCallPublic = Omit< + LlmTraceCall, + "request_object_key" | "response_object_key" +> & { + request_available: boolean; + response_available: boolean; +}; + +export interface LlmTraceManifest { + run: LlmTraceRun; + calls: LlmTraceCallPublic[]; +} + +export type LlmClientTraceEvent = + | { + type: "started"; + call_id: string; + task: LlmTaskName; + context: LlmTraceContext; + provider: LlmProviderName; + model: string; + request: unknown; + started_at: string; + } + | { + type: "responded"; + call_id: string; + response: unknown; + duration_ms: number; + responded_at: string; + } + | { + type: "validated"; + call_id: string; + schema_name: string; + schema_valid: boolean; + validation_issues: string[]; + validated_at: string; + } + | { + type: "failed"; + call_id: string; + error_type: LlmTraceErrorType; + error_summary: string; + duration_ms: number; + failed_at: string; + }; + +export type LlmClientTraceHandler = ( + event: LlmClientTraceEvent, +) => void | Promise; + +export type LlmTraceStreamEvent = + | { + type: "llm_call_started"; + job_id: string; + call_id: string; + sequence: number; + task: LlmTaskName; + workflow_stage: LlmTraceWorkflowStage; + rewrite_round: number | null; + provider: LlmProviderName; + model: string; + started_at: string; + request_available: boolean; + } + | { + type: "llm_call_responded"; + job_id: string; + call_id: string; + duration_ms: number; + token_usage: Record | null; + responded_at: string; + response_available: boolean; + } + | { + type: "llm_call_validated"; + job_id: string; + call_id: string; + schema_name: string; + schema_valid: boolean; + validation_issues: string[]; + validated_at: string; + } + | { + type: "llm_call_failed"; + job_id: string; + call_id: string; + error_type: LlmTraceErrorType; + error_summary: string; + failed_at: string; + } + | { + type: "trace_warning"; + job_id: string; + trace_completeness: "incomplete"; + error_summary: string; + }; +~~~ + +把 `LlmTaskName` 从 `client.ts` 移到该文件;`audit.ts` 和 `client.ts` 都从该文件导入。`stream-events.ts` 使用: + +~~~ts +import type { LlmTraceStreamEvent } from "../llm/trace-types"; + +type ExistingOptimizationStreamEvents = + | { + type: "job_created"; + job: { id: string }; + case?: { id: string; case_type: "article" }; + } + | { type: "fact_card_ready"; job_id: string; fact_card: OptimizationFactCard } + | { type: "draft_started"; job_id: string; message: string } + | { type: "draft_ready"; job_id: string; article: OptimizedArticle } + | { type: "qa_started"; job_id: string; message: string } + | { type: "qa_ready"; job_id: string; qa_report: QaReport } + | { type: "rewrite_started"; job_id: string; round: number } + | { + type: "rewrite_ready"; + job_id: string; + round: number; + article: OptimizedArticle; + } + | { + type: "final_ready"; + job_id: string; + case?: { id: string; case_type: "article" }; + result_version?: { id: string; version: number }; + optimized_article: OptimizedArticle; + qa_report: QaReport; + export_paths: Record; + } + | { + type: "failed"; + job_id?: string; + case?: { id: string; case_type: "article" }; + result_version?: { id: string; version: number }; + stage: OptimizationStreamStage; + error: string; + }; + +export type OptimizationStreamEvent = + | LlmTraceStreamEvent + | ExistingOptimizationStreamEvents; +~~~ + +现有业务事件字段和值保持原样,只增加类型别名和新的联合分支。 + +- [ ] **Step 4:运行流事件测试并确认通过** + +Run: `npm test -- src/lib/workflow/__tests__/stream-events.test.ts` + +Expected: PASS,且现有分块 NDJSON 测试继续通过。 + +- [ ] **Step 5:提交契约变更** + +~~~bash +git add src/lib/llm/trace-types.ts src/lib/llm/audit.ts src/lib/llm/client.ts src/lib/workflow/stream-events.ts src/lib/workflow/__tests__/stream-events.test.ts +git commit -m "定义LLM追踪事件契约" +~~~ + +## Task 2:增加 D1 和 SQLite 追踪索引 + +**Files:** +- Create: `migrations/0004_llm_trace_observability.sql` +- Create: `src/lib/llm/trace-repository.ts` +- Create: `src/lib/llm/sqlite-trace-repository.ts` +- Create: `src/lib/llm/d1-trace-repository.ts` +- Create: `src/lib/llm/__tests__/trace-repository.test.ts` +- Create: `src/lib/llm/__tests__/d1-trace-repository.test.ts` +- Modify: `src/lib/db/schema.ts` + +- [ ] **Step 1:写 SQLite 仓储失败测试** + +测试必须先创建真实文章任务,再验证运行、调用、最近任务和删除: + +~~~ts +it("stores a run and ordered calls, then deletes only trace rows", async () => { + const appRepository = createSqliteRepository(dbPath); + const job = await appRepository.createArticleJob({ + source_title: "Title", + source_body: "Body", + image_inputs: [], + publish_platform: "official_site", + user_instructions: "", + }); + const repository = createSqliteTraceRepository(dbPath); + + await repository.putRun(runFixture(job.id)); + await repository.putCall(callFixture(job.id, "llmcall_1", 1)); + await repository.putCall(callFixture(job.id, "llmcall_2", 2)); + + await expect(repository.getLatestRun()).resolves.toMatchObject({ + job_id: job.id, + status: "running", + }); + await expect(repository.listCalls(job.id)).resolves.toEqual([ + expect.objectContaining({ call_id: "llmcall_1", sequence: 1 }), + expect.objectContaining({ call_id: "llmcall_2", sequence: 2 }), + ]); + + await repository.deleteRun(job.id); + await expect(appRepository.getArticleJob(job.id)).resolves.not.toBeNull(); + await expect(repository.getRun(job.id)).resolves.toBeNull(); +}); +~~~ + +- [ ] **Step 2:运行测试并确认表或模块不存在** + +Run: `npm test -- src/lib/llm/__tests__/trace-repository.test.ts` + +Expected: FAIL,模块或 `llm_trace_runs` 表不存在。 + +- [ ] **Step 3:添加完整迁移和本地 Schema** + +`migrations/0004_llm_trace_observability.sql`: + +~~~sql +CREATE TABLE IF NOT EXISTS llm_trace_runs ( + job_id TEXT PRIMARY KEY, + case_id TEXT, + status TEXT NOT NULL, + current_stage TEXT NOT NULL, + trace_completeness TEXT NOT NULL, + error_stage TEXT, + error_summary TEXT, + started_at TEXT NOT NULL, + finished_at TEXT, + updated_at TEXT NOT NULL, + FOREIGN KEY (job_id) REFERENCES article_jobs(id) ON DELETE CASCADE, + FOREIGN KEY (case_id) REFERENCES optimization_cases(id) ON DELETE SET NULL +); + +CREATE TABLE IF NOT EXISTS llm_trace_calls ( + call_id TEXT PRIMARY KEY, + job_id TEXT NOT NULL, + sequence INTEGER NOT NULL, + task TEXT NOT NULL, + workflow_stage TEXT NOT NULL, + rewrite_round INTEGER, + provider TEXT NOT NULL, + model TEXT NOT NULL, + status TEXT NOT NULL, + request_object_key TEXT, + response_object_key TEXT, + token_usage TEXT, + schema_name TEXT, + schema_valid INTEGER, + validation_issues TEXT NOT NULL DEFAULT '[]', + business_status TEXT, + duration_ms INTEGER, + started_at TEXT NOT NULL, + responded_at TEXT, + validated_at TEXT, + failed_at TEXT, + error_type TEXT, + error_summary TEXT, + UNIQUE (job_id, sequence), + FOREIGN KEY (job_id) REFERENCES llm_trace_runs(job_id) ON DELETE CASCADE +); + +CREATE INDEX IF NOT EXISTS idx_llm_trace_runs_status_updated + ON llm_trace_runs(status, updated_at DESC); + +CREATE INDEX IF NOT EXISTS idx_llm_trace_calls_job_sequence + ON llm_trace_calls(job_id, sequence); +~~~ + +把同一表结构加入 `initializeSchema()`,保持本地 SQLite 与 D1 一致。 + +- [ ] **Step 4:定义仓储接口和运行时选择** + +`trace-repository.ts`: + +~~~ts +export interface LlmTraceRepository { + putRun(run: LlmTraceRun): Promise; + putCall(call: LlmTraceCall): Promise; + getRun(jobId: string): Promise; + getLatestRun(): Promise; + listCalls(jobId: string): Promise; + listTerminalRunsExcept(jobId: string): Promise; + deleteRun(jobId: string): Promise; +} + +export function getLlmTraceRepositoryFromRuntime(): LlmTraceRepository { + if (process.env.APP_RUNTIME === "cloudflare") { + const env = getAppCloudflareEnv(); + if (!env?.DB) throw new Error("Cloudflare D1 binding DB is required"); + return createD1TraceRepository(env.DB); + } + return createSqliteTraceRepository(getDefaultDatabasePath()); +} +~~~ + +使用 Task 1 已定义的 `LlmTraceRun` 和 `LlmTraceCall`;其字段与迁移一一对应。 + +- [ ] **Step 5:实现 SQLite 和 D1 UPSERT** + +两个实现都使用完整快照 UPSERT,避免动态拼 SQL。SQLite 的核心语句为: + +~~~sql +INSERT INTO llm_trace_calls ( + call_id, job_id, sequence, task, workflow_stage, rewrite_round, + provider, model, status, request_object_key, response_object_key, + token_usage, schema_name, schema_valid, validation_issues, + business_status, duration_ms, started_at, responded_at, validated_at, + failed_at, error_type, error_summary +) VALUES ( + @call_id, @job_id, @sequence, @task, @workflow_stage, @rewrite_round, + @provider, @model, @status, @request_object_key, @response_object_key, + @token_usage, @schema_name, @schema_valid, @validation_issues, + @business_status, @duration_ms, @started_at, @responded_at, @validated_at, + @failed_at, @error_type, @error_summary +) +ON CONFLICT(call_id) DO UPDATE SET + status = excluded.status, + response_object_key = excluded.response_object_key, + token_usage = excluded.token_usage, + schema_name = excluded.schema_name, + schema_valid = excluded.schema_valid, + validation_issues = excluded.validation_issues, + business_status = excluded.business_status, + duration_ms = excluded.duration_ms, + responded_at = excluded.responded_at, + validated_at = excluded.validated_at, + failed_at = excluded.failed_at, + error_type = excluded.error_type, + error_summary = excluded.error_summary; +~~~ + +D1 使用相同列顺序的 `prepare().bind().run()`;读取时把 JSON 文本和 `0 | 1 | null` 转回 TypeScript 类型。`getLatestRun()` 的顺序固定为运行中优先,再按 `updated_at DESC`: + +~~~sql +SELECT * FROM llm_trace_runs +ORDER BY CASE WHEN status = 'running' THEN 0 ELSE 1 END, updated_at DESC +LIMIT 1; +~~~ + +- [ ] **Step 6:增加 D1 绑定顺序测试并运行两套测试** + +D1 测试至少断言 `putRun()` 使用 `insert into llm_trace_runs`,`putCall()` 将 `validation_issues` 和 `token_usage` 序列化,`listCalls()` 按 `sequence` 读取。 + +Run: `npm test -- src/lib/llm/__tests__/trace-repository.test.ts src/lib/llm/__tests__/d1-trace-repository.test.ts src/lib/db/__tests__/repository.test.ts` + +Expected: PASS。 + +- [ ] **Step 7:提交索引存储** + +~~~bash +git add migrations/0004_llm_trace_observability.sql src/lib/db/schema.ts src/lib/llm/trace-repository.ts src/lib/llm/sqlite-trace-repository.ts src/lib/llm/d1-trace-repository.ts src/lib/llm/__tests__/trace-repository.test.ts src/lib/llm/__tests__/d1-trace-repository.test.ts +git commit -m "新增LLM追踪索引存储" +~~~ + +## Task 3:实现私有完整正文存储 + +**Files:** +- Create: `src/lib/llm/trace-payload-store.ts` +- Create: `src/lib/llm/__tests__/trace-payload-store.test.ts` + +- [ ] **Step 1:先写本地与 R2 失败测试** + +~~~ts +it("round-trips exact JSON locally and deletes one job prefix", async () => { + const store = createLocalTracePayloadStore(tempDir); + const payload = { model: "deepseek-v4-pro", messages: [{ role: "user", content: "原文" }] }; + const key = "llm-traces/job_1/llmcall_1/request.json"; + + await store.putJson(key, payload); + await expect(store.getJson(key)).resolves.toEqual(payload); + await store.deleteJob("job_1"); + await expect(store.getJson(key)).resolves.toBeNull(); +}); + +it("stores private JSON in R2 without a public URL", async () => { + const put = vi.fn().mockResolvedValue(undefined); + const get = vi.fn().mockResolvedValue({ json: async () => ({ ok: true }) }); + const bucket = { put, get } as unknown as R2Bucket; + const store = createR2TracePayloadStore(bucket); + + await store.putJson("llm-traces/job_1/llmcall_1/request.json", { ok: true }); + await expect(store.getJson("llm-traces/job_1/llmcall_1/request.json")) + .resolves.toEqual({ ok: true }); + expect(put).toHaveBeenCalledWith( + "llm-traces/job_1/llmcall_1/request.json", + JSON.stringify({ ok: true }), + { httpMetadata: { contentType: "application/json; charset=utf-8" } }, + ); +}); +~~~ + +- [ ] **Step 2:运行测试并确认模块不存在** + +Run: `npm test -- src/lib/llm/__tests__/trace-payload-store.test.ts` + +Expected: FAIL,正文存储工厂尚不存在。 + +- [ ] **Step 3:实现正文存储接口和工厂** + +~~~ts +export interface LlmTracePayloadStore { + putJson(key: string, value: unknown): Promise; + getJson(key: string): Promise; + deleteJob(jobId: string): Promise; +} + +export function tracePayloadKey( + jobId: string, + callId: string, + kind: "request" | "response", +) { + return `llm-traces/${jobId}/${callId}/${kind}.json`; +} + +export function getLlmTracePayloadStoreFromRuntime(): LlmTracePayloadStore { + if (process.env.APP_RUNTIME === "cloudflare") { + const bucket = getAppCloudflareEnv()?.EXPORT_BUCKET; + if (!bucket) throw new Error("Cloudflare R2 binding EXPORT_BUCKET is required"); + return createR2TracePayloadStore(bucket); + } + return createLocalTracePayloadStore(getAppDataDir()); +} +~~~ + +本地实现只能在 `join(dataDir, "llm-traces")` 下使用生成的键,使用 `mkdirSync`、`writeFileSync`、`readFileSync` 和 `rmSync`。R2 `deleteJob()` 必须循环 `bucket.list({ prefix, cursor })`,收集对象键后调用 `bucket.delete(keys)`,直到 `truncated` 为 false。 + +- [ ] **Step 4:运行正文存储和现有导出存储测试** + +Run: `npm test -- src/lib/llm/__tests__/trace-payload-store.test.ts src/lib/workflow/__tests__/export-store.test.ts` + +Expected: PASS;现有 `exports/` 行为不变。 + +- [ ] **Step 5:提交正文存储** + +~~~bash +git add src/lib/llm/trace-payload-store.ts src/lib/llm/__tests__/trace-payload-store.test.ts +git commit -m "新增LLM追踪正文存储" +~~~ + +## Task 4:实现追踪收集器、业务状态和保留策略 + +**Files:** +- Create: `src/lib/llm/trace-recorder.ts` +- Create: `src/lib/llm/__tests__/trace-recorder.test.ts` + +- [ ] **Step 1:写成功调用、QA 业务失败和保留策略测试** + +使用内存 fake 仓储和正文存储,覆盖完整顺序: + +~~~ts +it("persists exact bodies before publishing public metadata", async () => { + const published: LlmTraceStreamEvent[] = []; + const recorder = await createLlmTraceRecorder({ + jobId: "job_1", + caseId: "case_1", + repository, + payloadStore, + publish: (event) => published.push(event), + }); + + await recorder.onLlmEvent({ + type: "started", + call_id: "llmcall_1", + task: "quality_inspector", + context: { workflow_stage: "qa", schema_name: "llmQaPatchSchema" }, + provider: "deepseek", + model: "deepseek-v4-pro", + request: { model: "deepseek-v4-pro", messages: [] }, + started_at: "2026-07-16T00:00:00.000Z", + }); + + expect(payloadStore.values.get( + "llm-traces/job_1/llmcall_1/request.json", + )).toEqual({ model: "deepseek-v4-pro", messages: [] }); + expect(published[0]).toMatchObject({ + type: "llm_call_started", + request_available: true, + }); + expect(JSON.stringify(published)).not.toContain("messages"); +}); + +it("stores QA business failure separately from schema success", async () => { + await recorder.onWorkflowEvent({ + type: "qa_ready", + job_id: "job_1", + qa_report: { overall_status: "fail", checks: [] }, + }); + await expect(repository.listCalls("job_1")).resolves.toEqual([ + expect.objectContaining({ + task: "quality_inspector", + schema_valid: true, + business_status: "fail", + }), + ]); +}); + +it("keeps running runs and only the newest terminal full trace", async () => { + await recorder.finish({ status: "completed" }); + expect(payloadStore.deletedJobs).toEqual(["job_old_completed"]); + expect(repository.deletedRuns).toEqual(["job_old_completed"]); + expect(repository.deletedRuns).not.toContain("job_other_running"); +}); +~~~ + +- [ ] **Step 2:运行测试并确认收集器不存在** + +Run: `npm test -- src/lib/llm/__tests__/trace-recorder.test.ts` + +Expected: FAIL,`createLlmTraceRecorder` 尚不存在。 + +- [ ] **Step 3:实现收集器的公开接口** + +~~~ts +export interface LlmTraceRecorder { + onLlmEvent: LlmClientTraceHandler; + onWorkflowEvent(event: OptimizationStreamEvent): Promise; + finish(input: { + status: "completed" | "failed" | "interrupted"; + errorStage?: string; + errorSummary?: string; + }): Promise; +} + +export function createNoopLlmTraceRecorder(): LlmTraceRecorder { + return { + onLlmEvent: async () => undefined, + onWorkflowEvent: async () => undefined, + finish: async () => undefined, + }; +} + +export async function createLlmTraceRecorder({ + jobId, + caseId, + repository, + payloadStore, + publish, +}: CreateLlmTraceRecorderInput): Promise { + const calls = new Map(); + let sequence = 0; + let run = createRunningRun(jobId, caseId); + await repository.putRun(run); + + const warn = async (error: unknown) => { + run = { ...run, trace_completeness: "incomplete", updated_at: nowIso() }; + await repository.putRun(run).catch(() => undefined); + await publish({ + type: "trace_warning", + job_id: jobId, + trace_completeness: "incomplete", + error_summary: safeTraceError(error), + }); + }; + + return { + onLlmEvent: async (event) => { + try { + await applyLlmEvent(event); + } catch (error) { + await warn(error); + } + }, + onWorkflowEvent: async (event) => { + try { + await applyWorkflowEvent(event); + } catch (error) { + await warn(error); + } + }, + finish: async (input) => { + try { + await finishRun(input); + } catch (error) { + await warn(error); + } + }, + }; +} +~~~ + +`applyLlmEvent()` 的硬性顺序: + +1. `started`:先写 request.json,再 `putCall()`,最后发布 `llm_call_started`。 +2. `responded`:先写 response.json,再更新调用,最后发布 `llm_call_responded`。 +3. `validated`:更新 Schema 字段,再发布 `llm_call_validated`。 +4. `failed`:更新错误白名单字段,再发布 `llm_call_failed`。 + +若正文写入失败,`onLlmEvent()` 仍尝试写入不带对象键的调用索引并发布 `request_available: false` 或 `response_available: false`,随后发布 `trace_warning`;若索引也不可用,则至少发布 `trace_warning`。任何追踪失败都被收集器吞掉,不向 LLM 客户端或业务工作流抛出。 + +Token 用量只从完整响应的顶层 `usage` 读取为数字字典: + +~~~ts +function tokenUsageFromResponse(response: unknown) { + if (!response || typeof response !== "object") return null; + const usage = (response as { usage?: unknown }).usage; + if (!usage || typeof usage !== "object") return null; + return Object.fromEntries( + Object.entries(usage).filter( + (entry): entry is [string, number] => typeof entry[1] === "number", + ), + ); +} +~~~ + +- [ ] **Step 4:实现工作流状态和清理** + +`onWorkflowEvent()` 必须更新 `current_stage`;收到 `qa_ready` 时,把最近一个 `quality_inspector` 调用的 `business_status` 设置为 `qa_report.overall_status`。`finish()` 先写终态,再执行: + +~~~ts +const expired = await repository.listTerminalRunsExcept(jobId); +for (const oldRun of expired) { + await payloadStore.deleteJob(oldRun.job_id); + await repository.deleteRun(oldRun.job_id); +} +~~~ + +`listTerminalRunsExcept()` 不返回任何 `running` 记录,因此并发运行任务不会被删除。 + +- [ ] **Step 5:运行收集器测试** + +Run: `npm test -- src/lib/llm/__tests__/trace-recorder.test.ts` + +Expected: PASS,包含存储失败时业务回调不抛错、`trace_warning` 被发布的断言。 + +- [ ] **Step 6:提交追踪协调器** + +~~~bash +git add src/lib/llm/trace-recorder.ts src/lib/llm/__tests__/trace-recorder.test.ts +git commit -m "新增LLM任务追踪收集器" +~~~ + +## Task 5:在 LLM 客户端捕获真实 SDK 请求和响应 + +**Files:** +- Modify: `src/lib/llm/client.ts` +- Modify: `src/lib/llm/__tests__/client.test.ts` + +- [ ] **Step 1:写请求和响应深度相等的失败测试** + +~~~ts +it("traces the exact SDK request and full SDK response", async () => { + process.env.LLM_PROVIDER = "deepseek"; + process.env.DEEPSEEK_API_KEY = "test-key"; + const traced: LlmClientTraceEvent[] = []; + let sdkRequest: unknown; + const sdkResponse = { + id: "chatcmpl_1", + object: "chat.completion", + created: 1784188800, + model: "deepseek-v4-pro", + choices: [{ + index: 0, + message: { role: "assistant", content: "{\"value\":\"ok\"}" }, + finish_reason: "stop", + }], + usage: { prompt_tokens: 10, completion_tokens: 4, total_tokens: 14 }, + }; + client.setChatCompletionForTesting(async (request) => { + sdkRequest = request; + return sdkResponse; + }); + + await client.generateValidatedJson({ + schema: z.object({ value: z.string() }), + schemaName: "valueSchema", + prompt: "Return JSON.", + task: "article_optimizer", + traceStage: "draft", + onTraceEvent: (event) => traced.push(event), + }); + + expect(traced.find((event) => event.type === "started")).toMatchObject({ + type: "started", + request: sdkRequest, + }); + expect(traced.find((event) => event.type === "responded")).toMatchObject({ + type: "responded", + response: sdkResponse, + }); + expect(traced.map((event) => event.type)).toEqual([ + "started", + "responded", + "validated", + ]); +}); +~~~ + +再增加 JSON 解析失败顺序和 Schema 失败顺序测试: + +~~~ts +expect(jsonParseEvents.map((event) => event.type)).toEqual([ + "started", "responded", "failed", +]); +expect(schemaEvents.map((event) => event.type)).toEqual([ + "started", "responded", "validated", "failed", +]); +~~~ + +- [ ] **Step 2:运行客户端测试并确认失败** + +Run: `npm test -- src/lib/llm/__tests__/client.test.ts` + +Expected: FAIL,`schemaName`、`traceStage` 和 `onTraceEvent` 尚不存在,响应也会在解析后丢失。 + +- [ ] **Step 3:扩展 GenerateInput 并生成稳定调用 ID** + +~~~ts +export interface GenerateInput { + system?: string; + prompt: string; + model?: string; + temperature?: number; + task?: LlmTaskName; + schemaName?: string; + traceStage?: LlmTraceWorkflowStage; + rewriteRound?: number; + onTraceEvent?: LlmClientTraceHandler; + onAuditSummary?: (summary: LlmAuditSummary) => void | Promise; + traceCallId?: string; +} + +function callIdFor(input: GenerateInput) { + return input.traceCallId ?? `llmcall_${nanoid(12)}`; +} + +async function emitTrace(input: GenerateInput, event: LlmClientTraceEvent) { + try { + await input.onTraceEvent?.(event); + } catch (error) { + console.warn(`[llm:trace-warning] ${safeErrorSummary(error)}`); + } +} +~~~ + +`generateValidatedJson()` 在调用生成器前创建 `traceCallId`,并把同一个 ID 传给 `generateJsonForValidation()`,保证校验事件属于同一调用。 + +- [ ] **Step 4:拆开 Provider、JSON 解析和 Schema 校验阶段** + +`generateJson()` 必须按以下顺序实现,不能把 `JSON.parse()` 放回 Provider `try/catch`: + +~~~ts +const request = buildChatCompletionRequest(input, effectiveModel); +await emitTrace(input, { + type: "started", + call_id: callId, + task, + context: { + workflow_stage: input.traceStage ?? "unknown", + rewrite_round: input.rewriteRound, + schema_name: input.schemaName, + }, + provider: status.provider, + model: effectiveModel, + request, + started_at: new Date(startedAt).toISOString(), +}); + +let response: ChatCompletionResult; +try { + response = chatCompletionForTesting + ? await chatCompletionForTesting(request) + : await createClient().client.chat.completions.create(request); +} catch (error) { + await emitTrace(input, failedEvent(callId, "provider", error, startedAt)); + throw normalizeLlmError(error); +} + +await emitTrace(input, { + type: "responded", + call_id: callId, + response, + duration_ms: Date.now() - startedAt, + responded_at: new Date().toISOString(), +}); + +const content = response.choices[0]?.message.content ?? "{}"; +try { + return JSON.parse(content) as T; +} catch (error) { + await emitTrace(input, failedEvent(callId, "json_parse", error, startedAt)); + throw new Error(`LLM response is not valid JSON: ${safeErrorSummary(error)}`); +} +~~~ + +Schema 校验失败时先发送 `validated(schema_valid=false)`,再发送 `failed(error_type=schema_validation)`。安全错误只允许 `name` 和 `message`,不得序列化 SDK error、headers、request 或环境变量。 + +- [ ] **Step 5:保持现有审计和控制台日志兼容** + +现有 `[llm:start]`、`[llm:response]`、`[llm:validated]`、`[llm:error]` 和 `onAuditSummary` 行为必须保留。现有原始返回控制台日志仍按 `LLM_LOG_RAW_LIMIT` 截断;R2 追踪正文不截断。 + +- [ ] **Step 6:运行客户端与审计测试** + +Run: `npm test -- src/lib/llm/__tests__/client.test.ts src/lib/llm/__tests__/audit.test.ts` + +Expected: PASS,且失败测试证明 API Key 不出现在任何事件的 JSON 中。 + +- [ ] **Step 7:提交客户端追踪** + +~~~bash +git add src/lib/llm/client.ts src/lib/llm/__tests__/client.test.ts +git commit -m "记录真实LLM请求与响应" +~~~ + +## Task 6:把追踪器接入真实流式文章工作流 + +**Files:** +- Modify: `src/lib/workflow/fact-extractor.ts` +- Modify: `src/lib/workflow/article-optimizer.ts` +- Modify: `src/lib/workflow/quality-inspector.ts` +- Modify: `src/lib/workflow/targeted-rewriter.ts` +- Modify: `src/lib/workflow/streaming-optimizer.ts` +- Modify: `src/app/api/jobs/optimize-stream/route.ts` +- Modify: `src/app/api/__tests__/jobs.test.ts` + +- [ ] **Step 1:写流式事件顺序的失败测试** + +在 `jobs.test.ts` 中让 LLM mock 主动调用传入的追踪回调,并断言: + +~~~ts +expect(events.map((event) => event.type)).toEqual([ + "job_created", + "llm_call_started", + "llm_call_responded", + "llm_call_validated", + "fact_card_ready", + "draft_started", + "llm_call_started", + "llm_call_responded", + "llm_call_validated", + "draft_ready", + "qa_started", + "llm_call_started", + "llm_call_responded", + "llm_call_validated", + "qa_ready", + "final_ready", +]); +~~~ + +还要断言公开事件 JSON 不包含文章正文、`messages`、完整 `choices` 或访问密钥。 + +- [ ] **Step 2:运行 API 测试并确认缺少追踪事件** + +Run: `npm test -- src/app/api/__tests__/jobs.test.ts` + +Expected: FAIL,事件序列仍只有业务阶段事件。 + +- [ ] **Step 3:给四个工作流节点传递明确追踪上下文** + +每个输入接口增加: + +~~~ts +onTraceEvent?: GenerateInput["onTraceEvent"]; +rewriteRound?: number; +~~~ + +四个 `generateValidatedJson()` 调用必须分别传入: + +~~~ts +// fact-extractor.ts +schemaName: "candidateFactCardSchema", +traceStage: "fact_card", +onTraceEvent: options.onTraceEvent, + +// article-optimizer.ts +schemaName: "optimizedArticleSchema", +traceStage: "draft", +onTraceEvent, + +// quality-inspector.ts +schemaName: "llmQaPatchSchema", +traceStage: "qa", +rewriteRound: input.rewriteRound, +onTraceEvent: input.onTraceEvent, + +// targeted-rewriter.ts +schemaName: "optimizedArticleSchema", +traceStage: "rewrite", +rewriteRound, +onTraceEvent, +~~~ + +`streaming-optimizer.ts` 初次 QA 使用 `rewriteRound: 0`;每轮修复和复检使用当前 `nextRound`。 + +- [ ] **Step 4:在流式路由创建并连接追踪器** + +在 `ReadableStream.start()` 的业务 `try` 之前,与 `jobId`、`caseId`、`stage` 同级声明空追踪器,使失败 `catch` 始终能访问: + +~~~ts +let jobId: string | undefined; +let caseId: string | undefined; +let stage: OptimizationStreamStage = "job"; +let traceRecorder = createNoopLlmTraceRecorder(); +~~~ + +任务和案例创建完成后,再尝试替换为真实追踪器: + +~~~ts +try { + traceRecorder = await createLlmTraceRecorder({ + jobId: job.id, + caseId: optimizationCase.id, + repository: getLlmTraceRepositoryFromRuntime(), + payloadStore: getLlmTracePayloadStoreFromRuntime(), + publish: async (event) => send(event), + }); +} catch (error) { + send({ + type: "trace_warning", + job_id: job.id, + trace_completeness: "incomplete", + error_summary: safeTraceError(error), + }); +} +~~~ + +事实提取和 `runStreamingOptimizationWorkflow()` 都传 `onTraceEvent: traceRecorder.onLlmEvent`。保存事实卡后,把直接发送改为同一事件先入追踪器、再入 NDJSON: + +~~~ts +const factCardReadyEvent: OptimizationStreamEvent = { + type: "fact_card_ready", + job_id: job.id, + fact_card: savedFactCard, +}; +await traceRecorder.onWorkflowEvent(factCardReadyEvent); +send(factCardReadyEvent); +~~~ + +工作流的 `onEvent` 改为: + +~~~ts +onEvent: async (event) => { + stage = stageForEvent(event, stage); + await traceRecorder.onWorkflowEvent(event); + send(event); +}, +~~~ + +成功路径在 `final_ready` 前调用 `traceRecorder.finish({ status: "completed" })`;失败路径在 `failed` 前调用: + +~~~ts +await traceRecorder.finish({ + status: "failed", + errorStage: stage, + errorSummary: message, +}); +~~~ + +`safeTraceError()` 从 `trace-recorder.ts` 导出,只返回安全化的 `name: message`。追踪器创建或存储异常必须被隔离成 `trace_warning`,不得跳入业务失败分支。 + +- [ ] **Step 5:运行路由、工作流和流事件测试** + +Run: `npm test -- src/app/api/__tests__/jobs.test.ts src/lib/workflow/__tests__/streaming-optimizer.test.ts src/lib/workflow/__tests__/stream-events.test.ts` + +Expected: PASS,现有业务事件和结果版本断言保持不变。 + +- [ ] **Step 6:提交真实工作流接入** + +~~~bash +git add src/lib/workflow/fact-extractor.ts src/lib/workflow/article-optimizer.ts src/lib/workflow/quality-inspector.ts src/lib/workflow/targeted-rewriter.ts src/lib/workflow/streaming-optimizer.ts src/app/api/jobs/optimize-stream/route.ts src/app/api/__tests__/jobs.test.ts +git commit -m "接入文章优化LLM追踪" +~~~ + +## Task 7:增加受保护的追踪读取 API + +**Files:** +- Create: `src/lib/llm/trace-http.ts` +- Create: `src/app/api/llm-traces/latest/route.ts` +- Create: `src/app/api/jobs/[jobId]/llm-trace/route.ts` +- Create: `src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts` +- Create: `src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts` +- Create: `src/app/api/__tests__/llm-traces.test.ts` + +- [ ] **Step 1:先写鉴权、无缓存和正文测试** + +~~~ts +it("rejects trace reads without API access", async () => { + const response = await getLatestTrace(request(null)); + expect(response.status).toBe(401); +}); + +it("returns a no-store manifest without raw bodies", async () => { + const response = await getJobTrace( + request("test-key"), + params({ jobId: "job_1" }), + ); + const body = await response.json(); + expect(response.headers.get("cache-control")).toBe("no-store"); + expect(body.calls[0]).toMatchObject({ + call_id: "llmcall_1", + request_available: true, + response_available: true, + }); + expect(JSON.stringify(body)).not.toContain("messages"); + expect(JSON.stringify(body)).not.toContain("choices"); +}); + +it("returns the exact stored request through the protected payload route", async () => { + const response = await getTraceRequest( + request("test-key"), + params({ jobId: "job_1", callId: "llmcall_1" }), + ); + expect(response.headers.get("cache-control")).toBe("no-store"); + await expect(response.json()).resolves.toEqual(exactRequestFixture); +}); +~~~ + +- [ ] **Step 2:运行 API 测试并确认路由不存在** + +Run: `npm test -- src/app/api/__tests__/llm-traces.test.ts` + +Expected: FAIL,读取路由尚不存在。 + +- [ ] **Step 3:实现共享 HTTP 助手** + +~~~ts +export function noStoreJson(body: unknown, status = 200) { + return NextResponse.json(body, { + status, + headers: { "cache-control": "no-store" }, + }); +} + +export async function readTracePayload({ + jobId, + callId, + kind, +}: { + jobId: string; + callId: string; + kind: "request" | "response"; +}) { + const repository = getLlmTraceRepositoryFromRuntime(); + const call = (await repository.listCalls(jobId)).find( + (candidate) => candidate.call_id === callId, + ); + if (!call) return noStoreJson({ error: "追踪调用不存在" }, 404); + const key = kind === "request" + ? call.request_object_key + : call.response_object_key; + if (!key) { + if (kind === "response" && call.status === "started") { + return noStoreJson({ state: "waiting" }, 202); + } + return noStoreJson({ + error: kind === "response" ? "该调用未产生响应" : "追踪请求正文不存在", + }, 404); + } + const payload = await getLlmTracePayloadStoreFromRuntime().getJson(key); + return payload == null + ? noStoreJson({ error: "追踪正文不存在" }, 404) + : noStoreJson(payload); +} +~~~ + +- [ ] **Step 4:实现四个 GET 路由** + +每个路由第一段必须完全一致: + +~~~ts +const access = requireApiAccess(request); +if (!access.ok) return access.response; +~~~ + +`latest` 调用 `repository.getLatestRun()`;无记录时返回 `noStoreJson({ run: null, calls: [] })`。有记录时,`latest` 和任务清单路由都调用 `repository.listCalls(run.job_id)`,并通过同一个安全映射函数返回: + +~~~ts +export function toPublicTraceCall(call: LlmTraceCall): LlmTraceCallPublic { + const { + request_object_key: requestObjectKey, + response_object_key: responseObjectKey, + ...publicFields + } = call; + return { + ...publicFields, + request_available: Boolean(requestObjectKey), + response_available: Boolean(responseObjectKey), + }; +} + +return noStoreJson({ run, calls: calls.map(toPublicTraceCall) }); +~~~ + +`toPublicTraceCall()` 放在 `trace-http.ts`,保证内部 R2 对象键不会进入响应对象。 + +- [ ] **Step 5:运行追踪 API 和鉴权回归测试** + +Run: `npm test -- src/app/api/__tests__/llm-traces.test.ts src/lib/api/__tests__/auth.test.ts` + +Expected: PASS,401、404、202、200 和 `no-store` 均有断言。 + +- [ ] **Step 6:提交追踪读取 API** + +~~~bash +git add src/lib/llm/trace-http.ts src/app/api/llm-traces/latest/route.ts 'src/app/api/jobs/[jobId]/llm-trace/route.ts' 'src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts' 'src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts' src/app/api/__tests__/llm-traces.test.ts +git commit -m "新增LLM追踪只读接口" +~~~ + +## Task 8:实现浏览器 API 客户端和架构状态派生 + +**Files:** +- Create: `src/components/architecture/api-client.ts` +- Create: `src/components/architecture/trace-state.ts` +- Create: `src/components/architecture/__tests__/api-client.test.ts` +- Create: `src/components/architecture/__tests__/trace-state.test.ts` + +- [ ] **Step 1:先写按需加载和状态派生失败测试** + +~~~ts +it("loads request bodies only when explicitly requested", async () => { + vi.stubGlobal("fetch", vi.fn(async () => jsonResponse(exactRequest))); + await expect( + getTracePayload("job_1", "llmcall_1", "request", "test-key"), + ).resolves.toEqual(exactRequest); + expect(fetch).toHaveBeenCalledWith( + "/api/jobs/job_1/llm-trace/llmcall_1/request", + expect.objectContaining({ + credentials: "same-origin", + headers: { "x-api-key": "test-key" }, + cache: "no-store", + }), + ); +}); + +it("distinguishes QA business failure from schema failure", () => { + const nodes = deriveArchitectureNodes(run, [ + call({ task: "quality_inspector", schema_valid: true, business_status: "fail" }), + ]); + expect(nodes.qa).toMatchObject({ + status: "completed", + detail: "检查完成,需要修复", + }); + + const failedNodes = deriveArchitectureNodes(run, [ + call({ task: "quality_inspector", schema_valid: false, status: "failed" }), + ]); + expect(failedNodes.qa.status).toBe("failed"); +}); +~~~ + +- [ ] **Step 2:运行测试并确认模块不存在** + +Run: `npm test -- src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts` + +Expected: FAIL。 + +- [ ] **Step 3:实现 API 客户端** + +~~~ts +function traceHeaders(apiAccessKey: string) { + return apiAccessKey ? { "x-api-key": apiAccessKey } : {}; +} + +async function traceFetch(path: string, apiAccessKey: string): Promise { + const response = await fetch(path, { + credentials: "same-origin", + headers: traceHeaders(apiAccessKey), + cache: "no-store", + }); + const body = (await response.json().catch(() => ({}))) as T & { error?: string }; + if (!response.ok) throw new Error(body.error ?? "读取后台追踪失败"); + return body; +} + +export function getLatestTrace(apiAccessKey: string) { + return traceFetch( + "/api/llm-traces/latest", + apiAccessKey, + ); +} + +export function getJobTrace(jobId: string, apiAccessKey: string) { + return traceFetch( + `/api/jobs/${encodeURIComponent(jobId)}/llm-trace`, + apiAccessKey, + ); +} + +export function getTracePayload( + jobId: string, + callId: string, + kind: "request" | "response", + apiAccessKey: string, +) { + return traceFetch( + `/api/jobs/${encodeURIComponent(jobId)}/llm-trace/${encodeURIComponent(callId)}/${kind}`, + apiAccessKey, + ); +} +~~~ + +- [ ] **Step 4:实现实时事件归并和固定节点派生** + +`trace-state.ts` 导出: + +~~~ts +export type ArchitectureNodeStatus = + | "waiting" + | "running" + | "completed" + | "failed"; + +export interface ArchitectureNodeView { + id: "input" | "fact_card" | "draft" | "qa" | "rewrite" | "final"; + label: string; + status: ArchitectureNodeStatus; + detail: string; +} + +export function applyLiveTraceEvent( + state: LlmTraceManifest, + event: LlmTraceStreamEvent, +): LlmTraceManifest; + +export function deriveArchitectureNodes( + run: LlmTraceRun | null, + calls: LlmTraceCallPublic[], +): Record; +~~~ + +归并必须按 `call_id` 更新而不是追加重复调用;调用清单始终按 `sequence` 排序。`trace_warning` 只把运行记录改为 `incomplete`,不把业务状态改成 `failed`。 + +- [ ] **Step 5:运行前端纯逻辑测试** + +Run: `npm test -- src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts` + +Expected: PASS,覆盖重复事件、复检轮次、Provider 失败、Schema 失败和业务 QA 失败。 + +- [ ] **Step 6:提交前端数据层** + +~~~bash +git add src/components/architecture/api-client.ts src/components/architecture/trace-state.ts src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts +git commit -m "新增后台架构前端状态层" +~~~ + +## Task 9:构建固定架构图和完整日志详情组件 + +**Files:** +- Create: `src/components/architecture/architecture-flow.tsx` +- Create: `src/components/architecture/llm-call-detail.tsx` +- Create: `src/components/architecture/architecture-observer-panel.tsx` +- Modify: `src/app/globals.css` +- Test: `src/components/architecture/__tests__/trace-state.test.ts` + +- [ ] **Step 1:先写界面格式化失败测试** + +把界面格式化保持为纯函数并测试: + +~~~ts +it("formats every architecture node and call label in Chinese", () => { + expect(formatNodeStatus("running")).toBe("运行中"); + expect(formatNodeStatus("failed")).toBe("失败"); + expect(formatCallLabel(call({ + sequence: 4, + task: "targeted_rewriter", + rewrite_round: 1, + duration_ms: 22600, + }))).toBe("4 · 定向修复第 1 轮 · 22.6 秒"); +}); +~~~ + +- [ ] **Step 2:运行测试并确认格式化函数不存在** + +Run: `npm test -- src/components/architecture/__tests__/trace-state.test.ts` + +Expected: FAIL。 + +- [ ] **Step 3:实现固定拓扑组件** + +`architecture-flow.tsx` 只接受派生后的节点,不读取 API: + +~~~tsx +export function ArchitectureFlow({ + nodes, +}: { + nodes: Record; +}) { + const order: ArchitectureNodeView["id"][] = [ + "input", "fact_card", "draft", "qa", "rewrite", "final", + ]; + return ( +
+ {order.map((id) => { + const node = nodes[id]; + return ( +
+ {node.label} + {formatNodeStatus(node.status)} + {node.detail} +
+ ); + })} +

质量检查未通过时,定向修复后返回复检,最多两轮。

+
+ ); +} +~~~ + +固定节点不能按调用次数新增;复检只改变 QA 节点并在调用轨迹中增加一条调用。 + +- [ ] **Step 4:实现调用详情按需读取** + +`llm-call-detail.tsx` 接受 `jobId`、`call`、`apiAccessKey` 和 `technicalDetailsEnabled`。只有技术详情开启且用户选择 `请求` 或 `响应` 时才调用 `getTracePayload()`: + +~~~tsx +const [tab, setTab] = useState<"request" | "response" | "validation">( + "validation", +); +const [payload, setPayload] = useState(null); + +useEffect(() => { + if (!technicalDetailsEnabled || tab === "validation") return; + let current = true; + setPayload(null); + getTracePayload(jobId, call.call_id, tab, apiAccessKey) + .then((value) => current && setPayload(value)) + .catch((error) => current && setPayload({ error: errorMessage(error) })); + return () => { current = false; }; +}, [apiAccessKey, call.call_id, jobId, tab, technicalDetailsEnabled]); + +return ( +
+
+ {(["request", "response", "validation"] as const).map((value) => ( + + ))} +
+
+      {JSON.stringify(
+        tab === "validation" ? validationView(call) : payload,
+        null,
+        2,
+      )}
+    
+
+); +~~~ + +必须使用 `
` 文本渲染,不使用 `dangerouslySetInnerHTML`、Markdown HTML 或代码高亮器。
+
+- [ ] **Step 5:实现只读观察容器**
+
+`architecture-observer-panel.tsx` 的 props 固定为:
+
+~~~ts
+interface ArchitectureObserverPanelProps {
+  apiAccessKey: string;
+  currentJobId: string | null;
+  liveEvents: LlmTraceStreamEvent[];
+}
+~~~
+
+挂载时优先 `getJobTrace(currentJobId)`,没有当前任务时 `getLatestTrace()`;之后用 `applyLiveTraceEvent()` 合并 `liveEvents`。页面必须有:
+
+- “当前任务”或“最近一次任务”明确标签。
+- Provider、模型、运行状态和追踪完整性。
+- `ArchitectureFlow`。
+- 按 `sequence` 排列的调用按钮。
+- 默认关闭的“技术详情”开关。
+- 选中调用的 `LlmCallDetail`。
+- 无记录、401、加载失败和 `incomplete` 状态。
+
+组件不得出现开始、重试、取消、删除或编辑按钮。
+
+- [ ] **Step 6:增加桌面和窄屏 CSS**
+
+新增类:`.architecture-observer`、`.architecture-task-strip`、`.architecture-flow`、`.architecture-node-*`、`.architecture-workspace`、`.llm-call-list`、`.llm-call-detail`、`.llm-detail-tabs`、`.llm-json-view`。桌面 `.architecture-workspace` 使用左侧调用轨迹、右侧详情;在现有 `@media (max-width: 900px)` 中改成单列。JSON 使用 `white-space: pre-wrap` 和 `overflow-wrap: anywhere`,不得产生页面级横向滚动。
+
+- [ ] **Step 7:运行纯逻辑测试和 lint**
+
+Run: `npm test -- src/components/architecture/__tests__/trace-state.test.ts && npm run lint`
+
+Expected: PASS;ESLint 不出现 effect 同步 set-state 或未转义内容错误。
+
+- [ ] **Step 8:提交观测组件**
+
+~~~bash
+git add src/components/architecture/architecture-flow.tsx src/components/architecture/llm-call-detail.tsx src/components/architecture/architecture-observer-panel.tsx src/components/architecture/__tests__/trace-state.test.ts src/app/globals.css
+git commit -m "新增LLM后台架构观测界面"
+~~~
+
+## Task 10:接入第三标签并完成浏览器验收
+
+**Files:**
+- Modify: `src/app/page.tsx`
+- Create: `tests/e2e/architecture-observer.spec.ts`
+- Modify: `tests/e2e/mvp.spec.ts`
+- Modify: `tests/e2e/sample-flow/page-flow.ts`
+
+- [ ] **Step 1:写第三标签和切换不中断的 E2E 失败测试**
+
+`architecture-observer.spec.ts` 使用 `page.route()` 模拟分块 NDJSON 和追踪正文接口:
+
+~~~ts
+test("后台架构标签跟随当前任务并按需展示真实请求响应", async ({ page }) => {
+  await mockOptimizationStream(page);
+  await mockTracePayloads(page);
+  await page.goto("/");
+  await page.getByLabel("访问密钥").fill("local-dev-key");
+  await page.getByLabel("文章内容").fill("示例文章正文");
+  await page.getByRole("button", { name: "开始优化" }).click();
+
+  await page.getByRole("button", { name: "后台架构" }).click();
+  await expect(page.getByText("当前任务")).toBeVisible();
+  await expect(page.getByText("article_optimizer")).toBeVisible();
+
+  await page.getByRole("button", { name: /article_optimizer/ }).click();
+  await page.getByRole("checkbox", { name: "技术详情" }).check();
+  await page.getByRole("tab", { name: "请求" }).click();
+  await expect(page.locator(".llm-json-view")).toContainText("messages");
+  await page.getByRole("tab", { name: "响应" }).click();
+  await expect(page.locator(".llm-json-view")).toContainText("choices");
+  await expect(page.getByText("终稿已生成")).toBeVisible();
+});
+~~~
+
+Mock 路由记录请求次数,并断言关闭技术详情时正文接口调用次数为 0。
+
+- [ ] **Step 2:运行 E2E 并确认第三标签不存在**
+
+Run: `npx playwright test tests/e2e/architecture-observer.spec.ts --project=chromium`
+
+Expected: FAIL,找不到 `后台架构` 按钮。
+
+- [ ] **Step 3:在首页接入追踪事件状态**
+
+`page.tsx` 修改为:
+
+~~~tsx
+const [activeTab, setActiveTab] = useState<"geo" | "copy" | "architecture">("geo");
+const [llmTraceEvents, setLlmTraceEvents] = useState([]);
+
+function handleStreamEvent(event: OptimizationStreamEvent) {
+  switch (event.type) {
+    case "llm_call_started":
+    case "llm_call_responded":
+    case "llm_call_validated":
+    case "llm_call_failed":
+    case "trace_warning":
+      setLlmTraceEvents((current) => [...current, event]);
+      return;
+    case "job_created":
+      setJobId(event.job.id);
+      setLlmTraceEvents([]);
+      setStreamActivity("任务已创建");
+      return;
+    case "fact_card_ready":
+      setFactCard(event.fact_card);
+      setStreamActivity("事实卡已就绪");
+      return;
+    case "draft_started":
+      setStreamActivity(event.message);
+      return;
+    case "draft_ready":
+      setDraftArticle(event.article);
+      setStreamActivity("草稿已生成");
+      return;
+    case "qa_started":
+      setStreamActivity(event.message);
+      return;
+    case "qa_ready":
+      setQaReport(event.qa_report);
+      setStreamActivity("质量检查完成");
+      return;
+    case "rewrite_started":
+      setStreamActivity(`正在修复第 ${event.round} 轮`);
+      return;
+    case "rewrite_ready":
+      setDraftArticle(event.article);
+      setStreamActivity(`第 ${event.round} 轮修复完成`);
+      return;
+    case "final_ready":
+      setOptimizedArticle(event.optimized_article);
+      setDraftArticle(null);
+      setQaReport(event.qa_report);
+      setStreamActivity("终稿已生成");
+      setMessage(
+        event.qa_report.overall_status === "fail"
+          ? "优化完成,质检发现需要复核的问题。"
+          : "优化完成。",
+      );
+      return;
+    case "failed":
+      setStreamActivity("优化失败");
+      throw new Error(event.error);
+  }
+}
+~~~
+
+第三个标签按钮文字固定为 `后台架构`。内容选择使用三个明确分支,不能把 architecture 落入普通文案分支:
+
+~~~tsx
+{activeTab === "geo" ? (
+  <>
+    
+    
+ + + + + +
+ +) : activeTab === "copy" ? ( + +) : ( + +)} +~~~ + +在导航中紧跟“普通文案优化”增加 `后台架构` 按钮,并导入 `ArchitectureObserverPanel` 与 `LlmTraceStreamEvent`。现有 GEO JSX 原位保留。 + +- [ ] **Step 4:证明切换标签不取消流读取** + +不要把 `readOptimizationStream()` 移进标签组件,也不要在切换标签时 abort fetch。E2E mock 在点击 `后台架构` 后继续发送 `qa_ready` 和 `final_ready`,测试必须看到最终状态。 + +- [ ] **Step 5:更新旧 MVP 回归为当前一键优化流程** + +把 `tests/e2e/mvp.spec.ts` 中已不存在的“标题”“正文”“分析文章”“采纳为核心事实”“确认事实卡”步骤删除,改为填写 `文章内容`、`图片描述或图片链接`、`用户要求`,然后直接点击一次 `开始优化`;继续断言 `optimized.md`、质量报告、`optimized.docx` 和 `qa_report.json`。这只更新测试以匹配当前产品,不把产品改回旧流程。 + +- [ ] **Step 6:扩展真实样例验证但不落盘正文** + +在 `tests/e2e/sample-flow/page-flow.ts` 增加: + +~~~ts +async function validateLlmTrace({ request, baseURL, apiAccessKey, jobId }: { + request: APIRequestContext; + baseURL: string; + apiAccessKey: string; + jobId: string; +}) { + const headers = { "x-api-key": apiAccessKey }; + const manifestResponse = await request.get( + `${baseURL}/api/jobs/${jobId}/llm-trace`, + { headers }, + ); + expect(manifestResponse.ok()).toBe(true); + const manifest = await manifestResponse.json() as LlmTraceManifest; + expect(manifest.calls.length).toBeGreaterThan(0); + + for (const call of manifest.calls) { + const requestResponse = await request.get( + `${baseURL}/api/jobs/${jobId}/llm-trace/${call.call_id}/request`, + { headers }, + ); + expect(requestResponse.ok()).toBe(true); + const requestBody = await requestResponse.json() as { messages?: unknown[] }; + expect(Array.isArray(requestBody.messages)).toBe(true); + + if (call.response_available) { + const responseResponse = await request.get( + `${baseURL}/api/jobs/${jobId}/llm-trace/${call.call_id}/response`, + { headers }, + ); + expect(responseResponse.ok()).toBe(true); + const responseBody = await responseResponse.json() as { choices?: unknown[] }; + expect(Array.isArray(responseBody.choices)).toBe(true); + } + } + + return manifest.calls.map((call) => call.task); +} +~~~ + +把返回值写入现有 `SampleResult.llm_tasks`。不得把 request/response 正文写入 `test-results`、截图注释或 summary JSON。 + +- [ ] **Step 7:运行确定性 E2E 和相关单元测试** + +Run: `npx playwright test tests/e2e/architecture-observer.spec.ts tests/e2e/mvp.spec.ts --project=chromium` + +Run: `npm test -- src/components/architecture src/lib/workflow/__tests__/stream-events.test.ts` + +Expected: 全部 PASS,无控制台错误和横向溢出。 + +- [ ] **Step 8:提交首页和端到端接入** + +~~~bash +git add src/app/page.tsx tests/e2e/architecture-observer.spec.ts tests/e2e/mvp.spec.ts tests/e2e/sample-flow/page-flow.ts +git commit -m "接入后台架构标签页" +~~~ + +## Task 11:迁移、全量验证和真实 Provider 验收 + +**Files:** +- Verify only: all files from Tasks 1-10 + +- [ ] **Step 1:应用本地 D1 迁移并核对表** + +Run: `npm run d1:migrate:local` + +Expected: `0004_llm_trace_observability.sql` applied successfully;再次运行显示没有待执行迁移。 + +- [ ] **Step 2:运行完整质量门禁** + +Run: `npm run lint` + +Expected: PASS,0 errors。 + +Run: `npm test` + +Expected: PASS,包含新增 trace repository、payload store、recorder、client、API 和 UI 状态测试。 + +Run: `npm run build` + +Expected: PASS,Next.js 生产构建成功。 + +- [ ] **Step 3:运行完整 Playwright 回归** + +Run: `npx playwright test --project=chromium` + +Expected: PASS;Task 10 已把 `mvp.spec.ts` 更新为当前一键优化流程。 + +- [ ] **Step 4:运行一次真实 DeepSeek 安全样例** + +确保 `.env.local` 只在本地包含 `API_ACCESS_KEY` 和 `DEEPSEEK_API_KEY`,然后运行: + +Run: `npm run test:e2e:samples -- --limit 1 --timeout-ms 600000` + +Expected: PASS,并在生成的 `test-results/geo-sample-flow//summary.md` 中看到非空 `llm_tasks`。报告不得包含完整请求、完整响应、访问密钥或 Provider 密钥。 + +- [ ] **Step 5:检查敏感信息和非预期文件** + +Run: + +~~~bash +git status --short --ignored=matching +rg -n "auth\.token|secretKey|healthsource|Authorization|DEEPSEEK_API_KEY" . --glob '!node_modules/**' --glob '!.next/**' --glob '!.open-next/**' --glob '!deploy/*.toml' --glob '!test-results/**' +~~~ + +Expected: 没有新提交的真实密钥;`data/`、`test-results/`、追踪正文和生成构建目录保持忽略。用户原有未提交文件仍保持原状。 + +- [ ] **Step 6:核对提交边界和最终状态** + +Run: `git status --short && git log --oneline -12` + +Expected: Tasks 1-10 的功能文件均已包含在对应中文提交中;只剩用户原有的 `README.md`、`wrangler.jsonc`、样例或导出目录等未提交改动,没有本功能遗漏文件。若质量门禁暴露功能缺陷,返回拥有该文件的任务按“失败测试—最小修正—验证—中文提交”闭环处理,不创建内容不明的兜底提交。 + +## 完成定义 + +- 所有任务按 TDD 顺序完成,每个任务都有独立中文提交。 +- D1/SQLite 索引和 R2/本地正文存储均有测试。 +- SDK 请求对象和保存请求深度相等,SDK 响应对象和保存响应深度相等。 +- 公开 NDJSON 和清单 API 不含完整正文。 +- 完整正文只能通过受保护且 `no-store` 的按需接口读取。 +- 架构图由真实事件驱动,QA 业务失败与技术失败分开表达。 +- 切换标签不取消任务,刷新可恢复已持久化状态。 +- 只保留运行中任务和最近终态任务的完整正文。 +- `npm run lint`、`npm test`、`npm run build`、Playwright 回归和一次真实安全样例全部通过。