czj/GEOAgentArticleOptimizer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
001f0402b2fda7f81ccbf3485d177c4abb5a3368
GEOAgentArticleOptimizer/docs/superpowers/plans/2026-07-16-llm-architecture-observability-tab.md
T
czj 001f0402b2 记录LLM架构观测实施计划
2026-07-16 11:54:35 +08:00

63 KiB
Raw Blame History

LLM 后台架构观测标签页实施计划

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 在首页增加只读的 后台架构 标签，以真实后端事件驱动固定架构图，并让授权用户查看当前或最近一次任务的完整 LLM 请求、完整响应和校验结果。

Architecture: src/lib/llm/client.ts 产生包含真实 SDK 对象的内部追踪事件，独立追踪收集器把索引写入 D1/SQLite、把完整正文写入现有私有 R2/本地数据目录，并向现有 NDJSON 流发送不含正文的状态事件。前端共享当前任务事件，刷新后通过受保护的只读 API 恢复追踪清单，并按需加载请求或响应正文。

Tech Stack: Next.js 16、React 19、TypeScript、Vitest、Playwright、Zod、OpenAI-compatible SDK、Cloudflare Workers、D1、R2、better-sqlite3。

实施前文件结构

新建文件

  • migrations/0004_llm_trace_observability.sql：生产 D1 追踪表和索引。
  • src/lib/llm/trace-types.ts：Provider、任务、运行、调用、内部事件、公开流事件和 API 类型。
  • src/lib/llm/trace-repository.ts：追踪索引仓储接口和运行时选择。
  • src/lib/llm/sqlite-trace-repository.ts：本地 SQLite 追踪索引实现。
  • src/lib/llm/d1-trace-repository.ts：Cloudflare D1 追踪索引实现。
  • src/lib/llm/trace-payload-store.ts：本地目录与 R2 的完整 JSON 正文存储。
  • src/lib/llm/trace-recorder.ts：内部 LLM 事件、工作流事件、持久化、公开事件和保留策略协调器。
  • src/lib/llm/trace-http.ts：受保护追踪 API 的无缓存 JSON 响应和正文读取助手。
  • src/lib/llm/__tests__/trace-repository.test.ts：SQLite 追踪仓储测试。
  • src/lib/llm/__tests__/d1-trace-repository.test.ts：D1 追踪仓储测试。
  • src/lib/llm/__tests__/trace-payload-store.test.ts：本地和 R2 正文存储测试。
  • src/lib/llm/__tests__/trace-recorder.test.ts：事件顺序、业务结果、失败隔离和保留测试。
  • src/app/api/llm-traces/latest/route.ts：当前或最近终态追踪入口。
  • src/app/api/jobs/[jobId]/llm-trace/route.ts：任务追踪清单入口。
  • src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts：完整请求入口。
  • src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts：完整响应入口。
  • src/app/api/__tests__/llm-traces.test.ts：追踪 API 鉴权、正文和缓存测试。
  • src/components/architecture/api-client.ts：浏览器端追踪 API 客户端。
  • src/components/architecture/trace-state.ts：实时事件归并和固定架构节点派生。
  • src/components/architecture/architecture-flow.tsx：固定拓扑架构图。
  • src/components/architecture/llm-call-detail.tsx：请求、响应、校验详情。
  • src/components/architecture/architecture-observer-panel.tsx：只读观测标签页容器。
  • src/components/architecture/__tests__/api-client.test.ts：按需正文请求测试。
  • src/components/architecture/__tests__/trace-state.test.ts：状态归并和业务/技术失败区分测试。
  • tests/e2e/architecture-observer.spec.ts：标签切换、调用选择和技术详情端到端测试。

修改文件

  • src/lib/db/schema.ts：本地 SQLite 创建追踪表。
  • src/lib/llm/audit.ts：从共享类型导入 Provider 和任务名。
  • src/lib/llm/client.ts：记录真实请求、响应、解析、校验和安全错误。
  • src/lib/llm/__tests__/client.test.ts：请求/响应深度相等和事件顺序测试。
  • src/lib/workflow/fact-extractor.ts：传递事实提取追踪上下文。
  • src/lib/workflow/article-optimizer.ts：传递生成草稿追踪上下文。
  • src/lib/workflow/quality-inspector.ts：传递 QA 和复检轮次追踪上下文。
  • src/lib/workflow/targeted-rewriter.ts：传递定向修复轮次追踪上下文。
  • src/lib/workflow/streaming-optimizer.ts：向追踪器转发工作流事件和轮次。
  • src/lib/workflow/stream-events.ts：公开 LLM 状态事件加入 NDJSON 联合类型。
  • src/lib/workflow/__tests__/stream-events.test.ts：新事件编码和分块解析测试。
  • src/app/api/jobs/optimize-stream/route.ts：创建、连接和结束追踪器。
  • src/app/api/__tests__/jobs.test.ts：真实调用状态事件和失败现场测试。
  • src/app/page.tsx：第三标签、实时追踪状态和只读组件接入。
  • src/app/globals.css：架构图、调用轨迹、详情和窄屏布局。
  • tests/e2e/mvp.spec.ts：把旧的“分析—确认—优化”流程断言更新为当前一键优化界面。
  • tests/e2e/sample-flow/page-flow.ts：真实 Provider 样例核对追踪接口，不保存客户正文。

明确不修改

  • wrangler.jsonc：复用现有私有 EXPORT_BUCKET，不增加绑定。
  • README.md：本次规格已经完整记录功能，不扩大文档范围。
  • 用户当前未提交的样例、导出目录和本地数据。

Task 1：定义追踪契约和公开流事件

Files:

  • Create: src/lib/llm/trace-types.ts

  • Modify: src/lib/llm/audit.ts

  • Modify: src/lib/llm/client.ts

  • Modify: src/lib/workflow/stream-events.ts

  • Test: src/lib/workflow/__tests__/stream-events.test.ts

  • Step 1：先写公开追踪事件的失败测试

在 src/lib/workflow/__tests__/stream-events.test.ts 增加：

it("encodes LLM call metadata without raw request or response bodies", () => {
  const event: OptimizationStreamEvent = {
    type: "llm_call_started",
    job_id: "job_123",
    call_id: "llmcall_1",
    sequence: 1,
    task: "fact_extractor",
    workflow_stage: "fact_card",
    rewrite_round: null,
    provider: "deepseek",
    model: "deepseek-v4-pro",
    started_at: "2026-07-16T00:00:00.000Z",
    request_available: true,
  };

  const encoded = encodeOptimizationStreamEvent(event);
  expect(JSON.parse(encoded)).toEqual(event);
  expect(encoded).not.toContain("messages");
  expect(encoded).not.toContain("Authorization");
});
  • Step 2：运行测试并确认类型或断言失败

Run: npm test -- src/lib/workflow/__tests__/stream-events.test.ts

Expected: FAIL，因为 OptimizationStreamEvent 尚不接受 llm_call_started。

  • Step 3：创建共享类型并接入现有类型引用

src/lib/llm/trace-types.ts 必须定义以下完整公共边界：

export type LlmProviderName = "deepseek" | "openai";

export type LlmTaskName =
  | "unknown"
  | "fact_extractor"
  | "article_optimizer"
  | "quality_inspector"
  | "targeted_rewriter"
  | "renwei_copy_optimizer";

export type LlmTraceWorkflowStage =
  | "unknown"
  | "input"
  | "fact_card"
  | "draft"
  | "qa"
  | "rewrite"
  | "final";

export type LlmTraceRunStatus =
  | "running"
  | "completed"
  | "failed"
  | "interrupted";

export type LlmTraceCallStatus =
  | "started"
  | "responded"
  | "validated"
  | "failed";

export type LlmTraceCompleteness = "complete" | "incomplete";
export type LlmBusinessStatus = "pass" | "warn" | "fail";
export type LlmTraceErrorType =
  | "provider"
  | "json_parse"
  | "schema_validation";

export interface LlmTraceContext {
  workflow_stage: LlmTraceWorkflowStage;
  rewrite_round?: number;
  schema_name?: string;
}

export interface LlmTraceRun {
  job_id: string;
  case_id: string | null;
  status: LlmTraceRunStatus;
  current_stage: LlmTraceWorkflowStage;
  trace_completeness: LlmTraceCompleteness;
  error_stage: string | null;
  error_summary: string | null;
  started_at: string;
  finished_at: string | null;
  updated_at: string;
}

export interface LlmTraceCall {
  call_id: string;
  job_id: string;
  sequence: number;
  task: LlmTaskName;
  workflow_stage: LlmTraceWorkflowStage;
  rewrite_round: number | null;
  provider: LlmProviderName;
  model: string;
  status: LlmTraceCallStatus;
  request_object_key: string | null;
  response_object_key: string | null;
  token_usage: Record<string, number> | null;
  schema_name: string | null;
  schema_valid: boolean | null;
  validation_issues: string[];
  business_status: LlmBusinessStatus | null;
  duration_ms: number | null;
  started_at: string;
  responded_at: string | null;
  validated_at: string | null;
  failed_at: string | null;
  error_type: LlmTraceErrorType | null;
  error_summary: string | null;
}

export type LlmTraceCallPublic = Omit<
  LlmTraceCall,
  "request_object_key" | "response_object_key"
> & {
  request_available: boolean;
  response_available: boolean;
};

export interface LlmTraceManifest {
  run: LlmTraceRun;
  calls: LlmTraceCallPublic[];
}

export type LlmClientTraceEvent =
  | {
      type: "started";
      call_id: string;
      task: LlmTaskName;
      context: LlmTraceContext;
      provider: LlmProviderName;
      model: string;
      request: unknown;
      started_at: string;
    }
  | {
      type: "responded";
      call_id: string;
      response: unknown;
      duration_ms: number;
      responded_at: string;
    }
  | {
      type: "validated";
      call_id: string;
      schema_name: string;
      schema_valid: boolean;
      validation_issues: string[];
      validated_at: string;
    }
  | {
      type: "failed";
      call_id: string;
      error_type: LlmTraceErrorType;
      error_summary: string;
      duration_ms: number;
      failed_at: string;
    };

export type LlmClientTraceHandler = (
  event: LlmClientTraceEvent,
) => void | Promise<void>;

export type LlmTraceStreamEvent =
  | {
      type: "llm_call_started";
      job_id: string;
      call_id: string;
      sequence: number;
      task: LlmTaskName;
      workflow_stage: LlmTraceWorkflowStage;
      rewrite_round: number | null;
      provider: LlmProviderName;
      model: string;
      started_at: string;
      request_available: boolean;
    }
  | {
      type: "llm_call_responded";
      job_id: string;
      call_id: string;
      duration_ms: number;
      token_usage: Record<string, number> | null;
      responded_at: string;
      response_available: boolean;
    }
  | {
      type: "llm_call_validated";
      job_id: string;
      call_id: string;
      schema_name: string;
      schema_valid: boolean;
      validation_issues: string[];
      validated_at: string;
    }
  | {
      type: "llm_call_failed";
      job_id: string;
      call_id: string;
      error_type: LlmTraceErrorType;
      error_summary: string;
      failed_at: string;
    }
  | {
      type: "trace_warning";
      job_id: string;
      trace_completeness: "incomplete";
      error_summary: string;
    };

把 LlmTaskName 从 client.ts 移到该文件；audit.ts 和 client.ts 都从该文件导入。stream-events.ts 使用：

import type { LlmTraceStreamEvent } from "../llm/trace-types";

type ExistingOptimizationStreamEvents =
  | {
      type: "job_created";
      job: { id: string };
      case?: { id: string; case_type: "article" };
    }
  | { type: "fact_card_ready"; job_id: string; fact_card: OptimizationFactCard }
  | { type: "draft_started"; job_id: string; message: string }
  | { type: "draft_ready"; job_id: string; article: OptimizedArticle }
  | { type: "qa_started"; job_id: string; message: string }
  | { type: "qa_ready"; job_id: string; qa_report: QaReport }
  | { type: "rewrite_started"; job_id: string; round: number }
  | {
      type: "rewrite_ready";
      job_id: string;
      round: number;
      article: OptimizedArticle;
    }
  | {
      type: "final_ready";
      job_id: string;
      case?: { id: string; case_type: "article" };
      result_version?: { id: string; version: number };
      optimized_article: OptimizedArticle;
      qa_report: QaReport;
      export_paths: Record<string, string>;
    }
  | {
      type: "failed";
      job_id?: string;
      case?: { id: string; case_type: "article" };
      result_version?: { id: string; version: number };
      stage: OptimizationStreamStage;
      error: string;
    };

export type OptimizationStreamEvent =
  | LlmTraceStreamEvent
  | ExistingOptimizationStreamEvents;

现有业务事件字段和值保持原样，只增加类型别名和新的联合分支。

  • Step 4：运行流事件测试并确认通过

Run: npm test -- src/lib/workflow/__tests__/stream-events.test.ts

Expected: PASS，且现有分块 NDJSON 测试继续通过。

  • Step 5：提交契约变更
git add src/lib/llm/trace-types.ts src/lib/llm/audit.ts src/lib/llm/client.ts src/lib/workflow/stream-events.ts src/lib/workflow/__tests__/stream-events.test.ts
git commit -m "定义LLM追踪事件契约"

Task 2：增加 D1 和 SQLite 追踪索引

Files:

  • Create: migrations/0004_llm_trace_observability.sql

  • Create: src/lib/llm/trace-repository.ts

  • Create: src/lib/llm/sqlite-trace-repository.ts

  • Create: src/lib/llm/d1-trace-repository.ts

  • Create: src/lib/llm/__tests__/trace-repository.test.ts

  • Create: src/lib/llm/__tests__/d1-trace-repository.test.ts

  • Modify: src/lib/db/schema.ts

  • Step 1：写 SQLite 仓储失败测试

测试必须先创建真实文章任务，再验证运行、调用、最近任务和删除：

it("stores a run and ordered calls, then deletes only trace rows", async () => {
  const appRepository = createSqliteRepository(dbPath);
  const job = await appRepository.createArticleJob({
    source_title: "Title",
    source_body: "Body",
    image_inputs: [],
    publish_platform: "official_site",
    user_instructions: "",
  });
  const repository = createSqliteTraceRepository(dbPath);

  await repository.putRun(runFixture(job.id));
  await repository.putCall(callFixture(job.id, "llmcall_1", 1));
  await repository.putCall(callFixture(job.id, "llmcall_2", 2));

  await expect(repository.getLatestRun()).resolves.toMatchObject({
    job_id: job.id,
    status: "running",
  });
  await expect(repository.listCalls(job.id)).resolves.toEqual([
    expect.objectContaining({ call_id: "llmcall_1", sequence: 1 }),
    expect.objectContaining({ call_id: "llmcall_2", sequence: 2 }),
  ]);

  await repository.deleteRun(job.id);
  await expect(appRepository.getArticleJob(job.id)).resolves.not.toBeNull();
  await expect(repository.getRun(job.id)).resolves.toBeNull();
});
  • Step 2：运行测试并确认表或模块不存在

Run: npm test -- src/lib/llm/__tests__/trace-repository.test.ts

Expected: FAIL，模块或 llm_trace_runs 表不存在。

  • Step 3：添加完整迁移和本地 Schema

migrations/0004_llm_trace_observability.sql：

CREATE TABLE IF NOT EXISTS llm_trace_runs (
  job_id TEXT PRIMARY KEY,
  case_id TEXT,
  status TEXT NOT NULL,
  current_stage TEXT NOT NULL,
  trace_completeness TEXT NOT NULL,
  error_stage TEXT,
  error_summary TEXT,
  started_at TEXT NOT NULL,
  finished_at TEXT,
  updated_at TEXT NOT NULL,
  FOREIGN KEY (job_id) REFERENCES article_jobs(id) ON DELETE CASCADE,
  FOREIGN KEY (case_id) REFERENCES optimization_cases(id) ON DELETE SET NULL
);

CREATE TABLE IF NOT EXISTS llm_trace_calls (
  call_id TEXT PRIMARY KEY,
  job_id TEXT NOT NULL,
  sequence INTEGER NOT NULL,
  task TEXT NOT NULL,
  workflow_stage TEXT NOT NULL,
  rewrite_round INTEGER,
  provider TEXT NOT NULL,
  model TEXT NOT NULL,
  status TEXT NOT NULL,
  request_object_key TEXT,
  response_object_key TEXT,
  token_usage TEXT,
  schema_name TEXT,
  schema_valid INTEGER,
  validation_issues TEXT NOT NULL DEFAULT '[]',
  business_status TEXT,
  duration_ms INTEGER,
  started_at TEXT NOT NULL,
  responded_at TEXT,
  validated_at TEXT,
  failed_at TEXT,
  error_type TEXT,
  error_summary TEXT,
  UNIQUE (job_id, sequence),
  FOREIGN KEY (job_id) REFERENCES llm_trace_runs(job_id) ON DELETE CASCADE
);

CREATE INDEX IF NOT EXISTS idx_llm_trace_runs_status_updated
  ON llm_trace_runs(status, updated_at DESC);

CREATE INDEX IF NOT EXISTS idx_llm_trace_calls_job_sequence
  ON llm_trace_calls(job_id, sequence);

把同一表结构加入 initializeSchema()，保持本地 SQLite 与 D1 一致。

  • Step 4：定义仓储接口和运行时选择

trace-repository.ts：

export interface LlmTraceRepository {
  putRun(run: LlmTraceRun): Promise<void>;
  putCall(call: LlmTraceCall): Promise<void>;
  getRun(jobId: string): Promise<LlmTraceRun | null>;
  getLatestRun(): Promise<LlmTraceRun | null>;
  listCalls(jobId: string): Promise<LlmTraceCall[]>;
  listTerminalRunsExcept(jobId: string): Promise<LlmTraceRun[]>;
  deleteRun(jobId: string): Promise<void>;
}

export function getLlmTraceRepositoryFromRuntime(): LlmTraceRepository {
  if (process.env.APP_RUNTIME === "cloudflare") {
    const env = getAppCloudflareEnv();
    if (!env?.DB) throw new Error("Cloudflare D1 binding DB is required");
    return createD1TraceRepository(env.DB);
  }
  return createSqliteTraceRepository(getDefaultDatabasePath());
}

使用 Task 1 已定义的 LlmTraceRun 和 LlmTraceCall；其字段与迁移一一对应。

  • Step 5：实现 SQLite 和 D1 UPSERT

两个实现都使用完整快照 UPSERT，避免动态拼 SQL。SQLite 的核心语句为：

INSERT INTO llm_trace_calls (
  call_id, job_id, sequence, task, workflow_stage, rewrite_round,
  provider, model, status, request_object_key, response_object_key,
  token_usage, schema_name, schema_valid, validation_issues,
  business_status, duration_ms, started_at, responded_at, validated_at,
  failed_at, error_type, error_summary
) VALUES (
  @call_id, @job_id, @sequence, @task, @workflow_stage, @rewrite_round,
  @provider, @model, @status, @request_object_key, @response_object_key,
  @token_usage, @schema_name, @schema_valid, @validation_issues,
  @business_status, @duration_ms, @started_at, @responded_at, @validated_at,
  @failed_at, @error_type, @error_summary
)
ON CONFLICT(call_id) DO UPDATE SET
  status = excluded.status,
  response_object_key = excluded.response_object_key,
  token_usage = excluded.token_usage,
  schema_name = excluded.schema_name,
  schema_valid = excluded.schema_valid,
  validation_issues = excluded.validation_issues,
  business_status = excluded.business_status,
  duration_ms = excluded.duration_ms,
  responded_at = excluded.responded_at,
  validated_at = excluded.validated_at,
  failed_at = excluded.failed_at,
  error_type = excluded.error_type,
  error_summary = excluded.error_summary;

D1 使用相同列顺序的 prepare().bind().run()；读取时把 JSON 文本和 0 | 1 | null 转回 TypeScript 类型。getLatestRun() 的顺序固定为运行中优先，再按 updated_at DESC：

SELECT * FROM llm_trace_runs
ORDER BY CASE WHEN status = 'running' THEN 0 ELSE 1 END, updated_at DESC
LIMIT 1;
  • Step 6：增加 D1 绑定顺序测试并运行两套测试

D1 测试至少断言 putRun() 使用 insert into llm_trace_runs，putCall() 将 validation_issues 和 token_usage 序列化，listCalls() 按 sequence 读取。

Run: npm test -- src/lib/llm/__tests__/trace-repository.test.ts src/lib/llm/__tests__/d1-trace-repository.test.ts src/lib/db/__tests__/repository.test.ts

Expected: PASS。

  • Step 7：提交索引存储
git add migrations/0004_llm_trace_observability.sql src/lib/db/schema.ts src/lib/llm/trace-repository.ts src/lib/llm/sqlite-trace-repository.ts src/lib/llm/d1-trace-repository.ts src/lib/llm/__tests__/trace-repository.test.ts src/lib/llm/__tests__/d1-trace-repository.test.ts
git commit -m "新增LLM追踪索引存储"

Task 3：实现私有完整正文存储

Files:

  • Create: src/lib/llm/trace-payload-store.ts

  • Create: src/lib/llm/__tests__/trace-payload-store.test.ts

  • Step 1：先写本地与 R2 失败测试

it("round-trips exact JSON locally and deletes one job prefix", async () => {
  const store = createLocalTracePayloadStore(tempDir);
  const payload = { model: "deepseek-v4-pro", messages: [{ role: "user", content: "原文" }] };
  const key = "llm-traces/job_1/llmcall_1/request.json";

  await store.putJson(key, payload);
  await expect(store.getJson(key)).resolves.toEqual(payload);
  await store.deleteJob("job_1");
  await expect(store.getJson(key)).resolves.toBeNull();
});

it("stores private JSON in R2 without a public URL", async () => {
  const put = vi.fn().mockResolvedValue(undefined);
  const get = vi.fn().mockResolvedValue({ json: async () => ({ ok: true }) });
  const bucket = { put, get } as unknown as R2Bucket;
  const store = createR2TracePayloadStore(bucket);

  await store.putJson("llm-traces/job_1/llmcall_1/request.json", { ok: true });
  await expect(store.getJson("llm-traces/job_1/llmcall_1/request.json"))
    .resolves.toEqual({ ok: true });
  expect(put).toHaveBeenCalledWith(
    "llm-traces/job_1/llmcall_1/request.json",
    JSON.stringify({ ok: true }),
    { httpMetadata: { contentType: "application/json; charset=utf-8" } },
  );
});
  • Step 2：运行测试并确认模块不存在

Run: npm test -- src/lib/llm/__tests__/trace-payload-store.test.ts

Expected: FAIL，正文存储工厂尚不存在。

  • Step 3：实现正文存储接口和工厂
export interface LlmTracePayloadStore {
  putJson(key: string, value: unknown): Promise<void>;
  getJson(key: string): Promise<unknown | null>;
  deleteJob(jobId: string): Promise<void>;
}

export function tracePayloadKey(
  jobId: string,
  callId: string,
  kind: "request" | "response",
) {
  return `llm-traces/${jobId}/${callId}/${kind}.json`;
}

export function getLlmTracePayloadStoreFromRuntime(): LlmTracePayloadStore {
  if (process.env.APP_RUNTIME === "cloudflare") {
    const bucket = getAppCloudflareEnv()?.EXPORT_BUCKET;
    if (!bucket) throw new Error("Cloudflare R2 binding EXPORT_BUCKET is required");
    return createR2TracePayloadStore(bucket);
  }
  return createLocalTracePayloadStore(getAppDataDir());
}

本地实现只能在 join(dataDir, "llm-traces") 下使用生成的键，使用 mkdirSync、writeFileSync、readFileSync 和 rmSync。R2 deleteJob() 必须循环 bucket.list({ prefix, cursor })，收集对象键后调用 bucket.delete(keys)，直到 truncated 为 false。

  • Step 4：运行正文存储和现有导出存储测试

Run: npm test -- src/lib/llm/__tests__/trace-payload-store.test.ts src/lib/workflow/__tests__/export-store.test.ts

Expected: PASS；现有 exports/ 行为不变。

  • Step 5：提交正文存储
git add src/lib/llm/trace-payload-store.ts src/lib/llm/__tests__/trace-payload-store.test.ts
git commit -m "新增LLM追踪正文存储"

Task 4：实现追踪收集器、业务状态和保留策略

Files:

  • Create: src/lib/llm/trace-recorder.ts

  • Create: src/lib/llm/__tests__/trace-recorder.test.ts

  • Step 1：写成功调用、QA 业务失败和保留策略测试

使用内存 fake 仓储和正文存储，覆盖完整顺序：

it("persists exact bodies before publishing public metadata", async () => {
  const published: LlmTraceStreamEvent[] = [];
  const recorder = await createLlmTraceRecorder({
    jobId: "job_1",
    caseId: "case_1",
    repository,
    payloadStore,
    publish: (event) => published.push(event),
  });

  await recorder.onLlmEvent({
    type: "started",
    call_id: "llmcall_1",
    task: "quality_inspector",
    context: { workflow_stage: "qa", schema_name: "llmQaPatchSchema" },
    provider: "deepseek",
    model: "deepseek-v4-pro",
    request: { model: "deepseek-v4-pro", messages: [] },
    started_at: "2026-07-16T00:00:00.000Z",
  });

  expect(payloadStore.values.get(
    "llm-traces/job_1/llmcall_1/request.json",
  )).toEqual({ model: "deepseek-v4-pro", messages: [] });
  expect(published[0]).toMatchObject({
    type: "llm_call_started",
    request_available: true,
  });
  expect(JSON.stringify(published)).not.toContain("messages");
});

it("stores QA business failure separately from schema success", async () => {
  await recorder.onWorkflowEvent({
    type: "qa_ready",
    job_id: "job_1",
    qa_report: { overall_status: "fail", checks: [] },
  });
  await expect(repository.listCalls("job_1")).resolves.toEqual([
    expect.objectContaining({
      task: "quality_inspector",
      schema_valid: true,
      business_status: "fail",
    }),
  ]);
});

it("keeps running runs and only the newest terminal full trace", async () => {
  await recorder.finish({ status: "completed" });
  expect(payloadStore.deletedJobs).toEqual(["job_old_completed"]);
  expect(repository.deletedRuns).toEqual(["job_old_completed"]);
  expect(repository.deletedRuns).not.toContain("job_other_running");
});
  • Step 2：运行测试并确认收集器不存在

Run: npm test -- src/lib/llm/__tests__/trace-recorder.test.ts

Expected: FAIL，createLlmTraceRecorder 尚不存在。

  • Step 3：实现收集器的公开接口
export interface LlmTraceRecorder {
  onLlmEvent: LlmClientTraceHandler;
  onWorkflowEvent(event: OptimizationStreamEvent): Promise<void>;
  finish(input: {
    status: "completed" | "failed" | "interrupted";
    errorStage?: string;
    errorSummary?: string;
  }): Promise<void>;
}

export function createNoopLlmTraceRecorder(): LlmTraceRecorder {
  return {
    onLlmEvent: async () => undefined,
    onWorkflowEvent: async () => undefined,
    finish: async () => undefined,
  };
}

export async function createLlmTraceRecorder({
  jobId,
  caseId,
  repository,
  payloadStore,
  publish,
}: CreateLlmTraceRecorderInput): Promise<LlmTraceRecorder> {
  const calls = new Map<string, LlmTraceCall>();
  let sequence = 0;
  let run = createRunningRun(jobId, caseId);
  await repository.putRun(run);

  const warn = async (error: unknown) => {
    run = { ...run, trace_completeness: "incomplete", updated_at: nowIso() };
    await repository.putRun(run).catch(() => undefined);
    await publish({
      type: "trace_warning",
      job_id: jobId,
      trace_completeness: "incomplete",
      error_summary: safeTraceError(error),
    });
  };

  return {
    onLlmEvent: async (event) => {
      try {
        await applyLlmEvent(event);
      } catch (error) {
        await warn(error);
      }
    },
    onWorkflowEvent: async (event) => {
      try {
        await applyWorkflowEvent(event);
      } catch (error) {
        await warn(error);
      }
    },
    finish: async (input) => {
      try {
        await finishRun(input);
      } catch (error) {
        await warn(error);
      }
    },
  };
}

applyLlmEvent() 的硬性顺序：

  1. started：先写 request.json，再 putCall()，最后发布 llm_call_started。
  2. responded：先写 response.json，再更新调用，最后发布 llm_call_responded。
  3. validated：更新 Schema 字段，再发布 llm_call_validated。
  4. failed：更新错误白名单字段，再发布 llm_call_failed。

若正文写入失败，onLlmEvent() 仍尝试写入不带对象键的调用索引并发布 request_available: false 或 response_available: false，随后发布 trace_warning；若索引也不可用，则至少发布 trace_warning。任何追踪失败都被收集器吞掉，不向 LLM 客户端或业务工作流抛出。

Token 用量只从完整响应的顶层 usage 读取为数字字典：

function tokenUsageFromResponse(response: unknown) {
  if (!response || typeof response !== "object") return null;
  const usage = (response as { usage?: unknown }).usage;
  if (!usage || typeof usage !== "object") return null;
  return Object.fromEntries(
    Object.entries(usage).filter(
      (entry): entry is [string, number] => typeof entry[1] === "number",
    ),
  );
}
  • Step 4：实现工作流状态和清理

onWorkflowEvent() 必须更新 current_stage；收到 qa_ready 时，把最近一个 quality_inspector 调用的 business_status 设置为 qa_report.overall_status。finish() 先写终态，再执行：

const expired = await repository.listTerminalRunsExcept(jobId);
for (const oldRun of expired) {
  await payloadStore.deleteJob(oldRun.job_id);
  await repository.deleteRun(oldRun.job_id);
}

listTerminalRunsExcept() 不返回任何 running 记录，因此并发运行任务不会被删除。

  • Step 5：运行收集器测试

Run: npm test -- src/lib/llm/__tests__/trace-recorder.test.ts

Expected: PASS，包含存储失败时业务回调不抛错、trace_warning 被发布的断言。

  • Step 6：提交追踪协调器
git add src/lib/llm/trace-recorder.ts src/lib/llm/__tests__/trace-recorder.test.ts
git commit -m "新增LLM任务追踪收集器"

Task 5：在 LLM 客户端捕获真实 SDK 请求和响应

Files:

  • Modify: src/lib/llm/client.ts

  • Modify: src/lib/llm/__tests__/client.test.ts

  • Step 1：写请求和响应深度相等的失败测试

it("traces the exact SDK request and full SDK response", async () => {
  process.env.LLM_PROVIDER = "deepseek";
  process.env.DEEPSEEK_API_KEY = "test-key";
  const traced: LlmClientTraceEvent[] = [];
  let sdkRequest: unknown;
  const sdkResponse = {
    id: "chatcmpl_1",
    object: "chat.completion",
    created: 1784188800,
    model: "deepseek-v4-pro",
    choices: [{
      index: 0,
      message: { role: "assistant", content: "{\"value\":\"ok\"}" },
      finish_reason: "stop",
    }],
    usage: { prompt_tokens: 10, completion_tokens: 4, total_tokens: 14 },
  };
  client.setChatCompletionForTesting(async (request) => {
    sdkRequest = request;
    return sdkResponse;
  });

  await client.generateValidatedJson({
    schema: z.object({ value: z.string() }),
    schemaName: "valueSchema",
    prompt: "Return JSON.",
    task: "article_optimizer",
    traceStage: "draft",
    onTraceEvent: (event) => traced.push(event),
  });

  expect(traced.find((event) => event.type === "started")).toMatchObject({
    type: "started",
    request: sdkRequest,
  });
  expect(traced.find((event) => event.type === "responded")).toMatchObject({
    type: "responded",
    response: sdkResponse,
  });
  expect(traced.map((event) => event.type)).toEqual([
    "started",
    "responded",
    "validated",
  ]);
});

再增加 JSON 解析失败顺序和 Schema 失败顺序测试：

expect(jsonParseEvents.map((event) => event.type)).toEqual([
  "started", "responded", "failed",
]);
expect(schemaEvents.map((event) => event.type)).toEqual([
  "started", "responded", "validated", "failed",
]);
  • Step 2：运行客户端测试并确认失败

Run: npm test -- src/lib/llm/__tests__/client.test.ts

Expected: FAIL，schemaName、traceStage 和 onTraceEvent 尚不存在，响应也会在解析后丢失。

  • Step 3：扩展 GenerateInput 并生成稳定调用 ID
export interface GenerateInput {
  system?: string;
  prompt: string;
  model?: string;
  temperature?: number;
  task?: LlmTaskName;
  schemaName?: string;
  traceStage?: LlmTraceWorkflowStage;
  rewriteRound?: number;
  onTraceEvent?: LlmClientTraceHandler;
  onAuditSummary?: (summary: LlmAuditSummary) => void | Promise<void>;
  traceCallId?: string;
}

function callIdFor(input: GenerateInput) {
  return input.traceCallId ?? `llmcall_${nanoid(12)}`;
}

async function emitTrace(input: GenerateInput, event: LlmClientTraceEvent) {
  try {
    await input.onTraceEvent?.(event);
  } catch (error) {
    console.warn(`[llm:trace-warning] ${safeErrorSummary(error)}`);
  }
}

generateValidatedJson() 在调用生成器前创建 traceCallId，并把同一个 ID 传给 generateJsonForValidation()，保证校验事件属于同一调用。

  • Step 4：拆开 Provider、JSON 解析和 Schema 校验阶段

generateJson() 必须按以下顺序实现，不能把 JSON.parse() 放回 Provider try/catch：

const request = buildChatCompletionRequest(input, effectiveModel);
await emitTrace(input, {
  type: "started",
  call_id: callId,
  task,
  context: {
    workflow_stage: input.traceStage ?? "unknown",
    rewrite_round: input.rewriteRound,
    schema_name: input.schemaName,
  },
  provider: status.provider,
  model: effectiveModel,
  request,
  started_at: new Date(startedAt).toISOString(),
});

let response: ChatCompletionResult;
try {
  response = chatCompletionForTesting
    ? await chatCompletionForTesting(request)
    : await createClient().client.chat.completions.create(request);
} catch (error) {
  await emitTrace(input, failedEvent(callId, "provider", error, startedAt));
  throw normalizeLlmError(error);
}

await emitTrace(input, {
  type: "responded",
  call_id: callId,
  response,
  duration_ms: Date.now() - startedAt,
  responded_at: new Date().toISOString(),
});

const content = response.choices[0]?.message.content ?? "{}";
try {
  return JSON.parse(content) as T;
} catch (error) {
  await emitTrace(input, failedEvent(callId, "json_parse", error, startedAt));
  throw new Error(`LLM response is not valid JSON: ${safeErrorSummary(error)}`);
}

Schema 校验失败时先发送 validated(schema_valid=false)，再发送 failed(error_type=schema_validation)。安全错误只允许 name 和 message，不得序列化 SDK error、headers、request 或环境变量。

  • Step 5：保持现有审计和控制台日志兼容

现有 [llm:start]、[llm:response]、[llm:validated]、[llm:error] 和 onAuditSummary 行为必须保留。现有原始返回控制台日志仍按 LLM_LOG_RAW_LIMIT 截断；R2 追踪正文不截断。

  • Step 6：运行客户端与审计测试

Run: npm test -- src/lib/llm/__tests__/client.test.ts src/lib/llm/__tests__/audit.test.ts

Expected: PASS，且失败测试证明 API Key 不出现在任何事件的 JSON 中。

  • Step 7：提交客户端追踪
git add src/lib/llm/client.ts src/lib/llm/__tests__/client.test.ts
git commit -m "记录真实LLM请求与响应"

Task 6：把追踪器接入真实流式文章工作流

Files:

  • Modify: src/lib/workflow/fact-extractor.ts

  • Modify: src/lib/workflow/article-optimizer.ts

  • Modify: src/lib/workflow/quality-inspector.ts

  • Modify: src/lib/workflow/targeted-rewriter.ts

  • Modify: src/lib/workflow/streaming-optimizer.ts

  • Modify: src/app/api/jobs/optimize-stream/route.ts

  • Modify: src/app/api/__tests__/jobs.test.ts

  • Step 1：写流式事件顺序的失败测试

在 jobs.test.ts 中让 LLM mock 主动调用传入的追踪回调，并断言：

expect(events.map((event) => event.type)).toEqual([
  "job_created",
  "llm_call_started",
  "llm_call_responded",
  "llm_call_validated",
  "fact_card_ready",
  "draft_started",
  "llm_call_started",
  "llm_call_responded",
  "llm_call_validated",
  "draft_ready",
  "qa_started",
  "llm_call_started",
  "llm_call_responded",
  "llm_call_validated",
  "qa_ready",
  "final_ready",
]);

还要断言公开事件 JSON 不包含文章正文、messages、完整 choices 或访问密钥。

  • Step 2：运行 API 测试并确认缺少追踪事件

Run: npm test -- src/app/api/__tests__/jobs.test.ts

Expected: FAIL，事件序列仍只有业务阶段事件。

  • Step 3：给四个工作流节点传递明确追踪上下文

每个输入接口增加：

onTraceEvent?: GenerateInput["onTraceEvent"];
rewriteRound?: number;

四个 generateValidatedJson() 调用必须分别传入：

// fact-extractor.ts
schemaName: "candidateFactCardSchema",
traceStage: "fact_card",
onTraceEvent: options.onTraceEvent,

// article-optimizer.ts
schemaName: "optimizedArticleSchema",
traceStage: "draft",
onTraceEvent,

// quality-inspector.ts
schemaName: "llmQaPatchSchema",
traceStage: "qa",
rewriteRound: input.rewriteRound,
onTraceEvent: input.onTraceEvent,

// targeted-rewriter.ts
schemaName: "optimizedArticleSchema",
traceStage: "rewrite",
rewriteRound,
onTraceEvent,

streaming-optimizer.ts 初次 QA 使用 rewriteRound: 0；每轮修复和复检使用当前 nextRound。

  • Step 4：在流式路由创建并连接追踪器

在 ReadableStream.start() 的业务 try 之前，与 jobId、caseId、stage 同级声明空追踪器，使失败 catch 始终能访问：

let jobId: string | undefined;
let caseId: string | undefined;
let stage: OptimizationStreamStage = "job";
let traceRecorder = createNoopLlmTraceRecorder();

任务和案例创建完成后，再尝试替换为真实追踪器：

try {
  traceRecorder = await createLlmTraceRecorder({
    jobId: job.id,
    caseId: optimizationCase.id,
    repository: getLlmTraceRepositoryFromRuntime(),
    payloadStore: getLlmTracePayloadStoreFromRuntime(),
    publish: async (event) => send(event),
  });
} catch (error) {
  send({
    type: "trace_warning",
    job_id: job.id,
    trace_completeness: "incomplete",
    error_summary: safeTraceError(error),
  });
}

事实提取和 runStreamingOptimizationWorkflow() 都传 onTraceEvent: traceRecorder.onLlmEvent。保存事实卡后，把直接发送改为同一事件先入追踪器、再入 NDJSON：

const factCardReadyEvent: OptimizationStreamEvent = {
  type: "fact_card_ready",
  job_id: job.id,
  fact_card: savedFactCard,
};
await traceRecorder.onWorkflowEvent(factCardReadyEvent);
send(factCardReadyEvent);

工作流的 onEvent 改为：

onEvent: async (event) => {
  stage = stageForEvent(event, stage);
  await traceRecorder.onWorkflowEvent(event);
  send(event);
},

成功路径在 final_ready 前调用 traceRecorder.finish({ status: "completed" })；失败路径在 failed 前调用：

await traceRecorder.finish({
  status: "failed",
  errorStage: stage,
  errorSummary: message,
});

safeTraceError() 从 trace-recorder.ts 导出，只返回安全化的 name: message。追踪器创建或存储异常必须被隔离成 trace_warning，不得跳入业务失败分支。

  • Step 5：运行路由、工作流和流事件测试

Run: npm test -- src/app/api/__tests__/jobs.test.ts src/lib/workflow/__tests__/streaming-optimizer.test.ts src/lib/workflow/__tests__/stream-events.test.ts

Expected: PASS，现有业务事件和结果版本断言保持不变。

  • Step 6：提交真实工作流接入
git add src/lib/workflow/fact-extractor.ts src/lib/workflow/article-optimizer.ts src/lib/workflow/quality-inspector.ts src/lib/workflow/targeted-rewriter.ts src/lib/workflow/streaming-optimizer.ts src/app/api/jobs/optimize-stream/route.ts src/app/api/__tests__/jobs.test.ts
git commit -m "接入文章优化LLM追踪"

Task 7：增加受保护的追踪读取 API

Files:

  • Create: src/lib/llm/trace-http.ts

  • Create: src/app/api/llm-traces/latest/route.ts

  • Create: src/app/api/jobs/[jobId]/llm-trace/route.ts

  • Create: src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts

  • Create: src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts

  • Create: src/app/api/__tests__/llm-traces.test.ts

  • Step 1：先写鉴权、无缓存和正文测试

it("rejects trace reads without API access", async () => {
  const response = await getLatestTrace(request(null));
  expect(response.status).toBe(401);
});

it("returns a no-store manifest without raw bodies", async () => {
  const response = await getJobTrace(
    request("test-key"),
    params({ jobId: "job_1" }),
  );
  const body = await response.json();
  expect(response.headers.get("cache-control")).toBe("no-store");
  expect(body.calls[0]).toMatchObject({
    call_id: "llmcall_1",
    request_available: true,
    response_available: true,
  });
  expect(JSON.stringify(body)).not.toContain("messages");
  expect(JSON.stringify(body)).not.toContain("choices");
});

it("returns the exact stored request through the protected payload route", async () => {
  const response = await getTraceRequest(
    request("test-key"),
    params({ jobId: "job_1", callId: "llmcall_1" }),
  );
  expect(response.headers.get("cache-control")).toBe("no-store");
  await expect(response.json()).resolves.toEqual(exactRequestFixture);
});
  • Step 2：运行 API 测试并确认路由不存在

Run: npm test -- src/app/api/__tests__/llm-traces.test.ts

Expected: FAIL，读取路由尚不存在。

  • Step 3：实现共享 HTTP 助手
export function noStoreJson(body: unknown, status = 200) {
  return NextResponse.json(body, {
    status,
    headers: { "cache-control": "no-store" },
  });
}

export async function readTracePayload({
  jobId,
  callId,
  kind,
}: {
  jobId: string;
  callId: string;
  kind: "request" | "response";
}) {
  const repository = getLlmTraceRepositoryFromRuntime();
  const call = (await repository.listCalls(jobId)).find(
    (candidate) => candidate.call_id === callId,
  );
  if (!call) return noStoreJson({ error: "追踪调用不存在" }, 404);
  const key = kind === "request"
    ? call.request_object_key
    : call.response_object_key;
  if (!key) {
    if (kind === "response" && call.status === "started") {
      return noStoreJson({ state: "waiting" }, 202);
    }
    return noStoreJson({
      error: kind === "response" ? "该调用未产生响应" : "追踪请求正文不存在",
    }, 404);
  }
  const payload = await getLlmTracePayloadStoreFromRuntime().getJson(key);
  return payload == null
    ? noStoreJson({ error: "追踪正文不存在" }, 404)
    : noStoreJson(payload);
}
  • Step 4：实现四个 GET 路由

每个路由第一段必须完全一致：

const access = requireApiAccess(request);
if (!access.ok) return access.response;

latest 调用 repository.getLatestRun()；无记录时返回 noStoreJson({ run: null, calls: [] })。有记录时，latest 和任务清单路由都调用 repository.listCalls(run.job_id)，并通过同一个安全映射函数返回：

export function toPublicTraceCall(call: LlmTraceCall): LlmTraceCallPublic {
  const {
    request_object_key: requestObjectKey,
    response_object_key: responseObjectKey,
    ...publicFields
  } = call;
  return {
    ...publicFields,
    request_available: Boolean(requestObjectKey),
    response_available: Boolean(responseObjectKey),
  };
}

return noStoreJson({ run, calls: calls.map(toPublicTraceCall) });

toPublicTraceCall() 放在 trace-http.ts，保证内部 R2 对象键不会进入响应对象。

  • Step 5：运行追踪 API 和鉴权回归测试

Run: npm test -- src/app/api/__tests__/llm-traces.test.ts src/lib/api/__tests__/auth.test.ts

Expected: PASS，401、404、202、200 和 no-store 均有断言。

  • Step 6：提交追踪读取 API
git add src/lib/llm/trace-http.ts src/app/api/llm-traces/latest/route.ts 'src/app/api/jobs/[jobId]/llm-trace/route.ts' 'src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts' 'src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts' src/app/api/__tests__/llm-traces.test.ts
git commit -m "新增LLM追踪只读接口"

Task 8：实现浏览器 API 客户端和架构状态派生

Files:

  • Create: src/components/architecture/api-client.ts

  • Create: src/components/architecture/trace-state.ts

  • Create: src/components/architecture/__tests__/api-client.test.ts

  • Create: src/components/architecture/__tests__/trace-state.test.ts

  • Step 1：先写按需加载和状态派生失败测试

it("loads request bodies only when explicitly requested", async () => {
  vi.stubGlobal("fetch", vi.fn(async () => jsonResponse(exactRequest)));
  await expect(
    getTracePayload("job_1", "llmcall_1", "request", "test-key"),
  ).resolves.toEqual(exactRequest);
  expect(fetch).toHaveBeenCalledWith(
    "/api/jobs/job_1/llm-trace/llmcall_1/request",
    expect.objectContaining({
      credentials: "same-origin",
      headers: { "x-api-key": "test-key" },
      cache: "no-store",
    }),
  );
});

it("distinguishes QA business failure from schema failure", () => {
  const nodes = deriveArchitectureNodes(run, [
    call({ task: "quality_inspector", schema_valid: true, business_status: "fail" }),
  ]);
  expect(nodes.qa).toMatchObject({
    status: "completed",
    detail: "检查完成，需要修复",
  });

  const failedNodes = deriveArchitectureNodes(run, [
    call({ task: "quality_inspector", schema_valid: false, status: "failed" }),
  ]);
  expect(failedNodes.qa.status).toBe("failed");
});
  • Step 2：运行测试并确认模块不存在

Run: npm test -- src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts

Expected: FAIL。

  • Step 3：实现 API 客户端
function traceHeaders(apiAccessKey: string) {
  return apiAccessKey ? { "x-api-key": apiAccessKey } : {};
}

async function traceFetch<T>(path: string, apiAccessKey: string): Promise<T> {
  const response = await fetch(path, {
    credentials: "same-origin",
    headers: traceHeaders(apiAccessKey),
    cache: "no-store",
  });
  const body = (await response.json().catch(() => ({}))) as T & { error?: string };
  if (!response.ok) throw new Error(body.error ?? "读取后台追踪失败");
  return body;
}

export function getLatestTrace(apiAccessKey: string) {
  return traceFetch<LlmTraceManifest | { run: null; calls: [] }>(
    "/api/llm-traces/latest",
    apiAccessKey,
  );
}

export function getJobTrace(jobId: string, apiAccessKey: string) {
  return traceFetch<LlmTraceManifest>(
    `/api/jobs/${encodeURIComponent(jobId)}/llm-trace`,
    apiAccessKey,
  );
}

export function getTracePayload(
  jobId: string,
  callId: string,
  kind: "request" | "response",
  apiAccessKey: string,
) {
  return traceFetch<unknown>(
    `/api/jobs/${encodeURIComponent(jobId)}/llm-trace/${encodeURIComponent(callId)}/${kind}`,
    apiAccessKey,
  );
}
  • Step 4：实现实时事件归并和固定节点派生

trace-state.ts 导出：

export type ArchitectureNodeStatus =
  | "waiting"
  | "running"
  | "completed"
  | "failed";

export interface ArchitectureNodeView {
  id: "input" | "fact_card" | "draft" | "qa" | "rewrite" | "final";
  label: string;
  status: ArchitectureNodeStatus;
  detail: string;
}

export function applyLiveTraceEvent(
  state: LlmTraceManifest,
  event: LlmTraceStreamEvent,
): LlmTraceManifest;

export function deriveArchitectureNodes(
  run: LlmTraceRun | null,
  calls: LlmTraceCallPublic[],
): Record<ArchitectureNodeView["id"], ArchitectureNodeView>;

归并必须按 call_id 更新而不是追加重复调用；调用清单始终按 sequence 排序。trace_warning 只把运行记录改为 incomplete，不把业务状态改成 failed。

  • Step 5：运行前端纯逻辑测试

Run: npm test -- src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts

Expected: PASS，覆盖重复事件、复检轮次、Provider 失败、Schema 失败和业务 QA 失败。

  • Step 6：提交前端数据层
git add src/components/architecture/api-client.ts src/components/architecture/trace-state.ts src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts
git commit -m "新增后台架构前端状态层"

Task 9：构建固定架构图和完整日志详情组件

Files:

  • Create: src/components/architecture/architecture-flow.tsx

  • Create: src/components/architecture/llm-call-detail.tsx

  • Create: src/components/architecture/architecture-observer-panel.tsx

  • Modify: src/app/globals.css

  • Test: src/components/architecture/__tests__/trace-state.test.ts

  • Step 1：先写界面格式化失败测试

把界面格式化保持为纯函数并测试：

it("formats every architecture node and call label in Chinese", () => {
  expect(formatNodeStatus("running")).toBe("运行中");
  expect(formatNodeStatus("failed")).toBe("失败");
  expect(formatCallLabel(call({
    sequence: 4,
    task: "targeted_rewriter",
    rewrite_round: 1,
    duration_ms: 22600,
  }))).toBe("4 · 定向修复第 1 轮 · 22.6 秒");
});
  • Step 2：运行测试并确认格式化函数不存在

Run: npm test -- src/components/architecture/__tests__/trace-state.test.ts

Expected: FAIL。

  • Step 3：实现固定拓扑组件

architecture-flow.tsx 只接受派生后的节点，不读取 API：

export function ArchitectureFlow({
  nodes,
}: {
  nodes: Record<ArchitectureNodeView["id"], ArchitectureNodeView>;
}) {
  const order: ArchitectureNodeView["id"][] = [
    "input", "fact_card", "draft", "qa", "rewrite", "final",
  ];
  return (
    <section className="architecture-flow" aria-label="文章优化后台架构">
      {order.map((id) => {
        const node = nodes[id];
        return (
          <article
            className={`architecture-node architecture-node-${node.status}`}
            data-node={node.id}
            key={node.id}
          >
            <strong>{node.label}</strong>
            <span>{formatNodeStatus(node.status)}</span>
            <small>{node.detail}</small>
          </article>
        );
      })}
      <p className="architecture-loop-label">质量检查未通过时，定向修复后返回复检，最多两轮。</p>
    </section>
  );
}

固定节点不能按调用次数新增；复检只改变 QA 节点并在调用轨迹中增加一条调用。

  • Step 4：实现调用详情按需读取

llm-call-detail.tsx 接受 jobId、call、apiAccessKey 和 technicalDetailsEnabled。只有技术详情开启且用户选择 请求 或 响应 时才调用 getTracePayload()：

const [tab, setTab] = useState<"request" | "response" | "validation">(
  "validation",
);
const [payload, setPayload] = useState<unknown>(null);

useEffect(() => {
  if (!technicalDetailsEnabled || tab === "validation") return;
  let current = true;
  setPayload(null);
  getTracePayload(jobId, call.call_id, tab, apiAccessKey)
    .then((value) => current && setPayload(value))
    .catch((error) => current && setPayload({ error: errorMessage(error) }));
  return () => { current = false; };
}, [apiAccessKey, call.call_id, jobId, tab, technicalDetailsEnabled]);

return (
  <section className="panel llm-call-detail" aria-live="polite">
    <div className="llm-detail-tabs" role="tablist" aria-label="LLM 调用详情">
      {(["request", "response", "validation"] as const).map((value) => (
        <button
          aria-selected={tab === value}
          className={tab === value ? "active-tab" : undefined}
          key={value}
          onClick={() => setTab(value)}
          role="tab"
          type="button"
        >
          {detailTabLabel(value)}
        </button>
      ))}
    </div>
    <pre className="llm-json-view">
      {JSON.stringify(
        tab === "validation" ? validationView(call) : payload,
        null,
        2,
      )}
    </pre>
  </section>
);

必须使用 <pre> 文本渲染，不使用 dangerouslySetInnerHTML、Markdown HTML 或代码高亮器。

  • Step 5：实现只读观察容器

architecture-observer-panel.tsx 的 props 固定为：

interface ArchitectureObserverPanelProps {
  apiAccessKey: string;
  currentJobId: string | null;
  liveEvents: LlmTraceStreamEvent[];
}

挂载时优先 getJobTrace(currentJobId)，没有当前任务时 getLatestTrace()；之后用 applyLiveTraceEvent() 合并 liveEvents。页面必须有：

  • “当前任务”或“最近一次任务”明确标签。
  • Provider、模型、运行状态和追踪完整性。
  • ArchitectureFlow。
  • 按 sequence 排列的调用按钮。
  • 默认关闭的“技术详情”开关。
  • 选中调用的 LlmCallDetail。
  • 无记录、401、加载失败和 incomplete 状态。

组件不得出现开始、重试、取消、删除或编辑按钮。

  • Step 6：增加桌面和窄屏 CSS

新增类：.architecture-observer、.architecture-task-strip、.architecture-flow、.architecture-node-*、.architecture-workspace、.llm-call-list、.llm-call-detail、.llm-detail-tabs、.llm-json-view。桌面 .architecture-workspace 使用左侧调用轨迹、右侧详情；在现有 @media (max-width: 900px) 中改成单列。JSON 使用 white-space: pre-wrap 和 overflow-wrap: anywhere，不得产生页面级横向滚动。

  • Step 7：运行纯逻辑测试和 lint

Run: npm test -- src/components/architecture/__tests__/trace-state.test.ts && npm run lint

Expected: PASS；ESLint 不出现 effect 同步 set-state 或未转义内容错误。

  • Step 8：提交观测组件
git add src/components/architecture/architecture-flow.tsx src/components/architecture/llm-call-detail.tsx src/components/architecture/architecture-observer-panel.tsx src/components/architecture/__tests__/trace-state.test.ts src/app/globals.css
git commit -m "新增LLM后台架构观测界面"

Task 10：接入第三标签并完成浏览器验收

Files:

  • Modify: src/app/page.tsx

  • Create: tests/e2e/architecture-observer.spec.ts

  • Modify: tests/e2e/mvp.spec.ts

  • Modify: tests/e2e/sample-flow/page-flow.ts

  • Step 1：写第三标签和切换不中断的 E2E 失败测试

architecture-observer.spec.ts 使用 page.route() 模拟分块 NDJSON 和追踪正文接口：

test("后台架构标签跟随当前任务并按需展示真实请求响应", async ({ page }) => {
  await mockOptimizationStream(page);
  await mockTracePayloads(page);
  await page.goto("/");
  await page.getByLabel("访问密钥").fill("local-dev-key");
  await page.getByLabel("文章内容").fill("示例文章正文");
  await page.getByRole("button", { name: "开始优化" }).click();

  await page.getByRole("button", { name: "后台架构" }).click();
  await expect(page.getByText("当前任务")).toBeVisible();
  await expect(page.getByText("article_optimizer")).toBeVisible();

  await page.getByRole("button", { name: /article_optimizer/ }).click();
  await page.getByRole("checkbox", { name: "技术详情" }).check();
  await page.getByRole("tab", { name: "请求" }).click();
  await expect(page.locator(".llm-json-view")).toContainText("messages");
  await page.getByRole("tab", { name: "响应" }).click();
  await expect(page.locator(".llm-json-view")).toContainText("choices");
  await expect(page.getByText("终稿已生成")).toBeVisible();
});

Mock 路由记录请求次数，并断言关闭技术详情时正文接口调用次数为 0。

  • Step 2：运行 E2E 并确认第三标签不存在

Run: npx playwright test tests/e2e/architecture-observer.spec.ts --project=chromium

Expected: FAIL，找不到 后台架构 按钮。

  • Step 3：在首页接入追踪事件状态

page.tsx 修改为：

const [activeTab, setActiveTab] = useState<"geo" | "copy" | "architecture">("geo");
const [llmTraceEvents, setLlmTraceEvents] = useState<LlmTraceStreamEvent[]>([]);

function handleStreamEvent(event: OptimizationStreamEvent) {
  switch (event.type) {
    case "llm_call_started":
    case "llm_call_responded":
    case "llm_call_validated":
    case "llm_call_failed":
    case "trace_warning":
      setLlmTraceEvents((current) => [...current, event]);
      return;
    case "job_created":
      setJobId(event.job.id);
      setLlmTraceEvents([]);
      setStreamActivity("任务已创建");
      return;
    case "fact_card_ready":
      setFactCard(event.fact_card);
      setStreamActivity("事实卡已就绪");
      return;
    case "draft_started":
      setStreamActivity(event.message);
      return;
    case "draft_ready":
      setDraftArticle(event.article);
      setStreamActivity("草稿已生成");
      return;
    case "qa_started":
      setStreamActivity(event.message);
      return;
    case "qa_ready":
      setQaReport(event.qa_report);
      setStreamActivity("质量检查完成");
      return;
    case "rewrite_started":
      setStreamActivity(`正在修复第 ${event.round} 轮`);
      return;
    case "rewrite_ready":
      setDraftArticle(event.article);
      setStreamActivity(`第 ${event.round} 轮修复完成`);
      return;
    case "final_ready":
      setOptimizedArticle(event.optimized_article);
      setDraftArticle(null);
      setQaReport(event.qa_report);
      setStreamActivity("终稿已生成");
      setMessage(
        event.qa_report.overall_status === "fail"
          ? "优化完成，质检发现需要复核的问题。"
          : "优化完成。",
      );
      return;
    case "failed":
      setStreamActivity("优化失败");
      throw new Error(event.error);
  }
}

第三个标签按钮文字固定为 后台架构。内容选择使用三个明确分支，不能把 architecture 落入普通文案分支：

{activeTab === "geo" ? (
  <>
    <ProgressPanel
      action={busyAction}
      elapsedSeconds={elapsedSeconds}
      lastTiming={lastTiming}
      liveProgress={null}
    />
    <div className="workflow-grid">
      <ArticleInputForm
        isSubmitting={busyAction === "optimize"}
        value={input}
        onChange={setInput}
        onSubmit={optimize}
      />
      <FactCardEditor
        factCard={factCard}
        isSaving={busyAction === "optimize"}
        onChange={setFactCard}
      />
      <OptimizedPreview
        activityText={streamActivity}
        article={optimizedArticle}
        draftArticle={draftArticle}
        isStreaming={busyAction === "optimize"}
        jobId={jobId}
      />
      <QaReportPanel report={qaReport} />
      <PerformanceCalibrationPanel
        apiAccessKey={apiAccessKey}
        jobId={jobId}
        key={`${jobId ?? "no-job"}-${optimizedArticle?.revision ?? "no-revision"}`}
        optimizedRevision={optimizedArticle?.revision ?? null}
      />
    </div>
  </>
) : activeTab === "copy" ? (
  <RenweiCopyOptimizerPanel apiAccessKey={apiAccessKey} />
) : (
  <ArchitectureObserverPanel
    apiAccessKey={apiAccessKey}
    currentJobId={jobId}
    liveEvents={llmTraceEvents}
  />
)}

在导航中紧跟“普通文案优化”增加 后台架构 按钮，并导入 ArchitectureObserverPanel 与 LlmTraceStreamEvent。现有 GEO JSX 原位保留。

  • Step 4：证明切换标签不取消流读取

不要把 readOptimizationStream() 移进标签组件，也不要在切换标签时 abort fetch。E2E mock 在点击 后台架构 后继续发送 qa_ready 和 final_ready，测试必须看到最终状态。

  • Step 5：更新旧 MVP 回归为当前一键优化流程

把 tests/e2e/mvp.spec.ts 中已不存在的“标题”“正文”“分析文章”“采纳为核心事实”“确认事实卡”步骤删除，改为填写 文章内容、图片描述或图片链接、用户要求，然后直接点击一次 开始优化；继续断言 optimized.md、质量报告、optimized.docx 和 qa_report.json。这只更新测试以匹配当前产品，不把产品改回旧流程。

  • Step 6：扩展真实样例验证但不落盘正文

在 tests/e2e/sample-flow/page-flow.ts 增加：

async function validateLlmTrace({ request, baseURL, apiAccessKey, jobId }: {
  request: APIRequestContext;
  baseURL: string;
  apiAccessKey: string;
  jobId: string;
}) {
  const headers = { "x-api-key": apiAccessKey };
  const manifestResponse = await request.get(
    `${baseURL}/api/jobs/${jobId}/llm-trace`,
    { headers },
  );
  expect(manifestResponse.ok()).toBe(true);
  const manifest = await manifestResponse.json() as LlmTraceManifest;
  expect(manifest.calls.length).toBeGreaterThan(0);

  for (const call of manifest.calls) {
    const requestResponse = await request.get(
      `${baseURL}/api/jobs/${jobId}/llm-trace/${call.call_id}/request`,
      { headers },
    );
    expect(requestResponse.ok()).toBe(true);
    const requestBody = await requestResponse.json() as { messages?: unknown[] };
    expect(Array.isArray(requestBody.messages)).toBe(true);

    if (call.response_available) {
      const responseResponse = await request.get(
        `${baseURL}/api/jobs/${jobId}/llm-trace/${call.call_id}/response`,
        { headers },
      );
      expect(responseResponse.ok()).toBe(true);
      const responseBody = await responseResponse.json() as { choices?: unknown[] };
      expect(Array.isArray(responseBody.choices)).toBe(true);
    }
  }

  return manifest.calls.map((call) => call.task);
}

把返回值写入现有 SampleResult.llm_tasks。不得把 request/response 正文写入 test-results、截图注释或 summary JSON。

  • Step 7：运行确定性 E2E 和相关单元测试

Run: npx playwright test tests/e2e/architecture-observer.spec.ts tests/e2e/mvp.spec.ts --project=chromium

Run: npm test -- src/components/architecture src/lib/workflow/__tests__/stream-events.test.ts

Expected: 全部 PASS，无控制台错误和横向溢出。

  • Step 8：提交首页和端到端接入
git add src/app/page.tsx tests/e2e/architecture-observer.spec.ts tests/e2e/mvp.spec.ts tests/e2e/sample-flow/page-flow.ts
git commit -m "接入后台架构标签页"

Task 11：迁移、全量验证和真实 Provider 验收

Files:

  • Verify only: all files from Tasks 1-10

  • Step 1：应用本地 D1 迁移并核对表

Run: npm run d1:migrate:local

Expected: 0004_llm_trace_observability.sql applied successfully；再次运行显示没有待执行迁移。

  • Step 2：运行完整质量门禁

Run: npm run lint

Expected: PASS，0 errors。

Run: npm test

Expected: PASS，包含新增 trace repository、payload store、recorder、client、API 和 UI 状态测试。

Run: npm run build

Expected: PASS，Next.js 生产构建成功。

  • Step 3：运行完整 Playwright 回归

Run: npx playwright test --project=chromium

Expected: PASS；Task 10 已把 mvp.spec.ts 更新为当前一键优化流程。

  • Step 4：运行一次真实 DeepSeek 安全样例

确保 .env.local 只在本地包含 API_ACCESS_KEY 和 DEEPSEEK_API_KEY，然后运行：

Run: npm run test:e2e:samples -- --limit 1 --timeout-ms 600000

Expected: PASS，并在生成的 test-results/geo-sample-flow/<timestamp>/summary.md 中看到非空 llm_tasks。报告不得包含完整请求、完整响应、访问密钥或 Provider 密钥。

  • Step 5：检查敏感信息和非预期文件

Run:

git status --short --ignored=matching
rg -n "auth\.token|secretKey|healthsource|Authorization|DEEPSEEK_API_KEY" . --glob '!node_modules/**' --glob '!.next/**' --glob '!.open-next/**' --glob '!deploy/*.toml' --glob '!test-results/**'

Expected: 没有新提交的真实密钥；data/、test-results/、追踪正文和生成构建目录保持忽略。用户原有未提交文件仍保持原状。

  • Step 6：核对提交边界和最终状态

Run: git status --short && git log --oneline -12

Expected: Tasks 1-10 的功能文件均已包含在对应中文提交中；只剩用户原有的 README.md、wrangler.jsonc、样例或导出目录等未提交改动，没有本功能遗漏文件。若质量门禁暴露功能缺陷，返回拥有该文件的任务按“失败测试—最小修正—验证—中文提交”闭环处理，不创建内容不明的兜底提交。

完成定义

  • 所有任务按 TDD 顺序完成，每个任务都有独立中文提交。
  • D1/SQLite 索引和 R2/本地正文存储均有测试。
  • SDK 请求对象和保存请求深度相等，SDK 响应对象和保存响应深度相等。
  • 公开 NDJSON 和清单 API 不含完整正文。
  • 完整正文只能通过受保护且 no-store 的按需接口读取。
  • 架构图由真实事件驱动，QA 业务失败与技术失败分开表达。
  • 切换标签不取消任务，刷新可恢复已持久化状态。
  • 只保留运行中任务和最近终态任务的完整正文。
  • npm run lint、npm test、npm run build、Playwright 回归和一次真实安全样例全部通过。
Reference in New Issue View Git Blame Copy Permalink