1889 lines
63 KiB
Markdown
1889 lines
63 KiB
Markdown
# LLM 后台架构观测标签页实施计划
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** 在首页增加只读的 `后台架构` 标签，以真实后端事件驱动固定架构图，并让授权用户查看当前或最近一次任务的完整 LLM 请求、完整响应和校验结果。
|
||
|
||
**Architecture:** `src/lib/llm/client.ts` 产生包含真实 SDK 对象的内部追踪事件，独立追踪收集器把索引写入 D1/SQLite、把完整正文写入现有私有 R2/本地数据目录，并向现有 NDJSON 流发送不含正文的状态事件。前端共享当前任务事件，刷新后通过受保护的只读 API 恢复追踪清单，并按需加载请求或响应正文。
|
||
|
||
**Tech Stack:** Next.js 16、React 19、TypeScript、Vitest、Playwright、Zod、OpenAI-compatible SDK、Cloudflare Workers、D1、R2、better-sqlite3。
|
||
|
||
---
|
||
|
||
## 实施前文件结构
|
||
|
||
### 新建文件
|
||
|
||
- `migrations/0004_llm_trace_observability.sql`：生产 D1 追踪表和索引。
|
||
- `src/lib/llm/trace-types.ts`：Provider、任务、运行、调用、内部事件、公开流事件和 API 类型。
|
||
- `src/lib/llm/trace-repository.ts`：追踪索引仓储接口和运行时选择。
|
||
- `src/lib/llm/sqlite-trace-repository.ts`：本地 SQLite 追踪索引实现。
|
||
- `src/lib/llm/d1-trace-repository.ts`：Cloudflare D1 追踪索引实现。
|
||
- `src/lib/llm/trace-payload-store.ts`：本地目录与 R2 的完整 JSON 正文存储。
|
||
- `src/lib/llm/trace-recorder.ts`：内部 LLM 事件、工作流事件、持久化、公开事件和保留策略协调器。
|
||
- `src/lib/llm/trace-http.ts`：受保护追踪 API 的无缓存 JSON 响应和正文读取助手。
|
||
- `src/lib/llm/__tests__/trace-repository.test.ts`：SQLite 追踪仓储测试。
|
||
- `src/lib/llm/__tests__/d1-trace-repository.test.ts`：D1 追踪仓储测试。
|
||
- `src/lib/llm/__tests__/trace-payload-store.test.ts`：本地和 R2 正文存储测试。
|
||
- `src/lib/llm/__tests__/trace-recorder.test.ts`：事件顺序、业务结果、失败隔离和保留测试。
|
||
- `src/app/api/llm-traces/latest/route.ts`：当前或最近终态追踪入口。
|
||
- `src/app/api/jobs/[jobId]/llm-trace/route.ts`：任务追踪清单入口。
|
||
- `src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts`：完整请求入口。
|
||
- `src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts`：完整响应入口。
|
||
- `src/app/api/__tests__/llm-traces.test.ts`：追踪 API 鉴权、正文和缓存测试。
|
||
- `src/components/architecture/api-client.ts`：浏览器端追踪 API 客户端。
|
||
- `src/components/architecture/trace-state.ts`：实时事件归并和固定架构节点派生。
|
||
- `src/components/architecture/architecture-flow.tsx`：固定拓扑架构图。
|
||
- `src/components/architecture/llm-call-detail.tsx`：请求、响应、校验详情。
|
||
- `src/components/architecture/architecture-observer-panel.tsx`：只读观测标签页容器。
|
||
- `src/components/architecture/__tests__/api-client.test.ts`：按需正文请求测试。
|
||
- `src/components/architecture/__tests__/trace-state.test.ts`：状态归并和业务/技术失败区分测试。
|
||
- `tests/e2e/architecture-observer.spec.ts`：标签切换、调用选择和技术详情端到端测试。
|
||
|
||
### 修改文件
|
||
|
||
- `src/lib/db/schema.ts`：本地 SQLite 创建追踪表。
|
||
- `src/lib/llm/audit.ts`：从共享类型导入 Provider 和任务名。
|
||
- `src/lib/llm/client.ts`：记录真实请求、响应、解析、校验和安全错误。
|
||
- `src/lib/llm/__tests__/client.test.ts`：请求/响应深度相等和事件顺序测试。
|
||
- `src/lib/workflow/fact-extractor.ts`：传递事实提取追踪上下文。
|
||
- `src/lib/workflow/article-optimizer.ts`：传递生成草稿追踪上下文。
|
||
- `src/lib/workflow/quality-inspector.ts`：传递 QA 和复检轮次追踪上下文。
|
||
- `src/lib/workflow/targeted-rewriter.ts`：传递定向修复轮次追踪上下文。
|
||
- `src/lib/workflow/streaming-optimizer.ts`：向追踪器转发工作流事件和轮次。
|
||
- `src/lib/workflow/stream-events.ts`：公开 LLM 状态事件加入 NDJSON 联合类型。
|
||
- `src/lib/workflow/__tests__/stream-events.test.ts`：新事件编码和分块解析测试。
|
||
- `src/app/api/jobs/optimize-stream/route.ts`：创建、连接和结束追踪器。
|
||
- `src/app/api/__tests__/jobs.test.ts`：真实调用状态事件和失败现场测试。
|
||
- `src/app/page.tsx`：第三标签、实时追踪状态和只读组件接入。
|
||
- `src/app/globals.css`：架构图、调用轨迹、详情和窄屏布局。
|
||
- `tests/e2e/mvp.spec.ts`：把旧的“分析—确认—优化”流程断言更新为当前一键优化界面。
|
||
- `tests/e2e/sample-flow/page-flow.ts`：真实 Provider 样例核对追踪接口，不保存客户正文。
|
||
|
||
### 明确不修改
|
||
|
||
- `wrangler.jsonc`：复用现有私有 `EXPORT_BUCKET`，不增加绑定。
|
||
- `README.md`：本次规格已经完整记录功能，不扩大文档范围。
|
||
- 用户当前未提交的样例、导出目录和本地数据。
|
||
|
||
## Task 1：定义追踪契约和公开流事件
|
||
|
||
**Files:**
|
||
- Create: `src/lib/llm/trace-types.ts`
|
||
- Modify: `src/lib/llm/audit.ts`
|
||
- Modify: `src/lib/llm/client.ts`
|
||
- Modify: `src/lib/workflow/stream-events.ts`
|
||
- Test: `src/lib/workflow/__tests__/stream-events.test.ts`
|
||
|
||
- [ ] **Step 1：先写公开追踪事件的失败测试**
|
||
|
||
在 `src/lib/workflow/__tests__/stream-events.test.ts` 增加：
|
||
|
||
~~~ts
|
||
it("encodes LLM call metadata without raw request or response bodies", () => {
|
||
const event: OptimizationStreamEvent = {
|
||
type: "llm_call_started",
|
||
job_id: "job_123",
|
||
call_id: "llmcall_1",
|
||
sequence: 1,
|
||
task: "fact_extractor",
|
||
workflow_stage: "fact_card",
|
||
rewrite_round: null,
|
||
provider: "deepseek",
|
||
model: "deepseek-v4-pro",
|
||
started_at: "2026-07-16T00:00:00.000Z",
|
||
request_available: true,
|
||
};
|
||
|
||
const encoded = encodeOptimizationStreamEvent(event);
|
||
expect(JSON.parse(encoded)).toEqual(event);
|
||
expect(encoded).not.toContain("messages");
|
||
expect(encoded).not.toContain("Authorization");
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行测试并确认类型或断言失败**
|
||
|
||
Run: `npm test -- src/lib/workflow/__tests__/stream-events.test.ts`
|
||
|
||
Expected: FAIL，因为 `OptimizationStreamEvent` 尚不接受 `llm_call_started`。
|
||
|
||
- [ ] **Step 3：创建共享类型并接入现有类型引用**
|
||
|
||
`src/lib/llm/trace-types.ts` 必须定义以下完整公共边界：
|
||
|
||
~~~ts
|
||
export type LlmProviderName = "deepseek" | "openai";
|
||
|
||
export type LlmTaskName =
|
||
| "unknown"
|
||
| "fact_extractor"
|
||
| "article_optimizer"
|
||
| "quality_inspector"
|
||
| "targeted_rewriter"
|
||
| "renwei_copy_optimizer";
|
||
|
||
export type LlmTraceWorkflowStage =
|
||
| "unknown"
|
||
| "input"
|
||
| "fact_card"
|
||
| "draft"
|
||
| "qa"
|
||
| "rewrite"
|
||
| "final";
|
||
|
||
export type LlmTraceRunStatus =
|
||
| "running"
|
||
| "completed"
|
||
| "failed"
|
||
| "interrupted";
|
||
|
||
export type LlmTraceCallStatus =
|
||
| "started"
|
||
| "responded"
|
||
| "validated"
|
||
| "failed";
|
||
|
||
export type LlmTraceCompleteness = "complete" | "incomplete";
|
||
export type LlmBusinessStatus = "pass" | "warn" | "fail";
|
||
export type LlmTraceErrorType =
|
||
| "provider"
|
||
| "json_parse"
|
||
| "schema_validation";
|
||
|
||
export interface LlmTraceContext {
|
||
workflow_stage: LlmTraceWorkflowStage;
|
||
rewrite_round?: number;
|
||
schema_name?: string;
|
||
}
|
||
|
||
export interface LlmTraceRun {
|
||
job_id: string;
|
||
case_id: string | null;
|
||
status: LlmTraceRunStatus;
|
||
current_stage: LlmTraceWorkflowStage;
|
||
trace_completeness: LlmTraceCompleteness;
|
||
error_stage: string | null;
|
||
error_summary: string | null;
|
||
started_at: string;
|
||
finished_at: string | null;
|
||
updated_at: string;
|
||
}
|
||
|
||
export interface LlmTraceCall {
|
||
call_id: string;
|
||
job_id: string;
|
||
sequence: number;
|
||
task: LlmTaskName;
|
||
workflow_stage: LlmTraceWorkflowStage;
|
||
rewrite_round: number | null;
|
||
provider: LlmProviderName;
|
||
model: string;
|
||
status: LlmTraceCallStatus;
|
||
request_object_key: string | null;
|
||
response_object_key: string | null;
|
||
token_usage: Record<string, number> | null;
|
||
schema_name: string | null;
|
||
schema_valid: boolean | null;
|
||
validation_issues: string[];
|
||
business_status: LlmBusinessStatus | null;
|
||
duration_ms: number | null;
|
||
started_at: string;
|
||
responded_at: string | null;
|
||
validated_at: string | null;
|
||
failed_at: string | null;
|
||
error_type: LlmTraceErrorType | null;
|
||
error_summary: string | null;
|
||
}
|
||
|
||
export type LlmTraceCallPublic = Omit<
|
||
LlmTraceCall,
|
||
"request_object_key" | "response_object_key"
|
||
> & {
|
||
request_available: boolean;
|
||
response_available: boolean;
|
||
};
|
||
|
||
export interface LlmTraceManifest {
|
||
run: LlmTraceRun;
|
||
calls: LlmTraceCallPublic[];
|
||
}
|
||
|
||
export type LlmClientTraceEvent =
|
||
| {
|
||
type: "started";
|
||
call_id: string;
|
||
task: LlmTaskName;
|
||
context: LlmTraceContext;
|
||
provider: LlmProviderName;
|
||
model: string;
|
||
request: unknown;
|
||
started_at: string;
|
||
}
|
||
| {
|
||
type: "responded";
|
||
call_id: string;
|
||
response: unknown;
|
||
duration_ms: number;
|
||
responded_at: string;
|
||
}
|
||
| {
|
||
type: "validated";
|
||
call_id: string;
|
||
schema_name: string;
|
||
schema_valid: boolean;
|
||
validation_issues: string[];
|
||
validated_at: string;
|
||
}
|
||
| {
|
||
type: "failed";
|
||
call_id: string;
|
||
error_type: LlmTraceErrorType;
|
||
error_summary: string;
|
||
duration_ms: number;
|
||
failed_at: string;
|
||
};
|
||
|
||
export type LlmClientTraceHandler = (
|
||
event: LlmClientTraceEvent,
|
||
) => void | Promise<void>;
|
||
|
||
export type LlmTraceStreamEvent =
|
||
| {
|
||
type: "llm_call_started";
|
||
job_id: string;
|
||
call_id: string;
|
||
sequence: number;
|
||
task: LlmTaskName;
|
||
workflow_stage: LlmTraceWorkflowStage;
|
||
rewrite_round: number | null;
|
||
provider: LlmProviderName;
|
||
model: string;
|
||
started_at: string;
|
||
request_available: boolean;
|
||
}
|
||
| {
|
||
type: "llm_call_responded";
|
||
job_id: string;
|
||
call_id: string;
|
||
duration_ms: number;
|
||
token_usage: Record<string, number> | null;
|
||
responded_at: string;
|
||
response_available: boolean;
|
||
}
|
||
| {
|
||
type: "llm_call_validated";
|
||
job_id: string;
|
||
call_id: string;
|
||
schema_name: string;
|
||
schema_valid: boolean;
|
||
validation_issues: string[];
|
||
validated_at: string;
|
||
}
|
||
| {
|
||
type: "llm_call_failed";
|
||
job_id: string;
|
||
call_id: string;
|
||
error_type: LlmTraceErrorType;
|
||
error_summary: string;
|
||
failed_at: string;
|
||
}
|
||
| {
|
||
type: "trace_warning";
|
||
job_id: string;
|
||
trace_completeness: "incomplete";
|
||
error_summary: string;
|
||
};
|
||
~~~
|
||
|
||
把 `LlmTaskName` 从 `client.ts` 移到该文件；`audit.ts` 和 `client.ts` 都从该文件导入。`stream-events.ts` 使用：
|
||
|
||
~~~ts
|
||
import type { LlmTraceStreamEvent } from "../llm/trace-types";
|
||
|
||
type ExistingOptimizationStreamEvents =
|
||
| {
|
||
type: "job_created";
|
||
job: { id: string };
|
||
case?: { id: string; case_type: "article" };
|
||
}
|
||
| { type: "fact_card_ready"; job_id: string; fact_card: OptimizationFactCard }
|
||
| { type: "draft_started"; job_id: string; message: string }
|
||
| { type: "draft_ready"; job_id: string; article: OptimizedArticle }
|
||
| { type: "qa_started"; job_id: string; message: string }
|
||
| { type: "qa_ready"; job_id: string; qa_report: QaReport }
|
||
| { type: "rewrite_started"; job_id: string; round: number }
|
||
| {
|
||
type: "rewrite_ready";
|
||
job_id: string;
|
||
round: number;
|
||
article: OptimizedArticle;
|
||
}
|
||
| {
|
||
type: "final_ready";
|
||
job_id: string;
|
||
case?: { id: string; case_type: "article" };
|
||
result_version?: { id: string; version: number };
|
||
optimized_article: OptimizedArticle;
|
||
qa_report: QaReport;
|
||
export_paths: Record<string, string>;
|
||
}
|
||
| {
|
||
type: "failed";
|
||
job_id?: string;
|
||
case?: { id: string; case_type: "article" };
|
||
result_version?: { id: string; version: number };
|
||
stage: OptimizationStreamStage;
|
||
error: string;
|
||
};
|
||
|
||
export type OptimizationStreamEvent =
|
||
| LlmTraceStreamEvent
|
||
| ExistingOptimizationStreamEvents;
|
||
~~~
|
||
|
||
现有业务事件字段和值保持原样，只增加类型别名和新的联合分支。
|
||
|
||
- [ ] **Step 4：运行流事件测试并确认通过**
|
||
|
||
Run: `npm test -- src/lib/workflow/__tests__/stream-events.test.ts`
|
||
|
||
Expected: PASS，且现有分块 NDJSON 测试继续通过。
|
||
|
||
- [ ] **Step 5：提交契约变更**
|
||
|
||
~~~bash
|
||
git add src/lib/llm/trace-types.ts src/lib/llm/audit.ts src/lib/llm/client.ts src/lib/workflow/stream-events.ts src/lib/workflow/__tests__/stream-events.test.ts
|
||
git commit -m "定义LLM追踪事件契约"
|
||
~~~
|
||
|
||
## Task 2：增加 D1 和 SQLite 追踪索引
|
||
|
||
**Files:**
|
||
- Create: `migrations/0004_llm_trace_observability.sql`
|
||
- Create: `src/lib/llm/trace-repository.ts`
|
||
- Create: `src/lib/llm/sqlite-trace-repository.ts`
|
||
- Create: `src/lib/llm/d1-trace-repository.ts`
|
||
- Create: `src/lib/llm/__tests__/trace-repository.test.ts`
|
||
- Create: `src/lib/llm/__tests__/d1-trace-repository.test.ts`
|
||
- Modify: `src/lib/db/schema.ts`
|
||
|
||
- [ ] **Step 1：写 SQLite 仓储失败测试**
|
||
|
||
测试必须先创建真实文章任务，再验证运行、调用、最近任务和删除：
|
||
|
||
~~~ts
|
||
it("stores a run and ordered calls, then deletes only trace rows", async () => {
|
||
const appRepository = createSqliteRepository(dbPath);
|
||
const job = await appRepository.createArticleJob({
|
||
source_title: "Title",
|
||
source_body: "Body",
|
||
image_inputs: [],
|
||
publish_platform: "official_site",
|
||
user_instructions: "",
|
||
});
|
||
const repository = createSqliteTraceRepository(dbPath);
|
||
|
||
await repository.putRun(runFixture(job.id));
|
||
await repository.putCall(callFixture(job.id, "llmcall_1", 1));
|
||
await repository.putCall(callFixture(job.id, "llmcall_2", 2));
|
||
|
||
await expect(repository.getLatestRun()).resolves.toMatchObject({
|
||
job_id: job.id,
|
||
status: "running",
|
||
});
|
||
await expect(repository.listCalls(job.id)).resolves.toEqual([
|
||
expect.objectContaining({ call_id: "llmcall_1", sequence: 1 }),
|
||
expect.objectContaining({ call_id: "llmcall_2", sequence: 2 }),
|
||
]);
|
||
|
||
await repository.deleteRun(job.id);
|
||
await expect(appRepository.getArticleJob(job.id)).resolves.not.toBeNull();
|
||
await expect(repository.getRun(job.id)).resolves.toBeNull();
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行测试并确认表或模块不存在**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/trace-repository.test.ts`
|
||
|
||
Expected: FAIL，模块或 `llm_trace_runs` 表不存在。
|
||
|
||
- [ ] **Step 3：添加完整迁移和本地 Schema**
|
||
|
||
`migrations/0004_llm_trace_observability.sql`：
|
||
|
||
~~~sql
|
||
CREATE TABLE IF NOT EXISTS llm_trace_runs (
|
||
job_id TEXT PRIMARY KEY,
|
||
case_id TEXT,
|
||
status TEXT NOT NULL,
|
||
current_stage TEXT NOT NULL,
|
||
trace_completeness TEXT NOT NULL,
|
||
error_stage TEXT,
|
||
error_summary TEXT,
|
||
started_at TEXT NOT NULL,
|
||
finished_at TEXT,
|
||
updated_at TEXT NOT NULL,
|
||
FOREIGN KEY (job_id) REFERENCES article_jobs(id) ON DELETE CASCADE,
|
||
FOREIGN KEY (case_id) REFERENCES optimization_cases(id) ON DELETE SET NULL
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS llm_trace_calls (
|
||
call_id TEXT PRIMARY KEY,
|
||
job_id TEXT NOT NULL,
|
||
sequence INTEGER NOT NULL,
|
||
task TEXT NOT NULL,
|
||
workflow_stage TEXT NOT NULL,
|
||
rewrite_round INTEGER,
|
||
provider TEXT NOT NULL,
|
||
model TEXT NOT NULL,
|
||
status TEXT NOT NULL,
|
||
request_object_key TEXT,
|
||
response_object_key TEXT,
|
||
token_usage TEXT,
|
||
schema_name TEXT,
|
||
schema_valid INTEGER,
|
||
validation_issues TEXT NOT NULL DEFAULT '[]',
|
||
business_status TEXT,
|
||
duration_ms INTEGER,
|
||
started_at TEXT NOT NULL,
|
||
responded_at TEXT,
|
||
validated_at TEXT,
|
||
failed_at TEXT,
|
||
error_type TEXT,
|
||
error_summary TEXT,
|
||
UNIQUE (job_id, sequence),
|
||
FOREIGN KEY (job_id) REFERENCES llm_trace_runs(job_id) ON DELETE CASCADE
|
||
);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_llm_trace_runs_status_updated
|
||
ON llm_trace_runs(status, updated_at DESC);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_llm_trace_calls_job_sequence
|
||
ON llm_trace_calls(job_id, sequence);
|
||
~~~
|
||
|
||
把同一表结构加入 `initializeSchema()`，保持本地 SQLite 与 D1 一致。
|
||
|
||
- [ ] **Step 4：定义仓储接口和运行时选择**
|
||
|
||
`trace-repository.ts`：
|
||
|
||
~~~ts
|
||
export interface LlmTraceRepository {
|
||
putRun(run: LlmTraceRun): Promise<void>;
|
||
putCall(call: LlmTraceCall): Promise<void>;
|
||
getRun(jobId: string): Promise<LlmTraceRun | null>;
|
||
getLatestRun(): Promise<LlmTraceRun | null>;
|
||
listCalls(jobId: string): Promise<LlmTraceCall[]>;
|
||
listTerminalRunsExcept(jobId: string): Promise<LlmTraceRun[]>;
|
||
deleteRun(jobId: string): Promise<void>;
|
||
}
|
||
|
||
export function getLlmTraceRepositoryFromRuntime(): LlmTraceRepository {
|
||
if (process.env.APP_RUNTIME === "cloudflare") {
|
||
const env = getAppCloudflareEnv();
|
||
if (!env?.DB) throw new Error("Cloudflare D1 binding DB is required");
|
||
return createD1TraceRepository(env.DB);
|
||
}
|
||
return createSqliteTraceRepository(getDefaultDatabasePath());
|
||
}
|
||
~~~
|
||
|
||
使用 Task 1 已定义的 `LlmTraceRun` 和 `LlmTraceCall`；其字段与迁移一一对应。
|
||
|
||
- [ ] **Step 5：实现 SQLite 和 D1 UPSERT**
|
||
|
||
两个实现都使用完整快照 UPSERT，避免动态拼 SQL。SQLite 的核心语句为：
|
||
|
||
~~~sql
|
||
INSERT INTO llm_trace_calls (
|
||
call_id, job_id, sequence, task, workflow_stage, rewrite_round,
|
||
provider, model, status, request_object_key, response_object_key,
|
||
token_usage, schema_name, schema_valid, validation_issues,
|
||
business_status, duration_ms, started_at, responded_at, validated_at,
|
||
failed_at, error_type, error_summary
|
||
) VALUES (
|
||
@call_id, @job_id, @sequence, @task, @workflow_stage, @rewrite_round,
|
||
@provider, @model, @status, @request_object_key, @response_object_key,
|
||
@token_usage, @schema_name, @schema_valid, @validation_issues,
|
||
@business_status, @duration_ms, @started_at, @responded_at, @validated_at,
|
||
@failed_at, @error_type, @error_summary
|
||
)
|
||
ON CONFLICT(call_id) DO UPDATE SET
|
||
status = excluded.status,
|
||
response_object_key = excluded.response_object_key,
|
||
token_usage = excluded.token_usage,
|
||
schema_name = excluded.schema_name,
|
||
schema_valid = excluded.schema_valid,
|
||
validation_issues = excluded.validation_issues,
|
||
business_status = excluded.business_status,
|
||
duration_ms = excluded.duration_ms,
|
||
responded_at = excluded.responded_at,
|
||
validated_at = excluded.validated_at,
|
||
failed_at = excluded.failed_at,
|
||
error_type = excluded.error_type,
|
||
error_summary = excluded.error_summary;
|
||
~~~
|
||
|
||
D1 使用相同列顺序的 `prepare().bind().run()`；读取时把 JSON 文本和 `0 | 1 | null` 转回 TypeScript 类型。`getLatestRun()` 的顺序固定为运行中优先，再按 `updated_at DESC`：
|
||
|
||
~~~sql
|
||
SELECT * FROM llm_trace_runs
|
||
ORDER BY CASE WHEN status = 'running' THEN 0 ELSE 1 END, updated_at DESC
|
||
LIMIT 1;
|
||
~~~
|
||
|
||
- [ ] **Step 6：增加 D1 绑定顺序测试并运行两套测试**
|
||
|
||
D1 测试至少断言 `putRun()` 使用 `insert into llm_trace_runs`，`putCall()` 将 `validation_issues` 和 `token_usage` 序列化，`listCalls()` 按 `sequence` 读取。
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/trace-repository.test.ts src/lib/llm/__tests__/d1-trace-repository.test.ts src/lib/db/__tests__/repository.test.ts`
|
||
|
||
Expected: PASS。
|
||
|
||
- [ ] **Step 7：提交索引存储**
|
||
|
||
~~~bash
|
||
git add migrations/0004_llm_trace_observability.sql src/lib/db/schema.ts src/lib/llm/trace-repository.ts src/lib/llm/sqlite-trace-repository.ts src/lib/llm/d1-trace-repository.ts src/lib/llm/__tests__/trace-repository.test.ts src/lib/llm/__tests__/d1-trace-repository.test.ts
|
||
git commit -m "新增LLM追踪索引存储"
|
||
~~~
|
||
|
||
## Task 3：实现私有完整正文存储
|
||
|
||
**Files:**
|
||
- Create: `src/lib/llm/trace-payload-store.ts`
|
||
- Create: `src/lib/llm/__tests__/trace-payload-store.test.ts`
|
||
|
||
- [ ] **Step 1：先写本地与 R2 失败测试**
|
||
|
||
~~~ts
|
||
it("round-trips exact JSON locally and deletes one job prefix", async () => {
|
||
const store = createLocalTracePayloadStore(tempDir);
|
||
const payload = { model: "deepseek-v4-pro", messages: [{ role: "user", content: "原文" }] };
|
||
const key = "llm-traces/job_1/llmcall_1/request.json";
|
||
|
||
await store.putJson(key, payload);
|
||
await expect(store.getJson(key)).resolves.toEqual(payload);
|
||
await store.deleteJob("job_1");
|
||
await expect(store.getJson(key)).resolves.toBeNull();
|
||
});
|
||
|
||
it("stores private JSON in R2 without a public URL", async () => {
|
||
const put = vi.fn().mockResolvedValue(undefined);
|
||
const get = vi.fn().mockResolvedValue({ json: async () => ({ ok: true }) });
|
||
const bucket = { put, get } as unknown as R2Bucket;
|
||
const store = createR2TracePayloadStore(bucket);
|
||
|
||
await store.putJson("llm-traces/job_1/llmcall_1/request.json", { ok: true });
|
||
await expect(store.getJson("llm-traces/job_1/llmcall_1/request.json"))
|
||
.resolves.toEqual({ ok: true });
|
||
expect(put).toHaveBeenCalledWith(
|
||
"llm-traces/job_1/llmcall_1/request.json",
|
||
JSON.stringify({ ok: true }),
|
||
{ httpMetadata: { contentType: "application/json; charset=utf-8" } },
|
||
);
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行测试并确认模块不存在**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/trace-payload-store.test.ts`
|
||
|
||
Expected: FAIL，正文存储工厂尚不存在。
|
||
|
||
- [ ] **Step 3：实现正文存储接口和工厂**
|
||
|
||
~~~ts
|
||
export interface LlmTracePayloadStore {
|
||
putJson(key: string, value: unknown): Promise<void>;
|
||
getJson(key: string): Promise<unknown | null>;
|
||
deleteJob(jobId: string): Promise<void>;
|
||
}
|
||
|
||
export function tracePayloadKey(
|
||
jobId: string,
|
||
callId: string,
|
||
kind: "request" | "response",
|
||
) {
|
||
return `llm-traces/${jobId}/${callId}/${kind}.json`;
|
||
}
|
||
|
||
export function getLlmTracePayloadStoreFromRuntime(): LlmTracePayloadStore {
|
||
if (process.env.APP_RUNTIME === "cloudflare") {
|
||
const bucket = getAppCloudflareEnv()?.EXPORT_BUCKET;
|
||
if (!bucket) throw new Error("Cloudflare R2 binding EXPORT_BUCKET is required");
|
||
return createR2TracePayloadStore(bucket);
|
||
}
|
||
return createLocalTracePayloadStore(getAppDataDir());
|
||
}
|
||
~~~
|
||
|
||
本地实现只能在 `join(dataDir, "llm-traces")` 下使用生成的键，使用 `mkdirSync`、`writeFileSync`、`readFileSync` 和 `rmSync`。R2 `deleteJob()` 必须循环 `bucket.list({ prefix, cursor })`，收集对象键后调用 `bucket.delete(keys)`，直到 `truncated` 为 false。
|
||
|
||
- [ ] **Step 4：运行正文存储和现有导出存储测试**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/trace-payload-store.test.ts src/lib/workflow/__tests__/export-store.test.ts`
|
||
|
||
Expected: PASS；现有 `exports/` 行为不变。
|
||
|
||
- [ ] **Step 5：提交正文存储**
|
||
|
||
~~~bash
|
||
git add src/lib/llm/trace-payload-store.ts src/lib/llm/__tests__/trace-payload-store.test.ts
|
||
git commit -m "新增LLM追踪正文存储"
|
||
~~~
|
||
|
||
## Task 4：实现追踪收集器、业务状态和保留策略
|
||
|
||
**Files:**
|
||
- Create: `src/lib/llm/trace-recorder.ts`
|
||
- Create: `src/lib/llm/__tests__/trace-recorder.test.ts`
|
||
|
||
- [ ] **Step 1：写成功调用、QA 业务失败和保留策略测试**
|
||
|
||
使用内存 fake 仓储和正文存储，覆盖完整顺序：
|
||
|
||
~~~ts
|
||
it("persists exact bodies before publishing public metadata", async () => {
|
||
const published: LlmTraceStreamEvent[] = [];
|
||
const recorder = await createLlmTraceRecorder({
|
||
jobId: "job_1",
|
||
caseId: "case_1",
|
||
repository,
|
||
payloadStore,
|
||
publish: (event) => published.push(event),
|
||
});
|
||
|
||
await recorder.onLlmEvent({
|
||
type: "started",
|
||
call_id: "llmcall_1",
|
||
task: "quality_inspector",
|
||
context: { workflow_stage: "qa", schema_name: "llmQaPatchSchema" },
|
||
provider: "deepseek",
|
||
model: "deepseek-v4-pro",
|
||
request: { model: "deepseek-v4-pro", messages: [] },
|
||
started_at: "2026-07-16T00:00:00.000Z",
|
||
});
|
||
|
||
expect(payloadStore.values.get(
|
||
"llm-traces/job_1/llmcall_1/request.json",
|
||
)).toEqual({ model: "deepseek-v4-pro", messages: [] });
|
||
expect(published[0]).toMatchObject({
|
||
type: "llm_call_started",
|
||
request_available: true,
|
||
});
|
||
expect(JSON.stringify(published)).not.toContain("messages");
|
||
});
|
||
|
||
it("stores QA business failure separately from schema success", async () => {
|
||
await recorder.onWorkflowEvent({
|
||
type: "qa_ready",
|
||
job_id: "job_1",
|
||
qa_report: { overall_status: "fail", checks: [] },
|
||
});
|
||
await expect(repository.listCalls("job_1")).resolves.toEqual([
|
||
expect.objectContaining({
|
||
task: "quality_inspector",
|
||
schema_valid: true,
|
||
business_status: "fail",
|
||
}),
|
||
]);
|
||
});
|
||
|
||
it("keeps running runs and only the newest terminal full trace", async () => {
|
||
await recorder.finish({ status: "completed" });
|
||
expect(payloadStore.deletedJobs).toEqual(["job_old_completed"]);
|
||
expect(repository.deletedRuns).toEqual(["job_old_completed"]);
|
||
expect(repository.deletedRuns).not.toContain("job_other_running");
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行测试并确认收集器不存在**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/trace-recorder.test.ts`
|
||
|
||
Expected: FAIL，`createLlmTraceRecorder` 尚不存在。
|
||
|
||
- [ ] **Step 3：实现收集器的公开接口**
|
||
|
||
~~~ts
|
||
export interface LlmTraceRecorder {
|
||
onLlmEvent: LlmClientTraceHandler;
|
||
onWorkflowEvent(event: OptimizationStreamEvent): Promise<void>;
|
||
finish(input: {
|
||
status: "completed" | "failed" | "interrupted";
|
||
errorStage?: string;
|
||
errorSummary?: string;
|
||
}): Promise<void>;
|
||
}
|
||
|
||
export function createNoopLlmTraceRecorder(): LlmTraceRecorder {
|
||
return {
|
||
onLlmEvent: async () => undefined,
|
||
onWorkflowEvent: async () => undefined,
|
||
finish: async () => undefined,
|
||
};
|
||
}
|
||
|
||
export async function createLlmTraceRecorder({
|
||
jobId,
|
||
caseId,
|
||
repository,
|
||
payloadStore,
|
||
publish,
|
||
}: CreateLlmTraceRecorderInput): Promise<LlmTraceRecorder> {
|
||
const calls = new Map<string, LlmTraceCall>();
|
||
let sequence = 0;
|
||
let run = createRunningRun(jobId, caseId);
|
||
await repository.putRun(run);
|
||
|
||
const warn = async (error: unknown) => {
|
||
run = { ...run, trace_completeness: "incomplete", updated_at: nowIso() };
|
||
await repository.putRun(run).catch(() => undefined);
|
||
await publish({
|
||
type: "trace_warning",
|
||
job_id: jobId,
|
||
trace_completeness: "incomplete",
|
||
error_summary: safeTraceError(error),
|
||
});
|
||
};
|
||
|
||
return {
|
||
onLlmEvent: async (event) => {
|
||
try {
|
||
await applyLlmEvent(event);
|
||
} catch (error) {
|
||
await warn(error);
|
||
}
|
||
},
|
||
onWorkflowEvent: async (event) => {
|
||
try {
|
||
await applyWorkflowEvent(event);
|
||
} catch (error) {
|
||
await warn(error);
|
||
}
|
||
},
|
||
finish: async (input) => {
|
||
try {
|
||
await finishRun(input);
|
||
} catch (error) {
|
||
await warn(error);
|
||
}
|
||
},
|
||
};
|
||
}
|
||
~~~
|
||
|
||
`applyLlmEvent()` 的硬性顺序：
|
||
|
||
1. `started`：先写 request.json，再 `putCall()`，最后发布 `llm_call_started`。
|
||
2. `responded`：先写 response.json，再更新调用，最后发布 `llm_call_responded`。
|
||
3. `validated`：更新 Schema 字段，再发布 `llm_call_validated`。
|
||
4. `failed`：更新错误白名单字段，再发布 `llm_call_failed`。
|
||
|
||
若正文写入失败，`onLlmEvent()` 仍尝试写入不带对象键的调用索引并发布 `request_available: false` 或 `response_available: false`，随后发布 `trace_warning`；若索引也不可用，则至少发布 `trace_warning`。任何追踪失败都被收集器吞掉，不向 LLM 客户端或业务工作流抛出。
|
||
|
||
Token 用量只从完整响应的顶层 `usage` 读取为数字字典：
|
||
|
||
~~~ts
|
||
function tokenUsageFromResponse(response: unknown) {
|
||
if (!response || typeof response !== "object") return null;
|
||
const usage = (response as { usage?: unknown }).usage;
|
||
if (!usage || typeof usage !== "object") return null;
|
||
return Object.fromEntries(
|
||
Object.entries(usage).filter(
|
||
(entry): entry is [string, number] => typeof entry[1] === "number",
|
||
),
|
||
);
|
||
}
|
||
~~~
|
||
|
||
- [ ] **Step 4：实现工作流状态和清理**
|
||
|
||
`onWorkflowEvent()` 必须更新 `current_stage`；收到 `qa_ready` 时，把最近一个 `quality_inspector` 调用的 `business_status` 设置为 `qa_report.overall_status`。`finish()` 先写终态，再执行：
|
||
|
||
~~~ts
|
||
const expired = await repository.listTerminalRunsExcept(jobId);
|
||
for (const oldRun of expired) {
|
||
await payloadStore.deleteJob(oldRun.job_id);
|
||
await repository.deleteRun(oldRun.job_id);
|
||
}
|
||
~~~
|
||
|
||
`listTerminalRunsExcept()` 不返回任何 `running` 记录，因此并发运行任务不会被删除。
|
||
|
||
- [ ] **Step 5：运行收集器测试**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/trace-recorder.test.ts`
|
||
|
||
Expected: PASS，包含存储失败时业务回调不抛错、`trace_warning` 被发布的断言。
|
||
|
||
- [ ] **Step 6：提交追踪协调器**
|
||
|
||
~~~bash
|
||
git add src/lib/llm/trace-recorder.ts src/lib/llm/__tests__/trace-recorder.test.ts
|
||
git commit -m "新增LLM任务追踪收集器"
|
||
~~~
|
||
|
||
## Task 5：在 LLM 客户端捕获真实 SDK 请求和响应
|
||
|
||
**Files:**
|
||
- Modify: `src/lib/llm/client.ts`
|
||
- Modify: `src/lib/llm/__tests__/client.test.ts`
|
||
|
||
- [ ] **Step 1：写请求和响应深度相等的失败测试**
|
||
|
||
~~~ts
|
||
it("traces the exact SDK request and full SDK response", async () => {
|
||
process.env.LLM_PROVIDER = "deepseek";
|
||
process.env.DEEPSEEK_API_KEY = "test-key";
|
||
const traced: LlmClientTraceEvent[] = [];
|
||
let sdkRequest: unknown;
|
||
const sdkResponse = {
|
||
id: "chatcmpl_1",
|
||
object: "chat.completion",
|
||
created: 1784188800,
|
||
model: "deepseek-v4-pro",
|
||
choices: [{
|
||
index: 0,
|
||
message: { role: "assistant", content: "{\"value\":\"ok\"}" },
|
||
finish_reason: "stop",
|
||
}],
|
||
usage: { prompt_tokens: 10, completion_tokens: 4, total_tokens: 14 },
|
||
};
|
||
client.setChatCompletionForTesting(async (request) => {
|
||
sdkRequest = request;
|
||
return sdkResponse;
|
||
});
|
||
|
||
await client.generateValidatedJson({
|
||
schema: z.object({ value: z.string() }),
|
||
schemaName: "valueSchema",
|
||
prompt: "Return JSON.",
|
||
task: "article_optimizer",
|
||
traceStage: "draft",
|
||
onTraceEvent: (event) => traced.push(event),
|
||
});
|
||
|
||
expect(traced.find((event) => event.type === "started")).toMatchObject({
|
||
type: "started",
|
||
request: sdkRequest,
|
||
});
|
||
expect(traced.find((event) => event.type === "responded")).toMatchObject({
|
||
type: "responded",
|
||
response: sdkResponse,
|
||
});
|
||
expect(traced.map((event) => event.type)).toEqual([
|
||
"started",
|
||
"responded",
|
||
"validated",
|
||
]);
|
||
});
|
||
~~~
|
||
|
||
再增加 JSON 解析失败顺序和 Schema 失败顺序测试：
|
||
|
||
~~~ts
|
||
expect(jsonParseEvents.map((event) => event.type)).toEqual([
|
||
"started", "responded", "failed",
|
||
]);
|
||
expect(schemaEvents.map((event) => event.type)).toEqual([
|
||
"started", "responded", "validated", "failed",
|
||
]);
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行客户端测试并确认失败**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/client.test.ts`
|
||
|
||
Expected: FAIL，`schemaName`、`traceStage` 和 `onTraceEvent` 尚不存在，响应也会在解析后丢失。
|
||
|
||
- [ ] **Step 3：扩展 GenerateInput 并生成稳定调用 ID**
|
||
|
||
~~~ts
|
||
export interface GenerateInput {
|
||
system?: string;
|
||
prompt: string;
|
||
model?: string;
|
||
temperature?: number;
|
||
task?: LlmTaskName;
|
||
schemaName?: string;
|
||
traceStage?: LlmTraceWorkflowStage;
|
||
rewriteRound?: number;
|
||
onTraceEvent?: LlmClientTraceHandler;
|
||
onAuditSummary?: (summary: LlmAuditSummary) => void | Promise<void>;
|
||
traceCallId?: string;
|
||
}
|
||
|
||
function callIdFor(input: GenerateInput) {
|
||
return input.traceCallId ?? `llmcall_${nanoid(12)}`;
|
||
}
|
||
|
||
async function emitTrace(input: GenerateInput, event: LlmClientTraceEvent) {
|
||
try {
|
||
await input.onTraceEvent?.(event);
|
||
} catch (error) {
|
||
console.warn(`[llm:trace-warning] ${safeErrorSummary(error)}`);
|
||
}
|
||
}
|
||
~~~
|
||
|
||
`generateValidatedJson()` 在调用生成器前创建 `traceCallId`，并把同一个 ID 传给 `generateJsonForValidation()`，保证校验事件属于同一调用。
|
||
|
||
- [ ] **Step 4：拆开 Provider、JSON 解析和 Schema 校验阶段**
|
||
|
||
`generateJson()` 必须按以下顺序实现，不能把 `JSON.parse()` 放回 Provider `try/catch`：
|
||
|
||
~~~ts
|
||
const request = buildChatCompletionRequest(input, effectiveModel);
|
||
await emitTrace(input, {
|
||
type: "started",
|
||
call_id: callId,
|
||
task,
|
||
context: {
|
||
workflow_stage: input.traceStage ?? "unknown",
|
||
rewrite_round: input.rewriteRound,
|
||
schema_name: input.schemaName,
|
||
},
|
||
provider: status.provider,
|
||
model: effectiveModel,
|
||
request,
|
||
started_at: new Date(startedAt).toISOString(),
|
||
});
|
||
|
||
let response: ChatCompletionResult;
|
||
try {
|
||
response = chatCompletionForTesting
|
||
? await chatCompletionForTesting(request)
|
||
: await createClient().client.chat.completions.create(request);
|
||
} catch (error) {
|
||
await emitTrace(input, failedEvent(callId, "provider", error, startedAt));
|
||
throw normalizeLlmError(error);
|
||
}
|
||
|
||
await emitTrace(input, {
|
||
type: "responded",
|
||
call_id: callId,
|
||
response,
|
||
duration_ms: Date.now() - startedAt,
|
||
responded_at: new Date().toISOString(),
|
||
});
|
||
|
||
const content = response.choices[0]?.message.content ?? "{}";
|
||
try {
|
||
return JSON.parse(content) as T;
|
||
} catch (error) {
|
||
await emitTrace(input, failedEvent(callId, "json_parse", error, startedAt));
|
||
throw new Error(`LLM response is not valid JSON: ${safeErrorSummary(error)}`);
|
||
}
|
||
~~~
|
||
|
||
Schema 校验失败时先发送 `validated(schema_valid=false)`，再发送 `failed(error_type=schema_validation)`。安全错误只允许 `name` 和 `message`，不得序列化 SDK error、headers、request 或环境变量。
|
||
|
||
- [ ] **Step 5：保持现有审计和控制台日志兼容**
|
||
|
||
现有 `[llm:start]`、`[llm:response]`、`[llm:validated]`、`[llm:error]` 和 `onAuditSummary` 行为必须保留。现有原始返回控制台日志仍按 `LLM_LOG_RAW_LIMIT` 截断；R2 追踪正文不截断。
|
||
|
||
- [ ] **Step 6：运行客户端与审计测试**
|
||
|
||
Run: `npm test -- src/lib/llm/__tests__/client.test.ts src/lib/llm/__tests__/audit.test.ts`
|
||
|
||
Expected: PASS，且失败测试证明 API Key 不出现在任何事件的 JSON 中。
|
||
|
||
- [ ] **Step 7：提交客户端追踪**
|
||
|
||
~~~bash
|
||
git add src/lib/llm/client.ts src/lib/llm/__tests__/client.test.ts
|
||
git commit -m "记录真实LLM请求与响应"
|
||
~~~
|
||
|
||
## Task 6：把追踪器接入真实流式文章工作流
|
||
|
||
**Files:**
|
||
- Modify: `src/lib/workflow/fact-extractor.ts`
|
||
- Modify: `src/lib/workflow/article-optimizer.ts`
|
||
- Modify: `src/lib/workflow/quality-inspector.ts`
|
||
- Modify: `src/lib/workflow/targeted-rewriter.ts`
|
||
- Modify: `src/lib/workflow/streaming-optimizer.ts`
|
||
- Modify: `src/app/api/jobs/optimize-stream/route.ts`
|
||
- Modify: `src/app/api/__tests__/jobs.test.ts`
|
||
|
||
- [ ] **Step 1：写流式事件顺序的失败测试**
|
||
|
||
在 `jobs.test.ts` 中让 LLM mock 主动调用传入的追踪回调，并断言：
|
||
|
||
~~~ts
|
||
expect(events.map((event) => event.type)).toEqual([
|
||
"job_created",
|
||
"llm_call_started",
|
||
"llm_call_responded",
|
||
"llm_call_validated",
|
||
"fact_card_ready",
|
||
"draft_started",
|
||
"llm_call_started",
|
||
"llm_call_responded",
|
||
"llm_call_validated",
|
||
"draft_ready",
|
||
"qa_started",
|
||
"llm_call_started",
|
||
"llm_call_responded",
|
||
"llm_call_validated",
|
||
"qa_ready",
|
||
"final_ready",
|
||
]);
|
||
~~~
|
||
|
||
还要断言公开事件 JSON 不包含文章正文、`messages`、完整 `choices` 或访问密钥。
|
||
|
||
- [ ] **Step 2：运行 API 测试并确认缺少追踪事件**
|
||
|
||
Run: `npm test -- src/app/api/__tests__/jobs.test.ts`
|
||
|
||
Expected: FAIL，事件序列仍只有业务阶段事件。
|
||
|
||
- [ ] **Step 3：给四个工作流节点传递明确追踪上下文**
|
||
|
||
每个输入接口增加：
|
||
|
||
~~~ts
|
||
onTraceEvent?: GenerateInput["onTraceEvent"];
|
||
rewriteRound?: number;
|
||
~~~
|
||
|
||
四个 `generateValidatedJson()` 调用必须分别传入：
|
||
|
||
~~~ts
|
||
// fact-extractor.ts
|
||
schemaName: "candidateFactCardSchema",
|
||
traceStage: "fact_card",
|
||
onTraceEvent: options.onTraceEvent,
|
||
|
||
// article-optimizer.ts
|
||
schemaName: "optimizedArticleSchema",
|
||
traceStage: "draft",
|
||
onTraceEvent,
|
||
|
||
// quality-inspector.ts
|
||
schemaName: "llmQaPatchSchema",
|
||
traceStage: "qa",
|
||
rewriteRound: input.rewriteRound,
|
||
onTraceEvent: input.onTraceEvent,
|
||
|
||
// targeted-rewriter.ts
|
||
schemaName: "optimizedArticleSchema",
|
||
traceStage: "rewrite",
|
||
rewriteRound,
|
||
onTraceEvent,
|
||
~~~
|
||
|
||
`streaming-optimizer.ts` 初次 QA 使用 `rewriteRound: 0`；每轮修复和复检使用当前 `nextRound`。
|
||
|
||
- [ ] **Step 4：在流式路由创建并连接追踪器**
|
||
|
||
在 `ReadableStream.start()` 的业务 `try` 之前，与 `jobId`、`caseId`、`stage` 同级声明空追踪器，使失败 `catch` 始终能访问：
|
||
|
||
~~~ts
|
||
let jobId: string | undefined;
|
||
let caseId: string | undefined;
|
||
let stage: OptimizationStreamStage = "job";
|
||
let traceRecorder = createNoopLlmTraceRecorder();
|
||
~~~
|
||
|
||
任务和案例创建完成后，再尝试替换为真实追踪器：
|
||
|
||
~~~ts
|
||
try {
|
||
traceRecorder = await createLlmTraceRecorder({
|
||
jobId: job.id,
|
||
caseId: optimizationCase.id,
|
||
repository: getLlmTraceRepositoryFromRuntime(),
|
||
payloadStore: getLlmTracePayloadStoreFromRuntime(),
|
||
publish: async (event) => send(event),
|
||
});
|
||
} catch (error) {
|
||
send({
|
||
type: "trace_warning",
|
||
job_id: job.id,
|
||
trace_completeness: "incomplete",
|
||
error_summary: safeTraceError(error),
|
||
});
|
||
}
|
||
~~~
|
||
|
||
事实提取和 `runStreamingOptimizationWorkflow()` 都传 `onTraceEvent: traceRecorder.onLlmEvent`。保存事实卡后，把直接发送改为同一事件先入追踪器、再入 NDJSON：
|
||
|
||
~~~ts
|
||
const factCardReadyEvent: OptimizationStreamEvent = {
|
||
type: "fact_card_ready",
|
||
job_id: job.id,
|
||
fact_card: savedFactCard,
|
||
};
|
||
await traceRecorder.onWorkflowEvent(factCardReadyEvent);
|
||
send(factCardReadyEvent);
|
||
~~~
|
||
|
||
工作流的 `onEvent` 改为：
|
||
|
||
~~~ts
|
||
onEvent: async (event) => {
|
||
stage = stageForEvent(event, stage);
|
||
await traceRecorder.onWorkflowEvent(event);
|
||
send(event);
|
||
},
|
||
~~~
|
||
|
||
成功路径在 `final_ready` 前调用 `traceRecorder.finish({ status: "completed" })`；失败路径在 `failed` 前调用：
|
||
|
||
~~~ts
|
||
await traceRecorder.finish({
|
||
status: "failed",
|
||
errorStage: stage,
|
||
errorSummary: message,
|
||
});
|
||
~~~
|
||
|
||
`safeTraceError()` 从 `trace-recorder.ts` 导出，只返回安全化的 `name: message`。追踪器创建或存储异常必须被隔离成 `trace_warning`，不得跳入业务失败分支。
|
||
|
||
- [ ] **Step 5：运行路由、工作流和流事件测试**
|
||
|
||
Run: `npm test -- src/app/api/__tests__/jobs.test.ts src/lib/workflow/__tests__/streaming-optimizer.test.ts src/lib/workflow/__tests__/stream-events.test.ts`
|
||
|
||
Expected: PASS，现有业务事件和结果版本断言保持不变。
|
||
|
||
- [ ] **Step 6：提交真实工作流接入**
|
||
|
||
~~~bash
|
||
git add src/lib/workflow/fact-extractor.ts src/lib/workflow/article-optimizer.ts src/lib/workflow/quality-inspector.ts src/lib/workflow/targeted-rewriter.ts src/lib/workflow/streaming-optimizer.ts src/app/api/jobs/optimize-stream/route.ts src/app/api/__tests__/jobs.test.ts
|
||
git commit -m "接入文章优化LLM追踪"
|
||
~~~
|
||
|
||
## Task 7：增加受保护的追踪读取 API
|
||
|
||
**Files:**
|
||
- Create: `src/lib/llm/trace-http.ts`
|
||
- Create: `src/app/api/llm-traces/latest/route.ts`
|
||
- Create: `src/app/api/jobs/[jobId]/llm-trace/route.ts`
|
||
- Create: `src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts`
|
||
- Create: `src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts`
|
||
- Create: `src/app/api/__tests__/llm-traces.test.ts`
|
||
|
||
- [ ] **Step 1：先写鉴权、无缓存和正文测试**
|
||
|
||
~~~ts
|
||
it("rejects trace reads without API access", async () => {
|
||
const response = await getLatestTrace(request(null));
|
||
expect(response.status).toBe(401);
|
||
});
|
||
|
||
it("returns a no-store manifest without raw bodies", async () => {
|
||
const response = await getJobTrace(
|
||
request("test-key"),
|
||
params({ jobId: "job_1" }),
|
||
);
|
||
const body = await response.json();
|
||
expect(response.headers.get("cache-control")).toBe("no-store");
|
||
expect(body.calls[0]).toMatchObject({
|
||
call_id: "llmcall_1",
|
||
request_available: true,
|
||
response_available: true,
|
||
});
|
||
expect(JSON.stringify(body)).not.toContain("messages");
|
||
expect(JSON.stringify(body)).not.toContain("choices");
|
||
});
|
||
|
||
it("returns the exact stored request through the protected payload route", async () => {
|
||
const response = await getTraceRequest(
|
||
request("test-key"),
|
||
params({ jobId: "job_1", callId: "llmcall_1" }),
|
||
);
|
||
expect(response.headers.get("cache-control")).toBe("no-store");
|
||
await expect(response.json()).resolves.toEqual(exactRequestFixture);
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行 API 测试并确认路由不存在**
|
||
|
||
Run: `npm test -- src/app/api/__tests__/llm-traces.test.ts`
|
||
|
||
Expected: FAIL，读取路由尚不存在。
|
||
|
||
- [ ] **Step 3：实现共享 HTTP 助手**
|
||
|
||
~~~ts
|
||
export function noStoreJson(body: unknown, status = 200) {
|
||
return NextResponse.json(body, {
|
||
status,
|
||
headers: { "cache-control": "no-store" },
|
||
});
|
||
}
|
||
|
||
export async function readTracePayload({
|
||
jobId,
|
||
callId,
|
||
kind,
|
||
}: {
|
||
jobId: string;
|
||
callId: string;
|
||
kind: "request" | "response";
|
||
}) {
|
||
const repository = getLlmTraceRepositoryFromRuntime();
|
||
const call = (await repository.listCalls(jobId)).find(
|
||
(candidate) => candidate.call_id === callId,
|
||
);
|
||
if (!call) return noStoreJson({ error: "追踪调用不存在" }, 404);
|
||
const key = kind === "request"
|
||
? call.request_object_key
|
||
: call.response_object_key;
|
||
if (!key) {
|
||
if (kind === "response" && call.status === "started") {
|
||
return noStoreJson({ state: "waiting" }, 202);
|
||
}
|
||
return noStoreJson({
|
||
error: kind === "response" ? "该调用未产生响应" : "追踪请求正文不存在",
|
||
}, 404);
|
||
}
|
||
const payload = await getLlmTracePayloadStoreFromRuntime().getJson(key);
|
||
return payload == null
|
||
? noStoreJson({ error: "追踪正文不存在" }, 404)
|
||
: noStoreJson(payload);
|
||
}
|
||
~~~
|
||
|
||
- [ ] **Step 4：实现四个 GET 路由**
|
||
|
||
每个路由第一段必须完全一致：
|
||
|
||
~~~ts
|
||
const access = requireApiAccess(request);
|
||
if (!access.ok) return access.response;
|
||
~~~
|
||
|
||
`latest` 调用 `repository.getLatestRun()`；无记录时返回 `noStoreJson({ run: null, calls: [] })`。有记录时，`latest` 和任务清单路由都调用 `repository.listCalls(run.job_id)`，并通过同一个安全映射函数返回：
|
||
|
||
~~~ts
|
||
export function toPublicTraceCall(call: LlmTraceCall): LlmTraceCallPublic {
|
||
const {
|
||
request_object_key: requestObjectKey,
|
||
response_object_key: responseObjectKey,
|
||
...publicFields
|
||
} = call;
|
||
return {
|
||
...publicFields,
|
||
request_available: Boolean(requestObjectKey),
|
||
response_available: Boolean(responseObjectKey),
|
||
};
|
||
}
|
||
|
||
return noStoreJson({ run, calls: calls.map(toPublicTraceCall) });
|
||
~~~
|
||
|
||
`toPublicTraceCall()` 放在 `trace-http.ts`，保证内部 R2 对象键不会进入响应对象。
|
||
|
||
- [ ] **Step 5：运行追踪 API 和鉴权回归测试**
|
||
|
||
Run: `npm test -- src/app/api/__tests__/llm-traces.test.ts src/lib/api/__tests__/auth.test.ts`
|
||
|
||
Expected: PASS，401、404、202、200 和 `no-store` 均有断言。
|
||
|
||
- [ ] **Step 6：提交追踪读取 API**
|
||
|
||
~~~bash
|
||
git add src/lib/llm/trace-http.ts src/app/api/llm-traces/latest/route.ts 'src/app/api/jobs/[jobId]/llm-trace/route.ts' 'src/app/api/jobs/[jobId]/llm-trace/[callId]/request/route.ts' 'src/app/api/jobs/[jobId]/llm-trace/[callId]/response/route.ts' src/app/api/__tests__/llm-traces.test.ts
|
||
git commit -m "新增LLM追踪只读接口"
|
||
~~~
|
||
|
||
## Task 8：实现浏览器 API 客户端和架构状态派生
|
||
|
||
**Files:**
|
||
- Create: `src/components/architecture/api-client.ts`
|
||
- Create: `src/components/architecture/trace-state.ts`
|
||
- Create: `src/components/architecture/__tests__/api-client.test.ts`
|
||
- Create: `src/components/architecture/__tests__/trace-state.test.ts`
|
||
|
||
- [ ] **Step 1：先写按需加载和状态派生失败测试**
|
||
|
||
~~~ts
|
||
it("loads request bodies only when explicitly requested", async () => {
|
||
vi.stubGlobal("fetch", vi.fn(async () => jsonResponse(exactRequest)));
|
||
await expect(
|
||
getTracePayload("job_1", "llmcall_1", "request", "test-key"),
|
||
).resolves.toEqual(exactRequest);
|
||
expect(fetch).toHaveBeenCalledWith(
|
||
"/api/jobs/job_1/llm-trace/llmcall_1/request",
|
||
expect.objectContaining({
|
||
credentials: "same-origin",
|
||
headers: { "x-api-key": "test-key" },
|
||
cache: "no-store",
|
||
}),
|
||
);
|
||
});
|
||
|
||
it("distinguishes QA business failure from schema failure", () => {
|
||
const nodes = deriveArchitectureNodes(run, [
|
||
call({ task: "quality_inspector", schema_valid: true, business_status: "fail" }),
|
||
]);
|
||
expect(nodes.qa).toMatchObject({
|
||
status: "completed",
|
||
detail: "检查完成，需要修复",
|
||
});
|
||
|
||
const failedNodes = deriveArchitectureNodes(run, [
|
||
call({ task: "quality_inspector", schema_valid: false, status: "failed" }),
|
||
]);
|
||
expect(failedNodes.qa.status).toBe("failed");
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行测试并确认模块不存在**
|
||
|
||
Run: `npm test -- src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts`
|
||
|
||
Expected: FAIL。
|
||
|
||
- [ ] **Step 3：实现 API 客户端**
|
||
|
||
~~~ts
|
||
function traceHeaders(apiAccessKey: string) {
|
||
return apiAccessKey ? { "x-api-key": apiAccessKey } : {};
|
||
}
|
||
|
||
async function traceFetch<T>(path: string, apiAccessKey: string): Promise<T> {
|
||
const response = await fetch(path, {
|
||
credentials: "same-origin",
|
||
headers: traceHeaders(apiAccessKey),
|
||
cache: "no-store",
|
||
});
|
||
const body = (await response.json().catch(() => ({}))) as T & { error?: string };
|
||
if (!response.ok) throw new Error(body.error ?? "读取后台追踪失败");
|
||
return body;
|
||
}
|
||
|
||
export function getLatestTrace(apiAccessKey: string) {
|
||
return traceFetch<LlmTraceManifest | { run: null; calls: [] }>(
|
||
"/api/llm-traces/latest",
|
||
apiAccessKey,
|
||
);
|
||
}
|
||
|
||
export function getJobTrace(jobId: string, apiAccessKey: string) {
|
||
return traceFetch<LlmTraceManifest>(
|
||
`/api/jobs/${encodeURIComponent(jobId)}/llm-trace`,
|
||
apiAccessKey,
|
||
);
|
||
}
|
||
|
||
export function getTracePayload(
|
||
jobId: string,
|
||
callId: string,
|
||
kind: "request" | "response",
|
||
apiAccessKey: string,
|
||
) {
|
||
return traceFetch<unknown>(
|
||
`/api/jobs/${encodeURIComponent(jobId)}/llm-trace/${encodeURIComponent(callId)}/${kind}`,
|
||
apiAccessKey,
|
||
);
|
||
}
|
||
~~~
|
||
|
||
- [ ] **Step 4：实现实时事件归并和固定节点派生**
|
||
|
||
`trace-state.ts` 导出：
|
||
|
||
~~~ts
|
||
export type ArchitectureNodeStatus =
|
||
| "waiting"
|
||
| "running"
|
||
| "completed"
|
||
| "failed";
|
||
|
||
export interface ArchitectureNodeView {
|
||
id: "input" | "fact_card" | "draft" | "qa" | "rewrite" | "final";
|
||
label: string;
|
||
status: ArchitectureNodeStatus;
|
||
detail: string;
|
||
}
|
||
|
||
export function applyLiveTraceEvent(
|
||
state: LlmTraceManifest,
|
||
event: LlmTraceStreamEvent,
|
||
): LlmTraceManifest;
|
||
|
||
export function deriveArchitectureNodes(
|
||
run: LlmTraceRun | null,
|
||
calls: LlmTraceCallPublic[],
|
||
): Record<ArchitectureNodeView["id"], ArchitectureNodeView>;
|
||
~~~
|
||
|
||
归并必须按 `call_id` 更新而不是追加重复调用；调用清单始终按 `sequence` 排序。`trace_warning` 只把运行记录改为 `incomplete`，不把业务状态改成 `failed`。
|
||
|
||
- [ ] **Step 5：运行前端纯逻辑测试**
|
||
|
||
Run: `npm test -- src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts`
|
||
|
||
Expected: PASS，覆盖重复事件、复检轮次、Provider 失败、Schema 失败和业务 QA 失败。
|
||
|
||
- [ ] **Step 6：提交前端数据层**
|
||
|
||
~~~bash
|
||
git add src/components/architecture/api-client.ts src/components/architecture/trace-state.ts src/components/architecture/__tests__/api-client.test.ts src/components/architecture/__tests__/trace-state.test.ts
|
||
git commit -m "新增后台架构前端状态层"
|
||
~~~
|
||
|
||
## Task 9：构建固定架构图和完整日志详情组件
|
||
|
||
**Files:**
|
||
- Create: `src/components/architecture/architecture-flow.tsx`
|
||
- Create: `src/components/architecture/llm-call-detail.tsx`
|
||
- Create: `src/components/architecture/architecture-observer-panel.tsx`
|
||
- Modify: `src/app/globals.css`
|
||
- Test: `src/components/architecture/__tests__/trace-state.test.ts`
|
||
|
||
- [ ] **Step 1：先写界面格式化失败测试**
|
||
|
||
把界面格式化保持为纯函数并测试：
|
||
|
||
~~~ts
|
||
it("formats every architecture node and call label in Chinese", () => {
|
||
expect(formatNodeStatus("running")).toBe("运行中");
|
||
expect(formatNodeStatus("failed")).toBe("失败");
|
||
expect(formatCallLabel(call({
|
||
sequence: 4,
|
||
task: "targeted_rewriter",
|
||
rewrite_round: 1,
|
||
duration_ms: 22600,
|
||
}))).toBe("4 · 定向修复第 1 轮 · 22.6 秒");
|
||
});
|
||
~~~
|
||
|
||
- [ ] **Step 2：运行测试并确认格式化函数不存在**
|
||
|
||
Run: `npm test -- src/components/architecture/__tests__/trace-state.test.ts`
|
||
|
||
Expected: FAIL。
|
||
|
||
- [ ] **Step 3：实现固定拓扑组件**
|
||
|
||
`architecture-flow.tsx` 只接受派生后的节点，不读取 API：
|
||
|
||
~~~tsx
|
||
export function ArchitectureFlow({
|
||
nodes,
|
||
}: {
|
||
nodes: Record<ArchitectureNodeView["id"], ArchitectureNodeView>;
|
||
}) {
|
||
const order: ArchitectureNodeView["id"][] = [
|
||
"input", "fact_card", "draft", "qa", "rewrite", "final",
|
||
];
|
||
return (
|
||
<section className="architecture-flow" aria-label="文章优化后台架构">
|
||
{order.map((id) => {
|
||
const node = nodes[id];
|
||
return (
|
||
<article
|
||
className={`architecture-node architecture-node-${node.status}`}
|
||
data-node={node.id}
|
||
key={node.id}
|
||
>
|
||
<strong>{node.label}</strong>
|
||
<span>{formatNodeStatus(node.status)}</span>
|
||
<small>{node.detail}</small>
|
||
</article>
|
||
);
|
||
})}
|
||
<p className="architecture-loop-label">质量检查未通过时，定向修复后返回复检，最多两轮。</p>
|
||
</section>
|
||
);
|
||
}
|
||
~~~
|
||
|
||
固定节点不能按调用次数新增；复检只改变 QA 节点并在调用轨迹中增加一条调用。
|
||
|
||
- [ ] **Step 4：实现调用详情按需读取**
|
||
|
||
`llm-call-detail.tsx` 接受 `jobId`、`call`、`apiAccessKey` 和 `technicalDetailsEnabled`。只有技术详情开启且用户选择 `请求` 或 `响应` 时才调用 `getTracePayload()`：
|
||
|
||
~~~tsx
|
||
const [tab, setTab] = useState<"request" | "response" | "validation">(
|
||
"validation",
|
||
);
|
||
const [payload, setPayload] = useState<unknown>(null);
|
||
|
||
useEffect(() => {
|
||
if (!technicalDetailsEnabled || tab === "validation") return;
|
||
let current = true;
|
||
setPayload(null);
|
||
getTracePayload(jobId, call.call_id, tab, apiAccessKey)
|
||
.then((value) => current && setPayload(value))
|
||
.catch((error) => current && setPayload({ error: errorMessage(error) }));
|
||
return () => { current = false; };
|
||
}, [apiAccessKey, call.call_id, jobId, tab, technicalDetailsEnabled]);
|
||
|
||
return (
|
||
<section className="panel llm-call-detail" aria-live="polite">
|
||
<div className="llm-detail-tabs" role="tablist" aria-label="LLM 调用详情">
|
||
{(["request", "response", "validation"] as const).map((value) => (
|
||
<button
|
||
aria-selected={tab === value}
|
||
className={tab === value ? "active-tab" : undefined}
|
||
key={value}
|
||
onClick={() => setTab(value)}
|
||
role="tab"
|
||
type="button"
|
||
>
|
||
{detailTabLabel(value)}
|
||
</button>
|
||
))}
|
||
</div>
|
||
<pre className="llm-json-view">
|
||
{JSON.stringify(
|
||
tab === "validation" ? validationView(call) : payload,
|
||
null,
|
||
2,
|
||
)}
|
||
</pre>
|
||
</section>
|
||
);
|
||
~~~
|
||
|
||
必须使用 `<pre>` 文本渲染，不使用 `dangerouslySetInnerHTML`、Markdown HTML 或代码高亮器。
|
||
|
||
- [ ] **Step 5：实现只读观察容器**
|
||
|
||
`architecture-observer-panel.tsx` 的 props 固定为：
|
||
|
||
~~~ts
|
||
interface ArchitectureObserverPanelProps {
|
||
apiAccessKey: string;
|
||
currentJobId: string | null;
|
||
liveEvents: LlmTraceStreamEvent[];
|
||
}
|
||
~~~
|
||
|
||
挂载时优先 `getJobTrace(currentJobId)`，没有当前任务时 `getLatestTrace()`；之后用 `applyLiveTraceEvent()` 合并 `liveEvents`。页面必须有：
|
||
|
||
- “当前任务”或“最近一次任务”明确标签。
|
||
- Provider、模型、运行状态和追踪完整性。
|
||
- `ArchitectureFlow`。
|
||
- 按 `sequence` 排列的调用按钮。
|
||
- 默认关闭的“技术详情”开关。
|
||
- 选中调用的 `LlmCallDetail`。
|
||
- 无记录、401、加载失败和 `incomplete` 状态。
|
||
|
||
组件不得出现开始、重试、取消、删除或编辑按钮。
|
||
|
||
- [ ] **Step 6：增加桌面和窄屏 CSS**
|
||
|
||
新增类：`.architecture-observer`、`.architecture-task-strip`、`.architecture-flow`、`.architecture-node-*`、`.architecture-workspace`、`.llm-call-list`、`.llm-call-detail`、`.llm-detail-tabs`、`.llm-json-view`。桌面 `.architecture-workspace` 使用左侧调用轨迹、右侧详情；在现有 `@media (max-width: 900px)` 中改成单列。JSON 使用 `white-space: pre-wrap` 和 `overflow-wrap: anywhere`，不得产生页面级横向滚动。
|
||
|
||
- [ ] **Step 7：运行纯逻辑测试和 lint**
|
||
|
||
Run: `npm test -- src/components/architecture/__tests__/trace-state.test.ts && npm run lint`
|
||
|
||
Expected: PASS；ESLint 不出现 effect 同步 set-state 或未转义内容错误。
|
||
|
||
- [ ] **Step 8：提交观测组件**
|
||
|
||
~~~bash
|
||
git add src/components/architecture/architecture-flow.tsx src/components/architecture/llm-call-detail.tsx src/components/architecture/architecture-observer-panel.tsx src/components/architecture/__tests__/trace-state.test.ts src/app/globals.css
|
||
git commit -m "新增LLM后台架构观测界面"
|
||
~~~
|
||
|
||
## Task 10：接入第三标签并完成浏览器验收
|
||
|
||
**Files:**
|
||
- Modify: `src/app/page.tsx`
|
||
- Create: `tests/e2e/architecture-observer.spec.ts`
|
||
- Modify: `tests/e2e/mvp.spec.ts`
|
||
- Modify: `tests/e2e/sample-flow/page-flow.ts`
|
||
|
||
- [ ] **Step 1：写第三标签和切换不中断的 E2E 失败测试**
|
||
|
||
`architecture-observer.spec.ts` 使用 `page.route()` 模拟分块 NDJSON 和追踪正文接口：
|
||
|
||
~~~ts
|
||
test("后台架构标签跟随当前任务并按需展示真实请求响应", async ({ page }) => {
|
||
await mockOptimizationStream(page);
|
||
await mockTracePayloads(page);
|
||
await page.goto("/");
|
||
await page.getByLabel("访问密钥").fill("local-dev-key");
|
||
await page.getByLabel("文章内容").fill("示例文章正文");
|
||
await page.getByRole("button", { name: "开始优化" }).click();
|
||
|
||
await page.getByRole("button", { name: "后台架构" }).click();
|
||
await expect(page.getByText("当前任务")).toBeVisible();
|
||
await expect(page.getByText("article_optimizer")).toBeVisible();
|
||
|
||
await page.getByRole("button", { name: /article_optimizer/ }).click();
|
||
await page.getByRole("checkbox", { name: "技术详情" }).check();
|
||
await page.getByRole("tab", { name: "请求" }).click();
|
||
await expect(page.locator(".llm-json-view")).toContainText("messages");
|
||
await page.getByRole("tab", { name: "响应" }).click();
|
||
await expect(page.locator(".llm-json-view")).toContainText("choices");
|
||
await expect(page.getByText("终稿已生成")).toBeVisible();
|
||
});
|
||
~~~
|
||
|
||
Mock 路由记录请求次数，并断言关闭技术详情时正文接口调用次数为 0。
|
||
|
||
- [ ] **Step 2：运行 E2E 并确认第三标签不存在**
|
||
|
||
Run: `npx playwright test tests/e2e/architecture-observer.spec.ts --project=chromium`
|
||
|
||
Expected: FAIL，找不到 `后台架构` 按钮。
|
||
|
||
- [ ] **Step 3：在首页接入追踪事件状态**
|
||
|
||
`page.tsx` 修改为：
|
||
|
||
~~~tsx
|
||
const [activeTab, setActiveTab] = useState<"geo" | "copy" | "architecture">("geo");
|
||
const [llmTraceEvents, setLlmTraceEvents] = useState<LlmTraceStreamEvent[]>([]);
|
||
|
||
function handleStreamEvent(event: OptimizationStreamEvent) {
|
||
switch (event.type) {
|
||
case "llm_call_started":
|
||
case "llm_call_responded":
|
||
case "llm_call_validated":
|
||
case "llm_call_failed":
|
||
case "trace_warning":
|
||
setLlmTraceEvents((current) => [...current, event]);
|
||
return;
|
||
case "job_created":
|
||
setJobId(event.job.id);
|
||
setLlmTraceEvents([]);
|
||
setStreamActivity("任务已创建");
|
||
return;
|
||
case "fact_card_ready":
|
||
setFactCard(event.fact_card);
|
||
setStreamActivity("事实卡已就绪");
|
||
return;
|
||
case "draft_started":
|
||
setStreamActivity(event.message);
|
||
return;
|
||
case "draft_ready":
|
||
setDraftArticle(event.article);
|
||
setStreamActivity("草稿已生成");
|
||
return;
|
||
case "qa_started":
|
||
setStreamActivity(event.message);
|
||
return;
|
||
case "qa_ready":
|
||
setQaReport(event.qa_report);
|
||
setStreamActivity("质量检查完成");
|
||
return;
|
||
case "rewrite_started":
|
||
setStreamActivity(`正在修复第 ${event.round} 轮`);
|
||
return;
|
||
case "rewrite_ready":
|
||
setDraftArticle(event.article);
|
||
setStreamActivity(`第 ${event.round} 轮修复完成`);
|
||
return;
|
||
case "final_ready":
|
||
setOptimizedArticle(event.optimized_article);
|
||
setDraftArticle(null);
|
||
setQaReport(event.qa_report);
|
||
setStreamActivity("终稿已生成");
|
||
setMessage(
|
||
event.qa_report.overall_status === "fail"
|
||
? "优化完成，质检发现需要复核的问题。"
|
||
: "优化完成。",
|
||
);
|
||
return;
|
||
case "failed":
|
||
setStreamActivity("优化失败");
|
||
throw new Error(event.error);
|
||
}
|
||
}
|
||
~~~
|
||
|
||
第三个标签按钮文字固定为 `后台架构`。内容选择使用三个明确分支，不能把 architecture 落入普通文案分支：
|
||
|
||
~~~tsx
|
||
{activeTab === "geo" ? (
|
||
<>
|
||
<ProgressPanel
|
||
action={busyAction}
|
||
elapsedSeconds={elapsedSeconds}
|
||
lastTiming={lastTiming}
|
||
liveProgress={null}
|
||
/>
|
||
<div className="workflow-grid">
|
||
<ArticleInputForm
|
||
isSubmitting={busyAction === "optimize"}
|
||
value={input}
|
||
onChange={setInput}
|
||
onSubmit={optimize}
|
||
/>
|
||
<FactCardEditor
|
||
factCard={factCard}
|
||
isSaving={busyAction === "optimize"}
|
||
onChange={setFactCard}
|
||
/>
|
||
<OptimizedPreview
|
||
activityText={streamActivity}
|
||
article={optimizedArticle}
|
||
draftArticle={draftArticle}
|
||
isStreaming={busyAction === "optimize"}
|
||
jobId={jobId}
|
||
/>
|
||
<QaReportPanel report={qaReport} />
|
||
<PerformanceCalibrationPanel
|
||
apiAccessKey={apiAccessKey}
|
||
jobId={jobId}
|
||
key={`${jobId ?? "no-job"}-${optimizedArticle?.revision ?? "no-revision"}`}
|
||
optimizedRevision={optimizedArticle?.revision ?? null}
|
||
/>
|
||
</div>
|
||
</>
|
||
) : activeTab === "copy" ? (
|
||
<RenweiCopyOptimizerPanel apiAccessKey={apiAccessKey} />
|
||
) : (
|
||
<ArchitectureObserverPanel
|
||
apiAccessKey={apiAccessKey}
|
||
currentJobId={jobId}
|
||
liveEvents={llmTraceEvents}
|
||
/>
|
||
)}
|
||
~~~
|
||
|
||
在导航中紧跟“普通文案优化”增加 `后台架构` 按钮，并导入 `ArchitectureObserverPanel` 与 `LlmTraceStreamEvent`。现有 GEO JSX 原位保留。
|
||
|
||
- [ ] **Step 4：证明切换标签不取消流读取**
|
||
|
||
不要把 `readOptimizationStream()` 移进标签组件，也不要在切换标签时 abort fetch。E2E mock 在点击 `后台架构` 后继续发送 `qa_ready` 和 `final_ready`，测试必须看到最终状态。
|
||
|
||
- [ ] **Step 5：更新旧 MVP 回归为当前一键优化流程**
|
||
|
||
把 `tests/e2e/mvp.spec.ts` 中已不存在的“标题”“正文”“分析文章”“采纳为核心事实”“确认事实卡”步骤删除，改为填写 `文章内容`、`图片描述或图片链接`、`用户要求`，然后直接点击一次 `开始优化`；继续断言 `optimized.md`、质量报告、`optimized.docx` 和 `qa_report.json`。这只更新测试以匹配当前产品，不把产品改回旧流程。
|
||
|
||
- [ ] **Step 6：扩展真实样例验证但不落盘正文**
|
||
|
||
在 `tests/e2e/sample-flow/page-flow.ts` 增加：
|
||
|
||
~~~ts
|
||
async function validateLlmTrace({ request, baseURL, apiAccessKey, jobId }: {
|
||
request: APIRequestContext;
|
||
baseURL: string;
|
||
apiAccessKey: string;
|
||
jobId: string;
|
||
}) {
|
||
const headers = { "x-api-key": apiAccessKey };
|
||
const manifestResponse = await request.get(
|
||
`${baseURL}/api/jobs/${jobId}/llm-trace`,
|
||
{ headers },
|
||
);
|
||
expect(manifestResponse.ok()).toBe(true);
|
||
const manifest = await manifestResponse.json() as LlmTraceManifest;
|
||
expect(manifest.calls.length).toBeGreaterThan(0);
|
||
|
||
for (const call of manifest.calls) {
|
||
const requestResponse = await request.get(
|
||
`${baseURL}/api/jobs/${jobId}/llm-trace/${call.call_id}/request`,
|
||
{ headers },
|
||
);
|
||
expect(requestResponse.ok()).toBe(true);
|
||
const requestBody = await requestResponse.json() as { messages?: unknown[] };
|
||
expect(Array.isArray(requestBody.messages)).toBe(true);
|
||
|
||
if (call.response_available) {
|
||
const responseResponse = await request.get(
|
||
`${baseURL}/api/jobs/${jobId}/llm-trace/${call.call_id}/response`,
|
||
{ headers },
|
||
);
|
||
expect(responseResponse.ok()).toBe(true);
|
||
const responseBody = await responseResponse.json() as { choices?: unknown[] };
|
||
expect(Array.isArray(responseBody.choices)).toBe(true);
|
||
}
|
||
}
|
||
|
||
return manifest.calls.map((call) => call.task);
|
||
}
|
||
~~~
|
||
|
||
把返回值写入现有 `SampleResult.llm_tasks`。不得把 request/response 正文写入 `test-results`、截图注释或 summary JSON。
|
||
|
||
- [ ] **Step 7：运行确定性 E2E 和相关单元测试**
|
||
|
||
Run: `npx playwright test tests/e2e/architecture-observer.spec.ts tests/e2e/mvp.spec.ts --project=chromium`
|
||
|
||
Run: `npm test -- src/components/architecture src/lib/workflow/__tests__/stream-events.test.ts`
|
||
|
||
Expected: 全部 PASS，无控制台错误和横向溢出。
|
||
|
||
- [ ] **Step 8：提交首页和端到端接入**
|
||
|
||
~~~bash
|
||
git add src/app/page.tsx tests/e2e/architecture-observer.spec.ts tests/e2e/mvp.spec.ts tests/e2e/sample-flow/page-flow.ts
|
||
git commit -m "接入后台架构标签页"
|
||
~~~
|
||
|
||
## Task 11：迁移、全量验证和真实 Provider 验收
|
||
|
||
**Files:**
|
||
- Verify only: all files from Tasks 1-10
|
||
|
||
- [ ] **Step 1：应用本地 D1 迁移并核对表**
|
||
|
||
Run: `npm run d1:migrate:local`
|
||
|
||
Expected: `0004_llm_trace_observability.sql` applied successfully；再次运行显示没有待执行迁移。
|
||
|
||
- [ ] **Step 2：运行完整质量门禁**
|
||
|
||
Run: `npm run lint`
|
||
|
||
Expected: PASS，0 errors。
|
||
|
||
Run: `npm test`
|
||
|
||
Expected: PASS，包含新增 trace repository、payload store、recorder、client、API 和 UI 状态测试。
|
||
|
||
Run: `npm run build`
|
||
|
||
Expected: PASS，Next.js 生产构建成功。
|
||
|
||
- [ ] **Step 3：运行完整 Playwright 回归**
|
||
|
||
Run: `npx playwright test --project=chromium`
|
||
|
||
Expected: PASS；Task 10 已把 `mvp.spec.ts` 更新为当前一键优化流程。
|
||
|
||
- [ ] **Step 4：运行一次真实 DeepSeek 安全样例**
|
||
|
||
确保 `.env.local` 只在本地包含 `API_ACCESS_KEY` 和 `DEEPSEEK_API_KEY`，然后运行：
|
||
|
||
Run: `npm run test:e2e:samples -- --limit 1 --timeout-ms 600000`
|
||
|
||
Expected: PASS，并在生成的 `test-results/geo-sample-flow/<timestamp>/summary.md` 中看到非空 `llm_tasks`。报告不得包含完整请求、完整响应、访问密钥或 Provider 密钥。
|
||
|
||
- [ ] **Step 5：检查敏感信息和非预期文件**
|
||
|
||
Run:
|
||
|
||
~~~bash
|
||
git status --short --ignored=matching
|
||
rg -n "auth\.token|secretKey|healthsource|Authorization|DEEPSEEK_API_KEY" . --glob '!node_modules/**' --glob '!.next/**' --glob '!.open-next/**' --glob '!deploy/*.toml' --glob '!test-results/**'
|
||
~~~
|
||
|
||
Expected: 没有新提交的真实密钥；`data/`、`test-results/`、追踪正文和生成构建目录保持忽略。用户原有未提交文件仍保持原状。
|
||
|
||
- [ ] **Step 6：核对提交边界和最终状态**
|
||
|
||
Run: `git status --short && git log --oneline -12`
|
||
|
||
Expected: Tasks 1-10 的功能文件均已包含在对应中文提交中；只剩用户原有的 `README.md`、`wrangler.jsonc`、样例或导出目录等未提交改动，没有本功能遗漏文件。若质量门禁暴露功能缺陷，返回拥有该文件的任务按“失败测试—最小修正—验证—中文提交”闭环处理，不创建内容不明的兜底提交。
|
||
|
||
## 完成定义
|
||
|
||
- 所有任务按 TDD 顺序完成，每个任务都有独立中文提交。
|
||
- D1/SQLite 索引和 R2/本地正文存储均有测试。
|
||
- SDK 请求对象和保存请求深度相等，SDK 响应对象和保存响应深度相等。
|
||
- 公开 NDJSON 和清单 API 不含完整正文。
|
||
- 完整正文只能通过受保护且 `no-store` 的按需接口读取。
|
||
- 架构图由真实事件驱动，QA 业务失败与技术失败分开表达。
|
||
- 切换标签不取消任务，刷新可恢复已持久化状态。
|
||
- 只保留运行中任务和最近终态任务的完整正文。
|
||
- `npm run lint`、`npm test`、`npm run build`、Playwright 回归和一次真实安全样例全部通过。
|