2406 lines
70 KiB
Markdown
2406 lines
70 KiB
Markdown
# Publication Performance Calibration Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Build the first publication performance calibration loop: score optimized revisions, register publications, manually record performance snapshots, and generate calibration observations while keeping a future adapter boundary.
|
||
|
||
**Architecture:** Add a focused calibration domain beside the existing optimization workflow. Persist calibration data through the existing repository abstraction so local SQLite and Cloudflare D1 stay aligned. The first implementation uses a manual performance adapter only; future platform adapters will normalize into the same `PerformanceSnapshot` shape.
|
||
|
||
**Tech Stack:** Next.js 16 App Router, React 19, TypeScript, Zod 4, Vitest, better-sqlite3, Cloudflare D1, existing API key guard.
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
- Create `src/lib/calibration/types.ts`: domain types for rubric versions, scoring runs, publication records, sparse performance snapshots, adapter input/output, and calibration events.
|
||
- Create `src/lib/calibration/validation.ts`: Zod schemas for API inputs and stored calibration objects.
|
||
- Create `src/lib/calibration/scoring.ts`: deterministic v1 GEO scoring rubric and calibration event generation.
|
||
- Create `src/lib/calibration/manual-adapter.ts`: first `PerformanceAdapter` implementation that normalizes user-entered metrics.
|
||
- Create `src/lib/calibration/__tests__/validation.test.ts`: validation tests for sparse metrics and adapter-safe snapshots.
|
||
- Create `src/lib/calibration/__tests__/scoring.test.ts`: scoring and calibration-event tests.
|
||
- Modify `src/lib/db/schema.ts`: add local SQLite tables.
|
||
- Modify `migrations/0002_publication_performance_calibration.sql`: add D1 migration with the same tables.
|
||
- Modify `src/lib/db/repositories.ts`: add row types and SQLite helpers.
|
||
- Modify `src/lib/db/repository.ts`: extend `AppRepository` with calibration methods.
|
||
- Modify `src/lib/db/sqlite-repository.ts`: expose calibration methods.
|
||
- Modify `src/lib/db/d1-repository.ts`: expose calibration methods for Cloudflare.
|
||
- Modify `src/lib/db/__tests__/repository.test.ts`: test local persistence round trip.
|
||
- Modify `src/lib/db/__tests__/d1-repository.test.ts`: test D1 SQL/binds for one representative write.
|
||
- Create `src/app/api/jobs/[jobId]/calibration/score/route.ts`: create a scoring run for latest optimized revision.
|
||
- Create `src/app/api/jobs/[jobId]/publications/route.ts`: create and list publication records for a job.
|
||
- Create `src/app/api/publications/[publicationId]/performance/route.ts`: record a manual performance snapshot and calibration event.
|
||
- Modify `src/app/api/__tests__/jobs.test.ts`: route-level coverage for score, publication, and performance endpoints.
|
||
- Create `src/components/performance-calibration-panel.tsx`: compact UI for score, publication registration, and manual snapshot entry.
|
||
- Modify `src/app/page.tsx`: wire the calibration panel after optimization.
|
||
- Modify `src/app/globals.css`: add small calibration form/result styles.
|
||
|
||
## Task 1: Calibration Domain Types And Validation
|
||
|
||
**Files:**
|
||
- Create: `src/lib/calibration/types.ts`
|
||
- Create: `src/lib/calibration/validation.ts`
|
||
- Create: `src/lib/calibration/__tests__/validation.test.ts`
|
||
|
||
- [ ] **Step 1: Write failing validation tests**
|
||
|
||
Create `src/lib/calibration/__tests__/validation.test.ts`:
|
||
|
||
```ts
|
||
import { describe, expect, it } from "vitest";
|
||
|
||
import {
|
||
manualPerformanceInputSchema,
|
||
publicationInputSchema,
|
||
performanceSnapshotSchema,
|
||
} from "../validation";
|
||
|
||
describe("calibration validation", () => {
|
||
it("accepts a publication URL and normalizes optional notes", () => {
|
||
const parsed = publicationInputSchema.parse({
|
||
platform: "official_site",
|
||
url: "https://example.com/articles/geo",
|
||
published_at: "2026-06-24T12:00:00.000Z",
|
||
notes: " 官网首发 ",
|
||
});
|
||
|
||
expect(parsed.platform).toBe("official_site");
|
||
expect(parsed.notes).toBe("官网首发");
|
||
});
|
||
|
||
it("keeps sparse manual metrics absent instead of turning them into zeroes", () => {
|
||
const parsed = manualPerformanceInputSchema.parse({
|
||
window_label: "T+7d",
|
||
views: "1200",
|
||
clicks: "",
|
||
inquiries: undefined,
|
||
comments: "3",
|
||
feedback_summary: "用户追问案例依据",
|
||
});
|
||
|
||
expect(parsed.metrics).toEqual({
|
||
views: 1200,
|
||
comments: 3,
|
||
});
|
||
expect(parsed.feedback_summary).toBe("用户追问案例依据");
|
||
});
|
||
|
||
it("rejects unsafe raw adapter credentials in snapshots", () => {
|
||
expect(() =>
|
||
performanceSnapshotSchema.parse({
|
||
id: "perf_test",
|
||
publication_id: "pub_test",
|
||
source: "manual",
|
||
window_label: "T+3d",
|
||
metrics: { views: 10 },
|
||
feedback_summary: "",
|
||
raw_reference: "cookie=sessionid=secret",
|
||
snapshot_at: "2026-06-24T12:00:00.000Z",
|
||
}),
|
||
).toThrow(/raw_reference/);
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run the failing validation tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/calibration/__tests__/validation.test.ts
|
||
```
|
||
|
||
Expected: FAIL because `src/lib/calibration/validation.ts` does not exist.
|
||
|
||
- [ ] **Step 3: Add calibration domain types**
|
||
|
||
Create `src/lib/calibration/types.ts`:
|
||
|
||
```ts
|
||
import type { PublishPlatform, QaReport } from "../domain/types";
|
||
|
||
export type CalibrationDirection =
|
||
| "better_than_expected"
|
||
| "as_expected"
|
||
| "worse_than_expected"
|
||
| "needs_more_data";
|
||
|
||
export type PerformanceSource =
|
||
| "manual"
|
||
| `adapter:${string}`;
|
||
|
||
export interface RubricDimension {
|
||
id: string;
|
||
label: string;
|
||
weight: number;
|
||
description: string;
|
||
}
|
||
|
||
export interface RubricVersion {
|
||
id: string;
|
||
version: string;
|
||
name: string;
|
||
dimensions: RubricDimension[];
|
||
formula: "weighted_average_0_to_10";
|
||
is_active: boolean;
|
||
created_at: string;
|
||
}
|
||
|
||
export interface ScoringRun {
|
||
id: string;
|
||
job_id: string;
|
||
revision: number;
|
||
rubric_version_id: string;
|
||
dimension_scores: Record<string, number>;
|
||
composite_score: number;
|
||
rationale: string;
|
||
created_at: string;
|
||
}
|
||
|
||
export interface PublicationRecord {
|
||
id: string;
|
||
job_id: string;
|
||
revision: number;
|
||
platform: PublishPlatform;
|
||
url: string;
|
||
published_at: string;
|
||
status: "draft" | "published" | "archived";
|
||
notes: string;
|
||
created_at: string;
|
||
updated_at: string;
|
||
}
|
||
|
||
export interface PerformanceMetrics {
|
||
views?: number;
|
||
impressions?: number;
|
||
clicks?: number;
|
||
inquiries?: number;
|
||
likes?: number;
|
||
comments?: number;
|
||
shares?: number;
|
||
saves?: number;
|
||
average_position?: number;
|
||
}
|
||
|
||
export interface PerformanceSnapshot {
|
||
id: string;
|
||
publication_id: string;
|
||
source: PerformanceSource;
|
||
window_label: string;
|
||
metrics: PerformanceMetrics;
|
||
feedback_summary: string;
|
||
raw_reference?: string;
|
||
snapshot_at: string;
|
||
}
|
||
|
||
export interface CalibrationEvent {
|
||
id: string;
|
||
publication_id: string;
|
||
scoring_run_id: string;
|
||
performance_snapshot_id: string;
|
||
direction: CalibrationDirection;
|
||
observations: string[];
|
||
recommended_action: string;
|
||
created_at: string;
|
||
}
|
||
|
||
export interface AdapterFetchInput {
|
||
publication: PublicationRecord;
|
||
window_label: string;
|
||
}
|
||
|
||
export interface PerformanceAdapter {
|
||
source: PerformanceSource;
|
||
fetch(input: AdapterFetchInput): Promise<PerformanceSnapshot>;
|
||
}
|
||
|
||
export interface CalibrationContext {
|
||
scoringRun: ScoringRun;
|
||
qaReport: QaReport;
|
||
snapshot: PerformanceSnapshot;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Add validation schemas**
|
||
|
||
Create `src/lib/calibration/validation.ts`:
|
||
|
||
```ts
|
||
import { z } from "zod";
|
||
|
||
import { publishPlatformSchema } from "../domain/validation";
|
||
import type {
|
||
CalibrationDirection,
|
||
CalibrationEvent,
|
||
PerformanceMetrics,
|
||
PerformanceSnapshot,
|
||
PublicationRecord,
|
||
RubricVersion,
|
||
ScoringRun,
|
||
} from "./types";
|
||
|
||
const optionalTextSchema = z
|
||
.preprocess((value) => (value == null ? "" : value), z.string())
|
||
.transform((value) => value.trim());
|
||
|
||
function optionalMetric(value: unknown) {
|
||
if (value == null || value === "") return undefined;
|
||
const numberValue = typeof value === "number" ? value : Number(value);
|
||
return Number.isFinite(numberValue) && numberValue >= 0
|
||
? numberValue
|
||
: value;
|
||
}
|
||
|
||
const metricsShape = {
|
||
views: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
impressions: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
clicks: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
inquiries: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
likes: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
comments: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
shares: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
saves: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
average_position: z.preprocess(optionalMetric, z.number().nonnegative().optional()),
|
||
};
|
||
|
||
export const performanceMetricsSchema = z
|
||
.object(metricsShape)
|
||
.transform((metrics) =>
|
||
Object.fromEntries(
|
||
Object.entries(metrics).filter(([, value]) => value !== undefined),
|
||
) as PerformanceMetrics,
|
||
);
|
||
|
||
export const publicationInputSchema = z.object({
|
||
platform: publishPlatformSchema,
|
||
url: z.string().trim().url(),
|
||
published_at: z.string().datetime(),
|
||
notes: optionalTextSchema.default(""),
|
||
});
|
||
|
||
export const manualPerformanceInputSchema = z
|
||
.object({
|
||
window_label: z.string().trim().min(1),
|
||
feedback_summary: optionalTextSchema.default(""),
|
||
raw_reference: optionalTextSchema.optional(),
|
||
...metricsShape,
|
||
})
|
||
.transform(({ window_label, feedback_summary, raw_reference, ...metrics }) => ({
|
||
window_label,
|
||
feedback_summary,
|
||
raw_reference,
|
||
metrics: performanceMetricsSchema.parse(metrics),
|
||
}));
|
||
|
||
function rejectUnsafeRawReference(value: string | undefined) {
|
||
if (!value) return value;
|
||
if (/cookie|sessionid|token|secret|api[_-]?key|authorization/i.test(value)) {
|
||
throw new Error("raw_reference must not contain credentials");
|
||
}
|
||
return value;
|
||
}
|
||
|
||
export const rubricVersionSchema = z.object({
|
||
id: z.string().trim().min(1),
|
||
version: z.string().trim().min(1),
|
||
name: z.string().trim().min(1),
|
||
dimensions: z.array(
|
||
z.object({
|
||
id: z.string().trim().min(1),
|
||
label: z.string().trim().min(1),
|
||
weight: z.number().positive(),
|
||
description: z.string().trim().min(1),
|
||
}),
|
||
),
|
||
formula: z.literal("weighted_average_0_to_10"),
|
||
is_active: z.boolean(),
|
||
created_at: z.string().datetime(),
|
||
}) satisfies z.ZodType<RubricVersion>;
|
||
|
||
export const scoringRunSchema = z.object({
|
||
id: z.string().trim().min(1),
|
||
job_id: z.string().trim().min(1),
|
||
revision: z.number().int().positive(),
|
||
rubric_version_id: z.string().trim().min(1),
|
||
dimension_scores: z.record(z.string(), z.number().min(0).max(5)),
|
||
composite_score: z.number().min(0).max(10),
|
||
rationale: z.string().trim(),
|
||
created_at: z.string().datetime(),
|
||
}) satisfies z.ZodType<ScoringRun>;
|
||
|
||
export const publicationRecordSchema = z.object({
|
||
id: z.string().trim().min(1),
|
||
job_id: z.string().trim().min(1),
|
||
revision: z.number().int().positive(),
|
||
platform: publishPlatformSchema,
|
||
url: z.string().trim().url(),
|
||
published_at: z.string().datetime(),
|
||
status: z.enum(["draft", "published", "archived"]),
|
||
notes: z.string().trim(),
|
||
created_at: z.string().datetime(),
|
||
updated_at: z.string().datetime(),
|
||
}) satisfies z.ZodType<PublicationRecord>;
|
||
|
||
export const performanceSnapshotSchema = z
|
||
.object({
|
||
id: z.string().trim().min(1),
|
||
publication_id: z.string().trim().min(1),
|
||
source: z.union([z.literal("manual"), z.templateLiteral(["adapter:", z.string()])]),
|
||
window_label: z.string().trim().min(1),
|
||
metrics: performanceMetricsSchema,
|
||
feedback_summary: z.string().trim(),
|
||
raw_reference: z.string().trim().optional(),
|
||
snapshot_at: z.string().datetime(),
|
||
})
|
||
.transform((snapshot) => ({
|
||
...snapshot,
|
||
raw_reference: rejectUnsafeRawReference(snapshot.raw_reference),
|
||
})) satisfies z.ZodType<PerformanceSnapshot>;
|
||
|
||
export const calibrationDirectionSchema = z.enum([
|
||
"better_than_expected",
|
||
"as_expected",
|
||
"worse_than_expected",
|
||
"needs_more_data",
|
||
]) satisfies z.ZodType<CalibrationDirection>;
|
||
|
||
export const calibrationEventSchema = z.object({
|
||
id: z.string().trim().min(1),
|
||
publication_id: z.string().trim().min(1),
|
||
scoring_run_id: z.string().trim().min(1),
|
||
performance_snapshot_id: z.string().trim().min(1),
|
||
direction: calibrationDirectionSchema,
|
||
observations: z.array(z.string().trim().min(1)),
|
||
recommended_action: z.string().trim().min(1),
|
||
created_at: z.string().datetime(),
|
||
}) satisfies z.ZodType<CalibrationEvent>;
|
||
```
|
||
|
||
- [ ] **Step 5: Run validation tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/calibration/__tests__/validation.test.ts
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 6: Commit Task 1**
|
||
|
||
```bash
|
||
git add src/lib/calibration/types.ts src/lib/calibration/validation.ts src/lib/calibration/__tests__/validation.test.ts
|
||
git commit -m "新增发布校准领域类型"
|
||
```
|
||
|
||
## Task 2: Deterministic GEO Scoring And Manual Adapter
|
||
|
||
**Files:**
|
||
- Create: `src/lib/calibration/scoring.ts`
|
||
- Create: `src/lib/calibration/manual-adapter.ts`
|
||
- Create: `src/lib/calibration/__tests__/scoring.test.ts`
|
||
|
||
- [ ] **Step 1: Write failing scoring tests**
|
||
|
||
Create `src/lib/calibration/__tests__/scoring.test.ts`:
|
||
|
||
```ts
|
||
import { describe, expect, it } from "vitest";
|
||
|
||
import type { OptimizedArticle, QaReport } from "../../domain/types";
|
||
import { createManualPerformanceAdapter } from "../manual-adapter";
|
||
import {
|
||
GEO_RUBRIC_V1,
|
||
createCalibrationEvent,
|
||
scoreOptimizedArticle,
|
||
} from "../scoring";
|
||
|
||
const article: OptimizedArticle = {
|
||
job_id: "job_123",
|
||
revision: 2,
|
||
title: "示例科技 GEO 内容优化方案",
|
||
summary: "示例科技有限公司面向市场团队提供GEO内容优化服务。",
|
||
body_markdown:
|
||
"## 服务能力\n示例科技有限公司提供GEO内容优化服务，帮助市场团队提升AI搜索可见性。\n## 可信依据\n文章保留事实卡中的8年经验描述。",
|
||
image_suggestions: [],
|
||
changed_sections: ["title", "body"],
|
||
requires_user_confirmation: [],
|
||
};
|
||
|
||
const qaReport: QaReport = {
|
||
job_id: "job_123",
|
||
revision: 2,
|
||
overall_status: "warn",
|
||
checks: [
|
||
{
|
||
rule_id: "hallucination_risk",
|
||
status: "warn",
|
||
evidence: "8年经验",
|
||
reason: "需要人工复核经验年限依据。",
|
||
suggested_fix: "确认事实卡。",
|
||
target_agent: "body",
|
||
},
|
||
],
|
||
};
|
||
|
||
describe("calibration scoring", () => {
|
||
it("scores an optimized article with the active GEO rubric", () => {
|
||
const run = scoreOptimizedArticle({
|
||
jobId: "job_123",
|
||
article,
|
||
qaReport,
|
||
});
|
||
|
||
expect(run.rubric_version_id).toBe(GEO_RUBRIC_V1.id);
|
||
expect(run.dimension_scores.fact_integrity).toBe(4);
|
||
expect(run.dimension_scores.readability).toBeGreaterThanOrEqual(4);
|
||
expect(run.composite_score).toBeGreaterThan(6);
|
||
expect(run.composite_score).toBeLessThanOrEqual(10);
|
||
});
|
||
|
||
it("normalizes manual performance input through the adapter boundary", async () => {
|
||
const adapter = createManualPerformanceAdapter();
|
||
const snapshot = await adapter.fetch({
|
||
publication: {
|
||
id: "pub_123",
|
||
job_id: "job_123",
|
||
revision: 2,
|
||
platform: "official_site",
|
||
url: "https://example.com/article",
|
||
published_at: "2026-06-24T12:00:00.000Z",
|
||
status: "published",
|
||
notes: "",
|
||
created_at: "2026-06-24T12:00:00.000Z",
|
||
updated_at: "2026-06-24T12:00:00.000Z",
|
||
},
|
||
window_label: "T+7d",
|
||
manualInput: {
|
||
window_label: "T+7d",
|
||
views: "1200",
|
||
inquiries: "7",
|
||
feedback_summary: "用户追问案例依据",
|
||
},
|
||
});
|
||
|
||
expect(snapshot.source).toBe("manual");
|
||
expect(snapshot.metrics).toEqual({ views: 1200, inquiries: 7 });
|
||
});
|
||
|
||
it("creates a reviewable calibration event without changing the article", () => {
|
||
const scoringRun = scoreOptimizedArticle({
|
||
jobId: "job_123",
|
||
article,
|
||
qaReport,
|
||
});
|
||
const event = createCalibrationEvent({
|
||
scoringRun,
|
||
qaReport,
|
||
snapshot: {
|
||
id: "perf_123",
|
||
publication_id: "pub_123",
|
||
source: "manual",
|
||
window_label: "T+7d",
|
||
metrics: { views: 1200, inquiries: 7 },
|
||
feedback_summary: "用户追问案例依据",
|
||
snapshot_at: "2026-07-01T12:00:00.000Z",
|
||
},
|
||
});
|
||
|
||
expect(event.direction).toBe("better_than_expected");
|
||
expect(event.observations.join(" ")).toContain("询盘");
|
||
expect(event.recommended_action).toContain("积累");
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run failing scoring tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/calibration/__tests__/scoring.test.ts
|
||
```
|
||
|
||
Expected: FAIL because scoring/manual adapter files do not exist.
|
||
|
||
- [ ] **Step 3: Implement scoring service**
|
||
|
||
Create `src/lib/calibration/scoring.ts`:
|
||
|
||
```ts
|
||
import { nanoid } from "nanoid";
|
||
|
||
import type { OptimizedArticle, QaReport } from "../domain/types";
|
||
import type {
|
||
CalibrationContext,
|
||
CalibrationDirection,
|
||
CalibrationEvent,
|
||
RubricVersion,
|
||
ScoringRun,
|
||
} from "./types";
|
||
|
||
export const GEO_RUBRIC_V1: RubricVersion = {
|
||
id: "rubric_geo_v1",
|
||
version: "v1",
|
||
name: "GEO article performance rubric",
|
||
formula: "weighted_average_0_to_10",
|
||
is_active: true,
|
||
created_at: "2026-06-24T00:00:00.000Z",
|
||
dimensions: [
|
||
{
|
||
id: "fact_integrity",
|
||
label: "事实一致性",
|
||
weight: 2,
|
||
description: "事实、公司名、产品名和经验年限是否遵守事实卡。",
|
||
},
|
||
{
|
||
id: "platform_fit",
|
||
label: "平台适配",
|
||
weight: 1.5,
|
||
description: "表达是否匹配目标发布平台。",
|
||
},
|
||
{
|
||
id: "search_intent_fit",
|
||
label: "搜索意图匹配",
|
||
weight: 1.5,
|
||
description: "是否回答GEO/Search背后的用户问题。",
|
||
},
|
||
{
|
||
id: "answer_density",
|
||
label: "答案密度",
|
||
weight: 1.5,
|
||
description: "是否提供具体信息而不是泛泛宣传。",
|
||
},
|
||
{
|
||
id: "trust_signal_quality",
|
||
label: "信任信号质量",
|
||
weight: 1.5,
|
||
description: "可信依据是否具体、克制且可复核。",
|
||
},
|
||
{
|
||
id: "readability",
|
||
label: "可读性",
|
||
weight: 1,
|
||
description: "标题、摘要、正文是否清晰易读。",
|
||
},
|
||
],
|
||
};
|
||
|
||
interface ScoreOptimizedArticleInput {
|
||
jobId: string;
|
||
article: OptimizedArticle;
|
||
qaReport: QaReport;
|
||
}
|
||
|
||
export function scoreOptimizedArticle({
|
||
jobId,
|
||
article,
|
||
qaReport,
|
||
}: ScoreOptimizedArticleInput): ScoringRun {
|
||
const combined = `${article.title}\n${article.summary}\n${article.body_markdown}`;
|
||
const dimensionScores = {
|
||
fact_integrity: scoreFactIntegrity(qaReport),
|
||
platform_fit: scoreRuleGroup(qaReport, ["platform_fit", "voice_consistency"]),
|
||
search_intent_fit: hasGeoIntent(combined) ? 4 : 2,
|
||
answer_density: scoreAnswerDensity(combined),
|
||
trust_signal_quality: scoreTrustSignals(combined, qaReport),
|
||
readability: scoreReadability(article),
|
||
};
|
||
const composite = weightedComposite(dimensionScores);
|
||
|
||
return {
|
||
id: `score_${nanoid(10)}`,
|
||
job_id: jobId,
|
||
revision: article.revision ?? 1,
|
||
rubric_version_id: GEO_RUBRIC_V1.id,
|
||
dimension_scores: dimensionScores,
|
||
composite_score: composite,
|
||
rationale: buildRationale(dimensionScores),
|
||
created_at: new Date().toISOString(),
|
||
};
|
||
}
|
||
|
||
function scoreFactIntegrity(report: QaReport) {
|
||
const hardRules = ["company_name_integrity", "claim_consistency", "hallucination_risk"];
|
||
const statuses = report.checks
|
||
.filter((check) => hardRules.includes(check.rule_id))
|
||
.map((check) => check.status);
|
||
if (statuses.includes("fail")) return 1;
|
||
if (statuses.includes("warn")) return 4;
|
||
return 5;
|
||
}
|
||
|
||
function scoreRuleGroup(report: QaReport, ruleIds: string[]) {
|
||
const statuses = report.checks
|
||
.filter((check) => ruleIds.includes(check.rule_id))
|
||
.map((check) => check.status);
|
||
if (statuses.includes("fail")) return 2;
|
||
if (statuses.includes("warn")) return 3;
|
||
return 4;
|
||
}
|
||
|
||
function hasGeoIntent(text: string) {
|
||
return /GEO|AI搜索|生成式引擎|搜索|可见性|问答|推荐/i.test(text);
|
||
}
|
||
|
||
function scoreAnswerDensity(text: string) {
|
||
const headings = (text.match(/^##\s+/gm) ?? []).length;
|
||
const concreteSignals = (text.match(/服务|流程|方案|能力|团队|行业|客户|案例/g) ?? [])
|
||
.length;
|
||
if (headings >= 2 && concreteSignals >= 8) return 5;
|
||
if (headings >= 1 && concreteSignals >= 4) return 4;
|
||
if (concreteSignals >= 2) return 3;
|
||
return 2;
|
||
}
|
||
|
||
function scoreTrustSignals(text: string, report: QaReport) {
|
||
const hallucination = report.checks.find(
|
||
(check) => check.rule_id === "hallucination_risk",
|
||
);
|
||
if (hallucination?.status === "fail") return 1;
|
||
const trustSignals = (text.match(/依据|经验|资质|案例|事实卡|复核|客户/g) ?? [])
|
||
.length;
|
||
if (hallucination?.status === "warn") return trustSignals >= 2 ? 3 : 2;
|
||
return trustSignals >= 2 ? 4 : 3;
|
||
}
|
||
|
||
function scoreReadability(article: OptimizedArticle) {
|
||
const longSentence = `${article.summary}\n${article.body_markdown}`
|
||
.split(/[。！？.!?]/)
|
||
.some((sentence) => sentence.length > 180);
|
||
if (article.title.length > 42 || longSentence) return 3;
|
||
return 4;
|
||
}
|
||
|
||
function weightedComposite(scores: Record<string, number>) {
|
||
const totalWeight = GEO_RUBRIC_V1.dimensions.reduce(
|
||
(sum, dimension) => sum + dimension.weight,
|
||
0,
|
||
);
|
||
const weighted = GEO_RUBRIC_V1.dimensions.reduce(
|
||
(sum, dimension) => sum + (scores[dimension.id] ?? 0) * dimension.weight,
|
||
0,
|
||
);
|
||
return Math.round((weighted / totalWeight) * 2 * 10) / 10;
|
||
}
|
||
|
||
function buildRationale(scores: Record<string, number>) {
|
||
return `事实一致性 ${scores.fact_integrity}/5，平台适配 ${scores.platform_fit}/5，答案密度 ${scores.answer_density}/5，信任信号 ${scores.trust_signal_quality}/5。`;
|
||
}
|
||
|
||
export function createCalibrationEvent({
|
||
scoringRun,
|
||
qaReport,
|
||
snapshot,
|
||
}: CalibrationContext): CalibrationEvent {
|
||
const direction = inferDirection(scoringRun.composite_score, snapshot.metrics);
|
||
const observations = buildObservations(direction, scoringRun, qaReport, snapshot.feedback_summary);
|
||
|
||
return {
|
||
id: `cal_${nanoid(10)}`,
|
||
publication_id: snapshot.publication_id,
|
||
scoring_run_id: scoringRun.id,
|
||
performance_snapshot_id: snapshot.id,
|
||
direction,
|
||
observations,
|
||
recommended_action:
|
||
"先积累至少 5 篇同类样本，再评估是否调整 GEO rubric 权重。",
|
||
created_at: new Date().toISOString(),
|
||
};
|
||
}
|
||
|
||
function inferDirection(
|
||
composite: number,
|
||
metrics: { views?: number; clicks?: number; inquiries?: number },
|
||
): CalibrationDirection {
|
||
if (!metrics.views && !metrics.clicks && !metrics.inquiries) {
|
||
return "needs_more_data";
|
||
}
|
||
if ((metrics.inquiries ?? 0) >= 3 || (metrics.clicks ?? 0) >= 50) {
|
||
return composite >= 6 ? "better_than_expected" : "better_than_expected";
|
||
}
|
||
if ((metrics.views ?? 0) < 100 && composite >= 7) {
|
||
return "worse_than_expected";
|
||
}
|
||
return "as_expected";
|
||
}
|
||
|
||
function buildObservations(
|
||
direction: CalibrationDirection,
|
||
scoringRun: ScoringRun,
|
||
qaReport: QaReport,
|
||
feedbackSummary: string,
|
||
) {
|
||
const observations = [
|
||
`综合评分 ${scoringRun.composite_score}/10，真实表现方向为 ${direction}。`,
|
||
];
|
||
if (qaReport.overall_status !== "pass") {
|
||
observations.push(`QA 状态为 ${qaReport.overall_status}，需要和表现数据一起复盘。`);
|
||
}
|
||
if (feedbackSummary) {
|
||
observations.push(`反馈摘要：${feedbackSummary}`);
|
||
}
|
||
if ((scoringRun.dimension_scores.trust_signal_quality ?? 0) <= 3) {
|
||
observations.push("信任信号质量偏低，后续观察是否影响询盘。");
|
||
}
|
||
return observations;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Implement manual adapter**
|
||
|
||
Create `src/lib/calibration/manual-adapter.ts`:
|
||
|
||
```ts
|
||
import { nanoid } from "nanoid";
|
||
|
||
import type {
|
||
AdapterFetchInput,
|
||
PerformanceAdapter,
|
||
PerformanceSnapshot,
|
||
} from "./types";
|
||
import { manualPerformanceInputSchema, performanceSnapshotSchema } from "./validation";
|
||
|
||
interface ManualAdapterFetchInput extends AdapterFetchInput {
|
||
manualInput: unknown;
|
||
}
|
||
|
||
export function createManualPerformanceAdapter(): PerformanceAdapter & {
|
||
fetch(input: ManualAdapterFetchInput): Promise<PerformanceSnapshot>;
|
||
} {
|
||
return {
|
||
source: "manual",
|
||
async fetch(input) {
|
||
const parsed = manualPerformanceInputSchema.parse(input.manualInput);
|
||
return performanceSnapshotSchema.parse({
|
||
id: `perf_${nanoid(10)}`,
|
||
publication_id: input.publication.id,
|
||
source: "manual",
|
||
window_label: parsed.window_label,
|
||
metrics: parsed.metrics,
|
||
feedback_summary: parsed.feedback_summary,
|
||
raw_reference: parsed.raw_reference,
|
||
snapshot_at: new Date().toISOString(),
|
||
});
|
||
},
|
||
};
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Run scoring tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/calibration/__tests__/scoring.test.ts
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 6: Commit Task 2**
|
||
|
||
```bash
|
||
git add src/lib/calibration/scoring.ts src/lib/calibration/manual-adapter.ts src/lib/calibration/__tests__/scoring.test.ts
|
||
git commit -m "新增发布表现评分服务"
|
||
```
|
||
|
||
## Task 3: SQLite And D1 Schema
|
||
|
||
**Files:**
|
||
- Modify: `src/lib/db/schema.ts`
|
||
- Create: `migrations/0002_publication_performance_calibration.sql`
|
||
|
||
- [ ] **Step 1: Add local schema tables**
|
||
|
||
In `src/lib/db/schema.ts`, append these tables inside the existing `db.exec(\`...\`)` block after `qa_reports`:
|
||
|
||
```sql
|
||
create table if not exists rubric_versions (
|
||
id text primary key,
|
||
version text not null,
|
||
name text not null,
|
||
dimensions text not null,
|
||
formula text not null,
|
||
is_active integer not null,
|
||
created_at text not null
|
||
);
|
||
|
||
create table if not exists scoring_runs (
|
||
id text primary key,
|
||
job_id text not null,
|
||
revision integer not null,
|
||
rubric_version_id text not null,
|
||
dimension_scores text not null,
|
||
composite_score real not null,
|
||
rationale text not null,
|
||
created_at text not null,
|
||
foreign key (job_id, revision)
|
||
references optimized_articles(job_id, revision) on delete cascade,
|
||
foreign key (rubric_version_id) references rubric_versions(id)
|
||
);
|
||
|
||
create table if not exists publication_records (
|
||
id text primary key,
|
||
job_id text not null,
|
||
revision integer not null,
|
||
platform text not null,
|
||
url text not null,
|
||
published_at text not null,
|
||
status text not null,
|
||
notes text not null,
|
||
created_at text not null,
|
||
updated_at text not null,
|
||
foreign key (job_id, revision)
|
||
references optimized_articles(job_id, revision) on delete cascade
|
||
);
|
||
|
||
create table if not exists performance_snapshots (
|
||
id text primary key,
|
||
publication_id text not null,
|
||
source text not null,
|
||
window_label text not null,
|
||
metrics text not null,
|
||
feedback_summary text not null,
|
||
raw_reference text,
|
||
snapshot_at text not null,
|
||
foreign key (publication_id) references publication_records(id) on delete cascade
|
||
);
|
||
|
||
create table if not exists calibration_events (
|
||
id text primary key,
|
||
publication_id text not null,
|
||
scoring_run_id text not null,
|
||
performance_snapshot_id text not null,
|
||
direction text not null,
|
||
observations text not null,
|
||
recommended_action text not null,
|
||
created_at text not null,
|
||
foreign key (publication_id) references publication_records(id) on delete cascade,
|
||
foreign key (scoring_run_id) references scoring_runs(id) on delete cascade,
|
||
foreign key (performance_snapshot_id)
|
||
references performance_snapshots(id) on delete cascade
|
||
);
|
||
|
||
create index if not exists idx_scoring_runs_job_revision
|
||
on scoring_runs(job_id, revision);
|
||
|
||
create index if not exists idx_publication_records_job_revision
|
||
on publication_records(job_id, revision);
|
||
|
||
create index if not exists idx_performance_snapshots_publication
|
||
on performance_snapshots(publication_id);
|
||
```
|
||
|
||
- [ ] **Step 2: Add D1 migration**
|
||
|
||
Create `migrations/0002_publication_performance_calibration.sql` with the same SQL in Cloudflare-compatible uppercase style:
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS rubric_versions (
|
||
id TEXT PRIMARY KEY,
|
||
version TEXT NOT NULL,
|
||
name TEXT NOT NULL,
|
||
dimensions TEXT NOT NULL,
|
||
formula TEXT NOT NULL,
|
||
is_active INTEGER NOT NULL,
|
||
created_at TEXT NOT NULL
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS scoring_runs (
|
||
id TEXT PRIMARY KEY,
|
||
job_id TEXT NOT NULL,
|
||
revision INTEGER NOT NULL,
|
||
rubric_version_id TEXT NOT NULL,
|
||
dimension_scores TEXT NOT NULL,
|
||
composite_score REAL NOT NULL,
|
||
rationale TEXT NOT NULL,
|
||
created_at TEXT NOT NULL,
|
||
FOREIGN KEY (job_id, revision)
|
||
REFERENCES optimized_articles(job_id, revision) ON DELETE CASCADE,
|
||
FOREIGN KEY (rubric_version_id) REFERENCES rubric_versions(id)
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS publication_records (
|
||
id TEXT PRIMARY KEY,
|
||
job_id TEXT NOT NULL,
|
||
revision INTEGER NOT NULL,
|
||
platform TEXT NOT NULL,
|
||
url TEXT NOT NULL,
|
||
published_at TEXT NOT NULL,
|
||
status TEXT NOT NULL,
|
||
notes TEXT NOT NULL,
|
||
created_at TEXT NOT NULL,
|
||
updated_at TEXT NOT NULL,
|
||
FOREIGN KEY (job_id, revision)
|
||
REFERENCES optimized_articles(job_id, revision) ON DELETE CASCADE
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS performance_snapshots (
|
||
id TEXT PRIMARY KEY,
|
||
publication_id TEXT NOT NULL,
|
||
source TEXT NOT NULL,
|
||
window_label TEXT NOT NULL,
|
||
metrics TEXT NOT NULL,
|
||
feedback_summary TEXT NOT NULL,
|
||
raw_reference TEXT,
|
||
snapshot_at TEXT NOT NULL,
|
||
FOREIGN KEY (publication_id) REFERENCES publication_records(id) ON DELETE CASCADE
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS calibration_events (
|
||
id TEXT PRIMARY KEY,
|
||
publication_id TEXT NOT NULL,
|
||
scoring_run_id TEXT NOT NULL,
|
||
performance_snapshot_id TEXT NOT NULL,
|
||
direction TEXT NOT NULL,
|
||
observations TEXT NOT NULL,
|
||
recommended_action TEXT NOT NULL,
|
||
created_at TEXT NOT NULL,
|
||
FOREIGN KEY (publication_id) REFERENCES publication_records(id) ON DELETE CASCADE,
|
||
FOREIGN KEY (scoring_run_id) REFERENCES scoring_runs(id) ON DELETE CASCADE,
|
||
FOREIGN KEY (performance_snapshot_id)
|
||
REFERENCES performance_snapshots(id) ON DELETE CASCADE
|
||
);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_scoring_runs_job_revision
|
||
ON scoring_runs(job_id, revision);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_publication_records_job_revision
|
||
ON publication_records(job_id, revision);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_performance_snapshots_publication
|
||
ON performance_snapshots(publication_id);
|
||
```
|
||
|
||
- [ ] **Step 3: Run schema syntax check**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
node -e "const fs=require('fs'); const sql=fs.readFileSync('migrations/0002_publication_performance_calibration.sql','utf8'); console.log(sql.includes('performance_snapshots') ? 'migration ok' : 'missing table')"
|
||
```
|
||
|
||
Expected: prints `migration ok`.
|
||
|
||
- [ ] **Step 4: Commit Task 3**
|
||
|
||
```bash
|
||
git add src/lib/db/schema.ts migrations/0002_publication_performance_calibration.sql
|
||
git commit -m "新增发布校准数据库结构"
|
||
```
|
||
|
||
## Task 4: Repository Methods For Calibration
|
||
|
||
**Files:**
|
||
- Modify: `src/lib/db/repositories.ts`
|
||
- Modify: `src/lib/db/repository.ts`
|
||
- Modify: `src/lib/db/sqlite-repository.ts`
|
||
- Modify: `src/lib/db/d1-repository.ts`
|
||
- Modify: `src/lib/db/__tests__/repository.test.ts`
|
||
- Modify: `src/lib/db/__tests__/d1-repository.test.ts`
|
||
|
||
- [ ] **Step 1: Write failing SQLite repository test**
|
||
|
||
Append to `src/lib/db/__tests__/repository.test.ts`:
|
||
|
||
```ts
|
||
test("persists scoring, publication, performance, and calibration event", async () => {
|
||
const repository = createSqliteRepository(dbPath);
|
||
const job = await repository.createArticleJob({
|
||
source_title: "Title",
|
||
source_body: "Body",
|
||
image_inputs: [],
|
||
publish_platform: "official_site",
|
||
user_instructions: "",
|
||
});
|
||
const article = await repository.saveOptimizedArticle(job.id, {
|
||
title: "Optimized",
|
||
summary: "Summary",
|
||
body_markdown: "Body",
|
||
image_suggestions: [],
|
||
changed_sections: [],
|
||
requires_user_confirmation: [],
|
||
});
|
||
|
||
await repository.saveRubricVersion({
|
||
id: "rubric_geo_v1",
|
||
version: "v1",
|
||
name: "GEO rubric",
|
||
dimensions: [],
|
||
formula: "weighted_average_0_to_10",
|
||
is_active: true,
|
||
created_at: "2026-06-24T00:00:00.000Z",
|
||
});
|
||
const scoringRun = await repository.saveScoringRun({
|
||
id: "score_1",
|
||
job_id: job.id,
|
||
revision: article.revision ?? 1,
|
||
rubric_version_id: "rubric_geo_v1",
|
||
dimension_scores: { readability: 4 },
|
||
composite_score: 8,
|
||
rationale: "Readable",
|
||
created_at: "2026-06-24T00:00:00.000Z",
|
||
});
|
||
const publication = await repository.createPublicationRecord({
|
||
job_id: job.id,
|
||
revision: article.revision ?? 1,
|
||
platform: "official_site",
|
||
url: "https://example.com/article",
|
||
published_at: "2026-06-24T12:00:00.000Z",
|
||
status: "published",
|
||
notes: "官网首发",
|
||
});
|
||
const snapshot = await repository.savePerformanceSnapshot({
|
||
id: "perf_1",
|
||
publication_id: publication.id,
|
||
source: "manual",
|
||
window_label: "T+7d",
|
||
metrics: { views: 1200 },
|
||
feedback_summary: "用户追问案例依据",
|
||
snapshot_at: "2026-07-01T12:00:00.000Z",
|
||
});
|
||
const event = await repository.saveCalibrationEvent({
|
||
id: "cal_1",
|
||
publication_id: publication.id,
|
||
scoring_run_id: scoringRun.id,
|
||
performance_snapshot_id: snapshot.id,
|
||
direction: "better_than_expected",
|
||
observations: ["表现高于预期"],
|
||
recommended_action: "继续积累样本",
|
||
created_at: "2026-07-01T12:10:00.000Z",
|
||
});
|
||
|
||
await expect(repository.listPublicationRecords(job.id)).resolves.toHaveLength(1);
|
||
await expect(repository.getLatestScoringRun(job.id, article.revision ?? 1))
|
||
.resolves.toMatchObject({ id: scoringRun.id, composite_score: 8 });
|
||
await expect(repository.listPerformanceSnapshots(publication.id))
|
||
.resolves.toEqual([expect.objectContaining({ id: snapshot.id })]);
|
||
expect(event.observations).toEqual(["表现高于预期"]);
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run failing repository test**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/db/__tests__/repository.test.ts -t "persists scoring"
|
||
```
|
||
|
||
Expected: FAIL because repository methods do not exist.
|
||
|
||
- [ ] **Step 3: Extend repository interface**
|
||
|
||
In `src/lib/db/repository.ts`, import calibration types and add methods to `AppRepository`:
|
||
|
||
```ts
|
||
import type {
|
||
CalibrationEvent,
|
||
PerformanceSnapshot,
|
||
PublicationRecord,
|
||
RubricVersion,
|
||
ScoringRun,
|
||
} from "../calibration/types";
|
||
```
|
||
|
||
Add these interface methods:
|
||
|
||
```ts
|
||
saveRubricVersion(rubric: RubricVersion): Promise<RubricVersion>;
|
||
saveScoringRun(run: ScoringRun): Promise<ScoringRun>;
|
||
getLatestScoringRun(jobId: string, revision: number): Promise<ScoringRun | null>;
|
||
createPublicationRecord(
|
||
input: Omit<PublicationRecord, "id" | "created_at" | "updated_at">,
|
||
): Promise<PublicationRecord>;
|
||
listPublicationRecords(jobId: string): Promise<PublicationRecord[]>;
|
||
getPublicationRecord(id: string): Promise<PublicationRecord | null>;
|
||
savePerformanceSnapshot(snapshot: PerformanceSnapshot): Promise<PerformanceSnapshot>;
|
||
listPerformanceSnapshots(publicationId: string): Promise<PerformanceSnapshot[]>;
|
||
saveCalibrationEvent(event: CalibrationEvent): Promise<CalibrationEvent>;
|
||
```
|
||
|
||
- [ ] **Step 4: Add SQLite helper types and functions**
|
||
|
||
In `src/lib/db/repositories.ts`, import calibration types and add row interfaces:
|
||
|
||
```ts
|
||
import type {
|
||
CalibrationEvent,
|
||
PerformanceSnapshot,
|
||
PublicationRecord,
|
||
RubricVersion,
|
||
ScoringRun,
|
||
} from "../calibration/types";
|
||
```
|
||
|
||
Add row interfaces near existing row interfaces:
|
||
|
||
```ts
|
||
interface RubricVersionRow {
|
||
id: string;
|
||
version: string;
|
||
name: string;
|
||
dimensions: string;
|
||
formula: "weighted_average_0_to_10";
|
||
is_active: number;
|
||
created_at: string;
|
||
}
|
||
|
||
interface ScoringRunRow {
|
||
id: string;
|
||
job_id: string;
|
||
revision: number;
|
||
rubric_version_id: string;
|
||
dimension_scores: string;
|
||
composite_score: number;
|
||
rationale: string;
|
||
created_at: string;
|
||
}
|
||
|
||
interface PublicationRecordRow {
|
||
id: string;
|
||
job_id: string;
|
||
revision: number;
|
||
platform: PublishPlatform;
|
||
url: string;
|
||
published_at: string;
|
||
status: "draft" | "published" | "archived";
|
||
notes: string;
|
||
created_at: string;
|
||
updated_at: string;
|
||
}
|
||
|
||
interface PerformanceSnapshotRow {
|
||
id: string;
|
||
publication_id: string;
|
||
source: "manual" | `adapter:${string}`;
|
||
window_label: string;
|
||
metrics: string;
|
||
feedback_summary: string;
|
||
raw_reference: string | null;
|
||
snapshot_at: string;
|
||
}
|
||
|
||
interface CalibrationEventRow {
|
||
id: string;
|
||
publication_id: string;
|
||
scoring_run_id: string;
|
||
performance_snapshot_id: string;
|
||
direction: CalibrationEvent["direction"];
|
||
observations: string;
|
||
recommended_action: string;
|
||
created_at: string;
|
||
}
|
||
```
|
||
|
||
Add mapper helpers:
|
||
|
||
```ts
|
||
function toRubricVersion(row: RubricVersionRow): RubricVersion {
|
||
return {
|
||
...row,
|
||
dimensions: parseJson<RubricVersion["dimensions"]>(row.dimensions),
|
||
is_active: Boolean(row.is_active),
|
||
};
|
||
}
|
||
|
||
function toScoringRun(row: ScoringRunRow): ScoringRun {
|
||
return {
|
||
...row,
|
||
dimension_scores: parseJson<Record<string, number>>(row.dimension_scores),
|
||
};
|
||
}
|
||
|
||
function toPerformanceSnapshot(row: PerformanceSnapshotRow): PerformanceSnapshot {
|
||
return {
|
||
...row,
|
||
raw_reference: row.raw_reference ?? undefined,
|
||
metrics: parseJson<PerformanceSnapshot["metrics"]>(row.metrics),
|
||
};
|
||
}
|
||
|
||
function toCalibrationEvent(row: CalibrationEventRow): CalibrationEvent {
|
||
return {
|
||
...row,
|
||
observations: parseJson<string[]>(row.observations),
|
||
};
|
||
}
|
||
```
|
||
|
||
Add exported functions after `getLatestQaReport`:
|
||
|
||
```ts
|
||
export function saveRubricVersion(dbPath: string | undefined, rubric: RubricVersion) {
|
||
return withDb(dbPath, (db) => {
|
||
db.prepare(
|
||
`insert into rubric_versions (
|
||
id, version, name, dimensions, formula, is_active, created_at
|
||
) values (?, ?, ?, ?, ?, ?, ?)
|
||
on conflict(id) do update set
|
||
version = excluded.version,
|
||
name = excluded.name,
|
||
dimensions = excluded.dimensions,
|
||
formula = excluded.formula,
|
||
is_active = excluded.is_active`,
|
||
).run(
|
||
rubric.id,
|
||
rubric.version,
|
||
rubric.name,
|
||
serialize(rubric.dimensions),
|
||
rubric.formula,
|
||
rubric.is_active ? 1 : 0,
|
||
rubric.created_at,
|
||
);
|
||
return rubric;
|
||
});
|
||
}
|
||
|
||
export function saveScoringRun(dbPath: string | undefined, run: ScoringRun) {
|
||
return withDb(dbPath, (db) => {
|
||
db.prepare(
|
||
`insert into scoring_runs (
|
||
id, job_id, revision, rubric_version_id, dimension_scores,
|
||
composite_score, rationale, created_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
).run(
|
||
run.id,
|
||
run.job_id,
|
||
run.revision,
|
||
run.rubric_version_id,
|
||
serialize(run.dimension_scores),
|
||
run.composite_score,
|
||
run.rationale,
|
||
run.created_at,
|
||
);
|
||
return run;
|
||
});
|
||
}
|
||
|
||
export function getLatestScoringRun(
|
||
dbPath: string | undefined,
|
||
jobId: string,
|
||
revision: number,
|
||
) {
|
||
return withDb(dbPath, (db) => {
|
||
const row = db
|
||
.prepare(
|
||
`select * from scoring_runs
|
||
where job_id = ? and revision = ?
|
||
order by created_at desc
|
||
limit 1`,
|
||
)
|
||
.get(jobId, revision) as ScoringRunRow | undefined;
|
||
return row ? toScoringRun(row) : null;
|
||
});
|
||
}
|
||
|
||
export function createPublicationRecord(
|
||
dbPath: string | undefined,
|
||
input: Omit<PublicationRecord, "id" | "created_at" | "updated_at">,
|
||
) {
|
||
return withDb(dbPath, (db) => {
|
||
const timestamp = nowIso();
|
||
const record: PublicationRecord = {
|
||
id: `pub_${nanoid(10)}`,
|
||
...input,
|
||
created_at: timestamp,
|
||
updated_at: timestamp,
|
||
};
|
||
db.prepare(
|
||
`insert into publication_records (
|
||
id, job_id, revision, platform, url, published_at, status,
|
||
notes, created_at, updated_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
).run(
|
||
record.id,
|
||
record.job_id,
|
||
record.revision,
|
||
record.platform,
|
||
record.url,
|
||
record.published_at,
|
||
record.status,
|
||
record.notes,
|
||
record.created_at,
|
||
record.updated_at,
|
||
);
|
||
return record;
|
||
});
|
||
}
|
||
|
||
export function listPublicationRecords(dbPath: string | undefined, jobId: string) {
|
||
return withDb(dbPath, (db) =>
|
||
db
|
||
.prepare("select * from publication_records where job_id = ? order by published_at desc")
|
||
.all(jobId)
|
||
.map((row) => row as PublicationRecord),
|
||
);
|
||
}
|
||
|
||
export function getPublicationRecord(dbPath: string | undefined, id: string) {
|
||
return withDb(dbPath, (db) => {
|
||
const row = db
|
||
.prepare("select * from publication_records where id = ?")
|
||
.get(id) as PublicationRecordRow | undefined;
|
||
return row ?? null;
|
||
});
|
||
}
|
||
|
||
export function savePerformanceSnapshot(
|
||
dbPath: string | undefined,
|
||
snapshot: PerformanceSnapshot,
|
||
) {
|
||
return withDb(dbPath, (db) => {
|
||
db.prepare(
|
||
`insert into performance_snapshots (
|
||
id, publication_id, source, window_label, metrics,
|
||
feedback_summary, raw_reference, snapshot_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
).run(
|
||
snapshot.id,
|
||
snapshot.publication_id,
|
||
snapshot.source,
|
||
snapshot.window_label,
|
||
serialize(snapshot.metrics),
|
||
snapshot.feedback_summary,
|
||
snapshot.raw_reference ?? null,
|
||
snapshot.snapshot_at,
|
||
);
|
||
return snapshot;
|
||
});
|
||
}
|
||
|
||
export function listPerformanceSnapshots(
|
||
dbPath: string | undefined,
|
||
publicationId: string,
|
||
) {
|
||
return withDb(dbPath, (db) =>
|
||
db
|
||
.prepare(
|
||
"select * from performance_snapshots where publication_id = ? order by snapshot_at desc",
|
||
)
|
||
.all(publicationId)
|
||
.map((row) => toPerformanceSnapshot(row as PerformanceSnapshotRow)),
|
||
);
|
||
}
|
||
|
||
export function saveCalibrationEvent(
|
||
dbPath: string | undefined,
|
||
event: CalibrationEvent,
|
||
) {
|
||
return withDb(dbPath, (db) => {
|
||
db.prepare(
|
||
`insert into calibration_events (
|
||
id, publication_id, scoring_run_id, performance_snapshot_id,
|
||
direction, observations, recommended_action, created_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
).run(
|
||
event.id,
|
||
event.publication_id,
|
||
event.scoring_run_id,
|
||
event.performance_snapshot_id,
|
||
event.direction,
|
||
serialize(event.observations),
|
||
event.recommended_action,
|
||
event.created_at,
|
||
);
|
||
return event;
|
||
});
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Expose SQLite methods**
|
||
|
||
In `src/lib/db/sqlite-repository.ts`, add the imported helper names and methods:
|
||
|
||
```ts
|
||
createPublicationRecord,
|
||
getLatestScoringRun,
|
||
getPublicationRecord,
|
||
listPerformanceSnapshots,
|
||
listPublicationRecords,
|
||
saveCalibrationEvent,
|
||
savePerformanceSnapshot,
|
||
saveRubricVersion,
|
||
saveScoringRun,
|
||
```
|
||
|
||
Add implementations in `createSqliteRepository`:
|
||
|
||
```ts
|
||
saveRubricVersion(rubric) {
|
||
return Promise.resolve(saveRubricVersion(dbPath, rubric));
|
||
},
|
||
saveScoringRun(run) {
|
||
return Promise.resolve(saveScoringRun(dbPath, run));
|
||
},
|
||
getLatestScoringRun(jobId, revision) {
|
||
return Promise.resolve(getLatestScoringRun(dbPath, jobId, revision));
|
||
},
|
||
createPublicationRecord(input) {
|
||
return Promise.resolve(createPublicationRecord(dbPath, input));
|
||
},
|
||
listPublicationRecords(jobId) {
|
||
return Promise.resolve(listPublicationRecords(dbPath, jobId));
|
||
},
|
||
getPublicationRecord(id) {
|
||
return Promise.resolve(getPublicationRecord(dbPath, id));
|
||
},
|
||
savePerformanceSnapshot(snapshot) {
|
||
return Promise.resolve(savePerformanceSnapshot(dbPath, snapshot));
|
||
},
|
||
listPerformanceSnapshots(publicationId) {
|
||
return Promise.resolve(listPerformanceSnapshots(dbPath, publicationId));
|
||
},
|
||
saveCalibrationEvent(event) {
|
||
return Promise.resolve(saveCalibrationEvent(dbPath, event));
|
||
},
|
||
```
|
||
|
||
- [ ] **Step 6: Add D1 methods**
|
||
|
||
In `src/lib/db/d1-repository.ts`, import calibration types:
|
||
|
||
```ts
|
||
import type {
|
||
CalibrationEvent,
|
||
PerformanceSnapshot,
|
||
PublicationRecord,
|
||
RubricVersion,
|
||
ScoringRun,
|
||
} from "../calibration/types";
|
||
```
|
||
|
||
Add row interfaces near the existing D1 row interfaces:
|
||
|
||
```ts
|
||
interface RubricVersionRow {
|
||
id: string;
|
||
version: string;
|
||
name: string;
|
||
dimensions: string;
|
||
formula: "weighted_average_0_to_10";
|
||
is_active: number;
|
||
created_at: string;
|
||
}
|
||
|
||
interface ScoringRunRow {
|
||
id: string;
|
||
job_id: string;
|
||
revision: number;
|
||
rubric_version_id: string;
|
||
dimension_scores: string;
|
||
composite_score: number;
|
||
rationale: string;
|
||
created_at: string;
|
||
}
|
||
|
||
interface PublicationRecordRow {
|
||
id: string;
|
||
job_id: string;
|
||
revision: number;
|
||
platform: PublishPlatform;
|
||
url: string;
|
||
published_at: string;
|
||
status: "draft" | "published" | "archived";
|
||
notes: string;
|
||
created_at: string;
|
||
updated_at: string;
|
||
}
|
||
|
||
interface PerformanceSnapshotRow {
|
||
id: string;
|
||
publication_id: string;
|
||
source: "manual" | `adapter:${string}`;
|
||
window_label: string;
|
||
metrics: string;
|
||
feedback_summary: string;
|
||
raw_reference: string | null;
|
||
snapshot_at: string;
|
||
}
|
||
|
||
interface CalibrationEventRow {
|
||
id: string;
|
||
publication_id: string;
|
||
scoring_run_id: string;
|
||
performance_snapshot_id: string;
|
||
direction: CalibrationEvent["direction"];
|
||
observations: string;
|
||
recommended_action: string;
|
||
created_at: string;
|
||
}
|
||
```
|
||
|
||
Add mapper helpers below `toArticleJob`:
|
||
|
||
```ts
|
||
function toRubricVersion(row: RubricVersionRow): RubricVersion {
|
||
return {
|
||
...row,
|
||
dimensions: parseJson<RubricVersion["dimensions"]>(row.dimensions),
|
||
is_active: Boolean(row.is_active),
|
||
};
|
||
}
|
||
|
||
function toScoringRun(row: ScoringRunRow): ScoringRun {
|
||
return {
|
||
...row,
|
||
dimension_scores: parseJson<Record<string, number>>(row.dimension_scores),
|
||
};
|
||
}
|
||
|
||
function toPerformanceSnapshot(row: PerformanceSnapshotRow): PerformanceSnapshot {
|
||
return {
|
||
...row,
|
||
raw_reference: row.raw_reference ?? undefined,
|
||
metrics: parseJson<PerformanceSnapshot["metrics"]>(row.metrics),
|
||
};
|
||
}
|
||
|
||
function toCalibrationEvent(row: CalibrationEventRow): CalibrationEvent {
|
||
return {
|
||
...row,
|
||
observations: parseJson<string[]>(row.observations),
|
||
};
|
||
}
|
||
```
|
||
|
||
Add these methods inside the object returned by `createD1Repository` after `getLatestQaReport`:
|
||
|
||
```ts
|
||
async saveRubricVersion(rubric) {
|
||
await db
|
||
.prepare(
|
||
`insert into rubric_versions (
|
||
id, version, name, dimensions, formula, is_active, created_at
|
||
) values (?, ?, ?, ?, ?, ?, ?)
|
||
on conflict(id) do update set
|
||
version = excluded.version,
|
||
name = excluded.name,
|
||
dimensions = excluded.dimensions,
|
||
formula = excluded.formula,
|
||
is_active = excluded.is_active`,
|
||
)
|
||
.bind(
|
||
rubric.id,
|
||
rubric.version,
|
||
rubric.name,
|
||
serialize(rubric.dimensions),
|
||
rubric.formula,
|
||
rubric.is_active ? 1 : 0,
|
||
rubric.created_at,
|
||
)
|
||
.run();
|
||
return rubric;
|
||
},
|
||
async saveScoringRun(run) {
|
||
await db
|
||
.prepare(
|
||
`insert into scoring_runs (
|
||
id, job_id, revision, rubric_version_id, dimension_scores,
|
||
composite_score, rationale, created_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
)
|
||
.bind(
|
||
run.id,
|
||
run.job_id,
|
||
run.revision,
|
||
run.rubric_version_id,
|
||
serialize(run.dimension_scores),
|
||
run.composite_score,
|
||
run.rationale,
|
||
run.created_at,
|
||
)
|
||
.run();
|
||
return run;
|
||
},
|
||
async getLatestScoringRun(jobId, revision) {
|
||
const row = await db
|
||
.prepare(
|
||
`select * from scoring_runs
|
||
where job_id = ? and revision = ?
|
||
order by created_at desc
|
||
limit 1`,
|
||
)
|
||
.bind(jobId, revision)
|
||
.first<ScoringRunRow>();
|
||
return row ? toScoringRun(row) : null;
|
||
},
|
||
async createPublicationRecord(input) {
|
||
const timestamp = nowIso();
|
||
const record: PublicationRecord = {
|
||
id: `pub_${nanoid(10)}`,
|
||
...input,
|
||
created_at: timestamp,
|
||
updated_at: timestamp,
|
||
};
|
||
await db
|
||
.prepare(
|
||
`insert into publication_records (
|
||
id, job_id, revision, platform, url, published_at, status,
|
||
notes, created_at, updated_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
)
|
||
.bind(
|
||
record.id,
|
||
record.job_id,
|
||
record.revision,
|
||
record.platform,
|
||
record.url,
|
||
record.published_at,
|
||
record.status,
|
||
record.notes,
|
||
record.created_at,
|
||
record.updated_at,
|
||
)
|
||
.run();
|
||
return record;
|
||
},
|
||
async listPublicationRecords(jobId) {
|
||
const result = await db
|
||
.prepare(
|
||
"select * from publication_records where job_id = ? order by published_at desc",
|
||
)
|
||
.bind(jobId)
|
||
.all<PublicationRecordRow>();
|
||
return result.results;
|
||
},
|
||
async getPublicationRecord(id) {
|
||
return db
|
||
.prepare("select * from publication_records where id = ?")
|
||
.bind(id)
|
||
.first<PublicationRecordRow>();
|
||
},
|
||
async savePerformanceSnapshot(snapshot) {
|
||
await db
|
||
.prepare(
|
||
`insert into performance_snapshots (
|
||
id, publication_id, source, window_label, metrics,
|
||
feedback_summary, raw_reference, snapshot_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
)
|
||
.bind(
|
||
snapshot.id,
|
||
snapshot.publication_id,
|
||
snapshot.source,
|
||
snapshot.window_label,
|
||
serialize(snapshot.metrics),
|
||
snapshot.feedback_summary,
|
||
snapshot.raw_reference ?? null,
|
||
snapshot.snapshot_at,
|
||
)
|
||
.run();
|
||
return snapshot;
|
||
},
|
||
async listPerformanceSnapshots(publicationId) {
|
||
const result = await db
|
||
.prepare(
|
||
"select * from performance_snapshots where publication_id = ? order by snapshot_at desc",
|
||
)
|
||
.bind(publicationId)
|
||
.all<PerformanceSnapshotRow>();
|
||
return result.results.map(toPerformanceSnapshot);
|
||
},
|
||
async saveCalibrationEvent(event) {
|
||
await db
|
||
.prepare(
|
||
`insert into calibration_events (
|
||
id, publication_id, scoring_run_id, performance_snapshot_id,
|
||
direction, observations, recommended_action, created_at
|
||
) values (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||
)
|
||
.bind(
|
||
event.id,
|
||
event.publication_id,
|
||
event.scoring_run_id,
|
||
event.performance_snapshot_id,
|
||
event.direction,
|
||
serialize(event.observations),
|
||
event.recommended_action,
|
||
event.created_at,
|
||
)
|
||
.run();
|
||
return event;
|
||
},
|
||
```
|
||
|
||
- [ ] **Step 7: Add D1 representative test**
|
||
|
||
Append to `src/lib/db/__tests__/d1-repository.test.ts`:
|
||
|
||
```ts
|
||
test("saves a manual performance snapshot using D1 prepare and bind", async () => {
|
||
const run = vi.fn().mockResolvedValue({ success: true });
|
||
const bind = vi.fn().mockReturnValue({ run });
|
||
const prepare = vi.fn().mockReturnValue({ bind });
|
||
const db = { prepare } as unknown as D1Database;
|
||
|
||
const repository = createD1Repository(db);
|
||
|
||
await repository.savePerformanceSnapshot({
|
||
id: "perf_123",
|
||
publication_id: "pub_123",
|
||
source: "manual",
|
||
window_label: "T+7d",
|
||
metrics: { views: 1200 },
|
||
feedback_summary: "用户追问案例依据",
|
||
snapshot_at: "2026-07-01T12:00:00.000Z",
|
||
});
|
||
|
||
expect(prepare).toHaveBeenCalledWith(
|
||
expect.stringContaining("insert into performance_snapshots"),
|
||
);
|
||
expect(bind).toHaveBeenCalledWith(
|
||
"perf_123",
|
||
"pub_123",
|
||
"manual",
|
||
"T+7d",
|
||
'{"views":1200}',
|
||
"用户追问案例依据",
|
||
null,
|
||
"2026-07-01T12:00:00.000Z",
|
||
);
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 8: Run repository tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/db/__tests__/repository.test.ts src/lib/db/__tests__/d1-repository.test.ts
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 9: Commit Task 4**
|
||
|
||
```bash
|
||
git add src/lib/db/repositories.ts src/lib/db/repository.ts src/lib/db/sqlite-repository.ts src/lib/db/d1-repository.ts src/lib/db/__tests__/repository.test.ts src/lib/db/__tests__/d1-repository.test.ts
|
||
git commit -m "接入发布校准仓储接口"
|
||
```
|
||
|
||
## Task 5: Calibration API Routes
|
||
|
||
**Files:**
|
||
- Create: `src/app/api/jobs/[jobId]/calibration/score/route.ts`
|
||
- Create: `src/app/api/jobs/[jobId]/publications/route.ts`
|
||
- Create: `src/app/api/publications/[publicationId]/performance/route.ts`
|
||
- Modify: `src/app/api/__tests__/jobs.test.ts`
|
||
|
||
- [ ] **Step 1: Write route tests**
|
||
|
||
In `src/app/api/__tests__/jobs.test.ts`, import the new routes:
|
||
|
||
```ts
|
||
import { POST as scoreJob } from "../jobs/[jobId]/calibration/score/route";
|
||
import {
|
||
GET as listPublications,
|
||
POST as createPublication,
|
||
} from "../jobs/[jobId]/publications/route";
|
||
import { POST as recordPerformance } from "../publications/[publicationId]/performance/route";
|
||
```
|
||
|
||
Append this test inside `describe("job API routes", () => { ... })` after the optimize success tests:
|
||
|
||
```ts
|
||
it("scores a revision, registers publication, and records manual performance", async () => {
|
||
const { job } = await createJobFixture();
|
||
await confirmFactCard(
|
||
request(validFactCard),
|
||
params<{ jobId: string }>({ jobId: job.id }),
|
||
);
|
||
llmMocks.generateValidatedJson
|
||
.mockResolvedValueOnce({
|
||
title: "示例科技 GEO 内容优化方案",
|
||
summary: "示例科技有限公司面向市场团队提供GEO内容优化服务。",
|
||
body_markdown:
|
||
"## 服务能力\n示例科技有限公司提供GEO内容优化服务。\n## 可信依据\n保留事实卡中的8年经验。",
|
||
image_suggestions: [],
|
||
changed_sections: ["title", "body"],
|
||
requires_user_confirmation: [],
|
||
})
|
||
.mockResolvedValueOnce({ checks: [] });
|
||
await optimizeJob(request({}), params<{ jobId: string }>({ jobId: job.id }));
|
||
|
||
const scoreResponse = await scoreJob(
|
||
request({}),
|
||
params<{ jobId: string }>({ jobId: job.id }),
|
||
);
|
||
const scoreBody = (await scoreResponse.json()) as {
|
||
scoringRun: { id: string; composite_score: number };
|
||
};
|
||
|
||
expect(scoreResponse.status).toBe(201);
|
||
expect(scoreBody.scoringRun.composite_score).toBeGreaterThan(0);
|
||
|
||
const publicationResponse = await createPublication(
|
||
request({
|
||
platform: "official_site",
|
||
url: "https://example.com/article",
|
||
published_at: "2026-06-24T12:00:00.000Z",
|
||
notes: "官网首发",
|
||
}),
|
||
params<{ jobId: string }>({ jobId: job.id }),
|
||
);
|
||
const publicationBody = (await publicationResponse.json()) as {
|
||
publication: { id: string; notes: string };
|
||
};
|
||
|
||
expect(publicationResponse.status).toBe(201);
|
||
expect(publicationBody.publication.notes).toBe("官网首发");
|
||
|
||
const listResponse = await listPublications(
|
||
request({}),
|
||
params<{ jobId: string }>({ jobId: job.id }),
|
||
);
|
||
const listBody = (await listResponse.json()) as {
|
||
publications: Array<{ id: string }>;
|
||
};
|
||
|
||
expect(listResponse.status).toBe(200);
|
||
expect(listBody.publications).toHaveLength(1);
|
||
|
||
const performanceResponse = await recordPerformance(
|
||
request({
|
||
window_label: "T+7d",
|
||
views: "1200",
|
||
inquiries: "7",
|
||
feedback_summary: "用户追问案例依据",
|
||
}),
|
||
params<{ publicationId: string }>({
|
||
publicationId: publicationBody.publication.id,
|
||
}),
|
||
);
|
||
const performanceBody = (await performanceResponse.json()) as {
|
||
snapshot: { metrics: { views: number } };
|
||
calibrationEvent: { observations: string[] };
|
||
};
|
||
|
||
expect(performanceResponse.status).toBe(201);
|
||
expect(performanceBody.snapshot.metrics.views).toBe(1200);
|
||
expect(performanceBody.calibrationEvent.observations.length).toBeGreaterThan(0);
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run failing route test**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/app/api/__tests__/jobs.test.ts -t "scores a revision"
|
||
```
|
||
|
||
Expected: FAIL because route modules do not exist.
|
||
|
||
- [ ] **Step 3: Add scoring route**
|
||
|
||
Create `src/app/api/jobs/[jobId]/calibration/score/route.ts`:
|
||
|
||
```ts
|
||
import { NextResponse } from "next/server";
|
||
|
||
import { requireApiAccess } from "../../../../../../lib/api/auth";
|
||
import { GEO_RUBRIC_V1, scoreOptimizedArticle } from "../../../../../../lib/calibration/scoring";
|
||
import { getRepositoryFromRuntime } from "../../../../../../lib/db/repository";
|
||
|
||
interface RouteContext {
|
||
params: Promise<{ jobId: string }>;
|
||
}
|
||
|
||
export async function POST(request: Request, context: RouteContext) {
|
||
const access = requireApiAccess(request);
|
||
if (!access.ok) return access.response;
|
||
|
||
const { jobId } = await context.params;
|
||
const repository = getRepositoryFromRuntime();
|
||
const article = await repository.getLatestOptimizedArticle(jobId);
|
||
const qaReport = await repository.getLatestQaReport(jobId);
|
||
if (!article || !qaReport) {
|
||
return NextResponse.json(
|
||
{ error: "Optimize the article before scoring calibration" },
|
||
{ status: 409 },
|
||
);
|
||
}
|
||
|
||
await repository.saveRubricVersion(GEO_RUBRIC_V1);
|
||
const scoringRun = await repository.saveScoringRun(
|
||
scoreOptimizedArticle({ jobId, article, qaReport }),
|
||
);
|
||
|
||
return NextResponse.json({ scoringRun }, { status: 201 });
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Add publication route**
|
||
|
||
Create `src/app/api/jobs/[jobId]/publications/route.ts`:
|
||
|
||
```ts
|
||
import { NextResponse } from "next/server";
|
||
|
||
import { requireApiAccess } from "../../../../../lib/api/auth";
|
||
import { publicationInputSchema } from "../../../../../lib/calibration/validation";
|
||
import { getRepositoryFromRuntime } from "../../../../../lib/db/repository";
|
||
|
||
interface RouteContext {
|
||
params: Promise<{ jobId: string }>;
|
||
}
|
||
|
||
export async function GET(request: Request, context: RouteContext) {
|
||
const access = requireApiAccess(request);
|
||
if (!access.ok) return access.response;
|
||
|
||
const { jobId } = await context.params;
|
||
const repository = getRepositoryFromRuntime();
|
||
return NextResponse.json({
|
||
publications: await repository.listPublicationRecords(jobId),
|
||
});
|
||
}
|
||
|
||
export async function POST(request: Request, context: RouteContext) {
|
||
const access = requireApiAccess(request);
|
||
if (!access.ok) return access.response;
|
||
|
||
const { jobId } = await context.params;
|
||
const repository = getRepositoryFromRuntime();
|
||
const article = await repository.getLatestOptimizedArticle(jobId);
|
||
if (!article?.revision) {
|
||
return NextResponse.json(
|
||
{ error: "Optimize the article before registering publication" },
|
||
{ status: 409 },
|
||
);
|
||
}
|
||
|
||
const input = publicationInputSchema.parse(await request.json());
|
||
const publication = await repository.createPublicationRecord({
|
||
job_id: jobId,
|
||
revision: article.revision,
|
||
platform: input.platform,
|
||
url: input.url,
|
||
published_at: input.published_at,
|
||
status: "published",
|
||
notes: input.notes,
|
||
});
|
||
|
||
return NextResponse.json({ publication }, { status: 201 });
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Add performance route**
|
||
|
||
Create `src/app/api/publications/[publicationId]/performance/route.ts`:
|
||
|
||
```ts
|
||
import { NextResponse } from "next/server";
|
||
|
||
import { requireApiAccess } from "../../../../lib/api/auth";
|
||
import { createManualPerformanceAdapter } from "../../../../lib/calibration/manual-adapter";
|
||
import { createCalibrationEvent } from "../../../../lib/calibration/scoring";
|
||
import { getRepositoryFromRuntime } from "../../../../lib/db/repository";
|
||
|
||
interface RouteContext {
|
||
params: Promise<{ publicationId: string }>;
|
||
}
|
||
|
||
export async function POST(request: Request, context: RouteContext) {
|
||
const access = requireApiAccess(request);
|
||
if (!access.ok) return access.response;
|
||
|
||
const { publicationId } = await context.params;
|
||
const repository = getRepositoryFromRuntime();
|
||
const publication = await repository.getPublicationRecord(publicationId);
|
||
if (!publication) {
|
||
return NextResponse.json({ error: "Publication not found" }, { status: 404 });
|
||
}
|
||
|
||
const scoringRun = await repository.getLatestScoringRun(
|
||
publication.job_id,
|
||
publication.revision,
|
||
);
|
||
const qaReport = await repository.getLatestQaReport(publication.job_id);
|
||
if (!scoringRun || !qaReport) {
|
||
return NextResponse.json(
|
||
{ error: "Score the optimized revision before recording performance" },
|
||
{ status: 409 },
|
||
);
|
||
}
|
||
|
||
const adapter = createManualPerformanceAdapter();
|
||
const snapshot = await repository.savePerformanceSnapshot(
|
||
await adapter.fetch({
|
||
publication,
|
||
window_label: "manual",
|
||
manualInput: await request.json(),
|
||
}),
|
||
);
|
||
const calibrationEvent = await repository.saveCalibrationEvent(
|
||
createCalibrationEvent({ scoringRun, qaReport, snapshot }),
|
||
);
|
||
|
||
return NextResponse.json({ snapshot, calibrationEvent }, { status: 201 });
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 6: Run route tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/app/api/__tests__/jobs.test.ts -t "scores a revision"
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 7: Commit Task 5**
|
||
|
||
```bash
|
||
git add 'src/app/api/jobs/[jobId]/calibration/score/route.ts' 'src/app/api/jobs/[jobId]/publications/route.ts' 'src/app/api/publications/[publicationId]/performance/route.ts' src/app/api/__tests__/jobs.test.ts
|
||
git commit -m "新增发布校准接口"
|
||
```
|
||
|
||
## Task 6: Frontend Calibration Panel
|
||
|
||
**Files:**
|
||
- Create: `src/components/performance-calibration-panel.tsx`
|
||
- Modify: `src/app/page.tsx`
|
||
- Modify: `src/app/globals.css`
|
||
|
||
- [ ] **Step 1: Create panel component**
|
||
|
||
Create `src/components/performance-calibration-panel.tsx`:
|
||
|
||
```tsx
|
||
"use client";
|
||
|
||
import { useState } from "react";
|
||
|
||
import type { PublishPlatform } from "../lib/domain/types";
|
||
|
||
interface PerformanceCalibrationPanelProps {
|
||
apiAccessKey: string;
|
||
jobId: string | null;
|
||
optimizedRevision: number | null;
|
||
}
|
||
|
||
interface ScoreResponse {
|
||
scoringRun?: { id: string; composite_score: number; rationale: string };
|
||
error?: string;
|
||
}
|
||
|
||
interface PublicationResponse {
|
||
publication?: { id: string; url: string };
|
||
error?: string;
|
||
}
|
||
|
||
interface PerformanceResponse {
|
||
snapshot?: { metrics: Record<string, number> };
|
||
calibrationEvent?: { observations: string[]; recommended_action: string };
|
||
error?: string;
|
||
}
|
||
|
||
export function PerformanceCalibrationPanel({
|
||
apiAccessKey,
|
||
jobId,
|
||
optimizedRevision,
|
||
}: PerformanceCalibrationPanelProps) {
|
||
const [platform, setPlatform] = useState<PublishPlatform>("official_site");
|
||
const [url, setUrl] = useState("");
|
||
const [publishedAt, setPublishedAt] = useState(() =>
|
||
new Date().toISOString().slice(0, 16),
|
||
);
|
||
const [publicationId, setPublicationId] = useState<string | null>(null);
|
||
const [views, setViews] = useState("");
|
||
const [clicks, setClicks] = useState("");
|
||
const [inquiries, setInquiries] = useState("");
|
||
const [feedbackSummary, setFeedbackSummary] = useState("");
|
||
const [message, setMessage] = useState("");
|
||
const [observations, setObservations] = useState<string[]>([]);
|
||
|
||
const disabled = !jobId || !optimizedRevision;
|
||
|
||
async function scoreRevision() {
|
||
if (!jobId) return;
|
||
setMessage("");
|
||
const response = await fetch(`/api/jobs/${jobId}/calibration/score`, {
|
||
method: "POST",
|
||
headers: apiHeaders(apiAccessKey),
|
||
});
|
||
const body = (await response.json()) as ScoreResponse;
|
||
if (!response.ok || !body.scoringRun) {
|
||
setMessage(body.error ?? "评分失败");
|
||
return;
|
||
}
|
||
setMessage(
|
||
`校准评分 ${body.scoringRun.composite_score}/10：${body.scoringRun.rationale}`,
|
||
);
|
||
}
|
||
|
||
async function registerPublication() {
|
||
if (!jobId) return;
|
||
setMessage("");
|
||
const response = await fetch(`/api/jobs/${jobId}/publications`, {
|
||
method: "POST",
|
||
headers: apiHeaders(apiAccessKey),
|
||
body: JSON.stringify({
|
||
platform,
|
||
url,
|
||
published_at: new Date(publishedAt).toISOString(),
|
||
notes: "",
|
||
}),
|
||
});
|
||
const body = (await response.json()) as PublicationResponse;
|
||
if (!response.ok || !body.publication) {
|
||
setMessage(body.error ?? "发布记录保存失败");
|
||
return;
|
||
}
|
||
setPublicationId(body.publication.id);
|
||
setMessage("发布记录已保存。");
|
||
}
|
||
|
||
async function recordPerformance() {
|
||
if (!publicationId) return;
|
||
setMessage("");
|
||
const response = await fetch(`/api/publications/${publicationId}/performance`, {
|
||
method: "POST",
|
||
headers: apiHeaders(apiAccessKey),
|
||
body: JSON.stringify({
|
||
window_label: "T+7d",
|
||
views,
|
||
clicks,
|
||
inquiries,
|
||
feedback_summary: feedbackSummary,
|
||
}),
|
||
});
|
||
const body = (await response.json()) as PerformanceResponse;
|
||
if (!response.ok || !body.calibrationEvent) {
|
||
setMessage(body.error ?? "表现数据保存失败");
|
||
return;
|
||
}
|
||
setObservations(body.calibrationEvent.observations);
|
||
setMessage(body.calibrationEvent.recommended_action);
|
||
}
|
||
|
||
return (
|
||
<section className="panel stack">
|
||
<div className="panel-heading">
|
||
<span>发布表现校准</span>
|
||
<button disabled={disabled} onClick={scoreRevision} type="button">
|
||
生成评分
|
||
</button>
|
||
</div>
|
||
<label>
|
||
<span>发布平台</span>
|
||
<select
|
||
disabled={disabled}
|
||
value={platform}
|
||
onChange={(event) => setPlatform(event.target.value as PublishPlatform)}
|
||
>
|
||
<option value="official_site">官网文章</option>
|
||
<option value="media_article">媒体稿</option>
|
||
<option value="comparison_review">对比评测</option>
|
||
<option value="recommendation_list">推荐榜单</option>
|
||
</select>
|
||
</label>
|
||
<label>
|
||
<span>发布链接</span>
|
||
<input
|
||
disabled={disabled}
|
||
placeholder="https://example.com/article"
|
||
value={url}
|
||
onChange={(event) => setUrl(event.target.value)}
|
||
/>
|
||
</label>
|
||
<label>
|
||
<span>发布时间</span>
|
||
<input
|
||
disabled={disabled}
|
||
type="datetime-local"
|
||
value={publishedAt}
|
||
onChange={(event) => setPublishedAt(event.target.value)}
|
||
/>
|
||
</label>
|
||
<button disabled={disabled || !url} onClick={registerPublication} type="button">
|
||
保存发布记录
|
||
</button>
|
||
<div className="calibration-metrics">
|
||
<label>
|
||
<span>阅读/浏览</span>
|
||
<input value={views} onChange={(event) => setViews(event.target.value)} />
|
||
</label>
|
||
<label>
|
||
<span>点击</span>
|
||
<input value={clicks} onChange={(event) => setClicks(event.target.value)} />
|
||
</label>
|
||
<label>
|
||
<span>询盘</span>
|
||
<input
|
||
value={inquiries}
|
||
onChange={(event) => setInquiries(event.target.value)}
|
||
/>
|
||
</label>
|
||
</div>
|
||
<label>
|
||
<span>反馈摘要</span>
|
||
<textarea
|
||
value={feedbackSummary}
|
||
onChange={(event) => setFeedbackSummary(event.target.value)}
|
||
/>
|
||
</label>
|
||
<button
|
||
disabled={!publicationId}
|
||
onClick={recordPerformance}
|
||
type="button"
|
||
>
|
||
记录表现并生成复盘
|
||
</button>
|
||
{message && <p className="status-text">{message}</p>}
|
||
{observations.length > 0 && (
|
||
<ul className="calibration-observations">
|
||
{observations.map((observation) => (
|
||
<li key={observation}>{observation}</li>
|
||
))}
|
||
</ul>
|
||
)}
|
||
</section>
|
||
);
|
||
}
|
||
|
||
function apiHeaders(apiAccessKey: string) {
|
||
const headers: Record<string, string> = { "content-type": "application/json" };
|
||
if (apiAccessKey) {
|
||
headers["x-api-key"] = apiAccessKey;
|
||
}
|
||
return headers;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Wire panel into page**
|
||
|
||
In `src/app/page.tsx`, import:
|
||
|
||
```ts
|
||
import { PerformanceCalibrationPanel } from "../components/performance-calibration-panel";
|
||
```
|
||
|
||
Add this JSX after `<QaReportPanel report={qaReport} />`:
|
||
|
||
```tsx
|
||
<PerformanceCalibrationPanel
|
||
apiAccessKey={apiAccessKey}
|
||
jobId={jobId}
|
||
optimizedRevision={optimizedArticle?.revision ?? null}
|
||
/>
|
||
```
|
||
|
||
- [ ] **Step 3: Add compact styles**
|
||
|
||
Append to `src/app/globals.css` before the media query:
|
||
|
||
```css
|
||
.calibration-metrics {
|
||
display: grid;
|
||
gap: 0.75rem;
|
||
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||
}
|
||
|
||
.calibration-observations {
|
||
border: 1px solid #e5e9f0;
|
||
border-radius: 8px;
|
||
color: #586174;
|
||
display: grid;
|
||
gap: 0.45rem;
|
||
margin: 0;
|
||
padding: 0.75rem 0.75rem 0.75rem 1.4rem;
|
||
}
|
||
```
|
||
|
||
Inside the existing `@media (max-width: 900px)` block, change the selector:
|
||
|
||
```css
|
||
.workflow-grid,
|
||
.two-col,
|
||
.calibration-metrics {
|
||
grid-template-columns: 1fr;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run a TypeScript/build check**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm run lint
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 5: Commit Task 6**
|
||
|
||
```bash
|
||
git add src/components/performance-calibration-panel.tsx src/app/page.tsx src/app/globals.css
|
||
git commit -m "新增发布表现校准界面"
|
||
```
|
||
|
||
## Task 7: Full Verification
|
||
|
||
**Files:**
|
||
- No source edits unless verification reveals an issue.
|
||
|
||
- [ ] **Step 1: Run focused tests**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm test -- src/lib/calibration/__tests__/validation.test.ts src/lib/calibration/__tests__/scoring.test.ts src/lib/db/__tests__/repository.test.ts src/lib/db/__tests__/d1-repository.test.ts src/app/api/__tests__/jobs.test.ts
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 2: Run full validation suite**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
npm run lint
|
||
npm test
|
||
npm run build
|
||
```
|
||
|
||
Expected: all commands exit 0.
|
||
|
||
- [ ] **Step 3: Scan for public-repo credential risk**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
rg -n "auth\\.token|secretKey|healthsource|sessionid|li_at|web_session|cookie" . --glob '!node_modules/**' --glob '!.next/**' --glob '!.open-next/**' --glob '!deploy/*.toml'
|
||
```
|
||
|
||
Expected: no newly introduced credential values. Documentation-only mentions of cookie safety are acceptable.
|
||
|
||
- [ ] **Step 4: Review migration and local schema parity**
|
||
|
||
Run:
|
||
|
||
```bash
|
||
rg -n "rubric_versions|scoring_runs|publication_records|performance_snapshots|calibration_events" src/lib/db/schema.ts migrations/0002_publication_performance_calibration.sql
|
||
```
|
||
|
||
Expected: all five table names appear in both files.
|
||
|
||
- [ ] **Step 5: Commit final fixes if needed**
|
||
|
||
Only if verification required changes:
|
||
|
||
```bash
|
||
git add <changed-files>
|
||
git commit -m "修复发布校准验证问题"
|
||
```
|
||
|
||
If no changes were needed, do not create an empty commit.
|
||
|
||
## Self-Review
|
||
|
||
- Spec coverage: The plan implements the optional `PerformanceCalibrator`, pre-publish scoring, publication records, manual performance snapshots, adapter boundary through `PerformanceAdapter`, calibration events, sparse metrics, and D1 migration-only persistence.
|
||
- Placeholder scan: No `TBD`, `TODO`, "implement later", or cross-task shorthand remains. Each code-changing task includes concrete code or exact implementation signatures.
|
||
- Type consistency: The plan uses `RubricVersion`, `ScoringRun`, `PublicationRecord`, `PerformanceSnapshot`, `CalibrationEvent`, and `PerformanceAdapter` consistently across validation, service, repository, API, and UI tasks.
|