From 22c824d0dd72024145f22a64d48e50df9024ef64 Mon Sep 17 00:00:00 2001 From: Codex Date: Wed, 24 Jun 2026 10:37:24 +0800 Subject: [PATCH] =?UTF-8?q?=E5=AE=8C=E5=96=84=E5=8F=91=E5=B8=83=E8=A1=A8?= =?UTF-8?q?=E7=8E=B0=E6=A0=A1=E5=87=86=E8=AE=BE=E8=AE=A1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...6-16-geo-agent-article-optimizer-design.md | 256 +++++++++++++++++- 1 file changed, 253 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-06-16-geo-agent-article-optimizer-design.md b/docs/superpowers/specs/2026-06-16-geo-agent-article-optimizer-design.md index 3fe718c..56ca63f 100644 --- a/docs/superpowers/specs/2026-06-16-geo-agent-article-optimizer-design.md +++ b/docs/superpowers/specs/2026-06-16-geo-agent-article-optimizer-design.md @@ -19,6 +19,7 @@ The MVP is a local web application: 7. Failed checks trigger targeted rewriting for up to two rounds. 8. User previews optimized content and QA report. 9. User downloads Markdown and a basic Word document. +10. User can optionally register where an optimized revision was published and later record real performance data for calibration. ## Explicitly Out Of Scope @@ -29,6 +30,7 @@ The MVP is a local web application: - Direct `.docx` upload parsing. - Complex Word template layout. - Automatic use of unconfirmed facts. +- Automatic platform performance adapters in the first calibration release. The first release records performance manually while keeping an adapter interface for later. ## User Flow @@ -236,6 +238,34 @@ Examples: - Delete or mark unsupported claims for hallucination risk. - Warn instead of rewriting when image-text confidence is low. +### PerformanceCalibrator + +Purpose: turn exported GEO articles into a measurable quality-improvement loop after publication. + +This node is optional and runs after the optimization/export workflow. It does not rewrite the article, does not change the confirmed fact card, and does not block export. It records a pre-publication scoring snapshot, a publication record, post-publication performance snapshots, and calibration observations that can later improve GEO scoring rubrics. + +Inputs: + +- Optimized article revision. +- QA report for the same revision. +- Publish platform and optional URL. +- Manual performance data in the first release. + +Outputs: + +- `scoring_run` +- `publication_record` +- `performance_snapshot` +- `calibration_event` + +Rules: + +- Calibration is append-only for a published revision. Later data imports create new snapshots instead of overwriting earlier ones. +- The first release uses manual data entry only. +- The service boundary must support future adapters that return the same `PerformanceSnapshot` shape. +- Adapter code must never store platform cookies, tokens, or login state in the public repository. +- Calibration observations can recommend rubric changes, but rubric changes require a separate reviewed migration or plan. + ## Quality Gates | Rule ID | Issue Prevented | First Version Behavior | @@ -272,9 +302,108 @@ Examples: - Useless content. - Third-party voice when platform is official site. +## Publication Performance Calibration + +The project can borrow the useful part of `cheat-on-content`: content quality should become a measurable loop, not a one-time rewrite. GEO's version keeps the web app and database model, and adds a productized calibration layer instead of copying the external skill's file-based workflow. + +First-release flow: + +```mermaid +flowchart LR + A["Optimized Article Revision"] --> B["Pre-Publish Scoring"] + B --> C["Publication Record"] + C --> D["Manual Performance Snapshot"] + D --> E["Calibration Event"] + E --> F["Rubric Improvement Backlog"] +``` + +### Pre-Publish Scoring + +Before or after export, the system can score an optimized revision against a GEO rubric. The first rubric should focus on business article quality rather than viral-video prediction. + +Suggested first dimensions: + +| Dimension | Meaning | +| --- | --- | +| `fact_integrity` | Whether names, products, years, cases, and claims stay inside the confirmed fact card. | +| `platform_fit` | Whether the output matches official site, media article, comparison review, or recommendation list expectations. | +| `search_intent_fit` | Whether the article answers the likely GEO/search intent behind the topic. | +| `answer_density` | Whether the article gives useful, specific information instead of vague promotional filler. | +| `trust_signal_quality` | Whether credibility signals are specific, sourced, and not exaggerated. | +| `readability` | Whether title, summary, and body are clear enough for customers and AI answer engines. | + +The score is stored as a snapshot. It is not a replacement for QA gates. QA gates protect factual safety; scoring provides a baseline for later performance learning. + +### Publication Records + +A publication record links one optimized article revision to where it was published. + +Fields: + +- Job ID. +- Optimized revision. +- Platform. +- URL. +- Published at. +- Publication notes. +- Status: draft, published, archived. + +The same optimized revision may have multiple publication records if a user republishes it on multiple channels. + +### Manual Performance Snapshots + +The first release records post-publication performance manually. This avoids platform login, anti-scraping, and credential risk while validating the calibration loop. + +Baseline fields: + +- Views or reads. +- Impressions, when available. +- Clicks or inquiry actions, when available. +- Likes, comments, shares, saves, when available. +- Average ranking or citation position, when the user can observe it. +- Snapshot window, such as T+1d, T+3d, T+7d, or custom. +- Comment or feedback summary. +- Data source: `manual`. + +Manual snapshots should allow missing metrics. Different platforms expose different numbers, and forcing fake zeroes would corrupt later calibration. + +### Adapter Boundary + +Future adapters must write the same performance shape as manual entry: + +```ts +interface PerformanceAdapter { + source: string; + fetch(input: AdapterFetchInput): Promise; +} +``` + +Adapter output is normalized before storage: + +- `source`: `manual`, `adapter:xhs`, `adapter:bilibili`, `adapter:wechat`, or similar. +- `metrics`: sparse numeric metrics. +- `snapshot_at`: ISO timestamp. +- `window_label`: human label such as `T+3d`. +- `raw_reference`: optional safe reference to adapter output, never raw cookies or credentials. + +The first implementation should include a `manual` adapter only. Platform adapters are later work and must keep credentials out of Git, D1, logs, and public artifacts. + +### Calibration Events + +A calibration event compares the pre-publish scoring snapshot, QA report, and performance snapshot. + +Examples: + +- High `fact_integrity` and high `answer_density` correlate with stronger inquiry clicks. +- Weak `platform_fit` correlates with poor engagement on media articles. +- QA warning on `hallucination_risk` did not affect traffic but increased manual review burden. +- Articles with high readability but low trust-signal quality received views but no inquiries. + +Calibration events should be written as observations, not automatic rubric changes. A later rubric update must be reviewed separately and applied through migrations/tests so historical data remains interpretable. + ## Data Model -The first version uses local SQLite plus an export folder. +The first version uses local SQLite plus an export folder. Cloudflare deployments use D1/R2 bindings with migration-only schema changes; local and online data remain explicitly separated. ```text data/ @@ -392,6 +521,115 @@ Quality checks for one revision. } ``` +### `rubric_version` + +A versioned scoring rubric for GEO article performance calibration. + +```json +{ + "id": "rubric_geo_v1", + "version": "v1", + "name": "GEO article performance rubric", + "dimensions": [ + { + "id": "fact_integrity", + "label": "事实一致性", + "weight": 2, + "description": "事实、公司名、产品名和经验年限是否遵守事实卡" + } + ], + "formula": "weighted_average_0_to_10", + "is_active": true, + "created_at": "2026-06-24T10:00:00+08:00" +} +``` + +### `scoring_run` + +A scoring snapshot for one optimized article revision. + +```json +{ + "id": "score_xxx", + "job_id": "job_xxx", + "revision": 2, + "rubric_version_id": "rubric_geo_v1", + "dimension_scores": { + "fact_integrity": 5, + "platform_fit": 4, + "search_intent_fit": 4, + "answer_density": 3, + "trust_signal_quality": 3, + "readability": 4 + }, + "composite_score": 7.8, + "rationale": "事实一致性强，平台适配较好，但信任信号仍偏泛。", + "created_at": "2026-06-24T10:05:00+08:00" +} +``` + +### `publication_record` + +A publication instance for an optimized revision. + +```json +{ + "id": "pub_xxx", + "job_id": "job_xxx", + "revision": 2, + "platform": "official_site", + "url": "https://example.com/articles/geo-optimization", + "published_at": "2026-06-24T12:00:00+08:00", + "status": "published", + "notes": "官网文章首发" +} +``` + +### `performance_snapshot` + +One post-publication performance measurement. + +```json +{ + "id": "perf_xxx", + "publication_id": "pub_xxx", + "source": "manual", + "window_label": "T+7d", + "metrics": { + "views": 1200, + "impressions": 4300, + "clicks": 86, + "inquiries": 7, + "likes": 18, + "comments": 3, + "shares": 5, + "saves": 11 + }, + "feedback_summary": "用户主要询问服务流程和案例真实性。", + "snapshot_at": "2026-07-01T12:00:00+08:00" +} +``` + +### `calibration_event` + +An observation linking scoring, QA, and real performance. + +```json +{ + "id": "cal_xxx", + "publication_id": "pub_xxx", + "scoring_run_id": "score_xxx", + "performance_snapshot_id": "perf_xxx", + "direction": "better_than_expected", + "observations": [ + "高 answer_density 的段落带来更多服务流程咨询。", + "trust_signal_quality 偏低，用户仍追问案例依据。" + ], + "recommended_action": "后续 rubric 提高 trust_signal_quality 权重前，先积累至少 5 篇同类样本。", + "created_at": "2026-07-01T12:10:00+08:00" +} +``` + ## Error Handling ### Fact Extraction @@ -408,10 +646,18 @@ Examples: ### QA Failure -Hard failures block export. Warnings allow export with visible confirmation prompts. +QA failures do not hide exports. Hard failures and warnings are surfaced as risk signals with visible confirmation prompts, so the user can still download artifacts for review or customer handoff. Failed checks trigger targeted rewrite for up to two rounds. After two failed rounds, the app stops rewriting and shows manual review fields. +### Performance Data + +Manual performance snapshots accept sparse metrics. Missing metrics are stored as absent values, not zeroes. + +Adapter failures must degrade to manual entry. The app should show the source and failure reason, but it must not block the user from recording performance data manually. + +Calibration events never rewrite published articles automatically. They create a reviewable backlog for future rubric changes. + ## Acceptance Criteria The MVP is complete when: @@ -421,8 +667,12 @@ The MVP is complete when: 3. Confirmed fact card can be saved and reused as a local brand template. 4. System can generate an optimized article without changing confirmed facts. 5. System can generate a structured QA report for the 10 quality gates. -6. Hard failures block export until fixed or manually reviewed. +6. Hard failures and warnings are visible in the QA report without hiding export links. 7. User can download Markdown and a basic Word document. +8. User can create a publication record for an optimized revision. +9. User can manually record a post-publication performance snapshot. +10. System can create a calibration event that compares score, QA findings, and performance. +11. The performance collection boundary can later support platform adapters without changing the stored snapshot shape. ## Minimum Test Samples