Pensieve Docs

Understanding the 10-Stage Pipeline

tutorialpipelineagents

Understanding the 10-Stage Pipeline

Pensieve’s writing pipeline is a fixed, deterministic sequence of 10 stages. Each stage is owned by a single AI agent with a specific job, a set of tools, and a configured number of iteration passes. Agents never skip ahead or loop back unprompted — the code controls the flow.

This article explains what each stage does, what it produces, and how you can configure and control the pipeline.

The Pipeline at a Glance

# Stage Agent Output Core?
1 Topic Refinement Topic Refiner topic_brief.md ✅ Always runs
2 Research Researcher research_notes.md Optional
3 Outline Outliner outline.md Optional
4 First Draft Drafter draft_v1.md ✅ Always runs
5 Content Review Content Reviewer content_review.md Optional
6 Fact Check Fact Checker fact_check.md Optional
7 Style Review Style Reviewer style_review.md Optional
8 Revision Reviser draft_v2.md Optional
9 Final Polish Polisher final.md Optional
10 Consistency Check Consistency Checker consistency_report.md Optional

Core stages (1 and 4) always run. All other stages can be disabled in the workspace configuration.

Stage 1 — Topic Refinement

Input: The brief you entered when creating the project (brief.md) plus your identity memory.
Output: topic_brief.md

The Topic Refiner turns a rough idea into a structured writing brief. It clarifies the angle, identifies the target audience, lists key questions the article should answer, and notes any constraints (word count, tone, scope).

A good topic_brief.md is the foundation that every downstream agent reads. If the brief is vague or misaligned with your intent, it will affect every stage that follows. This is a good place to review and edit before continuing.

Stage 2 — Research

Input: topic_brief.md + your identity memory
Output: research_notes.md

The Researcher uses web_search and fetch_url tools to gather facts, examples, quotes, and sources relevant to the topic brief. It also calls search_topics to check whether any of your existing topic memory files contain useful context.

Research notes are unstructured by design — the agent collects what is relevant without worrying about the final article shape. The Outliner and Drafter decide how to use this material.

Configuration: Set iterations in research to give the agent more passes for thorough coverage. If you disable web search (no SEARCH_API_KEY), the agent falls back to topic memory only.

Stage 3 — Outline

Input: topic_brief.md + research_notes.md (optional)
Output: outline.md

The Outliner structures the article into sections. It decides what goes where, how to sequence the argument, and which research findings to foreground. The outline is the blueprint that the Drafter follows.

If you want to control the structure — for example, if you always write articles in a particular format — edit outline.md before running the First Draft stage.

Stage 4 — First Draft

Input: topic_brief.md + outline.md (optional) + research_notes.md (optional)
Output: draft_v1.md

The Drafter writes the complete first draft of the article. It reads the topic brief, follows the outline (if one exists), and draws on research notes for facts and examples. Your identity and style memory are injected into its prompt, so the draft reflects your voice.

This is the highest-stakes stage — it benefits most from a capable model and multiple iterations. Consider assigning the strong LLM provider in your workspace configuration.

Stage 5 — Content Review

Input: draft_v1.md + topic_brief.md (optional)
Output: content_review.md

The Content Reviewer reads the draft against the topic brief and produces a structured review: what is working, what is missing, where the argument is weak, and what should be cut or expanded. It does not rewrite anything — it only evaluates.

The Reviser (stage 8) reads this review along with the other reviews to produce draft_v2.md.

Stage 6 — Fact Check

Input: draft_v1.md + research_notes.md (optional)
Output: fact_check.md

The Fact Checker identifies specific claims in the draft and verifies them. It uses web_search and fetch_url to look up sources and flags anything that seems incorrect, unverifiable, or missing a citation.

Note: Fact checking is only as good as the search results. Always review fact_check.md yourself before publishing, especially for technical or time-sensitive topics.

Stage 7 — Style Review

Input: draft_v1.md + your identity memory
Output: style_review.md

The Style Reviewer compares the draft to your identity and style guide. It flags passages that don’t match your voice, suggests more fitting alternatives, and notes structural style issues (sentence length, passive voice, jargon, etc.).

This stage is particularly valuable if you maintain a detailed style_guide.md — the more specific your style rules, the more targeted the feedback.

Stage 8 — Revision

Input: draft_v1.md + content_review.md + fact_check.md + style_review.md (all optional)
Output: draft_v2.md

The Reviser synthesizes all three reviews and rewrites the draft. It does not just apply changes mechanically — it makes judgment calls about which feedback to follow and how to integrate it coherently.

draft_v2.md is typically the most substantive output in the pipeline. If only one artifact is worth reading carefully, this is it.

Stage 9 — Final Polish

Input: draft_v2.md (preferred) or draft_v1.md (fallback)
Output: final.md

The Polisher does a final pass focused on readability: tightening sentences, improving transitions, fixing awkward phrasing, and ensuring the opening and closing are strong. It does not change the substance — only the surface.

final.md is your publication-ready artifact.

Stage 10 — Consistency Check

Input: final.md (preferred) + topic_brief.md (optional) + your identity memory
Output: consistency_report.md

The Consistency Checker does a final quality check: are there any claims that contradict the topic brief? Does the voice drift in any section? Are there any formatting inconsistencies? It produces a report rather than a rewrite.

After the pipeline completes, Pensieve also updates your topic memory with a summary of the project — so future articles on the same topic have richer context. See Working with Memory for details.

Execution Control

Pause Modes

Set execution.pause_mode in your workspace configuration to control how the pipeline stops between stages:

Mode Behavior
after_each_stage Pause after every stage. You must click Execute Next to continue.
after_each_iteration Pause after every iteration within a stage. Maximum control.
none Run the entire pipeline to completion without stopping.

Runtime Controls

Regardless of the configured pause mode, you can override at runtime:

  • Execute Next — run the next stage/iteration, then pause
  • Execute All — run to completion, ignoring pause mode
  • Pause — pause after the current stage/iteration finishes
  • Cancel — stop the pipeline immediately (the current stage finishes first)
  • Retry Stage — re-run the current stage (useful after editing its input artifacts)

Resumability

If a pipeline is interrupted (cancelled, error, or manual stop), you can resume it at any time. Pensieve tracks state in status.json in the project directory. When you click Execute Next or Execute All on a partially-completed project, it picks up from where it left off.

Configuring the Pipeline

Enabling and Disabling Stages

In your workspace config.yaml, set enabled: false to skip a stage:

pipeline:
  fact_check:
    enabled: false
  consistency_check:
    enabled: false

Core stages (topic_refinement, first_draft) cannot be disabled.

Iteration Budgets

Increase iterations to give an agent more passes at its task:

pipeline:
  first_draft:
    iterations: 3    # agent can revise its draft up to 3 times
  research:
    iterations: 5    # agent can run more searches

More iterations generally improve quality but increase API costs and runtime.

Using Presets

Presets bundle stage configuration into a named profile. Select a preset when creating a project:

Preset What it does
minimal Core stages only — topic brief + first draft
quick_draft All stages at 1 iteration each; fact check and consistency check disabled
thorough Higher iteration counts across all stages
fiction Fact check disabled; minimal research

See Workspace Configuration for how to create custom presets.

What’s Next