Agentic AI for Academic Research

A Watch-Along Claude Code Mini-Course

Pedro H. C. Sant’Anna

Emory University

April 2026

Agentic AI for Academic Research

Where I’m coming from

A few facts about my last months:

• 5 kids at home (one of them very small)
• 3 papers in revision at top journals
• 3 more papers in active drafting
• A new PhD course I’m teaching for the first time

Something had to give. I started experimenting with Claude Code.

I’m still figuring it out.

Today is me sharing where I am.

What this actually is

Think of agentic AI as a super-RA:

• It reads your files. Runs your code. Edits on disk. Calls external services.
• It can plan steps before acting, and review its own work.
• You still drive. You still check. You still take responsibility for what ships.
• But it expands what’s possible — if you put in the practice.

Faster, better, more reliable. With practice.

What this is not

• ❌ A sales pitch.
  • Really. I have nothing to sell.
• ❌ A tutorial that teaches you Python, R, or Stata.
• ❌ “AI will write your paper.”
  • It won’t, at least not today. You still have to do the thinking. But it can help with the scaffolding.
• ❌ “Your first try will blow your mind.” It won’t either. Keep at it anyway.

How you get better at this

• Experimentation. Try small things first. See what breaks.

• On-the-job practice. Use it on a paper you’re already writing.

• Digging in. Read other people’s setups. Steal what works.

• Expanding the frontier. Once it works for X, ask whether it can also do Y.

• The first time, you’ll find errors. You’ll be picking on the wrong stuff — that’s normal, that’s the practice.

• Over time you build know-how. The agents get better at the things you care about.

What we’re going to do today

I’ll show you how I’ve been using it. Three live demos drawn from my actual workflow:

• A manuscript section I’m trying to improve

• A replication check on numbers in a paper draft

• A simulated peer review on a working paper

• You’re welcome to clone the repo.

• You’re welcome to skip it.

• Mainly: I’d love to hear what works for you afterward.

Roadmap (4 hours, watch-along)

Part	Topic
1	Agentic AI + the CLAUDE.md control center
2	End-to-end demo: improve a manuscript section
3	Building reusable skills (paper exercise)
4	Orchestration + multi-agent systems
5	Research + replication (live demo)
6	Refereeing + review simulation (live demo)
7	Closing + cross-tool strategies

Part 1 — Agentic AI + CLAUDE.md

What I think matters most

• In my experience, the highest-leverage file I write in this stack is not code. It’s a one-page file called CLAUDE.md.
• Everything else — skills, rules, hooks, multi-agent review — kind of derives from it.
• I’ll spend the next 40 minutes showing you what’s in mine and how to think about yours.
• I don’t write CLAUDE.md by myself - I always write it with the agent in a back-and-forth.
• Claude can help you figure out what to put in it.

A few things have shifted

• Replication scrutiny has gone up. AEA Data Editor checks every accepted paper.
• REStud + EJ run formal verification; field journals are heading the same direction.
• Referee reports got more methodical — pre-trends sensitivity, identification robustness,
  “what would change my mind”.
• Agentic AI matured fast. “Tool-using” went from research preview (2024) to production (2025).
• The gap between chatbot and agent ≈ spreadsheet vs programming language.

These don’t make this stuff necessary. They just make the upside of having a workflow bigger.

Where I think this might land

• Most academics I talk to use AI tools as chat replacements — basically a better Google.
• That’s fine, but it leaves a lot of capability on the table.
• The leap from chatbot-AI to agentic-AI is genuinely big — if you set things up to take advantage of it.

I’ll be honest: not all of what I show today is going to be relevant to all of you. Take what fits.

What I’ll be doing

1. Walking through a workflow that plans before acting, reviews its own work, and verifies before shipping.
2. Demonstrating it on real materials: manuscript improvement, replication check, simulated peer review.
3. Sharing the repo (Claude-Mini) and the parent template (claude-code-my-workflow):
   • If you want to clone it and adapt it, go for it.
   • If you want to steal bits and pieces, go for it.
   • If you want to ignore the repo and just take the ideas, also go for it.

• As agents get better at generating, the binding constraint quietly shifts to evaluating what they just produced.

• The workflow is built around that shift.

Tool landscape

The three big players (April 2026)

By April 2026, the agentic primitives have converged.

	Claude Code	Codex CLI (OpenAI)	Gemini CLI (Google)
Best for	Repo-context coding; deepest skills + agents ecosystem	Repo-context coding; sandboxed batch ops; can run as an MCP server	1M context standard; free tier; strongest multimodal input
Agentic primitives	Subagents, hooks (~26 events), MCP, plan mode — all first-class	All first-class (subagents default-on; hooks stable Apr 2026, v0.124)	All first-class (subagents Apr 15; hooks Apr 23, 2026, v0.39)
Skills / plugin ecosystem	Largest community library (awesome-claude-code)	Plugin marketplace (remote browsing in 0.122)	Smaller; subagent-scoped tools
Multimodal input	Images + PDFs via vision	Screenshots, design specs	PDFs, images, sketches; experimental visual agent
Pricing entry	Bundled with Claude Pro $20/mo	Free OSS + API key, or ChatGPT Plus $20	Free tier (Flash only as of Apr 1, 2026); AI Pro $19.99/mo
My experience	Daily driver	Cross-tool audits / second opinion	Occasional, multimodal-only

What this talk uses

We’ll be in Claude Code for everything live. Two reasons:

1. The infrastructure (skills, rules, agents, hooks) is the most mature here in April 2026.
2. The cross-tool pattern at the end (Part 7) uses Codex reviewing Claude output — same primitives generalize.

If you’re already in another tool, most concepts port.

Nothing here is Claude-exclusive in concept.

What “agentic” actually means

A chatbot

“Edit the abstract to mention the panel-data dimension.”

“Here’s a revision: [text]”

You: copy → paste → save → check the bib → repeat.

An agent

“Edit the abstract to mention the panel-data dimension.”

Opens the file → reads the abstract → edit it → saves → runs the bib check → opens the result → reports back.

You: read the diff. Approve.

The difference isn’t intelligence. It’s the ability to act — and to verify it acted correctly.

The four agentic primitives

🛠️ Tool use

Read files, run commands, edit on disk, call APIs.

The agent acts, not just talks.

🧭 Planning

Break a task into reviewable steps before acting.

A safety boundary you control.

🤝 Sub-agents

Focused workers (proofreader, code reviewer, fact-checker) in parallel.

Different agents, different blind spots.

⚙️ Hooks

Deterministic enforcement at lifecycle events.

When “Claude is supposed to” must become “this thing actually happens.”

The next four parts walk through these. Today’s talk in one sentence: a workflow is the pattern that connects them.

How Claude reads input

Markdown is Claude’s native format

Claude’s training corpus is dense in markdown — documentation, GitHub, Stack Overflow, technical blogs. Claude reads headers, code blocks, lists, and tables natively.

What this means in practice:

• ✅ Markdown: highest signal-to-noise. Claude reads it the way you read it.
• 🟡 PDF (via vision): works, but you pay 2–3× the tokens for layout artifacts (page breaks, fonts, columns) that don’t carry meaning.
• 🟡 DOCX / PPTX: XML noise. Convert first.
• 🔴 Scanned-only PDF: OCR layer is extra dependency; accuracy drops.

Rule of thumb: PDF → markdown saves 50–70% of tokens. Same content, less noise.

Convert PDFs first — two tools

Tool	Strength	Use when
microsoft/markitdown	Swiss-army knife: PDF, DOCX, XLSX, PPTX, HTML, EPUB. One-line CLI.	First choice — covers ~90% of cases.
opendataloader-pdf	PDF specialist. XY-cut++ reading order, bounding boxes, 0.907 benchmark accuracy.	Multi-column papers, dense tables, math-heavy PDFs.

markitdown paper.pdf > paper.md           # 90% of cases
opendataloader paper.pdf -o paper.md      # complex layouts

Workflow: convert PDFs as soon as they enter the project. Feed Claude the .md; let the original sit in master_supporting_docs/. There’s a /pdf-to-md skill in this repo that wraps both tools.

CLAUDE.md as control center

A very important file in this stack

CLAUDE.md lives at your project root.

It loads automatically at the start of every Claude session.

It tells the agent what your project is, what the rules are, and what good looks like — before the agent does anything.

It’s ~150 lines. It changes everything downstream.

What’s in mine

# CLAUDE.MD — Causal Panel Data Course

## Core Principles
- Plan first; verify after.
- Quality gates: nothing ships below 80/100.

## Folder Structure
- Slides/ — Beamer .tex
- scripts/R/ — analysis pipeline

## Commands
TEXINPUTS=../Preambles:$TEXINPUTS \
  xelatex -interaction=nonstopmode file.tex
quarto render file.qmd

I’ll open the actual file in a moment.

What that gives Claude

• Folder structure → knows where to put things, what to read
• Commands → knows the exact LaTeX flags, knows quarto render over pandoc
• Quality thresholds → won’t ship below 80/100 score; will halt and ask
• Source-of-truth contract → won’t edit Quarto without syncing Beamer

Live walkthrough — the actual CLAUDE.md

In live demo: opens CLAUDE.md, Claude-paper-DR_weak_overlap.md in editor. Walks line by line.
Audience sees that the file is short, specific, and prescriptive about what Claude can’t guess.

Three sections that always pull weight

1. Commands you’d type — exact CLI invocations Claude can’t infer (build flags, tool versions, environment variables).
2. Folder structure — where things live, with one-line “what’s here” notes per directory.
3. Quality thresholds + source-of-truth contracts — non-negotiables. “Never edit X without syncing Y.”

Three sections that look helpful but aren’t

• ❌ A list of every package you use. Claude knows R packages.
• ❌ “Write clean code.” Claude knows.
• ❌ A re-summary of files Claude can read. Skip — Claude reads them.

Test: if removing the line caused Claude to make a mistake, keep it. If not, cut it.

I often ask Claude to help me better improve: “review my CLAUDE.md following Anthropic best practices.
Let me know how to improve it, and why. Explain to me before you edit the file.”

Effective CLAUDE.md

✅ Include / ❌ Exclude

✅ Include	❌ Exclude
Bash commands Claude can’t guess	Standard language conventions
Code-style rules that differ from defaults	Detailed API docs (link to source)
Testing instructions + preferred runners	Self-evident practices (“write good code”)
Repository etiquette (branch naming, PR conventions)	File-by-file descriptions
Architectural decisions specific to your project	Information that changes frequently
Developer-environment quirks (env vars, paths)	A summary of files Claude can already read
Common gotchas / non-obvious behaviors	Long explanations or tutorials

Specificity wins

• ❌ “Use good R code style.”

• ✅ “Use fixest::feols, not lm. Cluster SE at firm level. set.seed(20260427) once at top of 00_run_all.R.”

• ❌ “Be careful with citations.”

• ✅ “Citations: \citep{key} for parenthetical, \citet{key} for narrative. All keys must exist in Bibliography_base.bib before commit.”

Specificity wins, part 2

• ❌ “Write clean Python.”

• ✅ “Use pandas for data, pyarrow for serialization. Type hints required for public functions. Black formatter.
  pytest -xvs is the test runner.”

• ❌ “The folder structure is intuitive.”

• ✅ “data/raw/ — never modified after download. data/clean/ — outputs of 02_clean.R only.
  results/ — read-only, regenerated by 00_run_all.R.”

An applied-causal example (your CLAUDE.md, draft 1)

# CLAUDE.MD — DiD on Medicaid Expansion

## Core Principles
- Default journal: AER.
- Replication policy: AEA Data and Code Availability Policy.
- Identification: staggered DiD with never-treated comparison.
- Estimator: Callaway-Sant'Anna (2021) ATT(g,t) — primary; TWFE robustness.
- Standard errors: clustered at state level (treatment unit).

## Data Sources
- ASEC monthly CPS, 2008-2022
- State Medicaid expansion dates from KFF
- Sample restrictions: 18-64, non-disabled, household head.

## Commands
- Cleaning: Rscript scripts/02_clean.R
- Estimation: Rscript scripts/03_main_did.R
- Pre-trends test: Rscript scripts/04_pretrends.R (Roth 2022, Rambachan-Roth 2023)
- Replication audit: /audit-reproducibility paper.tex scripts/

That’s enough. Stop there.

A causal R&R example (your CLAUDE.md, draft 2)

# CLAUDE.MD — QJE R&R Round 2

## Core Principles
- Target: QJE R&R round 2.
- Referee 1: alternative parallel-trends tests (Roth 2022 + Rambachan-Roth 2023 sensitivity).
- Referee 2: placebo tests using never-treated states only.
- Referee 3: treatment-effect heterogeneity by state-level covariates.

## Data
- State-year panel, 2000-2022
- Treatment dates: KFF + manual cross-check
- Outcomes: small-business formation (Census BFS), labor-force participation (CPS)

## Workflow
- New analyses go to scripts/r2_response/ (NOT main scripts/).
- Each analysis: ONE script that writes ONE table to tables/r2/.
- Update response-letter.tex with mapping: analysis script -> referee point.
- /respond-to-referees on this round's editor letter when ready.

Length budget

• Aim for 100-200 lines. That’s where most working CLAUDE.md files live.
• Hard ceiling: ~300. Past that, important rules can get lost.
• If yours is over 300: something belongs in a rule file (path-scoped), not the universal CLAUDE.md.

We’ll get to rule files in 5 minutes.

There’s evidence behind this. Gloaguen et al. (2026, ETH Zurich + INSAIT)
138-instance benchmark across 12 Python repos:

• LLM-generated AGENTS.md / CLAUDE.md files reduce task success ~3%; inflate steps + cost 20%+.
• Human-written ones gave only ~4% gains — and only when narrowly scoped to non-inferable details.
• The authors’ lesson: “omit LLM-generated context files entirely.”
• So: write yours by hand, with assistance of Claude, prune ruthlessly

Live: prune my Econ 730 CLAUDE.md (~5 min)

Mine is 1,056 lines — 3.5× over the ceiling I just told you about. Exactly the failure mode the evidence flags.

Live demo (~5 min)

Switching to VS Code on the Econ 730 course repo. We’ll prune CLAUDE.MD in real time:

1. Open the file — pause on the 100-line skill catalog (file-by-file descriptions Claude reads itself).
2. Apply the CLAUDE.md test: comment out a section → restart Claude → ask a normal question → see if anything breaks.
3. Move duplicated content to existing .claude/rules/*.md files (most of it is already there, twice).
4. Target: 1,056 → ~200 lines.

Abort trigger: if VS Code or the repo doesn’t open within 30 seconds, walk through the before/after diff in demos/02-skill-walkthrough/expected-claude-md-prune.md.

Rules and hooks (preview)

When CLAUDE.md gets too long

The fix: path-scoped rule files under .claude/rules/.

• A rule file is just like CLAUDE.md, but loads only when Claude touches a matching file.
• .claude/rules/r-code-conventions.md with paths: "*.R" → loads only when an .R file is in play.
• Keeps CLAUDE.md focused on universal stuff. Specialty knowledge waits for its moment.

A rule example

# .claude/rules/replication-protocol.md
---
paths: ["scripts/**/*.R", "scripts/**/*.do", "scripts/**/*.py"]
---

# Replication-First R Code

When asked to extend an analysis:
1. FIRST: replicate the original numbers in the manuscript.
2. THEN: extend.
3. NEVER: ship a script that produces numbers that don't match the paper.

If the original numbers don't replicate, STOP and assess whether you are doing something wrong. If you cannot reproduce the results, report what differs.

This rule auto-loads when Claude opens any .R / .do / .py file.
Outside that scope, the rule isn’t in context — staying lean.

Hooks: deterministic enforcement

CLAUDE.md and rules are advisory — Claude reads them and (usually) follows them.

Hooks are scripts that fire deterministically at lifecycle events. They don’t ask Claude — they enforce.

• PreToolUse — before any tool runs (block dangerous patterns)
• PostToolUse — after tool completes (sanity check, log, notify)
• Stop — when Claude is about to stop responding (verify, remind)
• PreCompact — before context auto-compaction (save plan to disk)

A hook example

# .claude/hooks/log-reminder.py — Stop hook
# Fires when Claude is about to stop responding.
# If session log hasn't been updated in N responses, remind to update it.

if responses_since_last_log_update > 50:
    print("Session log is stale. Update quality_reports/session_logs/", file=sys.stderr)

This isn’t AI judgment — it’s a Python script. Deterministic. Runs every Stop, no exceptions.

What’s coming next

• Part 2 (next, 30 min): see all of the above run end-to-end on a real manuscript section.
• Part 4 (later, 25 min): when multi-agent helps, when it hurts.
• Part 5 + 6 (after lunch): the headline demos — replication audit + simulated peer review.

Take a 30-second stretch. We’re going to a live demo.

Part 2 — End-to-End Demo

What you’re about to see

• A real working-paper introduction.
• I ask Claude: “Improve the introduction of this paper.” but I will provide a lot more infrastructure than that!
• We watch the full lifecycle — plan → spec → approve → implement → multi-agent review → fix → verify → score.

What to watch for, not what to listen to

The next 25 minutes is infrastructure, not prose.

• I’ll narrate at three checkpoints: plan appears, review fans out, score reports.
• The agent’s edit suggestions matter less than how the agent arrived at them.
• If Claude gets one of the edits wrong, that’s good — you’ll see the review loop catch it.

The full lifecycle (preview)

Step	What happens	Time mark
1	I paste the prompt. Claude enters plan mode.	T+0
2	Plan appears. I narrate.	T+3
3	I approve. Implementation begins.	T+5
4	Agent reads, edits, self-verifies.	T+6
5	Multi-agent review fans out (proofreader + domain-reviewer + claim-verifier).	T+12
6	Review reports surface.	T+18
7	Fixes applied.	T+21
8	Quality score, summary, done.	T+25

What I want you to notice

1. Plan mode is read-only. Claude can’t edit until I approve. This is the safety boundary.
2. The review fanout is parallel. Three agents in three contexts — no cross-contamination.
3. The verifier never sees my prompt. It checks the work in a fresh context (Chain-of-Verification, Dhuliawala et al. 2023).
4. The score is from a deterministic Python script, not the agent’s self-evaluation. Numerical, reproducible.

To the editor

Live demo

Switching to the IDE + terminal. Watch the screen, not the slides.

Coming back from the demo

What you just saw — in one sentence:

A 3-paragraph edit task became a 7-step structured workflow that produced an audit trail.

What got produced — six artifacts, one prompt

📋 Plan

saved to quality_reports/plans/

📝 Spec

MUST / SHOULD / MAY annotated; quality_reports/specs/

✏️ Edits

The actual manuscript-section changes

🔍 Review reports

3 reports (one per agent), quality_reports/

📊 Quality score

numerical, reproducible (quality_score.py)

📓 Session log

who did what, when — for collaborators

Six artifacts from one prompt. None of them existed when I typed it.

Why this matters academic work

• R&R round 2 = “what did I change since round 1?” This audit trail answers it automatically.
• Co-author collaboration: the next person opens the spec + session log and resumes without a 30-min sync call.
• Replication packages: the same lifecycle generates them on the analysis side (Part 5).

What this didn’t do

Honesty matters here.

• ❌ It didn’t write a better paper than I would have. The agent’s edits are suggestions, not authority.
• ❌ It didn’t catch every issue. One subtle error is best caught by a reading mind, not a fast one.
• ✅ It caught most of the surface stuff in 25 minutes, freeing me to spend the next hour on substance.

The point isn’t replacement. It’s leverage on the boring 70%.

What we cover next

We just saw the full lifecycle.

• Part 3 (next, 30 min): what’s a skill? How do you write your own? You’ll sketch one.
• Part 4 (25 min): the orchestration story — when does multi-agent help, when does it hurt?
• Part 5 + 6 (after lunch): the two headline demos — replication audit + simulated peer review.

Part 3 — Building Reusable Skills

What is a skill, really?

• A skill is a Markdown file at .claude/skills/<name>/SKILL.md.
• It has YAML frontmatter (when does Claude trigger it?) and a body (what to do).
• It auto-loads when a relevant prompt is detected — or you invoke it explicitly with /<name>.
• That’s it. No code, no installation, no plugin system.

A skill is just a written instruction — but a triggerable, version-controlled, reusable one.

What a skill is NOT

• ❌ A plugin (no compilation, no extension API)
• ❌ A chat template (skills can read files, run code, call other skills)
• ❌ A prompt library entry (those are static; skills run workflows)
• ❌ Magic (you’ll see exactly what’s in one in 90 seconds)

Anatomy of a skill

---
name: my-skill-name
description: What it does + trigger phrases. "Use when user says X, Y, Z."
allowed-tools: ["Read", "Write", "Bash"]
---

# My Skill Name

## Workflow

### Phase 1 — Read inputs (refuse if invalid)
### Phase 2 — Generate (annotate MUST / SHOULD / MAY)
### Phase 3 — Verify (cross-check domain rules)
### Phase 4 — Output (write to disk; report)

That’s a real skill template. Frontmatter ≤ 4 fields. Body = numbered phases. Other fields (argument-hint, disable-model-invocation) are optional polish — not required.

Why this beats writing the workflow into your prompts

• Reusable — write once, invoke anytime, share with co-authors
• Versionable — git diff your skill the way you git diff your code
• Auditable — anyone reading your repo sees the workflow as text
• Composable — a skill can invoke another skill (e.g., /preregister calls /verify-claims)

Walk-through: /preregister

What it does (one slide)

/preregister drafts a registry-ready preregistration document in your choice of style:

• OSF (default) — full template, social-science default
• AsPredicted — 9-question short form
• AEA RCT Registry — required for AEA-journal field experiments since 2018

It refuses to draft if your description contains results — preregistration is forward-looking by definition.

Live walkthrough: the actual SKILL.md

[Opens ~/Documents/GitHub/claude-code-my-workflow/.claude/skills/preregister/SKILL.md. Walks the frontmatter (description, argument-hint, allowed-tools), then the 6 phases. Then INVOKES it on a fake study description. Audience sees: input → output, end to end, ~3 minutes.]

Demo prompt (verbatim)

I’m planning a 2x2 between-subjects survey experiment on N=800 US adults via Prolific. Manipulating message-source partisanship × policy framing on attitudes toward climate policy.

Primary DV is a 4-item attitude index. Hypothesis: in-party + gain framing produces highest support. Pre-registered analysis: two-way ANOVA.

Run /preregister --style aspredicted on this.

Three things to notice in the SKILL.md

1. Trigger phrases in the description — “preregister”, “OSF”, “AsPredicted”, “AEA RCT”, “PAP”, “preanalysis plan”. Why so many? See Effective triggers below.
2. disable-model-invocation: true — this skill writes a commitment document; it must be user-intent.
3. Cross-check phase — refuses to mark “ready” if hypothesis isn’t directional, estimator isn’t named, exclusions aren’t ex-ante.

Effective trigger descriptions

A skill triggers when its description matches the user’s prompt. So the description must contain the words your future self / your co-author would actually use.

• ❌ description: A preregistration tool. — too generic; never triggers
• ❌ description: Pre-registers a study. — singular, formal; misses real usage
• ✅ description: Draft a preregistration document (OSF, AsPredicted, AEA RCT Registry style). Use when user says "preregister", "draft a preregistration", "OSF preregistration", "AsPredicted", "AEA RCT registry", "PAP", "preanalysis plan".

Skills vs MCPs (a quick aside)

A common question: “How is a skill different from an MCP?”

• A skill is a Markdown instruction Claude reads in-context. Triggered by user phrases. Lives in your repo as a .md file.
• An MCP (Model Context Protocol) is a runtime tool integration — a separate process that connects Claude to external services (databases, APIs, file systems, Notion, JIRA, your home Postgres).

Different layer of the stack. Skills tell Claude how to think; MCPs give Claude more things to talk to.

When to reach for which

Scenario	Use a skill	Use an MCP
“Format my regression tables in AER style”	✅	—
“Pull our latest WRDS query into the analysis”	—	✅
“Draft a preregistration document”	✅	—
“Update my Notion ‘papers under review’ page”	—	✅
“Audit reproducibility against the R script”	✅	—
“Query our shared Postgres database of working-paper revisions”	—	✅

• Most academic workflows are skill-heavy.
• MCPs become valuable when you connect Claude to live external data you don’t already have in your repo.
• Further reading: ai-mba.io/tutorials/skills-vs-mcps.

Where everything fits

CLAUDE.md, skills, MCPs, plugins. People mix them up. Here’s the map:

Piece	Where it lives	What it is	When it loads
CLAUDE.md	Repo root (also ~/.claude/CLAUDE.md for global)	Always-on rules + conventions	Every session, automatically
Skills	.claude/skills/<name>/ (repo) or ~/.claude/skills/ (global)	Workflows triggered by phrases you’d type	When you hint at the trigger
MCPs	External process + config entry	Access to systems Claude can’t otherwise reach	On invocation / auto-discovered tools
Plugins	~/.claude/plugins/ (after /plugin install)	Someone else’s bundle of the above (skills, MCPs, agents, hooks)	Behaves like its components, once installed

Community skills worth knowing about

Beyond claude-code-my-workflow’s 30 skills, two community skills are highly relevant for this audience:

• jusi-aalto/crossref — validates bibliographic references against the Crossref REST API.
  • Returns canonical APA citations with DOIs + match-confidence + diff flags. Pairs naturally with /verify-claims.
• jusi-aalto/strategic-revision — peer-review response planning for empirical accounting + finance R&R rounds.
  • Models reviewer requests as a directed acyclic graph (NetworkX).

What makes a good first skill

• Repetitive task — you do it more than 3 times a year
• Has a structure — there’s a “right way” you can specify
• Has a verifiable output — you’ll know if the agent got it wrong
• Is small — 1-3 phases. The first skill is the smallest skill that’s useful.

Most failures: trying to cram too much into skill #1. Aim small.

Design principles

Reusability

• Specific enough to do something useful, general enough to apply more than once.
• Trap: writing a skill that only solves this paper’s problem. Re-frame to the class of problems.
• Test: would your co-author / next-year-you actually invoke this?

Specificity (in the description AND the body)

• In the description: trigger phrases people actually say, examples of when to use it, when not to use it.
• In the body: literal commands, exact filenames, named estimators. “Run an analysis” → no. “Run Rscript scripts/02_main.R and read the output of Sys.time()” → yes.

Built-in verification

The best skills don’t just do — they check.

• /preregister refuses to mark MUST sections complete if hypothesis isn’t directional.
• /audit-reproducibility compares numerical claims to script outputs with tolerance thresholds.
• /checkpoint reads git status to verify state before writing the snapshot.

A skill without a verification step is a fancy macro. A skill with one is a workflow.

Part 4 — Orchestration + Multi-Agent

The honest framing

• Most “AI for research” demos online ship the first agent output.
• That’s not a workflow — it’s a draft.
• The leverage comes from running the loop: implement → verify → review → fix → re-verify → score.

A model alone is impressive. A model in a loop is useful.

The 6-step orchestrator loop

You saw this shape in Part 2’s demo. Now we generalize.

1. IMPLEMENT   — agent does the work
2. VERIFY      — does it compile / render / parse?
3. REVIEW      — what does a focused critic think?
4. FIX         — apply review findings
5. RE-VERIFY   — did fixes break anything?
6. SCORE       — numerical quality threshold
       │
       ├── Score ≥ threshold → ship
       └── Score < threshold → loop back to step 3 (max 5 rounds)

The point isn’t the specific 6 steps. It’s that steps 2 and 5 have to be deterministic — and that the loop runs automatically until quality clears.

Where this lives in the template

• The orchestrator is a pattern, not a daemon. There’s no background process running it.
• Specific skills implement it: /commit, /qa-quarto, /review-paper --adversarial, /slide-excellence, /create-lecture, /data-analysis, /review-paper --peer.
• Plan approval does NOT auto-trigger the loop. You invoke a skill; the skill runs the pattern within its own scope.

Honesty matters: this is a contract that skills implement, not a magic background service.

When agents help

Three patterns where multi-agent earns its keep

1. Parallel independent work — e.g., three papers to summarize, ten specs to estimate, six sections to proofread.
   • Each agent reads its file and reports back. No coordination needed.
2. Critic + fixer pairs — one finds problems (read-only); a different one applies fixes.
   • They alternate until the critic says “approved.” Eliminates the “I wrote this; it looks fine to me” trap.
3. Fresh-context verification — verifier never saw your draft.
   • Answers verification questions from source material alone. Can’t self-confirm a hallucinated claim.

Pattern 1: parallel independent work

• Three referee reports come in. You spawn three agents in parallel: one per report.
• Each reads its assigned report + the manuscript. Returns a structured response sketch.
• You synthesize.

When it works: the tasks are genuinely independent. When it breaks: the tasks need to share context (then it’s serial, not parallel).

Pattern 2: critic-fixer loop

• The critic reads the artifact (Beamer source, Quarto target, manuscript section). Read-only — it cannot edit.
• The fixer receives the critic’s findings and applies them.
• They alternate. Critic re-audits after each fixer pass. Loop until APPROVED (or max 5 rounds).

Why two agents and not one: the agent that wrote the fix shouldn’t be the one judging it. Role separation = honest review.

Pattern 3: fresh-context verification (CoVe)

• The verifier never sees the draft. It only sees: claims, verification questions, source material.
• Architecturally enforced via context forking — the verifier cannot know what was originally written.
• Adapted from Dhuliawala et al. 2023, Chain-of-Verification Reduces Hallucination in LLMs (arXiv:2309.11495).

Why this matters in practice:

• Suppose that I once spent an evening figuring out that I’d cited the wrong paper for a mechanism I was discussing — if a referee caught it, I could be in trouble.
• CoVe-style verification catches that class of mistake.
• The verifier doesn’t share the writer’s confidence in the original draft.

Three patterns, three failure modes

Pattern	Best for	Failure mode
Parallel	Genuinely independent tasks	Tasks need to share context — coordination chaos
Critic-fixer	Surface-level + structural review	Critic and fixer are the same agent — “self-confirmation”
Fresh-context CoVe	Factual claims (citations, numerical facts)	Forgetting to fork the verifier’s context — verifier confirms what it just generated

When multi-agent doesn’t help

The decomposition trap

• “Spawn 5 agents to refactor this codebase” — sounds good, fails badly.
• Each agent works on a slice; slices interact; no agent has the whole picture.
• You get 5 partial fixes that conflict and an integration nightmare.

Rule: multi-agent is for tasks where the decomposition is clear. If you can’t write the decomposition in 2 sentences, don’t fan out.

When ONE agent is enough

• Most editing tasks. One agent + the orchestrator loop is fine.
• Most analysis tasks. Multi-agent on a regression script is over-engineering.
• Quick fixes. The cost of fanning out exceeds the benefit if the task takes < 5 minutes.

Heuristic: add a second agent only when you can articulate exactly what that agent sees that the first one couldn’t.

The role of hooks

Hooks vs CLAUDE.md vs rules

Mechanism	What it does	Determinism
CLAUDE.md	Tells Claude what your project is	Advisory (Claude reads + usually follows)
.claude/rules/*.md	Tells Claude what’s required for specific files	Advisory + path-scoped
.claude/hooks/*.py	Runs a Python script at lifecycle events	Deterministic — runs every time, no exceptions

Hooks are how you turn “Claude is supposed to…” into “this thing actually happens.”

A real hook in this template

.claude/hooks/log-reminder.py — fires on every Stop event.

• If the session log under quality_reports/session_logs/ hasn’t been touched in N response cycles, prints a stderr reminder.
• Doesn’t block Claude — just nudges.
• Net effect: session logs get updated. Cross-session continuity becomes habit, not afterthought.

Another: pre-commit gates

/commit Step 0b runs scripts/check-surface-sync.sh + check-skill-integrity.py BEFORE writing the commit.

• Surface-sync: count assertions across 6 documents must match disk
• Skill-integrity: 4 mechanical parity classes (frontmatter ↔︎ body, etc.)
• Either fails → commit halts, user fixes, retries

This isn’t AI judgment. It’s a Python script. The same gate runs whether I’m tired or rested.

Mini demo: critic-fixer loop

Live demo (~3 min)

Run /qa-quarto on a Quarto slide with a known content drift from its Beamer counterpart (a typo in a formula). Audience watches:

1. Critic reads both files, flags the drift, hands findings to fixer.
2. Fixer applies the fix.
3. Critic re-audits. APPROVED.

Then run a git commit with no message; watch check-skill-integrity.py block.

Choreography: demos/03-orchestration/README.md.

What you saw

• Two agents, one task, one approval. That’s the shape.
• The fixer can’t approve itself — different role, different context.
• The hook caught a real issue at commit time that no agent would have caught — count drift across documentation.

A workflow I keep coming back to: /deep-audit

/deep-audit — multi-agent orchestration on my own work

When I finish a non-trivial chunk of work — a new lecture, a paper revision, even this deck — I run /deep-audit. Here’s what it does:

1. Phase 0 (mechanical): deterministic checks first — does it render? do internal links resolve? are tables consistent?
2. Phase 1 (4 parallel agents) in fresh contexts:
   • one for correctness (claims match disk reality)
   • one for link health (every URL resolves to claimed content)
   • one for pedagogy + visual (does it teach? does it look right?)
   • one for practicality (does it actually work end-to-end?)
3. Phase 2: I triage findings — genuine bugs vs false alarms.
4. Phase 3: fix. Loop if needed.

I ran this on the deck you’re looking at right now, last week.

What /deep-audit found in this deck (last week)

• 2 P0 ship-blockers: a stale value on one slide that contradicted the demo’s script.
• 3 broken external URLs I’d added without verifying.
• Multiple missing demo materials — The script would have errored live.
• Pedagogy issues: demo stakes that read as sales pitch; closing slides that trailed off into a URL dump.
• A few false alarms — agent thought one of my GitHub URLs was 404 when it was actually fine. Worth knowing the agent’s blind spots.

Why this works for me

• Independence. Each audit agent runs in a fresh context — it never saw me write the deck. So it can’t unconsciously confirm my framing.

• Specialization. Four narrow lenses each beat one broad reviewer. The link-health agent doesn’t get distracted by pedagogy.

• Lived experience. I’ve used this dozens of times now. Most runs find at least one thing I’d not want to ship.

• Cheap. ~10 min per pass. Worth the trade for the catches.

• It’s not a magic trick — just an orchestration pattern applied to my own work on a quick cadence.

• The pattern is reusable. Build your own audit skill with the lenses you care about.

Coming in Parts 5 + 6

• Part 5 (45 min, after lunch): orchestration applied to replication audit
• Part 6 (45 min): orchestration applied to simulated peer review — Editor + 2 referees + editorial decision.

Both demos use Pattern 1 (parallel) + Pattern 2 (critic-fixer) + Pattern 3 (fresh-context). Same primitives, bigger scope.

Part 5 — Research + Replication

A small reflection

A few months back, I had a manuscript bounce because a the simulations numbers in the table did not match the R scripts because I did’t verify them at the time of compilation. Took me a long evening to find it.

I now run this kind of check on every paper before submission. Doesn’t always catch everything, but it catches more than I would on a re-read. Let me show you what it looks like.

Why this part lands hardest for applied-causal work

• AEA Data Editor position created 2018; operational replication verification since 2019.
• Every accepted AEA paper (AER / AEJs / P&P) goes through a numerical reproduction check.
• AEA Data and Code Availability Policy is strict — a single non-reproducing number is grounds for revision.
• Top-5 econ + REStud + EJ all run formal verification at acceptance.
• The trend extends: more journals, tighter tolerances, faster cycles.

This is no longer “good practice.” It’s increasingly a real desk-reject and post-acceptance-revision hazard.

What the AEA Data Editor checks

• Every numerical claim in the manuscript must reproduce from the shipped code + data.
• Tolerance: typically 4 decimal places on point estimates; exact match on N, R², degrees of freedom.
• Discrepancies → major revision required to fix — even after acceptance.

You don’t want to discover the discrepancy after the editor’s email.

Replication-first as a workflow rule

The repo’s .claude/rules/replication-protocol.md says:

When asked to extend an analysis: FIRST replicate the original numbers. THEN extend. NEVER ship a script that produces numbers that don’t match the paper.

This rule auto-loads on *.R, *.do, *.py. Claude reads it at the start of every analysis session.

It’s a rule, not a hook — Claude can override with reason. But the default is replication-first.

Code inventory + comparison

The dependency graph

manuscript.tex ──cites──> Table 2
Table 2        ──from──> scripts/R/_outputs/results.rds
results.rds    ──by──> scripts/R/03_analyze.R
03_analyze.R   ──reads──> data/clean.rds
clean.rds      ──by──> scripts/R/02_clean.R
02_clean.R     ──reads──> data/raw/

A bug in 02_clean.R invalidates Table 2. Reviewing the manuscript without touching the code misses this class of error.

What /review-paper does about it

• Detects script references in the manuscript (\input{scripts/...}, %% source: scripts/..., table names matching script outputs).
• Auto-invokes /review-r on each referenced script (code-quality review).
• Auto-invokes /audit-reproducibility on the paper ↔︎ outputs pair (numeric verification).
• Surfaces cross-artifact findings at the top of the review report.

This isn’t a feature you trigger. It’s the default for /review-paper when scripts are detected.

The cross-artifact finding

## Cross-Artifact Findings

**Scripts reviewed:** 3
**Reproducibility:** FAIL — 1 of 4 numeric claims diverges from script output
**Code quality:** 0 critical, 2 major, 5 minor

### Critical cross-artifact issues
- Paper: Table 2, N = 14,562
  Code:  scripts/R/03_analyze.R:18 → N = 14,489
  Issue: silent `year < 2020` filter drops 73 obs

This block goes at the top of the review report. The paper-side findings come after — because if the numbers don’t reproduce, paper-side polish is moot.

AEA-style packaging

What an AEA-compliant package looks like

replication-package/
├── README.md                  # data sources, software versions, runtime estimate
├── 00_run_all.R               # ONE entry point that produces all outputs
├── data/
│   ├── raw/                   # downloaded, never modified
│   └── clean/                 # outputs of cleaning step
├── scripts/
│   ├── 01_load.R
│   ├── 02_clean.R
│   ├── 03_analyze.R
│   └── 04_figures.R
├── outputs/
│   ├── tables/                # match Table N in manuscript
│   └── figures/               # match Figure N in manuscript
└── sessionInfo.txt            # captured at the end of 00_run_all.R

What the workflow gives you

• /data-analysis scaffolds this structure on a new project.
• Numbered scripts (01_*, 02_*, etc.) — implicit dependency order; AEA Data Editor encourages it.
• 00_run_all.R is the entry point — a Data Editor runs it once, no flags, expects matching outputs.
• sessionInfo.txt captured automatically = R version + package versions reproducible.

You don’t have to remember any of this. The skill enforces the structure.

What the rule enforces

.claude/rules/r-code-conventions.md (v1.6.x) Section 8 — Numerical Discipline:

• No floating-point == comparison
• CDF clamping with named epsilon (eps <- 1e-12)
• Integer literals for counts (1L, not 1)
• Pre-allocated vectors
• Deterministic bootstrap seeding (set.seed(YYYYMMDD) once at the top)
• Explicit na.rm (no defaults)
• No T/F shorthands

These are the bugs that produce “the script ran but the numbers are slightly off” — exactly what AEA Data Editors catch.

A bug class /audit-reproducibility catches that I’d miss

• I write subset(df, year >= 2010) in the cleaning script.
• Manuscript reports N = 14,562.
• Three months later, I add & year < 2020 to drop COVID-era observations for a robustness check.
• Forget to update the manuscript; new N is 14,489 — manuscript still says 14,562.
• I’d never catch this on re-read. The skill catches it in 5 seconds.

Live demo: replication audit

What you’re about to see

• A manuscript-claims.md with 4 numerical claims (sanitized — looks like a real paper draft).
• An analyze.R script that allegedly produced them.
• I run /audit-reproducibility manuscript-claims.md analyze.R.
• The agent reads both, runs the script, compares to claims.
• One claim diverges — the kind of bug re-reading the manuscript wouldn’t catch.

This works most of the time. Sometimes the agent reports false positives or misses things. It’s a check, not a guarantee.

The 4 numerical claims (preview)

# Manuscript Claims to Audit

| ID | Claim | Source |
|----|-------|--------|
| C1 | ATT = 3.2 pp (SE 0.083) | Table 2, col (3) |
| C2 | N = 14,562 | Table 2, col (3) |
| C3 | R² = 0.31 | Table 2, col (3) |
| C4 | First-stage F-stat > 25 | Section 4.2 narrative |

The choreographed time-budget

Step	What happens	Time mark
1	I show the claims file	T+0
2	I show the R script	T+1
3	Run /audit-reproducibility ...	T+2
4	Agent reads both files	T+3
5	Agent runs the R script (~3 min for actual estimation)	T+5
6	Agent compares outputs to claims	T+9
7	Discrepancy surfaced	T+10
8	Generate report	T+12
9	Wrap	T+15

Watch the discrepancy moment

Live demo handoff

Switch to terminal. Run the audit.

The planted bug is filter(year < 2020) at line 18 of analyze.R; manuscript says N = 14,562, script produces N = 14,489.

Choreography: demos/04-replication/README.md.

Coming back from the demo

What you saw:

• The agent read manuscript-claims.md — extracted the 4 numerical claims into a checklist.
• The agent ran analyze.R — captured all output.
• For each claim, compared to the corresponding script output with explicit tolerance thresholds.
• Three claims passed. One failed. (Planted: N = 14,562 vs. actual 14,489.)
• The agent surfaced exactly which line of the script produces the divergent number.

Why this lands for me

• AEA Data Editors do this for every accepted paper. So do JF / REStud reviewers.
• Catching a discrepancy before submission means fixing in private rather than in public.
• The skill costs me ~5 minutes per run. Doesn’t always find something — but when it does, it’s saved me real time.

What it doesn’t replace

• ❌ Conceptual review. The skill checks numbers, not whether the question is interesting.
• ❌ Identification audits. It doesn’t ask “is parallel trends plausible here?”
• ❌ Editorial judgment. It can’t tell you the result is too small to publish.
• ✅ The boring 70%. All of which is mechanical and previously slow.

If you want to try this

1. Pick a paper you’ve already submitted somewhere.
2. Run /audit-reproducibility on the manuscript + replication code.
3. See what comes up.
4. If something comes up — decide what to do with it.

• I usually find something — sometimes a real issue, sometimes a false alarm.
• Both are useful: false alarms tell you what the agent is bad at — which helps you write better skills next time.

What is next

Part 6 — refereeing + review simulation.

Part 6 — Refereeing + Review Simulation

A pre-submission check that surprised me

• I started using this skill expecting it to be useful for surface stuff — typos, clarity.
• The interesting use is disposition modeling.
  • A credibility-skeptic referee would read your paper differently than a policy-relevance referee.
  • The skill simulates both — and shows what each would surface.
• The output isn’t the peer review. It’s a structured practice peer review.
  • Editor + two referees with different priors + an editorial-decision template.

Twenty minutes. Useful as a rehearsal — not a substitute for real referees.

Why this works (and where it doesn’t)

• ✅ Surface-level + structural issues: missing citations, unclear identification, weak first-stage, framing drift.
• ✅ Referee-perspective stress-tests: a “credibility skeptic” reads it differently than a “policy relevance” reviewer.
• ❌ Substantive contribution. The simulation can’t tell you the result is interesting. That’s the editor’s actual job.
• ❌ Field-specific norms beyond what’s in the journal profile.

Use it as a dress rehearsal, not a substitute for real referees.

The shape of what’s coming

One prompt → three agents → one revision plan.

                       ┌──────────────────┐
                       │   /review-paper  │
                       │     --peer AER   │
                       └────────┬─────────┘
                       ┌────────▼─────────┐
                       │  EDITOR AGENT    │  desk + novelty
                       └────────┬─────────┘
                          picks two referees
                          ◄──── ▼ ────►
            ┌──────────────────┐    ┌──────────────────┐
            │   REFEREE 1      │    │   REFEREE 2      │
            │  (CREDIBILITY)   │    │     (POLICY)     │
            └────────┬─────────┘    └─────────┬────────┘
                     ▼     non-overlapping    ▼
                       ┌──────────────────┐
                       │  EDITOR DECISION │ → FATAL /
                       │                  │   ADDRESSABLE /
                       └──────────────────┘   TASTE

The 6-way disposition taxonomy

Every applied-empirics referee has a type. We model them explicitly.

Disposition	Reads it asking…
STRUCTURAL	“Is the model specification right?”
CREDIBILITY	“Is identification airtight?”
MEASUREMENT	“Are the variables the right things?”
POLICY	“Does this matter outside of academia?”
THEORY	“Where’s the mechanism?”
SKEPTIC	“What’s the strongest case against this?”

Two referees with deliberately different dispositions stress-tests the paper from non-overlapping angles.

Journal calibration

The editor reads .claude/references/journal-profiles.md and adjusts:

• Disposition pool weights per journal (AER skews CREDIBILITY + POLICY; APSR skews THEORY; AJPS skews CREDIBILITY + MEASUREMENT)
• Domain-referee dimension weights (AER raises Contribution; QJE raises Identification)
• Methods-referee paper-type rubrics (formal-theory weights model-novelty; survey-experiment weights design + balance)

A simulated AER review should feel different from a simulated AJPS review. The journal is the prior.

What we ship

In claude-code-my-workflow/.claude/references/journal-profiles.md (v1.8.0):

• Econ top-5: AER, QJE, JPE, ECMA, ReStud
• Political science top-3: APSR, AJPS, JOP (added v1.8.0)
• Sociology, public health, psych: template + skeleton — your fork-and-extend

Each profile is ~40 lines. Adding your home journal: 30 minutes once.

“What would change my mind?”

Every MAJOR concern in a referee report from this pipeline must include:

What would change my mind: [specific test, estimator, robustness check, or evidence that would resolve this concern]

• This is a discipline, not a feature. Without it, “I’m worried about endogeneity” is taste; with it, “I’d be convinced if you ran an IV with X as instrument” is actionable.
• Same discipline for both referee agents (domain + methods).
• Without this constraint, the simulated reports drift toward un-actionable adversarialism.

Live demo: /review-paper --peer

What you’re about to see

• A sanitized manuscript snippet — intro + identification section, ~3 pages, a staggered-DiD application.
• I run /review-paper --peer AER.
• The editor agent does a desk review (with a WebSearch novelty probe).
• The editor picks two referees with different dispositions from the AER pool.
• The two referees fan out in parallel.
• The editor synthesizes → editorial decision (FATAL / ADDRESSABLE / TASTE).

Sometimes the simulated review surfaces something I hadn’t thought of. Sometimes it’s noise. It’s a useful first pass.

The choreographed time-budget

Step	What happens	Time mark
1	Show the manuscript snippet	T+0
2	Run /review-paper --peer AER	T+1
3	Editor: desk review + novelty probe	T+2
4	Editor: referee selection (2 dispositions)	T+5
5	Two referees fan out (parallel)	T+6
6	Domain referee report visible	T+11
7	Methods referee report visible	T+13
8	Editorial synthesis	T+15
9	Editorial decision	T+18
10	Wrap	T+20

The headline moment

Live demo handoff

Run /review-paper --peer AER.

This is the demo. The editorial decision moment is what we’re aiming for.

Choreography: demos/05-peer-review/README.md.

What you should have caught

• The disposition pairs were different (e.g., one CREDIBILITY, one EXTERNAL-VALIDITY / POLICY) — generating non-overlapping concerns.
• Each MAJOR concern had a “What would change my mind” clause — actionable.
• The editor’s synthesis classified concerns: FATAL (rejection unless mind-changed) / ADDRESSABLE (revise+resubmit territory) / TASTE (referee preference, not the paper’s fault).
• The decision wasn’t “accept” or “reject” — it was a revision plan.

A 30-second pause

I just got an editorial decision on a paper I haven’t published yet. From a simulated AER editor. With two referee reports. In ~20 minutes.

That’s the demo. The next demo is what you do next — turning that editorial decision into a structured revision plan.

Let’s go.

Drafting referee responses

The flip side

You just saw a simulated peer review. The other half of the workflow:

Now I have a real referee report I disagree with. How do I respond?

/respond-to-referees parses a referee report and maps each concern to a specific revision in the manuscript.

What it produces

• Concern-by-concern table: referee text → classification (Addressed / Partial / Deferred / Disagreement) → manuscript location of the response.
• Drafted response prose for each concern, polite-but-firm, in the project’s voice.
• A coverage summary at the top: X addressed, Y partial, Z deferred, W disagreement.
• Revision-letter scaffold ready to upload.

Why classification matters for R&R

• Addressed — you did the thing. Cite where.
• Partial — you did some of the thing. Cite where; explain what’s missing and why.
• Deferred — you didn’t do the thing this round. Explain why deferred (out of scope, future work, infeasible).
• Disagreement — you don’t think the thing should be done. Explain courteously and substantively.

Top journal editors want to see all four categories used appropriately. Avoiding “Disagreement” looks evasive.

A concrete example — the referee’s concern

A referee writes:

“The first-stage F-statistic of 9 is well below the conventional weak-instrument threshold (Stock & Yogo 2005). The IV results should not be interpreted causally.”

A real concern. The kind of concern that, on a bad day, you spend three hours drafting a response to.

A concrete example — what /respond-to-referees produces

Classification: Addressed.

Where: Manuscript p. 14, footnote 22; Online Appendix B.3.

Drafted response: “We thank the referee for raising the weak-instrument concern.

We now report Anderson-Rubin confidence intervals (Online Appendix B.3) which are robust to weak instruments; the result is qualitatively similar with much wider intervals.

We have added a footnote on p. 14 noting the limitation explicitly.”

Three hours → ten seconds. You still review and edit. But the scaffolding is done.

Alternative: strategic-revision (jusi-aalto)

A community Claude skill from outside our template. Different approach than ours.

• Where /respond-to-referees drafts concern-by-concern responses, jusi-aalto/strategic-revision treats reviewer requests as a directed acyclic graph.
• It identifies dependencies — e.g., “Referee 2’s IV concern depends on resolving Referee 1’s sample-selection concern first”.
• Validates the DAG with NetworkX, sequences work into execution blocks, assigns to co-authors.
• Worth knowing about. Whether it fits your workflow depends on how complex your typical R&R round is.

Different tool for different sub-task: /respond-to-referees for the prose; strategic-revision for the project plan.

Live demo: response letter

Live demo handoff (~8 min)

Take the editor agent’s output from the previous demo (still on screen).

Run:

/respond-to-referees \
  demos/05-peer-review/editor-output.md \
  demos/05-peer-review/manuscript-snippet.tex

Audience watches the agent map each editorial concern to a revision plan with classification + drafted response.

Choreography: demos/05-peer-review/README.md.

What just happened

• The skill read both files.
• For each concern in the editorial decision: classified it; drafted a response; pointed at the manuscript location where the response should land.
• Output is a structured Markdown response document — not a finished revision letter, but the scaffolding for one.
• Net time saved: 4-6 hours per R&R round.

The “What would change my mind” symmetry

• The referee agent attaches “what would change my mind” to MAJOR concerns.
• The response agent must engage with that explicitly: did the revision do that thing? If yes, claim it. If not, explain why.
• This forces a disciplined R&R conversation — the same kind real journals try to enforce, often with mixed success.

Wrap-up

What this session covered

In 45 minutes:

• A simulated peer review of a real working paper, with two differently-disposed referees + an editorial decision
• A drafted response to that editorial decision, with concern-classification + revision mapping
• All saved to disk as a reproducible audit trail

The cost: ~30 min of wall-clock time and one prompt.

Where this fits in your actual workflow

• Pre-submission: dress rehearsal. Catches the obvious 80%; surfaces the 10% that needs work; misses the 10% that needs a real referee.
• R&R round 2: classification scaffolding when you have 8 referee concerns and 2 weeks to respond.
• Co-author conflict: “the simulated review flagged exactly what my co-author has been pushing on. They’re right; here’s the data.”

What it does NOT do

• ❌ Predict whether your paper will be accepted.
• ❌ Tell you the paper isn’t novel (the editor agent has a WebSearch novelty probe, but it’s heuristic — not a substitute for real desk review).
• ❌ Replace the deep, paper-changing engagement of a great real referee.

It’s a first pass. The substance of peer review is still human.

Coming up: the close (25 min)

• Cross-tool strategies — using Claude vs. Codex to audit each other.
• Ecosystem tour — clo-author, claudeblattman, MixtapeTools, and what to read next.
• Three things to do Monday morning.

Final stretch.

Part 7 — Closing + Cross-Tool

The full stack, recap

Layer	What it is	Seen in
CLAUDE.md	The control center — universal project instructions	Part 1
.claude/rules/	Path-scoped, auto-loading domain knowledge	Parts 1, 5
.claude/skills/	Triggerable, version-controlled workflows	Parts 2, 3, 5, 6
.claude/agents/	Specialized review personas (proofreader, referees)	Parts 2, 6
.claude/hooks/	Deterministic lifecycle enforcement	Part 4

Five layers. None of them are AI-magical. All of them are written instructions.

How they connect

                   ┌─────────────────────────┐
                   │      CLAUDE.md          │  always loaded
                   └────────────┬────────────┘
        ┌───────────────────────┼───────────────────────┐
   path-scoped              triggered               lifecycle
   ┌────▼────┐           ┌─────▼─────┐          ┌─────▼─────┐
   │  rules  │           │  skills   │          │   hooks   │
   └─────────┘           └─────┬─────┘          └───────────┘
                               │
                          ┌────▼────┐
                          │ agents  │
                          └─────────┘

The diagram is the whole talk in one picture.

What we just covered

• A mental model for where each piece of the stack lives + why
• Three concrete demos from my own workflow
• A repo you can clone if you want the materials
• A parent template you can fork when you’re ready

If even one of these stuck, I’ll consider the four hours well spent.

Cross-tool strategies

Why use more than one AI tool

• No single tool is best at everything. Claude shines on repo-context tasks; Codex shines on focused-completion problems.
• A tool reviewing its own work has confirmation bias. A different tool, different training data, different failure modes — independent review.
• Cost arbitrage. Cheap Haiku-class models for triage; Opus-class for the hard part; Codex for the tasks where it’s strictly better.

The orchestration story extends: multi-agent across tools, not just within Claude.

The pattern: Claude reviewing Codex

1. Codex generates a 200-line refactor.
2. Claude reads the diff in a fresh context (not the prompt that generated it).
3. Claude flags issues from a different angle than Codex would.
4. You see both views before deciding.

The Codex plugin for Claude Code (openai/codex-plugin-cc, shipped by OpenAI) is exactly this. A different tool reviewing the first tool’s work.

When cross-tool actually helps

• Refactors with subtle correctness implications — different tools see different bugs.
• Code in languages you don’t read fluently — your one tool agreeing with itself isn’t reassurance.
• High-stakes one-shot tasks — you’d rather pay 2x to get 2x review than re-do it.

When it doesn’t

• Routine work. Cost-of-orchestration > value-of-second-opinion. Use one tool, save the round trip.
• Genuinely ambiguous decisions (taste, style). Two tools voting doesn’t tell you which is right.
• Speed-critical iteration loops. Cross-tool adds 30 sec / iteration. On a 50-iteration day = 25 min lost.

Heuristic: cross-tool for one in ten tasks — the high-stakes ones. Not as a default.

Codex plugin: setup in 4 commands

In any Claude Code session inside VS Code:

/plugin marketplace add openai/codex-plugin-cc  # one-time, account-wide
/plugin install codex@openai-codex
/reload-plugins
/codex:setup  # checks Codex CLI; offers install if missing

If /codex:setup says you need to authenticate: !codex login opens a browser for OAuth.

Canonical source: openai/codex-plugin-cc · Walkthrough: handouts/codex-setup.md.

Three modes, three different jobs

Command	What it does	When I reach for it
/codex:review	Read-only review of your diff	Sanity pass on a substantive change
/codex:adversarial-review	Pressure-tests the approach — assumptions, tradeoffs	Before shipping a design choice I’m unsure about
/codex:rescue	Hands a task to Codex (investigate / fix / continue)	When Claude’s stuck and I want a different angle

All three: --wait (foreground) or --background. Track background runs with /codex:status and /codex:result.

What it looks like in practice

The /codex:adversarial-review flow specifically — the cross-tool review I run most:

1. I finish a non-trivial change in Claude Code.
2. I invoke /codex:adversarial-review on the diff.
3. Codex reads the diff in a fresh context (it never saw my prompts).
4. Codex returns: things it would push back on, things it'd want me to verify.
5. I read both views and decide.

No tool gets the last word. The cross-tool review is information, not authority.

A small story from this week

• I asked Claude to do a substantial refactor — about 200 lines across a couple of files.

• I read the diff. It looked fine to me.

• I ran /codex:adversarial-review on it before pushing.

• Codex flagged three things Claude had quietly changed that I hadn’t noticed in my read — one was actually a regression I would have shipped.

• Took an extra 90 seconds. Saved me a follow-up commit.

• This isn’t every commit — I do it on the changes I care about most.

• The discipline: knowing when to spend the extra 90 seconds, and when not to.

A live-mini-demo (if time)

Optional demo (~3 min)

Run /codex:adversarial-review on a small refactor.

Audience watches: 1. Codex generates the refactor (already done, in the demo bundle) 2. Claude opens the diff fresh, reviews, surfaces 2-3 issues 3. Pedro narrates: “I’d have shipped this. Codex’s tool agreed with itself.”

Choreography: demos/06-cross-tool/README.md.

Ecosystem tour

You’re not alone in this

Two complements I’d point you at first:

• Paul Goldsmith-Pinkham × Princeton — Yale finance prof, multiple-episode video series for applied economists.
  • Different format, different angle.
• Claes Bäckman’s written guide — ~3,500-word practical companion.

Six more high-quality projects (clo-author, strategic-revision, crossref, claudeblattman, MixtapeTools, autoresearch, AI MBA) are in handouts/resources.md — won’t fit on a slide; will fit on your reference card.

Anthropic-shipped utilities

Built into Claude Code itself, no fork needed:

• /team-onboarding (Week 15 digest) — package your local setup into a replayable guide for collaborators.
  • Recommended for lab groups.
• /autofix-pr — runs Claude Code on a GitHub PR, applying CI-style fixes
• /powerup — interactive lessons (Anthropic’s tutorial system)
• Ultraplan (docs) — cloud-based plan editing, sync to local
• /loop self-pacing — recurring or self-paced background tasks
• /less-permission-prompts — diagnose permission-prompt fatigue

Cross-tool: Codex plugin (Part 7’s earlier demo)

Canonical: github.com/openai/codex-plugin-cc (OpenAI) · Announcement: OpenAI Developer Community.

• “It is Codex, just invoked from inside Claude Code” — reuses local Codex CLI auth, config, MCP setup.
• Three use cases: /codex:review (standard), /codex:adversarial-review (challenge the approach), /codex:rescue (delegate a task to Codex when Claude’s stuck).
• We saw /codex:adversarial-review in the optional Part 7 demo. Setup walkthrough in handouts/codex-setup.md.

Resources + start

If you only do one thing this week

Open the directory of a paper you’re working on. Try writing a CLAUDE.md. ~30 minutes.

That’s the smallest experiment I’d recommend. Once it exists, everything else in this stack has somewhere to anchor.

(If you want the slides and demos: clone Claude-Mini. Full template: fork claude-code-my-workflow. But neither of those changes a paper by itself.)

Claude-Mini’s .claude/ is a starter kit

This repo ships a curated set: 16 skills, 10 agents, 10 rules, 2 hooks — adapted for general academic use.

All of it is imported from claude-code-my-workflow, which is the maintained reference template.

Claude-Mini is a snapshot, not a mirror. Bug fixes and new skills land in the parent first. Don’t expect this repo to track them in real time.

Practical advice: clone Claude-Mini to learn the patterns; fork claude-code-my-workflow once you’re ready to use this seriously.

When you’re ready, three places to extend

When the CLAUDE.md habit is in place — these are next, in priority order:

1. Pick one repetitive task you do 3+ times a year. Write it as a skill. (Sample use cases on the handout.)
2. Run /audit-reproducibility on your most recent submitted paper. Find the issue. Fix it before someone else does.
3. Try /review-paper --peer AER (or your target journal) on a current draft. Treat the simulated review as a dress rehearsal.

These are menu, not assignments. Pick one when it’s the right week.

Honest closing

• This workflow has saved me probably 100+ hours in the last year.
• It’s also occasionally infuriating — agents misunderstanding, hooks firing inconveniently, demos failing live.
• The leverage is real. The friction is also real. You’ll calibrate by doing.
• The community of academics adopting this is small but growing fast. You’re early.

Resources for going deeper

Anthropic canon (read first if pressed for time):

• Best Practices — short, current, very good
• What’s New — weekly feature digest
• Permission modes, Hooks, Memory / CLAUDE.md, Skills

Resources for going deeper (community + academic)

• Goldsmith-Pinkham mini-series: Princeton BCF series — 5 episodes, applied-econ angle
• The claude-code-my-workflow guide: psantanna.com — full walkthrough
• TROUBLESHOOTING.md in the same repo — most issues catalogued
• Hugo Sant’Anna’s clo-author: github.com/hugosantanna — paper-authoring-focused fork
• AI MBA tutorials: ai-mba.io — applied tutorials and curriculum
• Take-home handouts/resources.md — every link from this talk in one file

Thank you

Thanks for spending your day with me. I’d genuinely love to hear what works for you.

Materials: github.com/pedrohcgs/Claude-Mini

Reach me: pedro.santanna@emory.edu · psantanna.com · @pedrohcgs