Session 3 · Wednesday, August 26, 2026

Harnessing AI Agents

Han Zhang and Bing Tian

Make the agent follow your standards in every session. Today you build the course's core deliverable: your own AGENTS.md and one Skill. When a section has something to do, it appears in the panel on the right.

Opening

Today

  • Goal: make the agent follow your standards in every session.
  • Today produces the course's core deliverable.
Why the agent is inconsistent
Your AGENTS.md
Break
Skills
Managing your usage
Wrap

Where we left off

  • You directed a full analysis: clean, describe, model, output.
  • Reading the code caught at least one silent choice.
  • Two open problems from yesterday:
    • long sessions drift
    • every new session starts blank
  • Both have the same fix, and it is today's topic.

Why the agent is inconsistent

Same prompt, different day

  • Run the same request on two different days and you can get:
    • a different table format
    • different standard errors
    • different treatment of missing data
  • Two structural reasons:
    • Every session starts blank, so the RA forgets between meetings.
    • A vague instruction leaves gaps, and the agent fills them with its own defaults.

What the agent decides for you when you do not

  • Unless you specify them, these are the agent's choices, made silently:
    • which standard errors to report
    • how to treat "don't know" and "refused"
    • whether to weight
    • how to round, sort, and label
    • what a table should look like
  • Any one of them can change what your reader concludes.

The fix: a specification that persists

  • Better prompting each morning does not fix this, because the same gaps reopen in the next session.
  • The fix is a written specification the agent reads at the start of every session, before you type anything.
  • The file is named AGENTS.md. Both Claude Code and Codex read it, at the root of your project.
  • Think of it as standard operating procedures for your AI assistant.

What goes into an AGENTS.md

  • Only what applies to every task in the project:
    • the research question and the data
    • methodological standards: weights, standard errors, missing rules
    • file and naming conventions
    • output formats
    • things the agent must never do
  • Keep it short. Write the page of rules you would give any new RA on day one, and nothing more.

Why this is worth the effort

  • Written down, your standards travel: the agent applies them in every session, on your machine or a collaborator's.
  • The agent stops re-guessing your conventions, so it handles the same task the same way each time.
  • In one study across 138 projects, an instructions file cut the agent's mistakes by a third to a half.
An AGENTS.md that encodes your standards is the AI equivalent of a pre-registration.

Your AGENTS.md · do it on the right

The same prompt, two ways

  • Without AGENTS.md. The agent guesses: default standard errors, its own table format, silent choices about missing data.
  • With AGENTS.md. It applies your weighting rule and your table format without being asked.
  • The prompt is identical, and only the context differs.

What to watch in the side-by-side

  • The standard errors: the default against the ones the file requires.
  • The table: the agent's habit against your journal's format.
  • The missing data: silently dropped against handled by your written rule.
  • Nothing about the model got smarter. What changed is that your standards were in the room.

The four sections

  • Write four sections for your own project:
    • Project context. The question, the data, the unit of analysis.
    • Data conventions. Key variables, units, missing-data policy.
    • Output standards. Table format, chart style, what every result must report.
    • Safety rules. What the agent must never do.
  • Concrete beats general in every line: name your variables, your units, your journal.

Example lines: context and data conventions

  • Project context:
    • "Panel survey of 4,000 respondents, waves 2020 to 2024; unit of analysis is person-wave."
    • "The outcome is political participation; the key predictor is education."
  • Data conventions:
    • "All models use survey-weighted standard errors."
    • "Missing codes are -1 to -5; recode to NA before any computation."
    • "Never modify anything in data/raw/."

Example lines: output standards and safety rules

  • Output standards:
    • "Every table reports N and confidence intervals."
    • "Charts use one message per chart, with a full-sentence title."
    • "Summaries follow: Question, Method, Finding, Limitation."
  • Safety rules:
    • "Never fabricate a number; if a value is unavailable, say so."
    • "Always state the sample restriction used."
    • "Flag every result you could not verify."

Skills · do it on the right

What a Skill is

  • A Skill is one routine you repeat often, written down once and invoked whenever you need it.
  • Technically it is a Markdown file in your repo with a description the agent reads.
  • AGENTS.md holds the rules that apply always; a Skill holds one procedure you invoke when needed.
  • Two ways it fires:
    • the agent loads it automatically when the task matches its description
    • or you call it by name, like a command

Skills that work, and skills that do not

  • The bar is specificity:
    • works: "My survey-weighted OLS table, with my SE choices, in my format."
    • fails: "A regression skill."
  • Good candidates from this room:
    • the descriptive panel you produce for every new dataset
    • your standard robustness battery
    • the data-prep sequence for each new wave of your survey
  • If a new RA could not follow the written routine, neither can the agent.

The format you just used is an open standard

  • The Skill you just wrote is a folder holding one plain-Markdown file, SKILL.md. Because it is plain text, you can audit a stranger's in minutes.
  • The format is an open standard (agentskills.io), adopted by roughly 20 agents including Cursor, Codex, and GitHub Copilot, so a Skill written once runs in all of them.
  • One public directory, skills.sh, counts over 1.1 million skill installs.
  • The practical consequence: for many research routines, someone has already written the Skill, and you can read exactly what you would be adopting.

Three collections by social scientists

CollectionAuthorWhat is inside
open-scholar-skillYongjun Zhang, Stony Brook36 skills: the whole workflow
claude-academic-setupByungkyu Lee, NYU22 skills: R, Stata, Python
vibe-research-skillYanbo Zhang, Tufts6 skills around your taste
  • Three examples of what is inside:
    • scholar-citation verifies every reference and checks it against retraction databases.
    • napkin keeps an append-only log of the mistakes you make with the agent.
    • scholar-rag searches your own Zotero library.
  • Yongjun Zhang removed the five "write the whole paper" skills from his public release, to discourage fully automated paper writing.
  • On GitHub: joshzyj/open-scholar-skill, letitbk/claude-academic-setup, Zhangyanbo/vibe-research-skill.

Three ways to install one

  • Clone. Clone the repository, then copy or symlink the skill folders you want. open-scholar-skill ships a bash setup.sh that does the copying for you.
  • Marketplace. Add the marketplace, then install by name. Claude Code shows the token cost and component inventory before anything installs.
  • Cross-agent. Pull from the skills.sh directory into whichever agent you use.
  • Whichever route you take, the result is a folder you can open. A skill in your home skills folder applies to everything you own; a skill inside a repository travels with that project.

Installing a Skill is a trust decision

  • A Skill is instructions your agent will follow, so installing one is closer to hiring than to downloading.
  • Snyk's ToxicSkills study (2026) found prompt injection in 36% of the agent skills it examined, and 1,467 malicious payloads on a single skill registry (ClawHub).
  • A skill body can embed dynamic-context syntax that runs shell commands the moment the skill loads, before any human reads it.
  • Anthropic's own documentation says: "Only install plugins and add marketplaces from sources you trust."
  • The rule in this course: read the SKILL.md before you install, and check the allowed-tools grant and any embedded shell command.

Make it yours

  • An adopted Skill is a draft: copy the folder into your home skills folder, edit SKILL.md, and the change applies immediately, with no restart.
  • Renaming the directory renames the command; rewriting the description field changes when it fires on its own.
  • The cleanest worked example: Yanbo Zhang's my-scientific-taste is a deliberately replaceable placeholder holding your scientific taste, which his other skills consult by name.
  • His rule for where things belong matches the split you built today:
What is true for everyonegoes in a shared Skill
What is true for yougoes in my-scientific-taste
What is true for this projectgoes in AGENTS.md

Managing your usage · do it on the right

The agent bills by the token

  • Everything you send and everything it writes is counted in tokens, and your plan has a limit.
  • A long session, a big file pasted in, or many rounds of back-and-forth all run the count up.
  • Running several agents at once spends it several times as fast.

How to spend less

  • Use a smaller, cheaper model for simple steps, and save the strong one for the hard ones.
  • Give a tight brief so it gets there in fewer rounds.
  • Start a fresh session for a new task, so it is not re-reading an old one.
  • Point it at just the one file it needs.

Wrap

Wrap

  • You now hold the course deliverable in embryo:
    • a working setup
    • a project AGENTS.md that survives a restart
    • one Skill that encodes your own routine
  • Tomorrow you use the same setup for literature and writing, where the mistakes are harder to catch.