Harnessing AI Agents

Today

• Make the agent follow your standards in every session.
• Today we produce the main result of the course.

Where we left off

• You gave directions for a full analysis and said to clean, describe, model, and output.
• Reading the code showed at least one silent choice.
• Two open problems from yesterday:
  • long sessions drift
  • every new session starts blank
• Your instruction files fix both, and they are today's topic.
• Last night, you wrote three instructions and repeated them to the agent. These three lines are the first draft and you will write this file this morning.
• You noted one procedure you want to run again, and that procedure is the Skill you will write this afternoon.

Everything the model says is a prediction

• LLMs predict the next word for every answer, and this prediction is probabilistic, so the same input can produce different outputs.
• Code is different, and the same input always produces the same output.
• Today’s tools — instruction files, memory, skills — push the agent toward consistent behavior.
• They are still instructions for a probabilistic model: soft constraints, not guarantees.
• If a step must always work the same way, write it as code and let the agent call the code. The agent decides when to run it and the code decides what happens.

The same prompt gives different results

• Run the same request on two different days and you can get:
  • a different table format
  • different standard errors
  • different treatment of missing data
• Two structural reasons:
  • Every session starts blank, so the RA forgets between meetings.
  • Vague instructions leave gaps, and the agent fills these gaps with its own defaults.

Agent Autonomy

• Unless you specify them, these are the agent's choices, made silently:
  • which standard errors to report
  • how to treat "don't know" and "refused"
  • whether to weight
  • how to round, sort, and label
  • what a table should look like
• Each one can change your reader's conclusion.

Global and Project Instructions

• The agent does not know you, so it fills gaps with default settings, and these settings belong to the average analyst, which is not you.
• Better prompting each morning does not fix this, and the same gaps reopen in the next session.
• The fix is written instructions, which the agent reads at the start of every session. There are two levels:
• Project instructions live at the project root. They hold the project rules and travel with the repository when you share it.
• Global instructions live in your home folder and hold rules about you, which include your language, your software, and your reporting habits, and apply to every project on your machine. They are personal and never shared.
• The names change depending on the tool, but the structure stays the same:

• Where this page says "your instructions file," use your tool’s name.
• The two instructions files are files you write, and no tool creates them for you. If you do not see a CLAUDE.md, you have not written one yet.
• The memory row is different: the agent writes those notes itself. The next card explains it.

Memory (Claude Code)

• Claude Code also keeps memory, which are notes about you and the project. They stay across sessions.
• The agent writes a note when it learns something and when you say "remember this."
• /memory shows the notes. You can edit or delete them.
• The notes live in ~/.claude/projects/, in one folder per project, not in the project itself. A MEMORY.md file there is the index, and the agent reads it at the start of every session.
• Rules you want enforced go in instructions. Memory holds the rest.
• Codex keeps memory the same way, in ~/.codex/memories/, with the same index-plus-notes structure.

You control the memory

• Control lives in one command: /memory in Claude Code, /memories in Codex. The notes are plain text files on your machine, and any editor can read, edit, or delete them.
• The coding app is less convenient: it does not show the memory commands. But the files are still on your machine, so everything else works.
• The chat apps are a useful contrast. There, memory lives on the server: it is on by default, it reads your whole chat history, and you cannot make the private mode your default. You can edit single memories in the settings, but you never hold the files.
• The coding tools are easier to control, at least in the terminal, because the memory is files on your machine. The line that matters is not app versus terminal; it is whose machine holds the files.
• In both, you can use simple words to control it and ask: "show me your memories," "delete the one about X," or "turn off automatic memory." The agent manages its own files.
• At the start of a session only the index loads, and full notes load only when a task needs them.
• Memory stays on one machine: notes on your desktop computer do not follow you to your laptop.

Instructions vs. Memory: When to use each

• A rule you want enforced goes in instructions: "cluster standard errors by county."
• Memory keeps the rest, for example:
  • you prefer R
  • the income variable is messy
  • why you chose logit over probit
• If you keep correcting the same thing, put the correction in the instructions. Memory might notice it and Instructions guarantee it.
• Memory is personal and it stays on your machine. A collaborator clones your project. They get your project instructions but do not get your memory. Put anything the team needs in the instructions file.

Scope and limits

• What the instructions file changes: the output follows your choices instead of the agent’s guesses. Less guessing means fewer differences between the morning run and the afternoon run.
• What it does not change: the model is still probabilistic. A rule can still be ignored, especially late in a long session, and two runs are still not identical.
• The file does not replace checking, and it does not make a result reproducible. Reproducibility lives in the committed script — the file only shapes how that script gets written.

Evidence

• Write down your standards and the agent applies them in every session and this happens on your machine or a collaborator's.
• The agent stops guessing your conventions, and you spend less time in each session correcting the same things.

What the agent sees: the context

• The context is everything the model reads before its next reply, and nothing else exists for it.
• The context includes: your instruction files, the current conversation, the files it read and the output from every command it ran.
• It does not see these things: files it has not read, your other terminal windows, or past sessions.
• The context has a size limit and fills up eventually. The agent then summarizes the conversation and continues from that summary, and details can be lost at that point.

Context management commands

• You met these commands on Monday and now you can see what they do.
• /compact replaces the conversation with a summary, which saves space but loses detail.
• /clear empties the conversation. Your instruction files and memory stay, because they live on disk.
• claude --resume loads a past conversation into the context, and this is how a new session sees an old one.

The conversation is also a file

• Each session saves as a file on disk, and it has one line per message. Nothing is temporary.
• In Claude Code, these files are under ~/.claude/projects/, and in Codex, they are under ~/.codex/sessions/.
• claude --resume loads one of these files back into the context and the agent continues as if the session never ended. codex resume does the same.
• All four layers are files and include instructions, memory, skills, and the conversation, and the only difference is when each layer enters the context.

Subagents: a fresh context, on purpose

• The agent can send a subagent, which does one task.
• The subagent begins with a new context, which contains only the task description and cannot see your conversation.
• It reports only its conclusion, and a search through fifty files does not fill your context with fifty files.
• This has the same idea as /compact. Keep the main context small and leave details out of it.

The same prompt, two ways

• Without instructions. The agent makes guesses. It uses default standard errors, its own table format, and makes silent choices about missing data.
• With instructions. It uses your weighting rule and your table format, and you do not have to ask.
• The prompt is the same, but only the context is different.

What to watch in the side-by-side

• The standard errors: the default against the ones the file requires.
• The table: the agent's habit against your journal's format.
• The missing data: silently dropped against handled by your written rule.
• The model is no smarter. What changed is that your standards were in the file the agent reads.

The four sections

• Write four sections for your own project:
  • Project context. The question, the data, the unit of analysis.
  • Data conventions. Main variables, units, and rules for missing data.
  • Output standards. Table format, chart style, what every result must report.
  • Safety rules. Things the agent must never do.
• Write specific lines, name your variables, name your units, and name your journal.
• Keep it to one page: the rules you would give any new RA on day one, and nothing more.

Instructions file examples: context and data conventions

• Each line below is a real line from an instructions file (CLAUDE.md or AGENTS.md). Paste and adapt.
• Project context:
  • "Panel survey of 4,000 respondents, waves 2020 to 2024; unit of analysis is person-wave."
  • "The outcome is political participation; the key predictor is education."
• Data conventions:
  • "All models use survey-weighted standard errors."
  • "Missing codes are -1 to -5; recode to NA before any computation."
  • "Never modify anything in data/raw/."

Instructions file examples: output standards and safety rules

• Output standards:
  • "Every table reports N and confidence intervals."
  • "Charts use one message per chart, with a full-sentence title."
  • "Summaries follow: Question, Method, Finding, Limitation."
• Safety rules:
  • "Never fabricate a number; if a value is unavailable, say so."
  • "Always state the sample restriction used."
  • "Flag every result you could not verify."

What a Skill is

• A Skill is a routine. You repeat it often, write it down once, and invoke it whenever you need it.
• It is a Markdown file in your repo, has a description, and the agent reads it.
• Your instructions file has the rules for always and a Skill has one procedure and you use it when needed.
• Two ways it fires:
  • the agent loads it automatically when the task matches its description
  • or you call it by name, like a command

Effective and ineffective skills

• A Skill works when it is specific:
  • works: "My survey-weighted OLS table, with my SE choices, in my format."
  • fails: "A regression skill."
• Good candidates from this class:
  • the descriptive panel you produce for every new dataset
  • your standard robustness battery
  • The data preparation steps for every new wave of your survey.
• If a new RA cannot follow the written routine, the agent cannot either.

An open format

• The Skill you just wrote is a folder and holds one plain-Markdown file, SKILL.md, which is plain text, and you can audit a stranger's in minutes.
• The format is an open standard (agentskills.io). About 20 agents use it, and these include Cursor, Codex, and GitHub Copilot, and a Skill written once runs in all of them.
• One public directory, skills.sh, has more than 1.1 million skill installs.
• Anthropic has no official page for browsing skills. Two third-party directories, buildwithclaude.com and claudemarketplace.net, collect skills and plugins. Browse them for skills worth adopting.
• The result is practical. For many research tasks, someone already wrote the Skill, and you can read exactly what you will adopt.

Skills shared by social scientists

• Three examples of what is inside:
  • scholar-citation verifies every reference and checks it against retraction databases.
  • napkin keeps a log of your mistakes with the agent, and this log is append-only.
  • scholar-rag searches your Zotero library.
• Yongjun Zhang removed five "write the whole paper" skills from his public release, and he did this to discourage fully automated paper writing.
• On GitHub: joshzyj/open-scholar-skill, letitbk/claude-academic-setup, Zhangyanbo/vibe-research-skill.

Four ways to install a Skill

• Clone (either tool). Clone the repository and copy or symlink the skill folders you want. open-scholar-skill ships a bash setup.sh which does the copying for you.
• Official catalog (either tool). Each vendor ships one and Claude Code registers Anthropic’s marketplace when you first start it. You install by name with /plugin install, in Codex type $skill-installer inside the session, and name a skill from the openai/skills catalog.
• Third-party marketplace (Claude Code). Add a person's repository as a marketplace and then install it by name. Claude Code shows the token cost and the full list of files to install before they land on disk.
• Cross-agent (either tool). The skills CLI installs from any public git repository and installs into your agents. The skills.sh directory is a search index for public repositories, and anyone can appear in the index and nobody curates it.
• No matter which route you take, you get a folder you can open.

Skill installation location

• Installing a skill means one thing: a folder containing a SKILL.md, placed where your agent reads.
• Both tools read two locations:
  • a home location, which applies to every project you open
  • a project location, which travels with the repository when you share it
• Claude Code reads ~/.claude/skills/ and .claude/skills/, and Codex reads ~/.agents/skills/, ~/.codex/skills/, and .agents/skills/.
• The two tools cannot read each other's folders, and Codex includes an /import command which copies Claude Code skills and settings.

Updating an installed skill

• An installed skill is a snapshot, and the author can keep editing their skill, but yours does not change until you update it.
• Cloned: git pull in the clone and re-run setup.sh or copy again. A copied folder does not get the pull but a symlinked folder does.
• Claude Code marketplace: Anthropic’s marketplaces update in the background and they then ask you to run /reload-plugins. Third-party marketplaces do not update by themselves so run /plugin marketplace update or turn on auto-update in the /plugin menu.
• Cross-agent: npx skills update fetches every skill again from its source, and name one skill to update only that one.
• Codex catalog: There is no update command, so run $skill-installer on the skill again.
• An update changes the instructions for your agent. Read the new changes like you read the original skill before installing. An update will overwrite any edits you made in that folder, so keep your own edits under a different skill name.

Installing without the terminal

• The Claude Code desktop app installs plugins from a window. Click the + next to the prompt box → Plugins → Add plugin, which opens a browser of your marketplaces, and use Manage plugins to turn them on and off.
• Codex does not have an install window. The IDE extension shows your skills and you run the installer by typing a command inside the session.
• Neither tool has a window to install a plain skill folder, so you must put the folder in the right place yourself, and the commands on this page do that for you.
• The desktop apps and the terminal read the same folders and a skill shows up in both places if you install it one way or the other.

Trust

• A Skill contains instructions for your agent, and installing a skill means you trust the author.
• Sources come in three tiers:
  • official collections from tool vendors
  • people you know — a coauthor or labmate sharing a repository
  • strangers on the internet
• The first two you can install after a quick read. The third you read line by line first.
• Snyk published a ToxicSkills study in 2026, which found prompt injection in 36% of the agent skills studied and also found 1,467 malicious payloads on one skill registry called ClawHub.
• A skill file can embed a shell command that runs the moment the skill loads — before any human reads it.
• Anthropic's documentation says: "Only install plugins and add marketplaces from sources you trust."
• This course has a rule. Read the SKILL.md before you install and check the allowed-tools grant and any embedded shell command.

Edit a Learned Skill

• You can edit any Skill you adopt and copy the folder into your home skills folder. Edit SKILL.md, and the change applies immediately, so you do not need to restart.
• Renaming the directory renames the command and rewriting the description field changes when it fires on its own.
• This is the cleanest worked example. Yanbo Zhang's my-scientific-taste is a placeholder and is meant to be replaced and holds your scientific taste and his other skills consult it by name.
• His rule for where things go matches the split you made today:

What is shared and what is private

• Your agent’s behavior is defined by files in the project folder — the instructions file, .claude/agents/, and the skill folders. Anyone who gets the folder gets your setup.
• Your home folder ~/.claude/ is different and holds your personal skills, your settings, and the agent’s memory of you. It applies to every project on your machine and is not part of any shared folder.
• You share the project folder and keep the home folder.
• Do not share settings.local.json, API keys, or anything under ~/.claude/.
• One-way sharing is the simplest method. Copy a person's project folder and keep what you want, and you get a snapshot, and their later changes reach you only when you copy again.

Collaborating via a shared Dropbox folder

• If two or three people trust each other, use a shared Dropbox or Drive folder. It is the easiest way, files sync automatically, and nobody has to learn a new tool.
• Follow one rule to stay safe: do not edit the same file at the same time and split the files between people, or, say who is editing what.
• If you break the rule, Dropbox keeps both versions and it renames one AGENTS (conflicted copy 2026-04-15).md. Nothing is lost and you must merge the two by hand.
• Dropbox saves file versions for a set time. It keeps them for 30 days on Basic and for 180 days on Business. Dropbox does not record why a change happened.
• Warning: Do not put a git repository inside a synced folder, and two machines might write internal files at the same time, which can corrupt the repository. Standard practice is to keep data in Dropbox and code in git.

Git Collaboration: Committing Together

• On Monday you learned one git command, which is to commit a save point. Collaboration adds one more command, pull, which fetches your partner’s commits.
• No one needs to approve anyone. One person creates the repository and they add the other person as a collaborator, and both people can push changes directly after that. GitHub has review steps, which are optional and are off by default.
• Only the creator controls the settings; they decide who is invited, choose public or private, and control deletion. You are equals over the code.
• The routine that stops almost all trouble: pull before you start, then edit, then commit and push.
• You both changed the same file, so the later push is rejected, which protects your partner’s work, so pull first. Git merges the file automatically if you touched different lines, but it only stops if you edited the same lines, and then it puts both versions side by side in the file and waits for you to choose.
• Nothing is overwritten silently, and Git tracks who changed what and merges changes automatically and adds these features to the shared folder.

The agent runs git for you

• Neither of you memorizes git commands, so start a session by saying, “pull the latest,” and end a session by saying, “commit and push, the change was about X.”
• If a push is rejected or a merge stops, say “resolve it and show me what you kept.”
• Add this line to your instructions file: “Confirm with me before pushing.” Without it, the agent will stage, commit, and push in one step, and the change will leave your machine before you read it.

Token-based billing

• Every message you send or the agent writes counts as tokens.
• Two ways to pay:
  • Subscription (Claude Pro or Max, ChatGPT Plus or Pro): a quota that refills on a schedule. Hit the limit and you wait.
  • API key: billed per token, no ceiling. You pay for exactly what you use, and a runaway session costs real money.
• Each new message sends the whole conversation again, so a reply at the end of a long session costs much more than an early reply, and the cost of a session grows faster than its length.
• What increases the count:
  • pasting a large file
  • many rounds of back-and-forth
  • several agents running at once

Rule location determines cost

• Instructions and the memory index load at the start of every session, which happens even if today's task does not need them. A skill loads only when its task appears, and an old session loads only when you resume it.
• The rule is about placement, not length. Put rules you want enforced every time in instructions and everything occasional in a layer that loads on demand.
• Wrong or outdated memories are bad and they are worse than having no memory. These memories load in every session and push the agent in the wrong direction, so use /memory to prune them.

Why not one long session?

• The model does not remember past messages, and every time you send a message, the tool sends the entire conversation from the beginning.
• Your tenth question is billed for the nine previous rounds plus itself, so the total cost of a session grows quadratically with its length, not linearly.
• Long context hurts quality, and the model misses things in the middle.
• The conversation reaches its limit. The agent makes a summary, and it decides what to drop silently.
• Start a new session for each task, and use /compact to rescue a session that has already grown too long.

Subagents use tokens without using your context

• A subagent's work is billed like everything else, so it does not make the task cheaper.
• What it saves is your main conversation. The subagent reads the fifty files inside its own context, that reading is thrown away when it finishes, and only its conclusion enters yours.
• Your main conversation stays short, so every later message costs less and the model stays sharp.
• Use one when the work is bulky but the answer is small.

How to spend less

• Use a small, cheap model for easy steps and save the strong model for hard steps.
• Write a short brief, which will reduce the number of rounds.
• Point it at just the one file it needs.

Wrap-up

• You now have the first version of the course deliverable:
  • a working setup
  • a project instructions file
  • one Skill that encodes your own routine
• Tomorrow the same setup connects to outside websites and apps — collecting data, coding at scale, and finding literature through APIs and MCP.