Connecting AI to Other Websites and Apps

Today

• The agent used your local files until now. Today, it connects to outside websites and apps.
• There are two ways to enter: API and MCP. This session teaches both.
• We use both connections on three tasks: collecting data, coding data at scale, and finding literature.
• The plan for the session:
  • two ways to connect
  • APIs: collecting data, coding data at scale
  • MCP
  • finding literature
  • discussion

So far the agent has only used your machine

• Everything stayed in one place this week. This includes your files, your code, and public web pages.
• Real projects need things that live elsewhere:
  • the Census
  • your Zotero library
  • a journal database
  • a government portal
• Those are other websites and apps. The agent cannot use them. It needs you to give it a way in.
• Today's question: How does the agent work with a website or an app that is not on your machine?

Two ways the agent works with other websites and apps

• Every outside website or app has the same problem. The agent needs a way to talk to it.
• There are two answers:
  • the site has an API: it accepts requests from programs, and the agent writes the request
  • Connect the app once through MCP. After that, the agent uses it directly.
• The morning covers APIs. MCP follows the break.

An API accepts requests from programs

• Many data sites have an API. An API is an address. A program uses it to ask for data instead of a person.
• Every site defines its own. The Census has one. arXiv has one. They all differ.
• Using an API requires writing code to send a request. The agent is very good at writing that code.

Three APIs, one shape: input in, output back

• All APIs have the same shape. A request goes in. Structured output comes back.
• Only the things you send and receive are different.

Constructing an API call: endpoint and parameters

• We use arXiv as our teaching example. It is an open paper archive. You do not need an account for its API.
• Paste this into a browser:

• The endpoint is the part before the question mark. It shows where you are calling.
• The parameters are the part after it: what you want. Here, search for “electron” and return the first 5 records.
• It does not return a webpage. It returns structured text. Each paper is one record. The records have named fields for title, authors, and date.

The agent uses the API for you

• In a programming course, you would read the API documentation. You would then write the request loop yourself.
• You say: "Get the 100 most recent arXiv papers mentioning 'election', with title, authors, and date, as a CSV." The agent reads the documentation. It writes the code to call the API. Then it runs the code.
• Your job has a new task. Check the CSV against the site. Make sure the agent follows the site's rate limits.

Where research data comes from

• The arXiv pull was data collection. An API returned records. A loop turned them into a table.
• The same pattern applies to most sources. These include the Census, a city open-data portal, and a journal database.
• Two fallbacks when there is no API:
  • a web page with a table on it: the agent reads the page itself and extracts the table
  • Download a file and unpack it. For example, a CSV inside a ZIP.
• You still choose the source. You choose the variables. You decide the amount.

Verify your collection

• These agents are weakest at gathering data.
• On a social-science replication benchmark, agents scored about 96 at running an analysis but about 31 at finding the data the analysis needed.
• Scrapes will break. Fields will be wrong. Check every pull against the source.

Nguyen et al., "ReplicatorBench" (2026), arXiv:2602.11354.

Some APIs need a key

• Open APIs like arXiv answer everyone. Most other APIs give you a key. This is a password for programs. It is usually free to request.
• The site uses the key to identify you. It limits your request speed. Sometimes it uses the key for billing.
• Paste your key one time. The agent uses it after that. The key stays in a file. It is never inside the shared code.

No API: the agent reads the page itself

• You say to the agent: “Go to this URL, get the monthly unemployment table, and put it in a CSV.”
• The agent reads the page, finds the table, and saves it.
• Watch for these problems: A table splits across pages. Footnotes mix into cells. A site asks you not to scrape.
• If a site's terms forbid scraping, stop. Find another source.

Track raw pull sources

• Save the raw result once, before any cleaning, and never overwrite that copy.
• Tell the agent to write the source, the date, and the exact query next to the file.
• Months later, you use that note. You use it to answer "where did this number come from."

Check your results

• Open the raw file. Read the first few rows yourself.
• Compare some values manually. Check them against the source page or the API.
• Match the data first. Then move it into your analysis.

Turn a repeated pull into a Skill

• Some pulls repeat — this month’s new records, again next month. Those are worth packaging.
• Turn the steps into a Skill, the same way you did on Wednesday, so next time is one command.
• You can share the Skill. A labmate can then run the same pull. They use their own key.

Scaling a pull: script or subagents?

• One question decides it: are the steps the same for every piece?
  • steps identical — 50 calls to one API, each with a different state code: a loop in a script, no agent needed
  • every piece different — 10 state websites, each with its own layout: subagent work, where each copy figures out its own page

The API you have been using all week

• The model is a website with an API. You send it text and instructions. It sends back an answer.
• The chat window and Claude Code wrap that API. All messages this week used it.
• The arXiv API returns data. The model's API returns a judgment. They use the same mechanism. A request goes out. A structured answer comes back.

Coding data through the model's API

• On Monday, you coded twenty cases. You pasted them into the chat. This works for twenty cases. It does not work for twenty thousand.
• When done at scale, the task becomes a script. For each case, the script sends the codebook and one text to the model's API. It then stores the returned label.
• This workflow is now common in social science text analysis.

What the script sends, one case at a time

• Each request has your codebook. It also has one text case.
• Codebook: 1 means it mentions cost. 0 means it does not. Text: 'The premiums went up again this year.' Answer with only the code.
• Returned: "1".
• The loop runs once per case. It writes each answer next to its case in a CSV.

Many model APIs, one request format

• Major labs sell their models in the same way. The request format is almost identical for OpenAI, Anthropic, and Google.
• OpenRouter connects the entire market to one address. It uses one account and one key for four hundred models. You choose a model by its name inside the request.
• This makes a robustness check cheap. Rerun the same coding with a second model. Report the agreement. Do it the same way you would for two human coders.
• Sometimes you cannot move text from your machine. This happens with interviews under an IRB protocol. In these cases, the same request shape works on a model running on your own laptop.
• OpenRouter can pick a model for you: model = openrouter/auto routes each request based on what other users run for similar tasks.
• For coded data, pin one model by name and report it. The model is part of your method.

What MCP is: a pre-packaged API

• Gmail has an API. Google Calendar, Slack, and Zotero also have APIs. Why is there a second way to enter?
• MCP is not another source. It is an API for agents. Someone wraps a site's API into ready-made tools. They handle the login. You connect it once with a click.
• Then the agent uses the app directly. It asks, "what is on my calendar Thursday?" No code is written. No key is pasted.

Why MCP exists

• Most people use an AI chat app on a phone or desktop. This AI cannot run code. Someone must give it tools. MCP is the standard way to give it those tools.
• It also carries the login. For a Gmail or Google Drive connector, you sign in once in the browser. You could not get that access by writing code without a developer account.
• One standard works for all AI apps. Claude and ChatGPT both use it. The same connector serves both.

Literature is where fabrication hurts most

• If you ask a basic chatbot for ten citations, it might invent some. A text predictor writes references that sound real without being real.
• The model cannot fix this from the inside. It cannot check its own memory. Saying "please double-check" only makes it predict again.
• Verification needs an outside authority. This is a source of record. You can check the reference against it.
• That is what the morning's APIs are for. Every reference must come through an API. It must include its real metadata. It must come from a source of record.

My own workflow: four steps

• You are writing, and a claim needs a citation. The citation must be real.
• Zotero is where your references live. It holds your library. It builds your bibliography.
• This is how I solve it myself, in four steps:
  • 1. search my Zotero library — is the paper already there?
  • 2. If not, search OpenAlex. Find it. Use its real metadata.
  • 3. Add it to Zotero. Write it into my library.
  • 4. One rule for the agent: Only cite works found in my Zotero.
• Steps 1 to 3 are API calls. They are like the ones from this morning. Step 4 makes the agent follow the steps. The next cards take them one by one.

Step 1: is it already in my library?

• Every citation starts with this question. This happens before any web search.
• The agent asks it using the Zotero API. You can search your library by title words. You can search by author. You can search by DOI.
• Not finding something does not prove it is gone. The paper might be under different words. Search a second way before you decide it is not there.

Step 2: find it in OpenAlex

• OpenAlex is a free and open index. It has more than 250 million scholarly works. It covers all fields. It uses the same kind of API as arXiv. It also has citation links between papers.
• Search by topic or by title. Follow citations to find nearby papers.
• The result is a database record. It includes the title, authors, year, venue, and DOI.
• That record goes into your draft. It is not the model's memory.

Why step 2 is slow

• Every database has a speed limit for requests. This protects their servers. They do not want people to copy their entire collection.
• arXiv asks for one request every three seconds. Semantic Scholar often refuses without a key. OpenAlex is generous if you include your email.
• If you hit the limit, the site says "too many requests". Slow down. Wait. Then retry.
• This process makes the search step slow, not the code. It takes minutes instead of seconds.

Step 3: write it into Zotero

• Adding the paper to your library is a write request. It uses your key for the Zotero API. It works like every other API call this morning.
• The record lands with its metadata. It is exactly as OpenAlex returned it.
• The system generates your bibliography from the library. No line was typed from memory.

Step 4: one rule the agent cannot skip

• Steps 1 to 3 only work if the agent does not go around them.
• The rule goes into the agent's instruction file in writing: "Only cite works in my Zotero library. If a work is not there, stop and tell me."
• That line prevents a citation from entering a draft from the model's memory. You must use the three steps above to include one.

Checking a finished draft works the same way

• The three steps keep fake references out. The same idea also checks references that are already on the page:
  • citation check: look up every reference in the draft against Crossref or OpenAlex. The DOI, title, authors, and venue must match.
  • retraction check: Check the references in the Retraction Watch database. This is now part of Crossref.
  • reading your own library: The agent answers using the PDFs in your Zotero. It does not use its own memory of the literature.
• There are three rules. All three follow this rule: if there is a database of record, check it. Do not trust the model's memory.

Subagents speed up steps 1 and 2

• The subagents from Tuesday return here. This happens because the steps repeat.
• Step 1 can run in parallel. A draft cites thirty works. One subagent checks the library for each work at the same time.
• Step 2 can also run in parallel. Several searches run at once. They use different keywords or different databases.
• Step 3 remains sequential. One writer adds the keepers to the library. This ensures nothing is added twice.

What this section showed

• Monday's WVS demo used one chart from one dataset. This workflow is more complex. It has four steps, three APIs, and one written rule.
• The shape is the standard one for a complex task:
  • plan first: plan mode turns the vague goal into concrete steps
  • Run the steps one by one. Check each step.
  • Inside a step, run the independent pieces in parallel using subagents.
• It solves a real problem: fake citations.
• My version is not done. I am still fixing parts. It is a working example. Use this design to build your own.

Where are the lines?

• What is acceptable AI help in your field? What goes too far?
• What parts must belong to you? Drafting and thinking: which parts?
• What do the rules for authorship and acknowledgment require?
• What do your target journals require you to disclose now?

Tomorrow

• The final session is a build day: 100 minutes on one small website that presents your own project.
• Think tonight about what goes on your page: your question in one sentence, one figure or table, and one sentence on what you found.