Auditing a federal AI system

Today

• The White House AI memo and the public list of federal AI systems.
• Exercise 1: description and classification of federal AI systems.
• Risks of government AI and the federal rules for high-impact AI.
• Exercise 2: an audit of one high-impact system.
• Comparing audit findings.
• Pairs for the Week 10 presentations.

Deliverables

• By the end of class, your repository holds:
  • a summary of the federal AI systems: count tables and charts, such as in output/inventory_summary/,
  • your codebook and your codes for the 31 systems,
  • a short note in your README on how your codes compare with a classmate’s,
  • an audit table for one high-impact system, with the evidence and your judgment for each required practice, such as output/audit_table.csv.
• In class, you also report on your system and choose your Week 10 partner and topic.

Assigned readings

• The White House AI memo, issued by the Office of Management and Budget (Accelerating Federal Use of AI through Innovation, Governance, and Public Trust, M-25-21, April 2025), pp. 14–17 and 21–23. These pages cover:
  • the definition of high-impact AI,
  • the minimum risk-management practices,
  • the uses presumed to be high-impact,
  • the list of risk factors.
• Engstrom, Ho, Sharkey & Cuéllar, Government by Algorithm (2020), pp. 6–8, the executive summary.

The White House AI memo

• In April 2025, the White House sent federal agencies a memo on their use of AI.
• The memo came from the Office of Management and Budget (OMB). OMB is the White House office that oversees the budgets and management of federal agencies.
• Its title is Accelerating Federal Use of AI through Innovation, Governance, and Public Trust.
• The memo tells agencies how to adopt AI and which safeguards some AI systems need.
• It also requires each agency to publish a list of its AI systems every year (p. 12).
• This page calls it the White House AI memo. Its official number is M-25-21.

Definition of AI in the White House memo

• AI includes (p. 18):
  • machine learning, including deep learning, supervised, unsupervised, and reinforcement learning,
  • generative AI,
  • statistical models that predict from data, such as a risk score.
• Simple models count too. The memo says a model’s size, type, or amount of training data does not exclude it.
• A system counts whether or not a person reviews its output.
• AI does not include:
  • systems that follow only rules written by people, such as a scoring formula with weights set by hand,
  • software that repeats a person’s steps exactly, called robotic process automation,
  • common commercial products with AI built in, such as a word processor or a map app.
• The Department of Defense and the intelligence agencies do not list individual systems (pp. 12 and 25).
• Each agency decides which of its systems to list. The list shows what agencies reported. It cannot show what they left out.

Public disclosure of federal AI

• OMB combines the agency lists into one public list on GitHub.
• The list is in the Data folder of that repository, with one row per system. Use the file 2025_individually_reported_AI_use_cases.
  • The .xlsx version opens in Excel. Use it to look through the rows.
  • The .csv version is for the agent.
• As of April 13, 2026, the list has 3,611 systems.
• Agencies marked 445 of them high-impact. A later section explains this label.
• OMB calls each entry a use case and the list an inventory.
• The data dictionary describes each column of the inventory.

Counts by agency, topic area, and stage

• Which agencies use AI, and for what? Start with the whole list.
• Use your agent to count the systems three ways, with a table and a bar chart for each:
  • by agency,
  • by topic area, the list’s own grouping of what each system is used for,
  • by stage of development, such as pilot or deployed.
• Many rows leave the topic area or the stage of development blank. Decide how to report those rows.

Classification of what the systems do

• The topic area says little about what a system does to people.
• Each row also has a short text, written by the agency, on the problem the system solves.
• Use that text to classify the systems yourself, with the text-coding method from Week 5. Possible categories:
  • decisions or scores about individual people, such as benefits, enforcement, or immigration,
  • internal agency work, such as translation or document processing,
  • science and research,
  • other.
• Write your own codebook. You may change these categories.
• Everyone codes the same 31 systems: the first deployed system listed for each agency. This way you can compare results.
• Then compare your codes with a classmate’s.
• Recall Week 5. The codebook, the wording of the prompt, and the choice of model all changed the codes you got. The same holds here.
• For each system you coded differently, read its text. Find the choice behind the difference.

Example: disability benefit reviews

• The Social Security Administration pays disability benefits.
• It uses a computer model to pick which people to review.
• The model looks for people whose health may have improved.
• A review can end a person’s benefits.
• If the model is wrong, people can lose benefits they need.
• The agency reports this model, the “Continuing Disability Review Model”, in the public list of federal AI systems.

Example: child-welfare screening in Allegheny County

• Local governments use such models too.
• Allegheny County, Pennsylvania, runs a hotline for reports of child abuse and neglect.
• Since 2016, a model has scored each call. Staff use the score to decide which families to investigate.
• Virginia Eubanks studied this model in Automating Inequality (2018), chapter 4.
• She argues that the model’s data come mostly from public services. Families who use those services, who are mostly poor, therefore look riskier to the model.
• This is a county system. It does not appear in the federal list.

Consequences of errors

• COMPAS, from Week 7, is a risk score used in criminal justice. Week 7 showed that its errors differed across groups.
• Federal systems such as the disability review model also shape decisions about people’s benefits, rights, or safety.
• When such a system is wrong, the cost falls on the people the decision is about.
• For this reason, the White House AI memo sets extra rules for these systems. The next section describes them.

Definition of high-impact AI

• The memo sets extra rules for AI that affects people’s rights or safety.
• The memo calls such AI high-impact.
• In the memo’s words, AI is high-impact “when its output serves as a principal basis for decisions or actions that have a legal, material, binding, or significant effect on rights or safety” (p. 14).
• Section 6 lists uses that the memo presumes to be high-impact (pp. 21–22).
• One of them is a risk assessment about individuals in law enforcement. COMPAS, from Week 7, is that kind of tool.
• Each agency decides for itself whether its system is high-impact.

Required risk-management practices

• High-impact AI must come with safeguards. Section 4(b) of the memo lists seven of them (pp. 15–17):
  • testing before the system is used,
  • an assessment of the system’s effects on people, called an AI impact assessment,
  • ongoing monitoring of how well the system works and of any harm it causes,
  • training and assessment for the staff who operate the system,
  • human oversight, intervention, and accountability,
  • remedies or appeals for the people affected,
  • feedback from end users and the public.
• The memo calls these seven safeguards the minimum risk-management practices.
• For each deployed high-impact system, the inventory reports what the agency says it did for each practice.

What an audit is

• The White House AI memo requires seven practices for every high-impact system.
• The public list reports, for each deployed high-impact system, what the agency says it did for each practice.
• An audit checks one system against these requirements. For each practice, it asks two questions:
  • What does the agency report?
  • Is there a document that lets the public verify it?

Example: an audit of the disability review model

• This example audits the Social Security Administration’s disability review model, described above under Risks of government AI.
• The public list has one row for this model. To find it, open the .xlsx file in the Data folder and search for “Continuing Disability Review Model”.
• The middle column below shows what SSA wrote in that row. The last column shows the instructor’s judgment, based on that row alone.

• SSA reports that it meets every practice.
• The row links no document. The privacy impact assessment and code fields are empty.
• The impact assessment describes what the model does for SSA. It says nothing about risks to the people whose benefits are reviewed.
• The conclusion: the public can see only the agency’s own answers. Nothing in the public record lets anyone verify them.

Your audit

• Audit a different system in the same way.
• Choose a deployed system that its agency marked high-impact.
• Many such systems leave the practice columns blank. Record that as a finding.
• If your system’s row links to documents, open them. They may let you judge a practice as met, or show a gap.
• Make the same table: one row per practice, with what the agency reports, any document, and your own judgment.
• Your judgment is one of three: the requirement is met, there is a gap, or there is not enough information to judge.
• Where information is missing, say what document would settle the question.

Findings across systems

• Each student speaks for two minutes about their system. Cover these points:
  • what the system does and which agency runs it,
  • why it meets the high-impact definition,
  • the practice with the strongest evidence behind it,
  • the practice where the public record says least,
  • the document or inventory column you read for each of those two practices.
• Then compare the systems. Discuss these questions:
  • Does a documented gap recur across systems?
  • Where is public information insufficient?
  • Did a reported practice ever come with a document you could open?
  • What further evidence would show that an agency carries out a practice it reports?

Limits of the public record

• Many rows may end with “not enough information to judge”.
• That result is itself a finding. It describes what the public can learn about government AI.
• It raises policy questions:
  • Should agencies publish more about their high-impact systems?
  • Should someone outside the agency audit these systems? Who?
  • What would each option cost, and what would it reveal?

Topics

• In Week 10, each pair presents one topic in AI governance.
• We encourage you to choose your own topic. Any debate about how to govern AI works.
• The Week 10 page lists six example topics:
  • frontier-model risk,
  • jurisdiction and regimes,
  • geopolitics and chips,
  • power and concentration,
  • the information ecosystem,
  • labor and the economy.
• Open-source AI models, the Week 9 topic, is another example.

Format

• The six students form three pairs.
• Each pair presents for 12–15 minutes in Week 10.
• Each presentation covers six points:
  • a concrete policy dispute,
  • its actors,
  • a sourced case,
  • one core reading,
  • a comparison of at least two policy alternatives,
  • a recommendation for a named decision-maker.
• The Week 10 page has the shared reading and example readings for each example topic.

Policy alternatives for government AI

• What can the published inventory show about your system?
• What would require further investigation?
• Which AI systems might an agency leave off the list? How could you find out?
• What alternatives could the agency consider?
• Which criteria should govern that choice?
• Where did the agent help you collect or check the evidence?

Checkpoint 3: project website · 5%

• Put your project website online with GitHub Pages.
• The website presents your project proposal. It covers the first four steps of policy analysis:
  • the policy problem and its audience,
  • the data source, with one figure or table that describes the data,
  • the policy alternatives you will compare,
  • the criteria you will use to judge them.
• Add the website address to the README of your repository.

Due Dec 3 before class.

Week 9: Open-source AI models · Nov 12

• In Week 9 you run an open-source model on your own computer with Ollama.
• Install Ollama before class. Follow the Ollama documentation. This is setup only. Nothing is submitted.