Skip to main content
Tokenline

Line 3 · Station 6 of 6

Human Oversight: Knowing When Not to Use AI

Why people must stay accountable for important decisions, what "human in the loop" means, and situations where AI is the wrong tool.

  • 7 min
  • +50 XP
  • 2 knowledge checks

By the end of this station, you can:

  • Explain why accountability for decisions stays with people
  • Identify high-stakes situations that need meaningful human review
  • Recognize everyday situations where not using AI is the better choice

AI can advise. People decide.

AI systems can sort, predict and suggest faster than any person. But they don't understand consequences, can't be held responsible, and can be wrong in ways that are hard to see. When a decision seriously affects someone's health, safety, rights, grades or future, a person needs to stay in charge.

That's the idea behind human oversight: AI can be part of the process, but a qualified human reviews the result, can override it, and is accountable for the final call.

Key terms

Human in the loop
A setup where a person reviews, approves or can override an AI system's output before it's acted on.
Accountability
Being responsible for a decision and its effects — including explaining it and fixing mistakes. It belongs to people and organizations, not to software.
High-stakes decision
A decision with serious or hard-to-reverse effects on someone's life, such as medical treatment, legal outcomes, school discipline, or hiring.

Areas that need a human in charge

  • Health and medical decisions — AI can support doctors, but diagnosis and treatment need qualified professionals.
  • Legal and disciplinary decisions — people deserve a human who can hear their side.
  • Grades and admissions — decisions about a student's future need human judgment and a way to appeal.
  • Safety-critical situations — when an error could hurt someone, humans must be able to step in.

Knowledge check 1

+10 XP on first try
A school considers using an AI system to automatically decide which students get suspended, with no staff review. What's the biggest problem?

Interactive lab

Who Should Decide?

For each task, choose how much of the decision AI should have. Think about the stakes and who is accountable if it goes wrong.

  1. 1.Suggest synonyms for a word in your essay
  2. 2.Decide a student's final grade for the semester
  3. 3.Summarize a long article so you can decide whether to read it
  4. 4.Decide whether a patient needs surgery
  5. 5.Generate extra practice problems for a math test
  6. 6.Decide whether to suspend a student
  7. 7.Flag emails that might be spam so someone can review them

Answered 0 of 7

What to notice: The higher the stakes — health, grades, safety, fairness — the more a human must stay in charge. AI can assist; people stay accountable.

When NOT to use AI

Human oversight isn't only for big institutions. In your own life, sometimes the smartest AI move is not using it at all.

Better without AI

Skip AI when…

  • The point of the task is to build your own skill, like practicing a new math method
  • It would mean sharing private information about you or others
  • You're going through something hard and need real support
  • You can't check whether the answer is right, and being wrong matters

Instead, try…

  • Working it through yourself first, then using AI to check or explain
  • Keeping it offline, or removing identifying details
  • Talking to a trusted adult, counselor, or support service
  • Going to an expert or a trustworthy primary source

Knowledge check 2

+10 XP on first try
Which situation is the best fit for using AI as a helper?
Learn moreHow organizations manage AI risk

Organizations that build and use AI increasingly follow risk-management frameworks. The NIST AI Risk Management Framework, for example, describes ways to identify, measure and manage risks from AI systems, including clear roles and human oversight.

UNESCO's guidance on generative AI in education also emphasizes a human-centered approach, where AI supports teachers and learners rather than replacing their judgment.

Sources for this lesson

  • National Institute of Standards and Technology (NIST). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 2023. Full citation

Finish this station

Answer every knowledge check (right or wrong) to unlock completion.

  • Knowledge check 1 — not answered yet
  • Knowledge check 2 — not answered yet

0/2 checks answered