Loading article...
Supervising AI in the workplace means keeping a trained person able to understand, check, and stop AI that affects work. Learn who must supervise, what the EU AI Act, US DOL guidance, and NIST require, and how to set up oversight that holds up in an audit.
Supervising AI in the workplace means a trained person can understand what the system does, check its output before it affects people, and stop or correct it when needed. The person is accountable for the outcome, not the model.
This guide explains what supervision covers, who must do it, how it works in practice, what the law requires in the EU and the US, and how to build a setup you can defend to workers, auditors, and regulators.
Supervision is the set of human and technical measures that keep AI within its intended limits while it is in use.
It includes:
Who owns the system, who can change it, and who can pause it. Roles are written down and assigned to real people.
Logging inputs, outputs, tool calls, and decisions so you can trace why an action happened.
Checking outputs for accuracy, bias, and safety before they drive a significant decision such as hiring, performance rating, task allocation, pay, discipline, or termination.
The ability to disregard, override, or halt the system. For high-risk systems under the EU AI Act, this includes a stop button or similar procedure that leaves the system in a safe state. Source: Regulation (EU) 2024/1689, Article 14 paragraphs 1 to 4(e) at https://artificialintelligenceact.eu/article/14/ and official text at https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202401689
Supervision does not end at design or training. Article 14 requires effective oversight during the period of use. The provider must create conditions for oversight, and the deployer must assign it to natural persons with competence, training, and authority. See also Article 26(2) on deployer duties. Oversight measures must be proportionate to risk, autonomy, and context of use.
NIST frames the same idea as human-led risk management across the AI life cycle. The AI Risk Management Framework 1.0, released January 26, 2023, organizes work into Govern, Map, Measure, and Manage. Documentation must explain knowledge limits and how output may be used and overseen by humans. Source: https://www.nist.gov/itl/ai-risk-management-framework and https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
ISO/IEC 42001:2023 covers the same ground for an AI management system. It requires an organization to define scope, assign leadership, assess risks, control the AI system life cycle, and run performance evaluation and improvement. Annex A lists controls for policies, human oversight, data management, and supplier oversight. Certification is voluntary, valid for three years with annual surveillance, and does not replace EU AI Act duties. Source: https://www.iso.org/standard/42001
If you only use AI for low-risk help such as drafting a job description or summarizing notes a human wrote, supervision can be light. If AI ranks candidates, monitors performance, assigns shifts, or sets pay, you need formal oversight.
Teams use these levels to match oversight to risk. NIST and EU guidance both point to proportional oversight.
The system proposes, a person approves before any significant action. Use for hiring screens, pay changes, and discipline. Highest friction, strongest control.
The system acts within pre-approved bounds but a person monitors live, reviews samples, and can intervene. Use for high-volume help desk drafts or shift suggestions where harm is reversible.
The organization sets policy, training, and audit that bounds the system over time. A person does not check each output, but the system cannot take significant actions and all uses are logged for later review. Use for low-risk writing help or internal knowledge search.
Pick the level by impact. An AI that suggests training modules needs less review than one that flags workers for performance action.
Map every step where AI touches worker or candidate data.
What data the system sees. Resume text, keystrokes, video, warehouse sensor feeds, or performance history. Inputs shape outputs, so limit collection to what is job-related.
Parser, matcher, ranker, or generative model. Record model name, version, and instructions for use. Note if the tool generates a score, tag, or ranking.
How the result moves to a person. Is it a draft, a shortlist with reasons, or an auto-reject? Auto-actions need tighter gates.
Who approved, when, and why. Keep the log immutable for a retention window so an auditor can replay the chain.
EU Article 14(4) lists five abilities the deployer must have, as appropriate and proportionate:
For remote biometric identification in Annex III point 1(a) cases, Article 14(5) adds a two-person rule. No action on an identification may be taken unless verified and confirmed by at least two competent persons. This does not apply where Union or national law treats it as disproportionate for law enforcement, migration, border control, or asylum.
You own the outcome of a tool you deploy. On July 12, 2024, the US District Court for the Northern District of California ruled in Mobley v. Workday, Inc., No. 3:23-cv-00770, that a vendor providing AI screening tools can be an agent of the employer under Title VII and related laws. The court later allowed the agent theory to proceed, and by mid-2025 the court addressed collective certification questions. Workday denies wrongdoing. The point for you is that both provider and deployer can face claims. See EEOC technical assistance of May 18, 2023 on assessing adverse impact, and EEOC ADA guidance of May 2022.
You cannot shift liability by contract. Put vendor cooperation on audits, logging, data, and notice duties in writing before you buy.
The US Department of Labor issued Artificial Intelligence and Worker Well-being: Principles for Developers and Employers on May 16, 2024, and detailed Principles and Best Practices on October 16, 2024. The documents are non-binding but map directly to existing law. Release: https://www.dol.gov/newsroom/releases/osec/osec20241016
Eight principles:
The October 2024 best practices repeat a clear line: do not rely on AI systems, or on information collected through electronic monitoring, to make significant employment decisions without meaningful human oversight.
On January 20, 2025 and January 23, 2025, executive orders revoked prior AI executive orders and related guidance pages. Several EEOC and DOL pages were removed or archived. Federal anti-discrimination statutes did not change. Title VII, the Americans with Disabilities Act, and the Age Discrimination in Employment Act still apply whether the decision involved a human or a system. See https://www.eeoc.gov/laws/guidance/select-issues-assessing-adverse-impact-software-algorithms-and-artificial-intelligence-used and iTutorGroup consent judgment of August 2023 on age bias for context.
Passed in 2021, effective January 1, 2023, enforced from July 5, 2023 by the Department of Consumer and Worker Protection. It applies when an automated employment decision tool substantially assists or replaces discretionary decisions for roles in New York City. The tool includes machine learning, statistical modeling, data analytics, or AI that generates a score, tag, or ranking.
Requirements:
The law requires you to have and publish an audit. It does not require a passing score. Published audits often show impact ratios below 0.80 or large unknown groups, which signals where to investigate. ACLU tracker: https://github.com/aclu-national/tracking-ll144-bias-audits Official page: https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page
For AI analysis of video interviews of Illinois applicants: give notice that AI may be used, explain how it works and what general characteristics it uses, get consent, limit sharing to those with expertise to evaluate fitness, and delete the video and copies within 30 days of a request including backups. If selection rests solely on AI for who gets an in-person interview, report race and ethnicity data to the Department of Commerce and Economic Opportunity by December 31 each year. Source: https://law.justia.com/codes/illinois/chapter-820/act-820-ilcs-42/
Defines AI to include generative AI, bars AI that causes discrimination or uses ZIP codes as a proxy, and requires notice when AI is used for recruitment, hiring, promotion, renewal, training selection, discharge, discipline, or terms of employment. Enforcement by the Illinois Department of Human Rights.
Requires written consent before facial recognition in interviews. Code: Md. Lab. and Empl. Sec. 3-717.
Requires reasonable care to avoid algorithmic discrimination for high-risk systems including employment, with impact assessments and disclosures.
Regulation (EU) 2024/1689 entered into force August 1, 2024. Source: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202401689
Annex III point 4 lists employment AI as high-risk. It covers AI used for recruitment or selection, especially to place targeted job ads, to analyse and filter applications, and to evaluate candidates, plus AI used to make decisions on promotion, termination, task allocation, or monitoring of performance. No minimum threshold.
Articles 9 to 15 require a risk management system, data governance, technical documentation, logging under Article 12, accuracy testing, and human oversight under Articles 14 and 26, plus transparency to deployers under Article 13. Deployers have duties under Article 26 and 50 to use the system per instructions, assign oversight to competent people, ensure input data is relevant, monitor operation, inform the provider of serious incidents, and inform workers that they are subject to an AI system.
Article 5 prohibits emotion inference in work and education from February 2, 2025. A video tool that claims to detect enthusiasm or stress from a face or voice in a work context is prohibited, not just high-risk.
Prohibitions applied February 2, 2025. General-purpose AI rules applied August 2, 2025. High-risk obligations for Annex III apply August 2, 2026, with the Digital Omnibus Regulation (EU) 2026/1744 deferring standalone Annex III compliance to December 2, 2027 and certain Annex I systems to 2028. Treat August 2, 2026 as the planning date until counsel confirms the deferral for your system. Build logging, documentation, and oversight now. Phasing depends on whether the system is embedded or standalone. See timeline at https://artificialintelligenceact.eu/implementation-timeline/
Article 99 sets fines for high-risk violations up to 15 million euros or 3 percent of worldwide annual turnover, whichever is higher. Higher for prohibited practices. Member state authorities, data protection authorities, and labor inspectorates can enforce, alongside GDPR Article 22 which already limits solely automated decisions with legal or similar effect.
Released January 26, 2023. Voluntary, but referenced by regulators and auditors. Use it to structure your program. See https://www.nist.gov/itl/ai-risk-management-framework
Set roles, policies, accountability, and worker input. Define who can grant, change, or revoke AI authority.
List where AI touches decisions, who is affected, and what harm could occur. Include context and intended use for each system.
Test accuracy and bias with sliced tests by group. Track precision and recall by requisition and stage, not only overall. Retest when you change criteria, thresholds, or model versions. Keep knowledge limits documented.
Put controls in place: human review before adverse action, logs, appeal paths, and periodic audits. Update controls when monitoring shows drift.
The companion Generative AI Profile NIST AI 600-1, released July 26, 2024, adds guidance on hallucinations, provenance, and grounding for generative tools. Source: https://doi.org/10.6028/NIST.AI.600-1
Where supervision helps:
A single screen checked once can affect hundreds of applicants. A human check catches parsing failures, mis-ranked candidates, and test drift before offers go out.
Logs of inputs, outputs, approvals, and overrules give you answers during a complaint or audit. SHRM reported in February 2025 that 43 percent of organizations now use AI in HR, up from 26 percent in 2024, with 64 percent of those using it in recruiting, so documentation gaps grow with volume.
Advance notice, plain-language reasons, and a clear path to review reduce disputes. Pew Research Center found 66 percent of Americans would not want to apply to an employer that uses AI to help make hiring decisions, and 71 percent oppose AI making a final decision. Transparency and review narrow that gap. Source: https://www.pewresearch.org/internet/2023/04/20/ai-in-hiring-and-evaluating-workers-what-americans-think/
The same inventory, impact assessment, and approval flow can cover hiring, scheduling, and support bots, which cuts later build cost. Limits and trade-offs:
In-loop review slows high-volume workflows and needs staffing. Treat it as operating cost, not a one-time project.
People tend to favor the model suggestion, especially under time pressure. NIST and Article 14 call this out. Counter it by showing the basis for the score, offering reasons to disagree, and rotating reviewers.
You need group data to test impact, but privacy rules limit what you can collect. Collect only what the law allows, disclose it, and keep testing data separate from the hiring decision where possible.
A published bias audit does not prove fairness. Audits often leave 18 to 25 percent of race or ethnicity as unknown, and they test only the period and groups reported. They show where to look, not that risk is zero.
Performance shifts as pools, policies, or prompts change. Supervision must be ongoing, not a pre-launch check.
List each tool, owner, purpose, data sources, model or rules version, and where the output appears in decisions. Mark whether it only assists a human or substantially assists or replaces a decision. That distinction triggers Local Law 144 and Annex III.
Add columns for on-chain proof if you use Web3 hiring: GitHub, verified contract addresses, Dune dashboards, and governance history. Note what the model does not see.
Start low-risk while you build the program:
Keep ranking as assistive. Show a shortlist with job-related reasons a human can verify. If you auto-reject, require a documented, job-related criterion and a second human check for edge cases.
Limit which APIs and on-chain contracts an agent can call. Cap spend per day and restrict data exports.
Give operators a stop control and a clear path to undo a bad batch, such as reopening a req or reinstating a schedule.
Require retrieval from trusted sources before a generative tool acts. Keep prompts, tools, retrieved context, and model version with each action.
Require model cards, data sheets, training data description, groups tested, bias results with selection rates and impact ratios, logging for traceability, and cooperation with your independent audit. Local Law 144 puts the audit duty on you, so require data access.
Do not rely on a vendor PDF.
Write a notice a person can act on. Include:
Keep the notice human-readable. Log the date sent and link the full audit PDF.
Time to fill, time to screen, recruiter hours saved, offer acceptance rate, 90-day retention, and pass-through rates by stage and by group with impact ratios. Pause automation for a req if ratios drop and no job-related justification exists.
Sampled human review of outputs: parsing errors, confabulated skills, and overrides with reasons. Review appeal and accommodation requests.
Independent bias audit within one year for NYC roles, broader impact assessment for all hiring systems, and management review per NIST Govern and ISO 42001 Clause 9.
Retain logs, notices, audit summaries, and version history per counsel advice. For EU high-risk systems, keep logging per Article 12, technical documentation per Article 11, and a serious-incident process per Article 73.
A named person checks the system's work at the right stage, can explain the basis for a decision, and can stop or correct the system before harm spreads. The team logs what was checked and why.
No. Match review to risk. An AI that drafts a help article needs sampled review. An AI that ranks applicants or flags a worker for discipline needs review before the adverse action. Batch or sampling review is acceptable for low-risk steps, not for significant employment decisions without additional controls.
No. EU Article 14 and NIST both reject a single checkbox. You need trained people, clear instructions, logging, testing for bias and accuracy, and a stop control. Over-reliance warnings and review of automation bias are required controls, not optional notes.
When its confidence is low, when data sources disagree, when gas or cost is unusually high, when it meets a contract or pool it has not seen before, or when the action is irreversible such as moving treasury funds or rejecting a candidate. Policy should be pause and ask for approval, with a global pause any operator can trigger.
Parts are assisted, such as summarizing notes or drafting schedules. Most teams keep hiring decisions, pay, and discipline behind human approval and contract gates. That is the pattern regulators expect.
No. 0.80 is a flag, not proof of lawfulness. A ratio above 0.80 can still be unlawful if the criterion is not job-related, and a ratio below 0.80 can be lawful if you show job-relatedness and no equally effective alternative with less impact. Use 0.80 to decide where to investigate.
Avoid it. Validity is weak, candidate trust is low, and rules are strict. Illinois requires detailed notice and consent plus deletion on request, Maryland requires written consent for facial recognition, and the EU bans emotion inference in work from February 2, 2025. Use human interviews with a consistent scorecard and limit recording retention.
You are, as the employer or employment agency. Mobley v. Workday shows vendors can also face liability as agents, but that does not remove your duty. Contracts should require vendor cooperation with audits, logging, and disclosure of training and testing.
No. ISO 42001 gives you an AI management system that aligns with many AI Act controls, but it does not replace conformity assessment or deployer duties. Treat it as a foundation for documentation and audits.
Not by default. Those platforms provide the ATS and optional AI features, but compliance depends on which features you enable, how you configure criteria, what data you collect, and whether you ran an independent audit within the past year, published the summary, and gave timely notice. Inventory what you turned on and run the seven steps.What record should we keep for an audit? Data period and reqs covered, total applicants and counts by group including unknown, selection rates and impact ratios by race or ethnicity and sex, auditor name and independence, model version, criteria used, notices sent with dates, human overrules with reasons, and next audit date. Keep version history to show what changed and when.
Sources and further reading: Regulation (EU) 2024/1689 Articles 5, 12 to 15, 26, 50, 99 and Annex III point 4 at https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202401689 and https://artificialintelligenceact.eu/article/14/ ; Digital Omnibus Regulation (EU) 2026/1744; NYC DCWP Automated Employment Decision Tools page, Rule, and FAQ at https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page ; 820 ILCS 42 and 775 ILCS 5/2-102 as amended by HB 3773; Md. Lab. & Empl. Sec. 3-717; Colorado SB 24-205; EEOC Technical Assistance on Assessing Adverse Impact under Title VII, May 18, 2023 and ADA guidance May 2022; DOL AI Principles May 16, 2024 and AI Best Practices October 16, 2024 at https://www.dol.gov/newsroom/releases/osec/osec20241016 ; NIST AI RMF 1.0 January 26, 2023 at https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf and Generative AI Profile NIST AI 600-1 July 26, 2024 at https://doi.org/10.6028/NIST.AI.600-1 ; SHRM Talent Trends February 2025; Pew Research Center April 20, 2023 at https://www.pewresearch.org/internet/2023/04/20/ai-in-hiring-and-evaluating-workers-what-americans-think/ ; EEOC v. iTutorGroup consent judgment August 2023; Mobley v. Workday, No. 3:23-cv-00770 (N.D. Cal.).
Explore more guides and career playbooks