Running a parallel pipeline without doubling the work
Rahul Khana
Product Analyst
Latest update
9 min read
IN THIS ARTICLE
The Four Fields Problem
Proxies that survive annoymization
When the rubric itself is the bias
Feedback loops, and why they hide
What monitoring actually catches
A short checklist
10 min remaining
Almost every hiring team using AI eventually faces the same question: what does the EU AI Act actually require when automated systems influence recruitment decisions? It may seem that compliance simply means choosing a compliant AI tool. But the requirements go further. AI systems used for recruitment and candidate selection can fall under the Act’s high-risk requirements, meaning hiring teams may need to address risk management, human oversight, documentation, record-keeping, and ongoing monitoring as part of the recruitment process itself.
Each of these requirements can shape how AI is used throughout the hiring process. Some address how candidates are assessed, while others focus on the safeguards required around automated systems. The two can be difficult to separate, because once AI influences screening, ranking, or evaluation, earlier outputs can affect who reaches the next stage — making human oversight, documentation, risk management, and ongoing monitoring important throughout the recruitment workflow.
The four fields problem
Consider what remains after an AI-assisted hiring system has screened, ranked, or evaluated candidates: records of automated outputs, risk assessments, human review steps, system decisions, monitoring results, and documentation showing how the system was used. Keeping these records can help hiring teams demonstrate that appropriate oversight and controls were in place, while also making it easier to identify recurring issues or patterns that may require further review.
Each of these requirements can influence how hiring teams use AI throughout the recruitment process. Some focus on how candidates are assessed, while others define the safeguards needed around automated systems. The two can overlap, because when AI helps screen, rank, or evaluate hundreds of applications, each automated output can influence who moves forward — making appropriate human review, oversight, and documentation essential at every stage.
The test we use internally
Take a candidate assessed by an AI-assisted hiring system and have two reviewers independently examine the outcome. Ask each reviewer whether they agree with the system’s recommendation, whether they would make the same decision, and which factors influenced their judgment. If their assessments differ, repeated reviews can help hiring teams identify where additional human oversight may be needed and whether the system’s outputs are being applied consistently across candidates.
Proxies that survive anonymisation
These are the areas that most often shape how hiring teams use AI in recruitment, ordered roughly by their potential compliance impact and how easily they can be overlooked:
Hiring teams need to determine whether an AI system used in recruitment falls under the EU AI Act’s high-risk requirements. Tools that screen, rank, or evaluate candidates may be subject to additional obligations covering risk management, human oversight, documentation, and monitoring. Failing to identify the system’s classification can leave teams without the safeguards needed to manage AI appropriately throughout the hiring process.
Hiring teams need to ensure that AI-generated recommendations do not become final decisions by default. Reviewers should be able to understand, question, and override automated outputs when appropriate, particularly when AI is used to screen, rank, or evaluate candidates. Without meaningful human oversight, automated recommendations can influence hiring outcomes without sufficient human review.
Hiring teams need to ensure that AI-generated recommendations do not become final decisions by default. Reviewers should be able to understand, question, and override automated outputs when appropriate, particularly when AI is used to screen, rank, or evaluate candidates. Without meaningful human oversight, automated recommendations can influence hiring outcomes without sufficient review or human judgment.
Hiring teams should identify and assess the risks associated with AI systems used in recruitment, including how automated outputs may affect different groups of candidates. A structured risk-management approach can help teams identify potential issues, assess their impact, and put appropriate safeguards in place before AI systems influence important hiring decisions.
Hiring teams should monitor AI systems after deployment to identify unexpected patterns, errors, or outcomes that may affect candidate assessment. Regular evaluation can help reveal when certain candidate information influences automated recommendations and give teams an opportunity to review, correct, or improve how the system is being used throughout the hiring process.
When the rubric itself is the bias
When screening rules narrow the candidate pool too aggressively, automated outputs can have a greater influence on who receives further consideration. Hiring teams therefore need appropriate safeguards, human oversight, documentation, and monitoring to ensure that AI-assisted decisions remain properly controlled and do not influence hiring outcomes without meaningful human review.
A hiring team may set a rule requiring a certain number of years in a similar role, prioritise specific career backgrounds, or make particular qualifications mandatory. An AI system can then apply those criteria consistently without understanding whether they are appropriate or proportionate. Teams should therefore review the criteria, assess potential risks, and ensure that automated screening supports a properly governed recruitment process rather than simply making candidate exclusion more efficient.
That is why hiring teams should monitor not only the AI system itself, but also the rules, criteria, and workflows surrounding its use. When recruitment outcomes or shortlist rates change, teams should be able to identify what changed, document its impact, and review whether additional human oversight or risk controls are needed rather than assuming the AI system itself is responsible.
Feedback loops, and why they hide
The most difficult risks can develop gradually as AI becomes part of a repeated hiring workflow. If the same screening rules and automated recommendations are used repeatedly, teams may reinforce existing patterns without noticing how they affect candidate selection. Regular monitoring, risk reviews, and documented changes can help hiring teams detect these patterns early and assess whether the system is still being used appropriately.
After months of use, an AI hiring system may appear reliable because it consistently produces similar recommendations and shortlist outcomes. But stable results do not necessarily show that the system is working appropriately. Hiring teams should continue to monitor performance, review risks, and assess whether the system is producing unintended patterns rather than assuming consistency means the process is compliant or effective.
Hiring teams should monitor AI use at each stage of the recruitment process rather than evaluating the system only by its final results. If a significant change appears during screening, ranking, or candidate evaluation, tracking each stage separately can help identify where the issue originates and whether additional human review, documentation, or risk controls are needed.
What monitoring actually catches
Small changes to how an AI hiring system is configured can affect who moves forward in the recruitment process. For example, teams may refine screening criteria, clarify acceptable qualifications, or include equivalent forms of relevant experience. Any meaningful change to the system or its use should be documented and reviewed so teams can understand its impact and maintain appropriate human oversight.
The lesson is not that every issue can be solved by changing the screening criteria. It is that potential risks should be identified while the AI system is being used, rather than only after recruitment decisions have been made. Ongoing monitoring and timely review can help hiring teams detect unexpected outcomes early and take corrective action before automated recommendations have a wider impact.
A short checklist
Go through the system’s intended use, screening criteria, and decision-making role, and ask whether each requirement is genuinely necessary for the recruitment process. This helps hiring teams identify potential risks, clarify responsibilities, and establish appropriate human oversight before an AI system begins influencing candidate assessment and selection.
Run the AI-assisted recruitment process with the same or comparable inputs and review whether the system produces consistent results. Significant differences in rankings, recommendations, or candidate outcomes can indicate that the system needs further evaluation. Regular testing helps hiring teams identify unexpected behaviour and determine whether additional monitoring or human oversight is required.
Hiring teams should monitor how AI is used across screening, ranking, and candidate evaluation without creating a separate process for each stage. Tracking AI performance as part of the existing hiring workflow makes it easier to identify unexpected results or risks, determine where additional controls are needed, and address issues before they influence important hiring decisions.
Hiring teams should not rely solely on past hiring decisions to determine whether an AI system is working appropriately. Historical data can reinforce existing selection patterns without confirming that the criteria remain suitable or that the system does not introduce new risks. Teams should combine historical information with ongoing testing, risk assessment, monitoring, and meaningful human oversight.
Hiring teams should keep a clear record of significant changes to the AI system, its screening criteria, and how it is used. Maintaining version history makes it easier to identify which configuration was applied at a particular point in time, understand how changes affected recruitment outcomes, and provide evidence for review when compliance questions arise.
None of this guarantees a perfect or risk-free hiring process. It does not ensure that every AI-assisted screening decision will be consistent or appropriate. What it does provide is visibility into how AI is being used, where potential risks or unexpected outcomes appear, and how those issues can be investigated, corrected, and explained—rather than allowing automated systems to influence important hiring decisions without meaningful human review.
