AI and employment: The risk behind using AI tools for the recruitment process

A recent investigation carried out by the Information Commissioner’s Office has raised concerns about UK businesses and their use of Artificial Intelligence when making employment decisions. Most importantly, the findings have exposed a widespread lack of meaningful human oversight in modern recruitment processes.

For those still stating that “AI is coming”, it’s evident that this comment no longer applies. AI is already here and running in the background of the working world. The reason it doesn’t fully feel like it yet is because the increase of tech adoption has outpaced the implementation of rigid, everyday rules. 

While these regulatory gaps should encourage businesses to act cautiously with their usage of AI, it isn’t yet having that effect. Instead, organisations are pushing the boundaries, often unknowingly crossing legal lines by letting software completely automate their employment processes.

Controlled, AI can be a useful tool for employers looking for their next ideal candidate. Unmonitored, it can be a company’s biggest nightmare, leaving senior leadership to face an expensive aftermath of data compliance breaches and high-profile discrimination claims.

ICO’s stance on automated HR 

The ICO’s investigation of over 30 major UK organisations uncovered some alarming truths about how employers are currently using artificial intelligence. The biggest takeaway is that while most employers believed their use of AI tools could be considered decision-support, in actuality, it constituted decision-making.

The act of allowing AI to make recruitment decisions opens the door to a range of compliance oversights and GDPR breaches that employers are likely unaware of. What seemed like an efficiency in the recruitment process is now risking businesses’ lengthy legal battles and complications.

How recruiters are using AI and the risks that come with it

AI is a learning system and is trained on historical data. It follows past patterns and trends and repeats them, opening the door to unintentional mishaps that may push businesses over the legal line. Here are the top risks identified when it comes to relying on AI for recruitment: 

The screening process and indirect discrimination

Due to AI functioning as an algorithm, it can sift through potential job candidates and compare them to past successful applicants. This has streamlined the shortlisting process for recruiters.

If the top applicants of previous roles at your company all share certain characteristics (e.g. age, gender, nationality, race) the algorithm will recognise this and consider it when scanning applications. If unmonitored, this can set your business on a path towards indirect discrimination claims without your realising.

Behavioural profiling tools and the penalisation of capable applicants

The ICO’s investigation outlines that profile-based tools like behavioural assessments are being used to find idealised sets of human traits such as eye contact patterns or pacing of speech. This sort of technology is supporting recruiters to pick between top applicants for customer-facing or advanced leadership roles. 

However, this is leading recruiters to filter out diverse applicants regardless of their qualifications and experience. Neurodivergent candidates or candidates with speech impairments can be poorly scored by an unadjusted, rigid AI algorithm that is looking for a narrow definition of ideal body language or tone. This lines businesses up once again for discrimination claims against them.

The reliance upon Solely Automated Decision-Making (ADMS)  

For roles with a high level of applicants, some employers are allowing AI tools to automatically issue rejection emails after its conclusion that an applicant isn’t qualified enough. Similarly, employers are being presented with a list of candidates to reject and are simply accepting the list with no review. 

Letting AI tools carry out these sorts of tasks without a human pair of eyes reviewing its decision falls right into the ICO’s definition of Solely Automated Decision-Making (ADMS). Under current UK GDPR rules, candidates have a strict right to know if an AI rejected them and a right to an explanation of the logic used.  

How does accountability work when AI is making decisions?

A common misconception is that when tech discriminates or breaches data privacy laws, it is the tech provider’s problem. This is not the case. It’s crucial for businesses using AI as a recruitment tool to understand that by choosing to use AI, they’re choosing to accept accountability for any of its decisions; if a candidate is filing a discrimination claim, they are filing it against your business and your business’ use of discriminatory tech. 

Here is what businesses must consider when choosing to use AI for employment:

Your business is in control of the data

AI, at its core, is a data processor. Employers set the parameters in order to get a collection of data that is highly tailored to what they’re looking for. This is one of the ways that employers are solidified as the liable source of recruitment outcomes. 

Why the role AI plays in your processes matters

AI that is used carefully with consistent monitoring from a human eye can absolutely speed up processes and support employers with large recruitment tasks. On the other hand, relying blindly on AI shortlists or recommendations without a human opinion to support it is how regulatory breaches occur, and businesses become liable for unjust application rejections.

Lack of defence

In the event that a rejected applicant chooses to challenge the decision a recruitment process came to, employers must be able to deconstruct the algorithm’s decision-making process to prove it was objective and non-discriminatory. If they can’t do this, they cannot legally defend the rejection. 

Getting back control 

The ICO has made it clear that it’s very easy for businesses to slip into a regular, unfiltered use of AI, meaning businesses are moving through a dangerous grey area where compliance breaches and GDPR risks are around every corner. This doesn’t have to be the reality. Directors and company leaders can implement changes and guidelines to prevent their business from heading down the wrong path in terms of AI usage. 

Ensuring and maximising human involvement

Despite AI’s rapid evolvement over the past few years, we aren’t yet able to allow it complete management of any workplace processes, including recruitment. For employers accepting AI input as the be-all and end-all, this habit needs to change. Recruiters must have the competence and time to review and influence the outcome of job applications before a final rejection is issued. 

Fairness and consistency

One of the ICO’s core expectations is that where human oversight is used to validate one candidate’s AI scoring, it must be used consistently across all candidate results at that stage of hiring. Inconsistent application of this will be deemed as a direct breach of regulations. 

Equally, keeping records of written or verbal human decisions about candidate success or failures is a simple and secure way of securing evidence of human involvement to provide if challenged by applicants. 

Statutory safeguards for employers

Under ICO rules, employers must highlight in job descriptions or privacy notices whether AI is being used to assess applicants in any way. Still, too many businesses are failing to distinguish between decision-support tools and ADMS, meaning this disclaimer is often not being announced. 

Finally, rigorous and regular testing must be carried out if employers are actively implementing AI in their hiring process. Whether this is through monthly bias reviews or another form of assessment, so long as these checks are carried out, businesses can ensure that they are within regulatory standards when it comes to AI and employment.

Keeping AI by your side

Until AI in employment is properly regulated, businesses using tools for recruitment purposes must remain highly vigilant. While the legislative landscape shifts (with updates like the UK’s Data (Use and Access) Act attempting to keep pace with innovation) widespread, formal regulation is still trying to catch up. This regulatory gap creates a dangerous grey area for employers. Until definitive legal frameworks are established, the burden of ethical and legal safety falls squarely on individual organisations. 

As much as the savings that AI tools can make are appealing (particularly for SMEs) this sort of tech cannot be relied upon to make decisions. When a business relies on software to make definitive, un-audited choices on human talent, these efficiencies translate to legal exposure. 

This doesn’t mean ruling out AI and its efficiencies; it means making clever decisions over how you want AI to be applied in your company. The objective for forward-thinking leadership isn’t to retreat from innovation or ban automation in HR. It means AI should be treated as the assistant it is advertised as, and nothing more. Protecting your organisation at a time of rapidly evolving tech means ensuring that your recruitment teams have the evidence to explain themselves when a decision is challenged.