Algorithmic bias: a real AI risk for SMEs?
Yes, algorithmic bias is a real risk for a small or mid-sized business, but not everywhere to the same degree. It becomes serious as soon as an AI tool contributes to a decision about a person (CV screening, performance evaluation, customer scoring), and at that point the liability sits with you as employer or data controller, not with the vendor alone. This article helps you spot the exposed uses, check a tool before relying on it, and document your controls. It is part of our guide to AI in business for SMEs.
What is algorithmic bias, in practice?
Algorithmic bias is a systematic error in an AI tool that favors or disadvantages certain groups of people for reasons unrelated to the task at hand. It almost never comes from intent. It comes from the training data, from how the problem was framed, or from the context in which the tool is used.
Three origins come up again and again:
- Unbalanced historical data. A model trained on ten years of mostly male hires learns that "successful profile" looks like "male profile". A recruiting tool dropped by a large US company, reported in the press in 2018, is the most quoted example.
- Indirect variables. Removing the first name or gender from a file is not enough. A postcode, a career break or a hobby can act as a proxy and rebuild the same information.
- Misuse. A tool designed to sort engineers' CVs does not behave the same way on care-assistant applications. Bias can come from the gap between the case the tool was validated for and the one you use it for.
Generative models (ChatGPT, Claude, Gemini, Copilot) have their own limits: they reproduce stereotypes found in their training corpora. This shows most when you ask them to judge a person rather than rephrase a text. For the basic vocabulary, see our glossary entries on LLMs and generative AI.
Is an SME really concerned?
An SME is concerned as soon as it uses AI to sort, score or route people, even with a consumer tool. The risk does not depend on company size but on how much the decision affects the individual.
The most common situations in an SME:
- an HR manager asks an AI assistant to rank the 80 CVs received for a position;
- a lead-scoring tool automatically filters out certain customer profiles;
- a phone pre-qualification assistant misroutes callers with a strong accent;
- a performance review written "with the help of AI" from a manager's notes.
By contrast, using AI to summarize a meeting, draft a quote or rephrase an email carries very little of this risk. The first question is therefore simple: does this output weigh on a person? If not, the topic is secondary. If yes, keep reading. Our article on shadow AI explains why these uses often exist without management knowing.
What the law says: three layers to know
Three bodies of law stack up: non-discrimination law, the GDPR and the AI Act, and none of them lets you off the hook because the bias came from a tool. The employer or organization makes the decision; the tool is only a means.
Non-discrimination. The French Labor Code (article L1132-1) and Criminal Code (articles 225-1 onwards) prohibit excluding a person on protected grounds: origin, sex, age, family situation, health, disability and so on. Whether the decision is made by hand or with an algorithm makes no difference to the offense. The Labor Code also requires that recruitment methods be disclosed to candidates (article L1221-8). Discrimination is a criminal offense, which goes well beyond an ordinary employment dispute.
The GDPR. Article 22 governs decisions based solely on automated processing that significantly affect a person. If a tool ranks candidates and nobody genuinely reviews the result, you may be in that situation. A data protection impact assessment (DPIA, article 35) should be considered for high-risk processing. To see how the two texts fit together, read about how the AI Act and the GDPR interact.
The AI Act (Regulation (EU) 2024/1689). AI systems used for recruitment, selection, promotion, termination or monitoring of workers are listed in Annex III as high-risk systems. Providers must govern their data so as to detect bias (article 10). Companies that use these systems ("deployers") must, among other things, ensure real human oversight and inform workers' representatives and the workers concerned before putting the system into service (article 26). The classification is detailed in our article on high-risk AI systems (article 6).
On timing, the high-risk obligations of Annex III were postponed under the "Digital Omnibus" package. The announced date is 2 December 2027, based on the political agreement of May 2026 (check the final text published in the Official Journal of the EU). This postponement does not suspend non-discrimination law or the GDPR, which already apply today.
On penalties, article 99 of the AI Act provides, for breaches of the obligations of operators of high-risk systems, fines of up to 15 million euros or 3% of worldwide turnover, whichever is higher, with a mitigation rule for SMEs (the lower of the two amounts applies). Our article on AI Act penalties covers the mechanics.
How do you check a tool before using it on people?
Test it on your own cases before trusting it with a decision, and keep a written record of the test. A vendor's compliance document does not replace a trial on your data.
A five-step method that fits in half a day:
- Describe the exact use. Which decision, about whom, with which input data, and who reviews the output.
- Ask the vendor for documentation. Training data, bias detection measures, known limits, intended conditions of use. A vendor who cannot answer is already telling you something.
- Build a test set. For example 20 real, anonymized applications, to which you add identical variants where only one element changes (first name, address, career gap).
- Compare the results. If two equivalent files get different scores or rankings, you have a signal.
- Document and set a rule. The tool helps prepare; a human decides and can justify the decision with criteria tied to the job.
To measure a selection gap, there is a simple benchmark: the selection rate of one group divided by that of the most selected group. Illustrative example (made-up figures): out of 100 applications from men and 100 from women, 40 men and 22 women are shortlisted. The ratio is 22/40, or 55%. A ratio below 80% is used in the United States as a warning sign (the "four-fifths rule"). This threshold has no legal value in France: it triggers an analysis, it does not prove discrimination.
| Situation | Risk level | Minimum reflex |
|---|---|---|
| Summarizing minutes, rephrasing an email | Low | Usual review |
| Ranking or scoring CVs | High (AI Act Annex III) | Test on real cases, reasoned human decision, inform candidates |
| Scoring customers or leads | Medium to high depending on impact | Documented criteria, ability to contest |
| Evaluating an employee's performance | High | No decision without human review, inform staff representatives |
| Customer service chatbot | Low to medium | Watch for unequal answers across profiles |
What you can put in place this week
Three actions are enough to remove the blind spot: an inventory, a rule, a record. None requires a lawyer or a data scientist.
- An inventory of uses that affect people. Map them in a simple table (tool, use, data, decision-maker). Our article on AI governance for SMEs offers a lightweight register.
- A written rule. Add one clear sentence to your AI policy: no decision about a person is made on the sole basis of an AI output.
- A record of controls. Test date, data set used, results, decision. In case of a dispute or inspection, this is the document that counts.
One point of attention: team training. A manager who understands why a model can get an atypical profile wrong reviews better than one who trusts by default. This is one of the concrete benefits of AI training for companies.
Frequently asked questions
Does a 20-person company need to worry about algorithmic bias? Yes if it uses an AI tool to sort, score or route people. Size does not change how non-discrimination law or the GDPR apply. It mainly changes the reasonable level of formality: a table and a documented test are often enough to start.
Who is liable if the bias comes from the vendor's tool? The vendor has its own obligations, but the company that makes the decision remains responsible for it. The vendor contract may share liability; it does not remove your responsibility toward the people concerned.
Can ChatGPT or Claude screen CVs safely? They can help structure or summarize files, but using them to rank candidates is recruitment, and therefore a sensitive use under the AI Act. You need to test, keep a human decision and inform candidates.
How do I know whether my tool is biased? You cannot know for certain, but you can detect gaps: compare results on equivalent files where a single element changes, and measure selection rates per group on a large enough sample.
Is a detected bias enough to ban the tool? Not automatically. What matters is understanding the gap, correcting it or limiting the use (for example by keeping the tool out of the final decision), and documenting that choice.
Do I have to inform candidates or employees? Yes for recruitment and employee monitoring: the Labor Code requires disclosure of recruitment methods, the GDPR of processing, and the AI Act of certain high-risk systems. Have the wording checked by your DPO or legal counsel.
What is the difference between bias and hallucination? A hallucination is an answer the model invents. A bias is a systematic gap across profiles. Both can be controlled, but not in the same way.
How GrowthPerf helps
GrowthPerf, a Qualiopi-certified training organization, supports SMEs and nonprofits on this topic with a short format: mapping the AI uses that affect people, testing on your real cases, drafting the internal rule and training the managers concerned. The goal is not to produce one more report, but to leave you with a register and a method you can repeat on your own.
To get started, contact us or see our AI training for companies. If you would rather take stock of your uses first, our AI audit for SMEs is the best entry point. To put the topic in a wider approach, read our complete guide to AI in business.