Why your AI might be biased (and what you can do about it)

Recently, I gave two interviews for my employer, OGD, where I discussed the technical challenges and responsibilities involved in building safe and fair AI systems. These interviews were published as Dutch blog posts:

• AI safety is onmisbaar. Het zou zonde zijn als AI zijn eigen voordelen tenietdoet met zijn bias

• “Als jouw AI iets doms doet, doet hij dat namens jou” | Leo Visser over het belang van goede datasets

Since these interviews were conducted in Dutch, I wanted to provide an English explanation of the key technical insights for a broader audience—especially for those working in AI development, operations, or governance.

AI systems are only as good as the data they are trained on. Whether you're working with a large language model (LLM), a recommendation engine, or a classification algorithm, the model learns patterns from historical data. If that data contains biases (such as overrepresentation of certain demographics) the model will replicate those patterns. For instance, if most examples of doctors in the training data are white men, the model will likely associate that demographic with the profession. This isn’t because the model is inherently biased, but because it is optimizing for statistical likelihood, not fairness. A 2024 study showed that five popular AI image generators overwhelmingly depicted physicians as white men, even when prompted neutrally.

Bias in AI isn’t always obvious. It can emerge through indirect signals in the data. A notable example involved an AI system used for screening job applicants. Even though it was designed to ignore names and ethnic backgrounds, it still filtered out candidates based on postal code, an indirect proxy for socioeconomic status or ethnicity. This kind of proxy bias is particularly difficult to detect because it hides in seemingly neutral variables. As Nolo explains , ZIP code-based filtering has been shown to lead to discriminatory outcomes in hiring systems.

If you're working with AI systems and helping others use them, one of the most important things you can do is teach users how their input shapes the output. AI doesn’t make decisions in a vacuum, it reflects the data and instructions we give it. So when an AI system makes a biased or flawed decision, that’s not just the system acting up. It’s reflecting the assumptions, gaps, or blind spots in the data or the way it was used. That means we, as professionals, are responsible for making sure users understand the impact of their choices.

One of the most effective ways to reduce bias is to train users to think critically about the data they provide. Encourage them to ask: is this data representative of the real world? Are we including enough variation in the examples we feed into the system? Are we unintentionally reinforcing stereotypes or outdated norms? These questions might seem abstract, but they have very real consequences in how AI behaves.

For example, if someone is using an AI tool to screen job applicants and they only feed it resumes from past hires (who all happen to come from similar backgrounds) the system will learn to prefer those profiles. Not because they’re better, but because that’s what it was shown. Helping users recognize this kind of pattern is key. It’s not about blaming them for bias, but about giving them the tools to spot and prevent it.

It also helps to demystify how AI works. If users understand that the system is just looking for patterns in historical data, they’re more likely to question whether those patterns are fair or useful. You don’t need to dive into the math behind machine learning, but a basic explanation of concepts like training data, model drift, or feedback loops can go a long way. A great starting point is Google’s Machine Learning Fairness guide, which breaks down these ideas in a very accessible way.

On the technical side, there are tools that can help you and your team detect and reduce bias. For example, IBM’s AI Fairness 360 is an open-source toolkit that can scan datasets and models for bias and suggest mitigation strategies. Another useful tool is Fairlearn, which integrates well with Python-based ML workflows and helps evaluate and improve fairness metrics. These tools don’t replace human judgment, but they do make it easier to spot issues early.

In my experience, it’s also important to involve a diverse group of people in the development and testing of AI systems. Bias often goes unnoticed because the people building the systems all think alike. Bringing in different perspectives (whether from different departments, backgrounds, or user groups) can help catch blind spots you didn’t even know were there.

Ultimately, reducing bias is a shared responsibility. Engineers build the systems, but users shape how they’re used. By training users to think critically and designing systems that support fairness, we can make AI not just smarter, but more trustworthy. Because if AI does something biased or dumb, it’s not just the AI that’s responsible, it’s us.

If you're building or supporting AI-powered tools inside your company, one of the most practical things you can do is help your users understand how their data shapes the system. AI doesn’t just magically know what to do. It learns from the data it’s given. So if that data is messy, biased, or inconsistent, the AI will be too. And that’s not just a technical issue, it’s something users can influence every day.

Let’s say you’ve got a Power App that uses AI to recommend actions based on support tickets. If users are tagging those tickets inconsistently or skipping fields, the model will learn from that noise. The result? Weird suggestions, irrelevant matches, or worse decisions that look confident but are totally off. That’s why it’s worth investing time in showing users how their input matters. Even a quick walkthrough or a tooltip that says “this field helps train the AI” can make a difference.

Transparency is another big one. Users don’t need to see the full model architecture, but they should know what data is being used and how it affects the outcome. You can surface this with simple UI elements like “this suggestion is based on X and Y” or “we used the last 100 entries to generate this result.” If you want to go deeper, tools like Model Cards and Datasheets for Datasets are great for documenting what’s going on under the hood in a way that’s readable and useful.

And finally, treat your data like code. Version it. Track changes. Know who touched what and when. If something goes wrong, you want to be able to trace it back, not just to the model, but to the data and the decisions behind it. That kind of traceability builds trust and makes it easier to improve things over time.

In summary, AI is not magic, it’s math. And like any tool, it must be used responsibly. By understanding how bias enters your models and applying the right technical safeguards from feature auditing and fairness-aware training to dataset curation and explainability you can build AI systems that are not only powerful but also fair and trustworthy.