This Toronto company is tackling AI bias in the hiring process

Winner of the final MaRS and CIBC Inclusive Design Challenge, Code for Canada is working to ensure everyone gets a fair chance at their dream job.

When looking to fill open positions, companies are increasingly look to artificial intelligence-driven platforms to help streamline — and speed up — the hiring process. AI tools can generate job postings, review and filter out applications and even run background checks. But for people living with disabilities, this means they’re rarely considered for the job.

That’s because these programs aren’t developed with disabilities in mind, and unconscious biases can be inadvertently embedded into the very programs meant to overcome prejudice. For the visually impaired, applying online often requires an expensive screen reader, which doesn’t work properly if job application webpages aren’t compatible. Those who have difficulty typing rely on voice-to-text software, which can misinterpret people with speech impairments. And application systems often don’t allow potential candidates to explain resume gaps, even if it’s due to an accident, surgery, a disability leave or rehabilitation.

In fact, a CIBC market research survey found 47 percent of people don’t disclose their disability in a job application out of fear of discrimination. For the fourth and final instalment of the Inclusive Design Challenge series, MaRS partnered with CIBC to help give AI bias the pink slip.

After the three month-long challenge, Code for Canada, a non-profit that helps companies and institutions develop accessible and inclusive tech, was announced as one of the four winners. Here, CEO Dorothy Eng explains her organization’s winning solution, the headway being made in tech and the changes that still needs to happen to ensure everyone gets a fair chance at their dream job.

How does Code for Canada tackle bias in hiring?

We partner with governments, other nonprofits and private-sector organizations to plan, co-develop and test digital solutions, whether it’s services that renew driver’s licenses, deal with immigration status or provide critical health services. Our inclusive user research service ensures digital products and services are tested with diverse people so that they work for everyone — not just some groups of users. Recently, we’ve been working with the RCMP on its national cybercrime reporting platform. Inputting information into an online portal seems fairly basic, but you need developers, coders, product managers and designers who can deliver an inclusive, user-centric portal.

So what does that look like in practice?

Our service connects people working on digital products and services with underrepresented groups — such as people with disabilities — to test their services. Often, teams talk to the customer segment they want to sell to, but the problem with that is marginalized people are then left out of the conversation. For instance, a graph that’s easy for someone to read is totally different for colour-blind people. And we need to consider the subtle use of language. A lot of job postings say “we’re looking for…”. We’re recognizing that sometimes words we take for granted insinuate ableism.

How does AI bias happen?

AI models are largely based on historical data, which includes feedback from an existing customer pool. Twitter, for example, built a tool to automatically crop photos, which was cutting out Black people’s faces. It turns out the data they used to train that model was from MySpace, which a lot of white people used. If AI models don’t have data from diverse populations, there’s potential for bias to happen, and for those biases to become reinforced. People with disabilities already face a lot of barriers when it comes to securing meaningful, high-quality employment. And when you layer in HR tech tools that make decisions about who they interview, who moves on and who gets offered employment, there’s so many added barriers facing people with disabilities.

What are the challenges in making sure those biases don’t show up in the first place?

It’s getting people to care. Important work is often not sexy. Bringing attention to responsible technology doesn’t get a lot of attention, investment or love. It takes a lot of energy and resilience. Getting those wins where partners value the work we’re doing is what keeps me going.

How can hiring managers create more inclusive job postings and application portals?

Hiring managers should ensure their job posting is accessible to the widest range of applicants. This includes using plain language and offering accessibility support. One of the most important things an organization should consider when building an application portal is building with — not for — their intended audience. Inclusive user research is key to ensuring that a digital product meets the needs of all, not just some, users.

What does tech need to do to be more inclusive?

Whether it’s the startup world, big tech or even the investment side, every single area of the industry has baked-in ableism, racism and gender stereotypes. But the modern wave of tech workers coming into this sector are more aware of these biases. So I think there’s hope for opportunities like this. Let’s do things ethically and with inclusivity as a core value. That will certainly help move the needle.

Learn more about the impact and lessons learned for the MaRS and CIBC Inclusive Design Challenge at a special closing event on June 4.

This interview was edited for clarity and length.

Photos courtesy of Code for Canada