Confronting the Problem of AI Bias

Posted on June 18th, 2021 by Ann-Marie Roche in AI & Data

When thinking of
artificial intelligence (AI), many people rely on cultural touchstones from
science fiction films and TV. They might think of HAL, the calm-voiced AI from 2001:
A Space Odyssey, or Data, the even-tempered android from Star Trek.
We tend to imagine that artificial intelligence will be driven by reason, logic
and facts – not by the emotions and prejudices that get us regular humans into
trouble so often.

However, the reality of AI is that it’s nothing without data, and that data is provided by humans. What’s more, currently, it isn’t uncommon for datasets used to train AI to be harvested from sources that are even notorious for having strong, biased opinions, like Reddit user forums or Amazon.com reviews. As the Future Today Institute’s 2021 Tech Trends Report recently highlighted, “AI bias” is a worrisome trend to be on the lookout for.

Denied
by an algorithm

“As computer systems
get better at making decisions, algorithms may sort each of us into groups that
don’t make any obvious sense to us—but could have massive repercussions,” warns
the report, noting that researchers at prestigious universities like Princeton
and Carnegie Mellon are now studying the unintentional effects of automatic
decision-making.

The report
highlights some concerning examples, like how the Apple card offered
significantly higher credit limits to men than women. “You, or someone you
know, could wind up on the wrong side of the algorithm and discover you’re
ineligible for a loan, or a particular medication, or the ability to rent an
apartment, for reasons that aren’t transparent or easy to understand,” it
explains.

Artificial neural
networks and other machine learning methods will often find and use features of
the data in their training sets that we don’t see or ignore. “This can mean
that neural networks will detect inherent bias in training sets and learn to
reflect that bias in their behavior,” says Ted Slater, Senior Director, Product
Management PaaS at Elsevier. “This bias can arise at any point in the data
collection process, from study design to data analysis and beyond.”

Bias
could hurt drug repurposing efforts

How, then, to
address this bias problem? Slater suggests three straightforward steps:

1. Find out that it’s happening
2. Find out why it’s happening
3. Fix the problem

“That may sound kind
of flippant,” he admits, “but that’s really it. Each step can be difficult, but
the key to success, as always, is good data.”

In an interview with Healthcare Global, Pistoia Alliance consultant Becky Upton agreed on the critical role of data, emphasizing that bias can only be dealt with by improving the quality of the data that is feeding the algorithms and ensuring the datasets are varied and drawn from reputable sources. “The Pistoia Alliance has created a Center of Excellence in AI and a project dedicated to Informed Consent using blockchain – to provide a space for the industry to share best practices and discuss common challenges,” she explained.

The Pistoia Alliance, which is a global non-profit committed to promoting innovation in life science and healthcare R&D, conducted a survey in which 38% of respondents said they believe algorithmic bias could be a barrier to the use of AI for drug repurposing. That would be very bad news for the pharma industry, since drug repurposing has become an important part of the development pipeline. All the more reason for organizations to join the alliance in its efforts to work on the bias problem.

A
history of bias in pharmaceutical development

In pharmaceutical research and the life sciences, bias has reared its head in a number of ways over time. It has been especially insidious in clinical trials, where for decades there was a major lack of diversity among trial participants. As a result, drugs would often go to market without having been adequately tested on women or people from various racial or ethnic backgrounds.

The inability to
recognize these kinds of biases often comes from having a largely homogenous
group conducting the research. This has historically been a problem in science,
and can be a big problem in tech as well. From the chemistry lab to the
computer lab, addressing diversity in who is managing the data can be
relevant to ensuring the integrity of the data itself.

Can
we debias machine learning?

Concerted efforts among many stakeholders, such as the Pistoia Alliance initiative, will hopefully lead to a variety of approaches to mitigating bias in AI and machine learning. Some researchers are already working on possible solutions. A paper published in Communications Biology (Systematic auditing is essential to debiasing machine learning in biology) earlier this year presented a “systematic, principled, and general approach” to auditing machine learning models in the life sciences.

A framework like
this, designed to examine and uncover biases in models, provides one very
promising approach to dealing with AI bias. We look forward to seeing others
emerge in the near future.

Please enable JavaScript to view the comments powered by Disqus.

R&D Solutions for Pharma & Life Sciences

We’re happy to discuss your needs and show you how Elsevier’s Solution can help.

Contact Sales