Making AI more ethical at Duke

At the crossroads of health care and technology, two leaders at Duke are not just driving innovation, they are ensuring that it is accessible and fair for all.  

Michael Pencina, a biostatistics and bioinformatics professor focusing on AI’s impact on health care, and computer science professor Nicki Washington, championing inclusivity in computer science education, share a core belief: Progress in AI and technology holds value when it benefits everyone.  

Obstacles such as bias in AI algorithms can have serious consequences, especially when the data used to train these systems is not representative. For instance, an AI tool that was designed to detect sepsis in patients performed poorly when deployed in hospitals outside the areas where the tool was trained. Another AI tool failed to properly identify Black patients who needed preventative care, demonstrating how systemic bias can persist in these technologies. ”

Pencina, director of Duke AI Health, emphasizes the critical need for thoughtful AI evaluation and implementation. AI’s potential to revolutionize health care is undeniable, with tools that help diagnose conditions, predict patient outcomes and improve medical treatments, he says. Still, there are issues that persist. 

“The opportunity is there, but there is a lot of potential to do things the wrong way,” says Pecina.  

Obstacles such as bias in AI algorithms can have serious consequences, especially when the data used to train these systems is not representative. For instance, an AI tool that was designed to detect sepsis in patients performed poorly when deployed in hospitals outside the areas where the tool was trained. Another AI tool failed to properly identify Black patients who needed preventative care, demonstrating how systemic bias can persist in these technologies.  

Pencina’s solution is clear: “I want to focus on evaluation,” ensuring AI tools work as intended and don’t perpetuate existing inequities in health care. 

While Pencina is focused on AI in health care, professor Washington is tackling similar issues within the realm of computer science. Her work is rooted in the belief that the tech industry can’t claim to be the best until it fosters inclusivity.  

“We need to make sure that every student, regardless of their identity, sees themselves in computing and no longer feels like they are othered,” she says.  

Washington is working to change the policies and perspectives in tech education that hinder marginalized students, advocating for systemic reforms that provide opportunities and remove barriers. Technology such as facial recognition has been shown not to work for people of color or people with disabilities, making it not only difficult to use, but marginalizing.  

“We’re all working towards the same solution,” she says. “We’re disrupting and dismantling the status quo.” 

Both Pencina and Washington are committed to the idea that progress should not come at the expense of equity. At Duke, their work aligns to ensure that technological advancement, whether in health care or computer science, is both ethical and inclusive. Washington’s focus is on creating pathways for students who have historically been excluded from tech, while Pencina’s work is focused on making sure AI is developed and deployed with fairness and accountability. Their efforts exemplify the understanding that true progress can only happen when every voice is heard, so everyone can succeed. 

“I see this becoming much more of a movement,” Washington says.