{"id":361,"date":"2023-08-10T09:58:44","date_gmt":"2023-08-10T13:58:44","guid":{"rendered":"https:\/\/www.yorku.ca\/ascend\/?post_type=article&p=361"},"modified":"2023-08-10T15:22:33","modified_gmt":"2023-08-10T19:22:33","slug":"uncovering-racial-bias-in-ai","status":"publish","type":"article","link":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/","title":{"rendered":"Uncovering racial bias in AI"},"content":{"rendered":"\n

For Laleh Seyyed-Kalantari, <\/strong>assistant professor in the Department of Electrical Engineering and Computer Science in the Lassonde School of Engineering, conducting research on the fairness of AI in healthcare and imaging diagnostics is personal. <\/p>\n\n\n\n

While working on research in developing optimization algorithms, Seyyed-Kalantari\u2019s life was derailed by a medical misdiagnosis that took two years to resolve. It was an agonizing experience, but it also inspired her to find a way to prevent the same thing from happening to others. <\/p>\n\n\n\n

\u201cI saw the impact of having an under-diagnosis in my personal life,\u201d she says. \u201cI remember wishing that my research could help patients reduce pain.\u201d <\/p>\n\n\n\n

Seyyed-Kalantari found a way to do just that during her post-doctoral fellowships at the Vector Institute and the University of Toronto, where she began investigating inaccuracies in AI diagnostics\u2014work that she continues to pursue at York. <\/p>\n\n\n\n

\"Laleh
Laleh Seyyed-Kalantari, Lassonde School of Engineering<\/figcaption><\/figure>\n\n\n\n

And she\u2019s uncovered some troubling findings that have caught even her by surprise.<\/p>\n\n\n\n

In a paper published in Nature Medicine<\/em>, Seyyed-Kalantari, as the lead author, examined data from multiple sources in the U.S. and discovered that AI-driven screening tools of chest X-rays had a concerning rate of under-diagnosis among underserved patient populations. <\/p>\n\n\n\n

\u201cHistorically vulnerable subpopulations\u2014for example, Black, Hispanic, female and low-income patients\u2014are suffering more from AI mistakes in these algorithms compared to other sub-populations,\u201d <\/p><\/blockquote>\n\n\n\n

she explains, noting that under-diagnoses are particularly harmful. \u201cUpon deployment of such AI models, these patients were wrongly diagnosed as healthy. That means they may not have received any treatment in a timely manner and were sent home without further assessments.\u201d<\/p>\n\n\n\n

Even radiologists were taken aback, she added. \u201cThey said that when reviewing patient results, they don\u2019t know anything about the patient\u2019s race. They are sitting in a dark room reviewing the images and asking themselves \u2018How can we be unfair to a patient that we have never seen?\u2019\u201d <\/p>\n\n\n\n

So, Seyyed-Kalantari and her multi-disciplinary, multi-institutional team delved further, looking into whether AI could determine the race of a patient from X-rays, CT scans and mammographic images alone. The results, published in The Lancet Digital Health<\/em>, were astonishing.<\/p>\n\n\n\n

\u201cWith a very high accuracy, AI models can determine the race of the patient by just looking at medical images,\u201d<\/p><\/blockquote>\n\n\n\n

says Seyyed-Kalantari. \u201cEverybody was surprised, and it was alarming. We don't know what AI is doing with this information. While we find that AI models can detect the race of patients, we find at the same time that AI is behaving against some races.\u201d<\/p>\n\n\n\n

The question now is: how is it happening? And what are the repercussions? <\/p>\n\n\n\n

Seyyed-Kalantari is actively looking for answers as she delves deeper into her work at Lassonde. In the meantime, she urges caution in embedding AI into healthcare. <\/p>\n\n\n\n

\u201cSome AI algorithms have received FDA approval in the U.S. for applications in radiology. But to the best of my knowledge, they haven\u2019t proven that the
algorithms are fair,\u201d she warns. <\/p><\/blockquote>\n\n\n\n

\u201cIf we\u2019re deploying these algorithms and using them for disease diagnoses -and we\u2019re not sure if they\u2019re fair or not -this could harm some groups in our society.\u201d<\/p>\n\n\n\n

This concern underscores why research like Seyyed-Kalantari\u2019s is so important, and how further research in this field can help ensure that the healthcare of tomorrow is truly fair and equitable. <\/p>\n","protected":false},"template":"","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":""},"categories":[3],"tags":[38],"class_list":["post-361","article","type-article","status-publish","hentry","category-tech-engineering","tag-lassonde-school-of-engineering"],"yoast_head":"\nUncovering racial bias in AI - Ascend Magazine<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Uncovering racial bias in AI - Ascend Magazine\" \/>\n<meta property=\"og:description\" content=\"For Laleh Seyyed-Kalantari, assistant professor in the Department of Electrical Engineering and Computer Science in the Lassonde School of Engineering, conducting research on the fairness of AI in healthcare and imaging diagnostics is personal.  While working on research in developing optimization algorithms, Seyyed-Kalantari\u2019s life was derailed by a medical misdiagnosis that took two years to […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/\" \/>\n<meta property=\"og:site_name\" content=\"Ascend Magazine\" \/>\n<meta property=\"article:modified_time\" content=\"2023-08-10T19:22:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.yorku.ca\/ascend\/wp-content\/uploads\/sites\/689\/2023\/07\/leah-1024x424.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/\",\"url\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/\",\"name\":\"Uncovering racial bias in AI - Ascend Magazine\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/wp-content\\\/uploads\\\/sites\\\/689\\\/2023\\\/07\\\/leah-1024x424.jpg\",\"datePublished\":\"2023-08-10T13:58:44+00:00\",\"dateModified\":\"2023-08-10T19:22:33+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/#breadcrumb\"},\"inLanguage\":\"en-CA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CA\",\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/wp-content\\\/uploads\\\/sites\\\/689\\\/2023\\\/07\\\/leah.jpg\",\"contentUrl\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/wp-content\\\/uploads\\\/sites\\\/689\\\/2023\\\/07\\\/leah.jpg\",\"width\":2048,\"height\":848},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/uncovering-racial-bias-in-ai\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Articles\",\"item\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/article\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Uncovering racial bias in AI\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/#website\",\"url\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/\",\"name\":\"Ascend Magazine\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.yorku.ca\\\/ascend\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-CA\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Uncovering racial bias in AI - Ascend Magazine","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/","og_locale":"en_US","og_type":"article","og_title":"Uncovering racial bias in AI - Ascend Magazine","og_description":"For Laleh Seyyed-Kalantari, assistant professor in the Department of Electrical Engineering and Computer Science in the Lassonde School of Engineering, conducting research on the fairness of AI in healthcare and imaging diagnostics is personal.  While working on research in developing optimization algorithms, Seyyed-Kalantari\u2019s life was derailed by a medical misdiagnosis that took two years to […]","og_url":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/","og_site_name":"Ascend Magazine","article_modified_time":"2023-08-10T19:22:33+00:00","og_image":[{"url":"https:\/\/www.yorku.ca\/ascend\/wp-content\/uploads\/sites\/689\/2023\/07\/leah-1024x424.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/","url":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/","name":"Uncovering racial bias in AI - Ascend Magazine","isPartOf":{"@id":"https:\/\/www.yorku.ca\/ascend\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/#primaryimage"},"image":{"@id":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/#primaryimage"},"thumbnailUrl":"https:\/\/www.yorku.ca\/ascend\/wp-content\/uploads\/sites\/689\/2023\/07\/leah-1024x424.jpg","datePublished":"2023-08-10T13:58:44+00:00","dateModified":"2023-08-10T19:22:33+00:00","breadcrumb":{"@id":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/#breadcrumb"},"inLanguage":"en-CA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/"]}]},{"@type":"ImageObject","inLanguage":"en-CA","@id":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/#primaryimage","url":"https:\/\/www.yorku.ca\/ascend\/wp-content\/uploads\/sites\/689\/2023\/07\/leah.jpg","contentUrl":"https:\/\/www.yorku.ca\/ascend\/wp-content\/uploads\/sites\/689\/2023\/07\/leah.jpg","width":2048,"height":848},{"@type":"BreadcrumbList","@id":"https:\/\/www.yorku.ca\/ascend\/article\/uncovering-racial-bias-in-ai\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.yorku.ca\/ascend\/"},{"@type":"ListItem","position":2,"name":"Articles","item":"https:\/\/www.yorku.ca\/ascend\/article\/"},{"@type":"ListItem","position":3,"name":"Uncovering racial bias in AI"}]},{"@type":"WebSite","@id":"https:\/\/www.yorku.ca\/ascend\/#website","url":"https:\/\/www.yorku.ca\/ascend\/","name":"Ascend Magazine","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.yorku.ca\/ascend\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-CA"}]}},"taxonomy_info":{"category":[{"value":3,"label":"Tech & Engineering"}],"post_tag":[{"value":38,"label":"Lassonde School of Engineering"}]},"featured_image_src_large":[],"author_info":[],"comment_info":"","_links":{"self":[{"href":"https:\/\/www.yorku.ca\/ascend\/wp-json\/wp\/v2\/article\/361","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.yorku.ca\/ascend\/wp-json\/wp\/v2\/article"}],"about":[{"href":"https:\/\/www.yorku.ca\/ascend\/wp-json\/wp\/v2\/types\/article"}],"wp:attachment":[{"href":"https:\/\/www.yorku.ca\/ascend\/wp-json\/wp\/v2\/media?parent=361"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.yorku.ca\/ascend\/wp-json\/wp\/v2\/categories?post=361"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.yorku.ca\/ascend\/wp-json\/wp\/v2\/tags?post=361"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}