Researchers in the Genetics-Biotechnology Center at the University of Wisconsin-Madison, led by associate professor Qiongshi Lu, published a study Sept. 30 in Nature Genetics raising concerns over persistent problems found in artificial intelligence-assisted genome-wide association studies.
Broadly, genome-wide association studies search for links between genetic variation and health traits. This process requires large amounts of genetic and health related data, which come from “biobanks” that collect and store biological data. Based on links found in these datasets, scientists can predict future health risks for people with certain genetic traits.
But attempting to predict certain health risks is challenging.
“Some outcomes are very easy to measure. If you want to study the genetics of height, it’s easy to measure everyone’s height and DNA and just link them to identify associations. But if you want to study, say, Alzheimer’s, it gets tricky,” Lu said.
Collecting and studying the traits of diseases like Alzheimer’s in large numbers is difficult because there isn’t enough prevalence in the datasets. This is mostly because collecting this type of data is extremely resource intensive. Without enough data, researchers don’t have the statistical power to make health-risk predictions.
As a result, it has become popular in recent years to leverage AI tools, which can infer unknown traits based on proxy data. This allows scientists to work around the challenge of missing data by assigning traits to individuals in whom that trait has not been actually observed. But Lu said the inferences don’t account for their own uncertainty.
Jiacheng Miao, a Ph.D. student in the Biomedical Data Science Program at UW-Madison and co-author of the study published in Nature Genetics, told The Daily Cardinal many scientists “are pretending that outcomes produced by AI are the gold standard, but we show that this is actually a pretty bad idea.”
“For example, we have hospital-recorded diabetes statuses of individuals. So we created an AI predicted diabetes status to compare the results, and we found that they differ a lot,” he said.
Further comparisons revealed persistent flaws in traits produced by AI. In response to these findings, Lu’s team developed a new statistical framework to improve the quality of AI inferences. While the results have been well-received within the field of genome research, they have not yet been widely implemented since Lu and his colleagues are some of the first to advance this discussion in genome-wide association studies, Lu said.
Impacts
The results of genome research are often used in developing therapeutic drugs. However, it’s possible that without the right statistical frameworks in place, these drugs will be developed based on contaminated data. While drugs built on false data would likely not make it past testing stages, Lu said developing those ineffective treatments “is a huge waste of resources.”
“You could spend 10 years focusing on a therapeutic intervention strategy just to realize it’s false,” Lu said. “We need frontier methodology to guard people against spurious findings, so that we can really efficiently advance the field of medical sciences.”
Marco is a features writer for The Daily Cardinal. He is an English and History major and has experience covering local businesses.
