DSU team researching Rag LLMs for trustworthy AI results
August 1, 2025
While large language models like ChatGPT have flourished in the last couple of years, their potential for hallucinating and making up information makes them an untrustworthy source, limiting their use in professional capacities. A faculty-student research team at Dakota State University is hoping to change that.
Dr. Andy Behrens, assistant professor and undergraduate coordinator for Information Systems, is leading the team in developing a design theory to improve the transparency and reliability of artificial intelligence (AI) systems. The rest of the team consists of Dr. Jason Mixon and undergraduate students Irina Pecherskaia and Andrew Smith. Together they are developing a design theory to provide more accurate results from a retrieval-augmented generation (RAG) large language models (LLMs) by sourcing the information from an external, verifiable database.
“Large language models like ChatGPT are incredibly powerful, but they have a known issue of hallucinations,” Behrens said. “They can make up information that sounds plausible but is completely fabricated. That’s a serious limitation in domains like healthcare where accuracy is critical.”
“You really want to avoid a model doing hallucinations,” Pecherskaia said. “You want to know the exact information because some of those scenarios are really critical in areas like healthcare, law, or cybersecurity.”
RAG LLMs are a relatively new AI system, first introduced in 2019. The DSU research team analyzed over 20 academic papers on the topic which helped them identify potential critical gaps the systems.
“The RAG system tells the AI to look in a specific database and generate responses only from that trusted information,” explained Smith. “That can make it significantly more accurate.”
The team’s next steps are to publish their findings in a research paper, followed by designing and implementing their own RAG LLM model to be used in a clinical decision-making environment.
In clinical settings, software is already used to provide decision-support tools, so a reliable RAG LLM would be a significant advancement in clinical or diagnostic settings. In the future, a RAG LLM could help clinicians make decisions and potentially diagnose issues faster, while increasing successful treatment plans.
“The advantages of RAG LLMs are that they pull from a lot of different sources, records, and databases,” Pecherskaia said. “It can be more helpful than a human because a human can’t remember millions of records of data. Humans can make a decision based on experience, but they don’t always have access to the existing, evolving data.”
Beyond industry impact, the project also offers a unique, resume building experience for undergraduate students.
“It’s a great opportunity to do a research project as an undergraduate student,” Pecherskaia said. “I don’t think a lot of students get that opportunity.”