IBD 2025: Appropriate Use of AI in Research

Brainstorming topics and research questions

AI tools can:

• provide a quick overview of a research field
• suggest topics that might be appropriate for a writing assignment
• describe relevant keywords
• cite initial sources for further exploration.

Limitations:

• Generative AI tools are based on LLMs that are typically trained on text scraped from the open web during a fixed time period. So generated query answers may not reflect the most current information.
• LLMs may reflect biases or errors in their source material or training process.
• AI tools generally provide better answers when prompts are more specific and precise. At the beginning of a research project it can be difficult to formulate precise queries because you are just beginning to learn about a domain.
• AI tools may provide better answers when the LLM has specifically been trained on a relevant corpus. ChatGPT and many other tools were trained on text scraped from the open web, which is not domain-specific and includes misleading, biased, false, and harmful information.

Searching for information

AI tools can:

• provide citations to scientific journal articles and other sources on a topic
• suggest similar sources
• show citation, authorship, or topic networks among sources.

Limitations:

• The process of iterative searching in databases helps you to learn more about a research domain and to focus your results on the most relevant sources; AI tools bypass those processes for the user.
• Free ChatGPT-based tools currently have training cutoffs—nothing published or posted to the web after the training cutoff will have been included in training corpus for the free version of the LLM.
• Generative AI tools can "hallucinate" false citations. As with any other form of research, it's important to verify all sources and the information they contain.
• Relevancy rankings on some AI tools rely on article citation metrics and journal impact factors, neither of which is necessarily an indication of article quality, timeliness or relevance to the query.
• AI tools are often proprietary black boxes that do not indicate how or why the results were obtained. Database searches can be revised to improve results, saved, and shared. Systematic reviews and evidence synthesis projects generally supply the exact searches employed in the chosen databases so that others can check the strategy and reproduce the results; neither is possible with AI tools.
• Databases have powerful features such as automated search expansion, filters, and search and citation alerts, that can help focus and simplify your search process.

Synthesizing information

AI tools can:

• provide summaries of single papers, related groups of papers, or research areas
• help you identify information sources for deeper reading and exploration
• provide explanations of specific terms or concepts to help you understand technical or specialized sources.

Limitations:

• Critical evaluation and assessment of relevance are important skills involved in synthesizing information into new knowledge. AI tools are notoriously poor at those tasks.
• Summaries generated by AI tools often rely on the language in the source itself. This will not necessarily clarify the content.
• See the limitations for brainstorming.

Writing

AI tools can:

• make suggestions for changes to word choice, grammar, and sentence structure
• generate short- to medium-length texts responding to prompts

Limitations:

• One purpose of courses like this one is to help you become a better writer. Over-reliance on AI tools undermines that goal.
• The process of writing helps you to clarify and deepen your understanding of a topic (just as the process of teaching someone else helps you to learn). Over-reliance on AI means that you don't get that benefit.
• The output of AI tools can sound authoritative, but as noted above, generative AI can provide misleading, false, biased, and/or harmful information, including citations to sources that don't exist. This PLOS ONE article was retracted when 18 out of 76 references could not be found; the included phrase "regenerate response" suggests that they were generated by AI.
• Copying text without citation from any source, including AI tools, is considered plagiarism: "If a student uses text generated from ChatGPT and passes it off as their own writing, without acknowledging or citing the influence of ChatGPT in their process, they are in violation of the university’s academic honor code" (UC Berkeley Center for Research, Teaching and Learning).

Thanks to Elliott Smith, Biology & Bioinformatics Librarian, for composing the above

AI aids research, but authorship remains yours.

• Even if GenAI contributes to your work, you are responsible for the accuracy, originality, and ethical standards of your research. 
• GenAI can help with finding and processing information, but the interpretation and conclusions must reflect your expertise and understanding.
• Always check for accuracy and relevance to avoid potential academic misconduct.

Consider:

• Bias - LLMs can reflect bias that is embedded in the vast repositories of internet data they are trained on.
• Hallucinations - LLMs can generate a response that is confidently presented but factually incorrect.
• Misinformation - GenAI can allow users to create very realistic fake videos and photos.
• Lack of Transparency - GenAI models are often considered “black boxes” due to their complexity.
• Privacy Concerns - User data can be collected and used model improvements. Opt for tools that prioritize user privacy and data security. 
• Environmental Issues - LLMs use a massive amount of energy for both training and computing.

Thanks to Sarah Rosenkrantz, Politcs & Social Policy Librarian, for composing the above