AI Literacy: Uses and limitations of AI tools
As when using any tool, it's important to understand the strengths and limitations of AI tools before choosing to use them. Some considerations to be aware of from the start:
- "Artificial Intelligence" tools are not intelligent: Large Language Models (LLMs) ingest a vast text corpus and are then trained to produce text based on statistical predictions of word probability and order based on context. Although they generate text that sounds authoritative, these agents do not, and cannot, evaluate the accuracy of their own output, and can produce responses that are biased, false, or harmful.
- Free generative AI tools currently have knowledge cut-off dates: Nothing published or posted to the web after the knowledge cut-off date will have been included in the training corpus for the free version of the LLM, so any information it references will date from the knowledge cut-off or earlier.
- Many AI resources are not supported tools at UC Berkeley: They have not been reviewed for accessibility, data privacy, or security.
For more information on each of these considerations, please see the AI Resources box on this page.
Outlined below are some possible uses of AI tools in literature searching and writing, along with limitations to consider. But when considering any use, be sure always to follow your instructor's AI policy.
Brainstorming topics and research questions
Image: Drtonymc, CC BY-SA 3.0, via Wikimedia Commons
AI tools can:
- provide a quick overview of a research field
- suggest topics that might be appropriate for a writing assignment
- describe relevant keywords
- cite initial sources for further exploration.
Limitations:
- Knowledge cut-offs: Generative AI tools are based on LLMs that are typically trained on text scraped from the open web during a fixed time period. So generated query answers may not reflect the most current information.
- Potential bias: LLMs may reflect biases or errors in their source material or training process.
- Ineffective prompts: AI tools generally provide better answers when prompts are more specific and precise. At the beginning of a research project it can be difficult to formulate precise queries because you are just beginning to learn about a domain.
- Generality: AI tools may provide better answers when the LLM has specifically been trained on a relevant corpus. ChatGPT and many other tools were trained on text scraped from the open web, which is not domain-specific and includes misleading, biased, false, and harmful information.
Searching for information
Image (detail): Jean-François Millet, Des glaneuses (The Gleaners), 1857. Source: Musée d'Orsay
AI tools can:
- provide citations to scientific journal articles and other sources on a topic
- suggest similar sources
- show citation, authorship, or topic networks among sources.
Limitations:
- Knowledge development is bypassed: The process of iterative searching in databases helps you to learn more about a research domain and to focus your results on the most relevant sources; AI tools bypass those processes for the user.
- Irreproducible results: AI tools are often proprietary black boxes that do not indicate how or why the results were obtained. Database searches can be revised to improve results, saved, and shared. Systematic reviews and evidence synthesis projects generally supply the exact searches employed in the chosen databases so that others can check the strategy and reproduce the results; neither is possible with AI tools.
- Lack of database search functions: Databases have powerful features such as automated search expansion, filters, and search and citation alerts, that can help focus and simplify your search process.
- Knowledge cut-offs: Free generative AI tools currently have knowledge cut-offs—nothing published or posted to the web after the knowledge cutoff will have been included in training corpus for the free version of the LLM, and so the information generated will not reflect the most current knowledge. GPT-5 has a cut-off of October 2024.
- Poor relevancy rankings: Relevancy rankings on some AI tools rely on article citation metrics and journal impact factors, neither of which is necessarily an indication of article quality, timeliness or relevance to the query.
- False citations: Generative AI tools can "hallucinate" false citations. As with any other form of research, it's important to verify all sources and the information they contain.
- Example: A September 2025 Nature News article points out that generative AI "hallucinations" are inevitable due to the intrinsic way in which LLMs work.
- Example: In August 2025 a researcher queried a generative AI chatbot and received the following citation:
Slater PJB. 2003. Song complexity in tropical birds: A review. Animal Behaviour 66(4): 1-10.
The journal Animal Behaviour has continuous page numbering: Vol. 66 Issue 4 (2003) starts on p. 617. While PJB Slater is the real author of an article on bird songs, no article of this title exists in any journal.
- Example: Retraction Watch reported in June 2025 that a book published by Springer Nature two months previously, Mastering Machine Learning: From Basics to Advanced by Govindakumar Madhavan, "is full of made-up citations." The author of the post, Rita Aksenfeld, notes that "nonexistent and error-prone citations are a hallmark of text generated by large language models like ChatGPT." The publisher has since retracted the book.
- Example: A summer reading list generated by AI that was published in the Chicago Sun-Times and other newspapers in May 2025 contained nonexistent books.
- Example: This 2024 PLOS ONE article was retracted when 18 out of 76 references could not be found; the included phrase "regenerate response" suggests that they were generated by AI.
Synthesizing information
Image (detail): ButuCC, CC BY-SA 3.0, via Wikimedia Commons
AI tools can:
- provide summaries of single papers, related groups of papers, or research areas
- help you identify information sources for deeper reading and exploration
- provide explanations of specific terms or concepts to help you understand technical or specialized sources.
Limitations:
- Lack of critical evaluation and assessment: Critical evaluation and assessment of relevance are important skills involved in synthesizing information into new knowledge. AI tools are notoriously poor at those tasks.
- Summaries can be circular: Summaries generated by AI tools often rely on the language in the source itself. This will not necessarily clarify the content.
- See the limitations for brainstorming.
Writing
Image (detail): Subhashish Panigrahi, Wikimedia Commons, CC-BY-SA 4.0
AI tools can:
- make suggestions for changes to word choice, grammar, and sentence structure
- generate short- to medium-length texts responding to prompts
Limitations:
- Undermines writing skills: One purpose of courses like this one is to help you become a better writer. Over-reliance on AI tools undermines that goal.
- Example: A 2025 MIT study. "Your brain on ChatGPT," found that using ChatGPT in writing assignments may negatively affect "student learning, creativity, and writing skills."
- Reduces understanding: The process of writing helps you to clarify and deepen your understanding of a topic (just as the process of teaching someone else helps you to learn). Over-reliance on AI means that you don't get that benefit.
- False or biased output: The output of AI tools can sound authoritative, but as noted above, generative AI can provide misleading, false, biased, and/or harmful information, including citations to sources that don't exist.
- Can violate the honor code: Copying text without citation from any source, including AI tools, is considered plagiarism: "If a student uses text generated from ChatGPT and passes it off as their own writing, without acknowledging or citing the influence of ChatGPT in their process, they are in violation of the university’s academic honor code" (UC Berkeley Center for Research, Teaching and Learning).
