There are a growing number of AI assistive tools available to help researchers. Be wary of how these tools are marketed. Ask yourself, if it is appropriate to use the tool for the stated purpose? For example, currently, it is not permissible to outsource your academic work to a third party and submit it as your own work. The development of AI assistive tools has not changed that. It is inadvisable, unless you are specifically instructed, to use a tool to write your entire assignment, report or literature review, no matter what the product says it can do.
- Are you permitted to use Gen AI or AI assistive tools in this context?
- If you are planning to publish, what are the publisher’s rules on using AI?
- You will be required to document where, how and why you used AI in your work, are you prepared acknowledge that?
- What problem are you trying to solve?
- Is there an appropriate tool available for this?
- When selecting a tool to use, ask yourself
- What are the information sources for this product? For example, Sematic Scholar, OpenAlex, etc.
- If the tool claims to use the “top” papers in a field, clarify the metric that is used to identify the top papers
- How does it deal with retracted papers?
- Does it have a generative search capability? Using Retrieval Augmented Generation (RAG).
- If it does use RAG, what does the knowledge base consist of? Does it have access to open access materials only or subscription materials?
- Are you aware of the limitations of this product?
- For products that offer to summarise documents, remember that in most cases, you already have a summary of the document, in the form of an abstract. Ask yourself what is the added value of an AI generated summary?
For products that say they analyse key points, ask yourself
-
- How can you check the quality and accuracy of the analysis?
- How do you plan to use the analysis?
- Do you plan to use the analysis to avoid reading the original text? If so, you may miss out on spotting something that is important to your research.
- Do you plan to quote from AI generated analysis? How would you cite that?
Use the ROBOT framework to evaluate the tool. Look for product reviews. Check the FAQ section and look for information on data privacy and the limitations of the product.
Evaluate the Output
Generative AI is continuing to develop new capabilities and improve, but it still makes mistakes or hallucinates. These errors may look credible, but you should check any material that is generated by an AI. This includes generated references.
- Generated summaries or analysis may not be accurate, these should be verified. Be critical in your evaluation of material.
- Does this content match what you already know?
- How does this content compare to your notes or other information you have gathered?
- Does this evidence support the conclusions presented in the generated content?
- AI tools have limitations. Providers are usually open about this, and you should be able to find details of known issues in the frequently asked questions (FAQ) section.
- Don’t use one single information source or rely on an AI tool to complete your work. Use authoritative sources from the library to validate information generated by AI products.
- Remember that bias is complex and can come from multiple sources including you. Using a lateral reading strategy, like SIFT, can help you to identify biases in material. Lateral Reading involves looking for information on your topic by different writers to compare how different authors deal with and view the same topic.
- Key to creating a new perspective on the literature is the unique voice that you, the author, brings to the work. Relying solely on any single AI tool or source can mean that the unique perspective of the author is potentially lost.
- Can you identify your voice in the work?
- How have you used the synthesis of literature to create something different and unique?
