Several recent conversations with colleagues at NMC have me mulling over emergent challenges at the intersection of Generative AI and academic practices regarding sourcing and citation. Among these challenges, two seem most prominent. Both involve preserving the integrity of a chain of information—what came from where, or who contributed what—but in slightly different ways. The first involves how GenAI platforms approach sourcing and transparency in their AI-generated text output. The second (which will get its own post in the next CIE newsletter) deals with developing new norms around transparency and disclosure as it relates to human use of AI tools and their output in the authorship of our own work and the work of our students.

The first challenge (and the focus of this post) pertains to ‘information integrity’ within the output of generative AI chatbots such as ChatGPT and Google Gemini. Different tools take different approaches when it comes to transparency and integrity of sourcing. ChatGPT’s responses rarely give any clear indication of or credit to the source material they draw on. In large part, this is because it does not operate by ‘looking things up,’ but rather by treating all of its training data more like one vast source and generating text by replicating patterns within that data.

When prompted, ChatGPT can create a plausible and well-formed citation. Clearly, there are enough examples in its training data of citations in common styles like APA and MLA for it to replicate the general pattern. In some cases, the citations it creates will credit real authors and real publications, but where it often falls apart is the title and link. In a recent example from the work of an NMC student this semester, the References section included a citation of a non-existent article by a well-known neuroscientist in the well-known publication Psychology Today. While the author and publication exist, and the author’s work has been discussed in that publication, the article the student cited was a fabrication. Neither an article of that title, nor an article by that author on the topic, had ever appeared in Psychology Today. What’s more, although the citation included a syntactically correct URL on the correct domain for the magazine, the link itself was broken because it pointed to a nonexistent page on that domain. While a student could conceivably fabricate such a citation without the use of AI, to do so would probably take at least as much work as simply finding and using a real article. (Plus, the instructor in this case had other reasons to think this was the result of undisclosed use of AI, and the student admitted as much.)

ChatGPT’s rival platforms Google Gemini and Perplexity AI, however, aim to combine the power and connectivity of a real-time web search engine with the conversational style of a chatbot like ChatGPT. As such, their responses reflect a kind of splitting of the difference between the probabilistic summary/synthesis of ChatGPT on one hand, and an effort to transparently link the user to concrete ‘sources’ on the other. In this way, these tools function more like search engines with a chat layer built on top of them (in fact, Perplexity AI licenses GPT-4 for its chat capabilities, linking it with Perplexity’s search engine). This difference is evidence of a key difference in design goals than those that resulted in ChatGPT.

Where ChatGPT’s initial purpose was to mimic human dialogue convincingly enough to seem like an omniscient conversation partner, Google Gemini and Perplexity aim for a balance between ChatGPT’s omniscient conversationalist and a separate or parallel goal of maintaining some level of ‘information integrity’ and sourcing transparency. These tools’ design acknowledges that an impressive and high-relevance word salad summary or synthesis of sources unknown is of limited use, that there are many instances that call for tracing a claim or tidbit of information upstream to its original source where it can be understood in its primary context.

Broadly speaking then, these more search-oriented tools seem more useful and transparent than purely chat-focused tools like ChatGPT. However, Google Gemini and Perplexity still have their downsides when it comes to sourcing and citation. Sometimes Gemini will fail to provide outbound links to human-authored sources, and even when it does link out, its chat replies draw on more than simply the material in the few sources it links to, so its true transparency is limited. I have experimented less with Perplexity AI than the other two, but from my limited experience it seems to be the best (at least among the ‘free tier’ of these three tools) at emphasizing sourcing and transparency.

What can we learn from this, and what does it mean for us as educators?

Here is a bulleted list of some of my stray thoughts and takeaways. I’d love to hear your thoughts and takeaways too – either in the comments or in conversation, or at the upcoming GenAI Drop-in Office Hours on Wednesday, March 20 in TJNIC 117 or via Zoom.

• Free, widely used AI tools can generate plausible, well-formed citations; the title and URL (or DOI) are the most likely giveaways. However, checking individual citations is impractical unless there are other red flags in the students’ work. Asking the student to furnish the source may be a followup step in cases where you notice this going on.
• Even when tools (Gemini, Perplexity) do manage to offer up valid links to outside sources, students will be tempted to simply cite the source and use the paraphrase/summary without visiting the original (this is similar to citing a scholarly journal article having only skimmed the abstract and not engaged at all with the material)
• Furthermore, as jobs like ‘AI editor’ with 200-article-a-week quotas start to pop up, an increasing proportion of the pool of linkable information on the web will be authored by bots rather than humans (with whatever light human editorial intervention is possible when one’s expected output is so high); this means that even ‘real’ articles students may cite could well be mostly AI regurgitations rather than original work
• Since generative AI tools are pattern replicators, they mostly repackage and repeat patterns present in their training data; this makes them inherently derivative. They can sometimes create strange, surprising, or novel text, but by and large, the more an author relies on AI-generated text, the more likely a piece of writing will be to be a kind of recycle or rehash of familiar connections and ideas
• Unsourced or nontransparent AI-powered chat is exactly the kind of writing that educators teach students not to do in an academic setting – lacking attribution, no sense of what work is the author’s own thinking and what or where they drew from elsewhere. The increasing ubiquity of these tools makes it increasingly vital for educators to have these conversations with students early and often about the ‘whys’ behind sourcing and citation, as well as the ‘whys’ behind reading and writing in general.
• The lack of ‘information integrity’ as a priority in most GenAI tools poses real challenges for academe. Not only ethically and legally (see recent lawsuits by NYT and other authors against OpenAI), but epistemologically as well. Scholarship and knowledge-making have been documented for centuries as incremental, transparent chains of information. What does it mean to integrate a ‘black box’ algorithmic text generator into these processes? What new norms are required to ensure clarity and integrity of scholarly evidence and information?