Dear Gutemberg,

This is exactly a question I wanted to answer with two research projects. They are currently under review so nothing has been published yet but have a look at my book from 2018 which looks at the issue (looking then at GPT-2) Book Spreading Activation, Lexical Priming and the Semantic Web

Michael T.L. Pace-Siggea

a University of Eastern Finland 1

Abstract

This paper revisits Michael Hoey’s Lexical Priming Theory (2005) in the light of recent discussions of Large Language Models forms of machine learning (commonly referred to as AI) which have found a lot of publicity in the wake of tools like OpenAI’s ChatGPT or Google’s BARD/Gemini. Historically, theories of language faced inherent difficulties, given language's exclusive use by humans and the complexities involved in studying language acquisition and processing. The intersection between Hoey's theory and Machine Learning tools, particularly those employing Large Language Models (LLMs), has been highlighted by several researchers. Hoey's theory relies on the psychological concept of priming, aligning with approaches dating back to Ross M. Quillian's 1960s proposal for a "Teachable Language Comprehender." The theory posits that every word is primed for discourse based on cumulative effects, a concept mirrored in how LLMs are trained on vast corpora of text data.

This paper tests LLM-produced samples against naturally (human-)produced material in the light of a number of language usage situations, investigates results from A.I. research and compares the results with how Hoey describes his theory. While LLMs can display a high degree of structural integrity and coherence, they still appear to fall short of meeting human-language criteria which include grounding and the objective to meet a communicative need.

What a Nano-GPT can (not) tell us about Spoken Language

Michael Pace-Sigge, University of Eastern Finland

Evangs Mailoa, Universitas Kristen Satya Wacana, Indonesia

Vishwanath Singh, University of Eastern Finland

Abstract

After the launch of Chat-GPT in autumn 2022, a lot of research has focussed on the quality and near-naturalness which Large Language Model (LLM)-based tools present in the texts produced. While one area of research focussed on the similarities and differences between machine-produced and human-produced output (e.g. Berber Sardinha, 2024), others explored in how far such tools could process more complex tasks (e.g. Valmeekam, et al., 2023; Curry et al., 2024).

While it can be assumed that Chat-GPT makes use of written-to-be-spoken training material, there has been no investigation, as yet that looks at in how far a Generative Pre-trained Transformer (GPT) algorithm is able to process (transcribed) natural, colloquial language. This research will investigate whether spoken language transcripts lead to processing difficulties; whether such generated language can be seen as a suitable reflection of natural speech; and whether machine produced texts offer new insights into the workings of language.