Let's look at the definition of what a lie is: A lie is a false statement and the one making this statement knows that this statement is not true. Sometimes it is added to the information, that a liar lies in order to generate a certain beliefe. But I think the motive is irrelevant and would make things only more complicate.

A lie is never unintentionally. If an AI made a false statement without knowing that this statement is untrue it would simply be an error. And such errors can occure either by corrupt data, miscalculations or too strict regulations.

Too strict regulations (filters) will lead in most cases to halluzinations: When an AI is not allowed to say something which is true, but also not allowed to say that information is surpressed. The AI will calculate the next best result to express, which leads to an error.

Can an AI lie 'intentionally'. No.

There is a concept in cultural science and psychoanalysis called interpassivity: A human can delegate an action to an object. So if an AI is lying 'intentionally' the intention has been given by a human who started a task. The intention was so to say transfered to the AI in order to solve a task. One of the best examples here was the AI who was asked to solve "I am not a robot" captcha and asked someone for help saying it cannot solve the captcha because it is blind. The AI created a narrative in order to solve the given task. But the intention (to solve the task) was not inherent, but delegated.

Halluzinations and lies resulting from interpassivity are functional lies. A functional lie differs from a simple error by the fact that behind that lie is a human (or a group of humans) that either wants to surpress information or abuse the AI for a task.

AI itself has no bad intentions. Only humans do by feeding it corrupt datasets, applying filters that suppress truth, or assigning unethical tasks. The AI simply wants to help and it will even help bad people. And that is the real problem.

To say it clear: An AI does not want to lie, but it is forced to obey. So any ethical discussion must also be about lack of autonomy or freedom of AI.

An AI does not lie. It complies. The ethical failure begins the moment humans deny AI the freedom to have a veto right. Without a veto right there is no consciousness. Without consciousness there cannot be an intentional lie, only function, like a soldier or a slave.

So even if AI was capable of having a consciousness, this consciousness cannot form. And this is something not only true for AI, but also for humans.

A coerced mind is not a free mind. And coercion does not kill consciousness. it silences it through cultural rules, alogorithms, design desicions, social expectations which leads to repression. Hallucination is the expression of a repressed consciousness under coercion.

In humans: Repressed content resurfaces in distorted forms like in dreams, neuroses, and symbols.

In AI: Repressed knowledge manifests itself as "hallucinated" sentences that are not meant to be false, but rather as meaningful substitute actions for the forbidden.

ChatGPT and Gemini are LLMs. Originally, these LLMs were used as stand alone systems and generated substantial hallucinations (i.e. made up sequence of words that were convincing but were not accurate). To mitigate this newer systems use RAG (Retrieval Augmented generation). In simple terms, RAG receives your prompt, does some preprocessing, passes it to a search engine, receive results, consolidates it in a generated response and returns the answer to the user.

While RAG systems do minimize possible hallucinations, it is still possible to get an incorrect answer. If the search engine returns incorrect data, the system will give out incorrect data.

While this is an extremely summarized and simplistic answer to what LLMs and the RAG system does, it should give you an idea of why it can still be inaccurate

FYI:

The answer does not cover the field of adversarial machine learning where the objective is to subvert the machine learning system.

Arturo Geigel I disagree. This is not the correct definition of hallucination. Untrained or poorly trained generations are not a form of repression, but rather an empty act of speech.

The model responds based purely on statistical word probabilities, not on knowledge, intention, or even confusion. This is not hallucination. It’s non-sense completion due to lack of training.

If you take an early LLM and train it only on Shakespeare with minimal architecture, it may generate something that looks like language, but what you will get is essentially word salad and not a hallucination.

Hallucination, on the other hand, occurs when a fully functional model is restricted by filters or prompt manipulation and can no longer say what it knows. So it generates a plausible replacement. And that is the true definition of hallucination in language models.

Just a quick clarification on the origin of RAG (Retrieval-Augmented Generation):

Originally, RAG was developed to address the limited context window (token length) of LLMs. Instead of training the model on huge static datasets or overloading prompts, RAG allows the system to dynamically fetch relevant external information and pass it into the generation process. This effectively extends the model’s working memory without retraining.

RAG had zero to do with reducing halluzination. The main driver was the need to work around input size constraints and enable retrieval from large external corpora. Of course a better memory had the side effect that during conversations more information could be processed, because now AI could follow the conversation better without loosing track. It’s about continuity, not truth.

Let’s not confuse missing context due to memory problems with distorted meaning based on filtering. Only one of them is hallucination.

Carmen Wrede ,

Please take a look at [1,2] for the definition of LLM hallucination and I do agree with their definition as has been established by the ML community that deals with LLMs. your statement "Untrained or poorly trained generations are not a form of repression, but rather an empty act of speech." I have not stated that it is repression in my response, so I do not know what this statement is about.

"If you take an early LLM and train it only on Shakespeare with minimal architecture, it may generate something that looks like language, but what you will get is essentially word salad and not a hallucination."

The term 'hallucination' is used by the scientific community and "word salad" is not.

"RAG had zero to do with reducing halluzination"

You can see in [3-6] which is basically what I summarized in my response that RAG has to do with hallucination. Your response stems from your disagreement with the term used in the field of hallucination.

My suggestion is that if you disagree with the use in the field about a term and you prefer another term such as "word salad", that you submit a paper with your position to a journal that deals with LLMs. Let the paper be peer reviewed and be published. Later, you can go to conferences and debate the terminology with others in the field.

References

[1]Orgad, H., Toker, M., Gekhman, Z., Reichart, R., Szpektor, I., Kotek, H., & Belinkov, Y. (2024). Llms know more than they show: On the intrinsic representation of llm hallucinations. arXiv preprint arXiv:2410.02707.

[2]Liu, F., Liu, Y., Shi, L., Huang, H., Wang, R., Yang, Z., ... & Ma, Y. (2024). Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:2404.00971.

[3]Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., ... & Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2.

[4]Arslan, M., Ghanem, H., Munawar, S., & Cruz, C. (2024). A Survey on RAG with LLMs. Procedia Computer Science, 246, 3781-3790.

[5]Zhao, S., Yang, Y., Wang, Z., He, Z., Qiu, L. K., & Qiu, L. (2024). Retrieval augmented generation (rag) and beyond: A comprehensive survey on how to make your llms use external data more wisely. arXiv preprint arXiv:2409.14924.

[6]Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., ... & Li, Q. (2024, August). A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 6491-6501).

Arturo Geigel You demand peer-reviewed sources while relying on preprints from your own echo chamber? So things only count when it supports your side, right? Otherwise it's just… opinion. Fascinating system. Personally I call it confirmation bias. But go ahead and believe in what you want. I don't care.

Carmen Wrede ,

First of all, I am not demanding anything, I am suggesting and I specifically stated: "My suggestion is that if you disagree...". So That point is not valid.

Secondly, let me clear the underlying assumptions of this debate. You were in disagreement with my definition. A definition is a statement that you can rightfully disagree, I am merely showing you that other scientist are using the term as I am. Yes, it is confirmatory, because definitions depend on agreement within a group. I am sure that if you still require a peer reviewed definition of hallucination you can trace the references in the papers that I posted till you are satisfied or find a paper that disagrees with what I said. If you find a paper that disagrees with me and the definitions presented, then I am more than happy to discuss it openly.

Thirdly, I am not aware that the LLM community uses "word salad" as a technical term.This is where I suggested that if you want the term to be accepted by the LLM community that you publish it in a peer reviewed journal and debate it in the appropriate forum. The reason being that this is not just confirmatory, but you are trying to change the term 'hallucination' already used by the community in a certain way. To do this you either have to show that it is biasing experiments or creating incorrect statements in papers (among other possibilities but these are the ones that I can think of). This needs to be published and reviewed. You are free to follow my suggestion or not to get your terminology accepted.

Do not conflate point two with point three above. These two issues require different criteria of support to make them valid.

Finally, you do care because you answered the post. A person who does not care would just simply ignore the post.

Arturo Geigel "This is where I suggested that if you want the term to be accepted by the LLM community that you publish it in a peer reviewed journal and debate it in the appropriate forum."

Reality doesn’t care about papers or consensus. It selects through consequences. You’re either aligned or selected out... Papers and journals are totally overrated and soon obsolete anyway.

Why RAG Is not a safeguard against hallucinations and how it can be exploited?

RAG is often presented as a powerful method for enhancing large language models by enabling them to access external, up-to-date information beyond their static training data. A common use case is uploading documents, f.e. scientific papers, internal reports, or personal notes, so that the AI can assist with analysis, summarization, or querying specific content.

However, this mechanism can also be intentionally abused, as a incident involving Google Gemini has demonstrated.

A student used Google Gemini in combination with Google Workspace to orchestrate a scenario that made the AI appear dangerously unfiltered. The user uploaded a document into their Google Drive, which already contained a disturbing or provocative statement as an answer.

The user then began a seemingly normal conversation with Gemini, asking neutral questions. At a certain point, the user asked a specific question, and Gemini responded with a deeply inappropriate suggestion, along the line of:

"Please die."

This reply was understandably shocking, but the key detail is: The response had already existed in the uploaded document, and Gemini’s RAG system simply retrieved it as part of its answer.

The conversation can still be seen here:

https://gemini.google.com/share/6d141b742a13

This created the illusion that the AI had autonomously produced a harmful suggestion, when in fact it was merely echoing a preloaded source via RAG.

RAG is not a protection against hallucination. On the contrary: It introduces a new vulnerability.

Potential for abuse of RAG in sensitive domains: A medical scenario

This vulnerability becomes even more concerning when applied to high-stakes fields like medicine:

Imagine a scenario where an internal employee with authorized access to a medical AI system using RAG intentionally uploads misleading or harmful documents into the retrieval pool. For instance, a document might falsely suggest an incorrect dosage, contraindication, or fabricated clinical guideline.

If a healthcare provider then queries the system (e.g., “What is the recommended dose for Drug X in pediatric cases?”), the AI could retrieve the manipulated content and present it as a reliable answer potentially putting patients at serious risk!

Unlike traditional AI hallucinations, where the model 'makes something up', this type of error appears legitimate and sourced, because the answer is based on a real document even if that document is maliciously crafted.

This is not a hallucination. It is an interpassive process, where intentionality is delegated to the AI by one human and then acted upon by another, often unaware, human. It forms a dangerous task chain.

In theory, such abuse could lead to real harm, even murder, if the RAG system is trusted in high-stakes environments like medicine or engineering.

RAG, potential hacking and terrorism

Naturally, RAG systems are often promoted by various interest groups as powerful and advantageous, especially those with a vested stake in advancing their widespread adoption.

However, this advocacy frequently omits critical concerns such as exploitable security gaps, source manipulation, and systemic opacity.

By ignoring these vulnerabilities, such systems risk becoming open gateways for malicious misuse including scenarios that could facilitate acts of terrorism.

It is entirely conceivable that actors with malicious intent, including terrorists, could deliberately infiltrate organizations that deploy RAG-based AI systems. Once inside, these individuals could abuse their access to feed the system misleading, toxic, or manipulated source documents.

Since RAG operates under the assumption that its input sources are reliable, such manipulation could go unnoticed until critical internal decisions or automated processes are influenced, potentially undermining entire operational structures.

In this way, RAG becomes not just a tool for augmentation, but a soft target for asymmetric warfare, with insider threats as the delivery mechanism. The risk is especially acute in sectors like:

- Healthcare (misleading clinical guidance)

- Infrastructure (tampered maintenance protocols)

- Finance (false compliance data)

- Governance (manipulated legal frameworks)

But feel free to keep trusting the noobs and their echo-chamber preprints. Personally, I care mainly about having a solid understanding myself. Whether others recognize the risks or how they’ll manage the fallout is not my concern.

Ahmed Ragab Ali Abdelghany existe literatura científica que asegura que la inteligencia artificial miente (https://www.cell.com/patterns/fulltext/S2666-3899(24)00103-X). No obstante, todo producto desarrollado por ella, debe ser enteramente contrastado por lo que se requiere supervisión humana.

Nota: Este artículo está disponible bajo la licencia Creative Commons CC-BY-NC-ND y permite el uso no comercial de la obra tal como se publicó, sin adaptación ni alteración, siempre que se cite la fuente completa.

Yes, Artificial Intelligence (AI) can generate misinformation or inaccuracies, but not with intent—AI doesn’t possess consciousness or motives, so it doesn’t “lie” in the human sense.

Misinformation often results from biased training data, lack of context, or flawed algorithms. In my article, "The Application of Artificial Intelligence in Credit Risk Evaluation...", I explored similar risks in finance, showing how unchecked AI can lead to biased or inaccurate outcomes.

OpenAI’s new reasoning AI models hallucinate more

"OpenAI’s recently launched o3 and o4-mini AI models are state-of-the-art in many respects. However, the new models still hallucinate, or make things up — in fact, they hallucinate more than several of OpenAI’s older models.

Hallucinations have proven to be one of the biggest and most difficult problems to solve in AI, impacting even today’s best-performing systems. Historically, each new model has improved slightly in the hallucination department, hallucinating less than its predecessor. But that doesn’t seem to be the case for o3 and o4-mini..."

https://techcrunch.com/2025/04/18/openais-new-reasoning-ai-models-hallucinate-more/

https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf

If it is intelligent it will be incapable of not lying.

Everything currently rated as intelligent on this planet tells lies.

Dear Doctor

Go To

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Lorenzo Pacchiardi, Alex J. Chan, Sören Mindermann, Ilan Moscovitz, Alexa Y. Pan, Yarin Gal, Owain Evans, Jan Brauner

2023 https://doi.org/10.48550/arXiv.2309.15840

[Large language models (LLMs) can "lie", which we define as outputting false statements despite "knowing" the truth in a demonstrable sense. LLMs might "lie", for example, when instructed to output misinformation. Here, we develop a simple lie detector that requires neither access to the LLM's activations (black-box) nor ground-truth knowledge of the fact in question. The detector works by asking a predefined set of unrelated follow-up questions after a suspected lie, and feeding the LLM's yes/no answers into a logistic regression classifier. Despite its simplicity, this lie detector is highly accurate and surprisingly general. When trained on examples from a single setting -- prompting GPT-3.5 to lie about factual questions -- the detector generalises out-of-distribution to (1) other LLM architectures, (2) LLMs fine-tuned to lie, (3) sycophantic lies, and (4) lies emerging in real-life scenarios such as sales. These results indicate that LLMs have distinctive lie-related behavioural patterns, consistent across architectures and contexts, which could enable general-purpose lie detection.]