Since the arrival of generative AI models, many organisations and users have been capitalising on these systems for their content, monitoring, documentary reports, and more. The responses are quick, structured, and often convincing. So much so that a confusion has set in: that of considering these systems as truly «intelligent».
This perception is not trivial. It directly influences the way decisions are made, sometimes overestimating the reliability of the results produced. Before relying on these tools, it is necessary to understand what they actually do... and what they do not.
Why do generative AIs give the impression of understanding?
Since the advent of generative AI models, we seem to be increasingly confusing performance with understanding. We can't see the mechanics, so we imagine a mind. Our brains, unable to tolerate emptiness, attribute intention, reasoning or meaning where there is merely a statistical sequence.
There's nothing modern about this confusion; it arises as soon as an inert object borrows our codes: it's anthropomorphism. This phenomenon can also be found in military robotics. Some soldiers deployed with robot dogs develop a form of attachment to the robot. The robot moves forward, explores and returns. It repeats a stable, almost familiar behaviour. When it is destroyed, it is not just a tool that is lost: it is an imagined presence. As soon as a system works, reacts or responds, we lend it a form of intention.
The tendency to overestimate the capabilities of a conversational system and attribute to it an understanding or intelligence that it does not possess. Named after ELIZA, a programme created by Joseph Weizenbaum at MIT (1964-1966) which simulated a psychotherapist: although it merely manipulated linguistic patterns, its users confided intimate thoughts to it, convinced that it truly «understood» them. See the full definition
The rise of language models has only amplified this illusion: when an AI nuances, contextualises, or reformulates, our minds naturally lean towards the idea of reasoning. This cognitive mechanism is not neutral: in business, it can lead to granting AI a level of reliability or understanding that it does not possess.
Why chatting with an AI isn't a real conversation.
Another source of confusion is the very notion of a «conversation» with LLMs. When a user interacts with a generative AI, they naturally feel as though they are having a continuous dialogue. However, the model does not «follow» a conversation in the way a human interlocutor would. For each response, it simply receives a context composed of the previous messages transmitted to it and generates the most probable text continuation. It retains no memory, no lasting understanding of the exchange, and no mental representation of what has been said.
This limitation explains why a model can lose track of a long conversation, forget information mentioned just a few exchanges earlier, or produce an incoherent response when the context becomes incomplete. It is not a lack of attention, but a direct consequence of its functioning. Without context, a language model «knows» nothing of the previous conversation.
This lack of understanding also explains the phenomenon of hallucinations. When information is missing, ambiguous, or out of context, the model doesn't seek the truth: it generates the answer that appears statistically most probable. It can thus invent a reference, attribute a quote to the wrong person, or present an erroneous fact with great confidence. The more fluent and convincing the phrasing, the harder it becomes to distinguish an accurate answer from an invented assertion.
What place should artificial intelligence be given within an organisation?
In this context, should all of one's projects and strategy be entrusted to generative AI? The question isn't so much whether one should use it, but how one does so. The real risk would be to do so without reflection. The real risk isn't using AI, but using it without a framework, without reflection, and without arbitration.
We talk about «artificial intelligence», but the term is already shaping our perception. As soon as the word intelligence is mentioned, our minds spontaneously project human attributes: understanding, intention, discernment. Yet these systems have no consciousness, will or capacity for judgement. They calculate, correlate and predict. Recognising this cognitive mechanism in no way diminishes the usefulness of AI. It simply avoids attributing to it what it has never had: an intention, a responsibility, a form of mind.
The challenge is to maintain a clear-headed stance. Use AI for what it truly is: an extremely powerful probabilistic tool, but incapable of discernment.
Clarity regarding artificial intelligence already means accepting that, despite its name, generative AI is not intelligence. It is a sophisticated probabilistic model, remarkable for what it allows, but devoid of subjectivity. It is only by looking at language models for what they truly are that we can decide what place we want to give them – or not.