.png)
What if the context of a conversation wasn't limited to its history?
By Joanne Breitfelder, Lead Data Scientist at Ask for the moon.
- ⏱️ Reading time: 8 min
- 🏷️ Common Ground, Conversational AI, memory
When we started designing the memory for our conversational assistants, we began with a simple hypothesis: to understand a user's next message, you just need to feed the model the latest exchanges from the conversation.
This approach seems natural. In a conversation, the context seems to be… the conversation itself.
But today's assistants no longer just produce messages. On the interface side, they also display images, sources, file previews, tables, text elements, interactive buttons, and more. Quite naturally, users have started referring to them:
"Can you explain the second diagram to me?"
"What happens if I click the button?"
"In the displayed document, what does the graph show?"
To them, these elements are now part of the conversation, and they naturally assume that the AI "sees" the same things. However, in many classic architectures, the AI is not even aware of their existence.
A first idea would be to simply transmit everything that has been displayed to the model. But this solution quickly raises other questions:
- Which objects has the user actually seen?
- Which ones have truly become points of reference?
- In what form should they be communicated to the AI?
Gradually, we realized that the problem is not just about keeping more information in the conversation history. It is primarily about maintaining, throughout the exchange, a consistent representation of what the user can reasonably consider as shared with the assistant.
The common ground as a framework
We wondered how this issue was addressed in the literature, both in terms of conversational systems and in linguistics and cognitive sciences. One concept quickly emerged as particularly relevant to our thinking: that of common ground.
Introduced by Robert Stalnaker in the late 1970s and later widely developed by Herbert H. Clark, common ground refers to the set of knowledge, beliefs, and information that two interlocutors mutually presume to be shared during a conversation, and upon which they rely to understand one another.
The idea championed by Clark is that dialogue is not just about transmitting information, but also about constantly building and updating this common ground. Thus, as the exchange progresses, certain elements become shared references that no longer need to be reintroduced or explained: everyone knows what the other is talking about.
This accumulation of common knowledge makes language progressively more efficient. As a result, detailed descriptions give way to expressions like "that one", "here" or the previous one, because an increasing amount of information is now conveyed by context rather than by the words themselves. This isn't a communication flaw; on the contrary, this efficiency is part of what makes human conversation so fluid!
For this mechanism to work, we must constantly verify that we share the same references. We therefore continuously check the common ground : we rephrase, ask for clarification, correct a reference, or confirm that we have understood correctly. We do this without thinking: this coordination work is spontaneous for humans.
It is precisely on this point that conversational assistants often still differ from their users. Rather than verifying that the common ground is actually shared, they frequently tend to resolve ambiguities on their own initiative, without asking for confirmation.
What the interface adds to the conversation
At Ask, this issue became apparent as our interface grew more complex.
Initially, our assistant primarily produced a text response. Then we began displaying elements around that response intended to make it more useful: a chart extracted from a document, a source viewable in a side panel, a button to forward the question to colleagues, and so on.
And the interface does not present these elements as independent components. On the contrary, it stages them to naturally create the impression that they belong to the continuity of the conversation.
For their part, the user has no reason to wonder whether an image comes from the model, the frontend, or an external tool. Everything visible spontaneously becomes an element of the exchange with the assistant. Consequently, if the AI is not informed of what has actually been presented to the user, certain implicit references become simply impossible to interpret.
What this changes in our architecture
This perspective has primarily led us to reframe the problem. The goal is no longer just to transmit a conversation history to the model, but to reconstruct, at each turn, a state sufficiently close to the common ground that the user assumes they share with the assistant.
In practical terms, this is now reflected in 3 complementary components of our architecture.
1. An enriched conversational history
The history is no longer limited to the messages exchanged. We add the main objects introduced throughout the interaction: attachments from the user, displayed sources, document previews, etc.
Our question is no longer just: "Does the model need this information?", but rather: "Is this element likely to become a shared reference in the conversation?" If the answer is yes, then it must be representable in the conversational state.
2. Shared context from the start
The common ground is not built solely during the conversation, however. Part of it exists even before the first message.
At Ask, every assistant has a system prompt that describes its role, its environment, and the domain in which it operates. This constitutes the foundational, stable knowledge that can be assumed to be shared from the very beginning of the conversation. For example, an assistant deployed at EDF knows it is operating within the context of nuclear power plants, that there are different "levels," and that equipment is identified by "functional reference systems."
These elements do not come from the history of the exchange, yet they form a common ground between the user and the assistant. They play a role comparable to the cultural or professional knowledge shared in human conversation: they allow for the correct interpretation of many implicit references without the need to explain everything from scratch.
3. A dialogue policy
Finally, representing the common ground is not enough. As we saw above, a conversation also consists of verifying and repairing this common ground when it is insufficient.
A conversational memory must therefore be accompanied by a dialogue policy. The assistant must be able to recognize when a reference is ambiguous, ask for clarification, rephrase, or admit when it lacks information. At Ask, this philosophy has been present in our assistant from the start: rather than inventing an answer, it is designed to be able to say "I don't know" when it does not have the necessary information. We have gradually extended this logic to the management of the common ground : when the assistant feels a reference is not sufficiently shared, it can now ask for clarification or verify its understanding before responding.
An impact on product organization
This evolution had another, more unexpected consequence: an assistant's memory can no longer be designed solely by the AI team.
If the goal is to represent the common ground, then every team that helps shape the user experience also indirectly helps shape the conversational memory.
The frontend determines which objects are actually displayed, in what order, and in what states. Design influences their visibility and salience—that is, what the user will spontaneously remember as central to the exchange. Product defines the possible actions and how they integrate into the conversation. The AI teams must then decide how to represent these events, which ones to keep, and how to reintroduce them in subsequent turns.
Thus, every new interface feature can have a conversational consequence. Adding an image gallery, a source panel, a document comparison tool, or a filter system does not just change the visual experience: it potentially creates new elements of the common ground, which the assistant must be able to represent and mobilize at the right moment.
Designing memory is therefore no longer just a matter of AI architecture. It is a topic that spans the entire product.
Conclusion
Ultimately, this reflection goes far beyond the question of AI assistant memory. It questions what it really means to "converse" with a computer system. As long as we view context as a simple succession of messages, an essential part of the exchange will remain invisible to the assistant.
Herbert Clark's common ground has provided us with a valuable framework for rethinking our architecture. This concept resonates with much contemporary work: grounding in dialogue systems, agent memory architectures, and cognitive science research on shared intentionality (see, for example, Tomasello in the “Further Reading” section). Each explores, in its own way, the same intuition: dialogue is not just about exchanging information, but about building and maintaining a shared representation of a situation.
Behind what may seem like purely technical problems lie questions that cognitive science has been exploring for decades. AI forces us to articulate and formalize mechanisms that humans apply without even realizing it. This is where software architecture meets the age-old quest to understand how the human mind works.
Learn more
Foundational texts
- Robert Stalnaker (1978) – Assertion
The seminal text that introduces the concept of common ground.
https://brill.com/display/book/edcoll/9789004368873/BP000013.xml - Herbert H. Clark (1996) – Using Language
The definitive work on common ground.
https://www.cambridge.org/core/books/using-language/4E7EBC4EC742C26436F6CF187C43F239
Recent research
- The common ground : a special issue of Cahiers de Praxématique
An excellent introduction in French.
https://journals.openedition.org/praxematique/9884 - Anikina, Leippert & Ostermann (2025) – Building Common Ground in Dialogue: A Survey
A state-of-the-art review connecting common ground and dialogue systems.
https://aclanthology.org/2025.luhme-1.2/ - Yalcin (2024) – Defining Common Ground
A contemporary philosophical discussion of the concept.
https://link.springer.com/article/10.1007/s10988-024-09415-7 - Michael Tomasello (2022) – The Origins of Human Cognition
On the origins of social cognition and shared intentionality.
https://shs.cairn.info/aux-origines-de-la-cognition-humaine--9782348075834?lang=fr - Park et al. (2023) – Generative Agents
On memory-based agent architectures.
https://doi.org/10.1145/3544548.3581318 - Anthropic (2024) – Building Effective Agents
A practical guide to designing AI agents.
https://www.anthropic.com/engineering/building-effective-agents
.png)
