Simu-Verse

A chatbot forgets you. Every reply it gives is built from whatever is on the screen right now, and when the conversation ends, that is gone. I wanted to build a small world of AI characters who could remember each other: who they talked to, what was said, where they left something. Simu-Verse was meant to be that world, in 3D. I led a team of four. My teammates worked on the environment and the 3D side, and on some of the code, and I took the memory.

My part was the memory. The idea is that each character gets a second AI that lives behind it. Think of the thing in your head that makes a name pop up when you see a face. You don't go looking through your past, the right thing just arrives. I called this second AI the subconscious. After every turn it writes a short note about what just happened: a summary, a few keywords, and a score from 1 to 10 for how much it matters. Before the next turn it reads the recent conversation, decides what is worth looking up, pulls a handful of old notes and hands them to the character as "flashbacks". The character never touches the shelf of notes itself.

What I built was the machinery around that: the subconscious, the notes, the search, and a 2D stand-in for the world. Four characters named James, Jade, Jesse and Jamal stand on a grid, each paired with whoever is closest to them. They can talk, walk to landmarks, look around, open a door and sit on a chair, at least in the code, and a side panel shows which memories were pulled up for each of them. The 3D world was my teammates' side, and some of it may be in their own repositories. In what I built, a flat grid takes its place.

What I never got to see is the memory working properly. When I ran it, the database rejected my login and the code quietly fell back to a fake, in-memory shelf. The way notes get ranked is also a bit silly: after searching, it sorts them by the score the AI gave them rather than by how well they match. The latest version I made doesn't start at all, because a rewrite left broken code behind. And there are no tests. So I would call it a prototype of an idea, not something that works.

What's next. The best part of the design is still empty: a third piece that edits memories, so that old ones can be merged or rewritten, the way yours do. A teammate started a part that turns memories into opinions, and that is a stub too. Before any of that, the shelf needs to work against a real database and be tested.

Simu-Verse was meant to be a world where LLM agents move and talk in 3D. I led a team of four. My teammates worked on the environment and the 3D side, and on some of the code, and I took the memory. What I wrote is public as Simu-Verse_Memory: the agent loop, the memory system and a 2D grid that stands in for the world, because the 3D work lives elsewhere. Some of it may sit in a teammate's own repositories. Unity was the plan for the 3D part. It ran from February to April 2025.

What I wanted

A language model forgets everything between calls. If agents are going to hold relationships or remember where things are, something has to decide what is worth keeping and what to bring back. I didn't want the agent doing that itself, with a database tool it has to remember to call. So each agent gets a paired "subconscious". It writes the memories, and before each turn it picks a few and hands them over as flashbacks. The agent never queries the database, and it never knows the database exists.

How it works

Every turn makes three kinds of LLM call. The first asks the subconscious for up to three search queries, written as XML. The second is the agent's own reply, also XML: a short reflection and one tool call. The third summarises the exchange into a memory: a summary, one of eight categories, keywords, "critical information" and an importance from 1 to 10. Step through the turn below to see each step and the format it uses.

<memory_queries>
  <query>
    <search_type>hybrid</search_type>
    <keywords>meeting, deadline</keywords>
    <query_text>the project deadline</query_text>
  </query>
</memory_queries>

The LLMManager talks to Ollama by default (the local model is qwen2.5:14b), or to Anthropic or OpenAI. The agent never gets a database tool. That was the point, and it is also why a bad retrieval silently hurts it: it can't ask again.

Each memory is stored in a collection per agent, in Weaviate, a vector database. Only the summary is embedded. The rest, category, keywords, critical information, importance, timestamp, location, agent and id, are stored as plain fields.

The grid demo

Since the 3D world isn't part of what I published, I made a 2D one with Dash and Cytoscape to run the agents in. Four agents (James, Jade, Jesse and Jamal) stand on a grid running from 50 to 550 in each direction, with three landmarks and four items to walk to or use. Each agent is linked to its nearest neighbour, with no range limit, and the links redraw as they move. Tap or click anything in the picture, or pick it from the list under it, to see what it is.

Each agent has a personality-strength slider. It sets how loosely the model answers, through the temperature of the reply. A strong personality gets a low temperature and sticks to its profile; a weak one gets a high temperature and wanders.

A panel shows the memories retrieved for each agent on its last turn. The profiles still tell agents to write [MOVE] when they want to leave a conversation, a leftover from an earlier version of the demo. In the grid demo, agents move through the movement tool instead.

Numbers

I never measured this system, so there is no accuracy figure to give. These are the fixed settings it runs with:

What didn't work

Retrieval was never shown working. When I ran it on 11 March 2025, Weaviate answered with an HTTP 401 and the code fell back to the in-memory mock. My Docker Compose file starts only a Weaviate service, with no embedding service behind its transformers module. I never saw it run against the real database.

In the grid demo the memories never reach the model. James, Jade, Jesse and Jamal take their whole prompt from a short profile, and those profiles have no slot for memories, no tool list and no XML format. They are told to answer in two sentences. The retrieved memories show up in the side panel but are not handed to the characters.

Ranking throws away relevance. After three searches, results are sorted by the importance number the model gave itself when it wrote the memory. A closer match loses to a higher-rated one.

The newest version doesn't start. One file, subconscious_agent.py, has duplicated docstrings and method stubs left over from an edit, and Python can't parse it. Another, agent_manager.py, has an await inside a plain function and never imports asyncio, which Python rejects at import. The earlier oli version, from 11 April, does not have that line.

The versions disagree with themselves. In the newest one, SubconsciousAgent renames its collection with capitalize(), so Memories_James becomes Memoriesjames, while AgentManager still creates Memories_James. Memories are also deduplicated on an id that results from a real Weaviate don't carry, and anything without one is dropped. So even if it parsed, a real database would hand back nothing. The mock does carry the id, which is why it looked fine with the mock.

Nothing is tested. There is no test suite. A memory is written every turn with no merging, and the merge_memories helper is never called.

What is unfinished