Jarvius

I built Jarvius because I wanted something like Jarvis in the Iron Man films, an assistant I could just talk to, and I wanted it to be useful on an ordinary day. Most of the time that meant the walk to class. I would send it voice messages while walking, and tell it to answer in text or in voice depending on whether I could listen or had to read. That worked, and it is the part I enjoyed most.

It started in late February 2025 as a Discord bot, because Discord was the quickest place to try it. It already handles accounts, text chats and voice calls, so I could skip building an interface and go straight to the interesting part. In the code the bot calls itself Jarbius. Jarvius is what I call the project now.

You type to it, or reply to one of its messages, and it answers in a sentence or two. The answers come from a language model, the kind of program behind chatbots, running on my own laptop instead of in someone else's cloud. That made it free to run, but it was only available when my laptop was on. It can also join a voice call and read its replies out loud, like a receptionist sitting in the call who answers what you typed, and in the first versions it could listen as well, writing down what was said and answering that. In the last version I wrote, it only speaks. It remembered what each person had said earlier, so a follow-up question made sense.

Around that I added two joke commands that print random words for farts and poops, and a party game called Death By AI. Everyone gets a scenario, types what they would do, and the model writes a short, dark little story about it and decides whether that person survived. Five rounds, and whoever survived the most wins.

It was never a polished thing. Each time it needed to listen or speak, it set up the speech software again from the start, which made it slow. The memory disappeared whenever the bot restarted. When I created the project I described it as a team of AI agents. In the Discord version that was never true. It was one model answering one message at a time.

Jarvius is the direct ancestor of VOS Gen 1, which is the same project carried on from February to September 2025. For that version I moved off my laptop to Google's Gemini AI service, with several AI helpers passing work to each other and a database to remember things between runs. That cost me far more money than it should have. I did learn a lot from it, about working with APIs and cloud models and about keeping track of what you are spending. I even gave the agents a tool to check how much they had spent. After it came VOS, which I built with Roman Slack.

What's next. I've stopped working on the Discord bot. If I picked it up again I would load the speech models once at start-up so replies come back faster.

Jarvius began as a Discord bot in Python, using discord.py. It ran a handful of commands, sent everything else to a language model, could speak its replies in a voice call, remembered what each person had said, and ran a party game. Jarvius is the direct ancestor of VOS Gen 1: the same project, with the Discord bot as its first stage, from February to September 2025.

What I wanted

I wanted Jarvis from the Iron Man films: an assistant you talk to out loud, that answers back and remembers you. The way I actually used it was on the walk to class. I would send it voice messages and ask it to reply in text or in voice, depending on whether I could listen or had to read. That worked. Discord was the shortcut. It already gives you accounts, text channels and voice calls, and a bot can sit in a call, so there was no interface to build before testing the idea. Every answer goes through Ollama, which runs open models on a machine you control. For me that was my own laptop, so the bot was not up around the clock, but it cost nothing to run. The first model was llama3.2:1b. By the end the default was dolphin3.

What is Ollama?

A program that downloads open language models and runs them on a machine you control, behind a small web address. The bot sends it a list of messages and gets text back, which is the same job a paid API does, without the bill. The cost is that you need a computer that can run the model.

When I created the project I described it as using a team of AI agents. The Discord version never had one. It is one model behind one short instruction. The agents belong to the Gemini version that came after.

How it works

There are three ways to get its attention: start a message with "hey jarbius", reply to one of its messages, or use a command prefix. If the first word is a known command it runs that directly: ping, help, memory, join, leave, generate, and the two joke ones, fart and poop, which print up to 100 random synonyms. Anything else goes to the model. The bot saves your message, builds a prompt from a short system instruction plus your recent history, sends it to Ollama and posts the answer. In the last version these normal chat replies run on a joke persona I swapped in, not on the Jarbius instruction. The Jarbius instruction is still what generate uses. Both ask for two sentences at most.

Speaking back uses Parler-TTS mini v1. It takes the reply text plus a plain sentence describing the voice, and mine asks for a clear speaker on a close, clean recording. The result is played into the call through ffmpeg. If the bot is already talking, a new reply cuts the old one off. The listening side records the whole call, measures volume, and hands finished clips to Whisper.

What is Whisper?

An open speech-recognition model from OpenAI. You give it an audio file and it returns the text. The medium.en version is a mid-sized model trained on English only. It is more accurate than the small ones and slower, which matters when you run it on every clip.

What does the volume check actually measure?

It is the root mean square, or RMS, of the audio samples: square each sample, average them, take the square root. That turns a burst of sound into one loudness number. A quiet room gives a small one and speech a large one. The audio here is 16-bit, so single samples can reach about 32,000, and I set the line between quiet and speech at 300.

The wake phrase check is a plain text match on "hey jarbius" in the transcript, so a transcript that spells the name any other way is ignored. The last version of the code no longer starts the in-call listener: the bot joins your call, says so, and tells you to use generate to make it speak.

Memory

Memory is a small class that keeps the last 200 messages for each person on each server, and a second log of the last 200 for the server as a whole. Each entry is a role, the text and a time. When the model is called, the prompt is a short system instruction plus that one person's history. That is why a follow-up like "and what about the second one?" works, and also why the bot has no idea what anyone else in the channel just said. The server-wide log is only counted by the memory command, which can show counts or clear one person or everything.

All of it sits in RAM, so a restart wipes it. The later Gemini version replaced it with Weaviate, which is a database that can keep memories between runs.

What is Weaviate?

A vector database. It stores each piece of text as a long list of numbers that captures roughly what it means, so a search can match on meaning and not just exact words. It also keeps everything on disk, so what it has stored survives a restart.

Death By AI

The game is a separate program in the same project, with its own commands and no shared code with the assistant. It uses the same model on the same Ollama machine. One person creates a game in a channel, others join, and the creator starts it. It works with a single player too. There is one game per channel, and it closes itself after ten minutes of silence.

A random player is asked for a scenario and gets 30 seconds. If they don't write one, the bot picks one of twenty built-in scenarios. Everyone then has 60 seconds to say what they do, and anyone who doesn't answer dies. Each answer goes to dolphin3 together with the scenario. The instruction asks for a small block of XML: a short reflection on the scenario, the player's approach and the story, then a story of at most three sentences, then a verdict of SURVIVED or DIED with a one-line reason. The narrator is told to be a dark, crude comedian. The story is revealed by editing one message word by word, and a scoreboard follows each round.

Game settings the creator can change

Setting	Default	Allowed
Rounds	5	a whole number
Time to write a scenario	30 s	10 to 120 s
Time to answer	60 s	10 to 300 s
Story reveal	word	character, word or instant
Delay between reveals	0.2 s	0.05 to 1 s

A small model doesn't always answer in the format it was asked for, so the parser is forgiving. It adds the missing wrapper tag, escapes stray ampersands, and if it still can't read the answer, the player survives. A model error counts as survival as well. So a broken answer never kills anyone.

What didn't work

Speech worked, but slowly. The speech models are loaded from scratch for every reply and for every clip, rather than once when the bot starts. That is the obvious first thing to fix. I also rewrote the call-listening code a few times as I changed how the bot receives audio.

Memory lives in RAM, so a restart forgets everyone, and it only ever follows one person's thread, never a group conversation. Everything depends on the Ollama machine being on, and that was my laptop, so when it was off the bot did not answer. The wake phrase is an exact text match, as above. Moving to Gemini later fixed the laptop problem and created a cost problem.

What's unfinished

I've stopped working on the Discord bot. The work went on as VOS Gen 1 (see the timeline above) and then VOS, built with Roman Slack. If I picked the bot up again I would load the speech models once at start-up so replies come back faster.