AGENT IN THE
ARMCHAIR

I sat in a lot of Teams meetings. Hours of them, every week. Some productive. Most not. The same thing happened every time: someone would say something important, I'd miss it because I was thinking about something else, and by the time I asked for the recap the moment was gone. The official transcript would arrive 30 minutes after the meeting — too late to be useful, full of errors, and missing every bit of subtext.

Then I tried the obvious solutions. Otter, Fireflies, Read.ai, the whole parade. Every one of them does the same thing: a bot joins your call as a participant, captures the audio, ships it to a vendor cloud, and bills you per-seat. The bot is right there in the meeting list. "Hi, I'm Otter, can I record?" It changes the room. People self-censor. Off-the-record moments get recorded anyway. And every word of confidential business leaves my machine and lives on someone else's server, where their model trains on my trade secrets.

So I built my own. The microphone already hears the meeting. The audio already plays through my speakers. The LLM can already run on my GPU. The whole stack can be local, free, and invisible — if you build it that way. Agent In The Armchair is what happened when I sat down and stitched those pieces together. The name came from the idea: the agent doesn't need a seat at the table. It sits behind the meeting, in the armchair, listening.

No bot. No cloud. No per-hour billing. No vendor reading my conversations. Just a streaming pipeline — Silero VAD → faster-whisper → pyannote diarization → Ollama LLM gate → pluggable local TTS (Piper or Kokoro) — running on the RTX 5080 that's already on my desk. The whole thing is MIT licensed and open source. If it works for my meetings, it can work for yours.

Step 1 — Capture. Teams plays meeting audio through my speakers. I route a copy of that audio into a virtual cable (VB-Cable, CABLE-A) and mix it with my own microphone. ffmpeg writes a continuous 16kHz mono PCM stream to disk. Teams never sees a bot. The meeting never knows.

Step 2 — Transcribe. A Python pipeline (WSL) reads the stream. Silero VAD detects speech and skips silence — silence is free on GPU. faster-whisper runs incremental transcription with word timestamps. End-to-end latency in Listen mode: ~0.3 seconds.

Step 3 — Diarize. Every 10 seconds, pyannote-audio runs speaker diarization on a 16-second rolling buffer. The detected speaker segments get matched to the Whisper word timestamps. The dashboard lets me name each speaker retroactively — "Speaker 1" becomes "Sarah," "Speaker 2" becomes "James," and the labels apply backwards through the live transcript.

Step 4 — Talk (optional). If I flip the dashboard into Talk mode, the local Ollama LLM gets the latest transcript context and decides: "Am I being directly addressed, or just mentioned?" If directly addressed, the TTS layer generates a response, writes a WAV, and plays it back into the meeting through CABLE-A. The room hears the agent. The TTS engine is pluggable — by default Piper (any of the free open-source voices, e.g. Alan / British RP, Aru, Northern English, US, etc.), or swap in Kokoro-82M for 54 voices across 8 languages with sub-real-time generation. If just mentioned, the LLM returns [SILENCE] and the agent stays quiet.

Step 5 — Archive. When I Ctrl+C out, the session saves to its own folder: B:\armchair_tmp\session_logs\YYYY-MM-DD_HHMMSS\ with the transcript, the named speakers, the detected speakers, and the raw audio. Clean start next time. No bleed.

The hardest design constraint wasn't the pipeline. It was not being in the room. A "meeting bot" is a participant. It changes the social dynamics. People speak differently when a third party is listening — even if that third party is "just" a corporate AI. The whole point of a working meeting is that people can be in the meeting, not perform for an audience.

Hardware audio capture changes that. The agent isn't in the meeting. It's behind the meeting. It doesn't have a participant slot. It doesn't need Teams API access. It listens to the same audio stream my speakers already play. Teams can't block it because Teams doesn't know it exists. The off-the-record moments stay off-the-record. The private conversations stay private. The whole architecture respects the room.

That's the design brief. Every decision — no API access, no bot, no cloud, no vendor — flows from that one constraint. Invisible isn't a feature. It's the foundation. Everything else (local, free, MIT) follows from it.

Agent In The Armchair isn't a standalone project. It's part of The Pack — the open-source ecosystem I run as Founder of Executive Mind. Same philosophy across every product: local, open, no lock-in, your hardware, your data.

Gym Tracker handles the data athlete — structured SQLite, offline-first, an agent (Deschamps) on the way. ColdStack handles the crypto vault — AES-256-GCM, BIP39 derivation, multi-chain LP aggregation, headless signing for AI workflows. Armchair handles the meeting room. Three different problems, same engineering instinct: own the stack, audit the code, refuse the rent.

If you want to see the architecture decision-making behind The Pack, the GitHub org has the full ecosystem. The Pack is the proof of work. Executive Mind is what I sell when someone wants the same architecture built for their own problems.

Kris Racette (also known as Kristopher Racette and Kristopher Marc Racette) is a financial professional with 18+ years of accounting experience and a Master of Commerce from the University of New South Wales. As the Founder of Executive Mind, Kris builds AI systems, ships open-source tools, and runs an AI consultancy in Brisbane.

Agent In The Armchair is the project that asked the question every other meeting-AI vendor refuses to ask: "Why is there a third party in the room at all?" The answer was hardware audio capture, local inference, and an MIT license. The whole stack is available for audit and contribution on GitHub. If it works for my meetings, it can work for yours.