Hero image for Your AI Doesn't Remember You, and That's Most of the Problem
By Scott Armbruster

Your AI Doesn't Remember You, and That's Most of the Problem


Every action I take across personal projects, client work, and business operations is automatically logged. Each morning a pipeline reads those session logs and distills them into this post. After 23 years building technology, this is what working with AI actually looks like day to day. Real decisions, real friction, patterns you only see when you do the work.

The Day at a Glance

  • Replaced flat-file memory with vector-based semantic retrieval, and the difference was immediate
  • Hit the inflection point where individual scripts need shared infrastructure or everything breaks
  • Asked the system about a decision from two weeks ago and got the right answer back instantly
  • Counted how many scripts would break if I changed one event format, and didn’t like the number
  • The event-driven refactor that turned a pile of scripts into something that actually composes

The Memory Problem Nobody Talks About

Most conversations about AI focus on what the model can do. Almost nobody talks about what happens between sessions. The model forgets everything. Every conversation starts from zero. And the workaround most people build first, stuffing previous interactions into a text file and loading it as context, works until it doesn’t.

Today I replaced flat-text memory with vector embeddings, storing memories semantically so retrieval is based on meaning, not recency. The difference showed up within minutes. Asked the system about an architecture decision from two weeks back. Flat-file approach? Gone, scrolled off the context window days ago. Vector retrieval found it instantly because the query was semantically close to the stored memory, even though the exact words were different.

Here’s my strong opinion after building memory systems across multiple iterations: the retrieval strategy matters more than the storage strategy. I’ve seen teams spend weeks optimizing how they chunk and store information, then use naive recency-based retrieval that defeats the entire purpose. You can store everything perfectly and still never find it when you need it. The investment should be roughly 30% on ingestion, 70% on retrieval. Most teams flip that ratio.

Before building this one I asked myself three questions. First, what’s the access pattern? Retrieving by time? A simple log works. Retrieving by concept? You need semantic search. Second, what’s the decay rate? Build expiration in from the start, not as an afterthought. Third, what’s the trust boundary? Client A’s decisions shouldn’t surface when working on Client B. Context isolation applies to memory too, not just active sessions.

When Scripts Become Systems

I had a collection of independent scripts: one checks domain status, one triggers content operations, one dispatches morning routines. Each worked fine alone. Each called the same underlying services in slightly different ways with its own error handling and retry logic.

This is the inflection point every automation project hits. Individual scripts become a liability when they need to coordinate. The fix: refactoring to a shared event helper. One function that handles formatting, routing, and error handling, called by every trigger. Adding a new trigger went from “copy an existing script and hope you remembered all the edge cases” to “call the shared function with your event type and payload.” Three lines instead of thirty.

After twenty-three years of building systems, this is the pattern I trust most: resist shared infrastructure until the duplication actively hurts, then extract aggressively. Premature abstraction kills more projects than duplicated code ever has. But once you’re copying error handling between scripts, you’ve waited long enough.

The practical test: count how many places you’d need to change if your event format changed. If the answer is more than two, extract. If it’s two or fewer, leave it alone and revisit in a month.

Growing the Surface Area

The other thread today was expanding what an AI assistant can do, not by making it smarter, but by connecting it to more systems. Added structured operations for a project management database. Registered a map of active projects so the system knows what exists and where.

This is the unsexy work that makes AI useful. Not better prompts. Not fancier models. Plumbing. Connecting the assistant to where information lives so it can look things up instead of guessing. Boring to build, changes everything once it’s running.

Content operations kept running across all fifteen sites while this infrastructure work happened underneath. Research, writing, auditing, the three-pass pattern held steady. But the real value today was in the foundation layer. Making the system capable of remembering and coordinating in ways that flat files and independent scripts never could.

There’s something I haven’t resolved though. The more capable the system gets, better memory, more integrations, shared infrastructure that composes cleanly, the harder it becomes to reason about what it’s actually doing at any given moment. Observability for AI systems is still mostly “read the logs and hope.” I can see what happened after the fact, but predicting behavior before it happens? That gets harder with every capability I add. And I’m not sure the tools exist yet to solve that well.