How I Gave My Jarvis a Memory

Jarvis is the AI that runs Tony Stark's suit in Iron Man. It sees the whole system, forgets nothing, and hands him what he needs before he asks for it. Mine remembered plenty. Just never the part I needed.

Iron Man was two parts. The human in the suit, and the AI that ran everything. The nerve and the intelligence were his, but the system had no holes in it, and it had memory.

Now put holes in that memory. Stark goes up against Thanos, and the one thing the suit does not have is how Thanos fights. Everything else is there. Not every time. Enough times that he loses. Nothing was wrong with the hardware.

That was mine. Not amnesia, which would have been obvious and easy to fix. It remembered, and what it was missing was the part I needed to move. Some weeks I re-explained the same context three or four times before I noticed it was a pattern rather than bad luck.

Trust is what you actually lose, and trust is the whole thing when you work with an AI.

Once you have found two holes, you check every time. And checking is the work you were trying to hand over.

I have been deep in the AI Champions programme at Yodeck for weeks. The tools are good and people are building real things with them. Capability stopped being the problem a while ago.

My own setup was the problem. Two loopholes, both of them mine, produced by how I work rather than by the technology. What I am trying to build is a real Jarvis for my day to day, and a Jarvis with holes in it is not one.

Loophole 1: my memory only updated when I remembered to update it

I checked last week. Nine days stale. In those nine days I had convened a cross-team call, written a proposal, changed my main focus for the quarter and opened a design review. None of it was written anywhere a machine could read.

Loophole 2: I had two memories and they did not agree

One inside Claude, one in my own files. Each held things the other did not, and nothing reconciled them. Whichever one I opened, I was reading half an answer without knowing which half.

Everyone wants Jarvis. Nobody wants the filing.

Where the idea came from

Not from work. I journal. Writing things down forces me to look at my thinking instead of carrying it around, and it changed how I decide. Metacognition, if you want the word for it.

An AI without that layer is a brilliant colleague with no memory. Ten minutes of re-explaining, then an answer shaped by whatever I happened to mention. That preparation time is what I set out to delete. Not the work. The re-explaining.

What I built

Two fixed moments in the day, and one rule about who is allowed to write what.

The daily loop. The journal writes itself. Everything else waits at the gate.

The loop closes the first loophole. At 17:00 something reads what actually happened, in Jira, Slack and my meetings, and writes it down whether or not I show up. Recording is no longer a habit I have to remember. At 09:00 the next morning it hands me back what needs me that day, and offers slots to do it in.

Which creates the next problem. Memory that writes itself grows, a file that grows stops being read, and keeping it short means rewriting it, and every rewrite deletes what was there.

The strip along the bottom of the diagram closes that one. It is RAM and hard disk. A short file that holds what is true right now, rewritten freely because nothing in it is precious. And long files that are only ever added to, never rewritten, where the journal and the reasons behind decisions pile up. The short one points at the long ones, so it stays small enough to read and complete at the same time.

Nobody asks why a computer does not keep everything in RAM.

That closed the second loophole too. Not one memory instead of two. One memory that says what is true, with everything it rests on kept underneath.

The three habits, none of them technical

• Say the memory out loud. When something is decided, say what and say why. The reasoning, not the outcome. A decision without its reason is useless in six weeks, because you cannot tell whether the conditions that produced it still hold.
  

• Version what changes. Mine has a changelog and a version number, like software. It sounds like overhead until the first time you ask why a rule exists and the answer is a date and one sentence.
  

• Connect the sources. Jira, Slack, calendar, the meeting recorder, the docs. Not so it can act everywhere. So it can see. Most of what happens to my work happens outside my conversations with it.

None of that needs a developer. All of it needs you to keep doing it on a Thursday when you are behind.

Guardrails, written before they were needed

Anything that writes to my files gets told what it may never do, in advance.

• It never changes the rules without telling me first, naming the file and what changes downstream. Even when I asked for the thing that requires it, because I do not always notice that my request touches the constitution.
  

• It never sends anything. Not Jira, not Slack, not email.
  

• It never deletes. Archiving moves text between files.
  

• Every line it adds carries the date it came from, so I can trace any claim back to the day and the source.

I checked it against the engineering writing

The overlap is closer than I expected. Effective context engineering for AI agents sets the goal as the smallest possible set of high-signal tokens, with detail loaded only when a task needs it and the rest kept in notes outside the window. That is my index file, my routing map and my journal, in that order.

And Agent Skills are folders of markdown loaded only when relevant. That is where the behaviour of my two daily jobs lives, so I change what they do by editing a file.

Two things I added that the engineering writing does not cover, because they already solve them in the system that they work. Who is allowed to change the rules. And how I see what changed, months later, without trusting my memory of it.

You do not need to be technical

I am not a developer. I am a product manager with a degree in banking and finance. What carried this was structured thinking and being organised, which is not the same as knowing how to code.

Saying that plainly matters, because the assumption stops people before they start. The gap between having a system in your head and having one that runs has closed, and it closed recently. There is enormous capability sitting unused, and unused capability is worth the same as none.

The snap that used to take six Infinity Stones now takes a subscription.

That is the part most people have not caught up with yet.

What I do not know yet

Whether the daily approval survives a busy month. That habit is the single point of failure. The design protects me from the obvious version, since the journal writes itself whether or not I answer, so a month of ignored digests costs me a catch-up pass rather than the month. But I have one week of data and one user, and the user is me.

Ask me in October.

If you want to try it

Start smaller than this. One file that says what is true right now. One file that never gets rewritten, where things pile up. A fixed time each day when something updates the first from the second, and asks you before it does.

That is the whole idea. Everything else in my setup is refinement I added after hitting a wall.

The nerve and the intelligence are yours. The system is the other part, and it decides whether any of it lands.

Tell me what broke. That is the part I want to read.