Your AI agent probably does not have a brain. What I mean by that is it does not have a system that allows it to store and retrieve knowledge that you have accumulated over months and years. Agents and models have lived in the present, but what we want is experience; a history that grows with you as your competence expands.
This system was inspired by Nate B. Jones's video on Open Brain. His implementation is probably better--I just didn't want to pay for the Substack subscription just to see the github. You can find my implementation on GitHub here.
What is this Shit?
It is vector database RAG for your coding agents. We do not need to make it sound any more fancy or mindblowing. People were already using vector db for retrieval before the agentic breakthrough. That is all it is. Is it novel? No, not really. Is it awesome? Yes.
Why Self-Sovereign?
There are several compelling reasons to build your own memory system rather than relying on the defaults that now come with AI coding agents like Claude Code:
Privacy: Your conversations, decisions, and knowledge belong in your own database, not in someone else's cloud. When you use off-the-shelf memory features, you are implicitly trusting a vendor with potentially sensitive information: PII, skills, trade secrets, etc.
Vendor Lock-in: Memory features tied to specific platforms can and do change. An API here, a pricing shift there. Building on open standards like MCP means your memory works regardless of which AI tool or provider you use today or tomorrow.
Capacity: The free tiers that come with AI agents have limits. Want to store thousands of embeddings? Build it yourself and scale as needed.
Portability: Your data should be yours. A self-hosted solution means you can export, migrate, or backup without friction.
Yes, there is maintenance and overhead. But for people who care about control and longevity, that trade-off makes sense.
The Memory Problem
Core Concept: Build a shared memory that works across different AI tools. Claude Code, OpenCode, Hermes Agent, or whatever comes next - they can all read from and write to the same database via MCP. No export/import between platforms. No locked-in memories. Just open source tools talking to each other.
Architecture
Currently running on localhost. Here is the architecture:
- PostgreSQL + pgvector - Vector embeddings stored directly in the database
- llama.cpp embedding server - Running nomic-embed-text on port 8080
- MCP server - Node.js server implementing the Model Context Protocol
- CLI tool - Bash wrapper for capture, search, list, and import commands
How It Works
1. Capture Thoughts
Any time you want to remember something, just capture it:
echo "Sarah mentioned considering a career change to consulting" | brain capture
brain capture --type decision "Migrated from MongoDB to PostgreSQL for vector search"2. Semantic Search
Search by meaning, not keywords:
brain search "career changes" --limit 5
brain search "PostgreSQL" --type decision3. Agent Integration
Configure Claude Code, Cursor, or OpenCode to connect via MCP:
{
"mcpServers": {
"openbrain": {
"command": "node",
"args": ["/path/to/open-brain/mcp-server/bin/server.js"]
}
}
}Available MCP Tools
| Tool | Description |
|---|---|
search | Search the knowledge base for semantically similar content |
capture | Capture a new thought or note to the knowledge base |
list-sessions | List all imported sessions |
get-stats | Get statistics about the knowledge base |
import-session | Import a session transcript into the knowledge base |
Why This Matters
Every time you start a fresh session, you are back at square one. You end up re-explaining the same context over and over. Switch from Claude to OpenCode? Context is gone. Close the tab? See you next time. It is Groundhog Day, but without the fun of knowing what is coming.
The lessons that are not immediately baked into your workflow get forgotten. That clever solution you found last week? That constraint you figured out after hours of debugging? Gone. Unless you wrote it down somewhere.
That is what this system solves. It is not magic. It is just a searchable place to put stuff so you do not have to repeat yourself.
Memory Architecture Determines Agent Capabilities
Memory architecture determines agent capabilities much more than model selection does. That is widely misunderstood. And when you construct memory incorrectly, you are stuck reexplaining yourself forever or you are stuck in a world where you know how to access memory and the agent does not.
If you are going to do context engineering, if you are going to do specification engineering, seriously, you need to invest in a memory system that is yours, that is agent readable, that makes calling and retrieving that context, that makes specifying easier.
What is Next
This system is now integrated with my LLM Garage setup. The GitHub repo is available at https://github.com/ashpoolin/open-brain with full setup instructions.