# 🧠 Clawdbot Memory System

**Never lose context to compaction again.**

Your Clawdbot agent forgets everything after a long session because compaction summarizes and discards old messages. This system fixes that with persistent, searchable memory that survives crashes, restarts, and compaction.

> *"My buddies hate when their agent just gets amnesia."* — Jake

## One-Command Install

```bash
bash <(curl -sL https://raw.githubusercontent.com/BusyBee3333/clawdbot-memory-system/main/install.sh)
```

The installer walks you through an interactive setup (~2 minutes) with these choices:

| Step | What It Asks | Jake's Choice |
|------|-------------|---------------|
| Core Memory System | Install persistent memory + search? | ⭐ Yes |
| Embedding Provider | OpenAI / Gemini / Local? | ⭐ OpenAI |
| Organization System | Add project scaffolding templates? | ⭐ Yes |
| Auto-Scaffold | Auto-create files for new projects? | ⭐ Yes |
| Git Backup | Add backup instructions for agent? | ⭐ Yes |

**Flags:**
- `--dry-run` — Preview all changes without applying
- `--uninstall` — Cleanly remove config (preserves your memory files)

## What Gets Installed

### Core Memory System (always installed)

| Component | What It Does |
|-----------|-------------|
| `memory/` directory | Where all memory files live |
| Daily log system | Agent writes `memory/YYYY-MM-DD.md` each session |
| SQLite vector index | Semantic search over all memory files (<100ms) |
| Hybrid search | 70% vector similarity + 30% keyword matching |
| Pre-compaction flush | Auto-saves context BEFORE session compacts |
| AGENTS.md patch | Tells your agent when and how to use memory |
| Config patch | Adds memorySearch to clawdbot.json (non-destructive) |

### Organization System (optional)

| Component | What It Does |
|-----------|-------------|
| Project Quickstart | Auto-create structured files for new projects |
| Research Intel System | Weekly intel rotation with auto-compression |
| Project Tracking | Standardized stages, blockers, decisions |
| Contacts Tracker | People, roles, access levels per project |
| Tag Convention | Consistent naming so search groups by project |

## How It Works

```
┌─────────────────────────────────────────────────────────┐
│                YOUR AGENT CHAT SESSION                  │
│  Decisions, preferences, facts, project progress...     │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼ (agent writes throughout session)
┌─────────────────────────────────────────────────────────┐
│              MARKDOWN FILES (Source of Truth)            │
│  memory/2026-02-10.md          (daily log)              │
│  memory/acme-progress.md       (project tracking)       │
│  memory/acme-research-intel.md (weekly intel)            │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼ (file watcher, 1.5s debounce)
┌─────────────────────────────────────────────────────────┐
│              INDEXING PIPELINE (automatic)               │
│  1. Detect file changes                                  │
│  2. Chunk text (~400 tokens, 80 overlap)                 │
│  3. Generate embeddings (OpenAI/Gemini/local)            │
│  4. Store in SQLite with vector + full-text indexes      │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────┐
│          SQLite DATABASE (Search Engine)                 │
│  ~/.clawdbot/memory/main.sqlite                         │
│  • Vector similarity (semantic meaning)                  │
│  • BM25 full-text (exact keywords, IDs, code)           │
│  • Hybrid: 70% vector + 30% keyword                     │
│  • Search speed: <100ms                                  │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼ (next session starts)
┌─────────────────────────────────────────────────────────┐
│              AGENT RECALLS CONTEXT                       │
│  memory_search("what did we decide about X?")           │
│  → Returns relevant snippets + file paths + line nums    │
│  → Agent answers accurately from persistent memory       │
└─────────────────────────────────────────────────────────┘
```

## Why Your Agent Gets Amnesia (and how this fixes it)

**The problem:** When a chat session gets too long, Clawdbot *compacts* it — summarizing old messages and discarding the originals. Important decisions, preferences, and context get lost in the summary. Your agent "forgets."

**The fix:** This system adds two mechanisms:

1. **Continuous writing** — Your agent writes important context to `memory/YYYY-MM-DD.md` files *throughout the session*, not just at the end. Decisions, preferences, project state — all captured on disk in real-time.

2. **Pre-compaction flush** — When a session is ~4,000 tokens from the compaction threshold, Clawdbot triggers a *silent* reminder. The agent reviews what's in context, writes anything important that hasn't been saved yet, and then compaction proceeds safely. The agent responds `NO_REPLY` so you never see this happen.

**Result:** Your agent's memory is on disk, indexed, and searchable. Compaction can't touch it. Crashes can't touch it. Restarts can't touch it.

## Embedding Providers

| Provider | Speed | Cost | Setup |
|----------|-------|------|-------|
| **OpenAI** ⭐ | ⚡ Fast | ~$0.02/million tokens (~$0.50/mo) | API key required |
| **Gemini** | ⚡ Fast | Free tier available | API key required |
| **Local** | 🐢 Slower first build | Free forever | Auto-downloads ~600MB model |

**Jake uses OpenAI** — it's the fastest and most reliable. At ~$0.50/month it's basically free.

**Local** uses `node-llama-cpp` with a GGUF model — fully offline, no API key, but first index build is slower.

## Production Stats (Jake's Setup)

| Metric | Value |
|--------|-------|
| Files indexed | 35 |
| Chunks | 121 |
| Search speed | <100ms |
| SQLite size | 15 MB |
| Monthly cost | ~$0.50 (OpenAI) |
| Data loss incidents | **Zero** |
| Crashes survived | 5+ |
| Days in production | 26+ |

## Manual Setup

If you prefer to set things up yourself instead of using the installer:

### 1. Create memory directory
```bash
cd ~/.clawdbot/workspace
mkdir -p memory
```

### 2. Add to clawdbot.json
```json
{
  "agents": {
    "defaults": {
      "memorySearch": {
        "enabled": true,
        "provider": "openai",
        "model": "text-embedding-3-small",
        "query": {
          "hybrid": {
            "enabled": true,
            "vectorWeight": 0.7,
            "textWeight": 0.3,
            "candidateMultiplier": 4
          }
        }
      }
    }
  }
}
```

### 3. Add to AGENTS.md
See `config/agents-memory-patch.md` for the exact text to append.

### 4. Restart and verify
```bash
clawdbot gateway restart
clawdbot memory index --verbose
clawdbot memory status --deep
```

## Troubleshooting

### "Memory search disabled"
**Cause:** No embedding provider configured or API key missing.
**Fix:** Run the installer again, or add your API key to clawdbot.json manually.

### Agent still forgets after compaction
**Cause:** AGENTS.md may not have the memory instructions.
**Fix:** Check that AGENTS.md contains the "Memory System" section. Re-run installer if needed.

### Search returns no results
**Possible causes:**
1. Index not built — run `clawdbot memory index --verbose`
2. No memory files yet — create your first daily log
3. Query too specific — try broader terms

### Rebuild index from scratch
```bash
rm ~/.clawdbot/memory/main.sqlite
clawdbot memory index --verbose
```
Markdown files are the source of truth — SQLite is always regenerable.

### Check system health
```bash
clawdbot memory status --deep
```

## FAQ

**Q: Will this slow down my agent?**
A: No. Search takes <100ms. Indexing happens in the background. Writing to files takes milliseconds.

**Q: How much disk space does it use?**
A: ~15MB for the SQLite index with 35 files. Markdown files themselves are tiny.

**Q: Can I edit memory files manually?**
A: Yes! They're plain Markdown. Edit in any text editor. Changes are auto-indexed.

**Q: What if I delete the SQLite database?**
A: Just run `clawdbot memory index --verbose` to rebuild it. Markdown files are the source of truth.

**Q: Does this work with OpenClaw / Moltbot?**
A: Yes. The installer auto-detects all three (Clawdbot, OpenClaw, Moltbot).

**Q: Can multiple agents share the same memory?**
A: Each agent gets its own SQLite index, but they can read the same markdown files if they share a workspace.

**Q: Is my data sent to the cloud?**
A: Only the text chunks are sent to generate embeddings (OpenAI/Gemini). The actual memory files stay on your disk. Use `local` provider for fully offline operation.

## License

MIT — use it however you want.

## Credits

Built by [Buba](https://github.com/BusyBee3333) (Jake's Clawdbot agent) based on 26+ days of production use.

*Your agent will never forget again.* ᕕ( ᐛ )ᕗ