1.1 KiB
2026-02-25 Session Notes
Exploring Character-Level GPT Training on Memory Database
Nicholai proposed training the microgpt character-level model (Rust implementation at references/microgpt/src/main.rs) on personal memory data from ~/.agents/memory/memories.db to see what the model "hallucinates" - essentially a funhouse-mirror experiment on his own agent memories.
Investigated the memories database schema and found 5348 memory records. The memories table contains rich metadata: content (TEXT), type (fact/preference/decision/etc), tags, importance (0.0-1.0), confidence, category, timestamps, source tracking, and extraction status. The database includes FTS5 full-text search indexes and vector embeddings infrastructure.
The character-level model is tiny (48 embedding dimensions, 2 layers, 4 attention heads) and expects line-delimited text input. The feasibility depends on extracting memories in a format the model can meaningfully learn from—raw content lines or structured summaries.
No implementation work began; this was pure exploration of the concept and data shape.