80 lines
2.8 KiB
Markdown
80 lines
2.8 KiB
Markdown
# 2026-01-26
|
|
|
|
## Genre Universe - 3D Artist Visualization
|
|
|
|
Built a full Three.js interactive 3D visualization showing where Das sits in genre space relative to other artists. See `2026-01-26-genre-universe.md` for full details.
|
|
|
|
**Key files:** `~/.clawdbot/workspace/genre-viz/`
|
|
**Live:** http://localhost:8420
|
|
|
|
---
|
|
|
|
## PageIndex Memory System
|
|
|
|
Implemented PageIndex-style hierarchical memory structure at `memory/INDEX.json`. Tree-based navigation with:
|
|
- Node IDs for each knowledge domain
|
|
- File references with line numbers
|
|
- Summaries and keywords for reasoning-based retrieval
|
|
- No vector search - uses tree traversal + reasoning
|
|
|
|
---
|
|
|
|
## Reonomy Scraper v13 - Complete Rebuild
|
|
|
|
### What We Built Today
|
|
|
|
**1. Production Scraper (`reonomy-scraper-v13.js`)**
|
|
- Full anti-detection: random delays (3-8s), shuffled property order, occasional breaks
|
|
- Daily limits: tracks properties scraped per day, caps at 50/day
|
|
- Session management: saves/loads auth state, auto-relogins if expired
|
|
- Appends to existing leads file (doesn't overwrite)
|
|
|
|
**2. Quick Demo Script (`reonomy-quick-demo.js`)**
|
|
- Single-contact extraction in ~60 seconds
|
|
- Opens headed browser so you can show someone
|
|
- Leaves browser open for inspection
|
|
|
|
### How to Run
|
|
|
|
```bash
|
|
# Demo (show someone how it works)
|
|
node ~/.clawdbot/workspace/reonomy-quick-demo.js
|
|
|
|
# Production scrape (20 properties, anti-detection)
|
|
node ~/.clawdbot/workspace/reonomy-scraper-v13.js
|
|
|
|
# Custom limits
|
|
MAX_PROPERTIES=5 node ~/.clawdbot/workspace/reonomy-scraper-v13.js
|
|
```
|
|
|
|
### Proven Extraction Workflow
|
|
|
|
1. Login → https://app.reonomy.com/#!/login
|
|
2. Search with filters → `/search/{search-id}`
|
|
3. Click property card → `/property/{id}/building`
|
|
4. Click "Owner" tab → `/property/{id}/ownership`
|
|
5. Click "View Contacts (X)" → navigates to company page
|
|
6. Click person link → `/person/{person-id}`
|
|
7. Click "Contact" button → modal with phones/emails
|
|
8. Extract from modal:
|
|
- Phones: `button "XXX-XXX-XXXX Company Name"`
|
|
- Emails: `button "email@domain.com"`
|
|
|
|
### Anti-Detection Features
|
|
- Random delays 3-8 seconds between actions
|
|
- Shuffled property order (not sequential)
|
|
- 20% chance of "coffee break" (8-15s pause)
|
|
- 30% chance of random scroll/hover actions
|
|
- Daily limit of 50 properties
|
|
- Session reuse (doesn't login/logout constantly)
|
|
|
|
### Files
|
|
- `/Users/jakeshore/.clawdbot/workspace/reonomy-scraper-v13.js` - Production
|
|
- `/Users/jakeshore/.clawdbot/workspace/reonomy-quick-demo.js` - Demo
|
|
- `/Users/jakeshore/.clawdbot/workspace/reonomy-leads-v13.json` - Output
|
|
- `/Users/jakeshore/.clawdbot/workspace/reonomy-daily-stats.json` - Daily tracking
|
|
- `/Users/jakeshore/.clawdbot/workspace/reonomy-auth.json` - Saved session
|
|
|
|
### Search ID (with phone+email filters)
|
|
`bacfd104-fed5-4cc4-aba1-933f899de3f8` - FL Multifamily with phone filter
|