jake/clawdbot-workspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
clawdbot-workspace/CLOUD-BACKUP-SYSTEM.md

348 lines
9.5 KiB
Markdown
Raw Blame History

# Cloud Backup System - Complete Guide
*Created: 2026-01-19*
*For: Computer reset protection*
---
## 🎯 What Was Done
Created a **three-tier backup system** for maximum protection:
1. **Local Backups** - Fast, accessible offline
2. **Cloud Backups** - Safe from computer reset/failure
3. **GitHub Backups** - Version history for code
---
## 📂 Files Created
| File | Purpose |
|------|---------|
| `backup_before_reset.sh` | Local backup (all data) |
| `restore_after_reset.sh` | Local restore (all data) |
| `backup_to_cloud.sh` | Cloud backup (via rclone) |
| `restore_from_cloud.sh` | Cloud restore (via rclone) |
| `backup_to_github.sh` | GitHub code backup (version control) |
| `RESET-IMPACT-ANALYSIS.md` | Detailed risk analysis |
| `BACKUP-RESTORE-QUICK-REF.md` | Quick reference |
| `CLOUD-BACKUP-SETUP.md` | Cloud storage setup guide |
---
## 🚀 Quick Start - Get Protected Now
### Step 1: Set Up Cloud Storage (One-Time)
```bash
# Choose your provider and run rclone config
rclone config
# Recommended: DigitalOcean Spaces
# 1. Get API key: https://cloud.digitalocean.com/account/api/tokens
# 2. Name remote: do-spaces
# 3. Type: s3 -> DigitalOcean
# 4. Region: nyc3
# 5. Endpoint: https://nyc3.digitaloceanspaces.com
# Test connection
rclone ls do-spaces:
```
### Step 2: Run First Cloud Backup
```bash
# Backup everything to cloud
~/.clawdbot/workspace/backup_to_cloud.sh do-spaces
```
### Step 3: (Optional) Set Up GitHub
```bash
# Backup code to GitHub
cd ~/projects/remix-sniper
~/.clawdbot/workspace/backup_to_github.sh jakeshore remix-sniper
# Follow prompts to create repository and push
```
---
## 📊 Backup Layers Comparison
```
┌─────────────────────────────────────────────────────────────┐
│ BACKUP SYSTEM │
├─────────────────────────────────────────────────────────────┤
│ │
│ LAYER 1: LOCAL BACKUP │
│ ├─ backup_before_reset.sh │
│ ├─ All data, configs, logs │
│ └─ Fast, offline │
│ │
│ LAYER 2: CLOUD BACKUP │
│ ├─ backup_to_cloud.sh │
│ ├─ All data, configs, logs │
│ ├─ Safe from computer failure │
│ └─ Requires rclone config │
│ │
│ LAYER 3: GITHUB BACKUP │
│ ├─ backup_to_github.sh │
│ ├─ Code only (no data/configs) │
│ ├─ Version history │
│ └─ Requires GitHub repo │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## 🔄 Recommended Backup Strategy
### Daily Workflow
```bash
# Run local backup (fast)
~/.clawdbot/workspace/backup_before_reset.sh
# Run cloud backup (safe)
~/.clawdbot/workspace/backup_to_cloud.sh do-spaces
# Commit code changes (optional)
cd ~/projects/remix-sniper
git add . && git commit -m "Daily backup" && git push
```
### Automatic Backups (Cron)
```bash
# Edit crontab
crontab -e
# Add daily cloud backup at 2am:
0 2 * * * ~/.clawdbot/workspace/backup_to_cloud.sh do-spaces >> ~/.clawdbot/workspace/cloud-backup.log 2>&1
```
---
## 📋 What Gets Protected
### Critical Data (Backed up to local + cloud)
| Item | Why Critical |
|------|--------------|
| Cron jobs (6) | Automated tasks - would stop working |
| Launchd service | Remi bot auto-restart - would stop |
| PostgreSQL database | All predictions, metrics, opportunities |
| Tracking data | 8 predictions, 1 remix outcome |
| Environment files | API tokens, database URL |
| Custom scripts | Pickle motivation, daily scan, etc. |
### Code (Backed up to GitHub)
| Item | Location |
|------|----------|
| Remix Sniper bot code | `~/projects/remix-sniper/packages/` |
| Scrapers | `packages/core/scrapers/` |
| Analyzers | `packages/core/analyzers/` |
| Database models | `packages/core/database/` |
| Bot commands | `packages/bot/cogs/` |
---
## 🔧 Setup Checklist
### Required for Local Backups
- [x] Scripts created
- [x] Scripts executable
- [ ] Run test backup: `~/.clawdbot/workspace/backup_before_reset.sh`
### Required for Cloud Backups
- [x] rclone installed
- [x] Scripts created
- [x] Scripts executable
- [ ] Configure rclone remote: `rclone config`
- [ ] Test connection: `rclone ls <remote-name>:`
- [ ] Run test backup: `~/.clawdbot/workspace/backup_to_cloud.sh <remote-name>`
### Required for GitHub Backups
- [x] Script created
- [x] Script executable
- [ ] Create GitHub repository: https://github.com/new
- [ ] Run first backup: `~/.clawdbot/workspace/backup_to_github.sh <username> <repo-name>`
---
## 📥 Restore Procedures
### Before Computer Reset
```bash
# 1. Run local backup
~/.clawdbot/workspace/backup_before_reset.sh
# 2. Run cloud backup (if configured)
~/.clawdbot/workspace/backup_to_cloud.sh do-spaces
# 3. Push to GitHub (if configured)
cd ~/projects/remix-sniper
git add . && git commit -m "Pre-reset backup" && git push
```
### After Computer Reset
```bash
# Option 1: Restore from local backup (if preserved)
~/.clawdbot/workspace/restore_after_reset.sh ~/.clawdbot/workspace/backup-before-reset-YYYYMMDD-HHMMSS
# Option 2: Restore from cloud backup
~/.clawdbot/workspace/restore_from_cloud.sh do-spaces remix-sniper-backup backup-cloud-YYYYMMDD-HHMMSS
# Option 3: Clone from GitHub (code only)
git clone https://github.com/<username>/remix-sniper.git ~/projects/remix-sniper
```
---
## 🌤️ Cloud Storage Options
| Provider | Free Tier | Setup | Notes |
|-----------|------------|--------|--------|
| **DigitalOcean Spaces** | 250 GB (10mo) | ⭐⭐⭐ Recommended |
| Google Drive | 15 GB | ⭐⭐ Easy |
| Dropbox | 2 GB | ⭐⭐ Easy |
| AWS S3 | 5 GB | ⭐⭐⭐ |
| OneDrive | 5 GB | ⭐⭐ |
### Why DigitalOcean Spaces?
- Already have DO account
- 250 GB free storage (huge for backups)
- S3-compatible (standard tools)
- Fast, reliable
- Good for long-term storage
---
## 🔍 Verification Commands
### Local Backup
```bash
# Check backup exists
ls -lh ~/.clawdbot/workspace/backup-before-reset-*
# Verify contents
cat ~/.clawdbot/workspace/backup-before-reset-*/MANIFEST.txt
```
### Cloud Backup
```bash
# List backups
rclone ls do-spaces:remix-sniper-backup/
# Check backup size
rclone size do-spaces:remix-sniper-backup/backup-cloud-YYYYMMDD-HHMMSS/
```
### GitHub Backup
```bash
# Check repository
cd ~/projects/remix-sniper
git status
git log --oneline -5
# Check remote
git remote -v
```
---
## ⚠️ Troubleshooting
### Local backup: "Permission denied"
```bash
# Fix permissions
chmod +x ~/.clawdbot/workspace/*.sh
```
### Cloud backup: "Remote not configured"
```bash
# List remotes
rclone listremotes
# Configure new remote
rclone config
```
### Cloud backup: "Authentication failed"
```bash
# Remove and reconfigure
rclone config delete <remote-name>
rclone config
```
### GitHub backup: "Repository not found"
```bash
# Create repository first
# 1. Go to: https://github.com/new
# 2. Name: remix-sniper
# 3. Don't initialize
# 4. Run: git remote add origin <url>
```
---
## 📚 Documentation
| File | Purpose |
|------|---------|
| `RESET-IMPACT-ANALYSIS.md` | What's at risk, detailed analysis |
| `BACKUP-RESTORE-QUICK-REF.md` | Quick reference for backup/restore |
| `CLOUD-BACKUP-SETUP.md` | Cloud storage setup guide |
| `memory/2026-01-19-backup-system.md` | Memory log |
| `remix-sniper-skill.md` | Remix Sniper quick reference |
| `memory/2026-01-19-remix-sniper-setup.md` | Remi setup log |
---
## ✅ What's Protected Now
| Threat | Local Backup | Cloud Backup | GitHub |
|---------|--------------|---------------|----------|
| Computer reset | ✅ | ✅ | ✅ (code) |
| Hard drive failure | ❌ | ✅ | ✅ (code) |
| Accidental deletion | ❌ | ✅ | ✅ (code) |
| Lost/destroyed computer | ❌ | ✅ | ✅ (code) |
**Recommendation:** Use **all three** for maximum protection.
---
## 🚀 Next Steps
1. **Set up cloud storage** - Run `rclone config`
2. **Run first cloud backup** - `~/.clawdbot/workspace/backup_to_cloud.sh do-spaces`
3. **Set up GitHub** - Create repo and run `~/.clawdbot/workspace/backup_to_github.sh`
4. **Test restore** - Ensure restore scripts work before you need them
5. **Schedule automatic backups** - Add to crontab for daily cloud backups
---
## 💛 Questions?
If anything goes wrong:
1. Check the backup scripts in `~/.clawdbot/workspace/`
2. Read the detailed guides (CLOUD-BACKUP-SETUP.md, RESET-IMPACT-ANALYSIS.md)
3. Tag Buba in Discord
---
## 📞 Emergency Contacts
**If all backups fail:**
1. **Reconstruct from notes** - Use `remix-sniper-skill.md` and memory logs
2. **Use 1Password** - Access credentials for API keys
3. **Reinstall tools** - PostgreSQL, Homebrew, rclone
4. **Contact support** - DigitalOcean, GitHub, etc.
---
**Last updated:** 2026-01-19
**Version:** 1.0
Reference in New Issue View Git Blame Copy Permalink