331 lines
7.4 KiB
Markdown
331 lines
7.4 KiB
Markdown
# Cloud Backup Setup Guide
|
|
|
|
*Created: 2026-01-19*
|
|
|
|
---
|
|
|
|
## 🌤️ Overview
|
|
|
|
This guide helps you set up **cloud backups** for all critical Remix Sniper and Clawdbot data. Cloud backups protect your data even if:
|
|
- Your computer is reset
|
|
- Hard drive fails
|
|
- Local backups are lost
|
|
|
|
---
|
|
|
|
## 🛠️ Prerequisites
|
|
|
|
```bash
|
|
# rclone is already installed
|
|
rclone --version
|
|
# Expected: rclone v1.72.1
|
|
```
|
|
|
|
---
|
|
|
|
## 📁 Step 1: Choose a Cloud Storage Provider
|
|
|
|
| Provider | Free Tier | Storage | Setup Difficulty |
|
|
|-----------|------------|----------|-----------------|
|
|
| **Google Drive** | 15 GB | Easy | ⭐⭐ |
|
|
| **Dropbox** | 2 GB | Easy | ⭐⭐ |
|
|
| **DigitalOcean Spaces** | 250 GB (10 months) | Medium | ⭐⭐⭐ |
|
|
| **AWS S3** | 5 GB | Medium | ⭐⭐⭐ |
|
|
| **OneDrive** | 5 GB | Easy | ⭐⭐ |
|
|
|
|
### Recommendation
|
|
|
|
**DigitalOcean Spaces** - Great for you since you already have a DO account:
|
|
- 250 GB free storage (10 months promo)
|
|
- S3-compatible (works with rclone)
|
|
- Fast upload/download
|
|
- Good API for automation
|
|
|
|
---
|
|
|
|
## 🔑 Step 2: Configure rclone with Your Provider
|
|
|
|
### Option A: DigitalOcean Spaces (Recommended)
|
|
|
|
```bash
|
|
# Run rclone config
|
|
rclone config
|
|
|
|
# Follow the prompts:
|
|
# 1. Enter "n" for new remote
|
|
# 2. Name it: do-spaces
|
|
# 3. Type: s3
|
|
# 4. Provider: DigitalOcean
|
|
# 5. Auth: Use your DO API key
|
|
# - Get from: https://cloud.digitalocean.com/account/api/tokens
|
|
# - Generate new token with "Write" scope
|
|
# 6. Region: nyc3 (or your region)
|
|
# 7. Endpoint: https://nyc3.digitaloceanspaces.com
|
|
# 8. ACL: private
|
|
# 9. Advanced options: press Enter (skip)
|
|
# 10. Confirm: yes
|
|
|
|
# Test connection
|
|
rclone ls do-spaces:
|
|
```
|
|
|
|
### Option B: Google Drive
|
|
|
|
```bash
|
|
# Run rclone config
|
|
rclone config
|
|
|
|
# Follow the prompts:
|
|
# 1. Enter "n" for new remote
|
|
# 2. Name it: gdrive
|
|
# 3. Type: drive
|
|
# 4. Scope: drive (full access)
|
|
# 5. Client ID: press Enter (auto)
|
|
# 6. Client Secret: press Enter (auto)
|
|
# 7. Choose advanced config: n
|
|
# 8. Use auto config: y
|
|
# 9. Browser will open - sign in to Google Drive
|
|
# 10. Allow rclone access
|
|
# 11. Confirm remote: y
|
|
|
|
# Test connection
|
|
rclone ls gdrive:
|
|
```
|
|
|
|
### Option C: Dropbox
|
|
|
|
```bash
|
|
# Run rclone config
|
|
rclone config
|
|
|
|
# Follow the prompts:
|
|
# 1. Enter "n" for new remote
|
|
# 2. Name it: dropbox
|
|
# 3. Type: dropbox
|
|
# 4. Client ID: press Enter (auto)
|
|
# 5. Client Secret: press Enter (auto)
|
|
# 6. Choose advanced config: n
|
|
# 7. Use auto config: y
|
|
# 8. Browser will open - sign in to Dropbox
|
|
# 9. Allow rclone access
|
|
# 10. Confirm remote: y
|
|
|
|
# Test connection
|
|
rclone ls dropbox:
|
|
```
|
|
|
|
---
|
|
|
|
## 🚀 Step 3: Run Your First Cloud Backup
|
|
|
|
```bash
|
|
# Make scripts executable
|
|
chmod +x ~/.clawdbot/workspace/backup_to_cloud.sh
|
|
chmod +x ~/.clawdbot/workspace/restore_from_cloud.sh
|
|
|
|
# Run backup
|
|
~/.clawdbot/workspace/backup_to_cloud.sh <remote-name>
|
|
|
|
# Examples:
|
|
~/.clawdbot/workspace/backup_to_cloud.sh do-spaces
|
|
~/.clawdbot/workspace/backup_to_cloud.sh gdrive
|
|
~/.clawdbot/workspace/backup_to_cloud.sh dropbox
|
|
```
|
|
|
|
---
|
|
|
|
## 🔄 Step 4: Set Up Automatic Cloud Backups (Optional)
|
|
|
|
### Add to crontab
|
|
|
|
```bash
|
|
# Edit crontab
|
|
crontab -e
|
|
|
|
# Add these lines (daily at 2am):
|
|
0 2 * * * ~/.clawdbot/workspace/backup_to_cloud.sh do-spaces >> ~/.clawdbot/workspace/cloud-backup.log 2>&1
|
|
|
|
# Save and exit (Ctrl+O, Enter, Ctrl+X in nano)
|
|
```
|
|
|
|
### Alternative: Weekly backups (less bandwidth)
|
|
|
|
```bash
|
|
# Add to crontab (Sundays at 2am):
|
|
0 2 * * 0 ~/.clawdbot/workspace/backup_to_cloud.sh do-spaces >> ~/.clawdbot/workspace/cloud-backup.log 2>&1
|
|
```
|
|
|
|
---
|
|
|
|
## 📥 Step 5: How to Restore from Cloud
|
|
|
|
### List Available Backups
|
|
|
|
```bash
|
|
# List all backups in cloud
|
|
rclone ls do-spaces:remix-sniper-backup/
|
|
|
|
# Expected output:
|
|
# backup-cloud-20260119-120000/
|
|
# backup-cloud-20260120-120000/
|
|
# ...
|
|
```
|
|
|
|
### Restore a Specific Backup
|
|
|
|
```bash
|
|
# Restore from cloud
|
|
~/.clawdbot/workspace/restore_from_cloud.sh do-spaces remix-sniper-backup backup-cloud-20260119-120000
|
|
|
|
# After restore, verify:
|
|
crontab -l # Check cron jobs
|
|
launchctl list | grep remix # Check launchd
|
|
psql -d remix_sniper -c '\l' # Check database
|
|
```
|
|
|
|
---
|
|
|
|
## 💾 What Gets Backed Up
|
|
|
|
| Item | Description |
|
|
|------|-------------|
|
|
| **Cron jobs** | All 6 cron tasks (pickles, scans, reports) |
|
|
| **Launchd services** | Remi bot auto-restart service |
|
|
| **PostgreSQL database** | Full database dump (songs, metrics, opportunities) |
|
|
| **Tracking data** | Predictions, remixes, snapshots |
|
|
| **Environment files** | `.env` with API keys and tokens |
|
|
| **Clawdbot workspace** | All workspace files and notes |
|
|
| **Custom scripts** | Shell scripts (pickle motivation, etc.) |
|
|
|
|
---
|
|
|
|
## 📊 Backup Comparison
|
|
|
|
| Backup Type | Location | Pros | Cons |
|
|
|-------------|-----------|--------|-------|
|
|
| **Local backup** | `~/.clawdbot/workspace/backup-*/` | Fast, no internet | Lost if computer reset/hard drive fails |
|
|
| **Cloud backup** | rclone remote | Safe from computer issues | Requires internet, may have storage limits |
|
|
| **Git (code only)** | GitHub | Version history, code safe | Doesn't backup data/configs |
|
|
|
|
**Recommended:** Use **both** local + cloud for maximum safety.
|
|
|
|
---
|
|
|
|
## 🔍 Troubleshooting
|
|
|
|
### rclone config: "remote not found"
|
|
```bash
|
|
# List configured remotes
|
|
rclone listremotes
|
|
|
|
# Check if your remote is listed
|
|
```
|
|
|
|
### rclone config: authentication failed
|
|
```bash
|
|
# Remove the remote and reconfigure
|
|
rclone config delete <remote-name>
|
|
rclone config
|
|
```
|
|
|
|
### Cloud backup fails: "permission denied"
|
|
```bash
|
|
# Check remote permissions
|
|
rclone lsd <remote-name>:
|
|
|
|
# May need to reconfigure with correct credentials
|
|
```
|
|
|
|
### Upload is very slow
|
|
```bash
|
|
# Increase transfers in backup script
|
|
# Edit backup_to_cloud.sh, change:
|
|
rclone sync ... --transfers 4
|
|
# To:
|
|
rclone sync ... --transfers 10
|
|
```
|
|
|
|
---
|
|
|
|
## 📋 Quick Reference
|
|
|
|
### Configure cloud storage
|
|
```bash
|
|
rclone config
|
|
```
|
|
|
|
### List configured remotes
|
|
```bash
|
|
rclone listremotes
|
|
```
|
|
|
|
### Run backup
|
|
```bash
|
|
~/.clawdbot/workspace/backup_to_cloud.sh <remote-name>
|
|
```
|
|
|
|
### List cloud backups
|
|
```bash
|
|
rclone ls <remote-name>:remix-sniper-backup/
|
|
```
|
|
|
|
### Restore from cloud
|
|
```bash
|
|
~/.clawdbot/workspace/restore_from_cloud.sh <remote-name> remix-sniper-backup <backup-folder>
|
|
```
|
|
|
|
### Check backup logs
|
|
```bash
|
|
tail -f ~/.clawdbot/workspace/cloud-backup.log
|
|
```
|
|
|
|
---
|
|
|
|
## 🔗 Useful Links
|
|
|
|
- **rclone Documentation**: https://rclone.org/docs/
|
|
- **DigitalOcean Spaces**: https://docs.digitalocean.com/products/spaces/
|
|
- **Google Drive**: https://drive.google.com/
|
|
- **Dropbox**: https://www.dropbox.com/
|
|
|
|
---
|
|
|
|
## ❓ Need Help?
|
|
|
|
1. Check the backup script: `~/.clawdbot/workspace/backup_to_cloud.sh`
|
|
2. Check the restore script: `~/.clawdbot/workspace/restore_from_cloud.sh`
|
|
3. Run test backup: `~/.clawdbot/workspace/backup_to_cloud.sh <remote-name>`
|
|
4. Tag Buba in Discord if stuck
|
|
|
|
---
|
|
|
|
## 💛 Security Tips
|
|
|
|
1. **Never share your rclone config file**
|
|
- Location: `~/.config/rclone/rclone.conf`
|
|
- Contains your API keys/tokens
|
|
|
|
2. **Use private cloud storage**
|
|
- Don't make backups public
|
|
- Access should be restricted
|
|
|
|
3. **Rotate credentials**
|
|
- Change API keys periodically
|
|
- Remove unused remotes: `rclone config delete <name>`
|
|
|
|
4. **Enable two-factor**
|
|
- On cloud provider accounts
|
|
- On 1Password
|
|
|
|
---
|
|
|
|
## ✅ Checklist
|
|
|
|
- [ ] Choose cloud storage provider
|
|
- [ ] Configure rclone with remote
|
|
- [ ] Test connection: `rclone ls <remote-name>:`
|
|
- [ ] Run first backup: `~/.clawdbot/workspace/backup_to_cloud.sh <remote-name>`
|
|
- [ ] Verify backup exists in cloud
|
|
- [ ] (Optional) Add to crontab for automatic backups
|
|
- [ ] (Optional) Test restore from cloud
|