Jake Shore df4aa799f8 Daily backup: 2026-01-24 - Workspace files including Discord bot automation research, Reonomy scraper versions, backup scripts, and project config

2026-01-24 05:09:55 -05:00

7.6 KiB

Raw Blame History

DigitalOcean Spaces Setup Guide

Updated: January 21, 2026

🌊 What is DigitalOcean Spaces?

DigitalOcean Spaces is S3-compatible object storage. Think of it like Dropbox but:

Programmable (rclone, boto3, etc.)
250 GB free for 10 months (new accounts)
S3-compatible (works with any S3 tool)
Fast and cheap (~$5/month for 250 GB after promo)

✅ Prerequisites

# Check if rclone is installed
rclone --version
# Expected: rclone v1.72.1 or higher

🔑 Step 1: Get Your DigitalOcean API Token

1.1 Log in to DigitalOcean

Go to: https://cloud.digitalocean.com/

1.2 Navigate to API Tokens

Click your avatar (top right)
Select "API" from the dropdown
You're now on the API page

1.3 Generate a New Token

Scroll to "Personal access tokens"
Click "Generate New Token"

1.4 Configure Your Token

Name: Enter something descriptive like rclone-spaces-backup-jan2026
Expiration: Leave blank (never expires) OR set to 1 year
Scopes: Select "Write" (you need write access for backups)

1.5 Copy Your Token

Click "Generate Token"
IMPORTANT: Copy the token NOW — you'll only see it once
It looks like: dop_v1_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Save it to 1Password with label: "DigitalOcean Spaces API Token - rclone"

📦 Step 2: Create Your First Space

2.1 Go to Spaces

In the left sidebar, click "Spaces"
Click "Create Space"

2.2 Configure Your Space

Name: remix-sniper-backup (or whatever you want)
Region: Choose the closest to you
- NYC3 = New York (recommended for you on East Coast)
- SFO2 = San Francisco
- AMS3 = Amsterdam
- FRA1 = Frankfurt
- SGP1 = Singapore
CDN: Leave unchecked (not needed for backups)
Visibility: Select "Private" (important!)

2.3 Create

Click "Create Space"
Wait for the space to be created (~5 seconds)

🔧 Step 3: Configure rclone

3.1 Start rclone config

rclone config

3.2 Follow the prompts

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

name> do-spaces

Type of storage to configure.
Choose a number from below, or type in your own value.
 1 / 1Fichier
...
 4 / Amazon S3
...
s/Memtype> 4

Choose your S3 provider.
Choose a number from below, or type in your own value.
 1 / AWS S3
 2 / Ceph Object Storage
 3 / DigitalOcean Spaces
 4 / Dreamhost DreamObjects
...
s/Memtype> 3

Get AWS credentials from runtime (environment variables or EC2/ECS meta data only,
no stdin), or enter them in the next step.
Enter a string value. Press Enter for the default (false).
choice> false

Access Key ID.
Leave blank for anonymous access or runtime credentials.
access_key_id> YOUR_SPACES_ACCESS_KEY

⚠️ How to find your Access Key & Secret Key

DigitalOcean uses the same API token for Spaces, but you may need to generate a separate "Spaces Access Key":

Go to: https://cloud.digitalocean.com/account/api/tokens
Look for "Spaces access keys"
Click "Generate New Key"
Name it: rclone-spaces-jan2026
Copy both:
- Access Key ID (shorter, looks like: xxxxxxxxxxxxxxxxxxxx)
- Secret Access Key (longer, base64-encoded)

Use these for rclone, NOT the token from step 1!

Continue with rclone config...

Secret Access Key.
Leave blank for anonymous access or runtime credentials.
secret_access_key> YOUR_SPACES_SECRET_KEY

Region to connect to.
Choose a number from below, or type in your own value.
 1 / nyc3
 2 / ams3
 3 / sgp1
 4 / sfo2
 5 / sfo3
 6 / fra1
region> 1

Endpoint for the S3 API.
Leave blank if using AWS defaults or the previous region default.
endpoint> https://nyc3.digitaloceanspaces.com

Location constraint.
Must be set to match the Region. Used when creating buckets.
Choose a number from below, or type in your own value.
 1 / Empty string (US East, N. Virginia or us-east-1)
...
location_constraint> nyc3

ACL.
Choose a number from below, or type in your own value.
 1 / Owner gets Full Control, others get no access
 2 / Owner gets Full Control, others get Full Control
...
acl> 1

Edit advanced config? (y/n)
y/n> n

Remote config
--------------------
[do-spaces]
type = s3
provider = DigitalOcean
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
region = nyc3
endpoint = https://nyc3.digitaloceanspaces.com
location_constraint = nyc3
acl = private
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

Current remotes:

Name                 Type
----                 ----
do-spaces            s3

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

✅ Step 4: Test Your Connection

# List contents of your space
rclone ls do-spaces:remix-sniper-backup

# Expected: Empty (or files if you already uploaded)

If this works, you're set up! 🎉

🚀 Step 5: Run Your First Backup

Use the existing backup script:

# Test backup (dry run first)
~/.clawdbot/workspace/backup_to_cloud.sh do-spaces

# This will backup:
# - PostgreSQL database dump
# - Environment files
# - Clawdbot workspace
# - Tracking data (JSON files)
# - Custom scripts

⏰ Step 6: Automate Daily Backups

Add to crontab:

# Edit crontab
crontab -e

# Add this line (runs daily at 2 AM):
0 2 * * * ~/.clawdbot/workspace/backup_to_cloud.sh do-spaces >> ~/.clawdbot/workspace/cloud-backup.log 2>&1

🔍 Useful Commands

# Check rclone config
rclone config show

# List all spaces
rclone lsd do-spaces:

# Sync local directory to cloud (mirror)
rclone sync /path/to/local do-spaces:remix-sniper-backup/folder

# Copy to cloud (adds new, doesn't delete)
rclone copy /path/to/local do-spaces:remix-sniper-backup/folder

# Check what would sync (dry run)
rclone check /path/to/local do-spaces:remix-sniper-backup/folder

# Monitor transfer in real-time
rclone sync /path/to/local do-spaces:remix-sniper-backup --progress

🛡️ Security Notes

⚠️ NEVER:

Commit ~/.config/rclone/rclone.conf to git (contains your secret keys)
Share your Access Key or Secret Key
Make your Space public (keep it "Private")

✅ ALWAYS:

Store keys in 1Password
Use "Private" ACL for backups
Rotate keys if you suspect a leak:
1. Delete old key in DigitalOcean
2. Generate new key
3. Update rclone config: rclone config edit do-spaces

💰 Pricing After Free Trial

Storage	Monthly Cost
250 GB	~$5.00/month
500 GB	~$10.00/month
1 TB	~$20.00/month

Bandwidth (outbound):

First 500 GB/month: FREE
After: $0.01/GB

🆘 Troubleshooting

"Access Denied"

Double-check your Access Key and Secret Key
Make sure you're using the right region

"No such host"

Check your endpoint URL
Should be: https://REGION.digitaloceanspaces.com

"Connection timeout"

Check your internet connection
Verify your region is correct

📞 Need Help?

DigitalOcean Spaces Docs: https://docs.digitalocean.com/products/spaces/
rclone S3 Docs: https://rclone.org/s3/
Tag Buba in Discord

✅ Checklist

DigitalOcean account created
API token generated
Spaces Access Key generated
Space created (name + region)
rclone configured with do-spaces remote
Test connection works: rclone ls do-spaces:remix-sniper-backup
First backup completed
Keys saved to 1Password
Daily cron job added

7.6 KiB Raw Blame History