SDXL-Lightning Image Generation for macOS (MPS)

We have successfully implemented a local image generation setup using SDXL-Lightning on macOS with Metal Performance Shaders (MPS) acceleration.

Features Status

Feature	Status	Notes
Generation Script	✅ Complete	`generate.py` supports text-to-image with custom resolution
Model	✅ Integrated	SDXL-Lightning 4-step UNet + SDXL Base Refiner components
Optimization	✅ Applied	Attention slicing enabled for 16GB RAM compatibility
Download	✅ Robust	Uses `hf_transfer` for fast, resilient model downloading
Git	✅ Pushed	Project initialized and pushed to remote repository

Usage Guide

To generate an image, open your terminal and run:

# Activate virtual environment
source venv/bin/activate

# Run generation (first run will download models)
HF_HUB_ENABLE_HF_TRANSFER=1 python generate.py "A futuristic cityscape, cyberpunk style" --width 1024 --height 1024

Note

: The first run requires downloading ~5-6GB of model weights. We use HF_HUB_ENABLE_HF_TRANSFER=1 to accelerate this process. Subsequent runs will be instant.

Project Structure

generate.py: Main script. Handles model loading, optimization, and generation.
models/: Stores the SDXL-Lightning UNet checkpoint.
hf_cache/: Stores the Diffusers cache (base model components).
requirements.txt: Python dependencies.

Optimizations Logic

To ensure smooth performance on macOS:

MPS Device: Automatically selects mps (Metal) for GPU acceleration.
Attention Slicing: Enabled via pipe.enable_attention_slicing() to reduce peak memory usage during generation.
Resolution: Defaults to 1920x1080 (customizable) or 1024x1024 (recommended). Note that higher resolutions use more RAM.
Local Caching: HF_HOME is set to ./hf_cache to avoid permission issues and keep the project self-contained.

Verification

The script has been verified to:

Download necessary components automatically.
Load the custom Lightning UNet correctly.
Execute the diffusion loop (4 steps).
Save images to ~/Documents/Image Generations/.

B{Check HF_HOME};

B --> C[Set Local Cache];
C --> D[Load Lightning UNet];
D --> E[Load Base Pipeline];
E --> F[Apply Optimizations];
F --> G[Generate Image];
G --> H[Save to Documents];

-->

2.8 KiB Raw Blame History