63 lines
2.8 KiB
Markdown
63 lines
2.8 KiB
Markdown
# SDXL-Lightning Image Generation for macOS (MPS)
|
|
|
|
We have successfully implemented a local image generation setup using SDXL-Lightning on macOS with Metal Performance Shaders (MPS) acceleration.
|
|
|
|
## Features Status
|
|
|
|
| Feature | Status | Notes |
|
|
| :--- | :--- | :--- |
|
|
| **Generation Script** | ✅ Complete | `generate.py` supports text-to-image with custom resolution |
|
|
| **Model** | ✅ Integrated | SDXL-Lightning 4-step UNet + SDXL Base Refiner components |
|
|
| **Optimization** | ✅ Applied | Attention slicing enabled for 16GB RAM compatibility |
|
|
| **Download** | ✅ Robust | Uses `hf_transfer` for fast, resilient model downloading |
|
|
| **Git** | ✅ Pushed | Project initialized and pushed to remote repository |
|
|
|
|
## Usage Guide
|
|
|
|
To generate an image, open your terminal and run:
|
|
|
|
```bash
|
|
# Activate virtual environment
|
|
source venv/bin/activate
|
|
|
|
# Run generation (first run will download models)
|
|
HF_HUB_ENABLE_HF_TRANSFER=1 python generate.py "A futuristic cityscape, cyberpunk style" --width 1024 --height 1024
|
|
```
|
|
|
|
> **Note**: The first run requires downloading ~5-6GB of model weights. We use `HF_HUB_ENABLE_HF_TRANSFER=1` to accelerate this process. Subsequent runs will be instant.
|
|
|
|
## Project Structure
|
|
|
|
- `generate.py`: Main script. Handles model loading, optimization, and generation.
|
|
- `models/`: Stores the SDXL-Lightning UNet checkpoint.
|
|
- `hf_cache/`: Stores the Diffusers cache (base model components).
|
|
- `requirements.txt`: Python dependencies.
|
|
|
|
## Optimizations Logic
|
|
|
|
To ensure smooth performance on macOS:
|
|
1. **MPS Device**: Automatically selects `mps` (Metal) for GPU acceleration.
|
|
2. **Attention Slicing**: Enabled via `pipe.enable_attention_slicing()` to reduce peak memory usage during generation.
|
|
3. **Resolution**: Defaults to `1920x1080` (customizable) or `1024x1024` (recommended). Note that higher resolutions use more RAM.
|
|
4. **Local Caching**: `HF_HOME` is set to `./hf_cache` to avoid permission issues and keep the project self-contained.
|
|
|
|
## Verification
|
|
|
|
The script has been verified to:
|
|
- Download necessary components automatically.
|
|
- Load the custom Lightning UNet correctly.
|
|
- Execute the diffusion loop (4 steps).
|
|
- Save images to `~/Documents/Image Generations/`.
|
|
|
|
|
|
<!-- above diagram text:
|
|
graph TD
|
|
A[Start Script] --> B{Check HF_HOME};
|
|
B --> C[Set Local Cache];
|
|
C --> D[Load Lightning UNet];
|
|
D --> E[Load Base Pipeline];
|
|
E --> F[Apply Optimizations];
|
|
F --> G[Generate Image];
|
|
G --> H[Save to Documents];
|
|
-->
|