Add project walkthrough

This commit is contained in:
Avery Felts 2026-01-26 01:54:34 -07:00
parent f330474208
commit 713bda3bfa

62
walkthrough.md Normal file
View File

@ -0,0 +1,62 @@
# SDXL-Lightning Image Generation for macOS (MPS)
We have successfully implemented a local image generation setup using SDXL-Lightning on macOS with Metal Performance Shaders (MPS) acceleration.
## Features Status
| Feature | Status | Notes |
| :--- | :--- | :--- |
| **Generation Script** | ✅ Complete | `generate.py` supports text-to-image with custom resolution |
| **Model** | ✅ Integrated | SDXL-Lightning 4-step UNet + SDXL Base Refiner components |
| **Optimization** | ✅ Applied | Attention slicing enabled for 16GB RAM compatibility |
| **Download** | ✅ Robust | Uses `hf_transfer` for fast, resilient model downloading |
| **Git** | ✅ Pushed | Project initialized and pushed to remote repository |
## Usage Guide
To generate an image, open your terminal and run:
```bash
# Activate virtual environment
source venv/bin/activate
# Run generation (first run will download models)
HF_HUB_ENABLE_HF_TRANSFER=1 python generate.py "A futuristic cityscape, cyberpunk style" --width 1024 --height 1024
```
> **Note**: The first run requires downloading ~5-6GB of model weights. We use `HF_HUB_ENABLE_HF_TRANSFER=1` to accelerate this process. Subsequent runs will be instant.
## Project Structure
- `generate.py`: Main script. Handles model loading, optimization, and generation.
- `models/`: Stores the SDXL-Lightning UNet checkpoint.
- `hf_cache/`: Stores the Diffusers cache (base model components).
- `requirements.txt`: Python dependencies.
## Optimizations Logic
To ensure smooth performance on macOS:
1. **MPS Device**: Automatically selects `mps` (Metal) for GPU acceleration.
2. **Attention Slicing**: Enabled via `pipe.enable_attention_slicing()` to reduce peak memory usage during generation.
3. **Resolution**: Defaults to `1920x1080` (customizable) or `1024x1024` (recommended). Note that higher resolutions use more RAM.
4. **Local Caching**: `HF_HOME` is set to `./hf_cache` to avoid permission issues and keep the project self-contained.
## Verification
The script has been verified to:
- Download necessary components automatically.
- Load the custom Lightning UNet correctly.
- Execute the diffusion loop (4 steps).
- Save images to `~/Documents/Image Generations/`.
![Workflow Diagram](https://mermaid.ink/img/pako:eNpVkE1rwzAMhv9KoFMO26XruWwH220wGEsvgx1CimNrqyNgyUlpKfvfJyljsN1EeT8eeR_0gDUWQQl9a6y8Yx8C3yObKe-94_w4HlE8j9Fqg1fbH8aA9-h8hU_Gk9nZ0-S8M_Z0PL9gz_Z0vFzhzXih12a4Q_8FfTigJ_uKfTmgZ_tS8HLFrxhe0A-HF_TjwSuKVyOvFl4zvGbcWrw2XC_x6uHdw3uA9wQfAT4SfCT4yPCR4aPAR4GPAl8Fvgp8Nfiq8dXgq8HXgG8A3xB-Q_hN4TeF3wy-OXxz-C3gt4LfCn4r-K3gt4LfCn4r-G_C7y9q1xK2)
<!-- above diagram text:
graph TD
A[Start Script] --> B{Check HF_HOME};
B --> C[Set Local Cache];
C --> D[Load Lightning UNet];
D --> E[Load Base Pipeline];
E --> F[Apply Optimizations];
F --> G[Generate Image];
G --> H[Save to Documents];
-->