From 713bda3bfaacf5148b7d1a72a8bda27d514f4964 Mon Sep 17 00:00:00 2001 From: Avery Felts Date: Mon, 26 Jan 2026 01:54:34 -0700 Subject: [PATCH] Add project walkthrough --- walkthrough.md | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 walkthrough.md diff --git a/walkthrough.md b/walkthrough.md new file mode 100644 index 0000000..b712907 --- /dev/null +++ b/walkthrough.md @@ -0,0 +1,62 @@ +# SDXL-Lightning Image Generation for macOS (MPS) + +We have successfully implemented a local image generation setup using SDXL-Lightning on macOS with Metal Performance Shaders (MPS) acceleration. + +## Features Status + +| Feature | Status | Notes | +| :--- | :--- | :--- | +| **Generation Script** | ✅ Complete | `generate.py` supports text-to-image with custom resolution | +| **Model** | ✅ Integrated | SDXL-Lightning 4-step UNet + SDXL Base Refiner components | +| **Optimization** | ✅ Applied | Attention slicing enabled for 16GB RAM compatibility | +| **Download** | ✅ Robust | Uses `hf_transfer` for fast, resilient model downloading | +| **Git** | ✅ Pushed | Project initialized and pushed to remote repository | + +## Usage Guide + +To generate an image, open your terminal and run: + +```bash +# Activate virtual environment +source venv/bin/activate + +# Run generation (first run will download models) +HF_HUB_ENABLE_HF_TRANSFER=1 python generate.py "A futuristic cityscape, cyberpunk style" --width 1024 --height 1024 +``` + +> **Note**: The first run requires downloading ~5-6GB of model weights. We use `HF_HUB_ENABLE_HF_TRANSFER=1` to accelerate this process. Subsequent runs will be instant. + +## Project Structure + +- `generate.py`: Main script. Handles model loading, optimization, and generation. +- `models/`: Stores the SDXL-Lightning UNet checkpoint. +- `hf_cache/`: Stores the Diffusers cache (base model components). +- `requirements.txt`: Python dependencies. + +## Optimizations Logic + +To ensure smooth performance on macOS: +1. **MPS Device**: Automatically selects `mps` (Metal) for GPU acceleration. +2. **Attention Slicing**: Enabled via `pipe.enable_attention_slicing()` to reduce peak memory usage during generation. +3. **Resolution**: Defaults to `1920x1080` (customizable) or `1024x1024` (recommended). Note that higher resolutions use more RAM. +4. **Local Caching**: `HF_HOME` is set to `./hf_cache` to avoid permission issues and keep the project self-contained. + +## Verification + +The script has been verified to: +- Download necessary components automatically. +- Load the custom Lightning UNet correctly. +- Execute the diffusion loop (4 steps). +- Save images to `~/Documents/Image Generations/`. + +![Workflow Diagram](https://mermaid.ink/img/pako:eNpVkE1rwzAMhv9KoFMO26XruWwH220wGEsvgx1CimNrqyNgyUlpKfvfJyljsN1EeT8eeR_0gDUWQQl9a6y8Yx8C3yObKe-94_w4HlE8j9Fqg1fbH8aA9-h8hU_Gk9nZ0-S8M_Z0PL9gz_Z0vFzhzXih12a4Q_8FfTigJ_uKfTmgZ_tS8HLFrxhe0A-HF_TjwSuKVyOvFl4zvGbcWrw2XC_x6uHdw3uA9wQfAT4SfCT4yPCR4aPAR4GPAl8Fvgp8Nfiq8dXgq8HXgG8A3xB-Q_hN4TeF3wy-OXxz-C3gt4LfCn4r-K3gt4LfCn4r-G_C7y9q1xK2) + B{Check HF_HOME}; + B --> C[Set Local Cache]; + C --> D[Load Lightning UNet]; + D --> E[Load Base Pipeline]; + E --> F[Apply Optimizations]; + F --> G[Generate Image]; + G --> H[Save to Documents]; +-->