SDXL-Lightning Image Generator for macOS

This project runs Stable Diffusion XL Lightning (4-step) natively on your Mac using Apple Silicon (M1/M2/M3) acceleration (MPS). It is optimized to generate high-quality 1024x1024 images in seconds with minimal setup.

Features

  • Blazing Fast: Uses SDXL-Lightning 4-step UNet for rapid generation (approx. ~10-30s per image on M1/M2).
  • Native Mac Support: Leverage your Mac's GPU with Metal Performance Shaders (MPS).
  • Memory Optimized: Automatic CPU offloading to run even on 8GB/16GB Macs without crashing.
  • Local Privacy: All models run locally on your machine. No cloud API keys needed.
  • Auto-Download: Automatically fetches required model weights on first run.

🚀 Prerequisites

  • macOS 12.3+ (Monterey or newer)
  • Mac with Apple Silicon (M1, M2, M3)
  • Python 3.9 or newer installed (check with python3 --version)

🛠️ Installation & Setup

  1. Clone the repository (if you haven't already):

    git clone <your-repo-url>
    cd "Image Generation"
    
  2. Create a virtual environment to keep dependencies clean:

    python3 -m venv venv
    source venv/bin/activate
    
  3. Install dependencies:

    pip install -r requirements.txt
    

    (This installs torch, diffusers, transformers, and accelerate optimized for Mac)

🎨 Usage

Basic Generation

Generate an image with a simple text prompt. The first time you run this, it will download the necessary models (~6GB).

# Make sure your virtual environment is active!
source venv/bin/activate

# Run the generator
python generate.py "A futuristic cityscape at sunset, highly detailed, cyberpunk style, neon lights"

Image-to-Image

Transform an existing image based on your prompt.

python generate.py "A cyberpunk version of this photo" --image "path/to/my_photo.jpg" --strength 0.75
  • --image: Path to your input image (JPG/PNG).
  • --strength: How much to change the image (0.0 to 1.0).
    • 0.3: Subtle changes
    • 0.75: Default, balanced mix
    • 1.0: Completely new image

Advanced Options

You can customize the resolution and quality settings:

python generate.py "An astronaut riding a horse on mars, realistic, 8k" --width 1024 --height 1024 --steps 4
Flag Default Description
prompt (Required) The description of the image you want to generate.
--width 1920 Width of the image. Standard SDXL is optimized for 1024.
--height 1080 Height of the image.
--steps 4 Number of inference steps. 4-8 is recommended for Lightning.

Output Location:
Images are saved automatically to the output/ folder in this directory.

First Run Note

The first time you run the script, it will download:

  1. SDXL Base Model: ~6GB (Cached in hf_cache/)
  2. Lightning UNet: ~5GB (Saved in models/)

Use a fast internet connection! Subsequent runs will be instant.

🔧 Troubleshooting

  • "MPS backend out of memory":
    This means your Mac ran out of GPU memory. The script includes pipe.enable_model_cpu_offload() to prevent this. try restarting your computer or closing other heavy apps.
  • "Permission denied" errors:
    You might see "mpsgraph" permission errors in the terminal. These are harmless warnings from macOS's Metal framework and can be ignored. The image generation will still work.
  • Slow First Generation:
    Shader compilation happens on the very first run. Future generations will be much faster.

🤝 Contributing

Feel free to open issues or submit PRs to improve performance or add features!

Description
Allows for local machine generation rather than online image generation (almost never available due to full server space). This implementation is meant to work with MacOs, windows is not tested.
Readme 36 KiB
Languages
Python 100%