MLX-Diffusion On Mac

MLX-Diffusion : Yet another open source image generator on Apple Silicon

MLX-DIFFUSION • SOVEREIGN LOCAL GENERATIVE STUDIO

Unleashing MLX-Diffusion on Apple Silicon

👤 By Ouinche 💻 Hardware: Mac M1 16GB Unified Memory 🚀 Dual-Engine: MLX Diffuser & mflux 🔓 License: Open-Source on GitHub

Don’t pay to generate censored image online ! Even with an 5 year old MAcBook M1 16Gb

What if you could take the absolute bleeding-edge of open-weights AI—massive 9-billion and 13-billion parameter models—and run them directly on the Mac sitting in front of you? Fast, free, uncensored, and private.

Today, we introduce MLX-Diffusion: an open-source generative studio built for hobbyists and researchers that turns your Apple Silicon Mac into a sovereign AI creative laboratory.

1. Interface & Technical Analysis: What It Does — And What It Does NOT Do

Before diving into empirical performance metrics, it is vital to understand the architectural foundation powering MLX-Diffusion.

What the Application Does (Features & Architecture)

  • Dual-Engine Local Generation: Powered natively on Apple Silicon by a robust dual-engine stack utilizing MLX Diffuser (for SDXL architectures) and mflux (for Flow-matching DiTs and Wan2.1). Both engines compile directly into Metal Performance Shaders, eliminating bloated runtime abstraction layers.
  • Intuitive, UnProfessional UI: A sleek, dark-themed interface resembling professional creative software, divided into an ergonomic parameter panel and a dedicated « Studio Canvas » equipped with built-in High-Resolution Visual Comparators and real-time generation previews.
  • SOTA Architecture Support: Flawlessly runs standard SDXL checkpoints alongside cutting-edge open-weight architectures like Flow-matching DiTs (FLUX.2-klein 4B & 9B, Z-Image Turbo 6B) and Wan2.1 (Krea 2 Turbo 13B) and Qwen-Image-2.1 ( ok only in 768×512 and Q4 quantized but Qwen-Image-2.1 at all.
  • Absolute Data Sovereignty (« Stealth Mode »): A core privacy feature: the Stealth Mode (No generation metadata) toggle. When active, it guarantees that absolutely zero generation metadata (prompts, seeds, CFG, models) is embedded into the exported image file. Your thoughts and creative recipes never leave your local drive.
  • Automatic Civitai Metadata Formatting: When sharing is desired, MLX-Diffusion automatically formats lossless PNG text chunks and EXIF tags to strictly comply with Civitai and Automatic1111 standards, enabling effortless drag-and-drop ingestion into Civitai galleries.
  • Frictionless Hobbyist Launch: Open-source on GitHub with zero python environment hell. No broken Conda environments or compiling C++ dependencies; simply clone and launch via a single bash command: ./Run.sh. ( NDLR : KEK ! See below )
  • Comprehensive Workflow Tools: Built-in multi-LoRA manager with auto-unloading on model swaps, upscaling suite featuring AI Neural 2x (SeedVR2 latent model) and Fast 4x Lanczos, multi-image reference inputs, and a fast local image browser.
  • Granular Parameter Control: Full interactive control over model checkpoints, image dimensions (portrait, square, landscape), step trajectories, Guidance scale (CFG), seed locking/randomization, and batch scheduling.
  • Deterministic Memory Clamping: Active denoise wired-memory clamping (limiting denoise allocations to 6.5–7GB), lazy pipeline unloader, and compiled Metal VAE decoders prevent macOS SSD swap thrashing even when generating at high resolutions.

What the Application Does NOT Do

  • No Cloud Processing: Relies 100% on your local Apple Silicon hardware. Zero cloud servers, zero background analytics, and zero monthly subscriptions.
  • Zero Content Moderation / Censorship: Completely offline. There are no hidden API safety filters, no blocked vocabulary, and no centralized gatekeepers policing your creative workflow.
  • Not an unoptimized PyTorch MPS wrapper: Standard PyTorch on macOS (torch.device("mps")) frequently leaks memory and misses custom fused Metal kernels. MLX-Diffusion executes native MLX array operations.
  • Not an unrealistic claim: A 16GB Mac cannot run 70B unquantized FP16 models. MLX-Diffusion achieves fluid local generation through disciplined 4-bit fused quantization and calibrated step trajectories.

2. The Grand SOTA Benchmark (Apple Silicon M1 16GB)

To establish empirical truths, the Apple Silicon Laboratory benchmarked ten state-of-the-art model configurations across ten canonical test prompts representing diverse visual challenges (environmental portraiture, Scandinavian landscapes, wildlife macro textures, glass architecture, and night urban lighting) at 512×768 resolution under fixed seeds (1001–1010) — with the brand-new Qwen-Image 2.1 (7B) matching the same 512×768 canvas as every other configuration (Qwen’s 64-channel RGBA VAE handled it without OOM; only 1024² exceeds the 16GB M1’s bf16 VAE decode headroom). All SDXL checkpoints and distilled LoRA adapters tested are available on Civitai.

# Model Configuration Engine & Architecture Steps CFG Avg Gen Time Speed Core Architectural Strength
1 Juggernaut XL Lightning SDXL MLX 4 1.0 9.73s 2.43 s/step Rapid prototyping (~10s), balanced composition. Available on Civitai.
2 RealVisXL V5.0 Lightning SDXL MLX 6 1.5 26.97s 4.50 s/step Skin micro-textures and soft lighting in under 27s. Available on Civitai.
3 RealVisXL V5.0 + Hyper-SD SDXL MLX 8 2.0 41.64s 5.21 s/step Deep textural density on full base UNet via Civitai.
4 Juggernaut XI v11 + Hyper-SD SDXL MLX 8 2.0 42.93s 5.37 s/step Surgical precision on metals, gears, and dynamic range on Civitai.
5 FLUX.2-klein 4B mflux (FlowMatch) 4 1.0 53.28s 13.32 s/step Optical physics, glass refraction, zero negative prompts.
6 FLUX.2-klein 9B mflux (FlowMatch) 4 1.0 107.04s 26.76 s/step Museum-grade 9B DiT, 100% local on 16GB memory.
7 Z-Image Turbo 6B mflux (FlowMatch) 8 1.0 170.44s 21.31 s/step Sharp geometric perspective and architectural alignment.
8 Krea 2 Turbo 13B (Distill) Wan2.1 + LoRA 4 1.0 250.83s 62.71 s/step Rich tactile surface rendering, cinematic warm lighting.
9 Krea 2 Turbo 13B (Native) Wan2.1 Native 8 1.0 368.91s 46.11 s/step Uncompressed trajectory with maximum specular precision.
10 Qwen-Image 2.1 mflux Qwen (q4) 20 1.0 457.8s 22.9 s/step Compact 7B single-stream DiT, Qwen3-VL text encoder, RGBA VAE — coherent alignment across all ten scenes at 512×768.

⚡ High-Resolution Visual Comparator: SOTA Arena (Cross-Model)

Welcome to the Grand SOTA Arena. Select any benchmark prompt from the buttons below, then choose any two models from the Left and Right dropdowns to compare them directly. Drag the center slider to inspect fine micro-textures, specular highlights, and optical physics side-by-side. You can also switch to DeepCache Lab to inspect 4-step vs 12-step caching.

⚡ High-Resolution Visual Comparator
Benchmark Scene Seed: 1001
Loading scene prompt…
Duel: Model Comparison Direct Comparison
Right Image Right Left Image Left
⮜ ⮞

💡 Interactive Grand Arena: Click any scene pill above to switch prompts. Choose any two models from the LEFT and RIGHT dropdowns to compare them head-to-head. Drag the divider to inspect differences at full resolution.

3. Open Source & Quick Start

MLX-Diffusion is free and open-source (MIT) on GitHub (github.com/OuincheWinch/MLX-Diffusion) — every benchmark above was generated locally with it. Requirements: an Apple Silicon Mac (M1–M4, 16GB+ unified memory recommended), macOS, Node.js 18+, and ~2–3 GB free disk for the first model download (one-time, into the Hugging Face cache).

0. Download

git clone https://github.com/OuincheWinch/MLX-Diffusion.git

1. Install

From the repo root, create the two Python virtual environments (both are required — the main engine venv/ and the isolated SDXL engine venv-sdxl/):

cd MLX-Diffusion
python3 -m venv venv
python3 -m venv venv-sdxl

Install the Python dependencies into each venv:

./venv/bin/python -m pip install -r backend/requirements.txt
./venv-sdxl/bin/python -m pip install -r backend/requirements-sdxl.txt

Install the frontend dependencies:

cd frontend && npm install && cd ..

2. Run

Launch MLX-Diffusion (backend port 8001, frontend port 5174, browser opens automatically). Press CTRL+C in the terminal to close it:

./run.sh

run.sh uses ./venv/bin/python -m uvicorn main:app and keeps the Mac awake with caffeinate during long renders. First generation downloads the model weights once (~2–3 GB, allow ~20 min); afterwards it runs 100% offline.

3. Your first prompt

Suggested starting prompt with FLUX.2-klein 4B, Z-Image Turbo 6B or Krea 2 Turbo 13B (4 steps for FLUX/Krea-distill, 8 steps for Z-Image):

A mischievous baby otter wearing a tiny yellow developer helmet, sitting in front of a futuristic glowing computer setup. The glowing computer screen clearly displays the words « HELLO WORLD » in vibrant neon text. Warm studio lighting, shallow depth of field, 8k resolution, cinematic photorealism. [Image 1]

HELLOWORLD render

In-context reference ([Image 1]): the tag at the end conditions the generation on a reference image, but you must load that image into the reference tray first, and only FLUX.2-klein 4B accepts image input. On Z-Image, Krea 2 or SDXL the studio refuses the prompt with: Cannot read "HELLOWORLD.png" (this model does not support image input). Either switch to FLUX.2-klein 4B or remove the [Image N] tag.

4. Uninstall (full removal) when needed

Ensure your terminal is open inside the project folder you wish to remove (check the path twice!):

cd /path/to/your/MLX-Diffusion
cd .. && rm -rf MLX-Diffusion

Purge the leftover caches so nothing lingers on the machine:

# purge pip cache (clears downloaded wheels and packages)
python3 -m pip cache purge

# clear npm global cache
npm cache clean –force

5. Development mode (2 terminals)

./dev-backend.sh  # FastAPI backend, port 8001, hot reload
cd frontend && npm run dev  # Vite frontend, port 5174, hot reload

Free stuck services: lsof -ti :8001,5174 | xargs kill -9 — ports are fixed at 8001 / 5174.

4. Tailored Prompting Rules by Architecture

Prompt engineering is not one-size-fits-all. Each architecture responds to distinct syntactic and stylistic conventions:

  • SDXL Models (Juggernaut XL, RealVisXL): Respond best to optical framing tags (85mm f/1.4, Hasselblad portrait, soft studio rim light) paired with an explicit negative prompt (ugly, deformed, blurry, bad anatomy) to prune unwanted latent modes.
  • FLUX.2-klein (4B & 9B): Requires continuous descriptive prose. Negative prompts are mathematically unsupported in guidance-distilled flow matching; describe textures, reflections, physical interactions, and lighting geometry explicitly.
  • Krea 2 Turbo (Wan2.1): Thrives on concrete tactile nouns (e.g., coarse fur, brushed steel, wet asphalt, warm sunset sidelight) and cinematic depth cues.
  • Z-Image Turbo: Excels when horizon lines, geometric vanishing points, and structural object relationships are articulated cleanly in structured sentences.
  • Qwen-Image 2.1: The compact 7B single-stream DiT (Qwen3-VL text encoder, 64-channel RGBA VAE) prefers the same continuous natural prose as FLUX flow-matching — explicit negative prompts are supported (guidance auto-raises to 3.0 when used) and its 20-step linear sampling delivers the full compositional spectrum without collapse.

5. Architectural Conclusion & Sovereign AI

On an Apple Silicon Mac equipped with 16GB of unified memory, local generative diffusion has graduated from experimental toy status into a reliable, uncensored, sovereign creative workstation.

  • For Ideation & High Throughput: Juggernaut XL Lightning (~10s per generation) is the undisputed speed benchmark. Grab weights on Civitai.
  • For Organic Portraiture: RealVisXL V5.0 Lightning (27s at 6 steps) provides industry-grade skin texture and expressive depth on Civitai.
  • For Complex Optics & Global Coherence: FLUX.2-klein 4B and 9B (53s to 107s) deliver museum-grade physical light transport without negative prompt tuning.
  • For Cinematic Tactile Renders: Krea 2 Turbo 13B (Wan2.1 architecture) provides deep shadows, warm grading, and exceptional atmospheric volume.
Reclaim Your Creative Sovereignty: Stop renting AI cycles and submitting your imagination to third-party cloud filters. With MLX-Diffusion — free and open-source at github.com/OuincheWinch/MLX-Diffusion — launch via ./Run.sh and experience 100% private, uncensored generation.

Explore thousands of community checkpoints and LoRAs on Civitai. All benchmark generations produced locally using MLX-Diffusion on Apple Silicon.

Leave a Reply

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

 

Ce site utilise Akismet pour réduire les indésirables. Découvrez comment les données de vos commentaires sont traitées.