Abstract

FLUX.1 is a diffusion-based text-to-image generation model developed by Black Forest Labs, designed to achieve faithful text-image alignment while maintaining high image quality and diversity. FLUX is considered state-of-the-art in text-to-image generation, outperforming popular models such as Midjourney, DALL·E 3, Stable Diffusion 3 (SD3), and SDXL. Although publicly available as open source, the authors have not released official technical documentation detailing the model’s architecture or training setup. This report summarizes an extensive reverse-engineering effort aimed at demystifying FLUX’s architecture directly from its source code, to support its adoption as a backbone for future research and development.

ML-Based Image Generation

Image generation, in the machine-learning sense, means learning to sample novel images from a distribution that mimics some source dataset. A common strategy is to learn a mapping from the complex data distribution (natural images) to something simple and easy to sample from, typically an isotropic Gaussian — and then invert that mapping at generation time.

Several major families implement this idea:

• Variational Autoencoders (VAEs) — learn an encoder/decoder pair with a regularized latent space.
• Generative Adversarial Networks (GANs) — pit a generator against a discriminator in an adversarial game.
• Normalizing Flows (NFs) — learn an invertible, exact-likelihood transformation between data and noise.
• Diffusion Models (DMs) — learn to reverse a gradual noising process.

Among these families, text-to-image models — DALL·E, Stable Diffusion, and FLUX among them — have become the dominant paradigm for producing high-quality images from natural-language prompts.

Diffusion Models

Diffusion Models (DMs) are today’s state-of-the-art approach for text-conditioned image generation. Training corrupts clean images with Gaussian noise according to a timestep-dependent schedule, and the network learns to undo that corruption. Early diffusion models used U-Net backbones — convolutional layers augmented with attention for image–text alignment — but newer models such as SD3 and FLUX.1 have moved to fully Transformer-based denoisers, trading convolutional inductive bias for scalability and better long-range context modeling.

In the most common formulation, the network is trained to predict the (normalized) noise \(\epsilon\) that was added to a clean sample, using a simple reconstruction loss:

\[ \mathcal{L}_{\epsilon} = \mathbb{E}_{x_t, \epsilon, t}\Big[\|\epsilon_\theta(x_t, t) - \epsilon\|^2\Big] \tag{1}\]

where the noisy sample at timestep \(t\) is

\[ x_t = \sqrt{\alpha_t}\, x + \sqrt{1-\alpha_t}\, \epsilon,\qquad \epsilon \sim \mathcal{N}(0,1) \]

and \(x_0\) denotes the clean image. At inference time, the model starts from pure Gaussian noise and iteratively denoises it into a realistic image (Equation 1). Compared to GANs’ adversarial objective, this reconstruction-based training is more stable and scales better to high resolutions, while also enabling fine-grained control for tasks like super-resolution, inpainting, and text-conditioned synthesis.

Stable Diffusion

Stable Diffusion is a family of Latent Diffusion Models (LDMs): rather than diffusing directly in pixel space, images are first compressed into a lower-dimensional latent space by a pretrained autoencoder, the diffusion process runs in that latent space, and a decoder maps the final latent back to pixels. This dramatically reduces compute cost while preserving visual fidelity.

The lineage, as covered in the paper:

• SD 1.x (1.0–1.5): frozen CLIP text encoder + U-Net latent denoiser, trained on LAION-2B subsets. Open-source release drove rapid adoption and fine-tuning.
• SD 2.1: switched to OpenCLIP, trained at higher resolution (768×768), improving structure and fidelity.
• SDXL: two-stage base + refiner architecture, better handling of complex prompts, closer to commercial-grade realism.
• SD-Turbo: near real-time generation via Adversarial Diffusion Distillation (ADD) — a student model distilled from a teacher (SD2.1 or SDXL) to require far fewer steps.
• SD3: adopts a diffusion-transformer backbone and multimodal training, integrating text and image understanding jointly — a direct architectural precursor to FLUX (see Section 3.3).

Rectified Flows

Rectified Flow (RF) is the training paradigm FLUX actually uses instead of standard DM noise-prediction. It’s a training scheme, not an architectural change, but it matters here because it shapes what the FLUX transformer is trained to predict. RF builds on the Flow Matching framework and replaces stochastic denoising with a deterministic vector field.

Standard diffusion models train \(\epsilon_\theta\) to predict the noise added to a sample (Equation 1). Rectified Flow instead trains a model \(v_\theta\) to predict the velocity — a vector pointing from a noise sample toward a data sample along a straight interpolation path:

\[ \mathcal{L}_v = \mathbb{E}_{x_0, x_1, t}\Big[\|v_\theta(x_t, t) - (x_1 - x_0)\|^2\Big] \tag{2}\]

where

\[ x_t = (1-t)\,x_0 + t\,x_1,\qquad x_0 \sim \mathcal{N}(0, I),\quad x_1 \sim p_{\text{data}} \]

By defining a deterministic transport path from noise to data, RF avoids explicit noise schedules and stochastic sampling, solving instead a simple ODE with a learned velocity field — generally faster and more stable to train and sample from than classic diffusion.

Transformers

Transformers, originally built for NLP, introduced the self-attention mechanism: a way for every element of a sequence to weigh its relationship to every other element. This has become the substrate for state-of-the-art models across text, image, video, and multimodal domains.

Each attention block projects the input embeddings into query (\(Q\)), key (\(K\)), and value (\(V\)) matrices via learned linear layers, then computes:

\[ \text{Attention}(Q, K, V) = \text{softmax}\!\left(\frac{QK^{T}}{\sqrt{d_k}}\right)\cdot V \tag{3}\]

where \(d_k\) is the key dimensionality and the softmax normalizes attention weights to sum to 1 (Equation 3). In practice, multi-head attention runs several \(Q\)/\(K\)/\(V\) projections in parallel so the model can capture different types of relationships simultaneously.

For vision, Vision Transformers (ViTs) tokenize an image into fixed-size, non-overlapping patches (e.g., 16×16 pixels), flatten and linearly project each patch, then feed the resulting sequence — with positional encodings added — through a standard transformer encoder. This lets ViTs model long-range dependencies without a convolutional inductive bias.

Two attention regimes matter for what follows:

• Self-Attention: \(Q\), \(K\), and \(V\) are all projected from the same input embedding.
• Cross-Attention: used in text-to-image generation to condition image synthesis on a prompt — \(Q\) comes from image tokens, while \(K\) and \(V\) come from text embeddings.

FLUX, as we’ll see, uses a joint scheme that blends both ideas: a Multi-Modal Attention where text and image tokens are concatenated and attend to each other simultaneously.

Introduction

FLUX.1 is a rectified-flow transformer trained in the latent space of an image autoencoder, released by Black Forest Labs in August 2024. The FLUX.1 family sets a new state-of-the-art for text-to-image tasks on both output quality and image–text alignment, as measured by ELO-score human-preference comparisons against Midjourney, DALL·E 3, SD3, and others across dimensions like prompt following, size/aspect variability, typography, output diversity, and visual quality. These ELO comparisons are bar-chart evaluations reproduced from the official Black Forest Labs announcement — not architecture diagrams — so they are summarized here in prose rather than redrawn.

While Black Forest Labs states FLUX follows the Rectified Flow training paradigm, the actual training setup — dataset, noise/timestep scheduling, hyperparameters — has never been publicly disclosed. What can be reverse-engineered is the model’s architecture and inference-time behavior, directly from the public inference code (the diffusers implementation). That’s the paper’s core contribution, and this guide follows the same top-down path: first a bird’s-eye view of the architecture, then the sampling pipeline, then a deep dive into the transformer itself.

flowchart LR
    P["Text Prompt"] --> TE["Text Encoders (CLIP + T5)"]
    TE --> EMB["Text Embeddings"]
    N["Gaussian Noise z0"] --> LAT["Latent Image Tokens"]
    EMB --> TB["FLUX Transformer Blocks (Double-Stream + Single-Stream)"]
    LAT --> TB
    TB -->|"repeat across sampling timesteps"| TB
    TB --> DEC["Pretrained VAE Decoder"]
    DEC --> IMG["Generated RGB Image"]

Figure 1 gives the top-level view: iterative refinement of latent + text tokens through transformer blocks, followed by VAE decoding.

FLUX.1 Sampling Pipeline

For simplicity, the paper (and this guide) describes sampling conditioned on a single prompt at a time. Like LDM, FLUX operates in a latent space: the final latent is decoded back into an RGB image. Following LDM’s approach, Black Forest Labs trained a convolutional autoencoder from scratch with an adversarial objective, but scaled the latent channel count from 4 (LDM) up to 16.

The sampling pipeline has three phases:

1. Initiation and Pre-processing
2. Iterative Refinement
3. Post-processing

The notation follows the official diffusers implementation of the FLUX.1 pipeline.

flowchart TD
    subgraph INIT["Initiation and Pre-processing"]
        TXT["text"] --> T5["T5 Encoder"]
        T5 --> EHS["encoder_hidden_states (dense, per-token)"]
        TXT --> CLIP["CLIP Encoder"]
        CLIP --> PP["pooled_projection (single vector)"]
        NIS["num_inference_steps"] --> TS["timestep subset, t in 0..T, T approx 1000"]
        RES["resolution H,W"] --> LATSHAPE["latent shape h,w = H,W // VAE_scale(8)"]
        LATSHAPE --> Z0["z0 ~ N(0,1)"]
        LATSHAPE --> IDS["img_ids per token; text_ids = n*(0,0,0)"]
        GS["guidance_scale"] --> GU["guidance"]
    end

    subgraph ITER["Iterative Refinement"]
        LOOP["for t in timesteps: z_(t+dt) = Samp(v_theta(z_t, t, Phi))"]
    end

    EHS --> LOOP
    PP --> LOOP
    GU --> LOOP
    Z0 --> LOOP
    IDS --> LOOP
    TS --> LOOP
    LOOP -->|"iterate t: 0 to 1"| LOOP
    LOOP --> Z1["z1 (clean latent)"]

    subgraph POST["Post-processing"]
        Z1 --> VAE["Pretrained VAE Decoder"]
        VAE --> IMG["RGB Image x1"]
    end

Transformer

The core of FLUX.1’s pipeline is the velocity predictor \(v_\theta\), optimized to estimate the velocity vector along the sampling trajectory. Like SD3, FLUX.1 replaces the conventional U-Net with a fully transformer-based design.

On every sampling iteration, inputs are preprocessed, then passed through a sequence of transformer blocks of two types:

1. Double-Stream blocks — separate weights for text and image tokens.
2. Single-Stream blocks — shared weights across both modalities.

Both block types use Multi-Modal Attention: a joint self-attention over concatenated text and image tokens, enabling bidirectional interaction that captures both within-modality and cross-modality information.

flowchart TD
    A["Per-Iteration Pre-process (build temb + pos_embeds)"] --> B["19 x Double-Stream Blocks (separate weights per modality)"]
    B --> C["Concatenate image + text token streams"]
    C --> D["Single-Stream Blocks (shared weights, parallel attn+MLP)"]
    D --> E["Velocity prediction v_theta(z_t, t, Phi)"]

flowchart TD
    HS["hidden_states (z_t)"] --> LP1["Linear Projection to 3072-dim"]
    EHS["encoder_hidden_states (T5)"] --> LP2["Linear Projection to 3072-dim"]

    subgraph TEMB["temb construction (red block)"]
        T["timestep"] --> SIN1["Sinusoidal Embedding"] --> LIN1["Linear Layer"]
        G["guidance"] --> SIN2["Sinusoidal Embedding"] --> LIN2["Linear Layer"]
        PP["pooled_projection (CLIP)"] --> LIN3["Linear Layer"]
        LIN1 --> SUM["Sum"]
        LIN2 --> SUM
        LIN3 --> SUM
        SUM --> TEMBOUT["temb (shared 3072-dim)"]
    end

    subgraph POSEMB["pos_embeds construction (purple block)"]
        IMGIDS["img_ids"] --> CAT["Concatenate ids"]
        TXTIDS["text_ids"] --> CAT
        CAT --> AXIS["Per-axis sinusoidal frequency times theta"]
        AXIS --> COSSIN["cos / sin components, interleaved"]
        COSSIN --> POSOUT["pos_embeds (RoPE-ready cos/sin tensors)"]
    end

    LP1 --> OUT["To Double-Stream Blocks"]
    LP2 --> OUT
    TEMBOUT --> OUT
    POSOUT --> OUT

flowchart TD
    subgraph LATENT["Latent Stream"]
        L0["Latent tokens"] --> LADALN["AdaLN (modulated by temb)"]
        LADALN --> LQKV["Q, K, V (image)"]
    end
    subgraph PROMPT["Prompt Stream"]
        P0["Text tokens"] --> PADALN["AdaLN (modulated by temb)"]
        PADALN --> PQKV["Q, K, V (text)"]
    end

    LQKV --> CATQKV["Concatenate Q, K, V across streams"]
    PQKV --> CATQKV
    CATQKV --> ROPE["Apply RoPE to concatenated Q, K"]
    ROPE --> ATTN["Joint Multi-Modal Attention"]
    ATTN --> SPLIT["Split output back into streams"]

    SPLIT --> GMSA_L["gate_msa (latent, from AdaLN)"]
    SPLIT --> GMSA_P["gate_msa (prompt, from AdaLN)"]
    GMSA_L --> ADD1L["Residual add"]
    GMSA_P --> ADD1P["Residual add"]

    ADD1L --> LN_L["LayerNorm"]
    ADD1P --> LN_P["LayerNorm"]
    LN_L --> MOD_L["scale_mlp / shift_mlp (latent, from AdaLN)"]
    LN_P --> MOD_P["scale_mlp / shift_mlp (prompt, from AdaLN)"]
    MOD_L --> MLP_L["Feed-Forward MLP"]
    MOD_P --> MLP_P["Feed-Forward MLP"]

    MLP_L --> GMLP_L["gate_mlp (latent)"] --> ADD2L["Residual add"] --> OUTL["Latent stream output"]
    MLP_P --> GMLP_P["gate_mlp (prompt)"] --> ADD2P["Residual add"] --> OUTP["Prompt stream output"]

flowchart LR
    IMGQKV["Image Q, K, V"] --> CAT["Concatenated token sequence"]
    TXTQKV["Text Q, K, V"] --> CAT
    CAT --> ATTN["softmax(QK^T / sqrt(d_k)) . V over the joint sequence"]
    ATTN --> SPLIT["Split into image / text outputs"]

flowchart TD
    IN["Input tensor"] --> NORM["LayerNorm (no learned affine params)"]
    COND["Conditioning vector (temb)"] --> LINP["Linear Projection"]
    LINP --> PARAMS["scale_msa, shift_msa, gate_msa, scale_mlp, shift_mlp, gate_mlp"]
    NORM --> MOD["Modulate: norm times (1 + scale) + shift"]
    PARAMS --> MOD
    MOD --> OUT["Modulated activations to MSA / MLP"]

flowchart TD
    IN["Concatenated latent + text tokens"] --> ADALN["Shared AdaLN"]
    ADALN --> QKV["Shared Q, K, V projection"]
    ADALN --> MLPIN["Same normalized input"]
    QKV --> ROPE["Apply RoPE to Q, K"]
    ROPE --> ATTN["Joint self-attention"]
    MLPIN --> MLP["Feed-Forward MLP (parallel)"]
    ATTN --> COMBINE["Combine attention + MLP outputs"]
    MLP --> COMBINE
    COMBINE --> GATE["gate (from AdaLN)"] --> ADD["Residual add"] --> OUT["Output tokens (latent + text)"]

flowchart LR
    CATIN["Concatenated latent + text representation"] --> QP["Q projection"]
    CATIN --> KP["K projection"]
    CATIN --> VP["V projection"]
    QP --> ATTN["Attention"]
    KP --> ATTN
    VP --> ATTN

Text-to-Image Variants

All FLUX.1 text-to-image variants share the same 12B-parameter architecture, differing in training/distillation strategy, licensing, and deployment:

• FLUX.1 [pro] — the highest-performing variant: best prompt alignment, visual detail, and output diversity. Commercial-only, hosted via licensed API endpoints (Replicate, Fal.ai); weights are not released.
• FLUX.1 [dev] — an open-weight, guidance-distilled alternative to Pro. Licensed strictly for non-commercial use (research/hobbyist). Downloadable and runnable locally via Hugging Face; may need more sampling steps than Pro to match its quality.
• FLUX.1 [schnell] — a speed-optimized variant distilled from Pro via timestep-distillation, able to generate in as few as a single step. Released under Apache 2.0 (unrestricted commercial use), trading some prompt fidelity/detail for latency — ideal for real-time or lightweight deployments.
• FLUX1.1 [pro] — an enhanced Pro variant: faster generation plus improved quality, prompt adherence, and diversity. Adds an Ultra mode (4× higher resolution at similar speed) and a Raw mode (hyper-realistic, candid-style images), plus LLM-based prompt upsampling. Not covered in architectural depth by this report; commercial licensing required via Replicate/Fal.ai.

Tools

Beyond text-to-image, Black Forest Labs has extended FLUX.1 with additional tools (current as of June 5, 2025; not explored in architectural depth in the source report):

• FLUX-Fill — inpainting/outpainting given a text description and a binary mask.
• FLUX-Canny — structural guidance from Canny edges extracted from an input image, combined with a text prompt.
• FLUX-Depth — structural guidance from a depth map extracted from an input image, combined with a text prompt.
• FLUX-Redux — an adaptor aligning dense per-token image embeddings (from a SigLIP image encoder) with T5’s text-embedding space, enabling image-conditioned generation from a pretrained FLUX.1 base. Limited to image variation in the Dev version, but unlocks prompt-driven image restyling when paired with FLUX1.1 [pro] Ultra.
• FLUX-Kontext — extends FLUX into in-context generation and editing: multimodal conditioning on both text and reference images, enabling localized edits, style transfer, and character/scene consistency across images. As of the report’s writing, Kontext [pro]/[max] were not yet publicly released and so weren’t directly evaluated.