We introduce PairFlow, a lightweight preprocessing step for training Discrete Flow Models (DFMs) to achieve few-step sampling without requiring a pretrained teacher. DFMs have recently emerged as a new class of generative models for discrete data, offering strong performance. However, they suffer from slow sampling due to their iterative nature. Existing acceleration methods largely depend on finetuning, which introduces substantial additional training overhead. PairFlow addresses this issue with a lightweight preprocessing step. Inspired by ReFlow and its extension to DFMs, we train DFMs from coupled samples of source and target distributions, without requiring any pretrained teacher.
At the core of our approach is a closed-form inversion for DFMs, which allows efficient construction of paired source–target samples. Despite its extremely low cost, taking only up to 1.7% of the compute needed for full model training, PairFlow matches or even surpasses the performance of two-stage training involving finetuning. Furthermore, models trained with our framework provide stronger base models for subsequent distillation, yielding further acceleration after finetuning. Experiments on molecular data as well as binary and RGB images demonstrate the broad applicability and effectiveness of our approach.
PairFlow is a training framework for DFMs that enables few-step sampling by constructing paired source–target samples using closed-form velocities. While inspired by ReDi-style coupling-driven training, our approach eliminates the need for a pretrained teacher by using closed-form formulations and achieves acceleration without finetuning. The algorithm for computing source–target pairs is fully parallelizable and requires at most 1.7% of compute needed for full model training.
Figure 1: Illustrations of data inversion in PairFlow and the standard corruption process in UDLM. PairFlow achieves a lower average Hamming distance (6.47 vs. 9.0), promoting straighter paths during training.
Algorithm 1: Procedure for generating source-target pairs using the closed-form backward velocity.
We derive the closed-form forward velocity field $\hat{v}_t(x^i, z)$ for discrete domains. For the empirical target distribution $\tilde{q}(x)$, the closed-form denoiser $p_{1|t}(x^i|z)$ and its associated velocity field $\hat{v}_t(x^i, z)$ are given by:
$$ \begin{align} p_{1|t}(x^i|z) = \cfrac{\sum_{m=1}^M \delta_{d_{m}^i}(x^i) \gamma^{- h(d_{m}, z)}}{ \sum_{m=1}^M \gamma^{- h(d_{m}, z)}}, \quad \Longrightarrow \quad \hat{v}_t(x^i, z) = \frac{\dot\kappa_t}{1 - \kappa_t} \left[p_{1|t}(x^i|z) - \delta_{z}(x^i) \right]. \tag{9} \end{align} $$
To effectively identify suitable source-target pairs, we derive the closed-form backward velocity. Specifically, we first derive the closed-form noise predictor $p_{0|t}(x^i|z)$:
$$ \begin{align} p_{0|t}(x^i|z) = \delta_{z}(x^i) - \frac{\kappa_t(K \delta_{x^i}(z^i) - 1)}{1 + (K-1)\kappa_t} \frac{\sum_{m=1}^M \delta_{d_m^i}(z^i)\gamma^{-h(d_m, z)}}{\sum_{m=1}^M \gamma^{-h(d_m, z)}}. \tag{11} \end{align} $$
Substituting this into the definition of the backward velocity field, we obtain the desired closed-form expression:
$$ \begin{align} \check{v}_t(x^i, z) = \frac{\dot{\kappa}_t(K \delta_{x^i}(z^i)-1)}{1 + (K-1)\kappa_t} \frac{\sum_{m=1}^M \delta_{d_m^i}(z^i)\gamma^{-h(d_m, z)}}{\sum_{m=1}^M \gamma^{-h(d_m, z)}}. \tag{12} \end{align} $$
Using the derived closed-form backward velocity, we can efficiently compute source-target pairs. As outlined in Algorithm 1, starting from a given target data sample $x_1$, we iteratively apply the backward update rule using $\check{v}_t(x^i, z)$ to trace back the probability path. The final state $x_0$ reached by this iterative process serves as the source noise. This directly constructs a well-aligned source-target pair $(x_0, x_1)$ purely from data, effectively eliminating the need for generating samples from a pretrained teacher model.
We validate the effectiveness of the proposed method and the source–target pairs it discovers across several discrete generative modeling benchmarks involving molecular data and images.
Figure 2: Step-wise performance analysis on the QM9 dataset.
Figure 3: Step-wise performance analysis on the ZINC-250k dataset.
Figure 4: Step-wise performance analysis on discretized image datasets.
@inproceedings{park2026:pairflow,
title={PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models},
author={Park, Mingue and Hwang, Jisung and Yoo, Seungwoo and Yeo, Kyeongmin and Sung, Minhyuk},
booktitle={ICLR},
year={2026}
}