FIRM: Flow-based Imaging via
Regularized Minimization

An operator-aware conditional velocity field, trained end-to-end

Shirin Shoushtari1, Edward P. Chandler2, Xiao Shi2, Ulugbek S. Kamilov2
1WashU 2UW-Madison
Preprint
Qualitative reconstructions on CelebA for super-resolution, random inpainting and box inpainting.

Qualitative reconstructions on CelebA. Values below each panel report PSNR/SSIM/LPIPS, and the insets show a zoomed crop next to its error map. FIRM recovers detail the baselines miss while using only $N\!=\!2$ ODE steps. Click the figure to enlarge.

up to 50×
fewer network evaluations than competitive flow-based methods
0.24 s
per image at the best distortion setting (vs. 13.03 s)
10
NFEs at $N\!=\!2$, $K\!=\!5$—no inference-time guidance
1 model
spans the distortion–perception trade-off, no retraining

Abstract

Flow matching methods for imaging inverse problems typically incorporate measurements through network conditioning or guidance during sampling. Neither approach explicitly applies the forward operator within the learned conditional velocity field. We develop a principled measurement-conditional velocity parameterization that does. For a linear interpolation path, we express the optimal velocity through the posterior mean $\mathbb{E}[x_1 \mid x_t, y]$ and show that this mean is the unique minimizer of a variational objective with an explicit data-consistency term. The velocity defined by this minimizer provably transports the source distribution to the measurement-conditioned posterior. This result leads to a forward operator-aware velocity field that is trained end-to-end and requires no separate guidance during sampling. Across five imaging tasks, our method achieves leading reconstruction quality with up to 50× fewer network evaluations than competitive flow-based methods. Varying the number of sampling steps also controls the distortion–perception trade-off without retraining.

Method

The conditional velocity is a posterior mean

Under linear interpolation $x_t = t x_1 + (1-t) x_0$, the state $x_t$ and the measurement $y = Ax_1 + \eta$ are two conditionally independent noisy observations of the same clean image $x_1$. Stacking them gives an augmented observation model whose MMSE estimator is exactly the measurement-conditioned posterior mean, and that mean determines the optimal conditional velocity:

$$ v(x_t, t, y) \;=\; \frac{\mathbb{E}[x_1 \mid x_t, y] - x_t}{1 - t}. $$

A variational characterization with explicit data consistency

Our main theoretical result (Proposition 1) shows that for a non-degenerate $p_1$ with bounded support, $\sigma > 0$ and $t \in (0,1)$, there is a regularizer $\phi_t$—unique up to an additive constant—for which the posterior mean is the unique global minimizer and unique stationary point of

$$ \mathbb{E}[x_1 \mid x_t, y] \;=\; \arg\min_{x \in \mathbb{R}^n}\; \frac{1}{2(1-t)^2}\lVert x_t - t x \rVert_2^2 \;+\; \frac{1}{2\sigma^2}\lVert y - A x \rVert_2^2 \;+\; \phi_t(x). $$

The first quadratic enforces interpolation consistency, the second enforces measurement consistency explicitly, and $\phi_t$ absorbs whatever structure remains. We further prove that this velocity satisfies the conditional continuity equation $\partial_t p_t(x \mid y) + \nabla \cdot \big(p_t(x \mid y) v(x,t,y)\big) = 0$, so the induced flow transports the Gaussian source to $p_\tau(\cdot \mid y)$, which converges weakly to the measurement-conditioned posterior $p(x_1 \mid y)$ as $\tau \to 1^-$. Data consistency becomes part of the generative dynamics rather than an external sampling correction.

An operator-aware conditional velocity

$\phi_t$ is not available in closed form, so we split the objective with half-quadratic splitting and alternate the two resulting updates. The $x$-update is a closed-form data-consistency solve that depends on $A$ and $y$; the $z$-update combines interpolation consistency with the implicit regularizer and is replaced by a learned estimator $R_\theta$, damped toward the data-consistent iterate. After $K$ iterations the output $z_\theta^K(x_t, t, y)$ estimates the posterior mean and defines the learned velocity.

The whole estimator is trained end-to-end with a weighted flow-matching loss, using a stabilized time scale $\tau_t = \max(1 - t, \tau_{\min})$ that keeps the velocity bounded near $t = 1$:

$$ \mathcal{L}(\theta) = \mathbb{E}_{x_1, x_0, t, y}\Big[\omega(t)\, \lVert v_\theta(x_t,t,y) - v_{\text{target}} \rVert^2 \Big], \qquad \omega(t) = \frac{1 + \tau_t^3}{\tau_t}. $$

The result is an operator-aware velocity field: every inner iteration of every velocity evaluation applies $A$ and $A^{\top}$ through the data-consistency solve, so data consistency is learned as part of the dynamics rather than imposed afterwards. Sampling therefore needs no guidance term, no projection, and no proximal correction at inference—just $N$ explicit Euler steps.

Algorithms

Training require: A, σ, [tmin, tmax], τmin
repeat
  x₁ ~ p₁,   x₀ ~ N(0,I),   t ~ U[tmin, tmax]
  η ~ N(0, σ²I),   y ← A x₁ + η
  xₜ ← t x₁ + (1−t) x₀,   τₜ = max(1−t, τmin)
  zK ← PostMean(xₜ, t, y, A)
  vθ ← (zK − xₜ) / τₜ
  vtarget ← (x₁ − xₜ) / τₜ
  update θ with the weighted FM loss
until convergence
Sampling require: y, A, trained model, step count N
x₀ ~ N(0, I)
Δt ← 1 / N
for i = 0 … N−1 do
  tᵢ ← i Δt
  zKi ← PostMean(xᵢ, tᵢ, y, A)
  vᵢ ← (zKi − xᵢ) / (1 − tᵢ)
  xᵢ⁺¹ ← xᵢ + Δt · vᵢ
return xN
 
no guidance, no projection, no proximal step
PostMean—operator-aware posterior mean require: xₜ, t < 1, y, A, Rθ, K, ρ, β
z⁰ ← xₜ
for k = 0 … K−1 do
  xᵏ⁺¹ ← (AᵀA + ρI)⁻¹ (Aᵀy + ρ zᵏ)
  ẑᵏ⁺¹ ← Rθ(xᵏ⁺¹, xₜ, t)
  zᵏ⁺¹ ← (1−β) xᵏ⁺¹ + β ẑᵏ⁺¹
return zK
 
θ enters only through Rθ; each call costs
K solves + K network evaluations

Qualitative Results

CelebA reconstructions for super-resolution, random inpainting and box inpainting
CelebA, 128×128. Super-resolution, random inpainting and box inpainting. Values below each panel report PSNR/SSIM/LPIPS. FIRM recovers details the baselines fail to recover while requiring only $N\!=\!2$ ODE steps.
AFHQ-Cat reconstructions for denoising and deblurring
AFHQ-Cat, 256×256. Denoising (top) and deblurring (bottom). FIRM preserves fine image detail—whiskers, fur texture—and produces faithful reconstructions using only $N\!=\!2$ sampling steps.
Compressed-sensing Fourier reconstruction on AFHQ-Cat
Compressed-sensing Fourier sampling on AFHQ-Cat (256×256), $\sigma = 0.002$, with a Cartesian mask retaining 21.88% of frequencies. Columns: ground truth, the zero-filled adjoint $A^{*}y$, Flower1-OT, and ours at $N\!=\!2$. The data-consistency solve is available in closed form for subsampled Fourier measurements, so the same construction carries over unchanged.

Inference Cost

Inference cost is $O(NK)$: $N$ ODE steps, each calling the posterior-mean estimator with $K = 5$ inner iterations. The default $N\!=\!2$ configuration therefore uses 10 network evaluations. Per-NFE cost is comparable to PnP-Flow and Flower, but FIRM needs 50× fewer of them—reconstructing an image in 0.24 s against 13.03 s for Flower5-OT. Peak memory is higher than PnP-Flow and Flower because intermediate iterates are retained, but far below the remaining baselines. The speedup stems from learning data consistency during training rather than enforcing it at inference.

PSNR versus wall-clock time per image; FIRM sits up and to the left of every baseline
PSNR averaged over the five CelebA restoration tasks against per-image wall-clock time (log scale, single NVIDIA RTX A6000). Up and to the left is better. Our curve sweeps $N \in \{2,4,10,25\}$ at fixed $K\!=\!5$; each baseline contributes one point. At $N\!=\!2$ we exceed the strongest baseline at roughly 50× lower cost.

Time, memory and NFEs—CelebA deblurring

MetricOT-ODED-FlowFlow PriorsPnP-Flow1PnP-Flow5Flower1Flower5Ours (N=2)Ours (N=4)Ours (N=10)Ours (N=25)
Time (s) ↓6.37299.2541.272.3311.462.6713.030.240.501.213.13
Peak memory (MB) ↓7436,1242,728192192193193331331331331
NFEs ↓180–100100500100500102050125

Averaged over 10 images on a single NVIDIA RTX A6000. Best and second best are highlighted.

Distortion–Perception Control at Test Time

The number of Euler steps $N$ is a free knob at inference. Coarse integration lands near the measurement-conditioned posterior mean, favouring PSNR and SSIM; finer integration moves toward perceptually realistic samples, favouring LPIPS. A single trained model therefore spans both regimes—no retraining, no reweighting of the loss.

Effect of the number of ODE steps on AFHQ-Cat deblurring
AFHQ-Cat deblurring. Values below each reconstruction report PSNR/LPIPS for the displayed image; the right panel reports averages over the test set. Increasing $N$ improves perceptual quality at the cost of PSNR.
Distortion-perception trade-off across all five tasks on CelebA
CelebA (128×128) across all five inverse problems at $N \in \{2,4,10,25\}$.
Distortion-perception trade-off across all five tasks on AFHQ
AFHQ (256×256) across all five inverse problems at $N \in \{2,4,10,25\}$.
CelebA random inpainting reconstructions as the number of sampling steps grows
Random inpainting as the step count sweeps $N = 1 \to 100$. Texture sharpens monotonically while the estimate drifts away from the posterior mean.
CelebA super-resolution reconstructions as the number of sampling steps grows
×2 super-resolution over the same sweep.

Quantitative Results

Five restoration tasks, two datasets, 100 test images each. Degradations follow the protocols of PnP-Flow and Flower without modification: denoising at $\sigma = 0.2$; deblurring with a $61 \times 61$ Gaussian kernel; $\times 2$ (CelebA) and $\times 4$ (AFHQ-Cat) super-resolution; random inpainting masking 70% of pixels; and box inpainting with a centered $40\times40$ (CelebA) or $80\times80$ (AFHQ-Cat) mask. All baselines except ISTA-Net and I²SB enforce measurement consistency at inference time using a pretrained prior.

DenoisingDeblurringSuper-resolutionRandom inpaintingBox inpainting
MethodPSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
Degraded20.000.3480.37127.810.7400.12510.250.1830.82711.950.1961.04122.260.7430.213
PnP-GS32.540.9080.03533.970.9240.04131.230.8900.06529.200.8750.069–––
PnP-HQS32.170.8970.04434.870.9460.03332.330.9110.07532.810.9450.023–––
ISTA-Net–––35.380.9500.03533.540.9370.03834.700.9600.022–––
DiffPIR31.120.8830.06032.680.9100.06031.460.8930.03431.690.9160.025–––
I²SB–––33.250.9240.01231.920.8910.01432.800.9380.01128.720.8030.042
OT-ODE30.490.8580.03232.960.9200.02931.340.9030.02728.650.8700.05129.370.9190.038
D-Flow26.010.6060.09231.210.8540.03730.440.8430.02633.610.9420.01530.590.8980.027
Flow-Priors29.310.7670.13531.510.8570.05628.360.7150.10032.870.9430.01930.060.8590.048
PnP-Flow131.750.9040.04434.480.9360.03931.040.9010.04533.020.9440.01830.450.9330.037
PnP-Flow532.240.9100.05634.800.9410.04631.440.9050.05633.950.9530.02231.060.9390.043
Flower1-OT32.220.9130.03334.950.9470.02532.300.9220.03433.050.9440.01731.170.9450.022
Flower5-OT33.070.9250.03835.650.9540.03133.030.9310.03933.920.9530.02031.850.9520.023
Ours (N=2)33.670.9320.03436.040.9570.02934.390.9490.02835.030.9630.01833.250.9600.022
Ours (N=4)33.240.9280.03435.780.9560.02833.960.9460.02634.670.9620.01632.570.9580.021
Ours (N=10)32.260.9140.03235.030.9500.02333.130.9370.02233.980.9560.01431.850.9530.020
Ours (N=25)31.560.9010.02831.460.8480.02630.390.8370.02933.420.9510.01331.330.9470.020

100 CelebA test images. Best and second best among all methods (the degraded reference is excluded). – marks a setting where a baseline does not apply. Scroll the table sideways on narrow screens.

DenoisingDeblurringSuper-resolutionRandom inpaintingBox inpainting
MethodPSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
Degraded20.000.2930.52924.380.5290.43611.590.2120.86713.230.2131.08221.520.7270.215
PnP-GS33.000.8950.07828.390.7870.38724.440.6390.41129.820.8440.143–––
PnP-HQS32.690.8960.10829.340.7930.36226.580.7240.44133.070.9160.051–––
ISTA-Net–––29.360.7900.34228.220.7840.29734.100.9290.049–––
DiffPIR31.050.8390.18628.240.7470.31924.240.6500.38532.310.8860.057–––
I²SB–––27.720.7250.07727.190.7390.07031.790.8790.02627.320.7630.072
OT-ODE30.480.8180.08127.820.7350.12626.710.7370.10829.990.8490.08724.470.8730.095
D-Flow26.440.5730.18028.600.7460.16725.200.6160.18832.790.8980.04427.020.8400.081
Flow-Priors29.670.7560.16527.280.7260.18823.950.5660.27432.950.9090.05126.130.8090.130
PnP-Flow131.620.8660.13928.710.7790.28527.560.7800.15833.710.9230.03826.450.8950.111
PnP-Flow531.870.8680.17029.010.7850.31228.010.7910.16734.440.9320.04527.140.9000.127
Flower1-OT32.120.8810.11329.380.7930.23726.760.7590.26233.680.9220.04226.670.9140.065
Flower5-OT32.750.8920.12629.730.8010.26427.090.7670.27234.360.9310.04827.320.9220.067
Ours (N=2)33.310.9040.08129.890.8090.22929.100.8160.16734.710.9360.04329.270.9240.060
Ours (N=4)32.930.8980.07929.420.7960.18828.500.8010.14134.300.9320.03928.650.9210.052
Ours (N=10)31.960.8790.07228.720.7750.14527.750.7770.11533.530.9220.03327.940.9160.047
Ours (N=25)29.690.7760.08128.160.7540.11926.540.6760.12132.910.9120.03026.760.8190.077

100 AFHQ-Cat test images. Best and second best among all methods (the degraded reference is excluded).

Ablations and Robustness

$K$ controls how many data-consistency / prior alternations happen inside one velocity evaluation. PSNR keeps improving with $K$ while LPIPS bottoms out at $K = 5$. Since cost scales as $O(NK)$, we use $K = 5$ throughout—roughly 70% of the cost of $K = 7$ for essentially the same quality.

KPSNR ↑SSIM ↑LPIPS ↓Time (ms) ↓
334.630.9590.020137
535.030.9630.018223
735.210.9640.019322

Random inpainting, $N = 2$.

Training with a fixed forward model ties the learned field to a particular $A$ and $\sigma$. A model trained at $\sigma = 0.01$ and evaluated at other noise levels still outperforms the operator-agnostic Flower5-OT on all three metrics, at both tested levels.

Methodσtest=0.005 · PSNR ↑SSIM ↑LPIPS ↓σtest=0.02 · PSNR ↑SSIM ↑LPIPS ↓
Flower5-OT34.020.9540.01933.650.9470.024
Ours35.190.9650.01834.570.9540.020

A model trained at a 0.7 inpainting mask ratio, applied without retraining to ratios from 0.5 to 0.8. Quality is monotone in the test ratio with no failure mode at either extreme. Because we do not train a matched model per ratio, this measures graceful degradation rather than quantifying the mismatch penalty. At test time the data-consistency step uses the actual test mask, while the learned regularizer and the step-size parameters stay fixed at their $p = 0.7$ values.

Test mask ratio pPSNR ↑SSIM ↑LPIPS ↓
0.538.260.9780.008
0.636.950.9730.012
0.7 † (training ratio)35.030.9630.018
0.832.100.9360.034
Reconstructions under forward-model mismatch on random inpainting
Reconstructions under forward-model mismatch on CelebA random inpainting, test mask ratios 0.5–0.8.

Does the Flow Really Reach the Posterior?

Proposition 1 says the ideal field transports the source to $p(x_1 \mid y)$, not to a single point estimate. Two experiments check that the trained model inherits this behaviour.

Posterior sample diversity on box inpainting; variation is confined to the masked region
Box inpainting, CelebA. The measurement $y$ is held fixed and only the source draw $x_0 \sim \mathcal{N}(0,I)$ varies. Columns show ground truth, measurement, three of eight reconstructions at $N\!=\!2$, the pixel-wise standard deviation over all eight, and the same at $N\!=\!25$. Variation is confined to the masked region and vanishes where measurements are available—exactly what transporting to the measurement-conditioned posterior predicts. Longer integration widens the spread, which is why LPIPS improves and PSNR drops.
Toy problem: samples against the analytical posterior
A two-dimensional problem where the posterior is available in closed form. At $N\!=\!50$ the generated samples (blue) match the analytical posterior (red) in both modes, while the MMSE mean sits in the low-density valley between them—the model is sampling, not regressing to the mean.
Posterior recovery degrades smoothly as the number of sampling steps drops
Posterior recovery degrades smoothly as $N$ drops: the Wasserstein-1 gap to the true posterior grows and mass collapses toward the mean. Few steps buy distortion performance by giving up sample fidelity, which is the same trade-off the image experiments show.
Internal solver trajectory across Euler steps and inner HQS iterations
Inside the solver. Box inpainting with $N\!=\!2$ Euler steps and $K\!=\!5$ inner iterations each. Rows are Euler steps labelled by $t$; columns walk the inner iterations, showing the data-consistency iterate $x^k$ above the damped learned update $z^k$.

Acknowledgments

This work was supported in part by the National Science Foundation under CAREER award CCF-2625643 and award CCF-2622128.

BibTeX

@misc{shoushtari2026firm,
  title         = {FIRM: Flow-based Imaging via Regularized Minimization},
  author        = {Shoushtari, Shirin and Chandler, Edward P. and Shi, Xiao
                   and Kamilov, Ulugbek S.},
  year          = {2026},
  journal        = {arXiv 2609.12953}
}