DDPM Project: Training and Evaluation (MNIST)

This repository contains the implementation of a Denoising Diffusion Probabilistic Model (DDPM) trained on the MNIST dataset.

The dependencies for this project are managed via pyproject.toml.

Github repo: https://github.com/locitran/diffusion

Three main components to keep in mind:

1. U-Net architecture + time embedding + attention block. A sketch is here

2. Annealing sampling in DDPM:

$$ x_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( x_t - \frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}} \epsilon_{\theta}(x_t,t) \right) + \sigma_t z $$

3. Training the DDPM model using the variational lower bound (or noise matching):

$$ \mathcal{L} = \mathrm{MSE}(\epsilon_\theta(x_t,t), \epsilon) $$

Clone the repository:

git clone https://github.com/locitran/diffusion.git
cd diffusion

Install dependencies:

pip install .

Running the Code

1. Training the DDPM

python src/ddpm/train.py

# Plot results
python src/ddpm/plot.py

Observe the loss curve during DDPM training on MNIST.

Observe the generated images during DDPM training.

Observation: We start to recognize the image class from 25 to 30 epochs.

2. The Reverse Process

The sample.py script contains the logic for the reverse diffusion process.

Observe the denoising timeline of 5 generated images.

Observation: We start to recognize the image class after t=150. It does not seem to change the image class, but rather fine-tune the texture in the final 200 steps.

3. Quantitative Evaluation

I generate 1000 samples and compute FID and IS. But before that we also need a trained classifier.

# Train the Classifier
python src/ddpm/classifier/train.py

# Compute FID and IS 
python src/ddpm/metrics.py

Metric	My Model	Baseline (Ho et al.)
FID ↓	6.27	3.17 (CIFAR-10)
IS ↑	7.80	9.46 (CIFAR-10)

FID: Frechet Inception Distance, the lower the better.

IS: Inception Score, the higher the better.