I kind of wanted to do a novel take on integrated gradients but got distracted. Anyhow, here’s the recipe:

  1. Train a flow model. I did it on MNIST digits.
  2. Get some image $x$.
  3. Use the flow model to map the image $x$ to latent $z^{\text{image}}$ by running the typical generation process (i.e. ODE parameterised by the flow model) in reverse.
  4. Define some arbitrary $z^{\text{init}}$ which could be zeros or random noise.
  5. Define the linear interpolant $\tilde{z}(\delta) = \delta z^{\text{image}} + (1 - \delta) z^{\text{init}}$ for $\delta \in [0, 1]$, and for some schedule $0 \le \delta_1 \le \dots \le \delta_D \le 1$ define the latents $z^{\text{interp}}_d = \tilde{z}(\delta_d)$.
  6. Reconstruct all the images $x^{\text{interp}}_{d}$ by running the ODE parameterised by the flow model forwards, and output that as a video.

It should be noted that for step (5), a linear schedule for $\delta$ is unlikely to be optimal, as we are optimising for distance in image space so our video is smoother. We can instead reconstruct $x^{\text{interp}}_{d}$ as we go, find the largest difference between two frames, and place a timestep in between.

For larger images that the ones we trained on, we can simply cut up the image into training data sized tiles and treat them all independently.

Keqing from Genshin Impact. Edge detection was used to create the original image.


Skirk from Genshin Impact. Artwork by ig:jyenartx (unaffiliated)