Part A: The Power of Diffusion Models
Precomputed Text Embeddings
num_inference_steps = 20
The pictures are all accurate to the prompts, but you can see there are some strange details, like the man being cross-eyed.
I think the middle rocket looks best, but the rightmost one could be considered the most "detailed", with more colors and shading.
I used 180 as my seed.
Forward Process
Classical Denoising
One-Step Denoising
Iterative Denoising
Diffusion Model Sampling
Classifier-Free Guidance
These were a lot better than in the previous part.
Image-to-Image Translation
With prompt "a high quality photo"
Trying this with my own images!
I then tried this with images from the web and hand drawn images.
For the hand-drawn images, the i-start=20 images came extremely close to the original image! It even looks a little better because it adds proper shading, which can be seen in the duck.
Inpainting
By applying masks to images and only denoising within those masks (keeping the rest of the image as the original), we can create inpainted images.
Text-Conditional Image to Image
This was essentially the same as the other image-to-imgae translation, but with new prompts instead of "a high quality photo"
"a rocket ship"
"a photo of a dog"
Visual Anagrams
We can create visual anagrams by iteratively denoising both an image and its flipped version, each with different prompts. We average the two noise estimates at each step.
Prompts: "an oil painting of people around a campfire", "an oil painting of an old man"
Prompts: "an oil painting of a snowy mountain village", "a photo of the amalfi cost"
Prompts: "a guitar", "a wine bottle"
Hybrid Images
This used a similar technique as the previous part, but with noise estimates from high and low frequencies instead of an image and its flipped version. We can see the low frequency image when the image is far away (or smaller), and the high frequency image when the image is up close, or bigger.
Prompts: "a lithograph of a skull", "a lithograph of waterfalls"
Prompts: "a photo of a dog", "an oil painting of an old man"
I thought this one was particularly interesting because it interpreted "oil painting" as a picture of the painting itself, not making the whole picture an oil painting. The legs of the easel enable the dog to have legs!
Prompts: "a lithograph of a skull", "an oil painting of a snowy mountain village"
