Diffusion

YouTube search... ...Google search

• Markov Model (Chain, Discrete Time, Continuous Time, Hidden)
• Stable Diffusion | Stable Diffusion ... a latent text-to-image diffusion model capable of generating photo-realistic images given any text input
• DALL-E 2 | OpenAI
  • How DALL-E 2 Actually Works | Ryan O'Connor - AssemblyAI
• Generative AI ... OpenAI's ChatGPT ... Perplexity ... Microsoft's BingAI ... You ...Google's Bard

Diffusion Models are generative models, meaning that they are used to generate data similar to the data on which they are trained. Fundamentally, Diffusion Models work by destroying training data through the successive addition of Gaussian noise, and then learning to recover the data by reversing this noising process. | Ryan O'Connor - AssmblyAI

Control Diffusion Models

• Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala
• Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery | Y. Wen, N. Jain, J. Kirchenbauer, M. Goldblum, J. Geiping, and T. Goldstein

ControlNet | llyasviel: We present a neural network structure, ControlNet, to control pretrained large diffusion models to support additional input conditions. The ControlNet learns task-specific conditions in an end-to-end way, and the learning is robust even when the training dataset is small (< 50k). Moreover, training a ControlNet is as fast as fine-tuning a diffusion model, and the model can be trained on a personal devices. Alternatively, if powerful computation clusters are available, the model can scale to large amounts (millions to billions) of data. We report that large diffusion models like Stable Diffusion can be augmented with ControlNets to enable conditional inputs like edge maps, segmentation maps, keypoints, etc. This may enrich the methods to control large diffusion models and further facilitate related applications. Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala

There exists a very subtle but powerful methodology for deep learning. This simple idea is to use a neural network to extract the “specification” from an example. Then use another neural network to generate examples from the extracted example. This is a universal idea. Deep Learning Modularity and Language Models | Carlos E. Perez - Medium