Jump to: navigation, search

YouTube search... ...Google search

Diffusion Models are generative models, meaning that they are used to generate data similar to the data on which they are trained. Fundamentally, Diffusion Models work by destroying training data through the successive addition of Gaussian noise, and then learning to recover the data by reversing this noising process. | Ryan O'Connor - AssmblyAI

Control Diffusion Models

ControlNet | llyasviel: We present a neural network structure, ControlNet, to control pretrained large diffusion models to support additional input conditions. The ControlNet learns task-specific conditions in an end-to-end way, and the learning is robust even when the training dataset is small (< 50k). Moreover, training a ControlNet is as fast as fine-tuning a diffusion model, and the model can be trained on a personal devices. Alternatively, if powerful computation clusters are available, the model can scale to large amounts (millions to billions) of data. We report that large diffusion models like Stable Diffusion can be augmented with ControlNets to enable conditional inputs like edge maps, segmentation maps, keypoints, etc. This may enrich the methods to control large diffusion models and further facilitate related applications. Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala

There exists a very subtle but powerful methodology for deep learning. This simple idea is to use a neural network to extract the “specification” from an example. Then use another neural network to generate examples from the extracted example. This is a universal idea. Deep Learning Modularity and Language Models | Carlos E. Perez - Medium