Difference between revisions of "Diffusion"
m |
m |
||
| Line 28: | Line 28: | ||
* [https://arxiv.org/abs/2302.03668 Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery | Y. Wen, N. Jain, J. Kirchenbauer, M. Goldblum, J. Geiping, and T. Goldstein] | * [https://arxiv.org/abs/2302.03668 Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery | Y. Wen, N. Jain, J. Kirchenbauer, M. Goldblum, J. Geiping, and T. Goldstein] | ||
| − | [https://github.com/lllyasviel/ControlNet ControlNet | llyasviel]: We present a neural network structure, ControlNet, to control pretrained large diffusion models to support additional input conditions. The ControlNet learns task-specific conditions in an end-to-end way, and the learning is robust even when the training dataset is small (< 50k). Moreover, training a ControlNet is as fast as fine-tuning a diffusion model, and the model can be trained on a personal devices. Alternatively, if powerful computation clusters are available, the model can scale to large amounts (millions to billions) of data. We report that large diffusion models like Stable Diffusion can be augmented with ControlNets to enable conditional inputs like edge maps, segmentation maps, keypoints, etc. This may enrich the methods to control large diffusion models and further facilitate related applications. [https://arxiv.org/abs/2302.05543 Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala] | + | [https://github.com/lllyasviel/ControlNet ControlNet | llyasviel]: We present a neural network structure, ControlNet, to control pretrained large diffusion models to support additional input conditions. The ControlNet learns task-specific conditions in an end-to-end way, and the learning is robust even when the training dataset is small (< 50k). Moreover, training a ControlNet is as fast as [[fine-tuning]] a diffusion model, and the model can be trained on a personal devices. Alternatively, if powerful computation clusters are available, the model can scale to large amounts (millions to billions) of data. We report that large diffusion models like Stable Diffusion can be augmented with ControlNets to enable conditional inputs like edge maps, segmentation maps, keypoints, etc. This may enrich the methods to control large diffusion models and further facilitate related applications. [https://arxiv.org/abs/2302.05543 Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala] |
There exists a very subtle but powerful methodology for deep learning. This simple idea is to use a neural network to extract the “specification” from an example. Then use another neural network to generate examples from the extracted example. This is a universal idea. [https://medium.com/intuitionmachine/deep-learning-modularity-and-language-models-bd726c5e3b58 Deep Learning Modularity and Language Models | Carlos E. Perez - Medium] | There exists a very subtle but powerful methodology for deep learning. This simple idea is to use a neural network to extract the “specification” from an example. Then use another neural network to generate examples from the extracted example. This is a universal idea. [https://medium.com/intuitionmachine/deep-learning-modularity-and-language-models-bd726c5e3b58 Deep Learning Modularity and Language Models | Carlos E. Perez - Medium] | ||
https://github.com/lllyasviel/ControlNet/raw/main/github_page/he.png | https://github.com/lllyasviel/ControlNet/raw/main/github_page/he.png | ||
Revision as of 19:41, 16 August 2023
YouTube search... ...Google search
- Markov Model (Chain, Discrete Time, Continuous Time, Hidden)
- Stable Diffusion | Stable Diffusion ... a latent text-to-image diffusion model capable of generating photo-realistic images given any text input
- DALL-E
- Generative AI ... Conversational AI ... ChatGPT | OpenAI ... Bing | Microsoft ... Bard | Google ... Claude | Anthropic ... Perplexity ... You ... Ernie | Baidu
Diffusion Models are generative models, meaning that they are used to generate data similar to the data on which they are trained. Fundamentally, Diffusion Models work by destroying training data through the successive addition of Gaussian noise, and then learning to recover the data by reversing this noising process. | Ryan O'Connor - AssmblyAI
Control Diffusion Models
- Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala
- Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery | Y. Wen, N. Jain, J. Kirchenbauer, M. Goldblum, J. Geiping, and T. Goldstein
ControlNet | llyasviel: We present a neural network structure, ControlNet, to control pretrained large diffusion models to support additional input conditions. The ControlNet learns task-specific conditions in an end-to-end way, and the learning is robust even when the training dataset is small (< 50k). Moreover, training a ControlNet is as fast as fine-tuning a diffusion model, and the model can be trained on a personal devices. Alternatively, if powerful computation clusters are available, the model can scale to large amounts (millions to billions) of data. We report that large diffusion models like Stable Diffusion can be augmented with ControlNets to enable conditional inputs like edge maps, segmentation maps, keypoints, etc. This may enrich the methods to control large diffusion models and further facilitate related applications. Adding Conditional Control to Text-to-Image Diffusion Models | Lvmin Zhang, Maneesh Agrawala
There exists a very subtle but powerful methodology for deep learning. This simple idea is to use a neural network to extract the “specification” from an example. Then use another neural network to generate examples from the extracted example. This is a universal idea. Deep Learning Modularity and Language Models | Carlos E. Perez - Medium
