"Moloch whose mind is pure machinery by Greg Rutkowski, oil on canvas" (CLIP guided diffusion + text to image embedding prior)

May 30, 2022 · 5:08 PM UTC

1
8
71
Replying to @RiversHaveWings
Hey neat. So... I understood everything you wrote there except for the "text to image embedding prior" part. Can I ask what that means?
1
1
It's the first component of DALL-E 2 (arxiv.org/abs/2204.06125), I trained a small one that I am using to sample CLIP image embeddings given text and am then using the resulting image embeddings in CLIP guided diffusion to see what they look like.

Hierarchical Text-Conditional Image Generation with CLIP Latents

Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a...

arxiv.org
3
2
more replies