just finetuned the SD 1.4 vae on a bunch of anime-styled images and finally it can reconstruct eyes and fingers muuuuch better! vae is available here: huggingface.co/hakurei/waifu…

Oct 10, 2022 · 1:45 AM UTC

7
71
7
322
massive thanks to @mahdimc for providing the compute used for finetuning! here's some more examples with the original image on the left, SD 1.4 VAE reconstruction in the middle, and the finetuned VAE on the right:
2
4
31
Replying to @haruu1367
extremely excited, i've wanted to do this for so long
1
10
yeah im surprised no one has done it yet, it seems more efficient than finetuning SD on image embeddings
7
Replying to @haruu1367 @_akhaliq
Did you fine tune the encoder together with the decoder, before fine tuning the unet?
1
the unet has not been touched at all, just the encoder and decoder. i'm still finetuning it more now with the encoder frozen. will post more results later
Replying to @haruu1367 @_akhaliq
Cool! How many images did you use for the fine tune?
1
1
250k images
2
Replying to @haruu1367
Unfortunately the bracelet on the redhead in the white dress doesn't like the fine tune
Replying to @haruu1367
Nice!
1
6