I just posted a big big update to my MultiClip + VQGAN notebook! Lots of improvements to how the cut sizes are picked, improved UI, and new advanced features for you all to play with. 😊
#aiart #vqganclip #vqgan #generativeart
May 8, 2022 · 3:19 PM UTC
Cuts are being generated with a truncated normal and a random choice from multiple sizes. The distribution generally matches the original jbusted1's version, but now includes two larger sizes of cuts as well. I also made it so you can generate your cut distribution heatmap.
There is an option to also oversample the edges so that the center is less "hot". YMMV. Sometimes this helps, but I generally have it off because I like the center focus that the hot spot creates.
I've also added a new feature that I've been using in my private notebooks to good effect. Noise bursting! Every so often this will create an increase in the noise augment before sending it in to CLIP, which decays over time. This can add more vibrancy to the gen.
This release also involved a huge refactor so that I could add a Batch Run feature, and a prompt templating system that I've posted before separately--
There are plenty more other improvements and new features to play with and discover. This is the culmination of a lot of time I've spent tinkering and improving this on my own, and then finally cleaned it up and pulled it all into a public releasable version. 😅
Thank you again to those that helped test this over the past week. @TRAPCHAT01 @manuW_atx @tonytribby
@OcularMagic @k_basali @EyeSeeThru @djcedr and others. 😁
I'd love to see what you all create with this.
I've also started a Discord server, come and post your questions, ideas, and art! discord.gg/cWQxUvwq
Oh, and if you want to go back to the old version, I also added links for all of those in a section in the first cell.
I'm already seeing one more improvement I need to make. The MSE weights need to work differently while doing animation. Otherwise I think it may start losing the thread.
Oh! I nearly forgot! One of the improvements is that each cut's impact on the generation is now weighted by their size. With the larger cuts included and this extra weighting, it really can help to make better panoramic images (especially if you turn on oversampling).
I tried to implement the advanced features in a way that will be easy for yall to mod. Having some good discussion on Discord already about getting the noise burst value from an audio data lookup instead. Would be easy, the hook to modify is right there. 😁