×
all 40 comments
sorted by:
best

[–]storods[S] 24 points25 points  (4 children)

I've been holding off on the AI craze until SD came around. Being able to steer the model into very specific styles was the major development that won me over. I've been learning and training for 1-2 weeks now and I'm beyond impressed. In these images I combined trained models with TI styles and some input images with random splashes of color. I just fell in love with the results and I had to share them.

The process:
1. I trained a small texture inversion bin on colab, with the inputs set to 6-7 images of blade runner las vegas. Mainly from places where large paneled concrete structures take up most of the scene. This was already a great little style to use in dark cityscapes.

  1. After firing up dreambooth on a rented machine I also processed 16 images of Blade Runner Joi into a model file. The results mainly focused on the face in most of the prompts.

  2. I loaded in the Model and Style just for fun. And Joi's own style hidden until now exploded into the raw concrete making everything more vibrant but still cool.

custom models + custom styles = breath of fresh air

[–]SandCheezy 0 points1 point  (2 children)

What repos are you using for both TI and DB?

Also, where do you prefer to get your rented GPU from?

[–]Nmanga90 0 points1 point  (1 child)

rented GPU from GCP is the best. $300 credit you literally cant go wrong. Just set up payment, and then request preemptible T4 or A100 or whatever device you want.

[–]SandCheezy 0 points1 point  (0 children)

Thanks!

[–]Leprechaun72 0 points1 point  (0 children)

hey what prompts did you use to get her like that

[–]Distinct-Quit6909 6 points7 points  (0 children)

wow I've been trying to render Ana De Armas in SD with txt2img with pretty poor results. But these, wow wow wow! Beyond stunning!! This is exactly what I've been going for. Well done. Time for me to jump on dreambooth!!

[–]KrankDamon 4 points5 points  (0 children)

really cool work, keep it up!

[–]AppropriateFlan3077 3 points4 points  (0 children)

That's really creative!

Using TI for style and dreambooth for object seems like the best of both worlds.

[–]Devilray_TT 2 points3 points  (0 children)

vtuber waifu RTX on!

Good work there buddy. That brutalist architechture is incredible!

[–][deleted] 2 points3 points  (0 children)

Thanks for sharing. She looks really good and pictures look very stylish. You could say that AI has very good taste or is very sophisticated. Of course you train it, configure, write prompt etc. but her sense of style (in my language Intelligence is a she) is usually flawless. Even when person has some extra fingers...

When the craze began my initial thought was that AI art will be soulless. Now I think this is quite opposite. This images reflect all of human culture from ancient art, through Warhol and beyond. AI this is us. Every generated image has a bit of everyone.

[–]In4theaction 1 point2 points  (2 children)

Is training how the AI can learn a specific character or item from the prompt? For example, you trained it on Joi and then it knew when you added her to the prompt. If so does this require having the AI running on a computer and not just through the browser version with image2image? Im just starting out, any sources for info would be appreciated.

[–]storods[S] 9 points10 points  (1 child)

Yes, and no. With dreambooth you can train it to do specific charactees for you. You give a training program 10-20 images of your subject, and 100 to 1500 classifier or regularizer images that helps the algorithm with learning the type of your subject. You also need to give a name to your subject so you can call it in the prompt.

For example. I had 16 images of Joi prepared. close, mid and far photos. Joi can be classified as a woman, or a person, so I chose a premade dataset of 1500 images of various persons to be my regularization source. (it's available in the newer notebooks) I named joi in the config file as bladejoy for the prompts.

So I let it run, and after a good 10-40 minutes I can download a singular model file. (you need some beefy machines for this but dont worry there are some great solutions already for that). This model file now can be moved across various versions of SD. It may be easier to do it locally because you need acces to your models folder. If you have an online web interface where you can swap your own model files then it should be possible to generate there too.

Now with the new model file loaded in. Inside the promts I just call the name bladejoy and the assigned and trained subject class person. "bladejoy person" And that should be it. If everything was done right it should look like Joi.

If you want to try it here's a great video: https://youtu.be/7m__xadX0z0

And don't forget there's always a lot to keep up with. New and better versions and methods are developed daily and it can only get easier to do this stuff.

[–]In4theaction 2 points3 points  (0 children)

Thank you so much that was the clarity I was looking for. I see where some of this can be ran in a Google Collab.

[–]AndThisGuyPeedOnIt 1 point2 points  (2 children)

Cool. This is exactly what I was looking to see if SD was able to do.

I saw your other comment about how you trained it on Joi. Would you similarly be able to locally train it with, say, images of characters you have created and get enough references for body position, facial looks, etc. that you'd be able to create scene descriptions like it were making comic book panels with your characters?

[–]storods[S] 1 point2 points  (1 child)

Yes it's possible. As far as I understand when you assign the class of your subject, for example to tell the training algorythm: here's my character its name and a bunch of images that explains what it is.. the explaining part is key to any kind of interaction with the world elements. if your character is assigned a person class then most of the things a person could do according the preexisting model of SD should also be possible. drive a car, sit on the beach, write a letter etc. If it wont listen to the words you can also give it an image with the rough body position and environemnt painted in as a blockout. most of the time it gets the situation and a few seed later something useable pops up.

Of course your results may vary as this is highly highly experimental. In the worst case scenario you can use inpainting to fix your characters onto different images generated with a different style.

[–]AndThisGuyPeedOnIt 0 points1 point  (0 children)

Thanks for the explanation. I've been waiting to install it locally and give this a shot, and your work has shown me it's possible. I wonder how good it would be at drawing environments if you block them out (sort of like Nvidia Canvas, but I believe that is limited strictly to landscapes).

[–]CarelessConference50 1 point2 points  (0 children)

I love Joi, and Brutalism. Nice work.

[–]biogoly 1 point2 points  (0 children)

Fantastic work!

[–]Low_Government_681 1 point2 points  (0 children)

Stunning work ♥

[–]MostlyRocketScience 1 point2 points  (2 children)

Would you mind sharing the Google Drive link for your joi.ckpt? I've noticed that Stable Diffusion can't really draw Ana de Armas well

[–]storods[S] 2 points3 points  (1 child)

Here you go:
https://drive.google.com/file/d/1ZggRNwDVBTYTgqqk1oUAhNjvJ2QpHgFB/view?usp=sharing
the calling prompt: "bladejoy person"

[–]MostlyRocketScience 1 point2 points  (0 children)

Thank you for sharing it! Very useful to me

[–]NoHopeHubert 1 point2 points  (2 children)

I think you’re one of the first people I’ve seen that used textual inversion in tandem with a dreambooth trained subject, I was wondering if it would work like that!

[–]storods[S] 1 point2 points  (1 child)

People are still sleeping on some possibilities. Now I'm at a point where I can consistently train architectural styles using dreambooth and apply TI for fine tuning the shape language. When hypernetworks hit the open source road It could be even better. The stuff is already more than production ready for gamedev.

[–]NoHopeHubert 1 point2 points  (0 children)

That’s absolutely wild, thank you for the inspiration! I figured that the mix of DB/TI would ease the burden and “heavy lifting” that the AI has to do to fill in the blanks sometimes within specific parameters.

[–]farcaller899 1 point2 points  (5 children)

This seems like a case in which graduation from common ‘prompt crafter’ to a new kind of ‘digital artist maestro’ has occurred.

Once we start training our own models, then using these tools in this manner, will we then be understood to be artists?

[–]PuddleCheese 1 point2 points  (4 children)

I mean, it's arguably still jacking the fruits of other people's artistic labor to generate an output without the requisite skills needed to produce the work that's being used for a given model in the first place, generally. I wouldn't worry about trying prove what an "Artist" may be, though...

[–]farcaller899 4 points5 points  (0 children)

If photographers are considered artists, I think master AI-using craftsmen can get there too. Not that the term matters, it’s just a conceptual matter.

[–]Collecto 2 points3 points  (2 children)

Yeah I totally get it, I have zero respect for painters that don't craft their own brushes and canvases or paint colors themselves, they are just lazy people free loading on the labor of other skilled crafters

[–]PuddleCheese 1 point2 points  (1 child)

Paintbrush making and canvas pulling are different markets from painting, generally. They have different clientele. They are not directly competing with each other in the same space.

Also painters generally pay for their supplies.

[–]Collecto 1 point2 points  (0 children)

Yeah, manufacturers of RAM and CPU don't compete in the same markets as internet providers and accessories like mouse and keyboards. Also the suppliers of horse hair for brushes probably don't compete in the market of wooden handle manufacturers if they didn't have an end product and only supplied part of the product to a third party.

[–]sergiohlb 0 points1 point  (1 child)

Whats the prompt?

[–]storods[S] 7 points8 points  (0 children)

Prompt: "portrait of bladejoy person in brutalist city in <bladerunner-vegas> style"

keep in mind this is running on a custom model specifically trained for the actress and stlye so It may be different on other generators.

[–]NateBerukAnjing 0 points1 point  (1 child)

can you train art styles with dreambooth?? like you train them to produce artgerm style, because the artgerm with the default sd doesn't look like his style at all

[–]storods[S] 1 point2 points  (0 children)

I've yet to find any evidence of Dreambooth working with styles. The style in my case applies to buildings and is done through TI.

[–]redrobcon 0 points1 point  (2 children)

How do you manage the proportions between the background and the human? (small human in huge Background) The positioning of the human is also good. Can you calibrate it?

[–]faldore 0 points1 point  (0 children)

Img2img or just keep generating until you get one you like

[–]Jujarmazak 0 points1 point  (0 children)

Try some photography terms like "close up" or "wide shot"

[–]Leprechaun72 0 points1 point  (0 children)

What prompts did you use to get such an outfits?

[–]LeKhang98 0 points1 point  (0 children)

Awesome works. May I ask which model of SD did you use to train TI for style? The original 1.5 or the model which you have trained for Blade Runner Joi character or something else? Thank you very much.