×
all 37 comments
sorted by:
best

[–]Leonum 20 points21 points  (6 children)

Very cool. I noticed that the most important part of the meme did not get pickup up though. Line of sight on the characters, and the emotional expressions are what makes this meme, and the AI hasn't been able to replicate that. Very cool pictures, but it's removed the most important 'point' of the source material

[–][deleted] 2 points3 points  (4 children)

Yep I noticed that too. Until we have models and tools that have much better "attention" on individual facial expressions, the best bet is to chop up the meme and focus on each face individually with emotion words in the prompt, then sew it back together.

[–]appenz 1 point2 points  (3 children)

Any reason why you couldn't train a ControlNet model on facial expressions? If it works with pose, expression should work as well.

[–][deleted] 1 point2 points  (1 child)

I'm not familiar with model training but I assume it's possible - you can after all prompt for expressions to a limited extent in regular SD without ControlNet.

But the problem is less the expressions and more that there are three different people in the image. Current techniques (although I may be missing some recent research) don't have good enough attention to focus on a smiling face for the person in the bottom left, an irritated face for the person on the right, and a stunned face for the person in the middle. Usually all of the people end up with the same expression (or in extreme cases, the same face).

[–]appenz 1 point2 points  (0 children)

The idea would be to use prameterized facial expression the same way you use pose. So at position x you have a face with parameters y. That works great for many people (and also makes SD work with very large canvas sizes).

[–]tekmen0 0 points1 point  (0 children)

Problem is data, not models. There may not be enough data to train on facial expression, but I think there is.

[–]notrealAI[S] 4 points5 points  (0 children)

Yes you're spot on! I noticed that right away which is why I continued adding more details to the prompt trying to reproduce the emotions of the original, but I never quite nailed it.

One thing I'd like to try is to use something like BLIP or CLIP interrogator to get a prompt that already matches the source image, rather than trying to manually piece it together.

Also one interesting possibility would be having a ControlNet that reads human facial expressions! That would be awesome.

[–]AhhhGeeze 14 points15 points  (1 child)

The fifth generation got a chuckle out of me 😅

[–]Inprobamur 7 points8 points  (0 children)

The ideal male form.

[–]notrealAI[S] 27 points28 points  (2 children)

Created on Artspark, this was the setup:

sampler: ddim

steps: 35

model: DreamShaper

cfg: 7.5

ControlNet: depth, openpose, segmentation

noise threshold: 15

gfpgan: 0.8

prompt:

(in the style of <david-martinez-cyberpunk>)- ([<bad\_prompt> <bad\_artist> <easy\_negative> grainy blurry]+) jealous girl on the right, boyfriend in center pursing lips, high quality, professional artwork, expressive face, detailed facial expressions, detailed eyes [blurry eyes]

And then I would change out <david-martinez-cyberpunk> any of the other textual inversions on Artspark, which come from the sd-concepts-library:

https://huggingface.co/sd-concepts-library

And here are the negative textual inversions:

https://huggingface.co/datasets/Nerfgun3/bad_prompt

https://huggingface.co/NiXXerHATTER59/bad-artist

https://huggingface.co/embed/EasyNegative

[–]Some_Leek3330 2 points3 points  (0 children)

Thanks, I will try it.

[–]Player13377 0 points1 point  (0 children)

Any way to know waht the art style of picture 7 is called? I really like the look of it

[–]EntrepreneurPlenty17 5 points6 points  (0 children)

Was thinking of if yesterday! Looks like memes from the multiverse kkkk

[–]Corsaer 1 point2 points  (0 children)

Oh man now I can't get out of my head all the memes recreated with the Metalocalypse gang. The fifth panel made me think of that and they don't really look like any of the band members except long hair lol.

[–]Justify_87 1 point2 points  (0 children)

Is there any good and up-to-date tutorial for automatic1111 with controlnet extension?

[–]starstruckmon 4 points5 points  (6 children)

I wish people would take the time to sort through their generations and presenting only the best, instead of dumping it all in the post.

[–]notrealAI[S] 9 points10 points  (3 children)

Can do next time, thank you for letting me know. Even this big batch was pared down from an even larger batch.

[–]FilterBubbles 8 points9 points  (0 children)

For what it's worth, I prefer the format you presented. It gives a better feel of what the workflow can produce.

[–]emotionalfool123 1 point2 points  (0 children)

Show the best in the main post. The dump can be an imgur album for others to explore in the comments.

[–]starstruckmon 0 points1 point  (0 children)

Much appreciated.

[–]InvidFlower 0 points1 point  (2 children)

Is ControlNet a pro-only feature of Artspark? I just checked out the site and don't see an option for it

[–]notrealAI[S] 2 points3 points  (1 child)

Anyone can use it, you just have to be in Img2Img mode or Canvas Mode to see it. Use the tabs on the left to switch the mode, and look for it underneath the Image Strength slider.

[–]InvidFlower 0 points1 point  (0 children)

Oh thank you, that makes sense.

[–]Creative_Praline2105 0 points1 point  (2 children)

Why do most of these look so bad? Is it the position of the people in the background? Seems like the models have a hard time generating in that angle

[–]notrealAI[S] 1 point2 points  (0 children)

I noticed a lot of things that needed improvement. I had difficulty with the eyes, as well as with each person having a distinct facial expression. I also should compare multi control on auto1111 with the same set up just to make sure the implementation is all there.

[–]kawasaki001 0 points1 point  (2 children)

Do you happen to know the embedding for that 7th image? My favorite by far

[–]Apprehensive-Pen1256 0 points1 point  (1 child)

How is this different from just using normal control net? I am sure there’s a trick, but I don’t quite see it.

[–]InvidFlower 2 points3 points  (0 children)

It is just that using multiple modes of controlnet at once can give improved results sometimes.

[–]jsideris 0 points1 point  (0 children)

Only the last one really captured the expression of the jealous girlfriend sort of right.

[–]Tr4sHCr4fT 0 points1 point  (0 children)

I can feel the cringe of 19

[–]Unreal_777 0 points1 point  (0 children)

whast artspark?

[–]C_Sobi 0 points1 point  (0 children)

Would you please tell me how to add to stable diffusion, Multi ControlNet and Artspark?

I ask because this is so cool if one wants to try out a piece in different styles :)

7,12,13,14,16,,18,19 and 20 are my favorites TOP is 16 for me haha.

Great Job, I will have to look into this haha!

[–]WickeDanneh 0 points1 point  (0 children)

Please do this with woman yelling at cat.