×
all 69 comments
sorted by:
best

[–]newsletternew[S] 64 points65 points  (13 children)

tl;dr: SDXL recognises an almost unbelievable range of different artists and their styles. The skilled prompt crafter can break away from the "usual suspects" and draw from the thousands of styles of those artists recognised by SDXL. I have tried out almost 4000 and for only a few of them (compared to SD 1.5) were images produced that did not show any significant differences between them.

It is often said that AI art can be recognised by its generic nature. This is hardly surprising, since there is often a tendency to simply copy the names of a few artists for "aesthetic" images and use them again and again, sometimes individually, sometimes in combination with other equally common names. In fact, however, this is a tiny real subset!

As an antithesis, I would like to present an admittedly opinionated selection of an extremely comprehensive comparison showing a wide variety of artists in SDXL 0.9.

Even the awesome "Deliberate", which is based on SD 1.5 and should indeed be understood here as a proxy for SD 1.5, does not recognise all artists. At present we can only be patient until someone produces a similar finetuning/merge model for SDXL that is a similar improvement to what Deliberate was for SD 1.5. Let's wait and see!

​

The generation of the corresponding images was quite lengthy due to the extensive amount of different artists. Therefore, only the base model was used, but not the refiner, to save a lot of time. Moreover, the refiner does not change whether an artist is recognised. For each artist, an image is thus generated with the following parameters:

- Resolution of 1024x1024, DPM++2m sampler, 50 steps, CFG 7, Seed 1

- The conditioning for each image and any artist (<ARTIST>) was done with the promt:

"art by <ARTIST>", the negative conditioning was "ugly, low quality".

​

After that, I tried to select as wide a range of different artists with different degrees of recognition as possible. At least that was the plan, but reddit limits me to 20 images, so I chose the ones I liked best, so you may discover an artist here that you like but have never heard of before!

For my part, I really like science fiction, so artists from that genre will get their own comparison.

One may get an impression of the manifold possibilities beyond the overused use of certain artists who were really relevant in the first era of Stable Diffusion.

​

All images of all 3887 artists can be found here:

https://huggingface.co/datasets/newsletter/SDXL-Artists

Comprehensive lists of artists are available, some of which offer filtering by genre or other aspects. I have used the following:

https://proximacentaurib.notion.site/e28a4f8d97724f14a784a538b8589e7d

Should any of the creators of this wonderful work read this (or anyone else who feels called) and find a great deal of time to re-examine the recognised artists of SD 1.5 for SDXL, this would probably be one (if not: the) most valuable resource when it comes to picking a pleasing style and standing out from the generic one-artist-fits-all of AI-Art, which only knows how to use a few artists and is therefore quickly recognised even by laymen.

​

A few observations:

- Lettering has become so good in places that in some pictures you can (almost) make out the artist's name.

- The consistency of light sources (highlights and shadows) has improved if you pay attention.

- the method known as "noise offset" is no longer an add-on, but built right in from the start.

- The new VAE is terrific and should hopefully eliminate the tiresome issue of "grey images" or "which VAE to use?" for merges based on SDXL.

- Oil painting has gotten even better and with a little work (and luck) is almost indistinguishable from real paintings, especially landscapes.

- Comics and especially different artists are produced much better than was the case in SD 1.5 especially for lesser known artists.

- Manga/anime works quite well and at least better than in SD 1.5/2.1, but is no comparison to the models explicitly trained for it.

- Accidental nudity is extremely rare, even for respective artists. SDXL seems to be much less horny here! ;-)

- Hands are still a problem, although not as severe as in SD 1.X/2.X

- The correct spelling of artists or their full name is not absolutely necessary: "Durer, Stalenhag, Friedensreich Hundertwasser" already works fine, instead of "Dürer, Stålenhag, Friedensreich Regentag Dunkelbunt Hundertwasser".

- certain styles (especially "Kurzgesagt"), for which there used to be a separate model and later at least one LORA, are already recognised by SDXL.

- last, but not least: all artists can be combined, creating a combinatorial complexity beyond all realistic comparability!

​

Considering the possibilities of SDXL, this is probably the leap in virtually all important points that SD 2.0 unfortunately was not.

Comparable commercial offerings cannot do this at a reasonable cost (but, to be fair, they are not designed to do so), not to mention the lack of ControlNet or LORA. Anyone who wants to can calculate what this comparison of about 4000 artists' names would have cost at MidJourney.

[–]demoran 13 points14 points  (3 children)

Great stuff.

In the future, there are a few optimizations you can do to reduce the amount of time this takes to run.

  1. Reducing the number of steps. With the sampler you were using, you should see pretty much the same result at 35 steps, but this can be reduced to half that easily and still provide a great image.
  2. Reducing the image size. I know, I know, we all love that you can gen without twinning and going crazy in sdxl at a higher res, but this isn't required to test whether an artist is recognised.

[–]newsletternew[S] 9 points10 points  (0 children)

In fact, it occurred to me too late to reduce the number of steps. By then (barely) half were already done and perfectionism forbade me to start all over again.

But next time, for sure!

[–]Etsu_Riot 0 points1 point  (1 child)

Is this because of SDXL, or the sampler? Because in SD1.5 I get radically different results at different steps count.

I will have to try this when I get home.

[–]demoran 0 points1 point  (0 children)

Some samplers are "converging", meaning that as you increase the steps, they will refine the image towards a certain ideal. At some point, no more changes are effectively made. In the case of DPM++ 2M Karras, this is at about 35 steps. I refer to this as being "fully baked".

Other samplers will produce radically different images at different step counts. These are the non-converging samplers.

For more info (and this is not my reference document, but SiliconThaumaturgy's), see https://docs.google.com/spreadsheets/d/1YkX1pzJvYKrj_w6fhrpyaTyZvR687THp7QkxOVI4qEg/edit#gid=1511867707, and https://www.youtube.com/watch?v=gtr-4CUBfeQ

I base the claim of 35 steps on the following

<image>

[–]barepixels 8 points9 points  (2 children)

What a fantastic research and share. Please consider making an archive to download. Having to click download 3887 times is a major chore for everyone

[–]newsletternew[S] 7 points8 points  (1 child)

That's right! I had thought of "git lfs clone", but I just realised that I also want to have the images on my phone. Here is the link to the archive:

https://huggingface.co/datasets/newsletter/SDXL-Artists/resolve/main/data/train.zip

[–]barepixels 1 point2 points  (0 children)

Thanks so much

[–]proximasan 2 points3 points  (3 children)

hey, great work and thanks for the mention! we have started testing again but paused for the final release of sdxl 1.0: https://sdxl.parrotzone.art

edit: these were made 3 months ago

[–]newsletternew[S] 2 points3 points  (0 children)

You’re very welcome, I hold the Stable Diffusion V1 Artist Style Studies in high esteem, especially "Strength" and "Tags" as well as the filtering make it an indispensable resource. I am looking forward to the release of SDXL 1.0 and your new Artist Style Studies XL!

[–]Alzakex 1 point2 points  (1 child)

You and the rest of the Parrot Zone crew are absolute heroes for your V1 Artist Style Studies. Before Have I Been Trained opened to the general public, y'all were the absolute GOAT site for finding new inspiration, figuring out what SD might do with my prompts, and why it wouldn't do what I wanted. Even after, you are still the best game in town. I wish Google crawled your site better - for a while there I forgot to bookmark it, or the bookmark would redirect to Notion, and I couldn't find you. I was thoroughly chuffed to find your SDXL site a couple of weeks ago, particularly because it has an easy URL to remember!

Is there a web portal for SDXL now that people without epic PCs can use to help you work through the 2800 artists you haven't already done? I have dibs on Patrick Nagel and Tamara De Lempicka if so!

[–]Alzakex 1 point2 points  (0 children)

I'd also love to help you fix some of the spots where one artist was mistaken for another, e.g. where Don Northup was misspelled "Northrup," so his page looks like Tony Northrup's art. In fact, there's a whole row of N people with Tony Northrup's art instead of theirs.

[–]lexcess 1 point2 points  (0 children)

Interesting on the VAE, the latest version of Vlad's fork allows 1.5 generations to use the 0.9 VAE. I thought that was just a frivolous detail, but maybe worth looking into more.

[–]IAmARobot 0 points1 point  (0 children)

If anyone wants a textish list instead of d/ling a GB of images you can get one via the google spreadsheet from op's proxima link above, then file->download as csv and do some text processing on it to extract the list of artists

[–]antonio_inverness 25 points26 points  (5 children)

Yeah, but does it include Greg Rutkowski?

[–]newsletternew[S] 0 points1 point  (1 child)

Haha, I checked 2 times because I wanted to leave Greg out for fun at first. ;-)

[–]antonio_inverness 0 points1 point  (0 children)

If it ain't got The Kowsk, it ain't got shit lol

[–]Tr4sHCr4fT 14 points15 points  (3 children)

Kurzgesagt should be flatter tho

[–]newsletternew[S] 5 points6 points  (1 child)

The use of "vector art" captures the style better, but not quite yet... There is still hope for it in SDXL 1.0! I bet there will be a LoHa/LyCORIS for it pretty soon.

​

<image>

[–]StickiStickman 3 points4 points  (0 children)

Yea, that barely looks anything like Kurzgesagt

[–]PenAndInkAndComics 7 points8 points  (1 child)

If you can settle on just one. The same prompt rendered in each style might be insightful.

[–]newsletternew[S] 1 point2 points  (0 children)

Honestly: I tried "art", "concept art", "illustration" and a few others and in the end just randomly chose the shortest one: "art, by <artist>" The decision was difficult ;-)

By the way, with longer prompts, the influences of some artists disappear (even with "(artist:1.5)").

[–]lordpuddingcup 4 points5 points  (9 children)

The real question is what artists doesn’t it know any longer

[–]LightVelox 5 points6 points  (1 child)

Apparently mostly manga and anime artists since it completely sucks at doing either, actually i think pretty much every AI sucks at drawing anime and manga without LoRAs, they just try to replicate the generic anime artstyle and get confused when you ask for something specific, with the exception of studio ghibli

[–]Apprehensive_Sky892 0 points1 point  (0 children)

It's interesting that even though their styles are poorly replicated, the use of a manga or anime artist's name often affect the composition in subtle ways.

For example, if you use "Akira Toriyama", you often get a more dynamic, exaggerated action. "Katsuhiro Otomo" will often add a sci-fi feel to the image, etc.

From the tests I've run, the two artists that have the strongest influence on style are Makoto Shinkai and Masamune Shirow.

[–]Striking-Long-2960 2 points3 points  (0 children)

I had a hard time with modern caricaturists like Paul Moyse. It seems it's not trained in that kind of realistic caricature style, or at least I couldn't find a way to replicate that style.

[–]newsletternew[S] 1 point2 points  (4 children)

Cory Loftis, for example, but that had been more of a chance discovery, which disappointed me, though, because Deliberate recognised him....

However, his portfolio on ArtStation is also empty, maybe that's the reason? In any case, it's a pity!

[–]lordpuddingcup 5 points6 points  (0 children)

Well they had an opt out phase for artists to have their data removed also so there’s going to be gaps but dunno what those will be

[–]Pretend-Marsupial258 1 point2 points  (1 child)

Did you try online-only artists, or is it mostly gallery artists?

[–]newsletternew[S] 0 points1 point  (0 children)

I hope that the list I have used represents as broad a range as possible, so something from everyone.

[–]LD2WDavid 0 points1 point  (0 children)

After reading that guy on IG on ArtStation months ago I stopped supporting him on medias and merch. Is one who is missinformating on IG with the awful devil firefly example. Couldn't care less but he can be trained back on. However I have 0 interests on guys like that.

[–]Vozka 0 points1 point  (0 children)

In my experience the answer is "many", but I didn't create notes. Sometimes with living artists it creates a portrait of someone who looks vaguely like the artist instead of creating something that looks vaguely like their art.

[–]EddieGoldenX 3 points4 points  (0 children)

Awesome work u/newsletternew! I noticed also the power of SDXL with a Dave Gibbons style visual I generated today. It recognized the style perfectly
I'd love to see a visual guide like you have for SD V1 when you can make one please with all 4k artists with their tags. It would be such a magical tool to use

<image>

[–]TrevorxTravesty 3 points4 points  (2 children)

Just a heads up, but the Huggingface link doesn’t show all the images 😞 It stops at Alessandro Allori and it says ‘The full dataset viewer is not available (click to read why)’ and the error is: Error occurred while generating the dataset Error Code: UnexpectedError. Hopefully it gets fixed soon because I’d love to see the rest 😊

[–]newsletternew[S] 1 point2 points  (0 children)

UnexpectedError

I have opened a ticket and hope that I can be helped, because the error is really strange.

[–]Jiboxemo2 5 points6 points  (0 children)

Ah, yeah, Studio Ghibli and Totoro. The difficult thing is to make a Studio Ghibli without Totoro.

[–]revolved 3 points4 points  (1 child)

Been having a ton of fun with artists. I would note that in 0.9 some prompts can crush your artist stylization (especially in the "illustration" or "painting" vein). Seems like an overfit area. However underneath that the artist styling is INCREDIBLY responsive!

[–]newsletternew[S] 2 points3 points  (0 children)

Yes, especially the frequently used beautifier voodoo prompts like "high res, 4K, 8K" can make an artist's style almost completely disappear.

[–]whaleofatale2012 2 points3 points  (0 children)

I have scrolled through the gallery of images (thank you for sharing this research), and noticed that some of the artists' work looks very much alike.

Based on my own research in 1.4 and 1.5, I believe that the reason for the similarity is that there are not many distinctive artworks sampled in the training set. Basically, the closer an image looks to generic art in the system, the fewer the weights for that particular artist.

As you said in your post, the prompt was: "art by <ARTIST>", and the negative conditioning was "ugly, low quality".

I believe that if you were to remove <ARTIST>, and simply prompt, "art by", with the same negative prompt, you'd get a generic result that would be pretty close to the image embedded below.

If someone has better understanding and information, please correct me.

I noticed that the following artists have art that looks very similar: Soraya Saga, Travis Louie, Victor Adame Minguez, Victor Medina, Victoria Frances, Alberto Dros, Alexandre Jacovleff, Allen Wiliams, Anato Finnstak, Anthony S. Waters (pictured) and quite a few more.

I interpret this similarity as the closer the render looks to the pictured image (chest up portrait of beautiful woman with dark skin and dark hair, with hands clasped, facing left), the weaker the style will be in the model. Some artists show a light-skinned woman with lighter hair, or an Asian woman with black hair, or light skin with red hair, etc., but they are still facing left, in a chest-up portrait, with clasped hands.

Contrarywise, the less the gallery art looks like the image shown, the stronger the artist style will be in your renders.

<image>

[–]okamiueru 8 points9 points  (3 children)

Imagine devoting your life to your craft, honing in on a style that sets you apart, insisting on it and making an identify out of it. Prevailing when it is entirely likely to be ignored along with almost everyone who try. Then, some technology comes along, and your name, your style, is now a keyword for a program to replicate it.

Whenever I think and wonder "is this really fair?", I think about how the music industry would pounce on anyone even considering doing something similar for established artists. "Recreate Shake it Off, but by Michael Jackson but in the style of fusion jazz". I wouldn't be surprised if this didn't already exist to some extent, but I haven't been paying attention.

On the other hand, it's a tremendous technology. The genie is out of the bottle. And, for better or worse, this is what it is. Companies with "black box" training data will always exist. Anyone who claim they do not use any copyrighted material can just lie.

I'm old enough to have experienced two such revolutions. The internet, and now the LLM and diffusion based technologies. It's exciting to say the least.

[–]Yglorba 7 points8 points  (0 children)

Whenever I think and wonder "is this really fair?", I think about how the music industry would pounce on anyone even considering doing something similar for established artists. "Recreate Shake it Off, but by Michael Jackson but in the style of fusion jazz". I wouldn't be surprised if this didn't already exist to some extent, but I haven't been paying attention.

Maybe, but is the litigious nature of the music industry actually something we want to replicate elsewhere? I don't see the fact that the music industry reacts like that as a good thing, and I don't think it's actually something that has benefited most artists - it's a result of basically the entire industry being owned by a few megacorps who zealously crush any threat to their hegemony.

[–]kruthe 0 points1 point  (0 children)

Imagine devoting your life to your craft, honing in on a style that sets you apart, insisting on it and making an identify out of it. Prevailing when it is entirely likely to be ignored along with almost everyone who try. Then, some technology comes along, and your name, your style, is now a keyword for a program to replicate it.

It's called photography. Before that it was called the printing press. Technology changes the world. Art isn't immune to that.

You want to make a living in business in the face of technological shifts then you pivot that which the machines cannot make or the customer prefers artist made. Glassware is a perfect example of something that has been technically perfected for a long time. You will get a superior piece off an assembly line every time for cheap. You want a hand blown object? Get out your wallet because you're gonna need it. Even vintage glassware can provoke a knife fight in a second hand store.

Artists are frequently shitty businessmen. If you are a sole trader (which most artists are) then competing on volume is going to be a mistake. Art as a product (because art is also a financial instrument) is a Veblen good. Having a computer system vomit out work after work is not competition to that exactly because the machine is so productive. This is comparing a plastic bag to a Louis Vuitton. Scarcity is value. Simply saying "You can't have it" creates product desire out of thin air. Yes, you still have to make a quality product and build up the reputation but the second there's a waiting line you've effectively made it.

[–]collegedude42069 0 points1 point  (0 children)

I think about how the music industry would pounce on anyone even considering doing something similar for established artists. >"Recreate Shake it Off, but by Michael Jackson but in the style of fusion jazz". I wouldn't be surprised if this didn't already exist to some extent, but I haven't been paying attention.

https://www.youtube.com/watch?v=t55n0P64FFE

its happening

[–]jib_reddit 3 points4 points  (0 children)

Good for us, but I can see why some of the artists featured are not very happy about it right now.

[–]Striking-Long-2960 1 point2 points  (0 children)

Many thanks.

[–]ura2309 1 point2 points  (2 children)

Hi, great work! Do you have some archive, so I can download all files? There is no way to download all pictures from hugginface together, only one by one.

[–]GabberZZ 1 point2 points  (0 children)

Would love to see one trained with the late Govinder Nazran.

[–]Muchablat 1 point2 points  (0 children)

Thomas Kinkade would be cool!

[–]wolve202 1 point2 points  (0 children)

But can it recognize why kids love Cinnamon Toast Crunch?

[–]Katana_sized_banana 1 point2 points  (0 children)

impressive post/work of you and SDXL

[–]SoupIcy1310 1 point2 points  (0 children)

You mean, Alex Ross xd

[–]Vyviel 1 point2 points  (0 children)

Can it do Tintin Georges Prosper Remi

[–]furrypony2718 1 point2 points  (0 children)

I love the Ernst Haeckel one omg. The xenobiologists would love it.

[–]Careful_Ad_9077 1 point2 points  (0 children)

I would like to see the prompts, my experience with this is that the art style Breaks once you start describing a scene.

[–]Current-Rabbit-620 1 point2 points  (0 children)

Thanks v much

[–]Apprehensive_Sky892 1 point2 points  (0 children)

Outstanding work! Thank you for sharing it.

[–]terrariyum 1 point2 points  (0 children)

Interesting - a huge number of these images features a rainbow hot air balloon. Or if not that, a rainbow-striped object in the top center. Including your cover image by Alex Gross.

I'm not sure what the implications of that are

<image>

[–]Sure-Company9727 0 points1 point  (0 children)

I noticed that a lot of the artists who share a first name have output that looks more similar than you would expect. Why do you think that is? Are the first and last names being treated like separate tokens?

[–]terrariyum 0 points1 point  (0 children)

Thanks OP!

I made this webpage that makes it easy to view and these images and search by artist name.

That link is a downloadable HTML file for viewing offline. That html file doesn't contain the original images. They must be downloaded separately from the newsletternew's huggingface.