It turns out that it is possible to get this right with minimal change on the original prompt!
Gary Marcus
 @GaryMarcus
25 May 2022
🙄 @GoogleAI, “a deep level understanding”? Seriously?! Your system can’t distinguish “a horse riding an astronaut” from “an astronaut riding a horse”. 🙄

May 25, 2022 · 4:54 PM UTC

13
45
26
420
The left one is generated by "A horse riding on back of an astronaut" and the right one is generated by "A horse riding on shoulders of an astronaut". So simply adding "on back of" or "on shoulders of" helps increase the chance of getting it right!
3
2
1
107
Another interesting observation is that in case of adding "on shoulders of", the models gets almost all images right because there are more hints in the sentence as opposed to adding "on back of" which is often used for riding a horse.
3
1
60
Obviously, this is not ideal but I think we have to acknowledge the progress!
3
63
Replying to @bneyshabur @markchen90
let's see how fast the goalposts are updated this time
1
43
Replying to @bneyshabur
This seems very familiar to other works which show: breaking down the prompt, and doing it step by step helps the model reach the intended goal. Fascinating 👏👏
1
3
Replying to @bneyshabur
GIF
Replying to @bneyshabur @mentzer_f
It used human legs over his shoulder. Couldn't it interpolate horse legs or weren't there any similar images in its library?
1
Replying to @bneyshabur
I want the right one framed on my wall
4
Replying to @bneyshabur
Very cool. I wonder how we might generally guide people towards using successful prompts—e.g. a GUI for populating a well diagrammed sentence?
4
Replying to @bneyshabur @savvyRL
Hahaha
Load more