“3DALL·E: Integrating Text-To-Image AI in 3D Design Workflows”, Vivian Liu, Jo Vermeulen, George Fitzmaurice, Justin Matejka2022-10-20 (CLIP, DALL·E 2)⁠:

[Autodesk] Text-to-image AI systems are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration is less understood. To investigate this, we integrated DALL·E 2 API, GPT-3 API, and CLIP within a CAD software in 3DALL·E, a plugin that allows users to construct text and image prompts based on what they are modeling.

In a study with 13 designers, we found that designers saw great potential to incorporate 3DALL·E into their workflows and to use text-to-image AI for reference images, renders, materials, and design considerations. Additionally, we elaborate on prompting patterns and provide measures of prompt complexity observed across participants.

We conclude on a discussion of how 3DALL·E can merge with existing generative design workflows and propose prompt bibliographies as a form of human-AI design history.