“HyperFields: Towards Zero-Shot Generation of NeRFs from Text”, Sudarshan Babu, Richard Liu, Avery Zhou, Michael Maire, Greg Shakhnarovich, Rana Hanocka2023-10-26 (MLP NN, knowledge distillation, meta-learning)⁠:

We introduce HyperFields, a method for generating text-conditioned Neural Radiance Fields (NeRFs) with a single forward pass and (optionally) some fine-tuning. Key to our approach are: (1) a dynamic hypernetwork, which learns a smooth mapping from text token embeddings to the space of NeRFs; (2) NeRF distillation training, which distills scenes encoded in individual NeRFs into one dynamic hypernetwork.

These techniques enable a single network to fit over a hundred unique scenes. We further demonstrate that HyperFields learns a more general map between text and NeRFs, and consequently is capable of predicting novel in-distribution and out-of-distribution scenes—either zero-shot or with a few finetuning steps.

Finetuning HyperFields benefits from accelerated convergence thanks to the learned general map, and is capable of synthesizing novel scenes 5 to 10× faster than existing neural optimization-based methods. Our ablation experiments show that both the dynamic architecture and NeRF distillation are critical to the expressivity of HyperFields.