“Generative AI Mode Collapse”, Gwern2022-11-28 (similar)⁠:

A side effect of preference-learning approaches like RLHF is a severe loss of ‘diversity’ or ‘creativity’ in outputs, analogous to broader mode collapse in generative models.