“Machine-Assisted Social Psychology Hypothesis Generation”, Sachin Banker, Promothesh Chatterjee, Himanshu Mishra, Arul Mishra2023-07-13 (GPT-3 nonfiction, GPT-4 nonfiction, psychology; similar)⁠:

This work illustrates how large language models (LLMs), such as GPT-3 and GPT-4, can be used as an aid to generate research hypotheses for social psychology.

The LLM-generated hypotheses were found to be on par with, or even better than, those written by human researchers.

As research findings proliferate, these LLMs can help streamline the process of creating testable ideas and offer new avenues to accelerate psychological research.

---

Social psychology research projects begin with generating a testable idea that relies heavily on a researcher’s ability to assimilate, recall, and accurately process available research findings. However, an exponential increase in new research findings is making the task of synthesizing ideas across the multitude of topics challenging, which could result in important overlooked research connections.

In this research, we leverage the fact that social psychology research is based on verbal models and employ large natural language models to generate hypotheses that can aid social psychology researchers in developing new research hypotheses. We adopted two methodological approaches. In the first approach, we fine-tuned the third-generation generative pre-trained transformer (GPT-3) language model on thousands of abstracts published in more than 50 social psychology journals in the past 55 years as well as on preprint repositories (PsyArXiv).

Social psychology experts rated model & human-generated hypotheses similarly on the dimensions of clarity, originality, and impact. In the second approach, without fine-tuning, we generated hypotheses using GPT-4 and found that social psychology experts rated these generated hypotheses as higher in quality than human-generated hypotheses on dimensions of clarity, originality, impact, plausibility, and relevance.

[Keywords: generative language models, deep learning, hypothesis formation, generative networks]

See Also:

• Automated Social Science: Language Models as Scientist and Subjects

• Using Large Language Models to Simulate Multiple Humans

• Large language models are able to downplay their cognitive abilities to fit the persona they simulate

• LLMs achieve adult human performance on higher-order theory of mind tasks

• Understanding Social Reasoning in Language Models with Language Models

• Large Language Models as Superpositions of Cultural Perspectives

• Assessing the nature of large language models: A caution against anthropocentrism

• Can a computer outfake a human [personality]?

• Social Simulacra: Creating Populated Prototypes for Social Computing Systems

• LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs

• Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews

• Can large language models provide useful feedback on research papers? A large-scale empirical analysis

• Language models accurately infer correlations between psychological items and scales from text alone

• Machine Learning as a Tool for Hypothesis Generation

Similar Links: