“Impossible Hypotheses and Effect-Size Limits”, Wijnand A. P. van Tilburg, Lennert J. A. van Tilburg2023-11-21 (power analysis, probability theory)⁠:

[piranha problem] Psychological science is moving toward further specification of effect sizes when formulating hypotheses, performing power analyses, and considering the relevance of findings. This development has sparked an appreciation for the wider context in which such effect sizes are found because the importance assigned to specific sizes may vary from situation to situation.

We add to this development a crucial but in psychology hitherto underappreciated contingency: There are mathematical limits to the magnitudes that population effect sizes can take within the common multivariate context in which psychology is situated, and these limits can be far more restrictive than typically assumed. The implication is that some hypothesized or preregistered effect sizes may be impossible. At the same time, these restrictions offer a way of statistically triangulating the plausible range of unknown effect sizes.

We explain the reason for the existence of these limits, illustrate how to identify them, and offer recommendations and tools for improving hypothesized effect sizes by exploiting the broader multivariate context in which they occur.

See Also:

• The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases

• Evaluating Effect Size in Psychological Research: Sense and Nonsense

• Psychological testing and psychological assessment: A review of evidence and issues

• When the Numbers Do Not Add Up: The Practical Limits of Stochastologicals for Soft Psychology

• Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology

• Maximal positive controls: A method for estimating the largest plausible effect size

• Estimating the magnitude of experimental effects

• So Useful as a Good Theory? The Practicality Crisis in (Social) Psychological Theory

• The Magnitude Heuristic: Larger Differences Increase Perceived Causality