Forking paths problem

[1] Exploring a forking decision-tree while analyzing data was at one point grouped with the multiple comparisons problem as an example of poor statistical method.

However Gelman and Loken demonstrated[2] that this can happen implicitly by researchers aware of best practices who only make a single comparison and only evaluate their data once.

The fallacy is believing an analysis to be free of multiple comparisons despite having had enough degrees of freedom in choosing the method, after seeing some or all of the data, to produce similarly-grounded false positives.

This approach is valuable in fields where research findings are sensitive to the methods of data analysis, such as psychology,[4] neuroscience,[5] economics, and social sciences.

Multiverse analysis aims to mitigate issues related to reproducibility and replicability by revealing how different analytical choices can lead to different conclusions from the same dataset.