[1] In other words, it is the extent to which the results of a study can generalize or transport to other situations, people, stimuli, and times.
That requires a test of whether the treatment effect being investigated is moderated by interactions with one or more background factors.
Using graph-based causal inference calculus,[9] they derived a necessary and sufficient condition for a problem instance to enable a valid generalization, and devised algorithms that automatically produce the needed re-calibration, whenever such exists.
The main difference between generalization from improperly sampled studies and generalization across disparate populations lies in the fact that disparities among populations are usually caused by preexisting factors, such as age or ethnicity, whereas selection bias is often caused by post-treatment conditions, for example, patients dropping out of the study, or patients selected by severity of injury.
[12][13] If age is judged to be a major factor causing treatment effect to vary from individual to individual, then age differences between the sampled students and the general population would lead to a biased estimate of the average treatment effect in that population.
Such bias can be corrected though by a simple re-weighing procedure: We take the age-specific effect in the student subpopulation and compute its average using the age distribution in the general population.
If, on the other hand, the relevant factor that distinguishes the study sample from the general population is in itself affected by the treatment, then a different re-weighing scheme need be invoked.
In other words, the new weight is the proportion of units attaining level Z=z had treatment X=x been administered to the entire population.
Suppose that subjects selected for the experimental study tend to have higher cholesterol levels than is typical in the general population.
They criticize the lack of ecological validity in many laboratory-based studies with a focus on artificially controlled and constricted environments.
There are two kinds of generalizability at issue: However, both of these considerations pertain to Cook and Campbell's concept of generalizing to some target population rather than the arguably more central task of assessing the generalizability of findings from an experiment across subpopulations that differ from the specific situation studied and people who differ from the respondents studied in some meaningful way.
However, if one's goal is to understand generalizability across subpopulations that differ in situational or personal background factors, these remedies do not have the efficacy in increasing external validity that is commonly ascribed to them.
If one's study is "unrealistic" on the level of some background factor that does not interact with the treatments, it has no effect on external validity.
[6] Research in psychology experiments attempted in universities is often criticized for being conducted in artificial situations and that it cannot be generalized to real life.
[21][22] To solve this problem, social psychologists attempt to increase the generalizability of their results by making their studies as realistic as possible.
To accomplish this, researchers sometimes tell the participants a cover story—a false description of the study's purpose.
If however, the experimenters were to tell the participants the purpose of the experiment then such a procedure would be low in psychological realism.
Several experiments have documented an interesting, unexpected example of social influence, whereby the mere knowledge that others were present reduced the likelihood that people helped.
Some social psychologist processes do vary in different cultures and in those cases, diverse samples of people have to be studied.
To make sense out of this, there is a statistical technique called meta-analysis that averages the results of two or more studies to see if the effect of an independent variable is reliable.
A meta analysis essentially tells us the probability that the findings across the results of many studies are attributable to chance or to the independent variable.
For example, increasing the number of bystanders has been found to inhibit helping behaviour with many kinds of people, including children, university students, and future ministers;[25] in Israel;[26] in small towns and large cities in the U.S.;[27] in a variety of settings, such as psychology laboratories, city streets, and subway trains;[28] and with a variety of types of emergencies, such as seizures, potential fires, fights, and accidents,[29] as well as with less serious events, such as having a flat tire.
Social psychologists opt first for internal validity, conducting laboratory experiments in which people are randomly assigned to different conditions and all extraneous variables are controlled.
Other social psychologists prefer external validity to control, conducting most of their research in field studies, and many do both.