JANUARY 1999

The Specificity of Norms
by John Martino

Any norm, however, expressed, is restricted to the particular normative population from which it was derived. We should never lose sight of the way in which norms are established. Test norms are in no sense absolute, universal, or permanent. They merely represent the test performance of those in the standardization sample. In choosing such a sample, an effort is usually made to obtain a representative cross section of the population for which the test is designed.

In the development and application of test norms, considerable attention should be given to the standardization sample. It is apparent that the sample on which the norms are based should be large enough to provide stable values. Another similarly chosen sample of the same population should not yield norms that diverge appreciably from those obtained. Norms with a large 3sampling error2 would obviously be of little value in the interpretation of test scores.

Equally important is the requirement that the sample be representative of the population under consideration. Subtle selective factors which might make the sample unrepresentative should be carefully investigated. A number of such selective factors are illustrated in institutional samples. Since such samples are usually large and readily available for testing purposes, they offer an alluring field for the accumulation of normative data. The special limitations of these samples should, however, be carefully analyzed. Testing subjects in school, for example, will yield an increasingly superior selection of cases in the successive grades, owing to the progressive dropping out of the less able pupils.

Selective factors likewise operate in other institutional samples, such as prisoners, patients in mental hospitals, or institutionalized feebleminded cases.

Closely related to the questions of representativeness of sample is the need for defining the specific population to which the norms apply. Obviously, one way of insuring that a sample is representative is to restrict the population to fit the specifications of the available sample. For example, if the population is defined to include only primary care physicians, rather than all specialties, then a random sample would be representative of primary care physicians. Ideally, of course, the desired population should be defined in advance in terms of the objectives of the test. Then a suitable sample should be assembled. Practical obstacles in obtaining subjects, however, often make such a goal unattainable. In such a case, it is far better to redefine the population more narrowly than to report norms on an ideal population which is not adequately represented by the standardization sample. In actual practice, very few tests are standardized on such broad populations as is popularly assumed. No test provides norms for the human species! And it is doubtful whether any tests give truly adequate norms for such broadly defined populations as "adult American men," "American 25 year-old homemakers," and the like.

For many purposes, however, highly specific norms are desirable. Thus, even when representative norms are available for a broadly defined population, it is often helpful to have separately reported subgroup norms. This is true whenever recognizable subgroups yield appreciably different scores. The subgroups may be formed with respect to age, sex, geographical region, urban or rural environment, socioeconomic level, specificity and many other factors. The use to be made of the test determines the type of separation that is more relevant. It has been suggested that, ideally, separate norms should be provided "for every conceivable kind of group with which those who take the tests are likely to compete."

In summary, normative populations should be clearly defined. The characteristics of such populations should be taken into consideration in interpreting test scores. For many purposes, moreover, specific norms, based on more narrowly defined normative populations, are more useful than general norms.