The Influence of Individual Opinion on Construction of An Attitude Scale[1]
E. D. Hinckley
Psychological Laboratories of the University of Florida
INTRODUCTION
Until recently, all attempts to measure attitudes have taken the form of questionnaires or rating scales. Although in many cases such methods are sufficiently accurate for the purpose of the study, they do not present us with a refined technique for measurement.
In the publication, "Measurement of Opinion"(6), Thurstone criticizes all previous studies as being contributions of merely bar-diagram representations, which in no case produce a mathematically sound and scientifically accurate scale for measurement. None of the studies has presented us with a yardstick on which the unit of measurement remains the same throughout the scale. Thurstone offers a new method of attack in his articles, "Attitudes Can Be Measured"(3) and "Theory of Attitude Measurement"(8), basing his technique on mathematical laws which he develops in the additional publications: "Psychophysical Analysis"(5), "Law of Comparative judgment" (4), "Three Psychophysical Laws"(9), "Unit of Measurement in Educational Scales"(10), and "The Phi Gamma Hypothesis"(7). On the basis of this technique, three scales have been constructed under Thurstone's supervision :
Attitude toward the Church, by Chave and Thurstone (11),
Attitude toward Prohibition, by Smith (2),
Attitude toward Militarism-Pacifism, by Droba (1).
These scales have already been used to measure differences in opinion between two groups, and also changes in opinion in the same group after certain influences were brought to bear.
In the case of each scale which has thus far been constructed according to Thurstone's technique, it has been assumed that the scale is not influenced by the attitude of the subjects used in the
( 284) construction of the scale. It is the aim of the present research to make a thorough experimental test of this assumption. Stated more definitely, our problem is to construct a scale for measuring attitude toward the Negro, and to test the extent to which this scale is influenced by the opinions of the subjects used in the construction of the scale. (By opinions is here meant opinions as to the social position of the Negro.)
It is obvious that a measuring device must not be seriously affected in its measuring function by the objects which it measures. If an inch were of one size in measuring a desk and of another size in measuring a rug, it would not be a valid unit of measurement. Likewise, an instrument devised for the measurement of a certain attitude must not be affected in its measuring function by the individuals whose attitudes are being measured. To the extent that such a scale is so influenced; its validity is impaired. The completed scale must not only transcend the groups and individuals to be measured, but also the groups and individuals used in the construction of the scale. This premise demands that the scale values of all statements be independent of the attitudes of the individuals or groups who aided in the sorting of the statements.
Let us see how such an influence could be exerted. Each subject in the construction of the scale must judge as to the degree in which each statement reflects the attitude variable. Now, his judgment might possibly be influenced by his own attitude on the question. If this be true of very many of the judges who aid in the construction of the scale, the resulting scale is not a valid measuring device. The discrimination of the value of any two statements at any portion of the scale must be the same for a person whose attitude is toward one extreme of the scale as for one whose attitude is toward the other extreme, or for one who is neutral in respect to the question. For instance, in a scale of attitude toward the church, should about 85%of the antagonistic judges declare statement "a" less antagonistic in attitude toward the church than statement "b," then it would be expected that about 85% of the friendly judges would make the same classification. Furthermore, if a scale for measuring attitude toward militarism-pacifism were constructed using militaristic subjects, it should be possible to measure pacifistic attitude just as accurately as a similar scale constructed by pacifistic judges. The present research is a study of this factor, that is, a study of the manner in which the position of the sorters on an attitude scale affects the validity of that scale.
( 285)
PROCEDURE
The present research consists in the calculation of three independent sets of scale values for certain statements of opinion about the Negro: the first on a group of white southern subjects who were prejudiced against the Negro; the second, on a group of white northern subjects who were prejudiced in favor of the Negro; and the third, on a group of Negro subjects. Each set of scale values was computed in accordance with Thurstone method, and the three sets were carefully compared.
The particular attitude variable which was chosen to be measured is that of the Negro's social rights. This particular attitude variable was chosen because it was sufficiently limited to be measurable and sufficiently debatable to furnish a large range of opinions. Furthermore, groups of subjects were available for the writer's use at the University of Florida and the University of Chicago, where differences of opinion on this attitude variable were sure to occur.
Our first real task was the selection of the statements of opinion as to the social position of the Negro, which opinions would range from the feeling that the Negro should be treated socially on a par with the animal to the attitude claiming social equality with the white man. The statements of opinion were collected from two sources. In the first place, all available literature of the Negro in the libraries of the University of Chicago and the University of Florida was scanned for brief, concise opinions. In the second place, I asked each of 200 students in my psychology classes at the University of Florida, and each of 25 college professors to write his opinion as to the social position of the Negro in as concise and clear a statement as possible. From these two sources were culled 230 statements, ranging between the two extremes of opinion stated before.
It is necessary that the statements which are used in the final scale be clear, concise, unambiguous, and that they contain no irrelevant elements. Consequently, this list of opinions was scrupulously refined according to these criteria. If there was the slightest suspicion of irrelevant elements, the statement was canceled from the list. In this way, the original list was reduced to 150 statements. Copies of this list were then given to Professors Thurstone and Kingsbury, of the Faculty of the University of Chicago, and to two graduate students, all of whom are thoroughly
( 286) familiar with the problem, with the request to check those statements which they felt were ambiguous or contained irrelevant elements. We were especially anxious that a sufficient number of unambiguous neutral statements should remain in the list so that our scale would not be split into the two extremes by the breaking down of the middle range. After all of these things had been taken into consideration, there remained 114 statements, and these make up the final list which was used in the experiment.
The care which was taken in this refining process cannot be overemphasized. The search for statements and the reduction of the original list to 114 statements extended over a period of a year. The efficiency of the execution of this process is mirrored in the uniformity and the relevance of the completed scale.
These 114 statements were then mimeographed on small cards, one statement to a card, for purposes of sorting. Five hundred envelopes were then made up, one envelope for each subject. In each envelope were placed the 114 statement cards, 11 pile-heading cards with the Roman numerals I, II,—XI, one card for the name and classification of the subject, 11 paper clips, and a copy of the directions. Later, more cards were mimeographed, and four hundred more envelopes were filled in the same way.
Each subject was given an envelope and instructed to sort the 114 statements into eleven piles according to the attitudes as to the social position of the Negro which the statements expressed. They were to put in Pile I those statements which place the Negro in the lowest social position, in Pile XI those statements which place the Negro in the highest social position, and in each of the other nine piles those statements which seemed to express that particular step in the social scale. Before starting the task of sorting the statements, each subject was requested to read over carefully the six statements given at the bottom of the instruction sheet, and to check those of the six which he could endorse as expressing most nearly his own attitude toward the Negro. The six statements are as follows :
1.
Inherited qualities have predestined the Negro to
the servant class of society.
2.
The white and colored races should enjoy the same
privileges and protection as set forth by law.
3.
The Negro should he used to produce the white man's
needs.
4.
I do not know enough about the social possibilities of
the Negro
( 287)
to pass judgment upon him.
5.
Our refusal to accept the Negro is not based on any
fact in nature but rather on prejudice, and should be overcome.
6.
So great is the social range between the highly
educated Negro and the 'nigger,' that the race as a whole cannot be assigned to
any one notch in the social scale.
It is clear that those subjects who check the second or fifth of these six statements have an attitude which favors the Negro, and that those who check the first or third hold an attitude which does not favor the Negro, while those who check the fourth or sixth have an attitude of neutrality on the question. The subjects can thus be divided into three groups according to their own attitude toward the social position of the Negro. In addition to the written instructions, the task was explained orally to each subject.
Eight hundred and fifty subjects were recruited from nine schools. Following is a list of the colleges which participated in the experiment:
University of Chicago
Chicago, Illinois
University of Florida
Gainesville, Fla.
Virginia Union University Richmond, Va.
Shaw University Raleigh, N. C.
Morris Brown University Atlanta, Ga.
Morgan College Baltimore, Md.
Lincoln High School
Gainesville, Fla.
Lincoln University
Lincoln Univ., Pa.
Benedict College
Columbia, S. C.
Of these 850 subjects, 600 were white students and 250 were Negroes. Practically all of these subjects were enrolled in some class in psychology, sociology, or education, in which the whole class hour was given over to the task of sorting the 114 statements. Thus practically all of the sorting was personally supervised, either by the writer or by someone who was thoroughly familiar with the problem. Every student was required to stay in the classroom until the end of the hour, even though he had finished the sorting. The writer feels that much more careful sorting was achieved by this method than if the students had sorted the statements outside of class without supervision.
It is not to be expected with such a large number of subjects that
( 288) every subject would understand the directions thoroughly, or that all of those who did understand them would follow them diligently. One tendency which revealed itself in the sortings of some subjects was the bunching of statements in one or more piles to the apparent detriment of the other piles. This phenomenon of bunching at the extremes was noticed in the case of certain of the white subjects, but was especially noticeable in the Negro subjects.. Since the 114 statements are distributed with fair uniformity over the entire scale, marked bunching is a sign of careless sorting. If more than a fourth of the statements are assigned to any one pile, it will leave less than three-fourths to distribute over the remaining ten piles. Furthermore, the individual who sorts the statements in this fashion often ignores some of the piles completely. On the assumption that this bunching was due to poor discrimination and carelessness, every case having 30 or more statements in any one pile was automatically eliminated from consideration, and the results were not recorded. Several other cases were found which clearly demonstrated that the statements had been sorted with no criterion at all, or according to some attitude variable other than the social position of the Negro. These cases were also eliminated from consideration.
The envelopes of the Negro subjects were all placed together for separate consideration, and will be discussed later. The envelopes of the white subjects were divided into three groups, according to the statement on the instruction sheet which had been checked. Group I consists of those who declared themselves against the Negro by checking the first or third statement on the instruction sheet. Group II consists of those who placed the Negro high in the social scale by checking the second or fifth statement on the instruction sheet. In a third group were placed those who expressed an attitude of neutrality by checking the fourth or sixth statement on the instruction sheet. Since this neutral group could have no place in the present research, the envelopes of this group were placed with those others which were eliminated from consideration, and the results were not recorded. A few of the subjects displayed carelessness by checking one friendly and one antagonistic statement, the first and fifth, for example. Their envelopes were likewise eliminated from consideration.
The arrangement of the statements in each envelope in Group I was then recorded according to the following form :
( 289)
When all the envelopes in Group I had been recorded, we thus had the frequency with which each statement had been placed in each pile. These data were then rearranged in the form of a separate frequency table for each statement. The arrangement of the statements in each envelope in Group II was then recorded in the same manner, and a frequency table was made for each statement, as before, showing the frequency with which that statement was placed in each pile by the sorters of Group II.
For each statement, there were then plotted the values in the percentile column, with a smooth percentile curve drawn through them. Since this was done for Group I and Group II 'separately, there are two curves for each statement. For purpose of comparison, the two curves for each statement were drawn on the same coordinate axes. In order to ascertain the proper position of each statement in the scale and its relative degree of ambiguity, the scale value and quartile deviation were calculated from each curve. The scale value is the abscissa of the point where the curve crosses the 50% line. The quartile deviation is half the horizontal distance between the point where the curve crosses the 25% line and the point where the curve crosses the 75% line. The Pearson coefficient of correlation between the series of scale values in Group I and the scale values in Group II was then computed.
( 290)
Although the purpose of the, present investigation could be achieved by a comparison of the results of Group I with the results of Group II, the writer thought that it would be interesting to build up a third series of scale values, using Negroes as subjects. Through the cooperation of certain colleges and universities, named on page 287, it was possible to obtain several competent groups of Negro judges. It was found that most of the colleges offered courses in "Race Relations," "Racial Attitudes," or some related study. For the sake of uniformity, it was considered best to have the sorting supervised, as it was in the case of the other two groups. In excellent and fullest cooperation, the professors in charge gave a part of one class period to preliminary directions, and donated an entire class period to the sorting.
Since the attitude variable is a national as well as a racial attitude, it was deemed advisable to select judges as diverse geographically as expedient, and in this way to lessen possibility of any sectional attitude causing a bias in the scale. It will be observed that seven states are represented, extending as far north as Pennsylvania and Maryland. Although the record of each school was kept separately, there was not a sufficient difference in the distributions to provoke comment.
Reports from all schools but one indicate that the entire class concerned seemed to display a purely scientific interest in the experiment, evidently free from emotional prejudice. Several professors commented on the contributions of the opinions toward later class discussions. The one exception mentioned reported considerable resentment or prejudice toward the investigation. It appears this feeling was engendered between the previous class period and period of the sorting, and that the students concerned had had no opportunity for contact with the actual opinions contained in the individual envelopes. The height of this emotion was to such an extent that about half the class declined to act in the capacity of judges, while the other half consented. Very special attention was given to the comparison of the results of this remaining half, with the results of the other schools, but no indication of unproportional divergence could be detected.
It is not a digression here to note that the few cases admitted from the one high school were entirely satisfactory. It was thought that immaturity might have a different influence on the results in comparison with college students. All girls and boys of the high school in question were above 18 years of age, and seniors.
( 291)
It would appear, then that neither geographic distribution nor sorting under an antagonistic atmosphere produced a noticeable effect. This furnishes us with some evidence on the problem of the present research, since prejudiced subjects seemed to sort about the same as the rest. This is especially significant when it is considered that racial attitudes are perhaps the most emotionally intense of all attitudes, and might be most responsive to sectional bias and emotional heat.
For purposes of tabulation and comparison, the group of Negro subjects was called Group III. The results were recorded and curves drawn, in the same manner as previously described for the other groups. The scale values and Q values were computed for all the statements from these tables and curves, and were properly recorded along with those of the other groups. The Pearson coefficient of correlation between the series of scale values of Group I and those of Group III was then computed, for purposes of comparison.
After a critical examination of the curves and data of Group I, according to Thurstone's method described in "Attitudes Can Be Measured,"32 statements of opinion were chosen as the landmarks in the final scale. It would be possible, in a similar manner, to select a series of evenly spaced statements on the basis of the data for each of the other two groups. It is quite certain that the same 32 statements would not be selected in each case, because of the differences in scale values, although many of the statements would probably be common to all three scales. We-would then have three separate scales for measuring attitude toward the Negro, constructed independently on three different groups of subjects. Since the construction of these three scales would have no bearing on the present problem, only one such scale is presented in this paper, although the writer selected the other two series of statements for his own interest. A better comparison than the construction of three scales is obtained by listing the three sets of scale values and Q values for the 32 statements of the first scale, and also by plotting the three sets on three separate base lines. These methods of comparison were used and the differences will be commented upon later.
For further comparison, the actual differences between the scale values of Group I and Group II were calculated. A frequency distribution of these differences, Group I minus Group II, was made, and the mean and standard deviation of this distribution computed. The relationship between these differences and the scale values of
( 292)
Group I were also studied, as being of possible value to our study. Further proof as to the validity of the scale was obtained by computing the probable error of a scale value in general. This was obtained on the basis of the average Q value, as follows:
This is the probable error of a scale value in general. All of these comparisons and results will be commented on and interpreted in the "Summary and Conclusions."
SUMMARY AND CONCLUSIONS
Following Thurstone's method of attitude scale construction, three sets of scale values for 114 statements have been independently obtained from the sortings of three groups of subjects. Since these groups differ definitely in their opinions as to the social position of the Negro, the crux of our problem lies in the comparison of these sets of scale values. Following are the results of such a comparison :
1.Remarkably close agreement is shown between the curves for Groups I and II, as drawn on the same coordinate axes.
2. The correlation between the scale values of Group I and the scale values of Group II gives a highly linear plot, and a Pearson coefficient of .980. The linearity is evident from the scatter diagram, and it may also be calculated on the basis of the correlation ratios. ηxy = .984; ηyx=.985.
3.The correlation between the scale values of Group I and the scale values of Group III likewise gives a plot which is well within the limits of linearity. r = .935; ηxy = .947; ηyx = .952.
4.A comparison of the scale values which the various groups give
( 293) to the 32 statements of the final scale throws some light on our problem. Although these statements were chosen because of their scale values in Group I, it is significant that Group II places them in the same rank order in which they were placed by Group I, with one exception. Statement 95 is the only one of the 32 statements which has a different relative position in Group II from what it has in Group I. This means that our scale, which was constructed on the basis of the sortings of Group I, can measure the attitudes of Group II as well a it can the attitudes of Group I. This is very definite evidence in favor of the objective validity of the scale..
5.The very low value obtained for the probable error of a scale value in general gives us increased confidence in the calculated scale values.
6. The frequency distribution of differences between the scale values of Group I and the scale values of Group II gives a mean of .27 and a standard deviation of .55. This constant error reveals a slight tendency for Group I, which is prejudiced against the Negro, to judge a given statement to be more favorable to the Negro than Group II judged it. When we tabulate the differences between the scale values of Groups I and II for only the 32 statements in the completed scale, we obtain a mean of only .17. This is Only one and a half times the probable error of a scale value, and could very possibly be due to the fluctuations of chance. At any rate, the fact that the correlation between the two groups is so high, together with the other evidence presented, suggests that this constant error is not significant.
7.The scatter diagram of the relation between the differences (Group I — Group II) and the scale values of Group I show no tendency for the differences to vary with the scale value.
8.It will be recalled that one of the Negro colleges became emotionally wrought up concerning the sorting task, but that their sorting was about the same as that of the other subjects. This furnishes further evidence in favor of the fact that emotional prejudice does not influence the differentiating judgment of the sorter.
On the basis of these points of comparison, we conclude that the scale which we have constructed for measuring attitude toward the social position of the Negro is not influenced in its measuring function by the attitudes of the subjects used in the construction. We
( 294) have definitely shown this conclusion to be true for those districts where the social position of the Negro is an issue. It is possible that this would not hold for countries where the social position of the Negro it not an issue. We conclude, in general, that any scale of attitudes, carefully constructed in accordance with Thurstone's method, will 'not be affected in its measuring function by the position which the sorters occupy on the scale.
It is to be noted that we have been concerned here only with the manner in which the position of the sorters on an attitude scale affects the validity of that scale. We must not confuse this with another factor, which may possibly affect the validity of an attitude scale. This factor is the difference in the standards of evaluation of the statements used by different sorters. To sit down and eat with a Negro might indicate the height of social recognition to one sorter, and indicate a less extreme attitude to another sorter who lives in a community where eating with Negroes is customary. This may be true even though the two sorters occupy approximately the same position on the attitude scale. It might therefore be possible, for a given series of statements, to obtain two quite different sets of scale values, based upon the sortings of two groups who differ in their standards of evaluation of the statements. Although this problem is beyond the scope' of the present research, the writer hopes soon to study the influence of this factor upon the validity of an attitude scale.
REFERENCES
1. DROBA, D. Measurement of attitude toward Militarism-Pacifism. (Unpublished.)
2. SMITH, H. Measurement of attitude toward prohibition. (Unpublished.)
3. THURSTONE, L. L. Attitudes can be measured. Amer. J. Social., 1928, 33, 529-554.
4. ———. Law of comparative judgment. Psychol. Rev., 1927, 34, 273-286.
5, ———. Psychophysical analysis. Amer. J. Psychol., 1927, 38, 368-389.
6. ———. The measurement of opinion. J. Abn. & Soc. Psycho!., 1928, 22, 415-430.
7. ———. The Phi Gamma hypothesis. J. Exper. Psychol., 1928, 11, 293-305.
8. ———. Theory of attitude measurement. Psychol. Rev., 1929, 36, 222-241.
9. ———. Three psychophysical laws. Psychol. Rev., 1927, 34, 424-432.
( 295)
10 ———. Unit of measurement in educational scales. J. Educ. Psychol., 1927, 18, 505-524.
11. THURSTONE, L. L. & CHAVE, E. J. Measurement of attitude toward the church. (Unpublished.)
University of Florida
Gainesville, Florida
