The Form of Statements in Attitude Tests
Daniel H. Kulp, II
Teachers College, Columbia University
INTRODUCTION
IN ANOTHER PAPER which is to be published shortly, the author has produced evidence to show that several of the existing "attitude" tests contain items which do not truly call forth attitude responses but may be items calling forth beliefs, judgments, opinions, or facts. In addition, experimental evidence is presented to show that experts in the field of sociology do not agree as to what constitutes an attitude item or a belief item or a judgment item or a fact item.
Droba,[1] too, has made a plea for a clear distinction between the types of statements used in attitude tests. He argues for uniformity of statement used in a single test which he feels will tend to increase the test's validity and reliability. He prefers the personal conduct form of statement, which he says would probably be the best indication of what the individual would do in a behavior situation.
PURPOSE OF THE STUDY
Consequently, an experiment was devised to test the validity of these ideas and to discover whether responses
( 19) to statements of varying types, although of the same con-tent, would elicit varying answers.
DEFINITION OF TERMS
Before discussing the experimental procedure, it will be necessary to describe briefly the distinction between attitude items, belief items, judgment items, opinion items, and fact items.
An attitude is a behavior tendency with reference to a value. Attitude items should contain an active verb and begin with "I." This type will more accurately than other forms reveal a person's behavior and, therefore, his attitude. An example of an attitude item is: I vote for the Republican Party. This is analogous to Droba's personal conduct form.
A belief is a verbal expression of one's highly personalized affective behaviors with reference to environmental qualities. There is more of an emotional than an intellectual content. Belief items should contain the words "should" or "must." An example of a belief item is : The United States should recognize Soviet Russia.
A judgment is a decision of an intellectual order, a cognitive experience concerning the quality aspects of one's environment, based upon personal knowledge. It involves an intellectual choice between two alternatives. An. example of a judgment item is: Negroes are born mentally inferior to white people.
A fact, of course, is an impersonal record of data based upon actual information. An example of a fact item is : The United States has not recognized Soviet Russia.
An opinion is a rationalization for a given act, attitude, belief, or judgment. It is a reason which satisfies the
( 20) person expressing it for self-justification. Opinions cannot stand by themselves; they must be attached to attitudes, beliefs, or judgments. An example of an opinion item is (in this instance the opinion is attached to an attitude) :
I vote for the Republican Party because it stands for a protective tariff.
With these distinctions in mind, let us proceed with the experimental evidence.
EXPERIMENTAL PROCEDURE
The procedure used in this study now outlined is as follows: —Twenty-four items collected from several of the existing attitude tests were changed into the four forms with the content kept constant—(belief [B], judgment [J], attitude [A], and fact [F] forms) as already indicated, thus making a total of ninety-six items. The opinion form was omitted because it was felt that its inclusion would complicate the problem unduly. Experiment to test the effects of opinion forms is now under way.
Two schedules were prepared: the first arranging the four forms of each item of the same contents contiguously in the order—B, J, A, F—and the second schedule arranging them in random order both as to form and content.
Their nature and the contrast between them is shown in the following items of each form :
Schedule I (Contiguous)
B[2] 77. There should be a federal law against lynching.
J 78. It is possible to have a federal law against lynching. A 79. I defend a federal law against lynching.
F 80. There is no federal law against lynching.
( 21)
Schedule II (Random)
J 77. It is necessary that we cut off immigration.
J 78. The United States is justified in keeping the Philippine Islands.
J 79. The United States is justified in not joining the League of Nations.
J 80. Colored people are justified in fighting for social equality.
The two schedules were administered to 180 students attending the Teachers College, Columbia University summer session, in 1931. One-half the students took Schedule I and the other half Schedule II.
The responses to each of the 96 items made by the 180 students were tabulated. In the final calculations, the F form was omitted, since responses to this form depend up-on a student's knowledge and not upon his attitude. In addition, inclusion of the F form would not aid at all in interpreting the results since our main problem is to determine whether statements, of varied forms, which are commonly found in attitude tests, tend to elicit varied responses.
Consequently the data were handled in this manner. The number answering the item (the A, B, J forms taken together) was found; the number and percentage answering the A, B, and J forms alike; the number and percentage answering the A form differently from the B and J forms; the number and percentage answering the B form differently from A and J forms; the number and percentage answering the J form differently from A and B forms were calculated. This was done for the two schedules.
RESULTS
The results are given in the table following :
(22)
Table I can be read as follows: On Schedule I, Item 1, the A, B, and J forms were answered alike by 90% of the persons answering the item; on Schedule II, the A, B, and J forms were answered alike by 92% of the persons answering the item. On Schedule I, Item 1, the A form was answered differently from the B and J forms by 1% and on Schedule II, by 2%. On Schedule I, Item 1, the B form was answered differently from the A and J forms by 1% and on Schedule II, by 1%. On Schedule I, Item 1, the J form was answered
( 23) differently from the A and B forms by 8%, and on Schedule II, by 4% of the persons answering the item. Similarly for the rest of the table.
There is somewhat greater variation on Schedule II, where the items were arranged at random. On Schedule II, there are only 4 items out of the 24 in which the A, B, and J forms were answered alike by more than 91% of the persons answering these items, while in Schedule I there are 7 items in which the A, B, and J forms were answered alike by more than 91%. The lowest amount of agreement occurred on item 21 in Schedule II, in which instance the A, B, and J forms were answered alike by only 48% of the persons answering the item, while in Schedule I the lowest amount of agreement occurred on item 8, in which instance the A, B, and J forms were answered alike by only 54% of the persons answering the item.
If the form of the item is not significant in test results, then all items in this experiment should be answered alike by 100% of the persons answering the item, but this is not the case since there is only one item (number 11) of the 24 in which there was 100% agreement on Schedule I and 99% agreement on Schedule II. In view of the amount of variation found in forms of attitude items in attitude tests, this is a most interesting result.
It may be worth while to note the kind of item in which there is more agreement than there is lack of agreement. Item 11, which showed practically perfect agreement, dealt with the issue of the cultivation of international friendship and good will among governments ; item 6, which showed 94% agreement on Schedule I and 91% on Schedule II, dealt with the issue of the condemnation of the church. Others which showed high agreement had to do with the independence of the Philippine Islands, the government ownership of railroads and the control of the United States
( 24) government by great financial interests. On the other hand, the items which showed least agreement dealt with more emotionalized and personal issues; such as, the recognition of Soviet Russia by the United States (Item 8) ; the immediate disarmament of all nations (Item 14) ; the fight for social equality by the colored people (Item 21) ; the policy of the United States in not allowing communists to openly advocate their policies here (Item 24) ; and, finally, the issue of patronizing a hotel that accommodates Negroes (Item 22).
CONCLUSIONS
In view of these results, it can be said that when the central idea of contrasted verbal forms of items is as identical as they can be made, (1) variations in verbal forms of test items produce some variations in markings. (2) Variations appear even when items of the same subject are contiguous. (3) Variations are somewhat greater when items on the same subject are not contiguous but random. (4) Variations while they do occur when items on the same subject are contiguous, are generally less than the similarities in markings. In other words, the test markings generally tend to be more alike than they tend to be different. But the variations are frequent enough to raise doubts as to the dependability of attitude test results when items are carelessly formulated without distinction as to whether they represent attitudes, or judgments, or beliefs. (5) The amount of variations of contiguous items on the same.-subject seem to depend upon the type of subject. Items symbolizing personal emotional complexes (prejudices, antipathies) organized about hot-spot experiences or social issues that involve the maintenance of personal status need exceptionally careful formulation for depend-
( 25) -able results. With respect to impersonal types of subjects in the items the methodological issue is not so significant.
It is judged, therefore, that attitude tests should contain statements of the same type, and that if they are attempting to measure the collectivity of attitudes, the statements should be phrased as simply as possible, begin with "I," and contain an active verb.
