The Methodology of Experimental Sociology
Dorothy Swaine Thomas
Assistant Professor of Education, Teachers College
Columbia University
THE STUDIES presented in preliminary form in this volume represent an experimental approach to social behavior. For this reason we are calling the methodological scheme on which they are based experimental sociology. It is sociological in the sense that its aim is the study of overt behavior in varying situations in the field of social interaction.' It is experimental in the sense of developing techniques for the control of the observer in order that scientific records may be obtained both of behavior and of situation, and that statistical analysis—ultimately the necessary tool for evaluating behavior-situation relationships--may eventually be applied.
Our present concern with the development of techniques is due to the fact that there have been so few attempts to obtain genuinely objective data in this field and, indeed, a rather general feeling that such attempts represented the measurement of the unmeasurable.
But, as Bowley has pointed out regarding statistical sociological investigations, "the difficulties are not greater than those overcome by scientific investigators in other fields who have had to invent their instruments and find, and even capture, their material below, above, and in remote parts of the earth. . . . The main task . . . is to discover exactly what is the critical thing to examine, and to devise the most perfect machinery for examining it with the minimum of effort." [2]
(2)
In this methodological scheme we have attempted to meet the problems of deciding what arc the critical things to examine, of inventing our units of measurement, and of devising means for controlling our instrument of measurement (i.e., the observer). Our emphasis on techniques is not the sterile interest of the metaphysician removed from contact with concrete materials. It has, on the contrary, grown out of an examination of existing materials, an appreciation of our present inability to better these materials, and a conviction that a purely empirical approach with an avoidance of the implications of methodology may lead to difficulties quite as serious as the philosophical morass in which the earlier investigators plunged us. In the one case there have been developed complicated systems of deductive methodology, logically more or less impeccable, leading to "laws" of social and economic behavior from which commonly acknowledged deviations in practice are put down to "interferences" of one sort or another with the normal working of the "law." Methodological discussions of this sort lead to little genuine knowledge about behavior. And in the case of the empiricists we find other difficulties. For the process of dealing with actual behavior cases has developed an approach which is more artistic than it is scientific. Practice, with a homely sort of experimentation, has advanced far beyond theory. The handling of behavior "problems" as they arise has become a highly specialized art—but an art in which the proportion of "successes" is still so relatively slight as to lead to serious dissatisfaction and the growing feeling that it is necessary to test out, by scientific methods, the hypotheses arising from these case materials. This has resulted in the application of statistical methods to existing behavior materials —and it is here again that serious difficulties with the problems of method have arisen. For it has not been widely recognized that it may be quite improper to apply the statistical methods commonly used in other fields to existing behavior data and that the adequacy of any particular method is very intimately dependent on the nature of the data to which it is applied. The very simplest statistical methods will not give meaningful results unless the variability of the instrument of measurement is con-
(3) -fined to very narrow limits. In measuring several fields we cannot have a yardstick that expands or contracts with each field if we wish to make a statistical comparison of fields, yet our statistical studies of behavior, individual and social, have done just that sort of thing, for we have failed to separate the observer from the observation. The more complicated statistical methods depend on assumptions regarding distribution of data, etc., which only a thorough knowledge of the derivation of our data will give us the right (scientifically speaking) either to accept or to disregard.
The available data in regard to social behavior consist largely of descriptive accounts—case histories and diary records. These are often very illuminating social behavior documents but they present certain difficulties as material for scientific analysis. The data obtained in such records are, at their best, objective, in the sense that they deal with certain verifiable facts, but they are selective, inconsistent, and usually incomparable with other records. This is due to the tremendous complexity of any social behavior act and the consequent recording of different elements of these complex acts at different times. At their worst, these records are such an intermixture of facts and interpretation as to be utterly worthless from the scientific point of view. Even at their objective best, the selection and emphasis are more or less dependent on the recorder. The control of this sort of error in our social data is one of the first problems claiming our attention. In other words, our data must become independent of our observers within a small and predictable range of error.
Our attempts to apply to data of this sort methods taken over from other fields have met with almost uniform disaster. We have thrown our data, obtained as described, into a pseudo-quantitative form by means of rating scales and similar devices. We have then proceeded with refined statistical analysis, using methods devised for genuinely quantitative data limited by very definite assumptions, and have presented our results in a form to suggest great mathematical accuracy. We have postulated cause-effect relationships from such analyses, but the failure of our results to give any reasonable predictions has shown us the fallacy of this procedure.
(4)
What, then, can we do to overcome these difficulties and to produce data on social behavior which will be more acceptable from the scientific point of view? The experiments described in this volume are attempting, in a limited way, to develop techniques which will produce more satisfactory data on certain phases of social behavior.
In developing these new techniques we had first to decide whether we would attempt to set up "controlled" situations. The purpose of controlled experiment is to measure the influence of some one factor or stimulus by excluding from the situation all interfering stimuli. Obviously, this can be ideally true only in certain of the natural sciences, such as chemistry or physics.[3] Where interfering stimuli cannot be excluded because of the very nature of the materials dealt with, as is the case in biology, physiology, and psychology, experiments are set up as perfectly as possible and statistical analysis resorted to as a means of evaluating and excluding the multiple "interfering stimuli."
In biology and physiology, even though the interfering stimuli may be quite imperfectly evaluated by statistical means, the data are recorded in genuinely quantitative units and are, to a large extent, independent of the observer. In most psychological experiments there is the additional difficulty, as we have indicated above, of the dependence of the data on the observer, so that the results, even in so-called controlled situations, arc subject to all the difficulties found in physiology and biology plus the more trying difficulties of unreliability of observation.
Sociological experimentation is closely allied to psychological experimentation with regard to this problem of method. A good deal may be said for controlled experiment in the study of individual behavior, but the main problem of control of the observer will still remain unsolved. In the study of group behavior or the relation of the individual to the group, the present methodological problem seems to center entirely around the control of the observer. We are skeptical of the possibility even of recognizing—much less of controlling—most of the "interfering" stimuli in
(5) behavior of the sort we wish to study. It seems to us that the very nature of the subject of investigation requires the development of techniques differing from those usually applied in psychology—not at present involving "control" in the sense of setting up specially prepared situations but approaching the study of social behavior in a genuinely social environment. The problem seems to be to evolve techniques for the accurate recording of the selective responses made by individuals to the multifarious stimuli which comprise their social milieu. In other words, we want to find means of recording the particular stimuli in the uncontrolled environment to which a given individual, at a given moment, reacts overtly—what consistency is observable in his selective responses over a period of time and what variability is shown among different individuals. It is to this end that our series of studies is directed.
These studies deal with the social behavior of young childhood, both because of the light that any results found at these age levels may throw on later development and problems in the field of social interaction and because of the relative simplicity of such studies at the early age levels. They are made possible by the fact that the nursery schools connected with the Child Development Institute at Teachers College provide an environment rich in material and social stimuli, thus forming a very desirable laboratory for developing techniques in the field of social behavior, and that graduate students, working for their degrees in Columbia University, are at hand to aid in collecting data and developing these techniques.
The nursery school children are divided into two groups of about twenty children each. The younger group ranges in age from about eighteen to thirty-two months and the older group, from thirty-three to forty-eight months. They are in school from nine a.m. to three p.m. daily for five days a week. The daily program includes a period of about two hours in the morning usually spent on the roof playroom where there is a great variety of equipment and toys interesting to young children, with relatively little direction by the teachers, and where the children's activities are, to a high degree, spontaneous. They are accus-
(6) -tomed to various adult observers, since one of the functions of the Institute is the training of nursery school teachers, so that observational studies may be made with a minimum of experiment-consciousness on the part of the subjects. The unselfconsciousness of the younger of the two groups is especially marked.
The first difficulty we met in our investigations arose through the question of determining the units in which our studies should be made,—in deciding the very difficult question of what is a datum in this field. The overt expressions of any act of behavior are so exceedingly complicated that a complete record is impossible. As a result the ordinary observational technique was shown to be totally invalid as an instrument for recording behavior for research purposes. An actual check-up was made by.having several observers make diary records of the social behavior of given children. There was a marked tendency for a given recorder to note one aspect in one record ; another in the next record. Although trained to make objective records in the sense of including overt reactions only, the result was necessarily subjective in that the recorder, usually unconsciously, selected specific parts of that total behavior act. This selection became, furthermore, inevitably inconsistent over a period of time. The obvious solution to this difficulty was to break up the behavior-complex into relatively simple units which would enable a record to be made of every recurrence of one of these behavior units.
The next problem was to decide which units could most satisfactorily be investigated. We have emphasized the fact that these units be genuinely quantitative and independent of the observer, i.e., variable within very narrow limits. Our ultimate aim is to work out the interrelationship among the various units of social behavior and their consistency as evidenced in varying situations. Our efforts toward evolving satisfactory techniques, therefore, attempted to take this aim into consideration when we selected units for measurement. In other words, we have tried to avoid the triviality which is often the concomitant of a sudden attempt to become scientific by looking for an a priori probability that those units which we are now measuring will prove "significant" indicators of the total social-behavior complex.
(7)
Our first efforts, therefore, were directed to hypothesis-forming materials—case studies and descriptive accounts of human behavior—and from these we selected certain apparently "significant" problems for investigation. We concluded that the study of social behavior could best be made by evolving indices of the overt actions made by different individuals involving other persons as compared with overt actions involving material objects (or abstractions) and the self. The aim of our techniques is the recording of the units into which these overt actions may be broken.
These techniques, several of which are discussed in detail in the following chapters, are of three general forms : (1) those in which each child is followed for a given period in the nursery school and a given overt social-behavior act recorded each time it recurs; (2) those in which, within the larger nursery school situation, a specific social situation is recorded each time it recurs; (3) those in which the psychological test situation, involving more limited social and material stimuli, is used instead of the nursery school for recording data of the sort described under (I) and (2).
(I)
The technique described in Chapter II, which is largely the work of Margaret Barker, involves the charting and timing of the child's activity as it is concerned with materials and persons. The child's actual progress as he moves about from person to person and thing to thing is charted on a floor plan (see Charts, pages 41 to 49). The recorder snaps a stop-watch each time the child takes up a new activity and indicates, briefly, the nature of the activity. The quantitative data yielded by such a charting are in terms of time spent on each activity, number of contacts with persons as compared with contacts with things, and gross activity (distance covered). These quantitative data should give us important indices of certain of the elements involved in social behavior. A priori it would seem that these observations should be to a high degree independent of the observer. The actual development of the technique shows the typical difficulties we are
(8) meeting throughout all our studies in achieving any such independence. The first study of reliability presented by Miss Barker was based on data collected by four observers, always working independently but simultaneously in pairs. Each observation was for a five minute period, and about twenty such observations were made .on each of sixteen children, making possible a study of reliability based on 324 records. When the times recorded for the same activities by different observers were paired and coefficients of correlation computed these coefficients were found to be very high (Pearson r ranges from .92 to .98 for the six possible combinations of pairs of observers). This indicates merely that when two observers timed the same activity the times were very highly correlated. But, as Miss Barker points out, in this early period of developing the technique one of the pair of observers was quite likely to time an activity which the other observer either neglected to time (noted only descriptively) or omitted altogether. Such omissions while not affecting the correlation made up a seriously large proportion of the whole—in 24 per cent of the cases the time was not recorded by either observer; in 9 per cent of the cases it was noted by one but not the other. Training the observers to snap the stop-watch with every change in activity reduced these omissions in the later study of reliability to a negligible proportion of cases where the time was not recorded by either observer (averaging less than 1 per cent). The proportion timed by one observer only was not materially reduced.
The correlation coefficients, based on these more complete data, were even higher than before (ranging from .95 to .98). The same high correlation is found for gross activity, or space covered. We have, then, in this technique for charting and timing a child's activity a very reliable instrument of measurement. Unfortunately, however, we have not been able to separate the material from the social units in either the charting or the timing. We find that we can record when one activity begins and another ends with almost perfect agreement, but the extent to which this activity is concerned with persons or things is still in a non-quantitative form. Our first procedure was to record a brief
(9) description of the activity in the margin of the chart, as indicated on page 39. These descriptions were in terms of overt behavior but showed the sort of unreliability inherent in descriptive or diary recording. From these brief descriptions we attempted to form a classification of the activities in terms of involvement with materials, with persons, or with the self. The statistics resulting from such a classification we recognize as unreliable, for we can be pretty sure that the recurrence of given elements was not always noted in these descriptions. Furthermore, the proportion of time spent, in complex activities, on persons and on things is indeterminate from the present data. The proportion spent on clear-cut simple activities (simple from the point of view of involving only materials or persons or the self) is more reliable. Miss Arrington's contribution is in the nature of an attempt to overcome this classification difficulty—to make it possible to record, in the accurate activity timing already achieved, the material-social aspects of these activities. This will be attempted by the coding of social contacts occurring within an activity as being spatial or functional (physical or verbal) and indicating the duration of the social elements. The materials involved have always been noted with the duration of the material activities. One reason for our low reliability on the number of social contacts (irrespective of timing, the Pearson is ranged from only .47 to .8o for the six observers in the preliminary reliability study) is our neglect to differentiate between the merely spatial and the functional social relationships. Hence, when a given child approached another the recorder made his own interpretation of whether it was a genuine social contact. These low
coefficients of correlation are a beautiful example of how unreliability creeps in where interpretation is permitted the recorder. The insistence on inclusion of all contacts is making for much higher reliability. The limitation of "functional" contacts to either physical or verbal throws the more subtle social relationships into the class of the merely spatial—obviously an arbitrary procedure but one that will bring us genuinely quantitative data,
(10) and the more subtle distinctions can be brought out by our non-quantitative studies. The earlier classification (page 3o) gave us data that were pseudo-quantitative—data dependent on the individual observer.
The question may arise as to why, when given this element of unreliability, we have applied any statistical analysis of the sort indicated in Miss Barker's tables of distribution of self, social and material activities according to the number of times each occurs and the per cent of time spent on each for our group of children. The reason these tables were formed was that the data were sufficiently reliable to give us a rough indication whether more reliable data would show up the individual differences which our technique was designed to discover. We know a priori that persons differ in their social adjustments and behavior. We suspect that these techniques are indicative of important elements in these differential adjustments. Unless the techniques show individual differences, therefore, it is probably futile to carry then much further. The individual differences found by this technique were great enough to encourage us to proceed to further attempts at improvement of its reliability. Any data we present on individual differences, then, must not be taken as accurate but as indicative of possible gross differences.
Our first use of this technique involved a five minute record of each child on twenty different days. The variability proved to be too great to give any consistent behavior patterns by such a sampling. In the present development of the method consecutive records of a given child are taken for one whole day every two weeks. Even the old method, however, succeeded in showing marked and interesting differentiation as between individuals. For instance, (on the old data) the proportion of time spent on clear-cut material activities averages 62 per cent for the group but is found to vary among individuals from 27 per cent to 79 per cent. The proportion of activities in which other persons were involved varies from 9 per cent to 49 per cent. Activities concerned neither with other persons nor with things but with the self alone vary from less than 1 per cent to 17 per cent. Thus, with regard to the ability of observers to record certain data ac-
(11) -curately and with regard to differentiation between individuals, this technique has proved satisfactory in certain respects and is being further developed.
The other techniques which we have developed are, in general, simpler and depend on less complex units for observation. It was thought that two of the most significant aspects of social behavior might be tapped by studies of the physical contacts persons make and receive from other persons and of the language (conversational contacts and monologue).
The technique for recording physical contacts has been developed by Alice Loomis and is described in Chapter III. The procedure in developing this technique was to make tentative categories of the kind of physical contacts children have with other children. A miscellaneous class was formed to include the contacts which did not fall definitely into these categories, and, where possible, the miscellaneous class was reduced by the formation of further objective categories from the contacts included in this class. The recorder follows a given child over a period of time and records every physical contact he makes with other children and every contact he receives from other children. There is no difficulty in recording for a given child the number of contacts made and received. There is, however, considerable possibility of unreliability in the description of the kind of contact made, i.e., in the formation of categories. These categories are in terms of hit, point, pull, push, caress, exploration, accident, and assistance, with an indication of whether the given child makes or receives the contact. The response of the other child involved in the contact is recorded in terms of passivity, cooperation, resistance, or flight. The placing of a given contact in one of these categories is not usually difficult as, for example, a hit, push, or pull is a readily recognizable behavior act. That data of this sort are not always free from the idiosyncrasies of the observer, however, is indicated by the reliability records in the social-material activities study where, of the two observers making simultaneous records, one recorded a situation as consistently "Conflict between William and Edward," whereas the other observer recorded it "William embraces Edward...."
(12)
A study of the reliability of this technique, as shown by the correlation of the records of two pairs of simultaneous observers, is now being made but is not included in Miss Loomis' preliminary report. As might be expected, there is found high correlation with regard to the ratio of contacts made to contacts received, but a lesser degree of correlation when the contacts and responses are thrown into categories. The improvement of these categories is the next step in the development of this technique, for it must be made independent of the observer's interpretation in cases such as cooperation, assistance, etc. In other words, we must concern ourselves with the behavior units involved in these complicated and subjectively-named actions. Part of the data obtained from this study, then, is genuinely quantitative and independent of the recorder; another part will necessarily at this stage of the investigation be pseudo-quantitative and not completely independent of the observer.
Important light is being shed by this study on the relation of various situations within the general nursery school situation to the physical contacts made and received. Miss Loomis points out the necessity of obtaining a representative sampling of these varied situations for each child in order to make a legitimate comparison among children.
The question of differentiation among individuals which is so important a criterion of the "significance" of the technique is considered in this preliminary study on the basis of records made of sixty minutes' physical contact data on each of twelve children and less than sixty minutes on six other children.
These data revealed an interesting differentiation between individuals when an index was worked out showing the ratio of contacts made to those received for each child.
The ratios of contacts made to those received tended to approximate to 1 for most of the children, but there were nicely differentiated extremes : one group of children who made twice as many contacts as they received and another who received twice as many contacts as they made. However, a check-up of the consistency in the contacts was made by comparing the ratios in the record of a child made by direct observation with the data
(13) regarding his contacts found on other children's records. The comparison showed a sixty minute observation on each child to be insufficient, since neither the ratios nor the rank of the children remained constant in such a comparison; whereas if the contacts as found on other children's records (depending on a longer period of observation) were numbered serially and the odd contacts compared with the even contacts, the subject-object ratios and the ranks of the children tended toward considerably more consistency. (The Pearsonian correlation was .8o for the odd-even correlation as compared with .5o for the correlation of contacts from a child's own records with those on other children's records.) A comparison of this sort is an important factor in our methodology, for we are interested not only in the independence of observations but in their consistency. These experimental studies will indicate how much material we must collect by each technique in order to have a valid sample of the particular activity tapped by this technique. If we wish to measure growth, or change, it will obviously be very important to know how many observations must be made at any given point in time.
This physical-contact technique which is being further developed in considerable detail has already been shown to differentiate between children in what we consider an important aspect of their social behavior if the children can be observed over a sufficiently long period of time. The amount of time necessary for consistent results has not yet been determined.
The importance of language as an index of social behavior has long been recognized. We are having accurate records made of the consecutive remarks and vocalizations of each child over a period of time. A stenographer follows a given child of the older age group each day and records everything he says and everything said to him, with as much of the non-verbal context as possible, from the moment he arrives at school until he takes his nap in the afternoon. The language of the younger children is recorded by a member of the staff who has had special training in phonetics. There is little doubt as to the accuracy and relative completeness of these records. The artificial situation for testing the reliability of several of the stenographers in recording lan-
(14) -guage of the sort used by these children is described by Miss Jenkins in Chapter X. The best stenographer obtained 85 per tent of all that was said. Miss Jenkins' evaluation of her omissions showed them to be unimportant for the purpose of these analyses, since almost no pronouns or nouns were omitted. No other stenographer has been used in this study. We have been uncertain as to the amount of recording necessary to get a fair sampling of the child's language responses.
A tentative analysis has been made of our records on each child of part of the older age group, comprising a total of some fourteen hours' language divided into four daily records of three and a fraction hours. From the point of view of an index of social behavior what is wanted is some simple quantitative expression which will be a partial indicator, as certain of our other techniques have been, of the degree to which the child directs his attention to himself, to other persons, and to material objects. We have not yet enough material to devise such an index adequately. Piaget, of the Rousseau Institute, analyzed the remarks made by children on the basis of whether or not they were adapted to a hearer. We find difficulty in making such an interpretation objective enough to give confidence in the statistical expression of the results. We have tried tentatively two simple indices but our evaluation has not yet reached a point where we are willing to claim any validity for them. These indices are designed a priori to obtain verbal data comparable with those of the material-social activities and physical-contacts studies. They depend on genuinely quantitative data in so far as the assumption will be accepted that a word is a unit and that different sorts of words may be added together for such an index. The first of these indices is designed to show the relation of verbal attention to the self to verbal attention to other persons. It is obtained by adding all nouns referring to the self and all first personal pronouns, and dividing by all nouns referring to other persons plus all second personal pronouns, plus all direct commands ("you" understood), etc. The second of the indices is designed to show the relation of verbal attention to others compared with verbal attention to material objects and is based (as in the
(15) other index) on the nouns and pronouns referring to these persons and objects. Tentative analyses of a number of records suggest that the desired differentiation among children will be shown.
Another study, not yet in form for a report, is based on our analysis of the data on conversations in these language records. It is hoped that the ratio of conversational contacts initiated to those received for each of a group of children will give an index suitable for comparison with the physical-contacts index.
These techniques which I have described have had the following common feature : They have attempted to define social-behavior units by following a given child over a period of time and recording his overt behavior in objectively defined and limited terms.
(2)
It is possible also to develop techniques that record the recurrence of certain situations significant for the study of social behavior. Miss Hubbard's study (Chapter IV) describes a technique of this sort which involved the recording, in terms of time and persons involved, of every social group. "Social group" was defined arbitrarily as two or more children together, either functionally or spatially, in the same activity, with no attempt to qualify the sort of group. The time to the nearest half-minute each child entered and left the group was noted. In the younger age group, where social contacts are tenuous in the extreme, there was a real difficulty in training an observer to record the group-changes quickly enough. The reliability, as tested by the consistency of two observers, was found to be of the order of .8 or .9 for the correlation of the children's rank in percentage of time spent in social situations, number of social situations recorded for each child, and number of children played with by each child. It is questionable how far a rank-order correlation is an adequate means of studying reliability. More numerous data would have made possible a Pearson r, whose meaning would have been more clear-cut. The indication of these correlation coefficients is of a high reliability. However, although the data recorded were
(16) perfectly objective, the observer was unable to keep track of all groups forming, and the percentage differences between the two observers averaged 10-20 per cent, a large margin of error. The same difficulty exists in this index as in our other indices of social participation; it fails to differentiate between the spatial and the functional. The large margin of error is undoubtedly due to this factor.
Interesting differentiation among children was indicated by the data. First, as to the ratio of time they spent in groups to the time they were in group-forming situations (i.e., playing in the same room with other children). For the younger age group this was found to vary from 33 to 8o per cent. The average number of situations per 15 minutes on the roof varied from 1.2 to 3.6, and the average number of children played with in 15 minutes varied from 4.4 to 7.8.
Another study of social situations was made on both age groups and on a kindergarten group, as reported in Chapter V. Every situation in which laughter occurred was described, briefly, with the names of the children involved in the situation recorded, and an indication made as to which of these children responded by either a laugh or a smile. In other words, the social, or contagion, element in laughter was emphasized. The reliability of this technique has not yet been determined. It is probably not so high as it is in the physical contacts study, for example. The recording of recurrent situations within a group introduces a different sort of unreliability front that found in the recording of recurrent acts by following an individual child. The recorder selects his data to a greater extent in studies of situational recording because of the conflicting foci of interest. Probably only the simplest sorts of studies may be made by this technique. Miss Hubbard's procedure represents a relatively satisfactory example of this situational technique, but this laughter study is probably a less satisfactory example, and definite checking of the technique by simultaneous recorders is necessary before we can use the results as statistically valid material.
Our plan in developing this technique was to compute an index of social responsiveness in so far as laughter may be considered
(17) the basis for such an index. The crucial data, then, consist of the number of laughs for each individual as compared with the number of times he was in a group where laughter occurred. The unreliability of this index is probably due to the unreliability of the determination of presence in a group, as has been shown in our previous studies. A tentative index for each child was worked out by computing the ratio of the number of laughter responses to the number of exposures to situations where laughter occurred. "Exposure" was defined as involving presence in a group where laughter occurred. This index varied for the three groups from 37 to 84 per cent for a kindergarten group, from 48 to 96 per cent for the older nursery school groups, and from 18 to 100 per cent for the youngest group. These percentages are presented only because of the interesting indication they give of the great variability in this mode of response which, it is believed, may be linked up with other social-behavior tendencies.
(3)
The studies which use the psychological examination as yielding certain indices of social behavior are largely attributable to Mrs. Nelson. In her introduction to Chapter VII she discusses their significance from the psychological point of view. They are, however, interesting to the sociologist because of the method used and the results obtained. A stenographer, whose reliability in recording data of this sort was known through Miss Jenkins' study, recorded all the spontaneous remarks of the child and the tester. Research students recorded specific units of the total situation, e.g., the number of times specific praise was given by the tester, and specific instances of resistance on the part of the child. Both "praise" and "resistance" were arbitrarily defined and included mainly their verbal aspects. Individual differences, among children, in the amount of resistance offered to the total test situation were striking, the range being 0 to 21 instances, with a median resistance of 3 1/2 instances. The type of test resisted throws light on certain possible factors in social conditioning, since the predominant type was one which called for a performance on the part of the child in which the self rather than
(18) materials or more overt activity was concerned. With regard to instances of praise given by the tester, less clear-cut results emerge. Certain extreme instances of a disproportionate amount of praise demanded by children before proceeding with their test-performances suggest another important factor in social behavior (dependence on social or personal approval) on which further studies in this field may throw light. Miss Wise's analysis (Chapter VIII) of the statements of "I can't" and "I don't know" has interesting implications, particularly as regards individual differences in the occurrence of these statements in test-performances below the child's known mental level, suggesting again that differences in social adjustment may be tapped by studies in the psychological field. Certain of these indices will be important for purposes of checking with data in certain of the other studies. To what extent, for example, is resistance found in the child's relation to his own group and his teachers, as indicated in the physical contacts study correlated with resistance in a more limited but intimate situation ?
These techniques are representative of what we are attempting in the way of developing a quantitative approach to the study of social behavior. Our present emphasis on the development of techniques, then, is not due merely to an interest in the instrument qua instrument but to what we conceive to be the necessary next step. So little unbiased material exists in this field that we are forced first to evolve techniques and then to go about the task of collecting data. There exists no body of accurate data such as the economist has at his disposal. It is our contention that there are in the field of social behavior possibilities of obtaining materials which will yield us data as objective as the best of those with which the statistical economists are dealing. But the task of getting at these elusive data is no easy one.
We are extremely chary of using statistical methods of analysis to any great extent on the meager data we have so far accumulated. We are trying to avoid the dangers of obtaining statistical reliability which may depend upon the indoctrination of the observers rather than upon the genuineness of the results. We are attempting to obtain data so objective and in such definite units
(19) that any observers with a minimum of training can get the same results. We are testing this reliability in all cases by having simultaneous records made by different observers and computing both correlation coefficients and percentage differences among the observers. Our ultimate aim is, of course, statistical. We are trying to develop techniques which will tap as many important aspects of social behavior as possible. Then we shall want to work out the interrelationships for a given group and the tendencies to change over a period of time, as well as the differences in response of groups already differentiated in other respects (geographically, racially, etc.).
But not only must we emphasize how we obtain data, but what we obtain must of itself be important. It may be questioned as to how we know we are getting at what I have called "important" aspects of social behavior. We say "important" advisedly—for some selection on an a priori basis is necessary in statistical investigations. We consider correlation coefficients and other instruments depending on the theory of probability as poor means of discovery without a preceding intimate knowledge of the way the data behave, but excellent instruments for controlling bias and evaluating relationships if we first know our materials.
Hence our research program includes another phase which seems to me to have considerable importance to our plan for developing quantitative techniques. We are having a series of studies made, depending on materials which are merely descriptive records of happenings—an historical rather than a statistical approach. For instance, we are developing the group personality study, as described by Mrs. Beaver in Chapter VI.
In our studies of groups made with a view to statistical analysis, descriptive material bearing on the more "interesting" details of group relationships has had to be sacrificed to data which would be valid statistically. It was considered desirable to make up for this loss not by changing the plan of the statistical studies but by formulating some other plan for recording in descriptive terms day-to-day changes in the formation of groups, the consistency of group play at the preschool level, and any tendency that might be shown toward the gang phenomenon. After some
(20) preliminary study of the nursery school situations, it was decided to keep a daily account of the group activities found among the children of the older age range. A choice had to be made between having the observer record items regarding the group behavior of twenty children, as far as she could observe them, or select some specific sub-group for intensive study. The latter plan had many advantages over the former in giving a picture of the "total situation" of the behavior of any group ; for in the first case the observer would be making records only of the things that struck her as particularly significant, thus overweighting her particular biases in the field and giving perhaps quite unrepresentative fragments because of the multifarious demands upon her attention and powers of recording, and in the second case the items selected for record will always be items from the same context, and probably a much fairer picture of the interplay of the group will result.
It was decided, therefore, to select a group which had already developed a certain solidarity, to follow through the activities that were developed in common, to make note of the attempts to exclude others from these activities and of the attempts of others to "crash" into the group, under what circumstances other children were played with, how projects were initiated, tendencies to the phenomenon known as "leadership," tendencies to alliances within the main group, what happens during the absence of certain of the group, etc. A specific group of three boys, then, becomes the focus for attention by the observer. A record is made of all their interplay with one another, and, so far as possible, details of their interplay and interrelationships with other children. The mass of data arising from this day-to-day recording is important as hypothesis-forming material for further studies, and is important in itself as giving an indication of the sorts of relationships young children develop at this early age-level. It obviously will not yield data appropriate for statistical analysis.
The long and detailed records of language are also an important source for behavior studies. The stenographic records of the child's responses to the psychological test situation, giving clues
(21) to elements involved in social resistance in the establishment and maintenance of "rapport" between adult and child, are a further important source. The value of such materials in analytical and hypothesis-forming studies is indicated in Mrs. Herben's study of rapport (Chapter IX). The presentation of a whole stenographic record in that chapter shows the intrinsically interesting nature of these materials as social-behavior documents.
This brief sketch of our research program, together with the detailed descriptions of methods found in Chapters II to X, indicates what we mean by experimental sociology. The materials with which we are dealing are certain elements of overt behavior in the field of social interaction. Our present emphasis is on developing techniques which will yield us genuinely quantitative data. We feel that it is more important in this field to control the observer than to control the experiment. For this reason we are using a relatively "uncontrolled" social environment, i.e., the nursery school, as a laboratory for developing our techniques. The advantages of developing our techniques on these children are the relative simplicity and un-self-consciousness of social behavior at this age-level. Our selection of units of behavior for investigation was made with a view to what seem to be "important" aspects of social behavior at this age-level and what seem, in the differential social adjustments of adults, to be factors the beginnings of which should be investigated in young childhood. In our selection of problems for investigation, then, we have accepted those aspects which, it seemed a priori probable, would yield significant results. When our techniques are sufficiently reliable our plan of procedure will be to make a statistical evaluation by appropriate methods of correlation of the interrelationships existing among these various units of social behavior with a view to selecting certain techniques for wider application among varying groups of children.
