The Journal of Experimental Education Volume 54 Issue 2 1986 [Doi 10.1080%2F00220973.1986.10806404] Paul, Steven M. -- The Advanced Raven’s Progressive Matrices

This article was downloaded by: [University of Otago] On: 31 December 2014, At: 11:16 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Journal of Experimental Education Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/vjxe20

The Advanced Raven’s Progressive Matrices Steven M. Paul

a

a

University of California, Berkeley Published online: 16 Apr 2014.

To cite this article: Steven M. Paul (1986) The Advanced Raven’s Progressive Matrices, The Journal of Experimental Education, 54:2, 95-100, DOI: 10.1080/00220973.1986.10806404 To link to this article: http://dx.doi.org/10.1080/00220973.1986.10806404

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Downloaded by [University of Otago] at 11:17 31 December 2014

The Advanced Raven's .Progressive Matrices: Normative Data for an American University Population and an Examination of the Relationship with Spearman's g STEVEN M. PAUL University of California, Berkeley

ABSTRACT Normative data for the Advanced Raven's Progressive Matrices are presented based on 300 University of California, Berkeley, students. Correlations with the Wechsler Adult Intelligence Scale and the Terman Concept Mastery Test are reported. The relationship between the Advanced Raven's Progressive Matrices and Spearman's g is explored.

THE RAVEN'S PROGRESSIVE MATRICES (RPM) are the best known and most widely used culture reduced tests of mental ability. British geneticist Lionel Penrose and British psychologist 5. C. Raven were the first to present perceptual analogy 'and inductive reasoning problems in the form of a matrix. In their matrices the perceptual analogies simultaneously involve both horizontal and vertical transformations. The variety of figures, relationships, and transformations are virtually limitless. Figures may increase or decrease in size, elements may be added or subtracted, shaded or unshaded, flipped, rotated, mirror imaged, or show many other progressive changes in pattern. In each case, the lower right corner of the total matrix is missing, and the subject must select the best one of the six or eight multiplechoice alternatives to fill the empty corner.

Raven described the Progressive Matrices as "a test of a person's present capacity to form comparisons, reason by analogy, and develop a logical method of thinking, regardless of previously acquired information'? (Raven, 1938, p. 12). He was responsible for publishing the first Progressive Matrices Test and its subsequent improvements and extensions (Raven, 1938, 1947, 1960). There are three forms of the RPM now in use: Standard Progressive Matrices (SPM), Colored Progressive Matrices (CPM), and Advanced Progressive Matrices (APM). Considerable research has been conducted involving the SPM and CPM, but little information is available concerning the APM. Adequate standardization norms are lacking in the United States. Research is notably absent in relation to university students, the type of population the APM is best suited to measure. The Standard Progressive Matrices consists of 60 items grouped in five sets (A, B, C, D, E) of 12 items each. Each set involves different principles of matrix transformation and within each set the items become progressively more difficult. It was designed to cover the widest possible range of mental ability and to be equally useful with persons of all ages, whatever their education, nationality, or physical condition. The scale is intended to cover the entire range of intellectual development starting with the time a child is able to grasp the idea of finding a missing piece to complete a pattern. It is sufficiently long to assess a person's maximum capac-

Downloaded by [University of Otago] at 11:17 31 December 2014

JOURNAL OF EXPERIMENTAL EDUCATION ity to form comparisons and reason by analogy without being overly taxing or unwieldy. A person's total score provides an index of his intellectual ability. The scores obtained by adults tend to cluster in the upper half of the scale. The Colored Progressive Matrices, Sets A, Ab, and B, were devised as a test for young children and old people, for anthropological studies, and for clinical work. It can be used with people who, for whatever reason, cannot understand or speak the English language, suffer from physical disabilities, are intellectually subnormal, or have deteriorated. To make the test independent of verbal instructions, the problems are printed on colored backgrounds and the scale is arranged so that it can be presented in the form of illustrations printed in a book or as boards with movable pieces. Success in Set Ab depends on the comprehension of discrete figures as spatially related "wholes" and in combination with Sets A and B adequately covers the cognitive processes of which children under 11 years of age are usually capable. The Advanced Progressive Matrices, Sets I and 11, were constructed as a test of intellectual efficiency that can be used with people of more than average intellectual ability and that will differentiate clearly between individuals of even superior ability. The difficulty level of the APM is such as to make it unsuitable for persons scoring below a raw score of about 50 on the SPM. For the general adult population the APM has too small a range of scores to be useful. The APM is intended for intellectually superior youths and adults, university students, and others for whom the SPM is too easy. The APM was originally created in 1943 for use'at the War Office Selection Boards. In 1947 a revision was prepared for general use as a nonverbal test of intellectual efficiency with which a person is able to form comparisons between figures and to develop a logical method of reasoning. Based on the experimental work with the 1947 edition of Foulds (Foulds & Raven, 1950) and an item analysis carried out by Forbes (1964), the 1962 edition of the APM dropped 12 problems that made no contribution to the score distributions for adults of more than average intellectual ability from Set I1 and arranged the remaining problems in order according to the frequency with which they were solved as the total score on the revised set increased from 0 to 36. Raven arranged the 1962 edition so that it could be used without a time limit, in order to assess a person9s total capacity for observation and clear thinking, or with a time limit, in order to assess the examinee's intellectual efficiency. It consists of two sets of tests. In Set I there are 12 problems designed to introduce a person to the method of working and cover all the intellectual processes needed for success in Set 11. The 36 problems in Set I1 are identical in presentation and argument with those in Set I. They only increase in difficulty more steadily and become considerably more complex.

To assess a person's total capacity for observation and clear thinking, Raven suggests that the examinee be shown the problems of Set I as examples to explain the principle of the test. The subject can then be allowed to work through Set I1 at his own speed from beginning to end without interruption. To assess a person's intellectual efficiency, Set.1 can be given as a short practice test followed by Set 11 as a speed test. The most common time limit is 40 minutes. Examination of the literature reveals a preference for the administration of the APM without a time limit. Yates, in particular, states that even the shorter 1962 edition has not overcome the problem of power and speed contamination when given a 40-minute time limit. In a study involving 960 freshman university students, he found that the number of persons not attempting to solve problems increases with the later items of the test. Consequently, the difficulty levels of the items cannot be determined (Yates, 1966). Unlimited working time practically eliminates the number of items not attempted and enables a determination of the true difficulty of each item, unconfounded by differences in speed of working. This present study was undertaken to provide normative information about the APM administered to an American university population. Comparisons to other mental ability tests are presented and the relationship between the APM and Spearman's g is explored. Method

Subjects Three hundred students (190 female, 110 male) from the University of California, Berkeley, served as subjects. Their average age was 252 months (21 years) with a standard deviation of 32 months.

Procedure Each subject was tested individually. The basic procedure of the matrices test was explained by the experimenter using examples (problems A1 and C5) from the SPM. Subjects were instructed to put some answer down for every question and were given a loose time limit of 1 hour. If the subject was not finished in an hour an additional 10 to 15 minutes was given to complete the test. A subject's score was the total number of items answered correctly. One hundred fifty of the subjects were also individually given the Terman Concept Mastery Test (CMT), a high level test of verbal ability. A different set of 62 subjects out of the 300 were also individually administered the Wechsler Adult Intelligence Scale (WAIS).

The mean total score for the sample of 300 students was 27.0 with a standard deviation of 5.14. The median

Downloaded by [University of Otago] at 11:17 31 December 2014

PAUL total score was also 27.0. The mean total score of the normative group of 170university students presented by Raven (1965) was only 21 (SD = 4). Gibson (1975) also found data on the APM which were significantly higher than the published university norms. The mean total score of 281 applicants to a psychology honors course at Hatfield Polytechnic in Great Britain was 24.28 (SD = '4.67). Table 1 presents the absolute frequency, cumulative frequency percentile, t score, and normalized t score for the total APM score values based on the sample of 300 students. The 95th percentile corresponds to a total score between 34 and 35 for this sample. The 95th percentile value based on Raven's normative group with similar ages is between 23 and 24. The Berkeley sample scored much higher overall than the normative sample of Raven's 1962 edition of the APM. The internal consistency reliability based on the Kuder-Richardson formula (KR-20) is .83. That is, approximately 83% of the variance in total test scores is attributable to true score variance, i.e., to what the APM is actually testing. There is strong agreement between the rank order of the items, according to the frequency with which they are solved, presented by Raven and those determined for this sample (r = .94). However, there is one noteworthy exception. The item Raven ranked 13th turned out to be much more difficult for the Berkeley students than would have been expected. It ranked as only the 22nd most frequently solved item. The item involves changes in three variables: object shape (diamond, square, circle), number of internal lines (one, two, three), and slant of internal lines (45", 90°, 135"). The majority of subjects who did not choose the correct response (#2) were attracted to a distractor (#5) that ignored the necessary change in the slant of the internal lines. Information beyond what is provided by just total score values can sometimes be found in an examination of the incorrect responses to the APM (Thissen, 1976). Selection of distractor items, incorrect multiple-choice alternatives, for each of the problems of the APM was examined to determine if patterns developed that would aid in the discrimination between subjects. Two subgroups of the total sample of 300 were formed. The low group came from the bottom 24th percentile receiving total scores less than or equal to 23 (n = 72). The high group comprised those in the top 26th percentile who scored greater than or equal to 31 (n = 78). A comparison was made between the two groups to see if distractors chosen by the high group were different from or perhaps better (i.e., closer to the correct response) than the incorrect responses chosen by the low group. No differences between the two groups were found. Unlike most studies of the Raven's Progressive Matrices, a significant difference (a = .05) was found between the average total score of males and females. In

97

TABLE 1-Absolute Frequency, Cumulative Frequency Percentile, t Score, and Normalized t Score for Total APM Score Values (N = 300) Total score

Absolute frequency

Cumulative frequency percentile

t score

Normalized t score

this sample the males (M = 28.40, SD = 4.85, n = 110) outscored the females (M = 26.23, SD 5.11, n = 190). Four percent of the variance in APM total scores can be explained by the differences in sexes. The sex differences occasionally reported in the literature are thought to be attributable to sampling errors. No true sex differences have been reliably demonstrated (Court & Kennedy, 1976). One hundred fifty of the Raven's testees were also individually given the Terman Concept Mastery Test. There was a moderate positive relationship (r = .44) between the total scores on the two tests (APM: M = 27.24, SD = 5.14; CMT: M = 81.69, SD = 32.80). Sixty-two of the subjects were also administered the WAIS. Full Scale IQ scores of the WAIS correlated .69 with the APM total scores. Correcting this correlation for restriction of range, based on the population WAIS IQ SD of 15, by the method given by McNemar (1949, p. 127), the correlation becomes. 84 (APM: M = 28.23, SD = 5.08; WAIS: M = 122.84, SD = 9.30). In a similar study, McLauren et al. (1973) reported a correlation of .55 (.74 corrected for restriction of range) between the APM and the WAIS based on 131 students at the University of Alabama in Birmingham. These results indicate that the APM, CMT, and the WAIS are tapping some of the same general ability. The possible nature of that ability is examined in the following section.

98

JOURNAL OF EXPERIMENTAL EDUCATION

Downloaded by [University of Otago] at 11:17 31 December 2014

Spearman's g One of the most solidly established phenomena in psychology is that scores on all mental ability tests, no matter how diverse the mental skills or areas they cover, are positively intercorrelated when they are obtained in a representative sample of the general population. It was Spearman who first hypothesized that there is some "general factor" of mental ability that is measured in common by all of the intercorrelated mental tests. He gave the label "g" to this general factor. Spearman developed the mathematical method known as factor analysis which enabled him to extract the g from all the intercorrelations among a collection of diverse tests and show the correlation between each test and the hypothetical general ability factor. The correlation of a particular test with the g factor common to all tests in the analysis is called the test's g loading. The square of a test's g loading indicates the proportion of the total variance in the scores on the test that is due to individual differences in this general ability. It is important to note that the g factor may not show up on some tests given to highly selected groups, such as the often tapped pool of university students, although these tests show moderate g loadings when given to the general population. The explanation is that these groups have already been highly selected on g-loaded tests, such as college entrance exams, and therefore their scores indicate less individual variation on the g factor. This limits the intercorrelations among the various tests and thereby prevents the g factor from showing up strongly in a factor analysis of the matrix of intercorrelations. Spearman originally hypothesized that each test measures only g plus some specific ability, s, which is tapped only by the particular test. This theory that any given test score is composed of only g + s, as well as measurement error, was soon refuted by the finding that there are other common factors besides g in many mental ability tests. However, they cannot be considered general factors because they do not enter into all tests, as does g, but do enter only into certain groups of tests. In a factor analysis of a large number of various mental tests, the first unrotated factor (or principal component) is g or general mental ability. It usually accounts for almost half of the total variance in a large battery of diverse tests. The several other smaller factors, the group factors, show highly differential loadings on tests that are often characterized as verbal, numerical, spatial, or involving memory. Factor analysis by itself does not and cannot explain the basis for the existence of g. Spearman himself stated that factor analysis cannot reveal the essential nature of g but only reveals where to look for it. Examination of the characteristics of a wide variety of tests in connection with their g loadings can provide some descriptive generalizations about the common features that charac-

terize tests that have relatively high g loadings as compared with tests that have relatively low g loadings. Spearman originally tried to get at the psychological nature of g by factor analyzing more than 100 tests, each fairly homogeneous in content, and then comparing their g loadings (Spearman & Jones, 1950). He characterized the most g-loaded tests essentially as those requiring "the eduction of relations and correlates," that is, perceiving relationships, inducing the general from the particular, and deducing the particular from the general. Such tests require inductive or inventive as contrasted to reproductive or rule-applying behavior. The most g-loaded test in the whole battery was the Raven's Progressive Matrices (RPM), which, as previously mentioned, depends almost entirely on perceiving key features and relationships and discovering the abstract rules that govern the differences among the elements in the matrix. There is much more test material available now than was available to Spearman more than 50 years ago. This had led to broader generalizations about g . The g factor is manifested in tests to the degree that they involve mental manipulation of the input elements, choice, decision, invention in contrast to selection, meaningful memory in contrast to rote memory, long-term memory in contrast to short-term memory, and distinguishing relevant information from irrelevant information in solving complex problems (Jensen, 1979). Task comple~rityand the amount of conscious mental manipulation required seem to be the most basic determinants of the g loading of a task. There are many examples in which a slight increase in task complexity is accompanied by an increase in the g loading of the task. Virtually any task involving mental activity that is complex enough to be recognized as involving some kind of conscious mental effort is substantially g loaded. It is the task's complexity rather than its content that is most related to g. An almost infinite variety of test items, regardless of sensory modality, substantive or cultural content, or the form of effector activity involved in the required response, is capable of measuring g. This observation led to Spearman's principle of "the indifference of the indicator," meaning that the manifestation of g is not limited to any particular types of information or item types. Previous research suggests that the Raven's Progressive Matrices administered to the general population measures g and little else (Burlte, 1958). The occasional loadings found on other factors, independently of g, are mostly trivial and inconsistent from one analysis to another. Although many other tests measure g to a similar extent, unlike the Raven, they also have loadings on the major group factors such as verbal, numerical, spatial, and memory. The RPM does not measure perceptual ability or spatial-visualization ability as is com-

Downloaded by [University of Otago] at 11:17 31 December 2014

PAUL monly believed. In fact, the Raven has very small loadings on these factors, when g is excluded. Factor analysis of the RPM at the item level should result in only a single factor. Some investigations that have found more than one factor have employed improper orthogonal rotations of the principal components. This method can artificially create the appearance of several factors even in correlation matrices that are artificially constructed so as to contain only one factor plus random error (Jensen, 1980). Some of the small spurious factors that emerge from factor analysis of the inter-item correlations are not really ability factors at all but are "difficulty" factors, due to varying degrees of restriction of variance on items of widely differing difficulty levels and to nonlinear regression of item difficulties on age and ability (McDonald, 1965). When these psychometric artifacts are taken into account, the RPM seems to measure only a single factor of mental ability, which can be termed g. The APM test results of this study were factor analyzed at the item level. The first principal factor of the intercorrelation matrix of the 36 items, scored correct or incorrect, accounts for 15% of the total inter-item variance. A factor loading correlation matrix was created from this first principal factor and subtracted from the original inter-item correlation matrix. The resulting residual matrix was tested and found not significantly different from zero at a! = .05. Therefore, the APM can be considered to measure only one factor. That this factor only accounts for 15% of the total inter-item variance indicates that the variance of each item is due mostly to uniqueness, that is, item specificity and error. The items are not highly intercorrelated. However, what they do have in common may indeed be Spearman's g. The first principal factor should not be considered just a difficulty factor. The correlation between the item loadings on the first principal factor and item difficulty levels (percent passing) is - .36. (Only 13% of the variance in the first principal factor loadings can be explained by differences in item difficulty.) The correlation between the item loadings on the first principal factor and item variance is .41. (Sixteen percent of the variance in the first principal factor loadings can be explained by differences in item variance.) The correlation between item difficulty and item variance is - 37. When item variance is held constant, i.e., partialled out, the correlation between item loadings on the first principal factor and difficulty levels is .01. The loadings of each item with the first principal factor and the correlations of each item with total score are shown in Table 2. The total score on the APM can be considered a reasonable measure of general mental ability. This notion is at the very least intuitively appealing. There is near perfect agreement between the correlations of each item with total score and the correlations of each item with the hypothesized g factor. The correlation be-

99

tween the 36 item by point biserial correlations and the 36 g loadings with the effect of item variance partialled out is .99. This evidence supports the claim that the first principal factor of this analysis is not just a difficulty factor and that it measures general mental ability. Further evidence that the APM measures g comes from a closer look at the relationship between the APM and the WAIS. Two subgroups consisting of 13 items each and matched on item variance were created from the 36 APM test items. The high g group had an average g loading of .46 and an average item variance of .15. The low g group had an average g loading of .27 and an average item variance of .IS. Correlations were obtained TABLE 2-APM Item Correlations with Total Score and First Principal Factor (N = 300)

Item

Total score

First principal factor

Item

Total score

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

.08 .20 .25 .38 .34 .17 .24 .35 .34 .34 .30 .42 .24 .30 .24 .45 .39 .35

.04 .24 .29 .43 .37 .17 .24 .37 .41 .37 .34 .46 .18 .29 .20 .47 .36 .31

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

.30 .27 .50 .41 .56 .53 .37 .36 .50 .45 .50 .45 .47 .39 .43 .53 .49 .43

First principal factor .26 .23 .49 .36 .57 .50 .30 .30

.44 .38 .44 .38 .41 .33 .37 .51 .45 .37

based on the 62 subjects who took both tests. The correlation between the high g items and WAIS Full Scale IQ scores was .67. Low g items and WAIS Full Scale IQ scores correlated 56. Although the two correlations are not significantly different from each other (t = 1.39, a = .05), a trend was apparent. Despite the fact that the items of the APM and the WAIS are drastically different in content, those items correlating highest with the hypothesized g factor derived from the APM show a stronger relationship to WAIS Full Scale IQ scores than those items with a low correlation with the hypothesized APM g. Since WAIS Full Scale IQ scores have been shown to be highly g loaded in previous research (Matarazzo, 1972; Jensen, 1980), the pattern found here can be interpreted to indicate that the hypothesized g of the APM is the same g that is measured by the WAIS. In summary then, the distribution of scores for a large cross section of University of California students on the Advanced Raven's Progressive Matrices is markedly higher than the estimated score distribution of

JOURNAL OF EXIPERXMENTAL EDUCATION university students that accompanies the 1962 version of the test. A moderate 'positive association exists between the APM and the Terman Concept Mastery Test. There is an even stronger positive relationship between the APM and the Wechsler Adult Intelligence Scale. Examination of the internal factor structure of the items of the test indicate that the APM measures only one factor. This factor is not just a difficulty factor. The results support the notion that the APM provides a measure of Spearman's g.

Downloaded by [University of Otago] at 11:17 31 December 2014

REFERENCES Burke, H. R. (1958). Raven's Progressive Matrices: A review and critical evaluation. Journal of Genetic Psychology, 93, 199-228. Court, J. H., & Kennedy, R. J. (1976). Sex as a variable in Raven's Standard Progressive Matrices. Proceedings of the 21st International Congress of Psychology, Paris, France. Forbes, A. R. (1964). An item analysis of the Advanced Matrices. British Journal of Educational Psychology, 34, 1-14. Foulds, G. A., & Raven, J. C. (1950). An experimental survey with Progressive Matrices (1947). British Journal of Educational Psychology, 20, 4-10. Gibson, H. B. (1975). Relations between performance on the Advanced Matrices and the EPI in high-intelligence subjects. British Journal qf Social and Clinical Psychology, 14, 363-369.

Jensen, A. R. (1979). g: Outmoded theory or unconquered frontier? Creative Science and Technology,2, 16-29. Jensen, A. R. (1980). Bias in mental testing. New York: The Free Press. Matarazzo, J. D. (1972). Wechsler's measurement and appraisal of adult intelligence (5th ed.). Baltimore: Williams & Wilkins. McDonald, R. P. (1965). Difficulty factors and nonlinear factor analysis. British Journal of Mathematical and Statistical Psychology, 18, 11-23. McLaurin, W. A., Jenkins, J. F., Farrar, W. E., & Rumore, M. C. (1973). Correlation of IQ's on verbal and nonverbal tests of intelligence. Psychological Reports, 33, 821-822. McNemar, Q. (1949). Psychological statistics. New York: Wiley. Raven, J. C. (1938). Progressive Matrices: A perceptual test of intelligence, 1938, Individual form. London: H . K. Lewis. Raven, J. C. (1947). Coloured Progressive Matrices. London: H . K. Lewis. Raven, J. C. (1960). Guide to the Standard Progressive Matrices. London: H. K. Lewis. Raven, J. C. (1965). Advanced Progressive Matrices Sets I and II. London: H. K. Lewis. Spearman, C., & Jones, L. L. W. (1950). Human ability. London: Macmillan. Thissen, D. M. (1976). Information in wrong responses to the Raven Progressive Matrices. Journal of Educational Measurement, 13, 201-214. Yates, A. J. (1966). A note on Progressive Matrices (1962). Australian Journal of Psychology, 18, 281-283.