June 21, 2016 | Author: IKetut Suena | Category: N/A
Contemporary Educational Psychology xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Contemporary Educational Psychology journal homepage: www.elsevier.com/locate/cedpsych
Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students q Isabel Festas a,⇑, Albertina L. Oliveira a, José A. Rebelo a, Maria H. Damião a, Karen Harris b, Steve Graham b a b
Faculty of Psychology and of Sciences of Education, University of Coimbra, Portugal Mary Lou Fulton College of Education, Arizona State University, Tempe, AZ, USA
a r t i c l e
i n f o
Article history: Available online xxxx Keywords: SRSD Writing Professional development Self-Regulated Strategy Development Instruction
a b s t r a c t We examined the effects of the Self-Regulated Strategy Development (SRSD) for opinion essay writing among 380 eighth grade students in six urban middle schools in a major city in Portugal. Fourteen teachers in six urban middle schools in Portugal participated in the present study; 7 of these teachers participated in practice-based professional development (PBPD) in SRSD before implementation, and follow-up support once instruction began. Schools were matched in pairs based on SES and teacher characteristics; a member of each pair was randomly assigned to either: (a) teacher led SRSD instruction for opinion essay writing; or (b) teacher implementation of the schools’ existing curriculum and language program prescriptions for opinion writing. Students in the experimental schools were taught strategies for planning and composing opinion essays once a week in 45 min sessions, over a three-month period. Multilevel modeling for repeated measures indicated SRSD instructed students made statistically greater gains in composition elements than the comparison students immediately after instruction and two months later. Teachers implemented SRSD with fidelity and teachers and students rated the intervention favorably. This study provides initial evidence for replication of the effects of PBPD and SRSD outside of the United States. Limitations, lessons learned, and directions for future research are discussed. Ó 2014 Elsevier Inc. All rights reserved.
1. Introduction Writing is an important skill that cuts across the school curriculum and is useful for a variety of functions in daily life. Although writing is important and challenging to learn, in Portugal as in the United States (e.g., Gilbert & Graham, 2010; Harris, Graham, Brindle, & Sandmel, 2009), its teaching has been neglected. In Portugal, recent reform of the language arts curriculum resulted in new guidelines and standards for language arts instruction (Ministério da Educação e Ciência/Ministry of Education and Science, 2009, 2012). These guidelines recognize the importance of writing, including it as a priority area of instruction. They require not only the development of writing skills related to q This research was supported by European FEDER funding through COMPETE: (Operational Program for Competitiveness Factors) FCOMP-01-0124-FEDER022660 and by national funding through FCT (Fundação para a Ciência e Tecnologia/Science and Technology Foundation) under the project PTDC/CPE-CED/ 102010/2008. ⇑ Corresponding author. Address: Faculty of Psychology and of Sciences of Education, Rua do Colégio Novo, Apartado 6153, 3001-802 Coimbra, Portugal. Fax: +351 239 851465. E-mail address:
[email protected] (I. Festas).
grammar, capitalization, punctuation, spelling, and sentence construction, but also development of writing processes related to the organization of the text, including planning and revision. Writing across multiple genres (e.g., narrative, expository, informative, opinion essay, argumentative) is also emphasized. The Standards for Elementary and Middle Grade Levels (2012), for instance, require the instruction on specific attributes for different genre texts, (e.g., premise, reasons, elaborations, and conclusion for opinion essay). These Standards also require development of high quality writing products and the evaluation of writing, but neither specific instructional approaches nor time dedicated to writing instruction are prescribed by the Portuguese curriculum. Teachers are free to choose the teaching methods they use in their classrooms. Although the importance of writing has been recognized in the Portuguese curriculum, teachers have not been trained to teach writing strategies (Almeida, 2012; Almeida & Simão, 2007) and students have difficulty planning and revising their writing. As in the United States, (National Center of Educational Statistics, 2012), national data in Portugal indicates Portuguese students experience severe problems mastering writing (Gabinete de Informação e Avaliação do Sistema Educativo/Office of
http://dx.doi.org/10.1016/j.cedpsych.2014.05.004 0361-476X/Ó 2014 Elsevier Inc. All rights reserved.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
2
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
Information and of Evaluation of Educational System, 2005; Report of Gabinete de Avaliação Educacional/Office of Educational Evaluation 2011, 2012). Difficulties with writing persist at least until university level, as research indicates many Portuguese university students do not plan their writing (Carvalho & Pimenta, 2005) and revise only superficial text features such as spelling and punctuation (Festas, Damião, & Martins, 2010). Thus, improving writing abilities and developing effective instructional procedures to help overcome problems with learning to write are national priorities in Portugal. The use of evidencebased practices in schools is critical to achieving this goal (Cook, Smith, & Tankersley, 2012). Supported by rigorous studies and research, evidence-based practices are a useful means for improving teaching and their application has been recommended by government policies in many countries (Cook et al., 2012). This is also the case in Portugal, where evidence-based practices are emphasized in the standards-based reform movement and recommended by current government policy (Ministério da Educação e Ciência/ Ministry of Education and Science, 2012). In the writing domain, one of the most effective evidence based methods for writing instruction is Self-Regulated Strategy Development (SRSD) (Baker, Chard, Ketterlin-Geller, Apichatabutra, & Doabler, 2009; Graham & Perin, 2007; Institute of Education Sciences, 2012; National Center on Intensive Intervention, 2013). SRSD is appropriate to the needs of Portuguese pupils and the demands of the current Portuguese language arts curriculum and Standards. In fact, as we address next, SRSD includes the teaching of writing processes and self-regulation strategies, features which have been neglected in Portuguese schools and that are essential components of proficient writing. 1.1. Self-Regulated Strategy Development (SRSD) Developed by Karen Harris and pioneered by Harris and Graham more than 30 years ago (Harris, 1982; Harris & Graham, 1992, 1996; Harris, Graham, Mason, & Friedlander, 2008), SRSD is an approach that combines the teaching of writing processes (including planning, drafting, composing, revising and evaluating); instruction in writing strategies; and development of self-regulation strategies, including goal-setting, self-assessment (self-monitoring and self-recording), self-instruction, and selfreinforcement. At the same time, SRSD helps students develop the knowledge and skills needed to use these strategies and purposively develops self-efficacy for writing, attributions to strategy knowledge and effort, and motivation for writing (Harris et al., 2009). SRSD is a complex, multicomponent intervention based on integrating multiple theories and lines of research which have been detailed elsewhere (Harris & Graham, 2009; Harris et al., 2009). Of particular importance to the present study, research indicates this approach is effective when teaching typically developing writers in a wide range of grade levels, from elementary to high school (Graham, Harris, & McKeown, 2013). SRSD promotes writing development through the explicit, situated, scaffolded instruction of genre-based and general writing strategies and self-regulation strategies. Specific writing strategies for multiple genres, such as story, personal narrative, expository, opinion, and persuasive essays have been developed (Harris et al., 2008). Such strategies for writing and self-regulation are developed in six recursive, interactive, individualized instructional stages with gradual release of responsibility for writing to students: (1) develop and activate background knowledge; (2) discuss and describe the strategies to be learned; (3) model the strategies; (4) memorize the strategies; (5) support the strategies; and (6) independent performance (Harris et al., 2008). Instruction proceeds based on students’ progress; students are given the time they need to make these strategies their own. Procedures for
maintaining what has been learned and determining how to use this knowledge across writing tasks are integrated throughout the stages of instruction. SRSD has proven to be a powerful instructional approach. Its application and effectiveness have been investigated in over 100 studies (Graham et al., 2013), and a number of meta-analyses have examined its impact on students’ writing. Some of these metaanalyses focused on strategy instruction in writing in general, including SRSD studies as part of the analysis (Graham, 2006b; Graham & Harris, 2003), whereas other reviews were broader in scope and examined a broader range of writing treatments, including strategy instruction and SRSD (Graham, McKeown, Kiuhara, & Harris, 2012; Graham & Perin, 2007; Rogers & Graham, 2008). These meta-analyses included studies conducted with students with learning disabilities as well as with poor, average or good writers in both special and regular school settings (Graham, 2006b; Graham & Harris, 2003; Rogers & Graham, 2008) or only students from regular school classrooms (Graham et al., 2012; Graham & Perin, 2007). Some of these reviews focused just on the elementary-levels (Graham et al., 2012), others on elementary and middle school pupils (Graham & Harris, 2003) and still others on pupils from elementary to 12th grade (Graham, 2006b; Graham et al., 2013; Graham & Perin, 2007; Rogers & Graham, 2008). Across these meta-analyses, SRSD was found to be a highly effective instructional practice, and it yielded better results than other writing instructional methods, including other methods for teaching writing strategies. Large effect sizes (ES) – above .80 – were found in true and quasi-experimental studies (Graham, 2006b; Graham & Harris, 2003; Graham et al., 2013, 2012; Graham & Perin, 2007), and a high percentage of non-overlapping data (PND) – almost above 90% – was obtained in single-subject design studies (Graham, 2006b; Graham et al., 2013; Rogers & Graham, 2008).
1.2. The present study While previous research has demonstrated that SRSD is a powerful tool for improving students’ writing, the present study was designed to address limitations in the data base on SRSD. First, most of the true- and quasi-experiments that have tested the effectiveness of SRSD in writing have involved children in the elementary grades (cf. Graham et al., 2013), and students have typically received SRSD instruction in small groups or one-on-one. Furthermore, and also of particular importance to the present study, instruction in nearly all of the published studies on SRSD was delivered by trained graduate assistants (Harris et al., 2009). Only three published studies have involved general education teacher implementation of SRSD in the middle school classroom (De La Paz, 2005; De La Paz & Graham, 2002; Wong, Hoskyn, Jay, Ellis, & Watson, 2008). These studies, however, involved only 2–4 classroom teachers, and little information was provided as to how teachers were prepared to use SRSD in their classrooms. Only one published study was found that focused on professional development for implementation of SRSD classwide in the general education classroom (Harris et al., 2012). This randomized controlled study involved 20 s and third grade teachers who participated in practice-based professional development (PBPD, cf. Ball & Cohen, 1999; Grossman & McDonald, 2008) in SRSD for either story or opinion essay writing (each genre served as the control condition for the other genre). PBPD focuses on teacher development of knowledge, understanding, and skills regarding an effective educational practice before they use it, with support once classroom use begins (cf. Ball & Cohen, 1999). PBPD rejects traditional approaches to professional development that are short-term and top down, do not allow teachers to actively engage in the practices
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
they are learning, and do not provide much support during implementation, which traditionally is primarily done in isolation. Congruent with sociocultural and social cognitive theories that stress the importance of meaningful learning in situated contexts, PBPD has six critical characteristics: (a) collective participation of teachers within the same school with similar needs; (b) basing professional development around the characteristics, strengths, and needs of the students in these teachers’ current classrooms; (c) attention to content knowledge needs of teachers, including pedagogical content knowledge; (d) opportunities for active learning and practice of the new methods being learned, including opportunities to see examples of these methods being used and to analyze the work; (e) use of materials and other artifacts during professional development that are identical to those to be used in the classroom, and (f) feedback on performance while learning, and before using these methods in the classroom, so that understandings and skills critical in implementation are developed. In the Harris et al. (2012) study, teachers who received PBPD implemented SRSD instruction that resulted in significant and meaningful changes in student writing outcomes for both story and opinion essay writing. Teachers implemented SRSD with fidelity, and SRSD was viewed as socially valid by teachers and students. Finally, the few published studies of teacher implementation of SRSD have all been conducted in the United States, as have the majority of studies on SRSD. Thus, there is a need to test the generalizability of SRSD results across populations outside of the United States. Similarly, PBPD has not been tested outside of the United States. If PBPD for SRSD’s promise is to be actualized, it is important to test its effectiveness in different contexts. Aggregated, consistent findings across a variety of studies in a variety of contexts are necessary to argue that an intervention is reliable and generalizable (Robinson, Levin, Schraw, Patall, & Hunt, in press). Recommendations for educational practice must be based upon such aggregated finding and research syntheses, thus replication and extension of previous research is critical to the field. Despite the difficulties Portuguese students experience learning to write, evidence-based practices in writing have rarely been tested in this country. Our study extended previous SRSD research by examining if SRSD instruction was effective when it was implemented by Portuguese teachers who received PBPD for SRSD, and conducted classwide with Portuguese middle school students. We examined the effects of SRSD instruction, implemented by classroom teachers following PBPD, on the opinion essay writing performance of eighth grade Portuguese students. Previously, we had adapted SRSD lessons and materials to the Portuguese language and to Portugal’s educational context (Rebelo et al., 2013). Eighth grade was selected as the target grade level because such instruction should help these students prepare for the national exam that is taken in the ninth grade. Writing opinion essays is an important part of the school curriculum in eighth- and ninthgrade and is targeted in writing tests and the Portuguese national exam. Criteria used in the national exam include evaluation of opinion essays’ structural elements (premise, reasons and conclusions) (Gabinete de Avaliação Educacional/Educational Evaluation Office, 2013), which are an important part of SRSD for opinion essays. Furthermore, the language arts teachers in the participating schools requested a focus on opinion essay writing. Fourteen teachers in six urban middle schools in Portugal participated in the present study; 7 of these teachers participated in PBPD for SRSD. The six schools were matched in pairs based on socioeconomic status and homogeneity/heterogeneity of students and on teacher characteristics including gender, experience, and preparation. One member of each pair was randomly assigned to one of two conditions: (a) classroom teacher implemented SRSD instruction for opinion essay writing; or (b) teacher implementation of the schools’ existing curriculum and language program
3
prescriptions for opinion writing. SRSD teachers in this study participated in PBPD before implementing SRSD, and received support and feedback throughout classroom instruction. We anticipated that SRSD would have a positive impact on Portuguese students’ opinion essays in terms of structural elements and writing output, and that SRSD instruction in this study would result in a significant increase in the number of opinion essay elements in students’ writing as well as a significant increase in the overall length of student essays. Such results have been found in other studies involving the opinion essay writing strategies taught in this study (e.g., Mason, Kubina, & Taft, 2009). We expected that these effects would be evident immediately after the intervention and after a two-month interval (such maintenance effects have been found in other studies; Graham et al., 2013). We further expected that following PBPD, teachers would implement SRSD with fidelity based on the study by Harris et al. (2012). Finally, we predicted that students and their teachers would be positive about the effects of the intervention, indicating social validity in Portugal, based on our anticipation that students’ opinion essay writing would improve. 2. Methods 2.1. Setting The study took place in six urban schools in a major city in Portugal. Three of the schools were middle schools; the other three combined middle and high school grades. As usual in Portugal, all of the schools involved in the study followed the national curriculum, as previously described. In an attempt to increase the likelihood that the experimental and comparison groups were equivalent, pairs of schools were matched for socioeconomic status prior to the intervention (Fraenkel & Wallen, 2003). Data were collected from the School Educational Project, which reports students’ socioeconomic status; SES is defined by the level of schooling completed by students’ parents and parents’ professions. School 1 (middle school with 838 students) was matched with School 2 (middle school with 690 students). According to the School Educational Project data, the majority of students in both schools were from a medium to high socioeconomic class. School 3 (middle/high school with 991 students) was matched with School 4 (middle school with 560). According to the School Educational Project, both schools had a more heterogeneous population from low, medium and high socioeconomic groups. School 5 (middle/high school with 950 students) was matched with School 6 (middle/high school with 700 students). According to the School Educational Project, these students had medium/high socioeconomic status. The schools were also matched on teacher characteristics including gender, experience, and preparation. All of the teachers completed a questionnaire to elicit information about: gender, age, licensing, writing training received during teacher preparation and professional career, teaching procedures, and teaching experience. One of each pair of schools was randomly assigned to the experimental condition and the other to the comparison group. The necessary consents were obtained from the Portugal Ministry of Education, the Director of each school, from the teachers involved in study, the students’ parents, and the pupils enrolled in the experiment before the study started. 2.2. Teachers Fourteen teachers in the six schools took part, seven each from the experimental and comparison schools. Teachers in the experimental and comparison conditions had very similar characteristics.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
4
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
All teachers were female and all taught Portuguese language arts. Additionally, all of the teachers had an undergraduate degree obtained from a university in Portugal and only one teacher (in the comparison group) had a master’s degree (no difference was found concerning this variable: Fisher’s Exact Test = 1, p = .50). All of the teachers held a teaching license and all had served as teachers for a long period of time, ranging between 16 and 34 years in the experimental group (M = 27; SD = 6.56) and between 23 and 39 years in the comparison group (M = 29.33; SD = 5.54). No statistically significant difference was found here between the two groups (t(12) = .686, p = 0.51]. Both groups included teachers who taught more than one section of language arts and teachers who taught only one section: in the experimental group there was 1 teacher with 3 sections, 4 teachers with 2 sections, and 2 teachers with 1 section; in the comparison group there was 1 teacher with 3 sections, 3 teachers with 2 sections, and 3 teachers with 1 section. Class size ranged from 10 to 23 students in the experimental group and from 9 to 24 students in the comparison group. In Portugal, there are no national guidelines for teacher preparation programs regarding preparation for teaching writing. Research, however, indicates that preparation to teach writing is poor (Pereira, 2001). None of the teachers had received either specific preparation in teaching writing in their preparation programs or professional development on writing instruction during their careers. In the present study, teachers in the comparison group followed the Portuguese language arts curriculum, while teachers in the experimental group followed the SRSD model for writing instruction in opinion essays.
2.3. Students A total of 380 students (214 in the experimental group and 166 in the comparison group) participated in this study. Initially, there were 507 students in the 14 eighth grade classrooms, with 285 students in the experimental classrooms (132 girls, 153 boys) and 222 in the comparison classrooms (118 girls and 104 boys). Only 436 students, however, participated in the pre-test (some students changed schools and others were not present during pretesting), and of these, consent and assent to participate was obtained for 380 students. Members of the research team collected the following information regarding students from school records: age, gender, grade retentions, grades in language arts and other subjects, and special education status; occupation and educational attainment of parents were also collected from these records. The age of the experimental group ranged from 11.92 to 15.42 years (M = 13.33; SD = .44) and the age of the comparison participants ranged between 12.67 and 16.42 years (M = 13.56; SD = .65). An independent t-test revealed a statistically significant difference in age [t(265.7) = 3.85, p < .001]. Concerning gender, 103 (48.1%) of the experimental group were male and 111 (51.9%) were female, while in the comparison schools the figures were 72 (43.4%) male and 94 (56.6%) female. The proportion of males and females in the two groups did not differ statistically (p = .36). In the experimental group, 21 students (10%) had repeated one grade or more, whereas 18 students (11.1%) in the comparison group had repeated one or more grades. The number of students repeating a grade in the two groups did not differ statistically (p = .72). The mean for the number of grade repetitions in the first group was .09 (SD = .37), ranging from 0 to 3, and for the comparison schools the mean was .20 (SD = .60) with the same range. There were four students (1.9%) with special needs in the experimental group and seven in the comparison group (4.3%). These numbers for the two groups did not differ statistically (p = .20).
The students from the experimental and comparison groups were also compared on their grades in language arts and overall school achievement (obtained at the end of the previous school year). For language arts, grades ranged from 2 to 5 for both groups (in the Portuguese educational system up to the ninth level, grades range from a low of one to a high of five), with M = 3.66 (SD = .82) for the experimental schools and M = 3.68 (SD = .82) for the comparison schools (p = 0.75). The general school achievement of the experimental group was M = 3.82 (SD = .77), and for the comparison group it was M = 3.69 (SD = .79), with the scores in both groups also ranging between 2 and 5. There was no statistical difference between the grades of the two groups (all p’s > 0.18). Finally, regarding the level of schooling completed by students’ parents (ranging from less than compulsory education up to the Ph.D.), no statistical differences were found for fathers (p = .11) or mothers (p = .25) between experimental and comparison groups. 2.4. Practice-based professional development for SRSD Before classroom instruction began, SRSD teachers participated in 14 h of professional development (across two days) in SRSD instructional practices. Professional development followed the PBPD model used by Harris et al. (2012). Teachers received notebooks with the guidelines and materials needed to implement all activities and lessons for opinion essays in their own classrooms. SRSD instruction was modeled, practiced, and discussed during professional development. Teachers and the research team also discussed how to adapt SRSD program and differentiate instruction to meet student needs in their classrooms (for more detail, see Harris et al., 2012). After PBPD, SRSD teachers met with research assistants weekly, after school for about an hour, to address any questions or concerns teachers had regarding SRSD instruction and how future lessons might need to be adjusted to meet teacher and student needs. The majority of teachers’ questions centered specific SRSD instructional activities (e.g., how to model the use of the selfregulation procedures) and on ways to differentiate instruction for students experiencing difficulty mastering the material taught. 2.5. SRSD instruction After the opinion writing pre-test was administered, teachers delivered SRSD instruction. The experimental teachers taught SRSD for opinion essays for one, 45 min class once a week, from October until January. Portuguese language arts classes are assigned 225 min a week and school directors can organize the class time as they wish. Class time is typically divided into 5 blocks of 45 min each. Teachers and researchers agreed that all writing activities would be conducted only in this one class a week to ensure that this was the only writing instruction students received during this study. In the other 180 min allocated to language arts curriculum, teachers focused on components such as oral and written comprehension, reading, and language knowledge in accordance with national guidelines. The materials for teaching opinion essay writing via SRSD were taken from Harris et al. (2008), including six lessons from Chapter 8 (lessons 1, 2, 3, 4, and 5 for older students) and one lesson from Chapter 9 (Lesson 1). We also used other SRSD material from the Harris et al. book, including mnemonic charts, graphic organizer, essays rockets graph, and cue cards. We adapted this material so that it was in Portuguese and appropriate for grade 8 classrooms in Portugal (Rebelo et al., 2013); we created materials that followed and respected the objectives and structure of SRSD but that would make sense to Portuguese teachers and students. The mnemonics for the SRSD strategies, in particular, required careful attention because a literal translation was not possible.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
We tried to find acronyms that had the same significance as the original mnemonics and matched a Portuguese word. The general planning strategy involving three steps, represented by the mnemonic POW (Pick my idea, i.e., pick an initial idea of what to write about; Organize my notes, i.e., write a plan using a graphic organizer; Write and say more, i.e., continue modify and upgrade the plan during writing), was translated to the Portuguese acronym PODE (Can in English). PODE signifies: Pega (Pick, in English), Organiza (Organize, in English), Dizer e Escrever (Say and Write, in English). The genre-specific planning strategy for opinion essays to help students to carry out the second step of POW (Organize my notes), TREE (Topic sentence, Tell what you believe; Reasons, three or more, and elaborate on or say more about each one; Ending, Wrap it up right; and Examine, do I have all my parts?), was translated to the Portuguese mnemonic TRAVE (Beam in English). TRAVE, the Portuguese mnemonic, represents: Tema (Topic sentence, in English), Razões (Reasons, in English), Acabar (Ending, in English), VE (Ver para Examinar) (See to Examine, in English). In order to verify that SRSD was implemented with fidelity (i.e., SRSD was delivered as intend), we developed lesson checklists. These checklists were used to monitor which steps of the lesson were completed by the teachers and should be filled by research assistants and by teachers. They contained a description of each activity in a lesson and a place to check if the activity was completed or not completed. Teachers carried out the six stages of the SRSD model, recursively as appropriate, with time spent on stage depending on the needs and rate of progress of their students. In the first stage, Develop Background Knowledge, the writing and self-regulation strategies (the general writing strategy POW/PODE, the genre specific writing strategy TREE/TRAVE, and self-regulation strategies including goal setting, self-instructions, self-assessment, and selfreinforcement) were introduced and discussed. At this stage, students acquired the knowledge, understanding, and vocabulary needed to apply the general and genre-specific writing strategies as well as the self-regulation strategies. During the Discuss It stage, teachers and students discussed the importance and utility of using the writing and self-regulation strategies. Next, teachers modeled how to use POW/PODE and TREE/TRAVE to write an opinion essay, showing students how to apply them (Model It stage). Simultaneously, they modeled the use of the self-regulation strategies, showing and explaining out loud such things as how to set goals, how and when self-instructions might be used, how to self-assess, and how to self-reinforce. If the students had not yet memorized the mnemonics (PODE and TRAVE) by this point, further practice was provided (Memorize It stage). During the Support It stage, teachers supported students through collaborative writing experiences by helping them to write opinion essays using POW/PODE and TREE/TRAVE, while self-regulating the writing task. Gradual release of control led to the last stage, Independence Performance, where students could use the strategies to write opinion essays without the teachers’ help.
2.6. Writing instruction in the comparison classrooms To document how writing was taught in the comparison classrooms, we administered a 64 item Writing Activities Questionnaire focused on writing instructional practices - those recommended in the eighth grade Portuguese curriculum, and those common in SRSD1. For each of the 64 items presented on the scale, teachers 1
This scale is available from the authors.
5
indicated how often it occurred in their classrooms across the school year using a 5-point scale: 0 (never), 1 (up to 30 min a week), 2 (from 31 min to 60 min a week), 3 (from 61 min to 90 min a week), or 4 (more than 90 min a week). Teachers also were asked if they had received inservice or preservice instruction in teaching writing. If a teacher answered Yes, they were asked to describe this instruction. Finally, teachers responded to an open ended question regarding any other features of their writing instruction they could report. Responses to the Writing Activities Questionnaire indicated that none of the teachers had received preservice or inservice instruction in teaching writing. The teachers generally followed the Portuguese language arts curriculum for grade 8 students, teaching grammar, capitalization, punctuation, spelling, sentence construction, organization of text, planning, and revision. Teachers reported spending more time each week on teaching grammar (one teacher reported spending more than 90 min a week, and the other six more than 60 min) and punctuation (two teachers reported spending more than 90 min a week, four from 61 min to 90 min, and one from 31 to 60 min) than on the development of planning (five reported spending up to 30 min a week and two from 31 to 60 min) and revision (one teacher answered never, two reported spending up to 30 min a week, and the other four reported spending from 31 to 60 min a week). Teachers reported teaching a variety of text genres, especially those recommended for this grade level, i.e., narrative, opinion essay, argumentative, and descriptive. The majority of teachers reported spending between 61 and 90 min per week teaching text genre, although all reported spending 31–60 min teaching opinion essay writing. Narrative writing received the most attention (four teachers reported spending more than 90 min and three reported 60–90 min). Regarding elements of instruction included in SRSD, teachers reported teaching only some features of the planning strategy POW/PODE (most reported up to 30 min per week) telling students they should organize their writing but not teaching any strategies for doing so. Regarding TREE/TRAVE, only two teachers indicated teaching all of its components. Two teachers reported teaching only premise and conclusion, one teacher only premise and reasons, one teacher only premise, and the last teacher only reasons. All of the teachers reported spending less than 30 min per week on these aspects of opinion essay writing. The two teachers who reported teaching all TREE/TRAVE components only explained them to students. None of the teachers reported following the stages of instruction in the SRSD model. Modeling was referred as a general method to teach writing, but not specifically reported for teaching writing opinion essays. Comparison group teachers did not report development of self-regulations strategies in their educational practices, including goal setting, self-instructions, self-assessment, and self-reinforcement. For example, they reported never teaching students to set goals and break them into sub-steps (goal setting), ‘‘talk to themselves’’ as they write in order to guide, organize and structure their writing (self-instruction), determine whether or not they included all the parts on an opinion essay (self-assessment), or reinforces themselves (selfreinforcement). The times teachers reported regarding components of writing instruction, however, must be interpreted in light of their responses to the questionnaire’ open ended question. All of the teachers explained that they included in these times the total time they spent on these elements across all of the language arts curriculum components (oral and written comprehension, reading, writing, language knowledge). Regarding text genre, for instance, teachers explained they focused on genre not only in teaching writing, but also when teaching oral comprehension and reading. The majority of the teachers reported spending up to 30 min a
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
6
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
week teaching writing, while some teachers acknowledged that they did not teach writing every week. Thus, the comparison group teachers reported generally spending less time a week specifically teaching writing than the SRSD teachers. 2.7. Assessments 2.7.1. Fidelity of SRSD instruction Lesson checklists were developed that included all activities and elements for each lesson. A research assistant observed 25% of instructional sessions, spread across instruction; this percentage of lessons observed is higher than the 20% of sessions of schoolbased intervention recommended for observation in establishing fidelity (Gresham, Gansle, Noell, Cohen, & Rosenblum, 1993; Hulleman & Cordray, 2009). Teachers told the observer where they would start and added any changes they had made in that lesson for their students before the lesson began; observers checked off lesson elements as they were completed. Treatment fidelity for each lesson was computed by dividing the number of lesson elements taught by the total number of elements possible and multiplying the outcome by 100%. Teachers were given a copy of the checklist for each lesson, and were asked to check off each step as it was completed each time they taught. These teacher checklists were collected from the teachers at the end of instruction. 2.7.2. Writing assessments Pre-test, posttest, and maintenance (two months after postintervention probes) writing probes were used to evaluate the effects of SRSD instruction on students’ opinion essay writing. The probes were administered by students’ classroom teachers, and the SRSD instruction and comparison groups were assessed at the same time. Students were given up to 45 min, the full class period, to complete their paper. Students wrote their opinion essays in response to writing prompts related to the use of technology (e.g., Would it be possible to live without a computer today?) teachers recommended as suitable and interesting for eighth grade students. Prior to scoring, each opinion essay was typed into a Word document. Identifying information was removed and spelling, capitalization, and punctuation were corrected to avoid any influence from surface level features, such as legibility and spelling errors, on examiners’ judgments about writing performance (Graham, 1999). Each opinion essay was scored for number of words and for number of structural elements. Structural elements included those taught to students in the experimental group: premise, reasons, explanations (why an author believed a particular premise or why they refuted a counter premise), conclusion, and elaborations (additional information on or examples of a premise, reason, or conclusion). For premise and conclusion, ‘‘0’’ was allocated if the element was absent and ‘‘1’’ if the element was present. For reasons and elaborations, 1 point was awarded for each unique reason and elaboration included. Structural elements were assessed for three reasons. First, the planning strategy students were taught encouraged them to use these structural markers to generate ideas and think about their topic (e.g., students were encouraged to generate and consider multiple reasons to support their premise). Thus, if SRSD instruction was effective, students’ writing should evidence an increase in these basic structural elements. Second, as noted earlier, national assessments in Portugal stress the assessments of these elements in the evaluation of opinion essays (Gabinete de Avaliação Educacional/Educational Evaluation Office, 2013). Third, these structural elements represent the basic building blocks of a good opinion essay and define this genre (see Scardamalia, Bereiter, & Goleman, 1982). Our second writing measure, number of words, is a common outcome measure in writing intervention research (see for
example Graham et al., 2013), as it is difficult to write a convincing argument or a good paper without a reasonable number of ideas and elaborations. While longer text is not always better text, it often is (Page & Petersen, 1995), as correlations between number of words and text quality are high (see Bangert-Drowns, 1993; Morphy & Graham, 2012). It should be noted, though, that text can be improved when students eliminate extraneous or repetitive material. While we anticipated that SRSD students would produce longer text essays in the current study, it was possible that instruction would result in more succinct text, as the planning process and self-evaluation processes built into instruction may eliminate unneeded material. Consequently, we viewed number of words written as an important outcome variable. All opinion essays were scored by a trained research assistant; one-third of the papers were independently scored by a second trained rater. Inter-rater reliability for the three evaluation points (pre-test, posttest and maintenance), respectively, was .88, .87 and .77 (M = .84, Cohen’s kappa). Each essay was also scored for number of words written using the word count tool in Word. 2.7.3. Social validity Social validity was assessed immediately after instruction. Teachers completed the Teachers’ Intervention Rating Profile and students the Students’ Intervention Rating Profile we constructed. Each rating profile presented a series of statements about the social validity of the SRSD intervention that teachers and students responded to using a 5-point Likert-type scale (strongly disagree = 1 to strongly agree = 5). The scale that the teachers completed included 15 items (e.g., SRSD instruction helped students to write better opinion essays; After SRSD instruction, students included more opinion essays’ structural elements in their writings; After the intervention, students showed a greater interest in writing; With SRSD instruction students became more self-regulated). The student scale contained 10 items (e.g., SRSD activities were interesting; With SRSD instruction I learned to write better; With SRSD instruction I learned to organize my ideas when I am writing; I liked to participate in this instruction programme). Internal consistency of the Teachers’ and Students’ Rating Profiles, measured by the Cronbach’ alpha, were .86 and .88, respectively. 3. Results We examined whether: SRSD instruction was implemented as intended (i.e., treatment fidelity), students in SRSD instruction outperformed students in the comparison condition on structural elements and number of words written, and teachers and students viewed the SRSD instruction positively. Means and standard deviations for structural elements and number of words are presented in Table 1. For both writing measures, multilevel modeling for repeated measures was conducted to examine the impact of SRSD in comparison to the control condition across pre-test (0), posttest (1), Table 1 Performance of student participants in opinion essay writing and number of words. Student performance
Time
Groups Experimental (n = 214) M (SD)
Control (n = 166) M (SD)
Opinion essay
Pre Post Follow-up
3.56 (3.85) 6.07 (3.70) 4.86 (3.20)
3.90 (3.45) 3.62 (2.96) 2.65 (2.34)
Word count
Pre Post Follow-up
229.06 (93.30) 176.47 (71.13) 167.00 (69.62)
178.07 (84.37) 148.89 (83.90) 155.41 (95.28)
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
7
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
and maintenance (2) (RStudio, 2013). Each of these analyses proceeded in the fashion as described below. For level-1 variables (time of testing and the outcome variable), we (1) determined the interclass correlation (ICC) looking at changes over time of testing; (2) examined if scores for the outcome measure randomly varied among individuals; (3) investigated the form of the relationship between time of testing (i.e., pre-test, posttest, and maintenance) and the outcome measure (i.e., Do scores generally increase, decrease, or show some other type of relationship with time?); (4) verified whether the relationship between time and the outcome measure was constant among students or whether it varied on a student-by-student basis; and (5) examined if the model fit improved by incorporating an autoregressive structure with serial correlations and heterogeneity in the error structure. We then conducted the analyses by adding the level-2 variables in order to try to explain the random variation in mean test results (intercepts) and in time of testing results slope. First, we examined the impact of adding treatment (SRSD versus comparison) to the model. Then we examined the effects of treatment, gender, and age. As noted earlier, there was a small, but statistically significant, difference between students’ age in the two groups. In addition, we included gender in the analyses, as girls are typically better writers than boys at these ages (Graham, 2006a). For both structural elements and number of words, we ran a model where the level-2 variables included school along with treatment, gender, and age. However, the model with only treatment, gender, and age provided a better fit to the data than the model where school was also included (structural elements: AIC = 5897.60; BIC = 5973.02; LogLik = 2933.80; p < 0.0001; number of words: AIC = 12726.25; BIC = 12806.7; LogLik = 6347.126; p < 0.05). The model where school was added to treatment, gender, and age did not converge, even though a model with school as the only level-2 variable was statistically significant. As a result, we only report the results for the model that included treatment, gender, and age. 3.1. Treatment fidelity The SRSD writing intervention was implemented with acceptable fidelity. Observations of 25% of the instructional sessions indicated that teachers completed 77.86% (SD = 11.61%; range 65–100%) of the writing activities. The lower score occurred in only one class session where the teacher was dealing with behavioral issues. In addition, SRSD teachers were asked to check each activity of a lesson as they completed it, using the same checklists. These checklists were collected at the end of instruction; teachers reported completing 82.38% (SD = 19.96%; ranged between 44
and 100) of the lesson elements. Finally, at the end of instruction all teachers confirmed that they did not teach writing in any other language arts classes with their students and that they did not use anything new for writing instruction other than SRSD. 3.2. Structural elements Analysis of level-1 variables (time of testing and number of structural elements) resulted in an ICC coefficient value of 14.3%, indicating that this amount of variance in students’ scores was explained by properties of the students. Examination of fixed effects (p < 0.001), slope variability (p < 0.464), autocorrelations (/ = 0.139) and variances of response changes over time (showing that a model that allowed for decreases in variance fit the data better than a model that only included the quadratic effect: AIC = 6018.80; BIC = 6049.01; LogLik = 3003.40; p < 0.0001) revealed that: (a) students randomly varied little in mean levels of structural elements in comparison to how they vary through time; (b) there was a quadratic relationship between time of testing and structural elements; (c) the strength of the quadratic relationship did not randomly vary among students and (d) there was significant variance heterogeneity in the data. When the level-2 variable of treatment alone was added to try to explain random variation in mean test results (intercepts) and in time of testing results slope, we found that the expected test score for students in the SRSD group was on average 1.74 structural elements higher than the expected score for students in the comparison group (p < 0.001). When the level-2 variables of sex, and age were subsequently added to the analysis (along with treatment; see Table 2), a statistically significant interaction between treatment and time of testing was obtained at both posttest and maintenance (as expected), with SRSD students making statistically greater gains from pre-test to posttest (p < 0.001) as well as pretest to maintenance (p < 0.001) than comparison students. While the interaction between gender and treatment was not statistically significant, girls in the study included on average 0.51 more structural elements than boys (p = 0.025). In addition, with all three level-2 variables included in the analyses, SRSD students included 1.4 more structural elements in their opinion essays than comparison students. 3.3. Number of words Analysis of level-1 variables (time of testing and number of words) resulted in an ICC coefficient value of 56.4%, indicating that this amount of variance in students’ scores was explained by properties of the students. Examination of fixed effects (p < 0.001), slope variability (p < 0.0001), autocorrelations (/ = 0.7) and
Table 2 Multilevel regression modeling analyses of the relationship between treatment, age, and gender and time of testing for structural elements in opinion essays. b Model with Treatment, Age, and Gender
(Intercept) Treatment Age Gender Timea Timeb Treatment Timea Treatment Timeb Age Time (0–1) Age Time (0–2) Gender Time (0–1) Gender Time (0–2)
3.657 1.400 0.383 0.508 22.650 10.941 36.725 23.890 9.759 1.835 9.236 10.999
SE
DF
T
P
0.197 0.231 0.209 0.266 5.398 5.311 6.333 6.231 5.712 5.620 6.183 6.083
752 376 376 376 752 752 752 752 752 752 752 752
18.560 6.055 1.836 2.253 4.196 2.060 5.799 3.834 1.715 0.327 1.494 1.808
0.000 0.000 0.067 0.025 0.000 0.040 0.000 0.000 0.087 0.744 0.136 0.071
Note: Treatment = SRSD versus control group; 0 = pre-test; 1 = posttest; 2 = maintenance. a Time = Linear effect of time; b Time = Quadratic effect of time.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
8
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
Table 3 Multilevel regression modeling analyses of the relationship between treatment, age, and gender and time of testing for number of words in opinion essays. b Model with Treatment, Age, and Gender
(Intercept) Treatment Age Gender Timea Timeb Treatment Timea Treatment Timeb Age Time (0–1) Age Time (0–2) Gender Time (0–1) Gender Time (0–2)
178.379 28.047 15.959 36.931 351.954 302.809 542.889 72.047 41.22 25.765 74.604 66.591
SE
DF
T
P
6.19 7.263 6.55 7.090 98.59 78.203 115.66 91.748 104.31 82.749 112.91 89.571
752 376 376 376 752 752 752 752 752 752 752 752
28.815 3.862 2.437 5.221 3.570 3.872 4.694 0.785 0.395 0.311 0.661 0.743
0.000 0.000 0.015 0.000 0.000 0.000 0.693 0.756 0.000 0.433 0.509 0.457
Note: Treatment = SRSD versus control group; 0 = pre-test; 1 = posttest; 2 = maintenance. a Time = Linear effect of time. b Time = Quadratic effect of time.
variances (AIC = 12859.25; BIC = 12899.54; LogLik = 6421.623; p = 0.065) revealed that: (a) students varied more in their mean levels of number of words in comparison to how they varied through time; (b) there was a quadratic relationship between time of testing and number of words; (c) the strength of the quadratic relationship varied randomly among students; (d) the variance heterogeneity in the data was not modeled. When we added the level-2 variable of treatment to try to explain random variation in mean test results (intercepts) and in time of testing results slope, we found that the expected score for students in the SRSD group dropped an average of 27.8 words more than the expected number of words for students in the comparison group (p < 0.001). When the level-2 variables of sex and age were subsequently added to the model (along with treatment; see Table 3), a statistically significant interaction between treatment and time of testing was obtained (p < 0.001), with students in the SRSD condition showing a greater drop in number of words written from pre-test to posttest than students in the comparison condition. Such a statistically significant difference, however, was not evident from pre-test to maintenance (p = 0.756). It is important to note that the average length of essays decreased from pre-test to posttest to maintenance for students in both groups (see Table 1). While the interaction between gender and treatment and age and treatment were not statistically significant, girls in the study wrote on average 36.9 more words than boys (p < 0.001), and students who were one year older wrote 16 words less than their younger counterparts (p = 0.015). Moreover, with all three level-2 variables included in the analyses, SRSD students wrote 28 words less than comparison students. 3.4. Social validity Responses to the Teachers’ Intervention Rating Profile and the Students’ Intervention Rating Profile were positive. Teacher responses (on a 5-point Likert-type scale from strongly disagree = 1 to strongly agree = 5) ranged from 4 to 5 (M = 4.58; SD = .41). Students’ perceptions of treatment acceptability were also positive (M = 3.80; SD = .51) with item scores ranting from 2.07 to 4.93.
Harris et al., 2012). Students who received SRSD instruction in opinion essay writing, as compared to students who received typical classroom instruction following Portuguese curriculum guidelines, showed meaningful improvements in the inclusion of basic structural elements (premise, reasons, conclusion, and elaborations) in their papers. These findings are consistent with earlier research in the United States, where similar SRSD professional development was provided (e.g., Harris et al., in press, 2012). SRSD did not increase the length of students’ compositions. While the length of SRSD instructed students’ essays decreased, after SRSD instruction these students included most or all of an opinion essay’s critical elements in their compositions. Their compositions became more organized and inappropriate text was eliminated, which may explain the lack of an effect on length. Previous SRSD research includes mixed results on length; several studies have found that after SRSD instruction students wrote better, but not longer, compositions (Harris et al., 2009, 2012). Length, or writing more, was not a goal for students in this study (as it was not in previous studies); this goal might be addressed in future research once initial competency in a genre is established. SRSD was implemented with fidelity and both teachers and students reported strong social validity. Teachers believed that SRSD had a positive impact on the writing of their students; students were positive about the instruction they received and found it very interesting. Treatment fidelity and social validity results are very important to the continuation and expansion of research on SRSD and to enabling its future implementation in Portugal and other countries. Importantly, this study demonstrated that PBPD for SRSD was effective when applied in a different cultural context, that of Portuguese schools. It is important to note, however, that we adapted the SRSD lessons used in this study so that they would fit within the Portuguese educational context. The process of adapting programs to other cultures, and researching their effects, needs greater attention if evidence-based practices are to be generalized and used globally. This study provides initial evidence for the generalizability of PBPD for SRSD outside of the United States. 4.1. Examining the effectiveness of SRSD
4. Discussion Given the complexity of writing, it seems reasonable to expect special care needs to be taken when teaching it. Nevertheless, the teaching of writing is neglected in Portugal (Pardal & Festas, 2011) and the United States (Gilbert & Graham, 2010). We provided intensive professional development in SRSD for opinion essay writing to teachers of eight grade students following the PBPD model (Ball & Cohen, 1999; Grossman & McDonald, 2008;
Teachers in the comparison group did use some methods in their classrooms common to those used in SRSD instruction (see Section 2.6). Thus, examining the differences in instruction between the SRSD and the comparison teachers can help build understanding as to why SRSD was effective in this study. We begin by noting one additional critical factor –teacher preparation for teaching writing. As noted previously, none of the teachers in this study reported receiving specific preservice or inservice
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
preparation in teaching writing. PBPD for SRSD supports teachers in developing deep understanding of the general and genre specific writing strategies to be taught, the changes in performance that can be expected in their students, and evidence-based methods for developing these strategies. In the United States, while most teachers report some preparation in teaching writing, most preparation occurs after initial certification and many teachers report that they are not adequately prepared to teach writing (Brindle, 2012; Graham & Harris, in press). Teaching writing is complex and challenging; teaching it skillfully requires considerable knowledge about what to teach, when to teach it, and how to teach it. PBPD for SRSD helps meet these needs. In addition, there were many meaningful differences between the SRSD instruction and business as usual in the comparison classrooms in this study. Comparison teachers in the present study, as compared to those who received PBPD for SRSD, spent more time teaching grammar and punctuation than they did teaching planning; did not devote as much time to teaching opinion essay writing; told students to organize their writing but did not offer or teach them any strategies for doing so; most commonly reported teaching students only 1–2 of the common elements of opinion essays; explained the few opinion essay elements they shared with students but did not model or support their inclusion in student writing; did not teach students any strategies for self-regulating the writing process; and did not model the writing process using self-regulation, general, and genre-specific writing strategies. We believe that all of these factors play a role in the results found in this study. One factor important to address here is whether or not these same results in terms of improvement of elements would occur without all of the stages and supports available in SRSD. While more research is needed here, several studies indicate that for most writers, and certainly for writers who are not making adequate progress in writing, all of the stages and components are beneficial (cf. Graham et al., 2013). These studies show that after the first three stages of SRSD instruction (see Section 1.1; the genre elements have been deeply examined and found in model essays, the mnemonics and importance of each part of the genre strategy have been discussed and students indicate their understanding of them, and the teacher has cognitively modeled the writing process using the self-regulation and writing strategies), little progress occurs in many students’ inclusion of the genre elements in their own writing. Thus, Harris and her colleagues have argued that ‘‘P.E.E.ing (Post, Explain, Even Model) in the classroom’’ is not enough for most students to deal with the complex factors involved in writing. More meaningful progress is typically seen in SRSD instruction only with the gradual release of control that occurs across stages 5–6. SRSD is a complex, multi-component instruction approach (see Section 1.1) developed for use across complex academic learning, not just for writing. Harris and her colleagues have traced the development of SRSD elsewhere more completely than we can here (cf. Harris, 1982; Harris & Graham, 1992, 1996, 2009; Harris et al., 2009), but all of the components and characteristics of SRSD instruction have been developed and continue to be fine-tuned based on multiple theories that have produced extensive empirical support for each characteristic and component. Clearly, components analyses of both SRSD and PBPD for SRSD are needed. Graham et al. (2013) identified five studies that compared SRSD with explicit development of self-regulation (the usual model) to SRSD with these components removed. They found that the added value of teaching self-regulations procedures specifically in the SRSD model was 0.48 standard deviations. No other components analyses studies of SRD were found; additional research is needed to determine what components of SRSD instruction are responsible for differing gains in students’ writing performance,
9
attitudes toward writing, writing self-efficacy, and so on. As Harris and her colleagues have argued, however, their experience indicates that each student brings a unique, complex set of personal characteristics and abilities to learning to write, thus necessitating the combination of the multiple components and characteristics of SRSD instruction when teaching multiple students (cf. Harris & Graham, 2009).
4.2. Lessons learned, limitations, future research and conclusions In the current study, we successfully addressed many of the difficulties of doing school-based intervention research, but others must be addressed in future research. We also learned several important lessons to share with other researchers. First, we were successful in revising SRSD instruction for Portuguese teachers and students. We adapted the SRSD lessons used in this study to the Portuguese educational and linguistic context. Adapting SRSD material, especially mnemonics, required carefully attention, as we have seen. Critical to our success was collaborating with a Portuguese language arts’ teacher; he played a key role in the adaptation of the original mnemonics and materials for SRSD to Portuguese. This kind of collaboration will be critical to investigating evidence-based practices developed in one country to the cultural and linguistic contexts of other countries. Similarly, our research team included members who had done extensive research on SRSD and PBPD for SRSD in the United States. This collaboration helped ensure the integrity of the PBPD and SRSD approaches. In order to test the generalizability of school-based interventions such as SRSD outside of their original countries, it is very important that such collaboration, or at least dialogue, be established. Another important lesson we learned was to work to find ways to increase the rate of parental consent and student assent in school-based research. Of the 436 students who took the pre-test, we received consent and assent for 380 students. In future studies, researchers might achieve even higher levels of participation by reaching out to parents to inform them further of the evidence base for the intervention we were studying. This has the added benefit of informing parents about the importance of school-based intervention research. One of the most important lessons we learned concerns resources needed to conduct school-based research. School-based research is complex and demanding, requiring numerous resources, and thus, funding. It is important to engage in discussion with national government and organizational leaders to establish the importance of such research and develop adequate funding for these efforts. Such funding is not readily available in many countries. Because our resources were limited, we faced challenges that resulted in limitations in our study. In the present study, we lacked the funds to holistically score students’ essays at each time point. Qualitative, holistic scoring is a common procedure for scoring writing quality in intervention studies (Graham & Perin, 2007). Trained raters read each paper to obtain a general impression of overall writing quality at that grade level, attending to features such as ideation, organization, sentence structure, aptness of word choice, and grammar. Training raters, achieving reliability, and scoring large numbers of essays is expensive. Future research on the generalizability of PBPD for SRSD should include assessment of effects on writing quality as well as structural elements. In addition, future research should investigate the effects of PBPD for SRSD on national writing exams. Because criteria used in scoring the Portuguese national exam include evaluation of opinion essays’ structural elements (premise, reasons, elaborations, and conclusions), PBPD for SRSD in opinion essay writing could impact results on national (or state) writing exams targeting opinion essay writing.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
10
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx
Another limitation of this study is the lack of observations of instruction in comparison classes, which we did not have the resources to conduct. Although we had the teachers’ answers to the Writing Activities Questionnaire, future research needs to include observations of typical instruction to rather than rely solely on teachers’ self-reports. In the present study, it appears some teachers in the business as usual comparison group spent less time directly teaching writing during the language arts blocks than did the teachers in the experimental group. While comparing school-based interventions to business as usual in schools is important, future studies might seek to control time spent specifically teaching writing. Research, however, indicates that spending more time teaching many of the elements of writing comparison condition teachers focused on (such as grammar and punctuation) typically does not result in better compositions (Graham & Perin, 2007). Further, we were not able to investigate an alternative, evidence-based writing intervention in comparison to either SRSD or typical instruction. Future research should examine the differing effects of different approaches to writing instruction. Another possibility in future research into the generalizability of PBPD for SRSD outside of the United States is to compare the effects of SRSD instruction in different writing genres (e.g., one condition receives SRSD for narrative writing and the other receives SRSD for opinion essay writing; both groups are tested in both genres), as done by Harris et al. (2012) with students in the United States. This is possible as research indicates little to no improvement in either of these genres given instruction in the other genre. This kind of design has the added advantage of allowing all teachers and students involved in school-based research to participate in evidence-based practices. Similarly, we were not able to investigate effects of PBPD for SRSD among teachers on variables such as knowledge, efficacy for teaching writing, attitudes toward writing instruction, and so on. In future research, it is important to procure funding to assess multiple outcomes among teachers in addition to fidelity of instruction (Harris et al., 2012). Further, in this study, as in Harris et al. (2012), some students did not make as much progress as their teachers thought they could have given more differentiated instruction. Further research is needed to determine how best to support teachers, during and after PBPD, in differentiating SRSD instruction for students with varying writing abilities in their classrooms. Establishing the importance of allocating more instructional time for writing should also be addressed in future research. Given the positive results obtained in the present study, we are hopeful that in future Portuguese schools and their directors will allow allocation of more time to this kind of writing instruction and research, allowing us to conduct studies further examining our approach to professional development. In conclusion, this is the first study of PBPD for SRSD in opinion essay writing carried out in Portugal, and the first to demonstrate that SRSD implemented by classroom teachers following PBPD can be effective in teaching opinion essay writing to students outside of the United States. Our experiences conducting this large scale school-based intervention study resulted in important lessons to share with future researchers. In addition, further research on scaling-up and cross-cultural generalizability is needed and warranted to explore the effects of PBPD for SRSD in other countries, cultures, populations, and genres of writing. References Almeida, T. (2012). Desafios ao desenvolvimento profissional: Do trabalho cooperativo ao nível da escola a um grupo sobre a escrita [Challenges to professional development: From school cooperative work to a writing group work]. Doctoral thesis not published. Lisboa: Faculdade de Psicologia e de Ciências da Educação da Universidade de Lisboa.
Almeida, T., & Simão, A. V. (2007). Concepções de professores sobre o processo de composição escrita [Teachers’ conceptions about composition writing processes]. In A. V. Simão, A. L. Silva, & I. Sá (Eds.), Auto-regulação da aprendizagem [Self-regulation learning] (pp. 41–61). Lisboa: EDUCA. Baker, S. K., Chard, D. J., Ketterlin-Geller, L. R., Apichatabutra, C., & Doabler, C. (2009). Teaching writing to at-risk students: The quality of evidence for SelfRegulated Strategy Development. Exceptional Children, 75, 303–318. Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In L. DarlingHammond & G. Sykes (Eds.), Teaching as a learning profession: Handbook for policy and practice (pp. 3–31). San Francisco: Jossey-Bass. Bangert-Drowns, R. (1993). The word processor as an instructional tool: A metaanalysis of word processing in writing instruction. Review of Educational Research, 63, 69–93. Brindle, M. (2012). Examining relationships among teachers’ preparation, efficacy, and writing practices. Unpublished dissertation. Vanderbilt University. Carvalho, J. B., & Pimenta, J. (2005). Escrever para aprender, escrever para exprimir o aprendido [Write to learn, write to express what was learned]. In B. Silva, & L. Almeida (Orgs.), Actas do Congresso Galaico-Português de Psicopedagogia [Proceedings of Psychopedagogy Galaico-Portuguese Congress] (pp. 1877–1885). Braga: Centro de Estudos em Educação e Psicologia da Universidade do Minho. Cook, B. G., Smith, G. J., & Tankersley, M. (2012). Evidence-based practices in education. In K. Harris, S. Graham, T. Urdan, C. B. McCormick, G. M. Sinatra, & J. Sweller (Eds.). APA educational psychology handbook: Theories, constructs, and critical issues (Vol. 1, pp. 495–527). Washington, DC, US: American Psychological Association. De La Paz, S. (2005). Teaching historical reasoning and argumentative writing in culturally and academically diverse middle school classrooms. Journal of Educational Psychology, 97, 139–158. De La Paz, S., & Graham, S. (2002). Explicitly teaching strategies, skills, and knowledge: Writing instruction in middle school classrooms. Journal of Educational Psychology, 94, 291–304. Festas, I., Damião, H., & Martins, I. (2010, April). Estratégias de escrita de textos em alunos universitários [Text writing strategies on university students]. In M. Simões (Chair), Experiências de Ensino e Aprendizagem na Universidade: Retenção de conhecimentos, contratos e estratégias de aprendizagem, reorganização curricular, job shadowing, avaliação [Teaching and learning experiences at the university: knowledge acquisition, learning contracts and strategies, curriculum organization, job shadowing, and evaluation]. Symposium conducted at the meeting Conference Learning and Teaching in Higher Education. Évora (Portugal). Fraenkel, J. R., & Wallen, N. E. (2003). How to design and evaluate research in education (5ª ed.). New York: McGraw-Hill. Gabinete de Avaliação Educacional [Office of Educational Evaluation] (2011). Provas de aferição 2.° ciclo – Língua Portuguesa [2.nd cycle exams- Portuguese Language]. Lisboa: Ministério da Educação. Gabinete de Avaliação Educacional [Office of Educational Evaluation] (2012). Provas de aferição 1.° ciclo – Língua Portuguesa [2.nd cycle exams- Portuguese Language]. Lisboa: Ministério da Educação. Gabinete de Avaliação Educacional [Office of Educational Evaluation] (2013). Provas finaise exames nacionais – Ensino básico e secundário [Final probes and national exams – Elementary and secondary levels]. . Gabinete de Informação e Avaliação do Sistema Educativo [Office of Information and of Evaluation of Educational System] (2005). Estatísticas da Educação 04/05 [04/05 Educational statistics]. Lisboa: Ministério da Educação. Gilbert, J., & Graham, S. (2010). Teaching writing to elementary students in grades 4–6: A national survey. Elementary School Journal, 110, 494–518. Graham, S. (1999). Handwriting and spelling instruction for students with learning disabilities: A review. Learning Disability Quarterly, 22, 79–98. Graham, S. (2006a). Writing. In P. Alexander & P. Winne (Eds.), Handbook of Educational Psychology (pp. 457–478). Mahwah, NJ: LEA. Graham, S., & Harris, K.R. (in press). Common Core State Standards and writing: An introduction. Elementary School Journal. Graham, S., Harris, K. R., & McKeown, D. (2013). The writing of students with LD and a meta-analysis of SRSD writing intervention studies: Redux. In L. Swanson, K. R. Harris, & S. Graham (Eds.), Handbook of Learning Disabilities (2nd ed., pp. 405–438). NY: Guilford Press. Graham, S., & Harris, K. (2003). Students with learning disabilities and the process of writing: A meta-analysis of RSD studies. In L. Swanson, K. Harris, & S. Graham (Eds.), Handbook of research on learning disabilities (pp. 323–344). New York, NY: Guilford Press. Graham, S. (2006b). Strategy instruction and the teaching of writing: A metaanalysis. In C. A. MacArthur, S. Graham, & J. Fritzgerald (Eds.), Handbook of writing research (pp. 187–207). New York, NY: Guilford Press. Graham, S., McKeown, D., Kiuhara, S., & Harris, K. (2012). A meta-analysis of writing instruction for students in the elementary grades. Journal of Educational Psychology, 104, 879–896. http://dx.doi.org/10.1037/a0029185. Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99, 445–476. Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S., & Rosenblum, S. (1993). Treatment integrity of school-based behavioral intervention studies: 1980– 1990. School Psychology Review, 22, 254–273. Grossman, P., & McDonald, M. (2008). Back to the future: Directions for research in teaching and teacher education. American Educational Research Journal, 45, 184–205.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
I. Festas et al. / Contemporary Educational Psychology xxx (2014) xxx–xxx Harris, K. (1982). Cognitive-behavior modification: Application with exceptional students. Focus on Exceptional Children, 15, 1–16. Harris, K., Lane, K., Driscoll, S., Graham, S., Wilson, K., Sandmel, K., et al. (2012). Tier 1, teacher-implemented self-regulated strategy development for students with and without behavioral challenges. Elementary School Journal, 113, 160–191. http://dx.doi.org/10.1086/667403. Harris, K., & Graham, S. (1996). Making the writing process work: Strategies for composition and self-regulation (2nd ed.). Cambridge, MA: Brookline Books. Harris, K., & Graham, S. (2009). Self-regulated strategy development in writing: Premises, evolution, and the future. British Journal of Educational Psychology, 6, 113–135. Harris, K., Graham, S., Brindle, M., & Sandmel, K. (2009). Metacognition and children’s writing. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Handbook of metacognition in education (pp. 131–153). Mahwah, NJ: Erlbaum. Harris, K., Graham, S., Mason, L., & Friedlander, B. (2008). Powerful writing strategies for all students. Baltimore: Paul H. Brookes Publishing Co. Harris, K., & Graham, S. (1992). Self-regulated strategy development: A part of the writing process. In M. Pressley, K. Harris, & J. Guthrie (Eds.), Promoting academic competence and literacy in school (pp. 277–309). New York: Academic Press. Harris, K., Lane, K., Graham, S., Driscoll, S., Sandmel, K., Brindle, M., et al. (2012). Practice-based professional development for self-regulated strategies development in writing: A randomized controlled study. Journal of Teacher Education, 63(2), 103–119. http://dx.doi.org/10.117/0022487111429005. Hulleman, C. S., & Cordray, D. (2009). Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Intervention Effectiveness, 2(1), 88–110. Institute of Education Sciences (2012). IES practice guide: Teaching elementary school students to be effective writers. NCEE 2012-4058, . Mason, L. H., Kubina, R., & Taft, R. (2009). Developing quick writing skills of middle school students with disabilities. Journal of Special Education, 44, 205–220. Ministério da Educação e Ciência [Ministry of Education and Science] (2009). Programa de Português do ensino básico [Portuguese language arts curriculum for elementary and middle levels]. . Ministério da Educação e Ciência [Ministry of Education and Science] (2012). Metas curriculares do ensino básico [Standards for elementary and middle grade levels]. .
11
Morphy, P., & Graham, S. (2012). Word processing programs and weaker writers/ readers: A meta-analysis of research findings. Reading and Writing: An Interdisciplinary Journal, 25, 641–678. National Center for Education Statistics (2012). The nation’s report card: Writing 2011. Washington, D.C.: Institute of Educational Sciences, U.S. Department of Education. National Center on Intensive Intervention (2013). Academic interventions tool chart. . Page, E., & Petersen, N. (1995). The computer moves into essay grading: Updating the ancient test. Phi Delta Kappan, 76, 561–566. Pardal, E., & Festas, I. (2011, September). A competência de escrita em Manuais do 10.° ano de Português [Writing competence on 10th level textebooks]. In I. Festas (Chair), Composição de textos [Text composition]. Symposium conducted at the meeting XI Congreso Internacional Galego-Portugés de Psicopedagoxia, Coruna (Spain). Pereira, L. A. (2001). A formação de professores para o ensino da escrita [Teacher training to the teaching of writing]. Cadernos de Formação de Professores [Notebooks of Teacher Training], 2, 35–49. Rebelo, J., Sousa, C., Inácio, M., Vaz, J., Festas, I., & Oliveira, A. (2013). O Programa de Escrita SRSD e a sua adaptação para um estudo em escolas de Coimbra. Revista Portuguesa de Pedagogia, 47(1), 31–51. Robinson, D. H., Levin, J. R., Schraw, G., Patall, E. A., & Hunt, E. B. (2013). On going (way) beyond one’s data: A proposal to restrict recommendations for practice in primary educational research journals. Educational Psychology Review, 25(2), 291–302. http://dx.doi.org/10.1007/s10648-013-9223-5. Rogers, L. A., & Graham, S. (2008). A meta-analysis of single subject design writing intervention research. Journal of Educational Psychology, 100, 879–906. http:// dx.doi.org/10.1037/0022-0663.100.4.879. RStudio (2013). RStudio: Integrated development environment for R (Version 0.98.490) [Computer software]. Boston, MA. Retrieved December 22, from http:// www.rstudio.org/. Scardamalia, M., Bereiter, C., & Goleman, H. (1982). The role of production factors in writing ability. In M. Nystrand (Ed.), What writers know: The language, process, and structure of written discourse (pp. 175–210). San Diego, CA: Academic Press. Wong, B. Y. L., Hoskyn, M., Jay, D., Ellis, P., & Watson, K. (2008). The comparative efficacy of two approaches to teaching sixth graders opinion essay writing. Contemporary Educational Psychology, 33, 757–784.
Please cite this article in press as: Festas, I., et al. Professional development in Self-Regulated Strategy Development: Effects on the writing performance of eighth grade Portuguese students. Contemporary Educational Psychology (2014), http://dx.doi.org/10.1016/j.cedpsych.2014.05.004
