Assessment and Evaluation - Chapters From Baxter's Evaluating Your Students

I -

L-F1t

Lr---

Lr--r L: L

l--al

Lr--r Lr---l

LLl---

r r l--

a-t

FOR EDUCATIONAL PURPOSES ONLY

Eualua,ting youf

Stud,ents

.J

AndyBaxter

lr-.-

Ll--r

lr--r

r

t--r'

LLl_ t-_ g

t-t l-r

r-.t

l_l_ a--

a--t

t_g

lLLr-.-

r-..I l-.-r

LLa._---

)--,f

l-

r a-J

IL

.--

l-J-

Ll---

lI

IE R i c h m o nP d ublishing 19 Berghem Mews BlytheRoad L o n d o nW 1 4 0 H N O Andy Baxter1997 P u b l i s h ebdy R i c h m o nP d u b i s h i n@ g F i r s pt u b l i s h e1d9 9 7 All rightsre.served. llo paft.of thisbook may be reproduced,storedin a retrieval systemor transmittedin any form, electronic,mechanical,photocopying,recordingor otherwise, withoutthe prior permissionin writing of the publishers. However,the pub.lisher grantspermissionfor the photocopyingof those pagesmarked 'photocopiable', for individualuseor for usein classestaughtby the puichiser only. Under no circumstances may any paft of this book be photocopiedfor resale. ISBN:84-294-5067-X Depbsito legal:M-45897-2002 Printed in Spainby Palgraphic, S.A.

Itr E Il-

D-

E trE E

DI:

rl'1 Design JonathanBarnard Layout CeckoLimited CoverDesign CeoffSida,ShipDesign l l l u s t r a t i o n s C e c k oL i m i t e d& J o h np l u m b

j

":

rl'1

I

Dedication To my father- a greateducator.

Dl I F

I

tr=I Di

^_

"-1 q

q

Di

4 q

rr:I

-,8: I

> I-

.> I-

-> l:

_+ --l-

->

b h l_

FOR EDUCATIONAL PURPOSES ONLY

pARTA Assessment, testihg,evaluation

l--I

Lr--r Lr--r L-

Baxter, A. (1997). Evaluating your students. London: Richmond Publishing.

=\<

S.

r Lr r--

1, cHAPTER -

Whydo weassess students'learning?

r-.r

r---t

LLr--r L: 4..-

Therearemanygroupswho havean interest in assessing a student's abilities: parents, governments teachers, headsof departments, and,of course, the students themselves. However, we allsharethe samefour mainreasons for assessment: ... to comoare students witheachother ... to seeif students meeta oarticular standard . . . t o h e l pt h es t u d e n tl'esa r n i n g ... to checkif theteaching programme is doingitsjob.

r V

r--rl

1: r-I

ll: L.t

r L-1]

T A

s

K

L-f

l:

givenin your Writea [ist of the typesof tests(notjust foreign[anguages) school. Whyaretheygiven?Whichgroupis eachoneprimarity aimedat?Who arethe resultsfor? parentsgovernments studentsteachersheadsof departments others

a-.-l

1_: lL.I

1 To comparestudents with eachother

lf yourstudents wantto entera university to studya popularsubject, the university hasto select whichstudents on a comparative it takes.lt decides basis, problem: it wants 20% is A e.g. thetop of candidates. Butthere a consistency. goodyearof candidates maybe compared with a weakyear:thisyear'stop 20% maynot be asgoodaslastyear'stop 20%. However, it is stillthetop 20% that getthroughtheexam.Thisapproach hasbeencalled'rationing the carrots': perform, wellallthecandidates however onlythe top 20% getthrough. Althoughthissystemmayappearunfair,it is stilloftenusedby governments and parents to judgethequalityof a school.

To seeif students meeta particular standard

Largeorganisations, likethestate,or international boards, have examining certainstandards of proficiency do not that studentsmustmeet.Thesestandards reflecttheteachingprogramme necessarily that the studentshavefollowed: mayusedifferentbooksor syLLABUsEs. differentschools Sotheselarge haveto settheirown standards organisations or criteria,andseeif the student canperformat thislevel. likeindividual Othersmaller organisations, schools, canalsoseta particular based on their individually-agreed standard own criteria. frequently, though, willbasetheirassessment on theirown More schools programme. Theyanalyse coverin class, andthen teaching whatthestudents whetherthe students havelearnedit, oftenby givingan ACHTEVEMENT TEsr. assess

L-.tl

l: lL-t

.J

L: -.-,l_ g

t: g

l:

r Lr l-J

r

a4

a5

l: 1-a-tl

.1

l-a--t

L: l---

.EE students'learning? Whydo we assess .-\.-/----./

Testers differoverwhatan AcHTEVEMENT resrshouldactuallycover.lt could testeither: (e.g.in English, ... the overallobjectives of the syLLABUS the abilityto express past time,or the abilityto writein a varietyof styles), or (e.g.in English, ... the individual itemson thesvLlneus the pastsimple, or writing advertisements). Anotherreasonfor assessment is initialplacement. We cananalyse the students' in orderto seewheretheyfit intothe system.Forexample, abilities if the school hasrestrictions on spacein classes, theymaybe placedaccording to what percentage theyget (e.g.the top 10% go into the top class). Alternatively, there maybe ceftaincriteria the students areexpected to meet.lf oneclass concentrates on writingwhileanotherspecialises in grammarrevision, the students' classwill be determined by theirsuccess according to thesecriteria.

To helpthe student's learning

To checkif the teachingprogramme is doingits job

proficiency Whetherwe assess or achievement, we cananalyse the student's in a diagnostic way.Instead abilities of usingthe assessment to gradethe we useit to seewherethestudentneedsmorehelp.Forexample, student, the gradein writingan adveftisement, studentgetsan excellent but makesmany errorsin the grammarsection, especially in the presentsimplethirdperson-s. we maythendecide to givehim/heradditional helpandteaching in thisarea. Butsuppose getexcellent allthe students gradesin writingadvertisements, but all makemanyerrorsin the present simplethirdperson-s.we maythendecide programme to alterthe wholeteaching to giveallthe students additional help in thisarea. andteaching if teachers On a largerscale, andinspectors identifya commonproblemacross all government schools', a maydecideto alterthe wholeof itseducation programme. Summary Thereare,aswe shallseein thisbook,manywaysof assessing students. But probablythe mostcommonmethodof assessment is a test. c pRoFtctENcy rEsrsexamine a generalstandardin ability,regardless of the programme. teaching c ACHTEVEMENT TESTS examine whetherstudents cando whattheyhavebeen taught,eitherby testingspecific syLLABUs itemsor generalobjectives. c PLACEMENT TESTS area mixtureof the abovetwo, depending on what criteria we useto placethe student. c DlAcNosrlc rEsrsusePRoFtcrENcy or AcHTEVEMENT TEsrs to analyse strengths and weaknesses in the studentor theteachingprogramme itself.

l-

.l-

.D= IP l!-

.trD-

.uE Jf-

.E ,E ,E .E= .E: !l:

,E=

.u .E r E v -1

-z

P

-_1.

.P

rlA

.g K

Thinkof two differentteststhat you knowwetLa tanguage test or othertest that is usedin yourschool"andoneof anothersubjector abil.ity(tikedriving). programme Is the test basedon the teaching or not? Whosetsthe tesfs standards/criteria? How are the resuttsused?To comparestudents?To assessthe teaching programme? Forotherreasons?

-1

-v -1

-D.1

Dz .1.

v _1.

I-

-v I

1

-v ,Jr

.>'

b

h L_

CHAPTER 2

FOR EDUCATIONAL PURPOSES ONLY

What'sthe differencebetween testitrg,teachingandevaluation?

l---

Lr--.r

r

r--r

LL-

L -

What is testing?

L r--

r r r r r r l---

r--I l-.t

Everytimewe askstudents to answera question to whichwe alreadyknowthe answer,we aregivingthema kindof test.Muchof whatwe do in classis,in fact,testingstudents' knowledge. Herearesomeexamples. goes He to the cinema.They...? Finda word in the text that means'angry'. On the tape,wheredoesJohn tell Susan he wantsto visit? What is the main ideaof paragraphthree? Dictation:write down the following... That'sthat part of the lessonfinished.Whatdo you think we'regoingto do next?

l---

L,'I

Testingandteaching

L-t

L: L: L-tl

a-,

l-:

r g

L-'

l: r--I

L: ll-tl

r r--t

r--.!l

L: L: l: la-t a-t L.t

r r

Turningperformance into numbers

Testinghas,traditionally, measured the results of studentperformance. o We choosesomerepresentative samples of language. et We measure whethera studentcanusethesesamples. o We thentry to quantifythisby turningit intoa markor grade. e We keepa recordof thesemarksandusethisto givean endassessment. Overtime,alltestingtheory(whether languages or shampoo development) has traditionally beenbasedon a semi-scientific procedure, namely: 1 Measurethe performance. 2 Do something to affectthe performance. 3 Measurethe performance againandcompare the difference. Applyingthistraditional testingprocedure or modelto language learners has meantthat the languagelearneristreatedasa kindof plant.We measure the plant,applythe newfertilisetandthenmeasure the plantagainto seewhateffect the fertiliserhashad.As language we applya (pmcEmrNr) teachers, test,teach,and then givean ACHTEVEMENT TEsr to seehow muchbetterthe studentsare. In otherwords,testingisgenerally with eruunnennrroru, concerned that is,turning into numbers. oerformance

a.4

Plants

Languagelearners

Stage'l

plant measure

testthe present simple

Stage2

addfertiliser

teachthe present simple

Stage3

measure plantagain comoarethe difference

testthe present simpleagain comoare the difference

a--J

a--J

lj a5

l: l)-tJ

a-J

lj l-r.-r ]a

1l"--

-

It4

the differencebetweentesting,teachingand evaluatjon? __l9tt

Testingactivitiesand teachingactivities

fWe oftenaskquestions to checkthat the andtestinggo hand-in-hand. Teaching we sometimes whatwe havesaid.Equally, haveunderstood aska students question know to find out whetherwe needto teacha point.We instinctively whetherit isto teachor to testsomething. why we aska question: the followingtwo exercises. Compare

F P

Fill the gap with an appropriate form of the verb. Franceeveryyear since 1993. Francelast year.

)r-

.I-

Exercise I a John b John

F

(visit)

(visit)

Exercise 2

I-

.trIl-

In groups,discussthe differencesbetweenthe wo sentences. a John hasvisited Franceeveryyearsince 1993. b John visited Francelast year.

1 assumes havesomeknowledge and asksthemto that the students Exercise proveit. lt is clearlya testingactivity.Notethat if the studentsget the right we don'tknowwhy theywrotethat answer.lt may be a guess,or it answer, just might soundright. 2 asksthe students a question aboutthe language. In otherwords,it is Exercise - a generalisable askingthemto formulatea ruletheycanusein othersituations theirawareness of how the language works.lt theory.lt is alsotryingto increase help learn: it is hand, to them a teaching activity. On the other some istrying wouldsaythat peopledon't needto knowwhy it is right,theyjust teachers needto get it right. two moreexercises. Let'scompare Exercise 3 Composition:A Summer'sDay at the Beach(150words) Exercise 4 Readthe following two compositionsentitled A Summer'sDay at the Beach'. \fhich do you prefer and why? Underline all the words and ideasrelating to summer.Underline all the words and ideasrelating to the beach.Put a tick next to the parts you like in eachessay. Put a crossnext to the parts you don't like in each essay. If all the paragraphsgot accidentailyjumbled up, could you put them back in the right order?Vhat would help you do this? Discussyour ideaswith another group.

DIl-

"[!E .F l.L.LJll-

"u

.ttr

D: .E .tr-

.,tr E P

Homework: write your own compositionon the sametheme (150 words). *

Usingthesameideasaswe outlinedabove,Exercise 3 isclearlya test:it wantsthe studentto showuswhathe/shecando. Exercise 4, on the otherhand,clearlytries to makethe studentmoreawareof what he/sheistryingto do: it triesto increase beforegivingthe task.lt triesto helpthe studentto learn. awareness

tr.1

D._1

I-

.-1

.P 4

10

-* 1

P

v

1

I

b

What's the difference between testing,teachingandevaluation?

Lr*Lr---

Lr-..-

Teachingor testing?

: Sometimes, though,teachers canget confused aboutwhethertheyareteaching ' or testing.We canthinkwe areteaching whenwe areactuallytesting. : Thisis particularly truewhenwe try to teachthefourskills: reading, writing, speaking and listening. Herelanguage teachers facea majorproblem.We don't I reallyknowenough; thatis,thereareno clearrulesaboutgoodlistening, : readingandotherskills. All we havearesomerathergeneralised ideassuchas skimming andscanning, andthesearenot detailed enoughto helpusworkout an effectiveand progressive programme. teaching In otherwords,whenfacedwith a skillthat is difficultto teach,suchasgood listening, we normally answer thisproblem in oneof two ways.Eitherwe give the students lotsof opportunities to showwhattheyknowsowe canseeif they're improving. We askthemto read,writeor listento textsof increasing linguistic complexity andhopetheykeepthesamegeneralresults or evenimprove; or we the complexity of the questions. ; keepthe sametextsandincrease Thisis a bit likea doctorsaying your illness I don'tknowwhatcaused or why you'regettingbetter,but yourtemperature is goingdown. All we cando to teachthe four skillsisexposestudents to language andtaketheirtemperature via testingto seeif they'regettingbefcer. Or we substitute the skillthat isdifficultto teachwith onethat is easyto teach. Whilethe rulesfor skillsarenotveryclear,we do havesomeverygoodrulesfor gf&mmdrandvocabulary, whichmakesthemeasierto teach(however, writinga grammar/vocabulary testcanbecomplex, aswe shallseelater).Sowe sometimes believe we areteaching or testinga skill,whenreallywe are or testinggrammar or vocabulary. Forexample, manyspeaking tests . practising grammarrevision: aredisguised theycanbecomean oraltestof grammar. They don'ttestrealspeaking skills suchasinterrupting withoutcausing offenceat all. Why isthis?Because plant thesemi-scientific modelof testingwhichwe looked at earlierhassomemajorproblems. Thenextpartcovers theseproblems.

Problemswith testing

Problem1: Skillsintonumbers On pnce9, we sawthattestingis basedon an ideafromscience: measure, make , changes, measure againandcompare. Oneproblemwiththescientific modelisthatnot everything cannecessarily be measured in thisway.Therearesomethingswe caneasilytestin thisway,e.g. -s. the present simplethirdperson

r

r-..r

L-

tr r r r r r r r t,

r-.l--t

vl

a-,J

l-J

3--tt

a-t

L: L: l--

r r r )--t

l-a

r--l

LL: LLLL: Lr-.L-J

]-1,

.J

a-----

r--{

r r r r L.-

'

l_--

l-:t

l _.

-

l.-a

LLa-4

t-A

La-4

L-

. : : , :

Butotherskillsaremoredifficult to measure. How,for example, canwe quantify a student's abilityto makeusefulcontributions to the class? o First,we wouldhaveto define'useful'and'contribution' in a wavthatwe couldmeasure them. c We coulddefine'useful' as'successfully explaining something to another student'. c We coulddefine'contribution' put to the wholeclass as'answering a question by the teacher'. e We couldnow counthow manytimesa studentsuccessfully answered a questionandthe majorityof the restof the classunderstood. teacher's Theproblemwith thisisthatwe arenow measuring how manytimesa student 'successfully answered questionandthe majorityof the restof the a teacher's Thisisnot necessarily classunderstood'. the samethingasmakinga useful contribution to theclass. 77

ar< S-

between testing.teachingandevatuation? What'sthe djfference Il-

so therearetwo dangerswhenassessing skillsthat aredifficultto measure. e We maytakesomething we all understand andre-define it to makeit measurable; but, in doingthis,we maychangethe verythingwe aretrying to measure. e lf something is too difficultto measure, we leaveit out of the test- evenif the skillis veryimportant. In the end,we arriveat a position wherewe areonlymeasuring the easilymeasurable, ratherthan assessing the performance we aretryingto improve.

t-

F atL

> I> I-

Listento yourcotleagues having(11)conversations in the staffroom. Whatpercentage of their naturatspoken[anguage consists of fut[ sentences? Whatpercentage consists of sentence fragments [inkedby intonational devic€s and ums and ers? Howoftendo you teachstudents to speakin fragmented sentences?

Otherproblems with testing

Problem2: Resultsversusprocesses, whatversuswhy Anotherproblemwith thissemi-scientific systemof euRrurrrnrrvr mensunrmerur is that it doesnot recordeuAlrrArvF onrR.Measuring will tell usif the planthas grown,but not why (orwhy not).lt givesus information aboutthe results, but doesn'ttellusanythingaboutthe process. pnce10),we wouldgeta muchbetterideaof the In the exampleessay(sEe from Exercise student's abilities 4, because we couldseesomeof the processes behindthe work,e.g.we couldlookat wherethe studentputtheticksandcrosses in the essays, andthenseeif andhowthesewerereflected in his/herown essav.

E .tr-

.tr-

E E .E Il-

IL-

.u| I-

Problem3: Standardisation andodd results A thirdproblemwith the scientific modelisthatthefertiliser givento the plant mustalwaysbe the same,or the results cannotbecompared. we mustremovethe variables in orderto assess the success of the programme. lt isdifficultto seehow thiscanworkin teaching. ln schools, alltheteaching wouldhaveto bethe same, or we couldn'treallycompare the progress of individual students. Thismodelof testingthereforeleadsto ratherauthoritarian teacher-proof methodologies. Thescientific modelis alsomoreinterested in generaltrends,andstrange individual resultsareoftenignored.Forexample, imaginethat in a listening test allyourstudentsget9OT",but yourbeststudentonlygets10%. Forus as it isthat one odd resultthat we wouldwantto investigate. teachers,

D=

tr'=

^u .L.F-

-D-

-v .v .v .-.-1

TA

s

K

- perhaps choosea coursebook the oneusedin yourschool- andsetectat random:threelisteningexercisesthreereadingexercisesthreespeaking exercises. Whatis the purpose of eachexercise? Is it ...testinggrammarorvocabu[ary?(e.g.t+/r8rown-theonenqetc.) ... testingthe student'sunderstanding? (e.g.via multipte-choice questions aboutinformationin the texb information gaps;etc.) ... teachingthe studentto read/tisten/speak better?(e.g.Doesit includeadvice abouthowto improvereadingor tistening,pnctisingintenupting,etc.) ..- teachingthe studentto study?(e.g.Doesit teachclassroom tanguage? Does it hel.pthe studentto find answers to their ownquestions?, etc.)

_1

_1

4

Uz _-1

lz llz

.P <

1?

.U

-v -v

U

What's the difference between testing,teaching andevatuatjon? w

l-Lr---

Testingandevatuation

r

The relationship betweentestingandevaluation is similarto the relationship betweenthe cunnrculum andthe svlLaeus Thesvllnsusis a setof itemsfor the teacherto coverin a term.Butthe syLLABUs - the cunnrculurvr. is partof a biggermethodological scheme A language teachingprogramme is not onlywhatyou cover(thesvllnaus), but alsohow you coverit (theclassroom procedures), andalsowhy you coverit (theeducational approachor rationale behindyoursyLLABUs andclassroom procedures).

r-er

L-

ts I

l_-.

L-

r r r r r l--

l---

Evaluation

r-.t

r--r

r--l-,-t

LLLLt--t

r-.f

In thisbook,we will seeevaluation aswiderthantesting.Evaluation seestesting asa usefultool, but alsothinksthereareotherimportantcriteriafor assessing performance. someone's we wantto assess students' abilityto usethe present simple,but we alsowantotherinformation abouttheir(language) learning, e.g. ei Canthey usea dictionary? e Do theyactuallyusethe targetlanguage in class(e.g.for chatting)? c Are theirnoteswellorganised? c Do theycontribute to groupwork? e' Aretheywell behaved? lf we comparethisto the syLLABUS{uRRtcuLUM diagram, we canseea (simplified) similarity:

a-,

r I r

curriculum

r--I

6";

r-t

'.\-7-

r-I

r a--]

r l--t

LLL)-t

|-D

evaluation

Problems with testing: Canevaluation solvethem?

Problem1: Skillsinto numbers Evaluation is not limitedto numbers or justgivingstudents marks.Insteadof tryingto countor measure a student's abilityto makeusefulcontributions to the class,we cansimplyjudgewhetherhe/shemakesa contribution or not. In other words,you canbe subjective aswellasobjective. Butwhenwe makejudgements, we mustrealise thatotherpeople, including teachers and students, maynot agreewith whatwe think.Evaluation meansthat sometimes we will haveto justify,negotiate and possibly modifyour opinions. We may needmorethanonejudge- we mayevenneeda jury.

a-,

I

.--.J

L]

r IJ

a--c

LLl: L: L: a-t t4

)-J

a

-

r l-_-t

lI

Problem2: Resultsversusprocesses, whatversuswhy In additionto EttuiurRnrtoru, evaluation looksfor tlumrrunrroru: How did you learn that?why did you writethat?we aredoingsomething with the student,rather than doingsomething to the student.lf we hadto assess Miguel'sperformance year, pnce1o),or his overthe wouldwe ratherhavehisessayfrom Exercise 3 (sEe essayfrom Exercise 4 with hisnotesstapledto the backof it? Exercise 3 tellsus what, but Exercise 4 tellsus what,how, andwhy. In addition,by askingthesequestions, we will learna lot of extrainformation: ... whatthe studentthinkshelsheis learning ... whatthe studentthinksiseasyldifficult ... whatthe studentenjoys/hates doingin class 13

FU

andevaLuation? What'sthe differencebetweentesting,teaching

tlz

Iprogramme andthe studentdon'tmeet ... wheretheteaching programme ... wheretheteaching needsto be re-designed. In otherwords,we canusethe assessment procedure to developand improve : notonlythestudent, butalsotheteaching programme, andeventhe school.By - and more procedures andattitudes, we gainmoreinformation evaluating - thanby simplylookingat testresults. usefulinformation andodd results Problem3: Standardisation process. Evaluation doesnot wantto removethe variables in the assessment is interested in odd results Evaluation asit is exactlythiskindof resultthat may process. illuminate something aboutthe learning Equally, it doesnot want materials andmethodologies. Instead, teacher-proof evaluation triesto includeas manypeopleas possible, because all information is seenas possibly usefulfor improving the designof a teachingprogramme.

Who evaluates?

4, writinga goodtestisan extremely As we willseein cHRprrR complextask, not onlya lot of timeandresources, andrequires in but alsosomeexpedise analysis. Forthisreason, it tendsto be largeorganisations statistical suchas governments anduniversities thatwritebigtests,mainlybecause theyneedto yearafter!ear. C, srr pnce31 standards keepthesameNoRM-REFERENcED however, we aretryingto helpthe studentto learn.Evaluation With evaluation, is notjustan assessment, butan aidto learning. Thismeansthatthe more peoplewho areinvolved in the process, the betterthe process is. Summary As we haveseenin thischapter, to teachstudents skillswhicharedifficultto (i.e. questions we ask them lots teach, either of micro-tests) effectively to seeif or we substitute they'reimproving theskillthat isdifficultto teachwith onethat is easyto teach. in a skillwhichis difficultto measure, To assess students we eitherre-defineit to possibly it measurable; make but change whatwe aremeasuring or we leaveit onlythe easily-measurable. out of the testand measure ratherthanthe individual. Testing alsolooksat thegeneral, Individuals, whether or students, arevariables theyareteachers that haveto be removedfromthe process. lndividuals assessment areturnedinto eunrurrrnrrvr datalikeresults; and onrn,likeprocesses or attitudes, arestatistically removed. euALtrATrvE

T A

s

K

language learners? Whocanevatuate Theheadof yourschoolhasdecided to developa newassessment systemfor hasaskedyou to providea list of atl the endof the nextschootyear.He/She the peoplewho mighthaveusefulinformation abouta student's[anguage learningabitity.Makean appropriate [ist,then consider the fotlowingquestions. coutdeachgroupprovide? Whatinformation Giventhe systemasit existsnow,whowouldactual.ty be consutted? Whatinformationwouldyou get us'ingthe presentsystem? giventhe presentsituation? wouldbe missing, Whatinformation Wh'ichpartsof the missinginformationarethe mostimportantto incl.ude? Canyouthink of anywaysof incorporating theseimportantareasinto the present systemwithoutneeding to re-design procedure? the wholeassessment

t: IL JP

> > IfFID-

E .E-

.tr-

E

r= r=

D=

tr=

.tr.tr.trb .--.4

>

--4

P 4

P

4

.D= flz,

-v -

.P 1

74

.U

-v .V

= t I

?:?_

CHAPTER 3

1-

FOR EDUCATIONAL PURPOSES ONLY

Whatdo weassess?

f

l---

Ll---

IJ IJ

Beforewe canassess a student's performance, we needto decidewhatwe are goingto assess. At firstsight,thislookslikean easyquestion. As foreignlanguage teachers we evaluate the student's abilityin a foreignlanguage. Ejrlier*e g"u" the examples belowastestquestions (i.e.theteacherarready knowsthe answers).

l-

lr

ljr-.-

l: l--

I

l-_r

lj

T A

s

K i1

l-I

L-

r l---

what do you think eachof the foLlowing questions is actualtytesting? Thinkof youranswers beforeyoulookat the keybelow. 1 Hegoesto the cinema. They...? 'angry,. 2 Finda wordin the text that means 3 0n the tape,wheredoesJohnte[ Susanhe wantsto visit? 4 Whatis the mainideaof paragraph three? 5 Dictation: writedownthe fottowing ... 6 That'sthat partof the lesson finished. whatdo youthink we'regoingto do next?

,-t

LLL: L: r-.t r-,t a-'t

rcv

r

1 Thisistestinggrammar (usingthepresent simplethirdpersonplural). 2 Thisistestingvocabulary (recognising thatfurious isa synonym of angry). 3 Thisistestingthe student's abilityto listenfor detail. 4 Thisistestingeitherlistening for general meaning or inferring froma text.

---J

r-rr

LLL: l-t

5 Thisistestinggeneral ability(writing, reading, pronunciation, listening, spelling, etc.). 5 Thisistestingtheirabilityto inferlesson phasing fromtheirprevious learning experience. so we alreadytestthe students on a widerangeof skillsandabirities.

a-f

r r r l-.]

a-,-J

a-t a-,

l_ a-.f

|;

r r r l-t

l-_!

a--'

lj L'

l-

r l-f

r aa

r

Message andmedium

However, questions canhavemorethanoneanswer.Forexampre: Teacher: Miguel, wheredoesthe presidentof the lJnitedstateslive? Miguel(1):He livesin London. Miguel(2):He live in the WhiteHouse. Miguelgivesthe teachera problemhere.Hisfirstansweris grammatically correct but factuallywrong.Hissecondanswerisgrammatically wrongbut factually correct.Whichansweris better? Theanswerto thisquestionislt depends why youaskedthe questionLanguage teaching isconcerned with bothmessage andmedium.lf we aretestingthe third person-s,Answer1 mustbe correct. on the otherhand,we arealsotryingto teachstudents to communicate in a differentlanguage. Thegrammatical mistake thatMiguelmakesin Answer2 doesnotstopcommunication of the idea. Language teachers haveto balance two different'correctnesses': the rightidea, i.e.the message andthe rightformof expression of thatidea,i.e.the medium. 15

t

Whatdo we assess? "-/-

>1 versuslanguageuse components Which language Language itemsthatwe abilitiesdo we test? Anothercommondistinction the individual iswhetherwe assess (grammar, of a language i.e.the components put togetherto makea sentence, puts these the student how whether we assess or pronunciation); vocabulary, (i.e. togetherwhentheyactuallyusethe language the four skillsof components andwriting). reading listening, speaking, Otherskillsof usinglanguage (e.g.formalversus informal appropriate that is socially We needto uselanguage makingwhatwe sayfit what has SKILLs: etc.).We needotscouRsE vocabulary, (e.g. wasgoingto the cinema,not / saw he He said / John. saw before said been SKILLS, too, suchas how going...). We need srRnreclc lohn. JohnsaidJohnwas get information from a text,listenfor gist,etc. to taketurnsin speaking, learningskills Language e the abilityto usea dictionary of unknownwords e the abilityto work out meanings suchasaskingthe teacherWhat'sthepasttenseof e learningmetalanguage thatverb?etc.

> .-I

tz Ill

v v >. IIfI!-

IGenerallearningskills c contributing to, andworkingin,groupsin class c the abilityto knowwhat you knowandwhatyou stillneedto learn you don't know e, strategies for findinginformation in tests,etc. ei followingthe instructions or socialskills Otherbehavioural isthe for anylearner wouldsaythatoneof the primaryskills Manyteachers to staysittingin his/herchairworking ability,for at leastpartof the lesson, the class. aroundanddisrupting ratherthanwandering shouldwe includein ourassessment? Whichof theseabilities How muchshouldeachskillbe worth? how shouldwe recordourassessment? And,if theyareincluded, areto markor record. Thistakesus on to how easyor difficultthesescores

Othercriteriafor ifficult i nclusion:Easy/d to markor record

As we havealreadyseen,thereis alsoa problemabouthow to markor recordanswers. etc.)are multiple-choice, asnumbers(gap-fills, that giveresults Assessments (or a pieceof on results nnw scoREs) write the veryeasyto record.We cansimply a markout of twentyor paper,or we canconvertthisnumberinto a percentage, an A-E grade. 9), of markingwriting, thereareways,aswe shallseelater(in cHnprER Similarly, not as we are counting correctresults, more complex much althoughtheseare but judgingthe qualityof a pieceof writing.We shallalsoseethat the same likespeaking and behaviour. canbe usedfor otherabilities, systems to the class, e.g.the student's contributions However,whenwe wantto assess, to write will have will we problem. This almost certainly mean bigger a we have

tr-

tr: E I-

> > > F > >

v * _-4

* 4

* 1

P

-1

P

4

P 16

*

P 4 f-4

* = { = {_ t--r-

LLr--Ll--r

u l: t-

Whatdo we assess?

notes.Notesaremoredifficultto record:differentteachers will writedifferent amountsaboutdifferentthings.lf we wantthisinformation to be kept,we will haveto havefilesfor eachstudent. Summary l a n g u a g ec o m p o n e n t s ( g r a m m a rv, o c a b u l a r y , pronunciation)

ranguage use ( r e a d i n gw, r i t i n g , l i s t e n i n gs, p e a k i n g )

l a n g u a g ec o m p e t e n c i e s (socio-linguistd i ci ,s c o u r s e andstrategic ompetencies)

l-e

LL1.'

'truth'or'fact'

l e a r n i n gs k i l l s

g

lr-.'

l a n g u a g el e a r n i n gs k i l l s

g e n e r a bl e h a v i o u r a l a n d s o c i a ls k i l l s

Ll-r-

t--

ASSESSMENT

)-a

LL_ L_ l_: La5

t h i n g st h a t a r e e a s y to mark or record

a--f

t-f

t h i n g st h a t a r e easy to test What we typically assess

t h i n g st h a t a r e easyto teach

t-.J

a-1I

lj l-t

LlL: L: Ll--r

a g e n e r a li m p r e s s i o no f t h e s t u d e n ta s a ( l a n g u a g el)e a r n e r

a g e n e r a il m p r e s s i o no f t h e s t u d e n ta s a l a n g u a g eu s e r

a g e n e r a il m p r e s s i o no f t h e s t u d e n ta s a m e m b e ro f t h e c l a s s

a-/' a-t

a-f-J'

t_ a--,f

L_ L: L: L: .-t

l-t

l-t

r.-!]

l_ L:

Thinkaboutyourcurrentsystemof evatuating a studentat the endof theyear. Which of the skilts mentionedabove are includedin your current assessment system? Whichof the skillsarenot inctuded? Canyouthinkwhy they arenot included? Is onetypeof skittmorevaluable Forexamp[e, than another? it getshigher marks,or determines the studenfsassessment? Howarethesemarksrecorded? Whichskiltsareforma[lyassessed (i.e. you recordthe informationon the studenfsrecords)? Whichskittsdo think aboutwhenassessing the student,but arenot recorded officiatty? Doesyourpresent systemwork?Dothe goodstudents getthroughandthe bad fait? ones So howdoesyoursystemdefinea goodlearner? Finishthe sentence betow: In ourschool,a goodlearnerissomeone whocan...

1--t

t" J--t

L:

77

F 5 CHAPTER #

FOR EDUCATIONAL PURPOSES ONLY

Whatformsof testingandevaluation shoutdweuse?

I!P ]z

Directandindirecttesting " Whatis directtesting?"

rcsrNcmeanswe askthe studentto performwhatwe want to test. DrREcr

v v )-

_+ l-

" Whotis indirect t esting?"

of the rEsrNcmeanswe testthingsthat giveusan indication tNDtREcr performance. student's theirideas to seeif theycancommunicate we talkto students rEslNG, In orREcr practical to it is not always This obvious, but conversation. sounds in interactive do this,e.g.a studentmaybe awaysick,or the classmaybe too big to speakto if thereisa strictsyLLABUs to adhereto. for enoughtime,especially eachmember of how well we wouldfindthingsthatgiveusan indication rEslNC In rNDrRrcr good uselonger we know that speakers For example, speak. can thestudents We couldtheninventa testwhere thanweakspeakers. or utterances sentences with goodspeaking, e.g.the averagelengthof each skillsassociated we measure grade. higher the utterance, the longer the the utterance: to writea Thesameistruefor writing.We coulddirectlyaskthe students However, theremay numberof texts.Thiswouldtellusabouttheirwritingskills. (e.g.restricted markingtime)why we can'taskthe studentsactually be reasons we mightgivethema teston linkerwords(e.g. to performthisskill.Therefore give of theirabilityto writewell. usan indication etc.).Thismay however, a connection betweenlengthof above,we areassuming In theexamples ability;and linkersandwritingability. andspeaking utterance goodindicator resrisan extremely Oneproblemis makingsurethat the rruorRrcr lf we find students with we to test. the skill are trying HtcH coRRELAToN has a or are highscores on our linkers test,thisshowslinkers whocannotwriteachieve Thiswill makethe testresultinvalid. not a goodindicator. a negativeBAcKWAsH effect(srrpnce28): rEsrNcalsooftenproduces tNDrREcr linkers ratherthan teaching willspendhoursin theclassroom someteachers is in what the test. that's writing,because teaching rEsrtNc methods. Therefore, cannotexistwhenwe useDlREcr Theseproblems possible, usethem. whenever

.> l!-

-> -v -> P U

-U

-> _v -P

v +

T A

30

s

K

to tHotnrcr resrs.Someteacherswoutdarguethat, as TEsTS are preferabte DTRECT - in otherwords, weareteachingstudents teachers, to communicate language language. use to in tal.king(speaking Weuse[anguage and [istening),readingand writing. good canbe at grammar Someone but unabteto commun'icate in speechor writing.In this case,grammar, vocabulary and pronunciation tests mustreatty a[[ beformsof indirecttesting.

,-

v _v _v t-

v .1

-a a

I

shoutd weuse? Whatformsof testingandevaluation

_\_-/__

I

?; L. L-

LL: l--a

r-...-

l_ lr

l_L-{

L: L: L:

So we shoutdptacemoreemphasis tests than grammar on resuttsfromskiLl.s andvocabulary tests. Doyou agree? Doyouthinkyourschoolagrees? Howmuchof your students' final assessment is basedon their abitityto - to useandexperiment communicate with the language they havelearned? Howmuchis basedon theirabiLityto manipulate grammar andvocabutary? Do youagreewith this balance?

r-..-

r-.-r J.J

LL:

Norm-referenced andcriteria-referenced testing

g

]--J

L: gl

lj

a--t

t_: a-t

L: L: l-f

r ut

L: L: L: LL: L: Lr--)1

a-'

a4

)-C

)-J

=

a-tA

L:

r .-4

L: -

.-r]

L:

r [:

L--rtl

L:

Thesetermsreferto the way a testis madeandthe way the results of a test arepresented.

" Whatis norn-referencedWhenthe resultsof the testcompare a studentwith otherstudents. Theresult testing?" doesnot giveanyinformation aboutindividual performance, onlya comparison - fromthat yearandfrom otheryears. with otherstudents' performances

Forexample, a university wantsto restrictentryto its (science) courses to the applicants who havethe bestchance of successfully completing a course. In the past,it hasfound- perhaps by trialanderror- that students who scoredgoy. or morein theirfinalschool yearexamsarethe candidates mostlikelyto succeed. Therefore theyofferplaces onlyto students your from schoolwho got g0%or 1 6 / 2 0i n t h e i rf i n a sl c i e n ceex a m s .

"Whatis criteriareferencedwhen the resulttellsyouabout whatthe individual studentcando,anddoes testing?" not compare him/herwith otherstudents. lt describes certaincriteria thatthe studenthasbeenableto meet. Forexample, a studentisapplying for a job whichrequires the abilityto usea word-processor. Theemployer doesnot wanta computer expert,onlysomeone who cando basicword-processing: typing;file-management; simplecut,copy andpastecommands. Thestudenttakes a word-processing course andthefinal examteststheseskills. Theemployer doesn'tneedto knowif anyoneelseon the course wasbefteror worse.He/shesimplywantsto knowwhatthecandidate cando. "So wh.ich ls better...?' As we sawabove,it isthe stateand largeinternational examination boards who aremostconcerned with comparing people. Thevalueof their qualifications depends uponyear-on-year comparison andconsistency. For example, manypeoplefeelthat 'examsthesedaysaren'tasdifficultaswhenI wasat school'.Lackof year-on-year consistency devalues the state'sawards, like university placesand university degrees. However, thisneedfor consistency meansit isverydifficult to improveor develop examsbecause results wouldn't be comparable. Thereis a similarity herewith DrREcr andrruorRecr rEsrNc.g seenncr3o

31

!]<

we use? shouLd Whatformsof testingandeva[uation

P ---.

+

_1.

by the user:an employercan rEsrscanbe directlyinterpreted cRtrERrA-REFERENCED what the studentcando. to discover reada description you canonlyusethe result TESTS arenot directlyinterpretable: NoRM-REFERENCED resultwith yourpastexperience of otherpeople onestudent's by comparing with the samescore. it is obviouslymoreusefulfor usto knowwhat a andstudents, Forteachers do so we canwork on the areaswherethereare cannot and can student problems. needto reduce the stateand otherlargeorganisations However, measurement. complexitytheywantsimpleandconsistent

_v y --1

-v y P P P

Formsof evaluation

P

-> In the difference betweensimpletestingandevaluation. Firstwe mustremember questions to thisbook,the term testingis usedwhenwe areaskingthe students to ask We areusingthe termevaluation whichwe alreadyknowthe answers. - genuine questions: questions to whichwe don'tknowthe answers the students Do the studentsfeel they aregettingbetter?Havethey foundthe courseuseful? rEsrs, AcHTEVEMENT severaldifferenttypesof test:pRoncrENcy We earlieridentified pLACEMENT (serpnce8). However, in allthesetests, rESTS rEsrsand orncruoslc TEsrs, andour to seewheretheyfit in to our system we aretestingthe students We arein control. criteria. we areaskingquestions to learnaboutthe is differentbecause Butevaluation and attitudes, and aboutthe teachingProgramme. learningprocess student's suMMAlvE, evaluation: Therearethreecommontermsusedwhendescribing and coNcnuENTEVALUATIoN. FoRMATTvE

"Whatis summative Thisisdoneat the endof (a stageof) a process. In teaching, thismightbe at the summarising euoluotion?" endof a termor a year.In thisway,it is a kindof finalassessment, EVALUAToN looksat throughoutthat course.suMMArvE what hasbeenachieved year's procedure next teaching used, so that course can to the generalfeedback to what hasbeenmoreor lesssuccessful. be changedaccording

I_r-F

P P

-v ^> !-

.> llt-

"Whotisformative Thisisdoneduringa process so that the process canbe changedto makeit more might be feedback that a teachergetsto checkhow In this teaching, eualuation?"effective. is.Thefeedbackfrom the studentscanalso the teachingprogramme successful whilethereis stilla chanceto changeit for the affectthe teachingprocedure better- to helpthisyear'sstudentsratherthan nextyear's.

*Whotis beforeit starts,in orderto congruent Lessoftenreferredto, thislooksat the wholeprocess and evaluation of the coursematchthe euoluotion?"makesurethat the aims,methodology the statedpurposeand beliefs.Forexample,imagineyour purposeisto increase to designa courseand a way to evaluate oralfluency.Youaskteachers students' it. Theyreturnit to you andyou noticethat the testsincludewriting:this isvery EVALUATIoN wouldn'tmatchyouroriginalaims.In thisway,coNcRUENr vALtDtry. similarto corurrrut ) seepnce18

tttl-

!D-

F rF

7,t

D-

-lr-

II = II >;1 t I l--

t-

t_ l---

l---

L-

lL'

fJ-

L4

t_ t_ a-{ l-_t

a-tf

1-l

a4

t_ a--.il a4

Whatformsof testingandevatuation shouldwe use? --^

Puttingthe three together

You will noticethat there is, in effect, litfle differencebetweentheseterms, becauseevaluationneverends.suMMATrvE EVALUATToN at the end of a course informsthe teacher- and students- about how to changethe coursenext time to make it more successfuland/or more closelyrelatedto the beliefsbehindthe course.suMMATlvE EVALUAToN will also have implicationsfor the next course:if there are certainproblemareasthe following coursewill have to be changedto allow more (or less)time on these;or focus on differentareas. In other words, the differencebetweensuMMArvE, FoRMAIvE and corucRuerur EVALUAIoN is not one of how evaluationis done, but when and why evaluation i sd o n e . c coNcRUENT EVALUAION triesto keep the processon the desiredcourse. o FoRMATtvE EVALUATTON triesto alter the processwhile it is still going on. e suMMAlvEEVALUAION triesto assessthe successof the completedprocess.

Summary In thischapterwe havecoveredthe following. o rNDrREcr resrstestabilities relatedto the skillwe areinterested in. o DrREcr rrsrstestthe skillitself. o NORM-REFERENCED examscompareone person'.s performance with manyothers. o CR|rER|A-REFERENCED examsdescribe what one personcando without comparing themwith others. o coNcRUENT, FORMATIVE and surunnnlvE EVALUATtolr describe whenevaluation is done:before,during,or after.Butit is importantto remember that evaluation is not linear,but cyclical. Eachpartinformsthe other.

t]-_!,

l-. l-t tla----I l--:l

tt-t I

t4

l-:t_

.-tf

--t_

Thinkaboutwhenandwhyyourschoolevatuates its students. Doesit askthe studentgenuinequestions to improvethe schoofsteaching: a duringthe yearor teachingprogramme? b at the endof theyearor teachingprogramme? Doesit askyou, the teacher,genuinequestionsto improvethe schoo['s teaching: a duringthe yearor teachingprogramme? b at the endof the yearor teachingprogramme? Doesyourschoolexamine newcurricula, syLLeBuses andassessment procedures beforeimplementing themin orderto checkthey matchthe schoofsaims? Doesyourschoofsteachingprogramme have'officiafaims? Why?Whynot?

-!.

-= -|t

----_ --..-, 33 ----t-