Assessment Methodology 1

Principle Six of CBT

Competency assessment tool

Judgment

Pointers…

A two-way process and means-

Judgment is used to conclude whether competence has been demonstrated, based on the available evidence

The decision made by the assessor on whether the evidence provided meet the principles of assessment and rules of evidence Characteristics of quality evidence

The decision made by the assessor on whether the evidence provided meets the principles of competency-based assessment and rules of evidence

Encompass the performance criteria to demonstrate achievement of the outcomes (elements)

Validity, reliability, flexibility, fairness

Reflect the skills, knowledge and attitudes defined in the relevant unit of competency

The decision made by the assessor, based on the evidence provided and evaluated, on whether the candidate is competent or not yet competent

Show application of skills in the context described in the Range

Pointers…

Characteristics of quality evidence

Relevant assessment system policies and procedures and organizational/legal/ ethical considerations are addressed in making the assessment decision

Demonstrate competence over a period of time Demonstrate repeatable competence Be the work of the candidate Be able to be verified Demonstrate current skills/knowledge of the candidate

Requirements for the collection of quality evidence Assessment must address the scope of the unit and reflect all components of the unit A range of appropriate assessment methods/evidence gathering techniques is used to determine competency Requirements for the collection of quality evidence Evidence must be gathered in the workplace or simulated workplace Evidence collected must relate to a number of performances assessed at different points in time

Requirements for the collection of quality evidence Assessment meets the rules of evidence A judgment of competence should only be made when the assessor is confident that the required outcomes of the unit have been achieved and that consistent performance has been demonstrated Pointers in making assessment decisions Collected evidence is examined and evaluated to ensure that it reflects the evidence required to demonstrate competency and which: Encompasses all component parts of the competency standards and the dimensions of competency Address other related documentation Complies with the rules of evidence

Clear and constructive feedback is provided to the candidate regarding the assessment decision and recommendations are made, where required Portfolio... √ is a purposeful collection of work (outputs/documents) that demonstrates progress/ achievement in specific areas of endeavors √ The collection of evidence that may be presented to an assessor to prove competence at a job or task √ it invites the reader of the portfolio to gain insight about the learning of the individual Types of Portfolio Showcase Portfolio best for summative evaluation of student’s mastery of key curriculum outcomes. a collection of an individual’s best or favorite work which the person has selected by the student & the teacher only completed work is included compatible with audio-visual artifact development Types of Portfolio Process Portfolio documents all facets or phases of the learning process particularly useful in documenting student’s overall learning process Documentation Portfolio a systematic on-going record of an individual’s progress includes everything from brainstorming activities to drafts to finished products

Related documentation Assessment guidelines Assessment criteria Evidence to be collected as set out in the Evidence Plan

Forms of portfolio evidences Self-assessment schedule

Allows the candidate to rate own performance against the requirements of the relevant unit/s of competency

Where do I go from here? (Utilization of Results)

Certificates



Criteria for Evaluation:

Provides evidence of prior education or training completed on or offthe job

Thoughtfulness

Assessment feedback or completed checklists

 Growth and development in relationship to key curriculum expectancies and indicators

Provides evidence of on-the-job performance, observer’s comments and future action as a result of assessment

 Understanding and application of key processes

Written statements or references

 Completeness, correctness, and appropriateness of products and processes presented in the portfolio

Provides evidence of the candidate’s job performance, responsibilities, achievements and skills levels

 Diversity of entries

Job descriptions

CHARACTERISTICS

Provides evidence of prior work experience

Clearly defined purpose

Work journals

systematic and organized collection of documents

Provides evidence of tasks, activities or other achievements, accomplished by the candidate in the course of days or weeks at the workplace or in a community/volunteer role

pre-established guidelines for what will be included clear and appropriate criteria for evaluating the individual’s documents PORTFOLIO EVIDENCES

Work samples (e.g., reports, designs, etc,) Provides evidence of the candidate’s ability to do part or whole of work tasks or processes Finished products Provides evidence of candidate’s ability to produce a product or service Statutory declarations Provides evidence that the samples are the candidate’s own work Why use a Portfolio? Encouraging self-directed learning enlarging the view of what is learned fostering learning about learning

Valid Must relate to the elements, performance criteria, range of variables and evidence guide Current Must reflect the candidate’s current knowledge and/or skills in the relevant aspect of work Sufficient Must cover the full range of activities described in the relevant unit/s Consistent Must show that the evidence presented by the candidate consistently meets the standards under workplace conditions Work samples must be compared in terms of approach, style and skills levels evident, to evaluate reasonable differences

demonstrating progress toward identified outcomes Authentic creating an intersection for instruction and assessment providing a way for students/trainees to value themselves as learners

Evidences/materials presented are the candidate’s own work or contribution

Offering opportunities for peer-supported growth

Assessors need to be aware of:

Some Concerns on Portfolio Assessment

 falsified evidence

Why do they want me

 falsified qualifications

to do this? (Purpose) What evidences do

 work samples completed by a team rather than the candidate  work samples that have lost currency

I present?(Design)

The assessor should:

Where do I get them?

Cite original certification rather than photocopied documents

How will my documents be evaluated?

Compare certification from organization Contact individuals who have provided documents to confirm validity

Question candidate about the evidence and ask for explanations of the candidate’s involvement

Check that equipment is safe and reflects the type of equipment currently used in the industry

Compare work samples in terms of approach, style and skills levels evident to evaluate consistency and reasonable differences

Conduct the demonstration at a site which reflects conditions in the workplace

Look for variations in dates between different documents, e.g., resume

Set realistic expectations, e.g., time limits and quality specifications

Characteristics of a Portfolio Assessor

Use materials and applications that are relevant to the workplace and the unit/s of competency



Committed to the Portfolio Assessment concept and process

Use the checklist to help focus on key aspects of the work activity or skill and record that the candidate has performed all required tasks



Analytical - able to understand cause and effect of data



Displays sound, objective judgment - free from bias

Ask the candidate questions to check underpinning knowledge, including contingency management, task management and job/role environment skills



Has high work standards



Is systematic and well-organized



Pays attention to detail



Flexible - receptive/open to new ways of doing things

Ensure that the equipment works prior to conducting the demonstration Ensure that all required materials are available Prepare instructions for the candidate Questioning is appropriate for ...

Third Party Report Evidence provided by person who has first hand information on candidate performance

Times when observation cannot be used Verifying observations Underpinning knowledge

Often supplied by person in authority

Dimensions of competency i.e. contingency skills

Useful for verifying assessor observation

Oral questioning

Useful in assessing consistency of candidate performance

Involves the assessor asking the candidate a number of questions about real, simulated or hypothetical situations

Third Party Report The Assessor should select the best person Prepare materials that will help the observer provide relevant feedback on the candidate’s performance that is directly related to the relevant unit/s of competency Third Party Report Send the Assessment Tool to the concerned party

Oral questioning Complements observations and demonstrations as it provides a means of probing the candidate’s understanding of work Oral questioning … a valuable tool for collecting evidence of underpinning knowledge and its application across a wide range of contexts

Retrieve and assess the accomplished Assessment Tool

… can be used where written questioning might not be appropriate due to literacy and language problems

Note: It is the Assessor, not the third party observer, who makes the assessment decision!

Oral questioning

Demonstration Used when safety, expense or time are involved Conducted in a simulated workplace environment Planning and implementing…

…can be standardized to enhance validity and reliability … provides immediate information about the candidate’s knowledge and understanding …assists in determining what other evidence may need to be collected

Check that the demonstration complies with the information in the unit of competency including:

Interview

√ elements

√ Involves the assessor asking the candidate a number of questions about real, simulated or hypothetical situations

√ performance criteria √ range of variables √ evidence guide

Suggestions for using interview 1.

Remember that the main purpose of the interview is to gather information from the candidate

2.

Prepare for an interview as you would prepare for other assessments

3.

Do not intimidate or frighten the candidate

Dos and Don’ts of Interview Body Language Characteristics of competency-based assessment Multiple forms of evidence

1.

Do not barrage the candidate with large numbers of rapid-fire question

Workplace performance

2.

Wait for the candidate to respond

Workplace skills main focus of assessment

3.

Begin with broader questions and gradually ask more specific and focused questions

The assessment is individual, there is no comparison among others

4.

Keep notes so that you do not have to rely on your memory

Opening the Interview Greet the candidate, giving your name and position Explain the interview’s objectives: √ to acquaint assessor and candidate √ to learn more about the candidate’s

It includes the recognition of acquired competencies Basis of training and development It provides a judgement for the assessed workers: competent or not yet competent It is the basis for the certification of competency of workers Competency-based system When we assess the candidate as COMPETENT, we are saying that we have justifiable grounds to say that this candidate can do ALL that the competency requires-

background and experience √ to gather relevant information to Questions ???... support employment decision. Is there a sufficiency of evidence to make judgement ? Opening the Interview… Has the competency standard been demonstrated? Describe the interview plan Active Listening √ Talk less, listen more.

Will the competence demonstrated in this setting transfer to other settings? Are my judgements consistent with other assessors judgements at different times and different context?

√ Summarize periodically Use thought speed

Are my judgements consistent with the judgements of other assessors?

√ Prepare your next question

Assessment Systems

√ Analyze what the candidate is saying

Traditional

√ Piece together what the candidate is saying now in relation to something said during the interview

•

Associated to a course or programme

•

Assessed by means of subject

•

Parts of the programme are included in final examinations

•

Passing criteria are based on marking scales

•

Questions are ignored

•

It is done within limited periods of time

•

Statistical comparisons are used

√ Take notes Benefits of asking follow-up or probing questions 4Enables the assessor to collect enough complete data to evaluate a candidate in all the target dimensions. Skillful probing is the key to collecting complete behavioral examples to illustrate the dimension under assessment.

Benefits of asking follow-up or probing questions…

Purpose of assessment

4 Respondent “faking”. Probing to get specific examples of actions every time the candidate provides vague or general information would make candidates reluctant to shade information to pass the interview.

Recognize current existing competency of candidate/s

Benefits of asking follow-up or probing questions… 4Discourages theoretical responses or responses that the candidate thinks are “right” or “desired”. Dos and Don’ts of Interview Body Language

Determine if competency has been achieved following learning Determine training gaps Measure work performance Classify employees/support career development certification

Read and interpret competency standards

Assessor must inform the candidate about:

Determine the evidence needed to demonstrate competency

the unit of competency

Principles of competency-based assessment

the evidence requirements

Validity

the evidence gathering process

Reliability

the role of the assessor

Flexibility

Forms of reasonable adjustment

Fairness

Translate English into local dialects

Communication skills

Use oral in place of written instructions

Using active listening

Adjustments to the physical environment or venue

Accurately interpreting verbal messages

Considerations relating to cultural beliefs, religious observances, age or gender

Providing constructive and supportive feedback Legal and ethical responsibilities Using appropriate questioning to clarify and confirm instructions for evidence gathering

Occupational Health & Safety

Interpersonal skills

Confidentiality

Engaging in two-way interaction

Free consent of candidates

Using language and concepts appropriate to cultural differences

Adheres to quality system, policies and procedures of the workplace

Accurately interpreting non-verbal messages

Transparent reporting and recording

Using culturally inclusive and sensitive techniques

Integrity of the assessor

Assessment policies and procedures

ASSESSMENT PACKAGE

Qualification of assessors

Parts of an Assessment Package

Candidate selection

Assessment package contains the following:

Assessment records

I. Assessor’s Guide

Reporting procedures

Contents:

Assessment reporting procedures Assessment appeals

1.

General Instructions @@@

contains stages in the Conduct of Assessment. Each stage contains pointers for the assessor in the conduct of assessment

Links with overall quality management system Stage I – Before the Assessment Characteristics of candidates Stage 2 – On the day of the assessment Level of work experience Stage 3 – During the assessment Motivation for assessment Stage 4 – Post Assessment Age Gender Assessor’s Guide Supportive Assessment Environment 2.

National Assessment and Certification Arrangements@@@

Suitable for the assessment √ List of competencies covered by the Qualification Candidate is relaxed and not apprehensive √ Ways of acquiring the Qualification Establish rapport with candidate Accumulation of COCs Provide overview of the process Full Qualification Advise candidate on assessment conditions √ and other provisions provided in Section 17 of TESDA Answer candidate questions Circular # 09 s. 2007 Inform the candidate

Assessor’s Guide@@@ 4.

Registration signifies compliance of the TVET program / course with minimum requirements promulgated for the program / course.

Evidence Plan@@@ What Programs / Courses are covered under UTPRAS?

A planning tool used by the Assessor which contains the competency to be assessed, details of the type of evidence to be collected, methods of assessment and the criteria for assessment decision II. Competency Assessor’s Script @@@

All technical-vocational education programs / courses need to be registered as required by the provision of TESDA Law, Section 8 and TESDA Board Resolution 98-03. It includes industrial trades, agriculture, fisheries, home industries, caregiver, security and Overseas Performing Artists (OPAs) among others.

guides the assessor in the different stages of assessment with corresponding reference materials to be used

E.O. 358

III. Candidate’s Guide

Executive Order No. 358 Sept. 15, 2004

Contains:

To Institutionalize a Ladderized Interface Between TechnicalVocational Education and Training (TVET) and Higher Education (HE)

III. Candidate’s Guide √ Competency Assessment Agreement@@@ A joint undertaking between the assessor and the candidate “I agree to undertake assessment in the knowledge that information gathered will only be used for professional development purposes and can only be accessed by concerned assessment personnel and my manager/supervisor.” III. Candidate’s Guide √ Instructions for the candidate@@@

Education Cohort Estimates Executive Order No. 358 Section 1 The TESDA and CHED, in consultation with the concerned sectors and stakeholders, shall develop and implement a unified national qualifications framework that establishes equivalency pathways and access ramps for a ladderized system allowing for easier transition and progression between TVET and HE Executive Order No. 358 Section 2

contains specific instructions on: Tasks to be performed/project to be undertaken

•

Ways on how evidences will be gathered

National System of Credit Transfer

•

Time allotment per task/project

Post TVET Bridging Programs

IV. Rating Sheets@@@

System of Enhanced Equivalency

A. Contents

Adoption of Ladderized Curricula/Program 

specific questions or activity developed from the selected assessment method to be used for assessment

Competency Assessment Results Summary@@@ 

Modularized Program Approach Competency-based Programs Network of Dual-Sector Colleges and Universities

IV. Rating Sheets B.

The Framework shall include, as far as practicable, but not limited to, the following unified qualification and articulation mechanism:

•

Contains the candidate’s performance in the different assessment methods (per unit or cluster of units of competency)



Assessor’s Recommendation



General comments (strengths/improvements needed)

Accreditation/Recognition of Prior Learning LADDERIZATION DEFINED refers to all education and training mechanisms that allow students and workers progression between Technical Vocational Education and Training (TVET) and Higher Education (HE) or college and vice-versa.

BRIEFING ON PROGRAM REGISTRATION PROCEDURES FOR TVET Providers

Ladderized education opens opportunities for career and educational advancement to students and workers. Specifically, it intends to create a seamless and borderless education and training system that will allow you mobility in terms of flexible entry and exit into the educational system.

OBJECTIVES

LADDERIZATION MODALITIES

What is UTPRAS?

1) Credit Transfer

The Unified TVET Program Registration and Accreditation System is the quality assurance mechanism for the mandatory registration of technical-vocational programs / courses with TESDA.

- recognition & carrying forward of overlapping learning from a TVET program to a Higher Education degree 2) Embedded TVET Qualification in a Ladderrized Degree Program

- a scheme by which the TVET content in the relevant higher education or bachelor degree program are mapped or identified and the curriculum is restructured to provide TVET exit points. The units of competencies in the TVET qualification are usually clustered together to allow for early attainment of a TVET qualification.

All ladderized programs shall have to be authorized by both TESDA and CHED. TESDA-CHED AGREEMENTS Creation of Permanent Joint Technical Secretariat

Mapping of TVET Content in Ladderized Degree Program STEPS IN DEVELOPING THE LADDERIZED CURRICULUM 1)

Identify or map the TVET content in the bachelor curriculum to be ladderized using the Training Regulation and the PSG

2)

Sequence the subjects in such a way that TVET exit points can be realized in the first two years of the curriculum with due regard to pre-requisite requirements

3)

Develop modules of training and course design using the competency-based format

1) TVET to Ladderized Degree Program 2) TVET to Traditional Degree Program PRIORITY DISCIPLINES (SY 2006-2007) Agriculture and Fisheries Health and Medical Services Information and Communication Technology (ICT) Maritime Tourism / Hotel and Restaurant Management(HRM) Criminology Education Engineering GENERAL GUIDELINES The degree program shall conform with the Policies, Standards and Guidelines (PSG) issued by CHED; The degree program to be ladderized shall be included in the priority disciplines; There shall be full tech-voc qualifications based on the Training Regulations issued by TESDA in the degree program; The full tech-voc qualification in the ladderized degree program shall be registered with TESDA under the Unified TVET Program Registration and Accreditation System (UTPRAS). MODEL LADDERIZED CURRICULUM •

Transition period of one year (SY 2006-2007) to mainstream other existing ladderization programs within the framework of E.O. 358

TESDA-CHED AGREEMENTS There will only be one (1) Ladderized Program, i.e. 

This Ladderized Education System under E.O. 358 is a program jointly agreed by TESDA and CHED.



Ladderized program offerings of institutions should be registered both with TESDA, for TVET courses, and with CHED for the degree programs.

Composition: - TESDA and CHED core staff - Representatives from COCOPEA,

PASUC, TEVSAPHIL

INSTALLATION OF LADDERIZED EDUCATION SYSTEM IN THE PHILIPPINES UNDER E.O. 358 July 14, 2006 Malacanang, Manila