(lif)e-portfolio

EDU928 Research Study Supervised by: Celia O’Hagan

(lif)e-Portfolio a framework for implementation

Lee D Ballantyne B00494857 May 2012

11/05/2012

Contents

1. Abstract

2

2. Introduction

3

3. Methodology

7

4. Themed Findings

10

5. Conclusions & Recommendations

14

Appendices

18

References

46

1. Abstract

Cambridge International Examinations (CIE) and University of Cambridge ESOL Examinations (ESOL) are investigating e-portfolios with a view to implementation. This research study examines e-portfolios in this, and in a wider context to gain a deeper understanding and lay the foundations for a successful adoption. Current drivers and issues are explored, as are stakeholder requirements. Two populations were involved in this requirements gathering: a focus group from CIE and ESOL, and a sample of teachers and candidates of teaching awards. This provided an opportunity to communicate potential benefits and existing guidance to key decision makers, as well as ensuring a learner-centred solution is delivered. It‘s hoped this approach will lead to a sense of ownership and informed decision making. A framework for implementation naturally emerged and it‘s the intention is that this framework and the outcomes from this research will inform the implementation process and subsequent related projects.

Keywords: e-portfolio, eportfolio, PLE, digital identity, learnscape, learning landscape, implementation

Terminology: User, learner and teacher are used interchangeably throughout in reference to the enduser of the e-portfolio, the learner role, who in this case happen to be teachers or trainee teachers. Similarly, assessment organisation, exam board and awarding body all represent those involved in the end-to-end administration of awards. Acknowledgements: I am indebted to an extended group of supportive people across Cambridge Assessment. Thank you all for your time and support. In particular I would like to thank: Irana Morrish, Phil Riding, Paul Beedle, and Elkie Wootten.

B00494857

2. Introduction

―Even with guidelines and case studies of exemplars those implementing e-portfolios seem often to reinvent the wheel, make really 'obvious' mistakes compared to those who have a deeper understanding of the area‖ (Joyes et al., 2010). This research aims to prevent reinventing the wheel by gaining a deeper understanding of e-portfolios and planning for a successful implementation. Firstly, by outlining the drivers and issues surrounding eportfolio adoption:

2.1 Policy E-portfolio use has grown steadily over the years. They evolved from a movement started by the Dearing Report (NCIHE, 1997), and specifically recommendation 20: ―institutions of Higher Education should (provide) a means by which students can monitor, build and reflect upon their personal development‖. Since 2001 a joint policy statement from Universities UK, SCOP, Quality Assurance Agency (QAA) and Learning and Teaching Support Network define this personal development as: ―a structured and supported process undertaken by an individual to reflect upon their learning, performance and/or achievements and to plan for their personal, educational and career development‖ (QAA, 2001, p 9). European, national and regional policies were implemented including the HEFCE Strategy for e-Learning (2005) and the DfES e-Strategy (2005, p 5) which states: ―We will encourage every institution to offer a personal online learning space to store coursework, course resources, results and achievements. We will work towards developing a personal identifier for each learner, so that organisations can support an individual‘s progression more effectively. Together, these facilities will become an electronic portfolio, making it simpler for learners to build their record of achievement throughout their lifelong learning‖. At the same time, e-portfolios were the key theme of a number of conferences worldwide. E-portfolio consortiums Europortfolio and LIfIA (Canada) along with various national governments stated a common goal that each citizen should have an e-portfolio for life by 2010 and the QCA, in their Proposed Blueprint for Delivering eAssessment (2004), stated that by 2009 all awarding bodies in the UK should be set up to accept and assess e-portfolios.

2.2 E-Portfolios for Assessment Assessment organisations are facing mounting pressure to provide sustainable, scalable e-assessment solutions. In addition to policy drivers, increasing candidate numbers (in an increasing number of countries for CIE and ESOL) are creating an administrative burden which is unfeasible in the longer term. Appropriate alternative assessment methods are crucial to sustain growth but ―technology should not be deployed simply for the opportunity to realise significant cost-savings or efficiency gains‖ but should be fit-for-purpose and, ideally, improve quality or service (Craven, 2009). An awarding body evaluates evidence of competency mapped to standards required for an award. This is most visible in the vocational sector and with teaching awards where paper-based portfolios have been in use for a number of years. The candidate collates a variety of evidence which is organised: tagged, referenced and cross-referenced; and presented for assessment and verification. There‘s an emphasis on this organisation and presentation as well as security, audit trails and archiving. Nonstandard features essential for assessment e-portfolios include ―publish‖ and ―audit‖ functions (Winkley & Roads, 2007, p 6). Current practice is to accept and, where appropriate, endorse e-portfolio systems which fulfil the requirements set by each awarding body (see Appendix I). This allows the awarding body to enforce good practices and there is significant evidence of quality improvement, particularly

3

B00494857

relating to achievement, retention, reliability and validity. While multiple methods are accepted there is an understandable focus on comparability with paper-based systems, which can ―slow down the rollout of some of the benefits‖ (Winkley & Roads, 2007, pp 8-9), however current methods have come under criticism. Both the Tomlinson Report (DfES, 2004, p 1) and the Intel, Microsoft and Cisco Education Taskforce (Kozma, 2009, p 13) state educational systems today (including assessment methods) are at odds with the skills and attributes required for the future. The Taskforce recommend that ―assessments should engage students in the use of technology and digital resources and the application of a deep understanding of subject knowledge to solve complex, real world tasks and create new ideas, content and knowledge‖. Similarly, the Re-engineering Assessment Practices (REAP, 2007) project found students‘ capacity to self-assess, reflect, and actively manage their own learning to be poorly developed even though they are highly valued skills in the workplace. This, in no doubt, contributes to the findings of the Leitch Report (HM Treasury, 2006) which predicts that the UK skills base will remain behind many other countries by 2020. A focus on developing and evidencing these skills, in addition to the prescribed objectives, would add significant value to the current assessment e-portfolio by narrowing this skills gap.

2.3 E-Portfolios for Learning JISC (2012a) reports ―emerging, often powerful evidence from practitioners and learners of how eportfolios can promote more profound forms of learning, as well their further potential in supporting for example transition between institutions and stages of education, and in supporting professional development and applications for professional accreditation.‖ These e-portfolios can be defined as a product created by learners, created using tools or systems, which facilitate certain e-portfolio related processes. These processes develop key skills such as reflecting, collaborating and self-directed learning. Reflective practice (embedded in an e-portfolio by the inclusion of a learner journal, blog or opportunity to comment) is seen as increasingly important in many professions (like teaching and healthcare); is a key element in a number of learning cycles (Dewey, 1938; Honey & Mumford, 1982; Kolb, 1984); and a widely acknowledged strategy to foster deep learning (Dewey, 1933; Donovan et al., 1999; Ewell, 1997; Schön, 1983). Reflection is notoriously difficult (JISC, 2012b), but case studies have demonstrated that providing support and structure helps (Reidinger, 2006) and for some learners their familiarity with social media and blogging is proving extremely valuable (JISC, 2012c). A simpler task is to place new content into the e-portfolio. To do this learners are forced to relate the new content to old. Schacter (1996, p 45) states that ―if we want to improve our chances of remembering an incident or learning a fact, we need to make sure that we carry out elaborative encoding by reflecting on the information and relating it to other things we already know‖. Therefore, in carrying out the simple task of placing new content in an e-portfolio we are forced to reflect and relate it to prior knowledge. The whole process of selection and presentation are deeply powerful learning events: constructing meaning out of diverse and seemingly unconnected material. Sharing is an intrinsic part of this process and yet social features which facilitate sharing, collaboration, and peer review are notably absent from many market leading assessment-driven e-portfolio systems. Collaboration or ―team-working‖ is ranked as second in importance only to ―communication skills‖ by 233 graduate employers in a recent survey (Archer & Davison, 2008, p 7). Both these skill sets can be developed and evidenced by the inclusion of social features in an e-portfolio. The learners can then construct personal meaning through interaction and dialogue, as well as develop self-assessment skills through peer assessment (JISC, 2010). The myriad of online niche communities is evidence of the effectiveness of contextualized, peer-based learning: the foundation of socio-constructivist and constructionist learning approaches (Brown, 2008). This integration of social tools and informal learning has somewhat redefined the e-portfolio as a ―personal learning environment‖ or ―learning landscape‖ to reflect the concept of aggregating information, evidence and networks from a wide variety of sources, wholly owned and directed by the learner using the same tools used in everyday lives. One aspect of this lifelong, life-wide ―learnscape‖ is the emergence of a professional digital identity (Cambridge, 2008).

4

B00494857

The European Institute for E-Learning (EIfEL) considers digital identity education to be as critical as physical and moral education and defines learning ―as the social construction of meaning and of one‘s identity‖ (Layte, 2008). Indeed there is evidence that digital identity, in relation to social media, is already factoring heavily in the recruitment process (Reppler, 2011; Swallow, 2011) in spite of a previous study suggesting 75% of employers are ―not familiar with e-portfolios‖ (Ward & Moser, 2008). Fundamental to these ideas of digital identity and a personal learning environment is the ability to create a personalised space. The benefits of personalisation are twofold. Firstly, ―a guiding principle behind reflective e-portfolio development – learner control – should apply to the tools learners use as well as to the content‖ (JISC, 2008, p 10). The ability to customise the e-portfolio (process), to integrate the learners‘ own choice of tools (tools or systems), and, ultimately, create a digital identity (product) is incredibly important to learners (Barrett, 2008; Tosh et al., 2005). Secondly, personalisation allows learners to take responsibility for their own learning: developing metacognitive skills and promoting autonomy (Rubín & Rümler, 2012). Personalisation is a key component of a number of government policies and it‘s recognised that using e-portfolios supports that aim (Gilbert, 2006, p 26; Ripley, 2007, p 10). This personalised, reflective environment provides a ―critical thinking space‖ where learners can ―take risks‖ and ―address what is important to them as individuals‖ (JISC, 2008, pp 17 & 35).

2.4 E-Portfolios for Life A number of leading luminaries are proponents of lifelong and life-wide usage (AeP, 2008, p 136; Barrett & Garrett, 2009; EIfEL, 2007). Lifelong access has been consistently identified as a critical success factor (Barrett, 2009; Jafari, 2004) and there are effective examples of life-wide use to achieve personal goals (Corbett, 2006; Robbins, 2009). The research company Gartner predict e-portfolios to be on the brink of global mainstream adoption in education (Lowendahl, 2011). Indeed there are already many successful large scale implementations: Career Wales offers an award-winning e-portfolio service to all age-groups using a variety of tools, potentially 3 million users, and e-portfolios have been widely adopted in medical and teacher education. Other examples of wide scale adoption include the Royal College of Nursing, a number of vocational projects across Europe, and a number of pilot projects supported by JISC. In the US eFolio Minnesota is offered to all Minnesota residents and now approximately half of all American universities have an e-portfolio system in place (Batson & Watson, 2011). Microsoft Research is investigating the feasibility of a ‗‗lifetime store of everything‘‘ with their MyLifeBits project (Gemmell, 2007), and e-portfolios and digital storytelling are used in some schools in the US with a view to being life-long (Barrett, 2009). It seems increasingly likely, therefore, that candidates studying for a specific award will require a flexible e-portfolio with lifelong access. If there is an advantage to maintaining this stakeholder relationship in the longer term then the hosting of an e-portfolio system would be one way to facilitate this, but hosting indefinite access is not sustainable. Instead JISC have committed ongoing resources into piloting and promoting the use of an interoperability standard to facilitate the transfer of data from one e-portfolio system to another. JISC have published the results of seven mini-projects to implement and test this LEAP2A standard which leads the way internationally in implementing e-portfolio interoperability (Grant, 2009). Currently many commercial, assessment-driven e-portfolios do not conform to this standard and only offer a basic export function (e.g. zip file). Researchers at Stanford University are piloting a ―large-scale robust e-portfolio system‖ which addresses issues of scalability, sustainability, adoptability and interoperability. With the rise of multimedia content even current users require an increasing amount of storage space (Kim et al., 2010). Stanford‘s system, which integrates cloud computing with semantic web architecture, organises artefacts stored in the ‗cloud‘ (e.g. Flickr, YouTube, GoogleDocs) into different views for appropriate sharing. A prototype is currently being tested (Kim et al., 2010). Another solution gaining traction is the use of blogs for e-portfolios (Wicks, 2011; Hopkins, 2011) and Seattle Pacific University currently use Wordpress b-portfolios for assessment. It offers many of the features of an e-portfolio and is free and cloud-based (therefore sustainable). Many users will be familiar with the platforms (e.g. Blogger,

5

B00494857

Wordpress); they integrate with other social media; they‘re accessible and initial studies report good student satisfaction and high pass rates (Wicks, 2011).

2.5 Challenges There has been much recent discussion (e.g. Barrett, 2009; JISC, 2012d) concerning the dichotomy of e-portfolios which have the primary purpose of learning versus those which have the primary purpose of assessment. E-portfolio systems developed specifically for assessment purposes often forgo key elements of the learner-centred e-portfolio: social tools, longevity, and personalisation. By contrast, eportfolios primarily for learning often lack the award-specific structure and reporting tools required for assessment (see Appendix II). A suitable e-portfolio solution must take into consideration the backwash of assessment and that ―from the students‘ point of view assessment always defines the actual curriculum‖ (Ramsden, 1992, p 187), and when the purpose of an e-portfolio changes from a learning tool to summative assessment it becomes ―something that is done to them rather than something they WANT to maintain as a lifelong learning tool‖ (Barrett, 2004a). There is a clear link between an assessment purpose and lack of engagement (Tosh et al., 2005) and yet CIE and ESOL both have stakeholder groups (teachers and trainee teachers) who straddle both learner (professional development) and candidate (teaching awards). The main challenge is to convey the value of the whole e-portfolio to all stakeholders; to find the right balance between assessment-driven (institution-centric) requirements and learner-driven (user-centric) requirements; and to achieve a level of standardisation yet allow for personalisation and creativity (Barrett, 2009). This unprecedented link between teaching, learning and high stakes assessment is fundamentally disruptive: pedagogically, organisationally and technologically (Baume cited Taylor & Gill, 2006, p 4; Cambridge, 2012; Eynon cited Shada et al., 2011. p 75), and planning for successful implementation is critical (JISC, 2012e; Joyes et al., 2010; Meyer & Latham, 2008; Shada at el., 2011).

6

B00494857

3. Methodology

This pragmatic research sets out to improve service and quality by contributing to the following aims: 

To improve assessment methods by incorporating existing research and emerging lessons with the specific requirements of ESOL and CIE.



To improve teaching standards by delivering a framework which embeds best practice and develops skills such as reflection, collaboration and self-directed learning.



To foster a lifelong relationship with learners by providing an engaging solution which fulfils their requirements in the short and longer term.



To increase the perceived value of Cambridge Teachers by providing a solution which allows the learner to showcase their achievements and develop and evidence employability skills.

3.1 Scope The scope of this study is limited to the planning stage of a wider, ongoing, collaborative, action research project. The experiential learning cycle is a theme of this research: the corner stone of reflective practice; action research and the systems lifecycle. All are based on the basic cycle of planning, acting and reviewing and have the common purpose of improving practice (see Appendix III for a comparison). Action research is defined as ―a spiral of steps, each of which is composed of a circle of planning, action and fact-finding‖ (Lewin, 1946, p 146), ―aimed at both solving a problem and generating new knowledge‖ (Coghlan & Brannick, 2005, p 14). In this context those spirals can defined as a series of cycles:

Figure 1: Action Research Cycles (Reil, 2010)

7

B00494857

This ―Study and Plan‖ stage is critical to the success of the wider implementation (Joyes et al., 2010; Meyer & Latham, 2008; Shada at el., 2011) and the outputs from this study will inform one or more pilots, forming one or more cycles. When the pilots are completed they will inform further implementations and further cycles, facilitating continuous improvement. This phased implementation is a strategy recommended by JISC (2012f) based on the outcomes of extensive research. JISC (2012g) also recommend spending time analysing requirements: consulting with a broad spectrum of stakeholders to elicit requirements and ensure all users‘ needs are fulfiled (JISC, 2012b; Meyer & Latham, 2008). This will ―develop a sense of ownership‖ and ―manage expectations‖ (SQA, 2007, p 47) as well as providing the learners‘ perspective to ensure a learner centred system is delivered (JISC, 2012g).

3.2 Sample Two main stakeholder groups (end-users) were identified to contribute to the requirements analysis: administrators (a team from CIE and ESOL representing end-to-end exam administration) and learners (teachers and trainee teachers). Administrators are represented by a focus group with members from CIE and ESOL. This focus group is the entire target population (in the context of this study) as they are potential administrators of the subsequent pilots (Cycle 2). Focus groups are ―a group of individuals selected and assembled by researchers to discuss and comment on, from personal experience, the topic that is the subject of the research‖ (Powell & Single, 1996, p 499). The group discuss and agree on assessment-driven requirements, based on existing checklists (see Appendix I), and their knowledge of current best practice. This is an effective way to generate hypotheses and develop concepts (Gibbs, 2011) both in relation to the requirements and also in the development of the learner survey. The group reviews the survey, develops it, pilots it and reviews the results. They all have experience of teaching and teacher training and have a unique understanding of the sample population. Including the group in the decision making process is empowering (Gibbs, 2011), and is an opportunity to engage staff and gain commitment, a further recommendation from research (JISC, 2012b). Learners are, in fact, teachers and trainee teachers who have a relationship with CIE or ESOL. This is the population of inference only in the wider context. For the purposes of this study the target population consists of CIE‘s ITC teachers and ESOL‘s teachers, with a view to running pilots with these groups. As this (international) population cannot be easily accessed, given the time and cost constraints, the frame of population is that ―which the survey materials or devices delimit, identify, and subsequently allow access to‖ (Wright & Tsao, 1983). A survey is distributed using the methods available: centre bulletin email; teacher e-zine; and online forums. Errors of coverage are acknowledged and one strategy considered was to use post-stratifying to weight the sample to better match the population, although it is accepted that this will not off-set the frame-coverage bias (Fricker, 2006). Therefore it cannot be concluded that the results from convenience sampling provide adequate representation of the population of inference without corroboration from secondary research.

3.3 Data Collection and Analysis The objective of this study is to create a series of recommendations for CIE and ESOL on how best to implement e-portfolios. Outcomes from primary and secondary research converge into a framework. Convergence or corroboration of different methods, known as triangulation, increases a study‘s validity and decreases investigator bias (Denzin, 1970). Qualitative data from the focus group informed the design and distribution of the survey which Greene, Caracelli, and Graham (1989, p 259) identify as a rationale for adopting a mixed-methods design. This is when the researcher collects, analyses, and mixes both quantitative and qualitative data (Creswell, 2003), for example, using the outcomes from one method in the application of another (Axinn & Pearce, 2006). This is an acknowledged strategy to ―draw on the strengths and minimize the weaknesses of both types of research‖ (Connelly, 2009, p 31).

8

B00494857

To facilitate this, data is collected in a variety of ways: email communication with the focus group; collaborative documents; and online survey. The focus group began by reviewing and feeding back on a draft survey by email, when consensus was reached the survey was piloted and adjusted once more. Timings were recorded and the approximate time commitment added to the rubric. A link to the survey was distributed online to the two sample populations by the methods available. The quantitative data was analysed using Microsoft Excel and the qualitative data manually. This was summarised and distributed to the focus group for comment. The focus group was also responsible for agreeing the requirements of assessment. ―Minimum standards‖ checklists from OCR, SQA and EdExcel (see Appendix I) were consolidated ahead of a group discussion. The outcomes of the discussion were added to a collaborative mind map. This data was cross-matched with secondary research and a framework for implementation was created. This framework was then distributed to the focus group for comment and further refinement ahead of piloting.

3.4 Limitations Selection bias is a limitation of the survey. Errors of coverage have already been acknowledged, additionally there is a risk of a non-response or self-selection error in the survey results: problems caused by differences between those who respond and those who do not, particularly in the event of a low response rate (Dillman, 2000). It may be the case that those most likely to respond are interested in e-portfolios, highly engaged with ESOL or CIE, or have strong positive or negative feelings about either. There is also a risk of response bias as respondents may want to present themselves in a positive light to these organisations, especially if they are undertaking a qualification. Using a focus group also has a risk of limitations: the role of moderator is critical in ensuring everyone contributes whilst remaining objective (Krueger, 1994), and it is acknowledged the results cannot be generalised to a wider population. With this in mind, success is measured by the effectiveness of the process, and those involved in further implementations (or subsequent cycles) are advised to evaluate their own specific requirements (JISC, 2012g), learning from the previous experiences.

3.5 Ethical Considerations Ethical approval and consent was obtained from the University of Ulster Research Ethics Committee before undertaking this research. Ethical consideration is integral to research governance policy and procedure as ―it ensures integrity and good conduct‖ (University of Ulster, 2011, p 3). There are no risks or issues of vulnerability and the likelihood of potential harm is low. The sample populations and focus group consist of adults who have chosen to take part and can withdraw at any time. Information sheets are provided and consent obtained from participants. No personal data (names, email addresses) is harvested from the survey, and focus group participants are not named. All participants are fully aware of the context of the research and due care has been taken to avoid publishing sensitive information. Anyone acting on the outcomes of this research does so at their own risk.

9

B00494857

4. Themed Findings

4.1 Focus Group The focus group had two deliverables. Firstly, was to collaboratively design, and then pilot the learner survey. After a kick-off meeting this took place primarily by email with a number of face to face meetings to ensure individual aims would be achieved. Secondly, the focus group was responsible for agreeing on requirements from an assessment perspective. Various checklists (see Appendix I) were consolidated and categorised ahead of a face to face meeting where the requirements were outlined. The findings were then added to a collaborative mind map (see Appendix IV) to allow further contribution and refinement. Usability: the group agreed the e-portfolio must allow the upload of various file types to allow rich and diverse content to be added to a repository. Import and integration with Web 2.0 tools will allow easy population which may facilitate ownership. Simple navigation along with templates and exemplars will ensure the system is easy to use. Future-proofing by choosing a market leading system was also an agreed requirement. Additionally, it was agreed that some collaborative space would be a nice feature to have. Training and support: the group agreed appropriate and scalable 24/7 support must be available. Online resources ought to include: FAQs; multimedia tutorials; peer support; and manuals. Email support should include auto password recovery and for other requests an acknowledgement email with expected timescales for resolution. An appropriate starter template will be provided which the user can then customise. Avatar help would be a nice feature to have. Ownership: the group agreed the e-portfolio is to be owned by the user and as such must offer privacy settings and various views controlled by the user, but must also be quality controlled. For example: if used for assessment an examiner must be given appropriate access; and if an award given this must be verifiable by employers. Even though the e-portfolio is owned by the user it‘s acknowledged that the organisations may have a responsibility to back up the system in case of failure. Tracking, security, and authentication: the group agreed the evidence for assessment must have a ―publish‖ function i.e. be ―locked down‖ so neither the user not the examiner can edit the evidence. It would be best if the user had a unique, authenticated identifier which is applicable in all contexts across the organisations. It would be nice if principal examiners could track completion rates. Accessibility: the e-portfolio must be personalisable for accessibility and learner engagement. Global accessibility is a requirement and devices, upload speeds and infrastructure must be considered, especially if the e-portfolio is to be used for assessment. Lifelong access is an agreed requirement which means the e-portfolio must fulfil the purposes of professional development, personal learning, assessment and showcasing. There are 4 agreed roles which fit into a minimum of 3 levels of access and this is controlled by the user: user (editor); peers (read/comment); tutor (read/comment); exam board (read only). Backups: there must be disaster recovery processes in place and an export function available to allow users to make their own backups if they want. Cross-referencing: performance criteria must be specified and checklists available. It would be nice if the appropriate evidence could be presented synoptically. In addition to these requirements there was some discussion around ―what would make this e-portfolio compelling‖. It was agreed it must be engaging, simple and accessible. It must be useful in terms of

10

B00494857

employment, improving practice and fostering community. It should be linked to recruitment and recognised by employers who could provide their own template and receive exactly the information they want to see from a prospective candidate. It must be capable of presenting different views so the user can display a C.V. version with specific highlights which they can email or print. Craven (2011) describes this as a ―lens view‖, where there is a rich view at learning level, a subset of that is tagged as assessment evidence, and another subset might contain a list of achievements plus highlights of key evidence for employers or admissions tutors. It should be capable of pulling everything together from around the web that is related to the users‘ professional life to create a professional digital identity. Finally, it must support triadic, formative assessment: self-, peer- and tutor-; as well as summative assessment in the form of a synoptic snapshot of ability at a particularly point in time.

4.2 Learner Survey (see Appendix V) There were 123 survey entries received from ESOL (Sample 1) and 10 from CIE (Sample 2). Demographically both samples are a good match with the target populations. Sample 2 results mirror Sample 1, in most cases. In terms of current practice, both samples recognise the importance of planning, recording, reflecting and sharing but Sample 2 use a wider variety of digital tools and more often than Sample 1. It‘s apparent from both sets of data that there are a number of teachers who never use social media tools for teaching and learning. Bookmarking sites and Twitter are the least used tools for learning, and email and web searching the most used. There‘s no correlation with age, adding some evidence to the ongoing debate regarding ―digital natives‖ (Wheeler, 2011): the theory that those who grow up with technology have different attributes and mindsets (Prensky, 2001). Other tools mentioned in the comments include: e-books; journals; audio recording sites and reference websites. The majority of both samples are happy to submit evidence for assessment online or by email (although there were a few comments from those who prefer to hand in written evidence). One significant difference is in Sample 2 more respondents selected ―to connect with other professionals‖ as a purpose, which may be linked to their more varied use of online tools, including e-portfolios. Both sets of results highlight that an e-portfolio is thought to fulfil multiple purposes but is mainly for professional development. Both samples expect a variety of content to go into an e-portfolio including presentations, links, audio, video, documents and images. The content and tools were cross-referenced to create a list of required tools (Table 1).

Must have:

Should have:

File upload and storage: documents, Blog integration (RSS?) multimedia including video, audio and photo Twitter integration Youtube/Flickr integration File sharing integration Social features: networking, sharing, (e.g. Dropbox) commenting

Could have

Presentation site integration (e.g. Slideshare) Bookmarking integration (e.g. Delicious)

Useful links News Table 1: List of Required Tools

Interestingly, websites for uploading presentations and bookmarking are not frequently used but presentations and saving links score highly when respondents are asked what type of content goes in

11

B00494857

to an e-portfolio. Also, reflective practice is acknowledged as important, and identified as a purpose of an e-portfolio, but blogging (an appropriate tool) is never used by 40-55% of respondents. This may highlight a need for better awareness of existing web tools and their uses. In terms of past experience, most of Sample 1 have no experience of e-portfolios and most of Sample 2 do. Of those with experience, most have used freely available web tools and only a few have used proprietary software. Most received little or no training and most were not assessed. Sample 2 rated their past experience more highly satisfactorily than Sample 1. The two respondents in Sample 1 who rated their experience as ―Not satisfactory at all‖ cited time constraints and working alone (unsupported) as their respective reasons. Ease of use is an important factor based on past experience, as is knowledge (how to use all the features); demonstrating the importance of appropriate support both technologically and pedagogically. Also important is whether or not it‘s useful: reinforcing the need to convey the purpose and benefits to the learners. Most respondents intend to use their e-portfolio for a few years or more with a large proportion expecting to use it throughout their career: adding evidence to the requirement for lifelong access and flexibility. In terms of future use, more respondents think they are likely rather than unlikely to use an e-portfolio in the future. This is mainly because they are perceived as useful but also because they are believed to be ―the way forward‖. Those unlikely to use an e-portfolio in the future cite lack of knowledge as a common reason. One interesting result is the demand for further training on e-portfolios. More than 90% of respondents answered ―Yes‖ to ―Would you like more training on e-portfolios?‖ and the comments reinforce that request for help and guidance. Self-selection-bias is an accepted limitation of this survey and it is acknowledged that those interested in finding out more about e-portfolios are more likely to complete a survey about e-portfolios; however in another recent national (US) survey of 3000 students, 40% wished they knew more about e-portfolios (EDUCAUSE, 2011).

4.3 Emerging themes

Usefulness Ease of use (or usability) is an essential attribute of a successful e-portfolio project (Jafari, 2004); a fact reinforced by various case studies (e.g. Gaitán et al., 2007, p13; JISC, 2012h) and from the learner survey and focus group. Improved ease of use can be achieved by considering a number of quality components: learnability; efficiency; memorability; errors; and satisfaction (Nielson, 2003). Usefulness is another important (related) factor based on the outcomes from the learner survey and focus group. If usability is defined as how easy and satisfying the e-portfolio is to use, and utility is defined as fitness for purpose, then usefulness can be defined as a combination of the two (Nielsen, 2003; Microsoft, 2012). Therefore defining the purpose and aligning that purpose to a specific context is a critical consideration (JISC, 2008, p 36; Joyes et al., 2010; Meyer & Latham, 2008; Miller, 2011). The focus group defined a common purpose of supporting their teachers throughout their career. Firstly, they must fulfil an assessment purpose, then continuing professional development. In fact, these separate purposes are closely aligned: reflective practice and professional development planning are common outcomes and are also clearly understood purposes of an e-portfolio from the learners‘ perspective (see Appendix V). It‘s important to reiterate these defined purposes to the learners, to set their expectations of the technology, and to communicate the benefits by including exemplars, videos of past experiences and mapping to objectives or standards (where used for formal assessment) (JISC, 2012i). Ownership However the context-specific purpose is defined,"the overarching purpose of portfolios is to create a sense of personal ownership over one's accomplishments, because ownership engenders feelings of pride, responsibility, and dedication" (Paris & Ayres, 1994, p 10). Emerging lessons from case studies

12

B00494857

agree the learner must ―feel that they own the e-portfolio before fully engaging‖ (JISC, 2012b) which is why personalisation and lifelong access are recurring themes in literature. Personalising is taking ownership of the e-portfolio. It‘s important, therefore, that the learner can customise the e-portfolio; that they can select and integrate their own choice of tools (Coolin & Harley, 2010, p 13) and be motivated to ultimately create not just a digital identity but a personalised learning environment employing metacognitive strategies to foster deep learning (JISC, 2012j). Lifelong access (or interoperability) ensures the learners‘ time and effort in this process is an investment worth making (JISC, 2012b) and reinforces the learner as the owner. The focus group agreed that the e-portfolio should be learner owned and controlled but it‘s noted that ownership refers to ―the centrality of the learner‘s own experience‖ and the legal issues of ownership are more complex (Charlesworth & Home, 2004, p 3). Central to that idea of ownership is control: control over which assets to share and who to share them with. The focus group agreed although the administrators would define roles the learner should be able to control who to share their content with. This also reduces the possibility of tension between reflection and assessment (Cotterill et al., 2010, p 16).

Training and Support Knowing how to use the features of an e-portfolio is an important factor from the results of the learner survey, and training and support is a recurrent theme from both primary and secondary research. JISC (2012i) identifies training and support as a common element in all their case studies and recommends ―processes are supported technologically and pedagogically‖. There are similar recommendations throughout literature (e.g. Coolin et al., 2010, p 30; Meyer & Latham, 2008) and specific tactics to support users include: having a support team in place; choosing an appropriate tool initially; providing templates with instructions; providing exemplars; supporting the pedagogic processes like reflective practice and peer review; and information on sharing and e-safety (JISC, 2012i). The focus group acknowledged much support will have to be provided remotely (and always be accessible) and so interactive, online resources (e.g. online courses, webinars and videos); peer support; email support; and manuals will be important support features.

Employability Skills Central to the theme of usefulness, and fundamentally driving the requirement for a multi-purpose eportfolio, is the idea that by developing and evidencing certain skills the perceived usefulness of an assessment-driven e-portfolio will increase. Instead of being an assessment tool with a finite lifespan it becomes a life-long learning tool, facilitating continually improving practice and, therefore, improved employability. The cycle of reflective practice and forward planning, identifying gaps and taking control is empowering for the learner (JISC, 2008, p 12), but, importantly, this also develops and evidences self-directed learning and critical thinking. These are skills highly valued by employers (REAP, 2007) and the focus group agreed that a compelling solution would include the opportunity to link closely with employment. Ideally, employers would provide a template to ensure they received the exact overview of skills and experience they required. Additionally learners could be supported by being provided with specific training on presenting to an external audience. Where employers cannot or will not accept an eportfolio as part of the recruitment process the desirable skills are still being developed and will be evidenced in other ways.

13

B00494857

5. Conclusions & Recommendations

This research aimed to avoid reinventing the wheel by gaining a deeper understanding of e-portfolios and planning for a successful implementation. It began by outlining the drivers and issues surrounding e-portfolio adoption. Government policies, driven by evidence of e-portfolio success, has led to increasing use, and it‘s now likely to they will become ubiquitous in education. Assessment drivers include pressure to provide scalable, sustainable e-assessment solutions which improve practice, possibly by developing and evidencing key skills like reflection, collaboration and self-directed learning. Indeed, emerging evidence of how e-portfolios can support deeper, more autonomous learning is a clear indicator that awarding bodies could improve practice, but the features which best support these more profound forms of learning are often absent from assessment-driven e-portfolios. This naturally led to involving stakeholders to investigate requirements and find that balance between the needs of the learner and those of assessment. This, furthermore, provided an opportunity to introduce e-portfolios and to disseminate lessons from secondary research, building capacity and laying the foundations for good decision-making. It was an opportunity to win hearts and minds by communicating evidenced benefits along with best practices. An appropriate framework for implementation has emerged:

5.1 Framework

This framework (see Figure 5) is adapted and influenced by various models from research (JISC, 2012b; JISC, 2012e; Joyes, et al., 2010; Madden, 2007, p 22; Meyer & Latham, 2008; Shada et al., 2011): 1. Define. Identify and engage all stakeholders in the planning and implementation process. Agree an approach and define purpose and objectives. Key stakeholders were identified as the end-users: teachers, trainee teachers and administrators. Administrators were also identified as key policy and decision makers, and given this is a ―middle out‖ initiation their buy-in is critical to foster ownership, define appropriate purpose, and secure resources for implementation (JISC, 2012k). The administrators defined a common purpose of supporting teachers throughout their career, for professional development and assessment. They also have a more urgent need to accept digital evidence for assessment. The recommendation is to implement incrementally: to meet that urgent need by first running pilots with these specific awards, then slowly expanding the adoption. 2. Understand. Collate and disseminate lessons from e-portfolio research. Win hearts and minds by communicating evidenced benefits along with best practices. Build capacity and lay foundations for good decision-making. A key outcome of this research has been the opportunity to engage stakeholders and communicate existing guidelines and case studies, to avoid reinventing the wheel. Current drivers and challenges were investigated and subsequent decisions were, therefore, informed. The focus group understood that any solution had to meet all the requirements of

14

B00494857

all stakeholders: it has to provide an environment suitable for learning, assessment and life; it has to be scalable and sustainable; and implementation has to be planned for.

3. Select: Agree functional and non-functional requirements, incorporating lessons from research and the needs of stakeholders. Choose a scalable, sustainable solution. The focus group agreed on a multi-view or lens view e-portfolio type to deliver a learnercentred e-portfolio which also meets an assessment need and could be an effective part of a recruitment process. It was agreed that it must be useful: easy to use, fit-for-purpose and the benefits understood. It must be learner-owned: personalisable with sharing controlled by the learner. There must be ongoing training and support; and the learner must be able to present a view of their e-portfolio to an employer or admissions tutor. The next step in this process is to revisit these requirements in more detail and evaluate available solutions. Also to be discussed is how to ensure the solution is scalable and sustainable. The minimum requirement is to be interoperable to ensure learners can transfer their content from one e-portfolio to another. This allows the learner to import their existing content and easily populate the e-portfolio but, importantly, it also allows the content to be exported if life-long hosting is unsustainable. Indeed the evidence suggests that without incorporating cloud-based Web 2.0 technologies it is unlikely to be sustainable due to the increasing amount of space required for multi-media content (Kim et al., 2010). Incorporating these technologies has additional advantages: they allow the learner to personalise the process, by using their preferred tools (Barrett, 2008; Tosh et al., 2005); as well as capture authentic evidence and real-time learning, in part due to the ubiquity of mobile devices (JISC, 2008, p 10; Barrett, 2011). 4. Preview: Pilot internally, with staff. Set up personal e-portfolios to learn the tools then set up dummy learner e-portfolios to understand how to best integrate the e-portfolio into a professional development or assessment framework. Share ideas and best practices. Review and evaluate. Once the e-portfolio has been selected the next step is to pilot this internally. Some recommend avoiding the word pilot or project, as these imply a lack of long-term commitment (Chesney, 2009, p 5; JISC, 2012f), reinforcing the importance of longevity on success. This stage provides an opportunity for staff to develop expertise in using eportfolios themselves, to build capacity across the organisation and identify champions (JISC, 2012i). They can begin to think about how best to embed e-portfolio use and experiment with dummy learner accounts. They can identify potential issues and areas where training will be required as a pre-requisite (for example, to improve digital literacy), and they can identify where e-portfolio use does not fit in with existing practice and change may be required (i.e. where it‘s disruptive). It is recommended that they share experiences and best practices, and consolidate preparation plans (JISC, 2012m). They should also identify those resistant to change and meet with them individually to communicate the vision and provide them with demonstrations (Meyer & Latham, 2008).

5. Prepare: Change existing practice where required. Design learning activities to suit the eportfolio. Create templates mapped to standards. Finally, create the training and support material and plan the delivery. Next any technological, organisational or pedagogical changes can be made which are necessary to embed e-portfolio use. This may involve re-designing activities to suit the context or changing the criteria for assessment. There is a substantial amount of evidence to suggest use must be mandatory in some way, ―at least in part or at the beginning to

15

B00494857

overcome initial resistance‖ (Madden, 2007). This is reinforced by Harrison (2011), recounting her experience of launching the Cambridge ESOL Teacher Portfolio: ―[teachers] are busy people and unless it‘s tied into a qualification, a government / organisation requirement or specific continuous professional development, they‘re unlikely to use it even if they like it and think it‘s a good idea‖. The e-portfolio should, at least, be demonstrably linked to the purpose and objectives (JISC, 2012i). The focus group agreed templates should be mapped to standards or criteria, and exemplars created. Any identified prerequisite training should take place. Induction training should be designed, along with ongoing support materials. Providing a template and instructions on how to customise the template is one method suggested by the focus group. Further resources identified by the group include: FAQs, multimedia tutorials, manuals, email support and peer support. The ―preparation phase is both critical and difficult‖ and the way the e-portfolio process is introduced to learners will determine its ultimate value (Shada et al., 2011). Rubrics and models are important at this stage, particularly in relation to reflective practice (JISC, 2008, p 11; Tosh et al., 2005) and JISC (2012m) specifically recommend providing examples of effective reflective writing compared to descriptive writing. Scaffolding is also important to support peer review, information selection, presentation and e-safety (JISC, 2012i). Data aggregation and programme evaluation should also be planned for before piloting (Meyer & Latham, 2008).

6. Pilot: Pilot with learners. Communicate purpose and benefits. Deliver training and advice on selecting, reflecting and sharing. Provide ongoing support and encouragement. Review, evaluate and apply lessons learned. With all the training ready and support resources in place, piloting the e-portfolio with learners is now recommended. First deliver induction training, communicating purpose and objectives as well as features, actions and benefits; and provide signposting to additional training and resources. Strong support is recommended (JISC 2012i; Hartnell-Young, E. et al., 2007, p 5) because lack of engagement at this point will result in flawed implementation (Yancey, 2009). Salmon‘s Five Stage model (Salmon, 2002) prioritises learner engagement and may provide an effective framework for implementation at course level. Encourage the learners to take ownership of the purpose and processes as well as the tool and product. Identify champions and use their voice to promote effective use (JISC, 2012b, 2012i). Continually review this process to identify areas for improvement: particularly in relation to training and support (JISC, 2012m). 7.

Implement: Begin ongoing implementation cycle: continue incrementally. On completion of one or more successful pilots, apply the lessons learned and continue to implement incrementally. Seek out open doors: those looking to implement change. Use a plan, act, review cycle to continually improve and ―systematically share effective e-portfolio practice‖ (JISC, 2012m), and evaluate the process as well as the product.

5.2 Conclusion This research has succeeded in identifying current issues in e-portfolio adoption and converging primary and secondary research with existing guidance and frameworks. There were accepted limitations in the sampling method and, as a result, the detailed requirements are worthy of further exploration. Improved access to a sample and larger sample sizes would increase validity and reduce bias. Learner focus groups and interviews would contribute to validating and expanding on the current results. In addition, the effectiveness of the process is yet to be evaluated and there is clearly much potential for further research in the context of this single implementation. Action research along with

16

B00494857

impact and efficacy studies would be of value as part of this phased implementation process but also in a much wider context. Key decision makers in CIE and ESOL are now in a position to make informed decisions and there is an understanding that, although the benefits are extolled, e-portfolio adoption must form part of a strategic approach and requires new practice due to their disruptive nature. Implementation has been planned for and with continued management should realise tangible benefits although it is acknowledged that this is a slow, iterative process and understanding will develop with experience and over time.

Figure 5: Implementation Framework [infographic version]

Word count: 7978

17

B00494857

Appendices

Appendix I: e-Portfolio Checklists Prospective E-Portfolio Product Provider: Pre-Evaluation Checklist. (OCR e-Assessment Team, 2007). Edexcel – Minimum Standards For NVQ E-Portfolios. (EdExcel, 2006). Requirements for e-portfolios. (SQA, 2011).

Appendix II: e-Portfolio Tools

Learner e-Portfolio

Assessment e-Portfolio (Assessment Management System)

Purpose

- Multiple purposes: Learning, Assessment, Showcasing, Employment

- Single purpose: Formative and Summative Assessment

Data Structure

- Data structure varies with the tools used to create the portfolio

- Data structure most often uses a relational database to record, report data

Type of Data

- Primary type of data: qualitative

- Primary type of data: qualitative and quantitative

Level of Personalisation

- Flexibility or total control, includes opportunities to integrate with existing digital identity (in some cases)

- Limited or no flexibility in layout and design

Social Interaction

- Peer interaction, opportunity to link to PLN external to the e-portfolio (in some cases)

- None or limited to tutor/learner roles, ringfenced

Locus of control - Student-centered

- Institution-centered

Selection of Contents

- Artifacts prescribed by institution

- Artifacts selected by learner

Technology skills - More advanced skills required required

- Minimal skills required

Technology competency demonstrated

- Low to medium, depending on the sophistication of the artifacts added to the portfolio

Medium to high, depending on tools used to create portfolio

Table 2: e-Portfolio Tools (adapted from Barrett, 2004b)

18

Appendix III: Comparison of Cycles

Figure 2: Experiential Learning Cycles (Batista, 2007)

Figure 3: Systems Lifecycles (JISC, 2012l)

B00494857

Figure 4: Action Research Cycles (Reil, 2010)

20

Appendix IV: Mindmap

Appendix V: Learner Survey Results

Sample and Population Population 1: ESOL stakeholder who are teachers (teachers of exams, teaching award candidates and alumni), limited for the purposes of this research to those ESOL actively engage with online: 100,000 Sample 1: 25,000 received the survey link, 123 complete entries were received. Population 2: CIE stakeholders who are ECIS International Teacher Certificate (ITC) candidates and alumni (teachers): 204 Sample 2: 82 had access to the survey link, 10 complete entries were received. This is a response rate of 12%. Initial impressions Demographically both samples are a good match with the target populations. Most respondents are 2655 in both surveys, substantially more women than men and most are teachers (as opposed to candidates). There is a global response in Sample 1 (although China and India are notably underrepresented, as are TKT candidates), but it‘s not a valid representation of Population 2 due to the small sample size. Nearly all of Sample 1 are teachers (have teaching experience). In Sample 2 some respondents are ECIS ITC candidates and some are alumni, all are teachers. Planning, recording, reflecting and sharing are recognised as important factors in teaching and learning. One significant difference is in the use of online tools for teaching and learning. Sample 2 use more digital tools, more frequently than Sample 1, possibly linked to the fact that ECIS ITC is an online course (although Sample 1 were accessed online). Both samples prefer to submit evidence for assessment online or by email. Responses to the purpose of an e-portfolio almost match with one notable exception: more of Sample 2 rate connecting with other professionals as a purpose, perhaps this is a result of personal experience with e-portfolios (as most have had some experience compared to Sample 1 where most have not), or perhaps it‘s reflective of the difference in digital literacy levels (especially in relation to social media and networking) The e-portfolio platforms used is also comparable with nearly all of both samples having used Web 2.0 tools (blogs, wikis, personal websites, Google sites). In the comments section both samples mentioned ease-of-use, time and lack of knowledge. Most had received little or no training and most e-portfolios were not assessed. Overall Sample 2 seems to have been more satisfied with their past experience than Sample 1 and a slightly higher percentage plan to use their e-portfolio throughout their career (50% versus 43%). All Sample 2 respondents are likely to use an e-portfolio in the future compared to just 54% of Sample 1. A surprising number of Sample 1 are interested in using the Cambridge Teacher Portfolio (not an option for Sample 2), but given most have no experience this is likely to based on brand recognition rather than informed choice. What is most surprising is the response to ―Would you like more training on eportfolios?‖ to which nearly all replied they would with online course and demo video being the top choices for delivery. One theme of the final comments is the desire for more information, training and support.

B00494857

Detailed Analysis: Demographics Most of Sample 1 (ESOL) and all of Sample 2 (CIE) are 26-55; the larger of the two samples has some outliers (younger than 25 and older than 56). Most are female, teachers (as opposed to candidates of teaching awards) and nearly all have teaching experience. This question was not asked to Sample 2 (CIE) as they are all teachers whereas some Sample 1 (ESOL) respondents may be at the start of their career, studying for a qualification with no professional experience. TKT candidates are underrepresented in this sample. This may be related to: the way we engage with stakeholders online in these countries; technology or infrastructure issues; or lower levels of digital literacy (anecdotal). This has a significant impact if e-portfolios are introduced for assessment but no impact otherwise. This is an area for further investigation before e-portfolios are considered for this award.

23

B00494857

Geographically, Sample 1 (ESOL) has a good spread of responses, although China and India are underrepresented in comparison to candidate numbers and visits to the Teacher Support website. The number of respondents in Sample 2 is too small to be representative of the target population. As this was an optional web survey (convenience sampling) it may be worth investigating any correlation between low response rates in India and China with TKT candidates and either low levels of English language or lack of interest in the subject.

0

1 1

Sample 1 (ESOL)

1

Sample 2 (CIE)

Detailed Analysis: Current Practice Both samples recognise the importance of planning, recording, reflecting and sharing with a slightly more positive response from Sample 2 (CIE) which could be attributed to the smaller sample. If we adjust for sample size (rescale the larger sample and remove counts of 1 or less (outliers)), the results are almost identical for this group of questions.

24

B00494857

In terms of the tools used in current practice the two graphs have a similar shape suggesting there are similar trends in both sets of results but Sample 2 (CIE) clearly use a wider variety of online tools more

25

B00494857

often. Even with adjustment for sample sizes there are still a significant number of Sample 1 (ESOL) respondents who do not use, for example, email. Bookmarking sites and Twitter are the least used tools for learning, and email and web searching the most used. There‘s no correlation with age, adding some evidence to the ongoing debate regarding ―digital natives‖.

Sample 1 (ESOL): How often do you use the following to record your work?

Sample 2 (CIE): How often do you use the following to record your work?

26

B00494857

Sample 1 (ESOL):

Sample 2 (CIE):

How often do you use each of the following tools for learning online?

How often do you use each of the following tools for learning online?

27

B00494857

Other tools:

• • • • • • • • • • • • • • • •

Telephone No,but I must be honest in this matter,that is that my country is not a developed country, so whatever I do are my personal efforts. Thank you. Concept Maps powerpoint books, magazines voice recording - vocaroo professional journals British Council, Flo-Joe, Cambridge University press, Oxford University Press, Mac Millan sites. recordings of my students to have an audio "portfolio" Reading books about the topic I want to learn. e-books My E-Library-everyday I try to read some stuff about my profession. No I have been creating wikispaces for my EFL students: http://amaldi-englishcorner.wikispaces.com http://nattaefl.wikispaces.com http://galliefl.wikispaces.com Shared documents - intra school drive on the computer systems across the school Teaching videos on different websites, online journals Online subscriptions that the School has like Sage Publications

Mentioned more than once: Journals Books or e-books Audio recordings websites

The question on current practice for submitting evidence for assessment, in hindsight was not well worded. It failed to include the option ―handing it in‖ which was then picked up in the comments. However the results show most respondents are happy to submit by email or online (which was the objective of the question).

Sample 1 (ESOL): send it by post

Sample 2 (CIE): send it by post

28

B00494857

Sample 1 (ESOL): email it

Sample 1 (ESOL): submit online

Sample 2 (CIE): email it

Sample 2 (CIE): submit online

Other:

• • • • • • • • • •

Create a wiki space with all the documents Hand in In my personal website: http://attiliogalimberti.webs.com I've never done it but I know I would like to do so No paper documentation post it online in the virtual platform we use Though I have said that I'll e-mail it ,but I'll try. write it, good old fashioned pen and paper writing lesson plans and reports about the classes.

29

B00494857

Detailed Analysis: e-Portfolio Purpose and Content In this set of questions there are, again, similar patterns between both sets of data. One significant difference is in Sample 2 (CIE) more respondents select ―to connect with other professionals‖ as a purpose, which may be linked to their more varied use of online tools, including e-portfolios. It‘s clear from both sets of results that an e-portfolio is thought to fulfil multiple purposes but is mainly for professional development. In hindsight there is one omission from this list which is picked up in the comments: to plan or set goals.

Sample 1 (ESOL): What do you think is the purpose of an e-portfolio?

Sample 2 (CIE): What do you think is the purpose of an e-portfolio?

Other:

• • • • • • •

to have a timeline of the work can be used as a research evidence in research to keep evidence of students' progress for self assessment To provide an example for teacher trainees. exchange experience to set new goals

30

B00494857

Sample 1 (ESOL): What type of content should go into an e-portfolio

Sample 1 (CIE): What type of content should go into an e-portfolio

Other:

• • • • • • • • • • •

Lesson plans Qualifications/projects Assessor's comments on my lessons?? Summative evaluation comments from learners as they related to my teaching ?? photos e.g. wall display, visits Photos, scans, Pinterest boards. the documents you've created to help your stufents achieve their tasks, the objectives of each task assignments for a teacher-training course Photographs of evidence such as students work scanned copies of documents Emails/Letters of recommendation or acknowledgement creative jpegs of work

31

B00494857

The responses from this section are cross-referenced with the tools already used in teaching and learning to create a MoSCoW list with ‗Won‘t have‘ omitted as no tools received a zero response.

Must have

•

Should have

File upload and storage: documents, multimedia including video, audio and photo

•

Youtube/Flickr integration

•

Option for email, skype ID, social media profile to be displayed

•

Useful links

•

News

•

Blog integration (RSS?)

•

Twitter integration

•

File sharing integration (e.g. Dropbox)

Could have

•

Presentation site integration (e.g. Slideshare)

•

Bookmarking integration (e.g. Delicious)

Interestingly, websites for uploading presentations and bookmarking are not frequently used but presentations and saving links score highly when respondents are asked what type of content goes in to an e-portfolio. Also, reflective practice is acknowledged as important, and a purpose of an e-portfolio, but blogging (an appropriate tool) is never used by 40-55% of respondents. This may highlight a need for better awareness of existing web tools and their uses.

Detailed Analysis: e-Portfolio Past Experience One significant difference between the two samples is their past experience with e-portfolios. Most of Sample 1 (ESOL) respondents have no experience and most of Sample 2 (CIE) respondents have experience. This correlates with the use of digital tools but it‘s also worth noting that ECIS ITC had an eportfolio tool a few years ago which will skew the results from Sample 2 (CIE).

32

B00494857

The survey branches here, applicable only to those who have experience of e-portfolios. Most respondents experience is with freely available Web 2.0 tools with very few having used proprietary software.

Sample 1 (ESOL): Which platforms have you used?

Sample 2 (CIE):Which platforms have you used?

Other: • In my computer • Broadlearning • We created e-portfolios for students on our shared documents website which could be accessed across the school by different teachers.

Most respondents received little or no training on the use of e-portfolios and most were not assessed (although a significant proportion (20-33%) was). Sample 2 (CIE) had a slightly more positive experience. The two respondents in Sample 1 who rated their experience as ―Not satisfactory at all‖ cited time constraints and working alone (unsupported) as their respective reasons.

33

B00494857

What did you like about the platform you used and why? (recurring points in bold, comments of interest underlined) ESOL

CIE

I like being able to categorise all of my bookmarked links with my own notes, so that when I need to do a lesson (for example) on Making Phonecalls in Business English, I know exactly where to find useful information that I can form into a personalised lesson

I could scan work with my comments and feedback, copy pictures/photographs of children engaged in different activities directly on to children's portfolios, share with parents when they would come for meetings or parent teacher conferences and it would be a ready reference document for next teachers to go through. For leaving parents , I used to make a copy of the e-portfoilio on a CD as a memory.

#organise #useful

#useful #multimedia

easy and simple to use

It was easy to follow and user-friendly for someone who isn't tech savvy.

#ease

#ease

user friendly

l like how it is structured

#ease

#ease #feature

I like it because I can apply some exercises on line.

Easy to use!

#useful #feature

#ease

34

B00494857

no "big-brother-is-watching-you" feeling individual style possible

Able to use videos very easily, these are very important to me as a music teacher

#personalised

#ease #multimedia

It's easy to use, bloggers

Keeps all work together in one place, easy to login. A great way to contact others.

#ease

#ease #organise #network Quite simple to use and add photos/video to.

everything #ease #multimedia Ability to keep things together - but Pebblepad isn't very user friendly.

I use edublogs.org and I like it for its simplicity.

#ease #organise

#ease

to share with my students, colleagues and other teachers, the website and the blog are very interactive.

Publishing online and getting replies as reflections from others #share

#share #network Freedom to create my own shape for it

Blogger is very easy to upload to...

#personalised

#ease

Sdfc More comfortable

#ease Wikispaces are user-friendly and very practical.

#ease #useful Tag count: ease 14; useful 4; organise 3; multimedia 3; feature 2; personalised 2; network 2; share 2

What did you dislike about the platform you used and why? (recurring points in bold, comments of interest underlined)

ESOL

CIE

I had to learn how to create my own website

It was sometimes time consuming e.g. to scan each child's piece of written work etc and there was no limit! That was a bit daunting as one could go on forever. The lack of consistency across the school in the understanding of the importance of maintaining an e-portfolio made sustaining it difficult.

#knowledge

#time #knowledge

too many places to go to gert to what I want

I liked everything I learned how to use.

#ease

#knowledge

I can not upload documents nothing

#knowledge #ease

35

B00494857

Still to get fully conversant with its usage! Nothing

#knowledge Complicated to use, especially uncomfortable with technology

for

those

#ease

harder to keep a copy for more long-term records

#ease #knowledge

Uploaded documents look messy. I want them to look as neat and ordered as my table of links to useful websites The continuous emails.

#feature #knowledge

#feature #knowledge

Not very intuitive software - difficult to manipulate. n/a

#ease I didn't like the attached documents, I would like to Sometimes it's too slow. have them linked to my page

#feature #knowledge I don't like those that don't have the check answers.

lack of technical knowledge

#feature

#knowledge

blog post was not used much

HTML features are still needed for really good presentation

#feature

#knowledge #feature

No negative comments. possibility to download videos, pictures and music

#feature No comments

Tag count: knowledge 11; feature 7; ease 5; time 1.

36

B00494857

Why did you rate your past experience like this? (recurring points in bold, comments of interest underlined) • • • • • • • • • • •

•

• •

• • • •

For the same reasons that I found many advantages but if it is not maintained in the same way throughout a child's school life then it loses meaning. #knowledge I haven't really had any past experience, only the one now, and I’m very satisfied with it. Everything is pretty explicit. #ease Limited experience so not really in a position to respond. #knowledge all worked fine. #ease Not much experience to base judgment on. #knowledge It is beneficial to my professional growth as a teacher. #useful I want to use it better but I like IT skills. #knowledge Because I found it time consuming on top of everything else I had to do #time Because I experiment by myself #knowledge To learn more #knowledge I see its usefulness, but I rarely refer to it. I improve my lessons not based on what happened the previous year with a different batch of students, but on what my current students need at the moment. With reference to information for teacher development, things are improving and changing all the time; what was discussed in 2009 often holds no more interest for me in 2012. #useful I wish it was perfect! I wish I could store all of my documents online easily and for free. I am nervous about sharing it with other people. At the moment it is just a personal resource that I use #organise #share In teacher training, it is useful to look at my students' work on Pebblepad, but I share their frustration with the limitations of the software. #useful #knowledge #feature My google site isn't really a tool to think on my teaching but a tool to help my students learn. It's more a large number of good links to go along the course. Sorry about being just that, but I've enjoyed it so far. It helped me to improve my teaching. #useful i was unable to use it to the full due to time restraints #time For professional development. #useful because it is a great experience and I learned a lot. #useful

Tag count: knowledge 7; useful 6; ease 2; time 2; organise 1; share 1; feature 1

Most respondents intend to use their e-portfolio for a few years or more with a large proportion expecting to use it throughout their career.

37

B00494857

Detailed Analysis: Future Plans More respondents think they are likely rather than unlikely to use an e-portfolio in the future, and in fact all of Sample 2 (CIE) responses were ―likely‖ or ―Very likely‖. This may be related to their past experience with e-portfolio or the fact that their qualification requires the submission of digital evidence. From the comments, many respondents are likely to use an e-portfolio in the future because they believe them to be essential in the future, easy to use and useful. Those who commented who are unlikely to use an e-portfolio cite lack of knowledge as a common reason.

38

B00494857

Why? -These respondents were likely to use an e-portfolio in the future. (recurring points in bold, comments of interest underlined) ESOL

CIE

Useful for professional development, sharing experiences and ideas with other teachers.

"I find maintaining student portfolios/learner profiles very useful as they give me a complete picture of the child.

#useful #share because it is essential to teacher development.

#useful

#essential good oportunity to share with students, colleagues and other teachers

I like the idea of creating a portfolio of my own professional practice. Over the summer, I might create one!"

#share can provide a collection that illustrates effort, progress and achievement

#like It's easy to use, submit, carry it as a form of cv for future job interviews. Future employers can see a wide range of my work using contemporary means of tech and education.

#useful My professional development))

#ease #useful "Is easier to have the information inside a Site

#useful It's becoming increasingly popular in our institution--whether it's relevant to learners' need is another thing

#useful #ease "

#trend It is very useful.

#useful I consider it an important tool. #essential

This is the way of the future

Excellent process as it ensures reflection and of course helps organise key learning events. #useful My school is currently looking into developing more formal requirements for teachers to make their own portfolios - and this is also in line with our students having their own e-portfolios. #essential More professional development. Contact with other teachers. Sharing of materials.

#essential #trend to make me a better facilitator

#useful #network #share It will become a required form of reflection. It is the simplest way to collate information.

#useful

#essential #ease

39

B00494857

not sure how it works

To ensure that I always strive for improving my teaching and learning skills.

#knowledge I would like to do the DELTA before the end of the year and it would be useful to start an eportfolio then.

#useful Next generation is less frightened or not frightened at all of ITC. Learning is fun and has great potential while it is on-line. #trend

#useful It would be a good organising tool.

#organise #useful

It is becoming a part of teaching and professional learning

I want to make more use of IT and think it is the way forward. My classroom is becoming cluttered with all the books and files. i would like to have a more stream-lined teaching system.

#essential #trend A lot of my work is accessed online so I find that it is easy for me to transfer it to an electronic form. My students also create their own e-portfolios, so it therefore follows that teachers should also be using them.

#trend #useful #organise

#ease #organise #useful #trend

Good for me

#like Why? -These respondents were unlikely to use an e-portfolio in the future. (recurring points in bold, comments of interest underlined) ESOL

CIE

I am unfamiliar with extensive technology, and feel that face to face presentations are best for learning.

#knowledge I don´t know how to use an e-portfolio.

#knowledge Lack of necessary information

#knowledge I am have retired after being in the teaching business for 38 years. I now do some remedial teaching if people ask me.

Tag count: useful 15; essential 6; trend 6; knowledge 4; ease 4; share 3; organise 3; like 2; network 1.

40

B00494857

The following graph illustrates which platforms respondents believe they are likely to use in the future. Note Cambridge ESOL Teacher Portfolio was not an option for Sample 2 (CIE). If we remove that option from the results most respondents don‘t know which platform they are likely to use.

Sample 2 (CIE) Sample 1 (ESOL)

Which e-portfolio would you like to use?

Probably the most interesting finding of the survey is the demand for further training on e-portfolios. More than 90% of respondents answered ―Yes‖ to ―Would you like more training on e-portfolios?‖. This is reinforced later, in the final comments. Most respondents prefer training delivered by a demonstration video or online course. Self-selection-bias is an accepted limitation of this survey and it is acknowledged that those interested in finding out more about e-portfolios are more likely to complete a survey about e-portfolios.

41

B00494857

Sample 1 (ESOL)

Sample 2 (CIE)

Further comments: (recurring points in bold, comments of interest underlined) ESOL

CIE

I'd like to introduce them to our trainees on our find them very useful to have all the TESOL training courses. I look forward to hearing information you need as a professional inside more about them! one place. #like I didn't know they existed.

#useful Thanks to the ITC I have learnt the art of creating e-portfolios. Great for teachers who are mobile, this resource is also useful for self-reflection! While I may not be 'applying' anywhere for a new job, I have found it fascinating to read my own work months later!

#useful #knowledge I'd really like to see how they might relate to both pre- Looking forward to be in contact. and in-service training (like Celta and Delta) but also for ongoing PD for teachers #knowledge 42

B00494857

I would really like to learn how to create an e-portfolio. #knowledge no,but need more information about e-portfolios #knowledge I don't know much about them These are definitely the way of the future. They are so much easier for everyone. I wish I knew what was the right platform for me! #knowledge #trend #ease It would be a great opportunity for new teachers to get exposed to seniors' experiences and comments through e-portfolios. #share #useful I would welcome an introductory email on eportfolios from the Cambridge English Teacher website. No, Thanks Frankly speaking I have no idea what it is, but I think it will be a great help. #knowledge They promote a positive attitude in designing your better future. #useful I don't know much about them, but anything that can connect technology and teaching/learning is interesting. Unluckily, schools don't go as quick as editors or pupils' needs. #knowledge Make them practical. Realistic. Not something that will make our life insufferable. Easy things. We are under a lot of pressure and we face hard times either from our employers or the parents. We just need a tool to keep a track of our lessons and achievements or failures within the classroom. Just make it simple and useful. #useful #ease I am looking forward to using an e-portfolio #like Not for now, thank you. It would be great if there will be a tutor who could

43

B00494857

guide me in teaching, #knowledge No it's complete for me it's well Are you doing this because you are offering e-portfolio training? leads to continuous professional development #useful Good idea. #like Tag count: knowledge 8, useful 6; like 3; ease 2; trend 1; share 1.

Limitations i) Sample Validity Population 1: ESOL stakeholder who are teachers (teachers of exams, teaching award candidates (preand post-)). There are 167,000 registered users on the Teacher support website (active and inactive). There are 60,000+ candidates. Estimated population, for the purposes of this research (those ESOL has an active relationship and are likely to engage in e-portfolio use i.e. digitally literate and willing): 100,000. Sample 1: 25,000 received the survey link and 123 complete entries were received. This is a response rate of 0.53%.

Confidence level: 90% Z: 1.64 Confidence interval: 7.44% Sample size for calculation: 123 Population for calculation: 100,000 Conclusion: 90% certainty that if the sample population response is 50% the true population is between 42.56% and 57.44%.

Population 2: CIE stakeholders who are ECIS International Teacher Certificate (ITC) candidates and alumni (teachers). There are 204 ITC teachers registered on the ECIS ITC VLE, 82 were active at some point while the survey was open. Sample 2: 82 had access to the survey link and 10 complete entries were received. This is a response rate of 12%.

44

B00494857

Confidence level: 90% Z: 1.64 Confidence interval: 25.5% Sample size for calculation: 10 Population for calculation: 204 Conclusion: 90% certainty that if the sample population response is 50% the true population response is between 24.5% and 75.5%. This sample has reduced validity and generalisations cannot be made from this data alone. The responses are compared to the ESOL survey results and secondary research.

ii) Self-selection bias Self-selection bias is a limitation of online survey research but can be comparable to non-response bias or voluntary response bias as they all result overrepresentation of a specific sector of the population. In this case respondents are more likely to respond if they are interested in e-portfolios, highly engaged with ESOL/CIE or have strong positive or negative feelings about either.

45

B00494857

References

AeP: Australian ePortfolio Project, (2008). Australian ePortfolio project—final project report. Sydney: QUT Department of Teaching and Learning Support Services. Available at: http://www.eportfoliopractice.qut.edu.au/docs/Aep_Final_Report/chapter9.pdf. (Accessed: 30 April 2012) Archer, W. & Davison, J. (2008). Graduate Employability – the views of employers. London: The Council for Industry and Higher Education. Axinn, W.G. & Pearce, L.D. (2006). Mixed Method Data Collection Strategies. In A. Tashakkori & C. Teddlie, eds. Cambridge University Press. Available at: http://ebooks.cambridge.org/ref/id/CBO9780511617898. (Accessed: 30 April 2012) Barrett, H. (2004a). Electronic Portfolios as Digital Stories of Deep Learning. Available at: http://electronicportfolios.org/digistory/epstory.html. (Accessed: 30 April 2012) Barrett, H. (2004b). Differentiating Electronic Portfolios and Online Assessment Management Systems. Available at: http://electronicportfolios.com/systems/concerns.html. (Accessed: 4 March 2012) Barrett, H. (2008). The REFLECT Initiative Research Project Final Report. Presentation at: National Educational Computing Conference July 1, 2008, San Antonio, Texas. Available at: http://electronicportfolios.com/reflect/NECC08.pdf. (Accessed: 29 April 2012) Barrett, H. (2009). Balancing the Two Faces of ePortfolios. Available at: http://electronicportfolios.com/balance/index.html. (Accessed: 4 March 2012) Barrett, H. (2011). Balancing the Two Faces of ePortfolios (updated version). Available at: http://electronicportfolios.org/balance/Balancing2.htm. (Accessed: 8 May 2012) Barrett, H.C. & Garrett, N. (2009). Online personal learning environments: structuring electronic portfolios for lifelong and life-wide learning. On the Horizon, 17(2), p 142-152. Available at: http://www.emeraldinsight.com/10.1108/10748120910965511. (Accessed: 30 April 2012) Batista, E. 21 October (2007). Experiential Learning Cycles. Executive Coaching and Change Management. Available at: http://www.edbatista.com/2007/10/experiential.html. (Accessed: 30 April 2012) Batson, T. & Watson, C. E. (2011). The Student Portfolio is the New Book: New Practices, Profession, and Scholarship. Campus Technology. Available at: http://campustechnology.com/Articles/2011/02/02/The-Student-Portfolio-is-the-NewBook.aspx?Page=4&p=1. (Accessed: 30 April 2012) Brown, J.S. (2008). How to Connect Technology and Passion in the Service of Learning. The Chronicle of Higher Education, 55(8), pp 8-10. Available at: http://www.johnseelybrown.com/howtoconnecttech.pdf. (Accessed: 4 January 2012) Cambridge, D. (2008). Audience, integrity, and the living document: eFolio Minnesota and lifelong and lifewide learning with ePortfolios. Computers & Education, 51(3), p 1227-1246. Cambridge, D. (2012). E-Portfolios: Go Big or Go Home. EDUCAUSE Review, 47(2). p 52-53. Available at: http://net.educause.edu/ir/library/pdf/ERM1226.pdf. (Accessed: 21 April 2012) Charlesworth, A. & Home, A. (2004). Legal Aspects of ePortfolio Systems: A Short FAQ. Bristol: University of Bristol/JISC. Available at:http://www.jisc.ac.uk/uploaded_documents/Legal_Aspects_FAQ.pdf. (Accessed: 28 February 2011) Chesney, S. (2009). Flourish—the eCPD project. Carlisle: University of Cumbria. Available at: http://www.pebblepad.co.uk/cs_newsmedia/CDLT_Flourish_Final.pdf. (Accessed: 8 May 2012)

46

B00494857

Coghlan, D. & Brannick, T. (2005). Doing Action Research in Your Own Organisation Second Edition. London: Sage. Connelly, L.M. (2009). Mixed methods studies. Medsurg nursing official journal of the Academy of MedicalSurgical Nurses, 18(1), p 31-32. Available at: http://www.ncbi.nlm.nih.gov/pubmed/19331297. (Accessed: 30 April 2012) Coolin, K. & Harley, P. (2010). Five ePortfolio Pilots. Nottingham: University of Nottingham. Available at: http://www.nottingham.ac.uk/eportfolio/leapahead/docs/65447_UON_CIEP_APPENDIX_A_PILOT.pdf. (Accessed 27 April 2012) Coolin, K., Harley, P., Smallwood, A. & Wood, S. (2010). ePortfolios and Lifelong Learners. Nottingham: University of Nottingham. Available at: http://www.nottingham.ac.uk/eportfolio/leapahead/docs/65364_UON_FULL%20REPORT_EPF.pdf. (Accessed: 27 April 2012) Corbett, D. (2006) Portfolio Life: The New Path to Work, Purpose, and Passion After 50. San Francisco: Jossey-Bass. Cotterill, S., Lowing, K., Cain, K., Lofthouse, R., Mackay, C., McShane, J., Stancliffe, D., & Wright, D. (2010). Blogs and e-Portfolios: can they support reflection, evidencing and dialogue in teacher training? Journal of Learning Development in Higher Education, 2 (Special Edition: Researching PDP Practice). Available at: http://www.aldinhe.ac.uk/ojs/index.php?journal=jldhe&page=article&op=viewArticle&path%5B%5D=97. (Accessed: 7 May 2012) Craven, P. (2009). Effective e-assessment: Making IT happen. Available at: http://www.cambridgeassessment.org.uk/ca/News_Room/Latest_News/News?id=131662. (Accessed: 27 February 2012) Craven, P. (2011). Re: Update: ePortfolio Research. Email to [email protected], on 11 December 2011. Denzin, N. K. (1970). The Research Act in Sociology. Chicago: Aldine. Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Boston: D.C. Heath and Company. Dewey, J. (1938). Experience and Education. New York: Macmillan, p 110. DfES: Department for Education and Skills, (2004). The Tomlinson Report (formerly known as the Final Report of the Working Group on 14-19 Reform). London: DfES. Available at: https://www.education.gov.uk/publications/standard/publicationdetail/Page1/DfE-09762004#downloadableparts. (Accessed: 4 January 2012) DfES: Department for Education and Skills, (2005). e-Strategy: Harnessing Technology: Transforming learning and children's service. Nottingham: DfES. Available at: http://webarchive.nationalarchives.gov.uk/20061209025756/dfes.gov.uk/publications/e-strategy/. (Accessed: 4 January 2012) Dillman, D. (2000). Mail and Internet surveys: The tailored design method (2nd Edition). New York: John Wiley and Sons. Donovan, M.S., Bransford, J.D. & Pellegrino, J.W. (1999). How people learn: Bridging research and practice. Washington, DC: The National Academies Press. Available at: http://books.nap.edu/catalog.php?record_id=9457. (Accessed: 4 January 2012) EdExcel (2006). Edexcel – Minimum Standards For NVQ E-Portfolios. Available at: http://www.edexcel.com/migrationdocuments/Family%20top%20level/Edexcel___Minimum_standards_for _e_portfolio_v0_7_Final.pdf. (Accessed: 30 April 2012)

47

B00494857

EDUCAUSE (2011). ECAR National Study of Undergraduate Students and Information Technology, 2011—Infographic. Available at: http://www.educause.edu/studentsAndTechnologyInfographic. (Accessed: 30 April 2012) EIfEL: European Institute for E-Learning, (2007). Europortfolio. Available at :http://www.eifel.org/about/europortfolio. (Accessed: 4 March 2012) Ewell, P.T. (1997). Organizing for learning: A point of entry. Draft prepared for discussion at the 1997 AAHE Summer Academy at Snowbird. National Center for Higher Education Management Systems. Available at: http://www.intime.uni.edu/model/learning/learn_summary.html. (Accessed: 4 January 2012) Fricker, R.D. (2006). Sampling Methods for Web and E-mail Surveys N. Fielding, R. Lee, & G. Blank, eds. October, 1(October), p 1-43. Available at: http://books.google.com/books?id=EeMKURpicCgC&pgis=1. (Accessed: 30 April 2012) Gaitán, A., Manton, R. & Jankowska, A. (2007). Piloting e-portfolios at the University of Bedfordshire. Available at: http://www.beds.ac.uk/__data/assets/pdf_file/0014/12065/pilotingeportfolios.pdf. (Accessed: 23 April 2012) Gemmell, J. (2007). MyLifeBits project. Microsoft Research. Available at: http://research.microsoft.com/en-us/projects/mylifebits/. (Accessed 30 April 2012) Gibbs, A. (2011). Issue 19 Winter 1997 Focus Groups. Social Research, (19), p 1-7. Gilbert, C. (2006) 2020 Vision: Report of the Teaching and Learning in 2020 Review Group. Nottingham: DfES Publications. Available at: http://dera.ioe.ac.uk/6347/1/6856-DfESTeaching%20and%20Learning.pdf. (Accessed: 4 March 2012) Grant, S. (2009) PIOP 2: the second round of Portfolio InterOperability Projects. JISC. Available at: http://wiki.cetis.ac.uk/images/8/83/PIOP2-summary.pdf. (Accessed: 4 March 2012) Greene, J.C., Caracelli, V.J. & Graham, W.F. (1989). Toward a Conceptual Framework for Mixed-Method Evaluation Designs. Educational Evaluation & Policy Analysis, 11(3), p 255-274. Available at: http://epa.sagepub.com/cgi/doi/10.3102/01623737011003255. (Accessed: 30 April 2012) Harrison, C. (2011). Re: e-Portfolios. Email to [email protected], on 25 October 2011. Hartnell-Young, E. et al. (2007). Impact study of e-portfolios on learning. Nottingham: BECTA. HEFCE: Higher Education Funding Council for England, (2005). HEFCE Strategy for E-Learning. Bristol: HEFCE. Available at: http://www.hefce.ac.uk/pubs/hefce/2005/05_12/. (Accessed: 4 March 2012) Hopkins, D. 23 October (2011). ‗ePortfolios‘ are out, ‗bPortfolios‘ are in. eLearning Blog Dont Waste Your Time. Available at: http://www.dontwasteyourtime.co.uk/eportfolio/eportfolios-are-out-bportfolios-are-inapparently-eportfolio-bportfolio/. (Accessed: 4 March 2012) Honey, P. & Mumford, A. (1982). Manual of Learning Styles. London: P Honey. HM Treasury (2006).The Leitch Review of Skills: Prosperity for All in the Global Economy - World Class Skills. London: HMSO. Available at: http://webarchive.nationalarchives.gov.uk/+/http://www.hmtreasury.gov.uk/leitch_review_index.htm. (Accessed: 4 March 2012) Jafari, A. (2004). The Sticky e-Portfolio System: Tackling Challenges and Identifying Attributes. Educause Review, 39(4), p 38-49. Available at: http://net.educause.edu/ir/library/pdf/ERM0442.pdf. (Accessed: 30 April 2012) JISC: Joint Information Systems Committee (2008). Effective Practice with e-Portfolios: Supporting 21st century learning. Bristol: JISC. Available at: http://www.jisc.ac.uk/media/documents/publications/effectivepracticeeportfolios.pdf. (Accessed: 10 March 2012) JISC: Joint Information Systems Committee (2010). Effective Assessment in a Digital Age. Assessment. Available at: http://www.jisc.ac.uk/digiassess. (Accessed: 4 January 2012) 48

B00494857

JISC: Joint Information Systems Committee (2012a). e-Portfolios - An overview. Available at: http://www.jisc.ac.uk/eportfolio. (Accessed: 4 January 2012) JISC: Joint Information Systems Committee (2012b). ePortfolio InfoKit: Emerging Lessons. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/emerging-lessons. (Accessed: 10 March 2012) JISC: Joint Information Systems Committee (2012c). ePortfolio InfoKit: Technologies. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/technologies. (Accessed: 4 January 2012) JISC: Joint Information Systems Committee (2012d). Assessing e-Portfolios: Guidance for Practitioners. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/assessment-guidance. (Accessed: 4 March 2012) JISC: Joint Information Systems Committee (2012e). e-Portfolio Implementations Toolkit: What Does Good Implementation Look Like. Available at: https://epip.pbworks.com/w/page/44803404/What%20does%20good%20implementation%20look%20like. (Accessed: 30 April 2012) JISC: Joint Information Systems Committee (2012f). ePortfolio InfoKit: Implementing. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/implementing. (Accessed: 4 January 2012) JISC: Joint Information Systems Committee (2012g). ePortfolio InfoKit: Choosing. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/choosing. (Accessed: 4 January 2012) JISC: Joint Information Systems Committee (2012h). PDP4XL2 Case Study: Success Factors. Available at: http://www.jiscinfonet.ac.uk/case-studies/e-portfolios/bournemouth-pdp4lx2/success. (Accessed: 30 April 2012) JISC: Joint Information Systems Committee (2012i). e-Portfolio Implementations Toolkit: Implementation guidance for practitioners. Available at: https://epip.pbworks.com/w/page/40808690/Implementation%20guidance%20for%20practitioners. (Accessed: 30 April 2012) JISC: Joint Information Systems Committee (2012j). ePortfolio InfoKit: Personalisation. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/personalisation. (Accessed: 30 April 2012) JISC: Joint Information Systems Committee (2012k). e-Portfolio Implementations Toolkit: Initiation Models. Available at: https://epip.pbworks.com/w/page/45691573/Initiation%20models. (Accessed: 30 April 2012) JISC: Joint Information Systems Committee (2012l). System Implementation InfoKit: Which Systems does this Kit cover? Available at: http://www.jiscinfonet.ac.uk/InfoKits/system-implementation/whichsystems/#lifecycle. (Accessed: 4 January 2012) JISC: Joint Information Systems Committee (2012m). ePortfolio InfoKit: Preparing for Success. Available at: http://www.jiscinfonet.ac.uk/infokits/e-portfolios/preparing-success. (Accessed: 30 April 2012) Joyes, G., Gray, L. & Hartnell-Young, E. (2010). Effective practice with e-portfolios: How can the UK experience inform implementation? Australasian Journal of Educational Technology, 26(1), p 15-27. Available at: http://www.ascilite.org.au/ajet/ajet26/joyes.html . (Accessed: 30 April 2012) Kim, P., Ng, C.K. & Lim, G. (2010) When cloud computing meets with Semantic Web: A new design for eportfolio systems in the social media era. British Journal of Educational Technology, 41(6), p 1018-1028. Available at: http://eport.uaa.alaska.edu/BJET_PRPL_Eportfolio.pdf. (Accessed: 30 April 2012) Kolb, D. (1984). Experiential Learning experience as a source of learning and development. New Jersey: Prentice Hall. Kozma, R. (2009) Transforming Education: Assessing and Teaching 21st Century Skills: A Call to Action. The Transition to Computer-Based Assessment: New Approaches to Skills Assessment and Implications for Large-scale Testing. Luxembourg: European Communities. Available at:

49

B00494857

http://www.emotionproject.eu/download/The%20Transition%20to%20ComputerBased%20Assessment.pdf#page=13. (Accessed: 4 March 2012)

Layte, M. (2008). The ePortfolio in Europe. Unpublished presentation at: ePortfolios, identity and personalised learning in healthcare education, 28 February 2008, Newcastle-upon-Tyne, England. Lewin, K. (1946). Action research and minority problems G. W. Lewin, ed. Journal of Social Issues, 2(4), p 34-46. Available at: http://onlinelibrary.wiley.com/doi/10.1111/j.1540-4560.1946.tb02295.x/full. (Accessed: 30 April 2012) Lowendahl, J. (2011) Hype Cycle for Education, 2011. Gartner. Available at http://www.gartner.com/DisplayDocument?docCode=214466. (Accessed: 29 February 2012) Madden, T. (2007). Supporting Student e-Portfolios. Hull: Higher Education Academy Physical Sciences Centre. Available at: http://www.heacademy.ac.uk/assets/ps/documents/practice_guides/eportfolios_jisc.pdf. (Accessed: 30 April 2012) Meyer, B. & Latham, N. (2008). Implementing Electronic Portfolios: Benefits, Challenges and Suggestions. EDUCAUSE Quarterly, 1(1), p 34-41. Available at: http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/ImplementingEle ctronicPortfoli/162514. (Accessed: 30 April 2012) Microsoft (2012). Usability in Software Design. Windows Desktop Development. Available at: http://msdn.microsoft.com/en-us/library/windows/desktop/ff728829(v=vs.85).aspx#defining_usability. (Accessed: 30 April 2012) Miller, A. (2011). Key Considerations when Implementing e-Portfolios. Workshop. Australian Flexible Learning Network. Available at: http://www.slideshare.net/taselu/key-considerations-when-implementingeportfolios. (Accessed: 30 April 2012) NCIHE: National Committee of Inquiry into Higher Education (1997). Higher education in the learning society. Report of the National Committee of Inquiry into Higher Education: The Dearing Report. London: HMSO. Nielsen, J. (2003). Usability 101: Introduction to Usability. Jakob Nielsen's Alertbox. Available at: http://www.useit.com/alertbox/20030825.html. (Accessed: 30 April 2012) OCR e-Assessment Team (2007). Prospective E-Portfolio Product Provider: Pre-Evaluation Checklist. Available at: http://www.ocr.org.uk/download/eassessment/ocr_24917_eassess_checklist.pdf. (Accessed: 30 April 2012) Paris, S. & Ayres, L. (1994). Becoming Reflective Students and Teachers. Washington: American Psychological Association. Powell, R. A. & Single, H. M. (1996). Focus groups. I. Bourgeault, R. Dingwall, & R. De Vries, eds. International journal for quality in health care journal of the International Society for Quality in Health Care ISQua, 8(5), p 499-504. Prensky, M. (2001). Digital Natives, Digital Immigrants. On the Horizon, 9(5), p 1-6. Available at: http://www.marcprensky.com/writing/Prensky%20%20Digital%20Natives,%20Digital%20Immigrants%20%20Part1.pdf. (Accessed: 4 January 2012) QAA: Quality Assurance Agency (2001). Guidelines for HE Progress. Available at: http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/progfile2001.pdf. (Accessed: 4 January 2012) QCA: Qualifications and Curriculum Authority (2004). A Proposed Blueprint for Delivering e-Assessment. The Qualifications and Curriculum Authority. Available at: http://webarchive.nationalarchives.gov.uk/20081105143757/http://qca.org.uk/libraryAssets/media/6995_bl ueprint_for_e-assessment.rtf. (Accessed: 4 January 2012)

50

B00494857

Ramsden, P. (1992). Learning to teach in higher education, London: Routledge REAP (2007). Re-engineering Assessment Practices in Higher Education. Available at: http://www.reap.ac.uk/. (Accessed: 4 January 2012) Reppler (2011). Managing Your Online Image Across Social Network. Available at: http://blog.reppler.com/2011/09/27/managing-your-online-image-across-social-networks/. (Accessed: 21 April 2012) Riedinger, B. (2006). Mining for meaning: teaching students how to reflect. In A. Jafari & C. Kaufman, eds. Handbook of Research on ePortfolios. Idea Group Reference, pp 90–101. Riel, M. (2010). Understanding Action Research. Center For Collaborative Action Research/Pepperdine University. Available at: http://cadres.pepperdine.edu/ccar/define.html. (Accessed: 30 April 2012) Ripley, M. (2007), Report 10 update: E-assessment – an update on research, policy and practice. Bristol: Futurelab. Available at: http://www.futurelab.org.uk/resources/documents/lit_reviews/Assessment_Review_update.pdf. (Accessed 21 April 2012) Robbins, A. (2009). Wealth Mastery Workshop. Brochure available at: http://www.anthonyrobbinsasia.com/downloads/WM09_brochure.pdf. (Accessed: 16 November 2009) Rubín, S. & Rümler M. (2012). E-PORTFOLIO: A METACOGNITIVE ACTIVITY TO DEVELOP PERSONAL AND PROFESSIONAL LIFE PLANS FOR STUDENTS, INTED2012 Proceedings, pp 43194327. Salmon, G. (2002). 5 Stage Model. E-tivities: the key to active online learning. London: Taylor & Francis. Available at: http://www.atimod.com/e-tivities/5stage.shtml. (Accessed: 8 May 2012) Schacter, D.L. (1996). Searching for Memory: The Brain, the Mind, and the Past. New York: BasicBooks. Schön, D. (1983). The Reflective Practitioner. How professionals think in action. London: Temple Smith. Shada, A., Kelly, K., Cox, R. & Malik, S. (2011). Growing a New Culture of Assessment: Planting ePortfolios in the Metro Academies Program. International Journal of ePortfolio, 1(1), 71-83. Available at: http://www.theijep.com/pdf/IJEP35.pdf. (Accessed: 30 April 2012) SQA: Scottish Qualifications Authority (2007). E-assessment: Guide to Effective Practice. Glasgow: SQA. Available at: http://www.sqa.org.uk/sqa/files_ccc/Section_4.pdf.http://www.sqa.org.uk/sqa/23369.html (Accessed: 4 March 2012) SQA: Scottish Qualifications Authority (2011). SQA requirements for e-portfolios. Available at: http://www.sqa.org.uk/sqa/files_ccc/SQA_requirements_for_e-portfolios-web_version_v6.1.docx. (Accessed: 30 April 2012) Swallow, E. (2011). How Businesses Use Social Media for Recruiting [INFOGRAPHIC]. Available at: http://mashable.com/2011/08/28/social-media-recruiting-infographic/. (Accessed: 21 April 2012) Taylor, L. & Gill, S. (2006). EPICS - Final Report. Newcastle: Newcastle University/JISC. Available at: http://www.jisc.ac.uk/media/documents/publications/effectivepracticeeportfolios.pdf. (Accessed: 10 March 2012) Tosh, D., Light, T.P., Fleming, K. & Haywood, J. (2005). Engagement with electronic portfolios: Challenges from the student perspective. Canadian Journal of Learning and Technology, 31(3), p 89–110. Available at: http://www.cjlt.ca/index.php/cjlt/article/view/97/91. (Accessed 4 March 2012) Ward, C. & Moser, C. (2008). E-Portfolios as a Hiring Tool: Do Employers Really Care? EDUCAUSE Quarterly 31 no 4 October December, 31(4), p 13-14. Wheeler, S. 2 May (2011). The natives are revolting. Learning with 'e's. Available at: http://stevewheeler.blogspot.com/2011/05/natives-are-revolting.html. (Accessed 21 April 2012)

51

B00494857

Wicks, D. (2011). bPortfolios: Blogging for Reflective Practice. Available at: http://sloanconsortium.org/effective_practices/bportfolios-blogging-reflective-practice. (Accessed 4 March 2012) Winkley, J. & Roads, G. (2007). Developing the use of assessment e portfolios. Manchester: AlphaPlus, 2011. Available at: http://www.alphaplusconsultancy.co.uk/pdf/Becta%20Research%20%20Assessment%20e-portfolio.pdf. (Accessed: 4 March 2012) Wright, T. & Tsao, H.J. (1983). A Frame on Frames: An Annotated Bibliography. In T. Wright ed. Statistical Methods and the Improvement of Data Quality. New York: Academic Press. University of Ulster (2006). Policy for the Governance of Research on Human Subjects and the Accompanying Procedures. University of Ulster. Available at: http://research.ulster.ac.uk/office/docs/d_policy.pdf (Accessed: 30 April 2012) Yancey, K. (2009). Electronic portfolios a decade into the twenty-first century: What we know, what we need to know. Peer Review, 11(1), p 28-32.

52