reference_paper_1_Sahana_Murthy (1).pdf

Design and Deployment of Clickers in Distance Education A Synchronous, Distributed approach for use of Student Response Systems

Abstract—While distance education is becoming increasingly popular, it suffers from the problem of a lack of real-time interactivity between the instructor and the students, as well as an absence of peer interaction between students. A technique used in faceto-face classrooms to improve student-teacher interaction and promote peer discussion among students is the use of clickers, or student response systems. In this paper we describe the adaptation of clickers in distance education for a multiple classroom environment. We discuss the design and development of a distributed architecture for the implementation of a student response system for multiple remote classrooms. We report on a pilot implementation and discuss problems faced and solutions implemented. Keywords- Student Response Systems, Distance education, Clickers, Pedagogy, Distributed, Client-Server, Remote, FTP

INTRODUCTION Distance education is increasingly being seen as a cost-effective solution to the universal problem of providing high quality and low cost education to growing numbers of students [1]. With the availability of new and better technologies, distance education is reshaping course structures and the pedagogy in the classrooms. Yet, studies on student perceptions towards distance and blended learning do not show a clear preference for these new modes of learning. A European-wide survey report [2] states that students are positive towards the use of ICT in university learning and teaching, but they have a preference for learning with traditional educational methods. As per a study conducted at Texas State University [3], students show a preference for conventional classroom teaching. One of the reasons for such a preference of conventional in-the-class education over distance education is the limited opportunity for synchronous teacher-student interactivity in a virtual environment [4]. Much of the present efforts in increasing studentteacher interactivity in distance education are aimed at diminishing the physical gap between the instructor and the students in the virtual classroom. Video and audio conferencing is one of the most commonly used systems for a scenario where students and instructors are separated physically. Other examples that promote interactivity in distance

classrooms include “A mouse on every desk” by Microsoft Research [5] and numerous virtual interactive learning environments. However, due to diminished visual cues, the instructor does not have sufficient means to gauge the comprehension level of the class in real-time. Another serious consequence is that students miss out on peer discussion and dialogue with the instructor as compared to the conventional classroom [3]. A possible way of addressing the above problems is to adapt known pedagogical techniques used in face-to-face classrooms. One effective method of promoting peer discussion among students is through the use of ‘Clickers’ which are also known as Student Response Systems, Classroom Response Systems, Personal Response Systems and Classroom Performance Systems among many other names [6]. Student Response Systems have proved to be very effective in conventional classrooms in facilitating active learning and improving teaching methodology [7]. The benefits of using clickers and peer discussion include improved class attendance, more active participation in class, increased collaborative activities, greater student satisfaction and better learning [8,9]. The use of such systems has improved teaching effectiveness as immediate feedback is available both to students, who learn via peer discussion and to the instructor who uses the feedback to amplify, clarify or review what has been taught. Another possible use of student response systems is the assessment of students. This can be easily and quickly done, especially with large numbers of students. Is it possible to embrace the implementation and benefits of student response systems for distance education? What modifications need to be done, both in terms of development of new technologies and in the pedagogy used? In this paper, we describe a solution to facilitate real-time interaction between the instructor and the remote classroom students through the use of a student response system involving clickers. We designed and developed a distributed architecture for a student response system. Our setup involved a central location at which the instructor delivered an interactive lecture. The lectures were transmitted via satellite to 22 remote centres in different cities all over India. Participants at the remote centres had access to clickers and the remote classrooms were equipped with receivers. The receivers could communicate to a server located in the

central classroom over the internet. Participants could respond to questions presented by the instructor through clickers, in the same manner as the conventional classroom students even though they were physically separated by their instructors by hundreds of miles. The instructor was able to collect and compare the responses to various questions posed during the lectures from participants in various remote centres. A similar solution was implemented by University of Oklahoma, College of Pharmacy for a dual classroom environment [10]. According to the study, students and faculty members felt that the immediate feedback the automated response system provided was more beneficial during non-graded activities, which is consistent with previous research [9]. In our implementation, we scaled up the solution and implemented it in a multiple classroom (23) environment. In this paper, we detail the design and architecture of the student response system we developed. We describe a pilot implementation of the system in multiple classrooms. We discuss in detail the problems we encountered, which were of technical as well as social nature. We report solutions implemented and propose several others.

HARDWARE AND SOFTWARE IN A STUDENT RESPONSE SYSTEM A. A Typical Student Response System A general student response system consists of two parts. • Hardware: Receiver and clickers The Hardware consists of a number of inexpensive keypads which function as clickers and one or more receivers connected to a computer placed within a classroom. Typically, each student has access to a clicker. The number of receivers depend on the number of clickers and the size of the classroom. Both clickers and receivers communicate using infra-red or radio frequencies based wireless technologies. •

Software: The software allows the instructor to initiate response collection, collect and store the response data in a database, represent the information collected in various ways such as graphs and charts, and generate reports analyzing the responses.

Figure 1 Distributed System Setup

In a typical scenario of a classroom implementation, each student’s clicker is associated with his or her identification details (such as a roll number). The instructor poses a multiple-choice question to the class using a projector or a board. When all the students have keyed in their preferred choices on their respective clickers, the instructor uses the software to collect all the response. These responses are stored in an xml file on the computer. Subsequently, these responses are processed by the software to be used by the instructor in form of graphs or charts to facilitate discussion in the class or to generate reports for student performance analysis. B. Our Approach The main idea behind the extension of a standard student response system to distance education lies in the use of the xml file that is created by the software to store all the responses. We have implemented a software interface wherein: - A central server initiates a response collection at all remote centres simultaneously. There is no need for the local co-coordinator at each remote centre to initiate response collection manually. - The response data collected in the xml file is immediately transferred back to the central server using FTP. Data from all remote centres are collected into a single database to be used by the instructor subsequently. Such a system can be easily implemented to collect responses for a synchronous real time questions presented by the instructor during the class. Figure 1 shows a schematic diagram explaining the model. ARCHITECTURE OF THE DISTRIBUTED STUDENT RESPONSE SYSTEM C. Features of the Distributed Student Response System To attempt a working model of our approach, we used a Student response system developed by the Indian Institute of Technology (IIT) Bombay in 2009. The system is intended to be used first in IIT Bombay lectures, and then to be extended to all colleges and schools across India. Our distributed student response system is developed using open source software and easily available hardware components. The design and source code of the system will be released in open-source and proposed as a standard to allow

interoperability between manufacturers. The salient features of the Clicker system developed at IIT Bombay are: - The complete system is developed using open source software and easily available hardware components. - The design and source code of the system will be made open-source as well as proposed as a standard to allow interoperability between manufacturers. The student response system was developed on the same technical lines as any standard clicker systems in the market, albeit at a fraction of the cost. This system was extended to support a distributed environment by enhancing the software. D. The Architecture The main components of the distributed system are as follows: • The Hardware The hardware consists of multiple radio frequency based clickers. Each clicker unit has a 9key keypad. The receiver is a similar device with an additional connection to the computer. Each remote centre is equipped with a receiver, which is connected to a computer that has access to the internet. • The Software The software is divided into two parts. The first part runs at the instructor’s computer at a central location. This software is used to: o Display the question to be asked during the session. o Read, store and interpret the data collected from the remote centres. The second part of software runs at the each of the remote centres on the computer system attached to the receiver. This software handles the responses collected by the receiver, writes them to an xml file and sends the file to the file server located at the instructor location. Figure 2 represents the control flow of the system. The instructor initiates the process and poses the question to the students as she would do in a conventional classroom. The students enter their response using their clicker device. The responses entered are collected when the call to collect responses is sent. The responses are written in an xml file which is then sent to the central FTP server. The instructor’s system accesses these xml files and stores them in a central database. The system uses these data to see how many students responded, and how many responses are correct/incorrect. As in a conventional classroom, the system can also be used

to collect attendance of the class. Figure 3 depicts the protocol diagram of the process followed.

REMOTE CLASSROOM

APPLICATION AT INSTRUCTOR’S END Display of Questions to all remote classrooms through live video

The receiver polls the clickers, gets the responses and stores them in an xml file

Collect Responses

Trigger by instructor to central server to initiate response collection at remote classroom

SERVER

Response File Sent Through FTP

Central Server sends a trigger to all remote classrooms to collect responses.

Figure 2: Flow Diagram

FTP SERVER Socket()

Bind() REMOTE CENTER Ready()

Socket()

Trigger Connect()

Connection Established Send_File()

Store()

Close_Connection()

Figure 3: Protocol Diagram

For the above process to work, each remote centre needs to install the software on the system on which the clicker receiver was attached. For the software to run on those systems, JRE 1.6 and Python 2.6 need to be installed. The software at the remote centre needs to be run when the instructor at the central location presents a question. This requires the presence of a coordinator at each remote centre. The instructor’s system at the central location also needs to be installed with JRE 1.6. As the instructor presents questions during the class, the software waits for the xml file from each remote centre to be received by the file server. Once all the files are collected, the software displays the results in the same manner as in a conventional classroom using a student response system. IMPLEMENTATION OF THE DISTRIBUTED STUDENT RESPONSE SYSTEM The ISTE workshop on Effective teaching techniques for teachers held under the eOutreach program conducted by IIT Bombay in December, 2009 was used to test the feasibility of the Distributed clicker system. The workshop was conducted via distance mode at one central location and 22 remote centres, with 473 participants spread across India. The lectures were delivered from a central location at our institute. The lectures were broadcast through EDUSAT, a satellite dedicated to the education sector by the Indian Space Research Organization (ISRO). Each remote centre had a local coordinator whose responsibility was to set up the initial hardware and software, test the clickers and receiver and be present during the workshop to facilitate participants in using the clickers. Two weeks before the workshop all remote centres were provided with the client software and the clicker system. Technical support was provided to the remote centres to help them with the setup of the system for the workshop. Three test sessions were also held before the workshop to iron out the setup problems faced by the remote centres. E. Data of number of responses collected Out of the 22 remote centres, 5 were not able to connect to the FTP server at the central location during the first two days. By the end of the workshop this number came down to 2. The results of the centres that were not connected are not included in the final data. Figure 4 shows the number of remote centres connected to the central server on each of the days a clicker question was posed.

Table 1

Figure 4: Remote Centers connected to the FTP server During the lectures, the instructor asked multiplechoice questions which required the workshop participants to enter their responses using the clickers. All together six questions were presented to the participants during the workshop. Out of the 473 participants present for the workshop, not all used the clickers to respond to the questions. If a participant tried to respond to a question but no response is registered, we regarded it as a time-out. Reasons for time-outs included improper handling of the clicker devices and malfunctioning clickers.

Remote Centre 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Total

No. of clickers used 33 32 30 27 25 22 21 19 13 12 12 12 12 7 4 3 3 1 0 0 0 0 288

No. of clickers not used 2 0 2 2 0 0 7 0 3 1 12 10 0 1 14 11 22 22 30 17 22 9 187

As a sample, the actual usage data for day 3 are shown in Table 1. Only 288 of the 473 participants used clickers to enter their responses for the question asked. This resulted in 187 time-outs in all (~40%), if we include all the clickers that we dispatched. One main reason for this was that clickers were an unfamiliar classroom device for many participants, which led to some discomfort with their usage. In Table 2, we discuss this and other problems we encountered in implementing the distributed clicker system. Figure 5 shows the response rate per question over six days of the workshop. Between 220 and 273 responses were collected per question. If we include only those clickers that were actually used (not all 473), the time-out rate was 5%-20%. As we see in Figure 5, there was an improvement in the time-out rate as the workshop progressed.

Figure 5: Responses collected per question F. Instructor and participant feedback on distributed clicker system We interviewed the instructor to get his reactions on the use of the distributed clicker system. His overall experience with the use of clickers in the distributed system was very positive. Based on the responses collected he was able to gauge the comprehension level of the participants. He gave the example of one such question asked during the session when he presented the class with a factual question about the history of computers. All of the participants answered the question correctly whereas in the very next question which he thought would be apparent to the participants since they answered the previous one correctly, all of the participants answered incorrectly. Such feedback from the class also initiated new discussions during the class. A sample of the question asked is shown in the figure 6.

Figure 6: Sample Question The sample question used in the above example (Fig. 6) is a factual question on the history of computers, yet it brought forth interesting discussions, and provided valuable feedback to the instructor. There could be other types of questions used, such as those that test the recall of an idea, ask students to calculate the numerical answer of a problem, apply a concept to a new scenario and so on. Each of these kinds of questions is suitable for a different purpose. For example, a recall question based on the previous lecture could be a revision for students at the beginning of the following lecture. Questions that test conceptual understanding, or ask students to predict results of an algorithm, or connect different representations of a concept have been shown to have a higher impact on learning [8]. An example of a question where students are asked to analyze a program code and predict the value of a variable, is shown below in Figure 7. Such questions could be used to initiate discussions among the participants as well as encourage them to discuss their doubts with the Instructor.

Figure 7: Sample Question

After the workshop the participants were requested for feedback about the workshop. 30 % of the participants said that their reason to join the workshop was to experience distance learning. Almost 30% of all the participants ranked clickers as one of the top three things they liked about the workshop. 91% agreed that clicker methodology was useful, out of which 34% were in strong agreement. 61% of participants wanted that such a workshop should be repeated in future. DISCUSSION The pilot implementation we conducted with the distributed student response system showed that instructors can consider using clickers as an effective tool in the teaching-learning process even over a distance mode, with multiple remote centers. Use of a system in distance education that would send realtime responses to the instructor was exciting to most participants. The use of the system motivated the participants to take part in future in such distance education classes. The instructor benefited largely as he could modify the pace of the session as per the comprehension level of such a diverse group of participants. Despite the distributed nature of the system, the instructor was able to collect responses from most of the students as effectively as in a regular classroom implementation of the clicker. The implementation also brought forth the shortcomings of the system. In Table 2, we list some of the common problems we faced, the solution we implemented and future solutions we plan. Some of the problems we faced dealt with the software installation and lack of synchronization during software upgrade processes. A large part of the Problem

Remote Center Coordinators not able to set up the software and hardware

Software updates were not being installed by the remote centers

The participants were afraid to use the clickers Responses collected had almost 40-50% timeouts

reason for this category of problems was that the clicker software installation and upgrade process had multiple steps. In this respect, we learned that a onestep installation process such as those used in commercial software systems would minimize these problems. As the remote centres were located in over 20 different cities, problems during the hardware and software installation appear more severe. Lack of trained manpower at the remote centres contributed largely to the inability to understand instructions in the manuals and handbooks provided with the system. During the pilot implementation, we attempted to solve this problem by having trained technicians from our clicker development team personally visit the remote centers to troubleshoot. We also provided technical support in the form of online chat sessions and phone calls. In future implementations, we plan to have training sessions for all remote centre coordinators. The clicker system was new to most of the participants hence they were not comfortable in using the clickers initially. But by the end of the workshop, we noticed that participants became more comfortable in using them. Future versions of the clickers would provide a more user-friendly interface to avoid these problems. Apart from resolving the issues faced in this version, the future version would also include two way communication via the clickers. Also, in future the design and the code of the system would be released in open-source.

Solution Implemented

1.

Detailed documentation was provided for set up. 2. Technical support provided to all the remote centers through chat sessions and phone from 9 AM to 6 PM. 3. Software and hardware team members were sent to the centers where the above two solutions did not work. All the centers were informed over mail as well over phone about the upgraded software. Part of the problem was that unlike the commercial software, the clicker software had multiple steps in up gradation process. The center coordinator would keep the initial version when they found the upgrades to be cumbersome. The system was new to all of the participants and they would not enter their response as they would get confused about which keys to press. The system was new to all of the participants and they would not enter their responses in the time in which the responses were collected by the system.

Future Solution

A training session will be held for all the center coordinators well in advance to enable them to handle any issues related to set up of the system.

The next version of the clicker system would have a one click installation like the commercial software.

The next version of the clicker system is very user friendly and has LED display that would tell the participants of the response they have entered. The next version is more user friendly so that the participants can easily understand and enter their responses.

The centers were not able to send their files through FTP

Some of the centers were behind a proxy server that would not allow the system to send the file through FTP. To resolve this the FTP server had to be added as an exception to the default browser of the remote center system.

The documentation provided would take care of such possible scenarios.

Lack of trained manpower at the remote centers

Clicker development team members went to some of the centers to resolve the technical issues faced by the center.

Training will be provided to the center coordinators before the commencement of the workshop so that they can handle any issue arising at their center.

Table 2: Problems faced and solutions implemented CONCLUSION In this paper, we described the design and architecture of a distributed student response system. We demonstrated its use in a multiple remote classroom environment. Such a system can enable to emulate the use of a student response system in a multiple remote classroom as effectively as in a conventional classroom. The results from the workshop in which we implemented the distributed student response system demonstrate that such a system can improve class participation in a distance education class. The system may also help the instructor to address the specific needs of the participant pool. With the correct choice of questions, such a system can also initiate peer discussions in a multiple remote classroom scenario. The system enables the instructor to collect data from a very large pool of students and can be extended to comprehend the needs of the participants on various classifications such as demographics, back ground etc. Such a system can also be scaled up to include more remote centers, however such a claim has its reservation due to problems faced by the remote centers in the experiment (Table 2). However these problems should not be faced in future implementations after the solutions are applied. While the use of clickers in a distance teachinglearning mode can be successful, specific problems exist due to the distributed nature of the system. We have enumerated some problems we encountered in our pilot implementation and suggested possible solutions. Instructors who want to use clickers in the distance mode need to pay extra attention to these problems in order to maximally benefit from the power of clickers. Acknowledgments Our sincere thanks to Prof. D.B. Phatak, Prof. Sridhar Iyer and the clicker software and hardware team at Affordable Systems Laboratory, Computer Science and Engineering Department, Indian Institute of Technology Bombay for their invaluable inputs which went into the writing of this paper. A special mention to Mr Manjurelahi Patil of the clicker

software team for providing a consolidated data for the workshop.

References [1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

Diana G. Oblinger, Carole A. Barone and Brian L. Hawkins “Distributed education and its challenges: An overview,” American Council on Education Centre for Policy Analysis, 2001. Mary Jo Garcia Biggs “Comparison of student percetions of classroom instruction:Traditional, Hybrid and Distance Education.” Turkish Online Journal of Distance Education, Volume 7, Number 2, Article: 4, April 2006. Survey report from the SPOT+ project supported by the European Commission. Available at http://www.spotplus.odl.org as of April 2010. Yuji Tokiwa, Koji Nonobe, and Masami Iwatsuki, “Web based tools to sustain the motivation of students in Distance Education,” 39th ASEE/IEEE Frontiers in Education Conference October 2009. Roger C. Lowery, “Clickers in the Classroom: A Comparison of Interactive Student-Response Keypad Systems,” National Social Science Association’s National Technology and Social Science Program, 5-7 April 2006, Las Vegas, NV, USA. Catherine Crouch and Eric Mazur, "Peer Instruction: Ten years of experience and results." American Journal of Physics -- September 2001 -- Volume 69, Issue 9, pp. 970977, 2001. Robert Kaleta and Tanya Joosten, “Student Response Systems: A university of Wisconsin system study of clickers,” Centre for Applied Research – Research Bulletin, Volume 2007, Issue 10, May 8, 2007. "Clicker resource guide” available at from the Carl Wieman Science Education Institute, as of April 2010 at: http://www.cwsei.ubc.ca/resources/clickers.htm. Patrick J. Medina, Nelson Er, Jane E. Wilson, Mark L. Britton, Melissa S. Medina, Donald S. Wanzer, "Use of an Audience Response System (ARS) in a Dual-Campus Classroom Environment". American Journal of Pharmaceutical Education, May 2008. [10] Neema Moreveji, Udai Pawar and Taiwei Kim, “A mouse on every desk: An inexpensive classroom interaction technique for remote teaching:”. Available as of April 2010 at http://moraveji.org/images/projects/mouse_on_each_desk_l ong.pdf [11] Information on IIT Bombay’s eOutreach program and workshop is available at: http://ekalavya.it.iitb.ac.in/EffectiveTeaching_Course.do