Students use intelligent tutors and other types of interactive learning environments in a considerable variety of ways. In this thesis, I detail my work to understand, automatically detect, and re-design an intelligent tutoring system to adapt to a behavior I term “gaming the system”. Students who game the system attempt to succeed in the learning environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly. Within this thesis, I present a set of studies aimed towards understanding what effects gaming has on learning, and why students game, using a combination of quantitative classroom observations and machine learning. In the course of these studies, I determine that gaming the system is replicably associated with low learning. I use data from these studies to develop a profile of students who game, showing that gaming students have a consistent pattern of negative affect towards many aspects of their classroom experience and studies. Another part of this thesis is the development and training of a detector that reliably detects gaming, in order to drive adaptive support. In this thesis, I validate that this detector transfers effectively between 4 different lessons within the middle school mathematics tutor curriculum without re-training. This detector uses Latent Response Models Maris 1995), combining labeled and unlabeled data at different-grain sizes, in order to train a model to accurately indicate both which students were gaming, and when they were gaming, and uses Fast Correlation-Based Filtering Yu and Liu 2003) to efficiently search the space of potential models. The final part of this thesis is the re-design of an existing intelligent tutoring lesson to respond to gaming. The re-designed lesson incorporates an animated agent “Scooter the Tutor”) who indicates to the student and their teacher whether the student has been gaming recently, and gives students supplemental exercises, in order to offer the student another chance to learn material he/she gamed through. Scooter reduced the frequency of gaming by over half, and Scooters supplementary exercises were associated with substantially improved learning； Scooter appeared to have little effect on non-gaming students.
Tag Archive: ComputerScience
Designing intelligent tutors that adapt to when students game the system (Education Papers posted on March 22nd, 2013 )
Marmoset: A programming project assignment framework to improve the feedback cycle for students, faculty and researchers (Education Papers posted on March 22nd, 2013 )
We developed Marmoset, a system that improves the feedback cycle on programming assignments for students, faculty and researchers alike. Using automation, Marmoset substantially lowers the burden on faculty for grading programming assignments, allowing faculty to give students more rapid feedback on their assignments. To further improve the feedback cycle, Marmoset provides students with limited access to the results of the instructors private test cases before the submission deadline using a novel token-based incentive system. This both encourages students to start their work early and to think critically about their work. Because students submit early, instructors can monitor all students progress on test cases and identify where in projects students are having problems in order to update the project requirements in a timely fashion and make the best use of time in lectures, discussion sections, and office hours. To study in more detail the development process of students, Marmoset can be configured to transparently capture snapshots to a central repository every-time students save their files. These detailed development histories offer a unique, detailed perspective of each students progress on a programming assignment, from the first line of code written and saved all the way through the final edit before the final submission. This type of data has proved extremely valuable for many uses, such as mining new bug patterns and evaluating existing bug-finding tools.
Incremental change and its application in software engineering courses (Education Papers posted on March 21st, 2013 )
In traditional software engineering courses, students develop small programs from scratch. Software engineering courses should prepare students for software industry careers and provide an experience that is close to the industrial practice. Nonetheless, most of the current courses offer projects where only small programs are developed from scratch, while in practice software engineers mostly work on evolution of large programs. In this thesis, we show how open-source software and expanded incremental change IC) can be used to narrow this gap without excessive student workloads or excessive instructor effort. We also show that standard tools like Concurrent Versioning Systems CVS) can be used to increase individual accountability of the project participants. The results of the course show that this type of software engineering course gives students a more realistic experience than traditional software engineering courses. In the survey at the end of the course, the students expressed a higher level of satisfaction with both rating the course and assessing how much they learned. Additionally, the resources required by such a course are not excessive.
Integral accessibility of digital resource centres for people with visual disabilities (Education Papers posted on March 18th, 2013 )
The relevance of accessibility is measured in terms of social, economical and personal welfare for all the agents involved in the definition, development, exploitation and use of Internet services and products. Some years ago the Web was an optional service from which some users could benefit, but nowadays it is necessary for many daily activities. In this new context, people with disabilities represent a large group of users with specific characteristics that have to be taken into account during the design of technological products. Moreover, their role is changing from users who are receptors of information to managers and administrators. It is in this new context where existing accessibility and usability guidelines do not cover the design needs for services like a Digital Resource Centre, if we want it to be managed by users with disability. Content managers have high interaction with the interface, which it is not usual in common commercial Web sites. They also manage a great amount of complex and interrelated information whose accessibility has not been previously studied in depth. This PhD describes accessibility guidelines for interface design of a Digital Resource Centre that can be extended to other contexts with similar interaction characteristics. These guidelines complement existing ones and are oriented to people with visual disabilities. This work also defines a digital resource type hierarchy, beyond HTML or Web content. That is why a set of metadata, or accessibility attributes, has been designed and attached to each resource type in the hierarchy. Their aim is to substitute the original digital resource in an accessible way, if it is not accessible for the user； provide some information when it is not possible to provide equivalent information in an accessible format； complement the information provided to improve the comprehension of the user with visual disability. The results and contributions of this thesis have been obtained working in close contact with the final users with visual disability, using a Web questionnaire, telephone and email contact and user testing. These methods are very valuable to know the real needs of this group of users, the way they interact with the interface and how they use technology to carry out different tasks.
Inferring cognitive learning styles in an e-learning environment (Education Papers posted on March 16th, 2013 )
Computer-aided instruction has been playing a crucial role in supporting learning. Early computer-aided instruction delivered a single style of content to all learners without any consideration of their learning styles. Recently, systems have been developed to adapt content based on the learners learning styles. These systems use instruments, such as questionnaires and interview, to infer the learning styles. Using such instruments costs learners extra time, and they have to be done explicitly. In addition, these systems do not adapt the learning styles of learners over time. These drawbacks are the problem addressed in this study. The purpose of this research was to infer the learning styles of students while they are browsing online instruction. This indicates that the inferred process can be done implicitly, in less time, and repeated over time. The focus of this study was on the three cognitive learning styles: holist, serialist, and versatile. In order to achieve this goal, a classification system was developed, which contains three online lessons and uses two mechanisms Tracking and Questions) to extract useful information about the users behaviors. The extracted features were used by a collection of classifiers to infer the users learning styles. These results were compared with those of the Study Preference Questionnaire by calculating the Pearson correlation between them. The major implication of this study is that the classification system developed for this study accurately infered the learning styles.
The politeness effect: Pedagogical agents and learning outcomes (Education Papers posted on March 12th, 2013 )
For over a decade, researchers have been investigating the potential of pedagogical agents to promote learning. Most of the research in this area has focused on addressing students’ cognitive needs at the expense of affective needs. Intelligent tutoring systems designed following this philosophy have been very effective compared to traditional methods of tutoring (Ritter, Anderson, Koedinger, & Corbett, 2007； VanLehn et al., 2005). However, in recent years, pedagogical agent researchers have begun to appreciate that ignoring student’s affective needs can potentially diminish the effect of cognitive tutorial strategies. To address students’ affective needs, some pedagogical agent researchers (Lester et al., 1997a； Johnson, Rickel, & Lester, 2000； Moreno, Mayer, Spires, & Lester, 2001) began to exploit Reeves and Nass’ (1996) Media Equation, which holds that users respond to interactive media as if they were social actors. Investigations following the Media Equation have tended to focus on the media used to realize the pedagogical agent, e.g., the use of animated talking heads and voices, and the results have been mixed. In my thesis, I focus instead on the manner in which a pedagogical agent communicates with students, i.e., the extent to which it exhibits social intelligence. I propose the hypothesis that pedagogical agents with socially intelligent strategies can affect learning outcomes. I investigated this hypothesis by studying the effect of politeness tactics in two different learning environments, across two different domains: industrial engineering and language learning. Results from these studies support this hypothesis, demonstrating the effect of socially intelligent tactics on learning outcomes. Comparison of results across studies helps to clarify our understanding of the processes whereby tactics employed by pedagogical agents can affect learning outcomes.
The central role of adaptation for curriculum enactment: Designing educational software for adaptation of curriculum using digital library resources (Education Papers posted on March 10th, 2013 )
Although educational digital libraries like the Digital Library for Earth System Education DLESE) offer a multitude of pedagogically relevant digital resources, educators still require support in reusing these resources in their classroom teaching. This research emphasizes the role of curriculum adaptation as central to enabling resource reuse and employs DLESE Teaching Boxes as examples of curricula supporting digital resource reuse. The questions guiding this research are as follows: 1) What adaptation processes and strategies do experts utilize in their planning for enactment i.e. classroom implementation) of curriculum? 2) How can computational models of curriculum be designed to support these adaptation processes and strategies? 3) How does the use of the Teaching Box Builder application influence or support educators adaptation processes and strategies? The outcomes of this work are a conceptual framework ENACT and the Teaching Box Builder application. ENACT ties expert educators adaptation processes with general design requirements. These general design requirements inform the specific design features of the Teaching Box Builder application which supports educators use and adaptation of Teaching Boxes. Teaching Box Builder utilizes the Federated Digital Object Repository Architecture FEDORA) to model Teaching Box components and their relationships. This research employs a two-phase methodology with each phase successively informing the design of Teaching Box Builder application design. The conceptual framework ENACT resulted from an analysis of a) frameworks describing digital library resource use and educators knowledge management practices and b) expert educators adaptation processes as describes in the literature review and observed in the Teaching Box workshops. The second phase included a verbal analysis which explored educators adaptation processes and which informed the design and implementation of the second version of Teaching Box Builder. In the evaluation study, educators completed adaptation tasks using the Teaching Box Builder while “thinking aloud”.
A methodology for design and development of an electronic learning (E-learning) network at the Regis University School for Professional Studies (Education Papers posted on March 7th, 2013 )
E-learning networks are used by academic institutions such as Regis University to support synchronous and asynchronous delivery of online courses and programs. An e-learning network supports a wide range of applications and services and facilitates access to diverse e-learning implementations including virtual laboratory V-lab) environments, knowledge repositories, storage area networks SANs), and content delivery networks CDNs), as well as metropolitan, regional, and international research and education configurations. As demonstrated in this dissertation, an e-learning network eliminates campus boundaries by providing infrastructure for information flows regionally, nationally, and internationally. A systems development life cycle SDLC) methodology enables academic institutions to design and develop e-learning networks that meet educational goals and objectives and provide access to current and next-generation research initiatives. The goal of the research was to assess the capabilities of the SDLC in enabling academic institutions to design and develop new e-learning networks and/or upgrade and enhance in-place e-learning networks. In this dissertation, the author applied the SDLC methodology for design and development of an e-learning network based on findings of a chronological case study of the e-learning Academic Research Network ARN) at Regis University between 2001 and 2005. The outcomes of this investigation demonstrate the role of the SDLC methodology as an enabler of design, development, and deployment of an e-learning network capable of meeting current and projected institutional goals and objectives.
A prosody-enriched dynamic text presentation technique for enhanced reading of electronic text (Education Papers posted on March 7th, 2013 )
Increasing amounts of textual information are generated, exchanged and interfaced electronically in our progression towards a paperless world. The human-level method of processing this information must progress in tandem to address the reading efficiency bottlenecks experienced when trying to traverse this growing information space. Dynamic text presentation techniques such as Rapid Serial Visual Presentation RSVP) offer the potential to induce high rates of reading through the temporal display of text, but at a significant cost of comprehension. This study explores the effectiveness of projecting prosody or the expressive properties of speech as visual effects on dynamic text as a means to sustain comprehension at higher-than-average rates of reading. The developed solution, a prosody-enriched dynamic text presentation technique, produces notable increases in comprehension in spite of low reading rates when compared to the traditional page format on electronic displays. Further, the majority of subjects that used the prosody-enriched technique indicated improved readability due to the visual projections of prosody on text. Keywords: dynamic text presentation, Rapid Serial Visual Presentation, prosody, reading efficiency, reading rate, comprehension.
A web-based system for course instruction and student advising (Education Papers posted on March 7th, 2013 )
Academic advising plays an important role in student success in college, freshman students who join the universities with certain goals may face difficulties in course selection, whom to contact, and where to get course information. Considering this, each student in the university is assigned an advisor from his/her department who can address students problems during their academic years in the college. The advisors spend considerable amount of time in analyzing student performance during the semester by viewing the students grades and transcripts. The rise in student enrollment in engineering majors at West Virginia University is making it increasingly difficult for the advisors to spend more time with their advisees. There is a need for an automated system which can provide information fast, accurate and available on the web so that the advisor can address students problems more efficiently and spend more time discussing the issues and less time on clerical. The focus of this project is to develop a web based system for course instruction and student advising for Bachelors Program in Industrial Engineering BSIE) at West Virginia University WVU). This could lay the ground work for expanding the system to college wide programs. The student advising sub system would help advisors in suggesting courses for next semester by automatically checking advisees completed courses and comparing them to the BSIE curriculum. The system would also help in verifying students eligibility for graduation by checking his/her completed academics to the graduation requirements. The course instruction sub system could help users in many ways. The instructors can post class material such as notes, home works, projects, and take home tests online. The instructor can post midterm grades online after every grade category in the semester. The user can provide access to the students registered for the class with a user id and password. The instructor can also keep track of past semesters grades. It has four different interfaces for four different user types namely, Department Administrator, Instructor, Teaching Assistant and Student. The department administrator user would have access to all methods available in the entire website. She/he would be able to access student and faculty information online. The department administrator can Add/Block/Delete any user in the database. The administrator can manage course catalog and assign courses to faculty. The department administrator will be able to update courses schedule, room/building, department, and college information. The faculty/instructor interface would allow the members to view student information and access course schedules information. It would provide specific information on advisee assigned to the faculty member. The faculty interface would also allow faculty members to update/upload course content to the web. They would also be able to manage class roster online. The teaching assistant user would have access to some of the methods designed for the instructor. This user can upload/update course content to the website. She/he can provide user access for students to the website. The teaching assistant can also upload course grades after every homework/project/tests. The student interface would provide the students with customized advice on which courses to take by referencing the most recent course information in the database. Students can view course material and grades online posted by the instructor. Students can get advice on courses to take next semester. It would provide information about the prerequisites to be taken for a specific course and also on selecting the approved number of General Education Curriculum GEC) courses from each objective.