This study explored assessment practices in online courses and the relationship between assessment and student learning strategies. Assessment practices are important in determining what kind of learning occurs in a course, and online learning environments by their very nature lend themselves towards different assessment practices. However, little is known about online assessment practices or their influence on student learning strategies. The study used a mixed-method, two-phase design. In Phase I, 60 online community college courses were sampled across disciplines. Instructor and student surveys and course observations were used to describe the status of course assessment practices and student learning strategies and explore which assessment practices relate to which learning strategies. In Phase II, follow up qualitative investigations of nine courses explored how certain assessment practices influence critical thinking strategies. The assessment practices in these online courses appear to match in many areas what is considered best practice in summative and formative assessment. Using multiple assessments and methods, grading student learning over time, and providing frequent and individualized feedback to students are beneficial practices. Both students and instructors seem to focus their efforts on elaboration, critical thinking, and self-regulation strategies, more than rehearsal and organization strategies. However, there are potential areas for improvement, such as using multiple assessors more effectively, adding only an appropriate number of assignments, and ensuring students utilize the frequent feedback they receive. Quantitative and qualitative data both indicated that discussions, written assignments, and papers were positively related to critical thinking strategy use whereas final/midterms and non-graded assignments were negatively related. Assignments that were successful in encouraging critical thinking had three areas in common: providing explicit intent to promote critical thinking, allowing time for reflective thinking, and using appropriate instructor guidance. In online discussions that promoted critical thinking, instructor postings were less frequent, more neutral, and used provoking questions. Knowledge retention multiple-choice questions were least useful for understanding student learning and played a formative instead of summative role. This study suggests that assessment methods are most important in determining the type of learning occurring and grading opportunities are ideal teachable moments in online courses.
Tag Archive: Technology
Course assessment practices and student learning strategies in online college courses (Education Papers posted on March 27th, 2013 )
The impact of teacher self efficacy and attitudes toward classroom computers(s) on the use of classroom technology (Education Papers posted on March 27th, 2013 )
This study focused on examining the integration of computers) into classrooms of schools in an urban school district. Teachers in the district had access to a laptop or desktop computers) for use in their classrooms. Teachers also had access to remote Internet access dial up) and web-based professional development at home or at school. Three instruments were used in this study: Teachers Computer Attitudes, developed specifically for this study, Teachers Sense of Efficacy Scale-Short Form Tschannen-Moran & Wollfolk, Hoy), and a demographic survey developed specifically for this study. Results of this study provided support that teacher efficacy was significantly related to teachers attitudes toward computers) for best practices, computer usefulness, and computer efficacy. The negative relationships between teacher efficacy and computer anxiety support the contention that teachers who had higher levels of confidence were less anxious about using computers) in their teaching practice. Teachers who lacked sufficient confidence in using computers) in their work were likely to avoid them altogether. The positive relationships between teacher efficacy and computer efficacy support the lack of anxiety. Perhaps computers) have been common in the classroom and teachers may be receiving sufficient training to encourage them to be used effectively. The teachers in the present study indicated that they used computers) for a variety of classroom tasks e.g., lesson planning, drill and practice, grading, education, classroom management and presentations) and were familiar with sever types of productivity software word processing, grade books, educational games, presentations, etc.). Teachers who recognize the value of computer use in their classroom are more likely to have higher levels of teacher efficacy. Teachers attitudes toward computers) were not related to their self-reported skills with computers). Although this question had not been addressed in previous literature, teachers who self-reported their skills as excellent or good were thought to have more positive attitudes toward the use of computers) in their classrooms. An interesting finding was the negative relationship between teachers person use of computers) and computer anxiety. Teachers who expressed higher levels of anxiety were less likely to use computers) for personal use.
The impact of individual learner characteristics and synchronous computer-mediated communication on language production in learners of English (Education Papers posted on March 27th, 2013 )
As computer technology advances rapidly in the past decades, incorporating computer technology into classroom teaching has been popular, especially in the area of foreign language education because of its potential benefits. Among computer technology, computer mediated communication henceforth, CMC) is widely used in language classrooms for its benefits, for example, more equal participation, a less stressful learning environment, and increased output Chun, 1998； Kern, 1995； Warschauer, 1996). In addition, due to the hybrid nature of CMC, learners are allowed to have more time to process input, monitor and edit output Kelm, 1992； Warschauer, 1996) resulting in more accurate and complex language production compared to face-to-face conversation. Beauvois, 1992, 1995； Kern, 1995； Pellettieri, 2000； Sotillo, 2000； Warschauer, 1995, 1996). Several studies examined another benefits of CMC – the development of oral proficiency Abrams, 2003； Beauvois, 1998b； Payne & Whitney, 2002； Kost, 2004)； however, the results of these studies are consistent, suggesting that oral proficiency may be enhanced by synchronous online discussion. Instead of investigating whether synchronous CMC is beneficial for all students, the present study aims to investigate the extent to which the factors such as personality types, amount of participation and previous experience using chat contribute most to the development of oral skills. That is, the present study focuses on individual, affective variables that could affect the effects of CMC on spoken language development among beginning learners of English. Employing a pre and post one-group comparison design, 16 students participated in the study and their oral skills were measure by two interviews and the interviews were scored by a native speaker informant and the researcher based on the scoring rubric that was adopted from the previous study Payne & Whitney, 2002). The results of repeated measures ANOVA revealed none of factors identified in the study were statistically significant. The results, however, should be interpreted that the use of CMC in language classroom does not disservice any group of students with a specific personality type or previous experience using chat. With numerous benefits known, chatting remains a valuable pedagogical tool to promote second language learning.
Teacher participation and feedback styles during classroom synchronous computer-mediated communication in intermediate German: A multiple case study (Education Papers posted on March 27th, 2013 )
This mixed design multiple case study of learners interactions explores the effects of teacher participation during third semester German in-class chatting activities. Three third-semester German courses taught by two different teachers were investigated over the course of one semester, during which the class members were asked to chat for 20 minutes per week using activities design by the researcher and adapted from the textbook. Multiple data sets were collected: teachers participation styles and feedback moves； students language learning achievement levels； students attitudes towards corrective feedback and technology； their experience with feedback and technology； and evidence in chat transcripts of errors, uptake, and error uptake. Students were administered a pre- and post-instruction achievement test on the structures taught during third semester German. In addition, they were surveyed at the beginning and the end of the semester on their attitudes and experiences with feedback and technology in the foreign language classroom. Furthermore, chat transcripts were analyzed to identify errors, corrective feedback, teacher moves, uptake, error uptake, student and teacher word count and words per minute, error rate, and target language use. In order to better understand the context of the transcripts, classroom observations were conducted once a month, and students completed a self-report form after each chat session. Informal conversations with the teachers provided additional insights. It was found that the students overwhelmingly appreciated teacher involvement and feedback, and that they saw chatting as both fun and beneficial for language learning. The corrective feedback rate was generally low, as were rates of uptake and error uptake. The two teachers were found to have different interaction and feedback styles. Furthermore, the three classes operated with differing levels of technical support during the lab sessions, which did not appear to influence the students experiences except for the amount of teacher output. Six case study subjects, namely the two students from each class who contributed the most to chat sessions, were selected for an in-depth analysis of their chat transcripts.
The impact of a multi-user virtual environment on teacher instructional time, voluntary student writing practice, and student writing achievement (Education Papers posted on March 26th, 2013 )
Two major obstacles to using PBL methods in K-12 classrooms are the time it takes to design the rich learning environment and the time required for students to interact at their own pace with ill-structured problems. The focus of this study was to determine whether game-design design principles can be used to both compliment a digital PBL environment and improve student learning. Further, this study sought to determine whether such a design could allow teachers to act as a challenger of poorly developed knowledge constructs instead of as a font of directional and procedural knowledge for students. To answer these questions a digital learning environment was designed that used embedded scaffolds, nested goals, clue trails, narrative context, and explicit rules to improve student writing. This unit was part of a larger multi-user virtual environment, but was designed to be a self-contained unit that leveraged advanced technologies to establish an immersive experience for learning writing skills. The unit was designed to be two-times per week for four weeks in total length which included student training on the active role of a reporter who investigated mysteries taking place in a virtual town. The learner then composed feature stories relating their understanding of the mystery. A comparison class was recruited and the teacher was observed teaching the same content and skill standards but through more didactic methods of instruction. The results of this study showed that the treatment condition had decreases in teacher time spent answering procedural and directional questions, increases in the amount of voluntary student writing activity, and improvements in standardized achievement scores on prompts that consisted of writing tasks similar to those that students participated in during the treatment. Students engaged fully with the learning environment although several tensions emerged. These included tensions between student perceptions of teacher rules versus system rules, student play versus completion of learning tasks, and whether they should learn through the system by reading versus being told what to do. Student disabilities were also encountered during the study which placed the system under a different kind of test than it was designed for, though it successfully engaged these students as well. A final tension arose in the result of the research methods themselves, bringing home the point that a need to capture data may interfere with the learners experience, possibly reducing or improving the impact of the treatment itself.
Administrators’ and teachers’ perceptions about surveillance technology (Education Papers posted on March 26th, 2013 )
Many schools have installed surveillance cameras, but little is known about the impact on the school environment as a result of the installation of surveillance technology. The purpose of this study was to investigate the perceptions of teachers and school administrators regarding surveillance technology as a useful tool in the educational setting. This study may inform policymakers and school leaders as they strive to achieve school goals, allocate resources, and carry out their jobs as described by law. Lastly, this research may provide more detailed information to Leadership Preparation Programs. The research questions that guided this study were: 1) What are the teachers perceptions of the impact of surveillance cameras on the school? 2) What are the school leaders perceptions of the impact of surveillance cameras on the school? 3) Is there a difference in the perceptions of teachers and school leaders of the impact of surveillance cameras on the school? The study found surveillance cameras were welcomed by the schools communities. The study has also shown that the cameras can be used as an additional tool for school administrators to maintain order inside and outside of their schools. The cameras have not only decreased discipline problems, but they have also been a deterrent to those who wish to cause problems. An additional finding was that teachers and administrators do not change their behaviors while in view of cameras and do not feel their right to privacy has been violated with the installation of cameras in the school.
Electronic government accessibility for people with blindness or low vision who utilize assistive computer technology (Education Papers posted on March 25th, 2013 )
Citizen access to electronic government information and services continues to enjoy an expansionary phase in local government. This expansionary phase holds a prominent place in service delivery strategic planning as governments address on-going operational challenges caused by increased fiscal pressures and greater accountability to the citizenry for their actions. Since the advent of the World Wide Web, in the mid 1990s, static information and interactive applications are available on government websites to facilitate information dissemination and citizen interaction. The presence of electronically delivered information and services may not address the accessibility needs of people with blindness or low vision who utilize assistive computer technology. Inaccessible websites can occur when accessibility-oriented development is absent from local governments web design process. Minimal research has been conducted to discover potential barriers preventing people with blindness or low vision who utilize assistive computer technology from accessing electronic delivery of government information and services through an official government website. This study consists of a cross-sectional survey of 472 local governments in a Midwestern state. Fifty-six local governments responded to the survey. The results indicate that 100% of respondents have accessibility design errors programmed into their official home pages. A key factor related to the number of accessibility design errors present on home pages is the work experience of a web developer. Additionally, subordinate web pages, linked to an official home page, demonstrate a similar number of instances of accessibility design errors as an official home page. This finding suggests errors are prevalent throughout entire websites. Finally, the presence of a formalized web development process is also a key factor related to the number of accessibility design errors present on home pages.
Algebraic functions in a graphing technology environment: Student performance after small group work using teacher-generated versus student-generated examples (Education Papers posted on March 25th, 2013 )
In this study I investigated the effects that two different levels of written guidance had on the performance of college level Precalculus students within the contextual environment of Graphing Calculators merged with Inquiry learning. There were two experimental groups and one control group. The control group took only the pretest and the posttest. The experimental groups were divided between High Structured HS) guidance, which consisted of teacher-generated examples and Low Structured LS) guidance, which consisted of student-generated examples. Based on Pretest scores, students were coded as High Prior Knowledge HPK) or Low Prior Knowledge LPK). Students were not allowed to have a calculator on either the Pretest or the Posttest； however, they used a calculator during the instructional intervention, which involved students working in small heterogeneous groups using the graphing calculators and examples theirs or the teachers) to explore the relationships between graphical and algebraic representations of parabolic functions. All students significantly and meaningfully improved their scores between the Pretest and the Posttest, but there was no evidence to support one structure over the other. However, on the Retention test the HPK students in the Low Structured groups significantly and meaningfully outperformed the HPK in the High Structured groups. My intervention included a group Quiz. The Quiz took place before any class discussion and before the Posttest. The mean Quiz score for the LS group was meaningfully and significantly higher then the mean Quiz score for the HS groups. There were no results favoring High Structure over Low structure on any measure.
Affordances of three-dimensional virtual environments for English language learning: An ecological psychological analysis (Education Papers posted on March 25th, 2013 )
The goal of this dissertation is to systematically explore teaching and learning English as a foreign language in a 3D game-like multi-user virtual learning environment MUVE), Quest Atlantis QA). This dissertation project is comprised of two studies, offering distinct theoretical perspectives and analytic data techniques for measuring and describing the phenomenon of English language learning in the virtual world. These studies, however, share the same meta-theoretical framework of learning by doing Dewey, 1910), in that thinking and the use of language are in service to action, not just about action. Study 1—Attitude and Self-Efficacy Change—used quasi-experimental design and quantitative analytic tools to measure the differences of Controls and a QA groups affective factors and English achievement test during a one academic years intervention, during which QA participants explored English-only QA worlds, completed content driven quests, and chatted with native English speakers. The findings, that the experimental group reported a more positive attitude and higher self-efficacy than the control group toward various aspects of language learning and use, are significantly important for both educational research and classroom practice. Failure to find differences on standardized post performance tests of English writing raises issues concerning the trade-offs between test preparation and engaging activities that may sustain learning. Study 2—A Cross-Cultural Interaction—took a finer grain analysis approach to look at the chat logs and other QA-generated artifacts and examines how cross-cultural interaction in a virtual space, specifically collaboration between NES and NNES in QA, provided resources for English language acquisition. Qualitative tools including the CHILDES CLAN program, Computer-Mediated Discourse Analysis and ethnography were used to examine the smallest unit of analysis, perception and action. Iterative multi-layered analyses revealed the affordances of QA at different levels. Co-questing was particularly interpretable based on the assumptions and underlying design intentions of ecological psychology and sociocultural cognition. Co-questing afforded a higher level of meaning making enacted in collaboration through the chat channel. In other words, this higher level of Negotiation for Meaning appeared to be the result of goal directed behavior within the social and cultural boundaries of the QA situation.
Universalizing universal design: Applying text-to-speech technology to English language learners’ process writing (Education Papers posted on March 25th, 2013 )
This paper presents findings from six case studies of English language learners (ELLs) applying text to speech (TTS) technology to the process of drafting and revising essays. The participants were all seniors at an urban public high school in Massachusetts where they were required to write quarterly expository essays in all major subjects. ELLs’ numbers in public schools are burgeoning, and even those who were skilled writers in their native countries face many barriers to writing. TTS is a technology designed to assist individuals with a need for audible input of computer text. It is built on the philosophy of Universal Design for Learning, which aims to make the technology usable by the broadest possible range of learners. The purpose of this study was to learn how TTS could support ELLs’ performing process writing on computers. Data was collected on the participants’ writing processes without and with TTS using questionnaires, documents, interviews, and observations. The findings suggested that when they used TTS, the participants wrote more drafts, spent more time on each draft, detected more errors, and increased revision of meaning-level features. At the same time, they struggled to express themselves in expository essays, having learned other ways of writing in their native countries. Students’ language learning goals also affected how they used the technology, because they also looked for ways to use TTS to improve their fluency and perfect their speech. Results suggest that ELLs must have sufficient time to acquire English proficiency as they meet academic requirements, and during this acquisition period must have latitude to write in culturally familiar ways. The audible input from TTS did help them conceptualize and revise their writing. Given direct instruction in how to apply it to process writing, ELLs could benefit even more from TTS input. However, to make the most of the universal design of TTS, teachers and others should explore letting ELLs use it to meet their individual goals. Exploring English language learners’ use of TTS software helps illustrate how individual learning processes must be considered when applying technology to diverse populations.