Challenges+of+Online+Assessment

Challenges of Assessing Online and Collaborative Learning

Authentic Assessment versus Traditional Assessment:

Some educators believe that assessment is a key element in the overall quality of teaching and learning. Educators that have well designed assessments give clear expectations, reasonable workload and provide students the opportunity to receive constructive feedback and self monitor their learning. Assessment should be treated as an essential part of the entire teaching and learning process, especially in the online environment. Instruction and assessment should be project based, authentic, and individualized in order to prepare students for learning and working in the 21st Century. Draves (2007) describes how assessment via online learning and the use of the Internet will change the nature of tests and quizzes, stating that assessment will “enhance learning and contribute positively to the learning experience” (p 115). Assessment in the 21st Century needs to measure not only core subject skills, but also learning, thinking and life skills. Standardized testing can measure some of these important skills but the other assessments will need to come from formative and summative assessments. Testing alone will not generate enough evidence of the skills sets that business and industry require of future employees in order to be competitive in the global economy. Useful, and meaningful feedback to students will help them work on their ability to come to understand that successful learning is as much about the process as it is about facts and figures. Additional support in offering authentic assessment as a tool of evaluation includes using alternative assessments to evaluate students with special needs, including those with disabilities, language or cultural barriers, gender and age differences (Draves, 2007). Authentic assessment requires the students to utilize their acquired knowledge to complete tasks or projects, taking into consideration all of the small components that are required to acquire knowledge of the task and sub-tasks. The workforce of the 21st Century will be employed based upon their ability to utilize their acquired knowledge and performance based skills will be essential.

Tests, quizzes, homework and the like are all traditional forms of assessing a student’s ability to learn and repeat information. Some educators are changing their belief that 21st Century assessment needs to focus on the process of learning and not the end result of memorizing and regurgitating information. Draves (2007) predicts that teachers and students will develop individual learning plans that will allow the student to demonstrate what knowledge skills have been acquired and what skills still need to be learned. Grading the work of the student rather than just the answer is an important part of this assessment plan. Authentic assessment takes a comprehensive look at the student, focusing on the long term learning of the student. Some believe that as a society, educators, administrators and governments, are heading in the wrong direction when it comes to high stakes testing. This form of testing only looks at math, reading, grammar, and science when there is so much more to education. It is also a very black and white form of testing where the student either gets all or nothing. This is a contradiction of 21st Century learning and authentic assessment, because the work place and schools are looking at collaboration between students and peers; they are looking for the process, whether it be creative, innovative, problem solving, or just unique. Trends in education continue to sway between authentic assessment and the more traditional forms of assessment. Understanding the differences between these two forms will help in implementing an effective form of assessment in online learning and collaborative work.

While traditional forms of assessment rely on tests and quizzes, requiring students to recall memorized information, authentic assessment requires the students to demonstrate understanding of concepts through more intellectual based tasks. Even in open-ended questions, students are not given the opportunity to create polished, justified answers because they are expected to write a correct response to the question. Standardized testing allows for one right answer, rather than providing the students an opportunity to present alternative possibilities to the problem. Wiggins (1990), states that “secretive tests composed of proxy items and scores that have no obvious meaning or usefulness undermine teachers' ability to improve instruction and students' ability to improve their performance” (para. 4). While tests can be a valid indicator of learning and are a demonstration of norms achieved through the right answer, they do not allow students to demonstrate an understanding of real problems and solutions. Teachers find the tests to be an after-thought to their lessons and students find tests to be irrelevant to their learning, and feel that the right answer matters more than the approach and results to a given problem.

In “a move toward more authentic tasks and outcomes thus improves teaching and learning: students have greater clarity about their obligations (and are asked to master more engaging tasks), and teachers can come to believe that assessment results are both meaningful and useful for improving instruction” (Wiggins, 1990, para. 8). Edelstien and Edwards (2002) describe assessment as the “procedure of observing learning” (para. 5). The idea of assessment, includes the use of describing, collecting, recording, scoring, and interpreting information about the student’s learning. In the 21st Century, in situations and job positions, accountability is not based off of a multiple choice, true or false, open-ended question, or a simple solution test. Current job assessment is based on performance evaluations that include continued engagement in training, project development and success, working collaboratively with others, increased abilities and the impact on the overall success of the company. Just as the adult workforce relies on authentic forms of assessment, education, including the K-12 and Higher Education institutions, have a need to look at alternative forms of assessment of student learning, mastery of skills, and meeting state and common core educational standards.

Assessing Collaborative Work:

Due to the increased need for collaboration in the workplace, classroom activities and authentic assessments are taking on the inclusion of collaborative work. Draves (2007) states that as educators “we already know that younger students often learn better collaboratively” (p. 121) which promotes and supports the rationale to include group projects as a tool for evaluating student progress and understanding. Online group work can pose several problems, including the need for students to meet face-to-face, assessing how the group functioned as a whole, the project outcome, and assessing individuals who may or may not have performed to the same level as the group. In 21st Century learning, there is no single method for assessment that can be considered ideal, therefore, educators should consider several methods of evaluation throughout a course. Evaluation of student learning, and collaborative work, may include the use of threaded discussions, demonstration of ability to apply knowledge in a real-life setting, relevant to the student and the learning community, and a continuing interaction amongst the group (Edelstien & Edwards, 2002).

Do’s of Assessment (Frey & Wojnar, n.d., Palloff & Pratt, 2005 & 2007, Wiggins, 1990):
 * Start small, use the examples created by others, or choose one activity to start with
 * Develop clear rubrics with standards and expectations
 * Be explicit from the start on what will be assessed and how
 * Allow for extra time, to develop and to feel comfortable using the new assessment
 * Plan assessment as part of the curriculum, it should not be an afterthought
 * Work together, collaborating with a peer will help you develop ideas
 * Create a collection of assessments to provide a variety based on the needs of the students and the curriculum
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Reflect on the assessment and activities, be willing to modify them for the next time
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Design learner-centered assessments that require self-reflection
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Include collaborative based assessments
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Use assessments techniques that fit the assignment and learning objectives
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Ask for student input and ideas
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Assess collaborative work through collaborative assessments
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Encourage self-assessment
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Model and encourage good feedback
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Provide end-of-course evaluations of synchronous / asynchronous discussions
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Evaluate discussions based on their purpose and outcomes
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Evaluate both qualitative and quantitative aspects of online discussions

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Don’ts of Assessment (Wiggins, 1990, Draves, 2007):
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Don’t give up, remember that “practice makes perfect” and even then you will need to be willing to revise your original plan
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Don’t rely on one form of assessment for every activity, provide a variety of assessments throughout the coursework
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Don’t rely on attendance as a form of assessment – it is difficult to measure and is irrelevant to learning
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Don’t post grades in a public forum

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Strategies to Assess Online and Collaborative Learning:

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Rubrics are a valuable tool in evaluating students as individuals and in collaborative work. Reviewing and revisiting the same concepts from different viewpoints improves the understanding of a lesson for students, therefore, teachers should create a rubric that can be slightly modified and applied to a variety of activities. While most requirements to projects, including the writing conventions, grammar, and spelling remain constant, other areas specific to the topic may change. The Teacher Vision website found below will guide teachers through designing, revising and implementing rubrics in any classroom and for any subject. Rubrics can also be used in assessing collaborative work and should include performance measures for the individual’s work, as well as the result of the group work. Palloff and Pratt (2007) describe the use of rubrics as a means for the student to complete a self-evaluation as well as for the instructor to assess student performance. They provide a simple guideline that helps establish a more substantive rubric later on, breaking the assessment down into simple standards and letter grades: A (synthesis level), B (analytic level), and C (summary level) (Palloff & Pratt, 2007, p. 210). Examples and resources of rubrics for evaluating learning can be found below. Each site is valuable in its own way. The Authentic Assessment Toolbox provides examples of rubrics as well as suggestions for other means to evaluate student learning. UW-Stout provides samples of rubrics that have been designed and used with students of all ages, subjects, and grade levels. The third link, Teacher Vision, provides a five-part series in creating, evaluating, revising and implementing the use of rubrics in the classroom, both online and face-to-face.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Rubric Based Tools for Assessment
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Authentic Assessment Toolbox] (with examples)
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|UW-Stout Rubrics for Assessment]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Teacher Vision Creating Rubrics] (Five-part series)

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Engaging students in evaluation is another way to support collaborative learning and assessment. Draves (2007) cites assessment ideas from Conrad and Donaldson (2004) including the use of online discussions, team projects, and self-assessments. The concept of engaging students in the evaluation process supports student learning, contributes to overall collaboration, allows for a different perspective of the learning processes from both the educator and the students, as well as allows the educator more time to assist learners rather than assess the learners. Encouraging students to contribute to the learning environment and then assessing them based on their learning may include ideas suggested by (Benfield, 2002). For example, students may select a number of contributions to the online discussion forum for assessment, creating essentially a portfolio of artifacts of learning outcomes. Additional ideas include a collaborative paper, multi-media project, annotated bibliography, group projects and problem-solving reports (Benfield, 2002). A similar concept to Benfield’s (2002) idea of students collaborating and being evaluated through authentic assessment is described by Alley and Jansak (2001) in the article, //Ten Keys to Quality Assurance and Assessment in Online Learning//. They describe how students develop a “webliography” (Alley & Jansak, 2001) where the educator assigns a topic and allows the students to construct their own knowledge through a series of gather information, sorting out and evaluating information and producing a final product that best demonstrates an understanding of the students’ learning related to the provided topic. This type of activity also allows the instructor to evaluate the students throughout the process, creating a more valuable and meaningful evaluation process.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Asynchronous discussions provide an opportunity for students to work collaboratively in an online environment while not having to worry about location or time constraints. They also offer an opportunity for the educator to evaluate learning through measures of authentic assessment. Vonderwell, Liang and Alderman (2007), suggest that educators observe the following guidelines when evaluating asynchronous discussions: <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Using discussion threads, educators are able to assess student learning and progress through the course, rather than at the end, while providing a variety of assessment methods and strategies will encourage student learning, interest, engagement and participation in the discussion forums (Vonderwell et al, 2007). By simply requiring students to participate in asynchronous, or synchronous, discussions the desired outcomes may not be reached, just as only evaluating students participation in the online environment may not be the only way to assess student learning. Other research by Balaji and Chakrabarti (2010) argues that “students learn faster and more effectively” (p. 6) when student participation is linked to assessment. Supporting their belief with examples from Palloff and Pratt (2005) including their description of the importance of community and fostering connectedness amongst the learners. However, Palloff and Pratt (2007) describe a method of evaluation that is based on an average of student assignments, exams and participation, as well as a series of rubrics to assess student performance. Therefore, it is suggested that online discussions are not merely graded on participation but the level of participation and the impact it has on the entire group.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Mastery of subject matter
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Application of ideas and concepts
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Clearly written with details
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Considerate of others’ ideas and opinions
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Provides reflective feedback by asking additional questions or providing useful information or additional suggestions (p. 313)

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Assessment in an online or face-to-face environment should consider assessment as a process as well as an outcome. It should not be what drives a course, nor should it be what comes at the end. Assessment as a process encourages online learners to facilitate self and peer-assessment, promotes learner autonomy and self-reflection and maximizes student learning. “The drive for new types of assessment, new requirements for evaluation, and new ways of assessing learning and knowledge, is helping to change the entire way we build and teach courses, both online and face-to-face” (Draves, 2007, p. 130). In order to promote student learning in an online environment, assignments and assessments should be varied, promote critical thinking skills, encourage collaboration, are designed to be shared with others, and promote a sense of responsibility to the whole group (Palloff & Pratt, 2007). However, not all courses, or schools promote the use of collaborative work and assessment to evaluate student learning. As Pallof and Pratt (2005) describe, when working alone, students can develop a sense of isolation and lack of involvement in the learning process. Although collaboration requires the presence of individuals, it can have its challenges and frustrations for both the instructor and the students. Assessing the level of participation and the presence was addressed throughout this report. Constructivist theorists argue that collaboration is essential to the learning environment and the level of understanding achieved by the individuals. When one person tries to solve a problem fewer possible solutions or remedies can be found, while a group of individuals may find many ways to solve the same problem. Do the frustrations of participation and presence outweigh the opportunity for individual growth? Does the evaluation of one person's assignment give the instructor a better sense of student learning than a group project? A constructivist would argue no. A student who has been alone in a very long, difficult process of obtaining a degree would argue no. When the Educational Technology Department was asked to evaluate the program, syllabi and coursework for possible updates, it took an entire team of five assistant professors who currently teach in the program to make the necessary modifications. Working as a team made the program better, and it continues to only improve. As Palloff and Pratt (2005) state "the online environment can be a lonely place" (p. 7). This student is tired of being lonely. The only reasons this student is continuing to follow through with the program is money and time. There is not enough money to quit this program and begin another, that won't accept more than 6 out of the 21 credits earned, there is not enough time to "start over", and although multiple residency requirements in another state or even out of the country sounds fabulous, there isn't enough money or time for that either. Too bad one can't jump in a time machine and go back to 2010 when a decision was being made. This student would have chosen a different path, one that led to a more collaborative, student-led, meaningful learning experience.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Resources to Assess Online and Collaborative Learning:

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">As listed earlier, rubrics are an excellent way to assess student learning, particularly of authentic learning activities. The following list contains websites to additional web-based tools that can be used in assessing learning, from the more traditional, test or quiz based assessment, to the more collaborative, creative and authentic forms of assessment.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Web-based Tools for Survey and Assessment Activities
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|SurveyMonkey]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|ProfilerPro]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Zoomerang]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Questionmark]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Quia!]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Quizstar]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Quiz Center]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Hot Potatoes]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|My Gradebook]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Quizlet]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|PollEverywhere]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Glogster]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|VoiceThread]
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">[|Bubbl.us]

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">References:

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Alley, L.R. & Jansak, K.E. (2001, Winter). The ten keys to quality assurance and assessment in online learning. //Journal of Interactive Instruction Development, (13)3, 3-18.// Retrieved from ERIC.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Balaji, M.S. & Chakrabarti, D. (2010, Spring). Student interactions in online discussion forum: Empirical research from ‘media richness theory’ perspective. //Journal of Interactive Online Learning, 9(1), 1-22.// Retrieved from ERIC.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Benfield, G. (2002, June 27). Designing and managing effective online discussions. //Oxford Centre for Staff and Learning Development Learning and Teaching Briefing Papers Series.// Retrieved from http://www.brookes.ac.uk/services/ocsd/2_learntch/briefing_papers/online_discussions.pdf

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Draves, W. A. (Ed). (2007). //Advanced teaching online//. River Falls, WI: LERN Books.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Edelstein, S. & Edwards, J. (2002, April). If you build it, they will come: Building learning communities through threaded discussions. //eLearn Magazine//. Retrieved February 18, 2012 from http://elearnmag.acm.org/featured.cfm?aid=566829

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Frey, B.A., & Wojnar, L.C. (n.d.). //Successful synchronous and asynchronous discussions: Plan, implement & evaluate.// Retrieved from http://net.educause.edu/ir/library/pdf/MAC0426.pdf

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Palloff, R.M. & Pratt, K. (2005). //Collaborating online: Learning together in community.// San Francisco, CA: Jossey-Bass.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Palloff, R.M. & Pratt, K. (Ed.) (2007). //Building online learning communities: Effective strategies for the virtual classroom.// San Francisco, CA: Jossey-Bass.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Vonderwell, S., Liang, X., & Alderman, K. (2007, Spring). Asynchronous discussions and assessment in online learning. //Journal of Research on Technology in Education, 39(3), 309-328.// Retrieved from ProQuest.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 110%;">Wiggins, Grant (1990). The case for authentic assessment. //Practical Assessment, Research & Evaluation//, 2(2). Retrieved February 17, 2012 from http://PAREonline.net/getvn.asp?v=2&n=2