Skip to content

Assessing Collaborative Efforts

January 12, 2010

 “How do we know they know?” is the fundamental question that determines the nature, scope, and value of learner assessment (Paloff & Pratt, 2009, p. 3). Assessment is central to defining and measuring the degree to which a learner achieves stated learning objectives. Instruction that includes appropriate assessment-based methodology utilizes constructive alignment, which as a fundamental principle of education, requires a good teaching system to bring teaching method and assessment into alignment so that all aspects of the system are in accord in supporting appropriate student learning (Biggs, 2003). (Here’s a rubric on developing an online course as defined by the California State University)

Assessment demands the alignment of teaching method and assessment to learning activities and collaboration is no different. Assessing participation in a collaborative community must include the following:

  1. a well-defined  collaborative objective that includes a clearly articulated rationale
  2. a distinct system to assess collaboration that includes a clearly articulated rationale

In #1, the instructor describes what is to be achieved and why. In #2, the instructor explains how he/she will perform the assessment and why he/she chose that method of performance. Paloff and Pratt (2009) indicated that, as a rule, collaborative activities are best assessed collaboratively. By requesting students submit self-evaluations of their individual activities and peer assessment, instructors could achieve this requirement. This is in agreement with Swan (2004) who identified four distinct interactions for learning effectiveness in online environments, interactions with content, instructors, classmates, and course interfaces. These interactions help develop divergent thinking where team members are not unanimous in their thought processes as much as they are harmonious. They also strengthen ongoing support for instructors, provide timely feedback, automate testing, and ongoing course development. The virtual university design and technology division of Michigan State University (MSU) has a well-defined system to develop online and peer assessment. MSU’s Faculty and Organization Development site includes links to several online instructional resources.

 Paloff and Pratt (2009) identified several principles that should guide student assessment in an online course in reference to collaboration. The pertinent ones include:

  1. Design and include grading rubrics for the assessment of collaboration
  2. Include collaborative assessments through public posting of papers that accompany comments from student to student
  3. Using assessment techniques that fit the context and align with learning objectives
  4. Asking students to incorporate their input into the method for conducting assessment

Using these four principles, an instructor could assess a student’s participation in a collaborative learning community. Since collaboration is the effective interactivity based on well-defined goal of a learning community, the participants are the best judges of each other’s activity in the progress toward a collaborative objective. The instructor may need to create a rubric that includes the evaluation of participants’ areas of strengths. Palloff and Pratt (2007) suggested that the human side of online learning appeals to the need for “connectedness, and coalescence” (p. 50). By involving the learners in the assessment process, expresses the conviction that students can be experts in areas of their own learning and that promotion of self-direction is important (Paloff & Pratt, 2009). Hurst and Thomas (2008) presented their experience with online team training and building skills in action and identified the accomplishing of a team project as the common goal toward which online education administrators need to strive.

Principles #1 and #2 above address the acceptance of students’ comments on each other’s work in thereby demonstrating, to a degree, the instructor’s attempt at “fair and equitable assessment” based on well-defined grading rubrics (See article by Swan, Shen, and Hiltz for sample rubrics for assessing collaborative work). Macdonald’s work on the assessment of online collaborative learning as a process and product highlights the importance of assessment in ensuring online participation (You will find the abstract to Macdonald’s article here ). Furthermore, by linking theoretical discussions to one’s own experience can give opportunities for students with particular skills and knowledge to contribute to achieving collaborative goals. (See Bobby Elliot’s article “Online Collaborative Assessment”  

I think that incorporating principles #4 and #5 above will further enhance the students’ perspective of the instructor’s intent to be “fair and equitable” in assessing their work. Arnold, Ducate, Lomicka, and Lord (2009) in a study of computer-supported collaborative learning among foreign language graduate students determined that success in interaction depends largely on the group members themselves. Consequently, it is incumbent upon the members of the collaborative community to encourage one another to participate and hold each one accountable.

Anne Marie highlighted a few questions that address collaborative learning. It is interesting to note that regardless of the field, students and educators find online collaboration hard. She opined as to why evaluators are concerned about assessing online collaboration when instructors are happy for students to work in small groups in a tutorial without assessing relative contributions. Anthony Cocciolo included an excellent diagram to visualize the components of a community network which he noted defines how one analyzes and assesses in terms of member interaction which he referred to as “sociality.” As Siemens (2009) suggested it is the intentional creation of connections between the participants that leads to effective collaboration and mutual accomplishment of clearly defined goals. Consequently, it may be important to focus on the rationale for creating the connections between members of a learning community in the real world and in its virtual counterpart rather than on achieving collaborative steady state.

References

Arnold, N., Ducate, L., Lomicka, L., & Lord, G. (2009). Assessing online collaboration among language teachers: A cross-institutional case study. Journal of Interactive Online Learning, 8(2), 121-139.

Biggs, J. (2003). Teaching for quality learning at university (2nd ed.). Buckingham: SRHE and the Open University Press.

Hurst, D. & Thomas, J. (2008). Developing team skills and accomplishing team projects online in Anderson, T. (Ed.). The theory and practice of online learning, (2nd ed.). Edmonton, AB: Athabasca University Press.

Palloff, R. M. & Pratt, K. (2007). Building online learning communities: Effective strategies for the virtual classroom, (2nd ed.). San Francisco: Jossey-Bass.

Paloff, R. M. & Pratt, K. (2009). Assessing the online learner: Resources and strategies for faculty. San Francisco, CA: Jossey-Bass.

Siemens, G. (2009). Learning Communities. Resource vodcast in Principles of Distance Education (EDUC 8842, Winter 2009), Walden University, Laureate Education.

 Swan, K. (2004). Relationships between interactions and learning in online environments. The Sloan Consortium.

Advertisements
7 Comments leave one →
  1. Andy Goswick permalink
    January 13, 2010 2:46 pm

    Why do you think we find online collaboration hard? Is there an underlying factor in the online setup or do you think it’s more of “us” not use to the non-social interaction? I find it a little harder because I’m not sure exactly what is expected of me therefore I get easily upset at myself and the class when I think something is right but it isn’t. What about online collaboration but with a video link to where you can “see and watch” the professor and/or other students if accessible? Do you think this would alleviate the pressure we’re feeling at times?

    Andy Goswick

    • edutechtalks permalink*
      January 16, 2010 1:07 pm

      Andy,

      You ask questions that most of us are asking. I think it has to do with the definition of collaboration. Apparently, the manner in which the word is used in the online setting differs from its usage in the face-to-face world. Furthermore, online collaboration appears to infringe upon personal freedom in the manner in which one may wish to achieve well-established, individual academic goals. It appears to be one thing to collaborate online for non-academic purposes and quite another if it were for academic reasons. Most of us who are pursuing a doctoral degree online, bring academic experiences that were defined by individual work. Those experiences included collaboration with instructors and fellow students for the purpose of receiving guidance, help, and resources to achieve personal, academic goals. Translating those experiences to the collaboration required because of the online nature of our education could be restrictive. Perhaps this expands on your current experience as it does mine.

      Just thinking!

      David

  2. Amy permalink
    January 14, 2010 8:43 pm

    David

    I agree that everyone involved in teaching or learning is concerned with the question, “How do we know they know?” The answer is simple; the more we engage in meaningful dialogue, the more we will learn about each other, their knowledge, and the subject at hand. Everyday, my students say that the answer is 4 or 9 and my first question is “How do you know?” Without prompting my students to participate in more collaboration or thinking skills, the learning process stops instantly. In an online setting, people engage in metacognition automatically because online communication requires validation, self-reflection, and planning. According to Rick West, several studies imply that online learning environments enhance metacognition naturally and aid in the learning process better than face-to-face environments (2006, October 3). It would very interesting to analyze more studies and evaluate if metacognition should be a mandatory component of learning. Therefore, I believe the most critical element of assessment should be self-evaluation and the use of metacognitive thinking. According to Palloff & Pratt, (2005) “students should receive credit for self-reflection and the activity should be incorporated into the design and expectations of online courses” (p. 43). Do you agree that our learning and abilities would be drastically different if we never thought about how we reached our conclusions?

    Palloff, R. M., & Pratt, K. (2005). Collaborating online: Learning together in community. San
    Francisco: Jossey-Bass. p. 43.

    West, R. (2006, October 3). Metacognition in online learning. Retrieved from http://rickwest.edublogs.org/2006/10/03/metacognition-in-online-learning-3/

    • edutechtalks permalink*
      January 16, 2010 1:28 pm

      Amy,

      I appreciate your comments.

      I think your indication that engagement in meaningful dialog is central to answering the question, “How do we know they know?” At the doctoral level, meaningful dialog may take different forms. The discussion and collaborative components would affect the methodology and outcomes of assessment.

      I am not sure I clearly understand your question, “Do you agree that our learning and abilities would be drastically different if we never thought about how we reached our conclusions?” I would say it depends on what you mean by “learning,” “abilities,” and “how we reached our conclusions.” I do not see how the manner in which one reaches conclusions should affect one’s abilities. Epistemological and logical necessity of the conclusions dictate the nature and scope, and, consequently, the value of learning. If your statement were based on a theoretical foundation, then since theories develop when researchers repeatedly test a prediction, a proposed new perspective or variation to a study would be meaningful and would objectively add to the body of literature if an existing theory could clarify one’s projected claims based on the history of past research and current development in similar fields. Consequently, based on the mindset that justifies the testing and assessment process, the relationship of a theory to the philosophy of learning is as tenable as it is ubiquitous in terms of the intent, content, and extent of a scientific inquiry.

      Just thinking!

      David

  3. Kimberly C Davis permalink
    January 16, 2010 10:44 pm

    Very reflective thoughts on collaboration assessment (online). Do you think collaborative insrtuction assessment is different for adult learning environments verse high school learning environments? How much responsibility do you think lies on the instructor as far as them making sure that assessment is fair when they are doing the assessment and when they are getting peer assessed? How much responsibility lies on students as far as pulling their weight in a collaborative environment?

  4. Tim Powell permalink
    January 21, 2010 12:22 am

    Great work, David. I bookmarked your link to the Cal State rubric for online courses. Not too much new information, but a good resource nonetheless!

    Tim.

  5. February 12, 2010 6:12 pm

    Very useful information on collaboration assessment. I can imagine how hard it is to make sure collaboration is taking place within an online classroom. Sometimes, it’s hard to get students to collaborate in an actual classroom. I wonder what happens when students don’t collaborate in the online setting? Do you think there’s some type of steps the professor takes to get that student to step up to the plate?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: