Just like assessment, it’s important to think of evaluation as something you do throughout the learning process.

This is module seven of a Learning To Teach Online MOOC with Coursera, you can read about module 1 here and about module 2 here, and about module 3 here and about module 4 here and about module 5 here and about module 6 here and about module 7 here.

Module Learning Outcomes

By engaging with the content, and completing the activities and related discussion in this module, you should be able to:

  1. Describe why it is important to evaluate your teaching practice
  2. Identify a range of strategies you might employ in order to holistically evaluate your own course design

Overview Video

Once you’ve introduced a new technology in a course you need to be able to evaluate it:

  • Was the change successful?
  • Did it meet your intended outcome/goal?
  • What was the feedback from students? Peers?
  • How do you think it worked?
  • What you use it again or change it?

Capturing feedback from multiple angles is important.

Using data from students’ online footprints is important too:

  • Are they using the online technologies in the way you intended?
  • Are they engaging with the activities/assessments as designed?
  • Should you change your course design/practice through the way in which they’ve engaged with the technologies?

Setting a Learning Goal

goal-374490_640

Image available from here

For this module (chosen from a list):

Gain confidence with tools and methods that I can use in my practice

For the whole MOOC (chosen from a list):

I want to gain knowledge about blended/online learning and use this to re-design my course.

Key Concepts Videos

idea_by_ssandiiee-d4gfknd

Image available from here

Evaluation Strategies (Video 1)

It’s important to know whether your strategies are working or not.

Just like assessment, it’s important to think of evaluation as something you do throughout the learning process.

Consider the evaluation from multiple angles too.

Self- reflection:

  • Did your teaching strategy achieve the outcome you were expecting?
  • Is there something you would change next time?
  • Would you recommend the approach to a colleague?
  • Was the technology appropiate

Peer-feedback:

  • What feedback can they offer?

Student feedback:

  • A short survey during the course?
  • Conduct focus groups?

Theory and scholarship:

  • How have others approached to evaluation?
  • How do their outcomes relate to your own experience?

Use objective data (learning analytics):

You can start evaluating your online initiative using more behavioural data

This data can be provided by LMS but also by more open technologies such as YouTube, Twitter and Google.

Learning Analytics to inform teaching practice (Video 2)

The data that you collect about the participation in an online environment. How can we use this data to improve the environment?

The interactions that take place:

  • students-students
  • students-material
  • students-tutor

Any interaction that occurs can be tracked.

Planning:

STEP 1: Think about what problem/aspect do you want to detect and then act upon? For instance, I want to make sure my students sustain their engagement throughout the course.

STEP 2: Think about what kind of data will show you this- what kind of activities will you need to analyse?

STEP 3: What actions/adjustments do you need to employ in the environment to solve the issue?

You can then speak to students to gather their feedback so that your data is triangulated and you can make full sense of it.

Acting upon the data is vital- don’t just look at it/observe it.

PDF to support these videos

Activities

images

Image available from here

Activity 8.1- Self-Awareness

Why don’t you evaluate your teaching more regularly? (tick all that apply)

  • I am not sure of the best way to perform the evaluation
  • I’m worried about what I might discover

Which of the following statements below do you most agree with?

  • I have evaluated my course design or teaching practice from a single perspective only

Which of the following statements do you think is the most accurate definition of learning analytics?

  • The measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs

One of your colleagues has asked you to provide peer evaluation of their online teaching. Which factors will you keep in mind when observing your colleague? (tick all that apply)

  • I would focus on the areas on which my colleague has specifically asked for feedback on

Activity 8.2- Knowledge

Scenario: A teacher in a course teaches through blended mode – a combination of face-to-face activities in class and online activities using a learning management system. The online activities involve a combination of lecture videos on YouTube, discussion forums in the learning management system (LMS), and online self-tests. In class, the teacher usually provides a short mini-lecture based on those questions in the self-test that students answer incorrectly, and on the questions raised in the online discussion forums. The students then work in teams to discuss the concepts introduced in the online lecture videos that they were required to view prior to attending the class. Assessments in the class include a mid-term and final exam based on the content covered in the course including videos, and one 1500 word essay on a topic that the student wishes to research and critically analyse. This teacher has been teaching the class in this format for two consecutive years and is now interested to know how effective his teaching is by employing various evaluation techniques.

Referring to the previous scenario, if you were the teacher, how would you go about evaluating your teaching using self-reflection strategies? (tick all that apply)

  • Have a reflective teaching portfolio where reflections on different teaching strategies are documented
  • Record videos of myself when teaching in the classroom and reflect later on to identify strengths and weaknesses to improve my teaching performance
  • Keep asking myself if the content, learning activities, technology, and assessment tasks help achieve the learning outcomes or not

Referring to the previous scenario, if you were the teacher, how would you go about evaluating your teaching using peer feedback strategies? (tick all that apply)

  • Ask a couple of colleagues to sit in on a couple of my in class sessions and/or view my LMS site to offer feedback

Referring to the previous scenario, if you were the teacher, how would you go about evaluating your teaching using student feedback strategies? (tick all that apply)

  • Short surveys, focus groups, and review of end of semester class evaluations

Referring to the previous scenario, if you were the teacher, how would you go about evaluating your teaching to collect evidence for your promotion application using theory/literature strategies? (tick all that apply)

  • Review related scholarly literature on blended learning class design (eg. in peer-reviewed journals) to see whether the feedback received from my students and peers matches with the findings in published studies
  • Publish my findings of the evaluation of my teaching and class in peer-reviewed conference proceedings or journals

Referring to the previous scenario, if you were the teacher, how would you go about evaluating your teaching using learning analytics strategies? (tick all that apply)

  • Access reports from the Learning Management System (LMS) to show how students are engaging with the online material on the LMS. The reports show how many times students have accessed the self-tests, clicked on a video link, or posted in the discussion forum
Q. In light of the concepts covered in Module 8, how would you go about evaluating your teaching practice or course design using self-reflection strategies?
A. During and at the end of each of my online projects this year, I will reflect upon what I wanted the level of engagement and skills development to be and I will consider how far each of these was met.
Q. In light of the concepts covered in Module 8, how would you go about evaluating your teaching practice or course design using peer feedback strategies?
A. As I am making my first journeys into real online learning this academic year, peer feedback will be valued highly. During and at the end of projects, I will discuss the challenges with my peers and ask them to review the projects and learner participation to help me consider what I could do facilitate learning more effectively. 
Q. In light of the concepts covered in Module 8, how would you go about evaluating your teaching practice or course design using student feedback strategies?
A. Student feedback will be vital this year as their engagement in online learning will be different and more structured to how it has occurred previously. Part way through the first project, I will gather student feedback via a Google form (ratings) which I can then analyse. After analysis, I will then meet with a small focus group to discuss what I could do to improve the learning experience for them in the online learning environment. 
Q. In light of the concepts covered in Module 8, how would you go about evaluating your teaching practice or course design using learning analytics strategies?
A. I will use these analytics initially to learn about student engagement: when they’re asking, how long they’re staying for and what they’re using. I will ensure that I then use this data to structure the next short online project. It will also help me to contact those learners (more quickly) who are not interacted as well as they should be online. 
Q. In light of the concepts covered in Module 8, how would you go about evaluating your teaching practice or course design using the literature and/or theory related to your practice?
A. I have made use of the literature in the initial design of my online projects and will continue to do so as the projects progress- using them at the start but also when things don’t go as planned to see what others have done in similar situations and what their (hopefully more positive) experiences have been. 

Discussion Forums

chairs-358404_640

Image available from here

One forum, related to the evaluation of practice, made me think about the process of designing this years lesson plans for staff- I am starting to develop a quick lesson evaluation sheet and much of what I have learnt on this course will help me in designing this.

This resource, about Writing a Teacher Diary will help with my design of this reflection sheet.

In another forum, I found this document related to evaluation. There was also this link shared and this, to the journal of online learning.

Advertisements