Instructional Quality of Massive Open Online Courses (MOOCs)

I was pleased to hear that our paper on the quality of instructional design of massive open online courses (MOOCs) has been accepted for publication in Computers and Education.  

Abstract:  We present an analysis of instructional design quality of 76 randomly selected Massive Open Online Courses (MOOCs). The quality of MOOCs was determined from first principles of instruction, using a course survey instrument. Two types of MOOCs – xMOOCs and cMOOCs – were analysed and their instructional design quality was assessed and compared. We found that the majority of MOOCs scored poorly on most instructional design principles. However, most MOOCs scored highly on organisation and presentation of course material. The results indicate that although most MOOCs are well-packaged, their instructional design quality is low. We outline implications for practice and ideas for future research.

The full paper will be available online shortly; in the meantime, a draft is available for download.

It took 18 months to get to this stage from the conception of the study:

• conception of the project idea and securing internal funding for a research assistant – Feb 2013;
• search and recruitment of a research assistant – Feb-Jul 2013 (6 months);
• data collection – Sep-Dec 2013 (4 months);
• data analysis – Jan 2014;
• publication – Feb-Aug 2014 (7 months), including: (i)submission of the first draft of the article – Feb 2014; (ii) review received – Apr 2014; (iii) resubmission of 2nd draft – May 2014; (iv)second review received – Jul 2014; (v) resubmission and acceptance of the final draft – Aug 2014.  

So the actual conception and execution of the study took just 6/18 months…

Citation: Margaryan, A., Bianco, M., & Littlejohn, A. (in press). Instructional quality of Massive Open Online Courses (MOOCs). Computers and Education.

Open teaching, open learning

Cormier and Siemens reflect on their experiences with teaching open courses, in particular a form of course that they term ‘massive online open course’ or MOOC.

Of particular interest to me were their observations on educators’ roles in open courses: “Open learning adjusts the role of the educator with respect to access to new content and engagement tools now under the control of the learner. Educators continue to play an important role in facilitating interaction, sharing information and resources, challenging assertions, and contributing to learners’ growth of knowledge.” They list the following teacher roles:

• Amplifying – drawing attention to important ideas/concepts
• Curating – arranging readings and resources to scaffold concepts
• Wayfinding – assisting learners to rely on social sense-making through networks
• Aggregating – displaying patterns in discussions and content
• Filtering – assisting learners in thinking critically about information/conversations available in networks
• Modeling – displaying successful information and interaction patterns
• Staying present – maintaining continual instructor presence during the course, particularly during natural activity lulls

Compare this with the more familar list – delivering lectures, marking assignments, monitoring attendance, and so on.

Other ideas and principles they put forward that resonated with me:

• “Content itself is not a sufficient value point on which to build the future of higher education”
• “The registration process is the approach to the conversation; the filtering, the deciding whether or not to participate, happens after registration”
• “The opening up of the teaching process is an important dimension of openness in education more broadly.”
• “The community-as-curriculum model inverts the position of curriculum: rather than being a prerequisite for a course, curriculum becomes an output of a course.”

Reference: Cormier, D., & Siemens, G. (2010). Through the open door: Open courses as research, learning, and engagement. EDUCAUSE Review, 45(4), 30-39. [Online] http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume45/ThroughtheOpenDoorOpenCoursesa/209320

Book on work-based learning published

My book “Work-based learning: A blend of pedagogy and technology” has been published by VDM Verlag.  It is based on my PhD research at Shell EP in the Netherlands in 2002-2005.

Synopsis
Global socio-economic changes are transforming workplaces. Organisations require employees skilled in strategic problem-solving; learning quickly in response to rapidly changing environment; working in distributed, culturally-diverse teams; building knowledge from different sources and applying it in a flexible way. The new workplace demands are in sharp contrast with the traditional ways of learning and teaching. The assumption that these highly complex skills can be learned in traditional formal learning settings focused on transferring content from experts to novices is no longer tenable. It is equally untenable that the knowledge and skills for effective performance in the workplace can be picked up from experienced peers or coaches in informal learning settings alone. New learning approaches enabling integration of the worlds of learning and work are needed. This book describes a new model for technology-enhanced work-based learning.The model aims to support managers, instructors and learning designers in organisations in devising strategy and curriculum for learning; in developing course design, evaluation and assessment processes; and in capturing and sharing good practice in learning within organisation.  The model was implemented at Shell EP Netherlands. The book is addressed to researchers and practitioners in work-related and technology-enhanced learning.

Table of Contents (high level)

List of Figures

List of Tables

Acknowledgements

Introduction: Work and Learning.

1. Work-Based Learning in Organisations.

1.1. Current challenges faced by organisations

1.2. Characteristics of work-based learning.

1.3. Work-based learning: Link between formal and informal learning.

1.4. Work-based activities as vehicles for integration of work and learning.

1.5. Technology affordances

2. Activity Theory: A Holistic Framework for Work-Based Learning.

2.1. Socio-cultural perspectives of learning.

2.2. Activity systems

2.3. Technology-enhanced work-based learning as an activity system.

3. Reference Model for Work-Based Learning.

3.1. Concretising work-based learning activity system instruments for implementation in practice.

3.2. First Principles of Instruction.

3.3. First Principles of Instruction extended.

4 Context of the Study: Shell Exploration and Production.

4.1. Shell Exploration and Production Learning and Leadership Development

4.2. Organisational needs for new forms of learning: The EP Learning Strategy.

4.3. Work-based blended learning in Shell EP LLD.

4.4. Technology for learning support at Shell EP LLD.

4.5. Work-based learning in Shell EP LLD as an activity system. 59

5. Investigation and Applications of the First Principles Plus Reference Model at Shell EP LLD

5.1. Methodology for investigating and applying the Reference Model

5.2. Investigations of the local elements of the Reference Model

5.3. Course scan: An instrument to measure the elements of the Reference Model

5.4. Application 1: Support for organisational learning strategy and across-course consistency.

5.5. Application 2: Support for after-action reviews and reflection within a faculty.

5.6. Application 3: Support for course evaluation procedures

5.7. Synthesis and guidelines

5.8. Conceptual reflections

5.9. Next steps: Capturing and sharing good practices in implementation of Reference Model

6. Design and Development of the “Work-based Learning Practices Worth Replication” Digital Repository.

6.1. Design methodology and procedures

6.2. Design, development, and evaluation of the repository: First cycle.

6.3. Design, development, and evaluation of the repository: Second cycle.

6.4. Design, development, and evaluation of the repository: Third cycle.

7. Conclusions.

7.1. Research questions revisited.

7.2. Conceptual contributions

7.3. Contributions to practice.

7.4. Future research.

References.

Appendices.

Appendix 1. Course Analysis Spreadsheet

Appendix 2. Course Scan Instrument

Appendix 3. P-310 course evaluation data.

Appendix 4. Guidelines for implementation of the Reference Model

Appendix 5. WBL-PWR repository, Prototype 3, July 18, 2005.

Appendix 6. Evaluation of the third prototype: Questionnaire.