Learning within crowdwork platforms

My paper on crowdworkers’ learning within microwork and online freelancing platforms has been accepted at Internet, Policy and Politics 2016 Conference organised by Oxford Internet Institute. I’m very much looking forward to the conference.


This paper reports findings of a survey exploring how crowdworkers develop their knowledge and skills in the course of their work on digital platforms. The focus is on informal learning initiated and self-regulated by crowdworkers: engaging in challenging tasks; studying professional literature/online resources; sharing knowledge and collaborating with others. The survey was run within two platforms representing two types of crowdwork – microwork (CrowdFlower) and online freelancing (Upwork). The survey uncovered evidence for considerable individual and social learning activity within both types of crowdwork. Findings suggest that both microwork and online freelancing are learning-intensive and both groups of workers are learning-oriented and self-regulated. Crowdwork is a growing form of employment in developed and developing countries. Improved understanding of learning practices within crowdwork would inform the design of crowdwork platforms; empower crowdworkers to direct their own learning and work; and help platforms, employers, and policymakers enhance the learning potential of crowdwork.


Reference: Margaryan, A. (22 September, 2016). Understanding crowdworkers’ learning practices. Paper presented at Internet, Policy and Politics 2016 Conference, Oxford Internet Institute, University of Oxford, UK.  [Online]


Abstract accepted for Dynamics of Virtual Work symposium

Pleased to find out that my abstract ‘Reconceptualising professional learning within emergent digitally-mediated work practices’ has been accepted for the forthcoming symposium of the Dynamics of Virtual Work research network.

Abstract: In many domains work has become increasingly complex, reliant on expertise distributed across a range of specialisms and involving novel problems at the boundaries of human knowledge (Boisot et al, 2011). In parallel, the unfolding digital transformation of work is catalysing new formations and constellations in the workplace that challenge traditional patterns of individual agency, organisation, power, responsibility and learning (Littlejohn and Margaryan, 2013). New forms of organisation mediated by digital technology include crowdwork, networked science, nomadic work and other types of distributed work (Bietz, 2013; Nickerson, 2013; Nielsen, 2012). These developments are having a profound effect on society and work, but are yet to have a significant effect on how professional learning is conceptualised and organised. Contemporary work practices require new forms of professional learning that align with the new spatial and temporal reconfigurations of workplaces, new work cultures, new networks of knowledge, and new requirements pertaining to the development and use of digital technologies. Conventional forms of professional learning such as formal training enable large numbers of people to reach a specific level of competency; however these forms of learning are unlikely to meet the learning needs of people in these new work contexts. Established forms of professional learning have largely not taken advantages of the opportunities around how people collaborate to learn, emergent knowledge networks, multiple ways in which people and knowledge resources can be brought together to enhance learning, and how digital technologies can extend access to these learning opportunities and resources. A fundamental rethink of how professional learning aligns with current trends in work, technology and society is required. In this presentation, I will discuss key implications of digital reinstrumentation and the emergent work practices for professional learning. Drawing on four concepts from learning sciences, sociology of work and technology-enhanced learning – self-regulation (Zimmerman, 2006), objectual practice (Knorr-Cetina, 2001), networked learning (Milligan, Littlejohn and Margaryan, 2014) and charting (Littlejohn, Milligan, and Margaryan, 2012) – I will outline some ways in which learning within emergent digitally-mediated work practices may be reconceptualised and fostered.


Bietz, M. (2013). Distributed work: Working and learning at a distance. In Littlejohn, A., & Margaryan, A. (Eds.). Technology-enhanced professional learning: Processes, practices and tools (pp. 28-38). London: Routledge.

Boisot, M., Norberg, M., Yami, S., & Nicquevert, B. (2011) (Eds.) Collisions and collaboration: The organisation of learning in the Atlas Experiment at the LHC. Oxford: Oxford University Press.

Knorr-Cetina, K. (2001). Objectual practice. In Schatzki, T., Knorr-Cetina, K., & Savigny, E. (Eds), The practice turn in contemporary theory (pp. 175-188). London: Routledge.

Littlejohn, A., & Margaryan, A. (2013) (Eds.). Technology-enhanced professional learning: Processes, practices and tools. London: Routledge.

Littlejohn, A., Milligan, C., & Margaryan, A. (2012). Collective knowledge: Supporting self-regulated learning in the workplace. Journal of Workplace Learning, 24(3), 226-238.

Milligan, C., Margaryan, A., & Littlejohn, A. (2014). Workplace learning in informal networksJournal of Interactive Media Environments, special issue ‘Reusing Resources – Open for Learning. [Online] file:///Users/ama11/Downloads/325-2585-1-PB%20(2).pdf

Nickerson, J. (2013). Crowd work and collective learning. In Littlejohn, A., & Margaryan, A. (Eds.). Technology-enhanced professional learning: Processes, practices and tools (pp. 39-49). London: Routledge.

Nielsen, M. (2012). Reinventing discovery: The new era of networked science. Princeton: Princeton University Press.

Zimmerman, B. (2006). Development and adaptation of expertise: The role of self-regulatory processes and beliefs. In Ericsson, A., Charness, N., Feltovich, P., & Hoffman, R. (Eds.), The Cambridge handbook of expertise and expert performance (pp. 705-722). New York: Cambridge University Press.

Presenting at Learning and Professional Development SIG conference in Oslo

I’m looking forward to the EARLI’s Learning and Professional Development Special Interest Group Conference in Oslo next week.  I’m contributing to two session as follows:

  • As a discussant at symposium “A social network perspective on learning and professional development” on Wed 27/08/14 at 16:30-18:00
  • As a co-author at paper session 3C ‘Motivation and self-regulation at work’, paper titled “Self-regulated learning in the financial services industry” (Milligan, Fontana, Littlejohn, Margaryan) on Thur 28/08/14 at 09:00-11:00

Let me know if you are around and would like to meet up.

My Online Educa Berlin 2013 schedule

It’s that time of the year again and I am at Online Educa conference in Berlin.

There is an extensive programme of plenary and parallel sessions over the next couple of days. I’m  planning to attend the following sessions:


09:15-11:00 Opening plenary in which the talk by Viktor Mayer-Schönberger on big data is the one I’m looking forward to most.

12:00-13:30 BUS04 Learning Moves in the Corporate Sector parallel session, with representatives from Challenge Stretching Talent The Netherlands; UK Medical Research Council; DNV GL Oil and Gas Norway, and Swiss Post, Switzerland

14:30-16:00 My own talk on ‘narrating work’ experiment in BUS20 Doing Things Differently parallel session

16:30-17:30 BUS36 The Battle of Benchmarks, with presentations from ASTD and others.

17:45-19:00 The OEB Debate

20:00+ OEB Dinner and Dance


09:30-11:00 Plenary on Lifelong Learning

11:45-13:00 BUS50: A Global Dialogue on a New Way of Working,Learning and Innovating

14:15-16:00 BUS65: Transforming Learning, with case studies from Toyota Europe and IBM Germany

I’m very much looking forward to a stimulating conference and to meeting old and new colleagues.


I am at Online Educa Berlin this week.  A conversation with another conference participants crystallised in my mind the three types of conferences I prefer:

  1. Conferences which cover a broad variety of research domains within my research field, Learning Sciences. A key conference exemplifying this category for me is the biannual conference of  the European Association of Research in Learning and Instruction (EARLI).   
  2. Conferences which bring together a variety of sectors and organisations – universities or other public sector organisations and the industry, research centers and publishers, etc.  Online Educa Berlin (OEB) is an example of this type of conference.
  3. Conferences focused on my particular research area (Technology-Enhanced Professional Learning).  Typical conferences in this category are EARLI Professional Learning SIG, Researching Work and Learning (RWL), EDMEDIA

My EARLI2011 diary

As I said earlier, I’m heading off to Exeter tomorrow to participate in the EARLI2011 conference.  If anybody wants to meet up, my conference diary is as follows:



  • 10:15-11:00: EARLI Opening/JURE Closing session
  • 11:00-12:30: Session A16 Professional Development
  • 12:30-13:30: Lunch
  • 13:30-15:00: Sessions B4 Culture and Education and B16 Training of young researchers
  • 15:30-17:00: Session C16 Learning Theory and Educational Attainment
  • 17:15-18:45: Presidential address and awards
  • 19:00+ Opening Reception


  • 09:00-10:30: Symposium D5 Different Perspectives on Understanding Learning in the Workplace
  • 11:00-12:30: Poster sessions (not sure which thematic session I am going to, I think I will check all of them out)
  • 12:30-13:30: Lunch
  • 13:30-15:00: Session F18 Training of young researchers
  • 15:30-17:00: Keynote 1 Deliberate Practice and the Future of Education & Professional Training
  • 17:00-18:30 Session G12 Professional Development


  • 09:00-10:30: Panel Discussion H24 Developing expertise development in the classroom: A utopia or reality?
  • 11:00-12:30: Round Table I14 Typology of informal workplace learning [I am leading this round table discussion together with Colin Milligan and Allison Littlejohn]
  • 12:30-13:30: Lunch
  • 13:30-15:00: Symposium J2 The Question of Research Methodology: Where next for EARLI?
  • 15:30-17:00: Keynote 1 Deliberate Practice and the Future of Education & Professional Training
  • 18:00-19:00: SIG 14 Professional Development Meeting


  • 09:00-10:30: Session K3 Measuring Outcomes of Professional Education
  • 11:00-12:30: Poster Sessions (TBI)
  • 12:30-13:30: Lunch
  • 13:30-15:00: Session M12 Lifelong Learning and Professional Development session [I am chairing this session; also our paper on Learning From Incidents in Organisations, co-authored with Dane Lukic and Allison Littlejohn, will be presented here]
  • 15:30-17:00: Keynote 3 TLRP’s ten principles for effective pedagogy: rationale, development, evidence, argument and impact
  • 17:00-18:30: General Assembly
  • 19:30 + Conference Dinner
I am leaving on Saturday morning to return to Glasgow, so Friday will be my last day at the conference.

Metacognition, epistemological beliefs and the division of cognitive labor (EARLI09 Keynote, Rainer Bromme)

Yesterday’s EARLI09 keynote by Rainer Bromme was very interesting, in my view the best one of all of this year’s EARLI keynotes that I had a chance to attend. I have captured some of the ideas and research findings he shared, what follows below is the abstract of his talk and the summary of my notes.

Abstract: “Due to the division of labor in modern societies, knowledge is distributed and used unevenly. Most of the knowledge we acquire through lifetime has been produced by specialized experts, is provided by specialized experts, and it is organized into disciplines, reflecting such specialization. This  division of cognitive labor has implications for our understanding of learning, epistemological beliefs and metacognition. Recent approaches on learning (especially those inspired by constructivist ideas) and research on epistemological beliefs undervalue the division of cognitive labor. Instead, they are in favor of personal knowledge construction, of first hand experiences as the main ways of learning. The implicit assumption ‘own knowledge is better than knowledge attained from others’ is underlying many approaches in educational research. In contrast to this normative assumption, we all remain laypersons with regard to the most domains of knowledge in modern societies. Therefore we will have to cope with experts and with specialized expert knowledge for the whole lifetime. It is necessary to learn how to evaluate knowledge which we do attain from experts. Such judgments are necessary even when they are based on a fragmentary understanding of the knowledge claims. At first I will review such positions held within research on epistemological beliefs and on learning. Secondly I will sketch a research program on students’ capacities to understand how specialized knowledge is distributed (who knows what?) and to evaluate expert sources (whom to believe?). Such capacities entail epistemological beliefs about expert knowledge and metacognitive awareness about ones’ own knowledge. I will then resume some empirical evidence from developmental psychology about children’s intuitive understanding of the division of cognitive labor and I will ask how schooling fosters and impedes such understanding. Finally, some of our own studies about the cognitive division of labor will be sketched. This research focuses on laypersons’ capacities for the evaluation of health related expert knowledge found in the Internet. Based on our research the relationship between metacognition and such capacities will be discussed.”

He started off by defining the notion of division of cognitive labour – “the reliance on the deeper understanding of knowledge held by others when using this knowledge for communication, for categorising our environment, cooperation and decision making”.  Argues this notion is linked with division of labour which has developed in human societies froom prehistoric times, but has increased dramatically after the industrial revolution and expecially recently in the context of knowledge economy and advancements in ICT.  Everincreasing specialisation of knowledge means people need skills in continuous assessment of knowledge claims made by experts.

Argues there is a conceptual distinction between first-hand and second-hand knowledge claims.  The former refers to assessment of “what is true?” and the latter to “whom to believe?”  Criteria of assessment of first hand knowledge claims are well known and well researched (critical thinking, evidence, cohesiveness and logic of argument) and a lot of instructional and research efforts have been focused on dealing with this type of knowledge claims.  In contrast, there hasn’t been enough focus on the second-hand knowledge claims (whom to believe) in learning theory or development of instruction.   [An exception that comes to mind is Harry Collins’s research programme on expertise and his recent book co-authored with Robert Evans].  In addition, traditional approaches to first-hand knowledge claims have been influenced by Piagetian focus on personal, first-hand authentic experience, disregarding the social nature of our knowledge, ie that it does not derive only from indidvidual experience but we have to rely on what others know.

Goes on to argue that laypersons are not novices who want to become experts-above and beyond basic literacy and focus on specific areas, most laypeople become “acclimatised” to broader knowledge but not proficient in it.

Research on metacognition has focused on person’s own cognition, but studying metacognition about others’ metacognition is necessary (who knows what, who has the knowledge to corraborate knowledge claims). This area is currently underresearched. Some people argue that research on theory of mind (TOM) is focusing on this, but that’s not correct – the focus of TOM is on comparing others’ minds to one’s own, ie the focus is still on individual.

Then shares early findings of a range of empirical studies conducted by his group testing students’ abilities to evaluate knowledge claims, in particular identify who to ask when presented with tasks they don’t have sufficient knowledge to complete. Found that majority of  students identified the right experts to ask for further information above chance  [note to self to follow up on these studies, some are not published yet]. Concludes that humans have capabilities not to rely on knowledge by experts and to make accurate judgements on how the necessary knowledge can be accessed even if they are not faimilar with the domain.

Question and answer session focused on discussion of the implications of this research for democratisation of knowledge through web 2.0 phenomenon (how people can assess knowledge that is not created by experts and that has not been vetted by gatekeepers) and implications of this research for educational system (“should there be a subject in schools teaching children how to judge expert knowledge”).