Charting Possible Performance Indicators Related to Digital Learning Content Supported Learning
Copyright (c) 2022 Balkányi Péter
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The copyright to this article is transferred to the University of Public Service Budapest, Hungary (for U.S. government employees: to the extent transferable) effective if and when the article is accepted for publication. The copyright transfer covers the exclusive right to reproduce and distribute the article, including reprints, translations, photographic reproductions, microform, electronic form (offline, online) or any other reproductions of similar nature.
The author warrants that this contribution is original and that he/she has full power to make this grant. The author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors.
An author may make an article published by University of Public Service available on a personal home page provided the source of the published article is cited and University of Public Service is mentioned as copyright holder
Abstract
During my professional carrier, I participated in several digital content development projects, and in my doctoral dissertation I examined the ecosystem of e-learning development according to the Design Science (DS) methodology. Partly from the practical experience, partly from the comprehensive research, I perceived that few indicators – which can be applied in practice – were determined about the effectiveness and efficiency of a digital learning content development process.
Traditionally, projects can be evaluated by the ‘iron triangle’: the project is completed on time, within budget in right quality. This approach is not fundamentally wrong, but as several studies point out, it does not provide a complete picture, can even lead to false conclusions, and can determine wrong decisions.
The present work aims to focus specifically on the e-learning content created as a product of professional authoring tools (e.g. the end result is a SCORM standard package). This paper does not seek to take into account the whole development process. In addition to the focus on the content development, it is also important to underline that the background of these findings is in adult learning environments in large or multinational corporations.
In this article, I summarise the key technological, methodological and financial indicators (both key performance indicators and key result indicators) for the learner- and content-centred e-learning materials. Together, these indicators can form a system that gives the opportunity to evaluate and compare e-learning materials in its complexity.
Keywords:
How to Cite
References
Adelman, Clem, ‘Kurt Lewin and the Origins of Action Research’. Educational Action Research 1, 1 (1993), 7–24. Online: https://doi.org/10.1080/0965079930010102
Aebli, Hans, Didactique psychologique: Application à la didactique de la psychologie de Jean Piaget. Delachaux et Niestlé, 1951.
Atkinson, Roger, ‘Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria’. International Journal of Project Management 17, no 6 (1999), 337–342. Online: https://doi.org/10.1016/S0263-7863(98)00069-6
Au-Yong Oliveira, Manuel and João José Pinto Ferreira, ‘Business Model Generation: A Handbook for Visionaries, Game Changers and Challengers, Book Review’. African Journal of Business Management 5, no 7 (2011), 22–30.
Balkányi, Péter, Az e-learning tananyagfejlesztés ökoszisztémájának Design Science módszertan szerinti vizsgálata. Doctoral thesis, Budapesti Corvinus Egyetem, 2019.
Blass, Eddie and Ann Davis, ‘Building on solid foundations: establishing criteria for e-learning development’. Journal of Further and Higher Education 27, no 3 (2003), 227–245. Online: https://doi.org/10.1080/0309877032000098662
Bradbury-Huang, Hilary, ‘What is good action research? Why the resurgent interest?’. Action Research 8, no 1 (2010), 93–109. Online: https://doi.org/10.1177/1476750310362435
Busstra, Maria C, Rob Hartog, Sander Kersten and Michael Müller, ‘Design guidelines for the development of digital nutrigenomics learning material for heterogeneous target groups’. Advances in Physiology Education 31, 1 (2007), 67–75. Online: https://doi.org/10.1152/advan.00090.2006
Cantoni, Virginio, Massimo Cellario and Marco Porta, ‘Perspectives and challenges in e-learning: towards natural interaction paradigms’. Journal of Visual Languages & Computing 15, no 5 (2004), 333–345. Online: https://doi.org/10.1016/j.jvlc.2003.10.002
Capper, Joanne, ‘E-Learning Growth and Promise’. TechKnowLogia 2, no 2 (2001), 7–10.
Chakraborty, Ayon and Kay Chuan Tan, ‘Case study analysis of Six Sigma implementation in service organisations’. Business Process Management Journal 18, no 6 (2012), 992–1019. Online: https://doi.org/10.1108/14637151211283384
Csedő, Zoltán, Tamás Tóth, András Égler and Zoltán Sára, ‘Online képzési stratégiák és módszerek a közszolgálati továbbképzésekben’. Információs Társadalom 14, no 1 (2014), 9–28. Online: https://doi.org/10.22503/inftars.XIV.2014.1.1
Dichev, Christo, Darina Dicheva, Galia Angelova and Gennady Agre, ‘From Gamification to Gameful Design and Gameful Experience in Learning’. Cybernetics and Information Technologies 14, 4 (2015), 80–100. Online: https://doi.org/10.1515/cait-2014-0007
Diederen, Julia, Harry Gruppen, Rob Hartog, Gerard Moerland and Alphous G J Voragen, ‘Design of Activating Digital Learning Material for Food Chemistry Education’. Chemistry Education Research and Practice 4, no 3 (2003), 353–371. Online: https://doi.org/10.1039/B3RP90020G
Galloway, Dominique L, ‘Evaluating Distance Delivery and E-Learning: Is Kirkpatrick’s Model Relevant?’ Performance Improvement 44, 4 (2005), 21–27. Online: https://doi.org/10.1002/pfi.4140440407
Gardiner, Paul D. and Kenneth Stewart, ‘Revisiting the golden triangle of cost, time and quality: the role of NPV in project control, success and failure’. International Journal of Project Management 18, no 4 (2000), 251–256. Online: https://doi.org/10.1016/S0263-7863(99)00022-8
Hevner, Alan, Salvatore T March, Jinsoo Park and Sudha Ram, ‘Design Science in Information Systems Research’. MIS Quarterly 28, no 1 (2004), 75–105. Online: https://doi.org/10.2307/25148625
Hevner, Alan and Samir Chatterjee, Design Research in Information Systems: Theory and Practice. Springer, 2010. Online: https://doi.org/10.1007/978-1-4419-5653-8
Jung, Insung, ‘The dimensions of e-learning quality: from the learner’s perspective’. Educational Technology Research and Development 59, no 4 (2011), 445–464. Online: https://doi.org/10.1007/s11423-010-9171-4
Kaplan, Robert S, ‘Innovation Action Research: Creating New Management Theory and Practice’. Harvard Business School, 1998. Online: https://www.hbs.edu/faculty/Pages/item.aspx?num=3674
Kirkpatrick, Donald L, ‘Evaluating training programs: Evidence vs. proof’. Training and Development Journal 31, no 11 (1977), 9–12.
Lee, Ming-Chi, ‘Explaining and predicting users’ continuance intention toward e-learning: An extension of the expectation–confirmation model’. Computers & Education 54, no 2 (2010), 506–516. Online: https://doi.org/10.1016/j.compedu.2009.09.002
Marques, Pedro A and José G Requeijo, ‘SIPOC: A Six Sigma tool helping on ISO 9000 quality management systems’. 3rd International Conference on Industrial Engineering and Industrial Management, 2–4 September 2009, 1229–1238.
Maurya, Ash, Running Lean : Iterate from Plan A to a Plan that Works. Sebastopol, CA: O’Reilly, 2012.
Moore, James F, The Death of Competition: Leadership and Strategy in the Age of Business Ecosystems. New York: Harper Business, 1996.
Nahalka, István, ‘Konstruktív pedagógia-egy új paradigma a láthatáron (I.)’. Iskolakultúra 7, no 2 (1997), 21–33.
Nahalka, István, Hogyan alakul ki a tudás a gyerekekben? Konstruktivizmus és pedagógia [How Does Knowledge Develop in Children? Constructivism and Pedagogy]. Budapest: Nemzeti Tankönyvkiadó, 2002.
Parmenter, David, Key Performance Indicators: Developing, Implementing, and Using Winning KPIs. John Wiley & Sons, 2015. Online: https://doi.org/10.1002/9781119019855
Rapoport, Robert N, ‘Three Dilemmas in Action Research: With Special Reference to the Tavistock Experience’. Human Relations 23, no 6 (1970), 499–513. Online: https://doi.org/10.1177/001872677002300601
Ryan, Richard M and Edward L Deci, ‘Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being’. American Psychologist 55, no 1 (2000), 68–78. Online: https://doi.org/10.1037/0003-066X.55.1.68
Sabir, Brahim, Bouzekri Touri and Mohamed Moussetad, ‘Using the Integrated Management System and Approach SIPOC in Higher Education for the Evaluation and Improving the Quality of Life of Students’. The Online Journal of Quality in Higher Education 2, no 3 (2015).
Sántha, Kálmán, ‘A kvalitatív metodológiai követelmények problémái’. Iskolakultúra 17, no 6–7 (2007), 168–177.
Savage, Jan, ‘Participative Observation: Standing in the Shoes of Others?’ Qualitative Health Research 10, no 3 (2000), 324–339. Online: https://doi.org/10.1177/104973200129118471
Shelton, Kaye, ‘A Quality Scorecard for the Administration of Online Education Programs: A Delphi Study’. Journal of Asynchronous Learning Networks 14, no 4 (2010), 36–62. Online: https://doi.org/10.24059/olj.v14i4.163
Sinclair, Peter M, Ashley Kable, Tracy Levett-Jones and Debbie Booth, ‘The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: A systematic review’. International Journal of Nursing Studies 57 (2016), 70–81. Online: https://doi.org/10.1016/j.ijnurstu.2016.01.011
Szokolszky, Ágnes, Kutatómunka a pszichológiában. Budapest: Osiris Kiadó, 2004.
Van der Westhuizen, Danie and Edmond P Fitzgerald, ‘Defining and measuring project success’. Proceedings of the European Conference on IS Management, Leadership and Governance, 7–8 July 2005, 157–163.