Knowledge Area 8: Learning Analytics

Knowledge Area 8 covers the skills, methods, and conceptual frameworks that enable a learning engineering team to collect, analyze, and act on data about learners and learning systems. Learning analytics (LA) is the measurement, collection, analysis, and reporting of data about learners and their contexts, with the purpose of understanding and optimizing learning and the environments in which it occurs. [LE-LS-GL-007] Within learning engineering, analytics serves as the empirical engine of the design-iterate-evaluate cycle: it converts learner interaction logs, assessment responses, and behavioral signals into evidence that drives instructional improvement. [LE-LS-AP-003]

What Learning Analytics Is

At its core, learning analytics asks: what is this learner doing, what do they know, and what should the system do next? These questions require three interlocked capabilities. First, instrumentation — the capture of fine-grained interaction data such as response latencies, error sequences, hint requests, and navigation choices. Second, modeling — the use of statistical or machine-learning algorithms to infer latent states (knowledge, affect, engagement) from observable behavior. Third, action — the translation of model outputs into adaptive decisions, instructor alerts, or curriculum revisions. [LE-LS-AP-005]

The field is distinct from — but deeply connected to — Educational Data Mining (EDM). EDM is oriented toward discovery: finding previously unknown patterns in large educational datasets using algorithmic techniques. [LE-LS-AP-005] Learning analytics is more action-oriented: it applies known methods to support real-time or near-real-time decisions for learners, instructors, and administrators. [LE-LS-CO-003] In practice, LA teams draw heavily on EDM methods and the two communities share conferences and journals. [LE-LS-CE-002] [LE-LS-CE-003]

Subfields and Core Topics

Student Modeling is the representation of what a learner knows and can do at any given moment. The foundational technique is Bayesian Knowledge Tracing (BKT), introduced by Albert Corbett and John Anderson. [LE-LS-AP-011] BKT models each knowledge component (KC) as a hidden binary state — known or not known — and updates probability estimates after each practice attempt using four parameters: initial knowledge probability, learning rate, slip rate, and guess rate. [LE-LS-AP-011] BKT remains the most widely deployed student model in production intelligent tutoring systems worldwide. [LE-LS-AP-002]

Educational Data Mining encompasses prediction models (which learners will fail?), clustering (what student subgroups exist?), relationship mining (which KCs co-occur in difficulty?), and distillation for human judgment (what visualizations help teachers act?). [LE-LS-AP-005] Ryan Baker codified these categories and founded the field's primary institutional homes. [LE-LS-PP-008]

Dashboards and Actionable Feedback convert analytic outputs into instructor-facing or learner-facing interfaces. The LAK Conference has been the primary venue for sociotechnical research on dashboard design, algorithmic fairness, and data privacy. [LE-LS-CE-002] [LE-LS-JO-004]

Ethical Considerations — including algorithmic bias, data minimization, consent, and equity — are foundational to responsible LA practice. The U.S. Department of Education's 2023 AI report documented algorithmic bias in e-proctoring systems and established civil-rights framing as a baseline for LE practice. [LE-LS-GL-004] Strategic priorities for addressing equity at scale were synthesized in the "High-Leverage Opportunities" report produced by the Penn Center for Learning Analytics. [LE-LS-GL-003]

Role in Learning Engineering

Learning engineering is an iterative, evidence-based process of designing, building, and improving learning experiences. [LE-LS-GL-007] Learning analytics provides the measurement arm of that process. When a learning engineer asks "did this instructional design work?", the answer comes from analytics — pre/post test gains, embedded assessment performance, completion rates, time-on-task distributions, and model-estimated knowledge trajectories. Design-based research, as codified by Ann Brown, established the epistemological standards for evaluating interventions in authentic educational contexts, and LA operationalizes that evaluation at scale. [LE-LS-AP-003]

A/B testing infrastructure, such as the E-TRIALS platform developed by Neil Heffernan at WPI for use with the ASSISTments system (serving 500,000+ students), enables learning engineers to run randomized controlled trials within live platforms. [LE-LS-PP-011] The doer effect — the causal advantage of active practice over passive reading — has been demonstrated at scale using such infrastructure. [LE-LS-AP-012]

Major Institutions and Venues

The Simon Initiative at CMU is the world's premier ecosystem for learning engineering research and infrastructure, encompassing LearnLab, the Open Learning Initiative (OLI), DataShop/LearnSphere (the world's largest open repository of educational log data), and the METALS master's program. [LE-LS-CO-002] Kenneth Koedinger, who co-founded Carnegie Learning and led development of Cognitive Tutors deployed in thousands of schools, directs much of this infrastructure. [LE-LS-PP-005]

The Penn Center for Learning Analytics, led by Ryan Baker, has been the field's most active policy-oriented research center and produced the strategic roadmap for the field. [LE-LS-CO-004] The HCII at Carnegie Mellon houses Koedinger, Corbett, and the PACT Center, making it the academic home of the Cognitive Tutor and BKT lineage. [LE-LS-CO-005]

Conference venues serve distinct roles: the LAK Conference (organized by SoLAR) focuses on ethical deployment, dashboards, predictive modeling, and algorithmic fairness in practice; [LE-LS-CE-002] the EDM Conference focuses on algorithmic approaches including knowledge tracing, early warning systems, and student behavior modeling; [LE-LS-CE-003] and the AIED Conference addresses AI-driven educational systems including generative AI tutors and conversational agents. [LE-LS-CE-004] Primary journals are the Journal of Educational Data Mining [LE-LS-JO-002] and the Journal of Learning Analytics. [LE-LS-JO-004]

Standards infrastructure for data collection includes IEEE 9274 / xAPI, which standardizes tracking of learning experiences across platforms via JSON-LD profiles and vocabulary registries, [LE-LS-SG-002] and IEEE P2247 for Adaptive Instructional Systems interoperability. [LE-LS-SG-003] Open infrastructure for LA research is provided by the OpenSimon Toolkit, which includes DataShop, LearnSphere, CTAT, OLI, and DiscourseDB. [LE-LS-GL-010]

Organizational Maturity

Organizations vary substantially in their capacity to do data-informed learning engineering. The Generalizable Learning Engineering Adoption Maturity Model (Blake-Plock et al., 2025) provides a multi-level framework for assessing organizational capability — from ad hoc data use through systematic, infrastructure-supported continuous improvement. [LE-LS-AP-013] LA competency is a key dimension in this assessment.

¶ Subsections:

• 8.1 Overview of Learning Analytics
• 8.2 Assessing Learning Outcomes and Learner Progress
• 8.3 Categories of Methods, Algorithms and Models in Learning Analytics
• 8.4 Artificial Intelligence and Machine Learning in Learning Analytics
• 8.5 Ethical Considerations in Learning Analytics

From the Learning Engineering Toolkit

The Toolkit defines analytics as the computer-driven, systematic examination of data and casts it as central to the investigation phase of the learning engineering process. [LET-06] It is the second half of how learning engineers put data to work: instrumentation gathers data from learning experiences, and analytics interprets that data to feed back to learners and their supporters and to drive successive improvements to what has been engineered. [LET-06] The book makes the case that design decisions need to be grounded in data because people's hunches about what helps learning are so often wrong; examining data from simple trials keeps the effort trained on the design features that count and heads off expensive detours. [LET-06]

One theme it returns to is that analyzing data is a collective endeavor rather than a solo one. [LET-06] Even a straightforward descriptive step — say, computing the fraction of students who pass a formative assessment — is better for the input of several roles, among them learning engineers, learning designers, content and assessment developers, psychometricians, implementation consultants, software engineers, and UI/UX designers. [LET-06] And as analyses become more advanced, moving through predictive and on to prescriptive forms, that shared interpretation matters just as much if not more, since sophisticated methods like deep learning can be harder to see into. [LET-06]

The book also separates learning engineering research from the generalizable research of the learning sciences. Learning engineering research concerns highly context-bound cases within particular courses or products for particular learner populations, so its findings frequently will not transfer elsewhere; the point is to try a solution out in context and let the data settle which option is best. [LET-06] Its companion tools chapter offers a learning analytics process model — assembled from Ryan Baker's open online materials — that walks a team from a real-world question, through recasting it as a data question and confirming that enough data exist, to choosing an analytical approach and judging how good the resulting model is. [LET-18]