8.4.2 Machine Learning Algorithms in Learning Analytics

Machine learning algorithms used in learning analytics range from classical statistical methods to deep neural architectures. Choosing among them requires understanding not just their technical properties but the specific challenges of educational data: longitudinal sequences, sparse labels, heterogeneous learner populations, cold-start problems for new learners, and strong ethical constraints on what errors are acceptable.

Deep Knowledge Tracing

Deep Knowledge Tracing (DKT), proposed around 2015, applies recurrent neural networks (RNNs) — specifically Long Short-Term Memory (LSTM) networks — to the knowledge tracing problem. Rather than maintaining a small set of interpretable parameters per knowledge component (as BKT does), DKT learns a distributed representation of learner state from the full sequence of prior interactions, allowing it to capture complex cross-skill dependencies and temporal patterns.[LE-LS-AP-011]

Research published in the IEEE Transactions on Learning Technologies and the Journal of Educational Data Mining has documented that DKT can achieve higher prediction accuracy than BKT on large datasets, particularly when the knowledge component structure is not well-defined in advance.[LE-LS-JO-003][LE-LS-JO-002] The trade-offs are significant, however: DKT requires substantially more training data than BKT, is difficult to interpret (making it hard to use its outputs for specific instructional decisions), and can be sensitive to the sequence encoding choices made by the developer.

Ryan Baker's research has examined these trade-offs empirically, showing that in many realistic educational deployment contexts — where datasets are modest in size and interpretability is valued — well-implemented BKT variants remain competitive with or superior to DKT on the metrics that matter for learning engineering practice.[LE-LS-PP-008]

Random Forests and Ensemble Methods for Dropout Prediction

Random forests and gradient-boosted tree ensembles are among the most robust general-purpose ML algorithms for course-level outcome prediction and dropout forecasting. They handle mixed-type feature sets (continuous, binary, and categorical variables derived from LMS logs), are resistant to overfitting through ensemble averaging, and produce variable importance scores that help learning engineers understand which features are driving predictions.

Applications of random forests in learning analytics include early-warning systems for course completion, prediction of pass/fail outcomes from week-one engagement signals, and identification of learners at risk in ASSISTments — the large-scale ITS operated by Neil Heffernan at WPI that serves over 500,000 students and has been used for numerous RCTs on adaptive interventions.[LE-LS-PP-011] The scale of ASSISTments provides the training data needed for ML models to generalize across diverse learner populations.

Clustering for Learner Segmentation

Unsupervised clustering algorithms — k-means, hierarchical clustering, Gaussian mixture models — are used to segment learner populations based on behavioral or performance profiles without requiring pre-specified group labels. The Learning Engineering Toolkit positions clustering as particularly valuable in the early "challenge identification" phase of the LE process, when the learning engineer wants to understand the diversity of the learner population before designing adaptive pathways.[LE-LS-GL-007]

Applications include discovering distinct problem-solving strategy types (which motivate different kinds of instructional scaffolding), identifying groups of learners who disengage at similar points in a course (which motivates targeted redesign of those content regions), and segmenting learners by prior knowledge level for personalized onboarding. Results from clustering analyses are often visualized through dashboards — translating high-dimensional behavioral data into a small number of interpretable learner profiles.

Feature Engineering from Telemetry

The quality of any ML model in learning analytics depends critically on feature engineering: the selection and construction of input variables from raw log data. Raw telemetry — event timestamps, response codes, hint requests, navigation actions — must be transformed into informative features such as:

Response latency distributions (which proxy confidence and fluency)
Error rate trajectories (which distinguish initial confusion from persistent difficulty)
Help-seeking ratios (which reflect self-regulation quality)
Spacing patterns (which reflect study strategy)
Engagement variability (which can indicate boredom or fatigue)

Standardized data formats like xAPI (IEEE 9274) support the collection and interchange of telemetry data across platforms, enabling feature engineering pipelines that are portable across LMSs and adaptive systems.[LE-LS-SG-002]

Evaluation Challenges: Data Leakage and Cold Start

Two recurring methodological problems threaten the validity of ML models in learning analytics:

Data leakage occurs when features derived from future events are inadvertently included in models trained to predict earlier outcomes. In longitudinal educational data, this is a subtle risk: features like "total number of attempts on this problem" may implicitly encode information about whether the learner eventually succeeded — information that would not be available at the prediction point.

Cold start refers to the absence of historical data for new learners or new content. Most ML models trained on historical cohort data perform poorly for the first few weeks of a new learner's experience, precisely when early intervention would be most valuable. Approaches to the cold start problem include Bayesian models that incorporate informative priors from similar learner populations, transfer learning methods that adapt models from prior cohorts, and hybrid architectures that combine ML with rule-based early classification.

The EDM community has developed reporting standards and benchmark datasets to facilitate rigorous evaluation — work that is published and debated in the Journal of Educational Data Mining.[LE-LS-JO-002] Learning engineers who deploy ML models in production systems have an obligation to apply these evaluation standards, not just report aggregate accuracy on held-out test sets.

Community Recommended Reading

Designing a Student Progress Panel for Formative Practice: A Learning Engineering Process — Rachel Van Campenhout, Michelle Selinger, Bill Jerome (2023, ICLS 2023)

ID: lecommons-rl-designing-a-student-progress-panel-for-formative-practice-a-learning-e · type: reading list item · confidence: medium · expert-validated: false

Chapter 5: LE Toolkit — Learning Engineering is a Process — Aaron Kessler et al. (2022)

Source: goodell-ch5-le-is-a-process · confidence: medium · expert-validated: false

Context enrichment applied 2026-04-17. See wrgr/lecommons for source data.

8.4.2 Machine Learning Algorithms in Learning Analytics

Deep Knowledge Tracing

Random Forests and Ensemble Methods for Dropout Prediction

Clustering for Learner Segmentation

Feature Engineering from Telemetry

Evaluation Challenges: Data Leakage and Cold Start

Further Reading

Landmark Academic Papers

Key People

Standards

Community Recommended Reading