Learning engineers draw on a broad taxonomy of data types to characterize learner states, diagnose system performance, and drive iterative design improvements. Understanding what each data type measures — and what it cannot reveal — is a prerequisite for constructing valid data-informed design cycles. Ryan Baker and Kalina Yacef's foundational codification of educational data mining identified five major analytic tasks (prediction, clustering, knowledge discovery, relationship mining, and distillation for human judgment) that each place different demands on the underlying data [LE-LS-AP-005], implying that data collection strategies must be planned in light of the design questions the team needs to answer.
Quantitative Interaction and Assessment Data
Quantitative data in learning engineering encompasses the rich interaction logs generated by digital learning platforms: time on task, number of attempts per problem, hint requests, error patterns, completion rates, navigation sequences, and assessment scores. These data streams allow engineers to apply probabilistic models such as Bayesian Knowledge Tracing, which uses the pattern of correct and incorrect responses across practice items to estimate the probability that a learner has acquired a given knowledge component. [LE-LS-AP-011] BKT and its successors make it possible to infer latent learner states — mastery, confusion, disengagement — from observable response data, converting raw interaction logs into actionable signals for adaptive systems and human reviewers alike.
At scale, quantitative interaction data has enabled causal analysis of fundamental design principles. The doer effect research used telemetry from seven online courses to isolate the causal contribution of active practice to learning gains, providing an empirical warrant for one of learning engineering's most consequential design commitments: that engagement with practice problems, not passive exposure to content, is the primary driver of durable learning. [LE-LS-AP-012] Kenneth Koedinger's DataShop infrastructure at Carnegie Mellon made this kind of large-scale quantitative analysis possible by aggregating authentic classroom interaction data from thousands of students across multiple platforms and subject areas. [LE-LS-PP-005]
Qualitative, Physiological, and Contextual Data
Qualitative data — including learner interviews, think-aloud protocols, observation notes, and open-ended survey responses — captures dimensions of the learning experience that quantitative metrics systematically miss. A learner may complete all required problems with high accuracy while experiencing significant frustration with the interface design, a frustration that will only be visible in verbal report or behavioral observation. Cognitive Load Theory research has long used a mix of quantitative performance measures and self-report ratings to distinguish between intrinsic load (inherent content difficulty), extraneous load (load imposed by poor design), and germane load (cognitive effort contributing to schema formation). [LE-LS-AP-008] Learning engineers who rely exclusively on performance metrics risk optimizing for outcomes that are measurable while overlooking design problems that undermine engagement, motivation, and transfer.
Emerging data streams extend the taxonomy further. Physiological measures — eye tracking, galvanic skin response, EEG-based attention indicators — offer real-time estimates of cognitive and affective states that neither performance logs nor self-report can fully capture. Ryan Baker's research on detecting gaming-the-system, off-task behavior, and boredom from ITS interaction logs demonstrated that behavioral pattern recognition can surface affective states from existing interaction data without requiring additional sensor hardware. [LE-LS-PP-008] Contextual data — including information about the learning environment, instructor practices, institutional constraints, and learner demographics — is equally essential for interpreting quantitative findings correctly. A performance gap that appears uniform in aggregate data may, when disaggregated by demographic group, reveal an equity issue that demands a design response. [LE-LS-GL-007]