7.4.5 LRS Pipeline Development – Software Development

This subtopic covers standards-based technical skills for creating software and systems multi-stage xAPI-complaint LRS pipelines, i.e. stages that transform raw event data to transactional data to summary assertions and inferences or formats supporting analytics and decision-making use cases.

Refer to the Software Engineering Body of Knowledge.

From the Learning Engineering Toolkit

The Learning Engineering Toolkit frames a data pipeline as the combined software and hardware that captures, processes, and stores data [LET-05]. Building an LRS pipeline therefore means engineering the software stages that move learning events from capture through to analysis-ready form. In practice, xAPI supplies three main parts a developer works with: coded statements written into the learning technology, a learning record store implemented to catch and validate those statement outputs, and xAPI Profiles that keep data elements and terminology consistent across experiences and systems [LET-13].

The toolkit describes the multi-stage transformation such pipelines perform. Raw log data may first be collected in a triplestore (a subject–predicate–object statement), then transformed and cleansed into another triplestore for analytical processing; facts from those data may then be moved into a normalized operational data store for learner progress tracking, and transformed once more into a denormalized data warehouse for enterprise analytics and reporting [LET-14]. This staged design matches the pipeline's goal of turning raw event data into transactional data and then into summary assertions and inferences suited to decision-making.

Software architecture choices shape whether the pipeline can keep up. Applications that produce high volumes of data continuously — an intelligent tutoring system with clickstream monitoring might generate thousands of messages per second — can overwhelm traditional point-to-point designs, so a streaming data architecture is required to process data immediately [LET-14]. Many such systems, and xAPI itself, use a REST architectural style over HTTP for storing, retrieving, and exchanging data [LET-14]. The toolkit notes that teams building these platforms will require experienced data engineers to help design the platforms and pipelines that gather, move, hold, and reshape the data [LET-14]. Because instrumentation can be an iterative process, developers should also expect to revise pipeline stages as new questions emerge rather than assuming an initial design is final [LET-05].

Further Reading

Source: wrgr/lecommons — curated by the learning engineering community. Confidence: medium — lecommons-curated; not yet independently expert-validated in this context. To validate or challenge any item: use the Edit button on this page. Upgrading confidence from medium → high requires expert sign-off.

Landmark Academic Papers

• Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge — Albert T. Corbett, John R. Anderson (1994). User Modeling and User-Adapted Interaction · doi:10.1007/BF01099821 · ~2,500 citations · tier: foundational

  The formal presentation of Bayesian Knowledge Tracing (BKT) — the probabilistic model for tracking per-student, per-skill mastery. BKT remains the most widely deployed student model in production ITS and learning platforms worldwide, forming the core of personalization algorithms. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-011 · confidence: medium · expert-validated: false

• Computer Support for Knowledge-Building Communities — Marlene Scardamalia, Carl Bereiter (1994). Journal of the Learning Sciences · doi:10.1207/s15327809jls0303_3 · ~1,800 citations · tier: foundational

  Engineered the socio-cognitive framework for collaborative digital learning platforms. Established that effective online learning environments must support collective knowledge construction. Foundational for CSCL platform design, modern learning management systems, and collaborative LE tools. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-004 · confidence: medium · expert-validated: false

• The State of Educational Data Mining in 2009: A Review and Future Visions — Ryan S. J. d. Baker, Kalina Yacef (2009). Journal of Educational Data Mining · link · ~1,500 citations · tier: foundational

  Codified Educational Data Mining as a distinct discipline, distinguishing it from learning analytics and positioning it with its own methods, journals, and research questions. Established the taxonomy of EDM methods (prediction, clustering, knowledge discovery, relationship mining, distillation for human judgment) still used today. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-005 · confidence: medium · expert-validated: false

• Intelligent tutoring systems — Albert T. Corbett, Kenneth R. Koedinger, John R. Anderson (1997). Handbook of Human-Computer Interaction · ~1,200 citations · tier: foundational

  Documented the full architecture of Bayesian Knowledge Tracing and the Cognitive Tutor student model. The chapter that operationalized real-time student modeling and served as the blueprint for adaptive learning software for the next two decades. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-002 · confidence: medium · expert-validated: false

• The doer effect at scale: Investigating correlation and causation across seven courses — Rachel Van Campenhout, Bill Jerome, Benny G. Johnson (2023). Proceedings of LAK23: 13th International Learning Analytics and Knowledge Conference · doi:10.1145/3576050.3576088 · ~80 citations · tier: contemporary

  Leveraged telemetry from large-scale online platforms to isolate the causal effect of active practice (doing) vs. passive reading on learning outcomes across seven courses. Provides large-sample empirical validation of one of learning engineering's core design principles at scale. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-012 · confidence: medium · expert-validated: false

Policy, Reports & Grey Literature

• Online Education: A Catalyst for Higher Education Reforms — Karen Willcox, Sanjay Sarma, Philip Lippel (2016). MIT Online Education Policy Initiative (MIT OEPI) · link

  Re-injected 'learning engineer' into higher education discourse at the moment MOOCs reached 58M+ global students. Recommended universities create dedicated learning engineering roles. Catalyzed institutional adoption of LE language and practices across US higher education. Widely cited as the document that triggered the modern LE movement. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-002 · confidence: low · expert-validated: false

• High-Leverage Opportunities for Learning Engineering — Ryan S. Baker, Ulrich Boser, Allison Shelley (2021). University of Pennsylvania Center for Learning Analytics · link

  Synthesized a 2020 convening of 100+ academics, policymakers, and practitioners to define ten strategic priorities for the field in three domains: Better LE Infrastructure (shared R&D architectures, A/B testing at scale, reusable algorithmic components), Supporting Human Processes (teacher dashboards, predictive advising), and Better Learning Technologies (algorithmic equity, complex-skills measurement). The field's most comprehensive contemporary roadmap. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-003 · confidence: low · expert-validated: false

• Artificial Intelligence and the Future of Teaching and Learning: Insights and Recommendations — U.S. Department of Education, Office of Educational Technology (2023). U.S. Department of Education · link

  First major federal policy document addressing AI in education with explicit civil-rights framing. Documented algorithmic bias in e-proctoring systems (facial recognition failures for darker-skinned, transgender, and neurodivergent students). Mandated algorithmic fairness and data representation requirements that now set the ethical floor for learning engineering practice. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-004 · confidence: low · expert-validated: false

• EDUCAUSE Horizon Report: Teaching and Learning Edition — EDUCAUSE (2017). EDUCAUSE · link

  The annual predictive literature for edtech adoption across global higher education. Tracks short-, mid-, and long-term adoption horizons. Documents the gradual mainstreaming of LE concepts (analytics, adaptive systems, AI tutors) from forecast to widespread adoption across the 2017–2024 arc. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-005 · confidence: low · expert-validated: false

• NMC Horizon Report: 2017 Higher Education Edition — Samantha Adams Becker, Malcolm Cummins, Annie Davis et al. (2017). The New Media Consortium / EDUCAUSE · link

  Identified adaptive learning technologies and data-driven personalization as short-term adoption trends in higher education. Forecast the long-term restructuring of degrees around competency measurement. Part of the annual Horizon series that documented the field's gradual integration into mainstream institutional practice. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-006 · confidence: low · expert-validated: false

Key People

• Kenneth R. Koedinger (profile), Carnegie Mellon University, HCII (active 1988–present) — Co-originator of learning engineering as a named field; Cognitive Tutor pioneer; DataShop founder

  Led development of Cognitive Tutors deployed in thousands of schools; co-founded Carnegie Learning Inc. Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-005 · confidence: medium · expert-validated: false

• Ryan S. J. d. Baker (profile), University of Pennsylvania, Graduate School of Education (active 2000s–present) — EDM pioneer; director of Penn Center for Learning Analytics; high-leverage LE policy voice

  Codified Educational Data Mining as a formal discipline with Baker & Yacef 2009 (1,500+ citations) Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-008 · confidence: medium · expert-validated: false

• Neil Heffernan (profile), Worcester Polytechnic Institute (active 2000s–present) — Creator of ASSISTments; large-scale LE experimentation pioneer

  Created ASSISTments, a free ITS serving 500,000+ students and 20,000+ teachers Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-011 · confidence: medium · expert-validated: false

• Jim Goodell (profile), Quality Information Partners (QIP); IEEE ICICLE (active 2010s–present) — LE standards architect; Learning Engineering Toolkit co-editor; ICICLE community builder

  Co-edited the Learning Engineering Toolkit (Taylor & Francis, 2023), the field's primary practitioner handbook Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-012 · confidence: medium · expert-validated: false

• Karen Willcox (profile), University of Texas at Austin, Oden Institute (active 2000s–present) — MIT OEPI lead; LE in higher education advocate

  Led MIT's Online Education Policy Initiative (OEPI), producing the 2016 catalyst report that re-introduced 'learning engineer' to higher education discourse Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-013 · confidence: medium · expert-validated: false

Organizations, Conferences & Journals

• The Simon Initiative — Carnegie Mellon University (research_center) · link

  Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CO-002 · confidence: medium · expert-validated: false

• Society for Learning Analytics Research (academic_society) · link

  Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CO-003 · confidence: medium · expert-validated: false

• Penn Center for Learning Analytics (research_center) · link

  Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CO-004 · confidence: medium · expert-validated: false

• International Educational Data Mining Society (academic_society) · link

  Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CO-006 · confidence: medium · expert-validated: false

• International Conference on Intelligent Tutoring Systems (conference) · link

  Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CE-001 · confidence: medium · expert-validated: false

• International Conference on Learning Analytics and Knowledge (conference) · link

  Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CE-002 · confidence: medium · expert-validated: false

Programs & Initiatives

• Educational Data Mining (EDM) (CE) · link

  Annual international conference and journal. Primary venue for learning analytics and EDM research. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-021 · confidence: medium · expert-validated: false

• Learning Analytics & Knowledge (LAK) (CE) · link

  Premier analytics in education conference. Strong on data infrastructure and equity. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-023 · confidence: medium · expert-validated: false

• Society for Learning Analytics Research (SoLAR) (CO) · link

  International nonprofit dedicated to research, practice, and policy of learning analytics. Home of LAK conference and companion journal. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-041 · confidence: medium · expert-validated: false

• Penn Center for Learning Analytics (UPenn GSE) (PC) · link

  GSE hub for learning analytics and AI in education research, including work that explicitly intersects learning engineering, educational data mining, and design of data-informed learning systems. Related graduate offerings include the online M.S. in Learning Analytics and Artificial Intelligence. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-084 · confidence: medium · expert-validated: false

• Penn Center for Learning Analytics (CO) · link

  Research center directing educational data mining and learning engineering policy work, led by Ryan Baker. Produced 'High-Leverage Opportunities for Learning Engineering' (2021) and several landmark EDM papers. Baker is editor of the Journal of Educational Data Mining and a primary organizer of the EDM conference. Major hub connecting academic EDM research to policy influence and practitioner adoption. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-108 · confidence: medium · expert-validated: false

• Journal of Educational Data Mining (CO) · link

  Open-access journal dedicated to algorithms and methodologies for analyzing educational data: knowledge tracing, automated feature engineering, predictive student modeling, and behavioral analysis. Baker & Yacef's founding codification of EDM (2009) published here. Premier venue for computational LE methods. Diamond open access — no fees for authors or readers. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-110 · confidence: medium · expert-validated: false

Lecommons enrichment applied 2026-04-17. All items pending expert validation. See wrgr/lecommons for source data and curation methodology.