LEBOK Wiki
Prototype — not authoritative. A learning-engineering product under development, cloned from the lebok.wiki of record. Much content is AI-drafted and pending expert review.About & pedagogy →
📖

Authored from the Learning Engineering Toolkit — pending expert review

This page was authored on 2026-07-16 directly from the source text of the Learning Engineering Toolkit (Jim Goodell & Janet Kolodner (Eds.), 2023). Every factual claim carries an inline <cite> citation to the specific chapter it draws on; the full references are listed at the foot of the page. The prose is grounded in the primary source but has not yet been validated by a subject-matter expert. Use the Edit button to validate, correct, or expand.

Chapters: LET-11 (Chapter 11), LET-01 (Chapter 1)

9.4.3 Effectively Integrating User Feedback

Source:lebok.wiki/en/knowledge-area-9-lean-agile-methodologies-in-learning-engineering/9-4-challenges-and-considerations-for-lean-agile-in-learning-engineering/9-4-3-effectively-integrating-user-feedback
Retrieved:2026-04-17
Confidence:HIGH

  • Diverse User Feedback: Learners have diverse needs, and feedback can vary widely. Learning engineers must effectively prioritize user feedback, distinguishing between conflicting suggestions and aligning actions with educational objectives.

  • Agile Backlog Management: The agile backlog must reflect user needs, balancing immediate requests for improvements with longer-term objectives. Ensuring user feedback is appropriately incorporated into the backlog helps maintain user alignment while meeting the broader goals of the learning solution.

From the Learning Engineering Toolkit

Chapter 11 of the Learning Engineering Toolkit centers Lean-Agile on delivering the greatest possible value to the customer, and for learning engineering teams that customer takes in learners along with the people who support them [LET-11]. The opening Agile principle is about pleasing the customer through prompt and ongoing delivery, and the Manifesto prizes working with customers and adapting to change ahead of adhering to a fixed plan [LET-11]. Those commitments put user feedback at the heart of every iteration instead of treating it as a later add-on.

In Scrum, user needs enter the work by way of the product backlog, a ranked list whose items are each phrased as a story told from the standpoint of the user who stands to gain incremental value, concentrating on what needs doing rather than the method [LET-11]. The toolkit's recommended story template records the user, their situation, an aim, and the problem to solve, together with acceptance criteria that spell out when the work counts as finished and satisfactory for that user [LET-11]. In grooming, the Product Owner reviews those criteria with the team so the output tracks genuine user needs [LET-11]. At the sprint review the team shows off the working value produced, opening a regular moment to collect feedback [LET-11].

The chapter also emphasizes drawing feedback from each team member by way of retrospectives, which it advises pairing with Kanban as the one Lean-Agile pairing a learning engineering team ought to adopt even if it adopts nothing more [LET-11]. This ties into the wider learning engineering process, where the outcomes of each cycle circle back to shape the next round of design and refinement [LET-01]. Since learning is situated and learners in differing circumstances behave quite differently, folding end-user feedback into successive iterations helps confirm that solutions perform as intended across the varied settings where they are put to use [LET-01].

Sources from the Learning Engineering Toolkit

  1. [LET-11]Michelle Barrett & Jim Goodell (2023). Chapter 11: Lean-Agile Development Tools. In Jim Goodell & Janet Kolodner, Learning Engineering Toolkit (pp. 269–277). Routledge / Taylor & Francis. doi:10.4324/9781003276579
  2. [LET-01]Aaron Kessler, Scotty D. Craig, Jim Goodell, Dina Kurzweil & Scott W. Greenwald (2023). Chapter 1: Learning Engineering is a Process. In Jim Goodell & Janet Kolodner, Learning Engineering Toolkit (pp. 29–46). Routledge / Taylor & Francis. Open Access

Further Reading

Source: wrgr/lecommons — curated by the learning engineering community. Confidence: medium — lecommons-curated; not yet independently expert-validated in this context. To validate or challenge any item: use the Edit button on this page. Upgrading confidence from medium → high requires expert sign-off.

Landmark Academic Papers

  • Cognitive load during problem solving: Effects on learning — John Sweller (1988). Cognitive Science · doi:10.1207/s15516709cog1202_4 · ~4,000 citations · tier: foundational

    The founding paper of Cognitive Load Theory. Established that working memory limitations impose hard constraints on instruction design. CLT-derived principles (worked examples, split-attention, redundancy) are among the most widely replicated findings in educational psychology and are standard design rules in learning engineering. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-008 · confidence: medium · expert-validated: false

  • Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings — Ann L. Brown (1992). Journal of the Learning Sciences · doi:10.1207/s15327809jls0202_2 · ~2,800 citations · tier: foundational

    The most-cited paper in JLS history. Established design-based research as the canonical methodology for testing engineered learning interventions in authentic classroom settings. Defined the epistemological standards that learning engineering uses when validating systems in the wild. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-003 · confidence: medium · expert-validated: false

  • Improving students' help-seeking skills using metacognitive feedback in an intelligent tutoring system — Ido Roll, Vincent Aleven, Bruce M. McLaren et al. (2011). Learning and Instruction · doi:10.1016/j.learninstruc.2010.07.004 · ~600 citations · tier: highly_cited

    Demonstrated that LE methods can be applied beyond domain knowledge to model and improve learner metacognition and self-regulation. Proved that intelligent tutors can engineer help-seeking behavior, not just subject-matter proficiency — broadening the scope of what LE can target. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-007 · confidence: medium · expert-validated: false

  • The doer effect at scale: Investigating correlation and causation across seven courses — Rachel Van Campenhout, Bill Jerome, Benny G. Johnson (2023). Proceedings of LAK23: 13th International Learning Analytics and Knowledge Conference · doi:10.1145/3576050.3576088 · ~80 citations · tier: contemporary

    Leveraged telemetry from large-scale online platforms to isolate the causal effect of active practice (doing) vs. passive reading on learning outcomes across seven courses. Provides large-sample empirical validation of one of learning engineering's core design principles at scale. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-012 · confidence: medium · expert-validated: false

  • Generalizable Learning Engineering Adoption Maturity Model — Shawn Blake-Plock, Scotty D. Craig, Emily Czerwinski et al. (2025). Journal of Learning Engineering (preprint/proceedings) · tier: contemporary

    Defines a multi-level organizational capability model for assessing and advancing an institution's LE practice. Provides a structured framework for moving from ad-hoc instructional design toward systematic, data-driven learning engineering — the field's primary tool for organizational self-assessment. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-013 · confidence: medium · expert-validated: false

Policy, Reports & Grey Literature

  • The Job of a College President — Herbert A. Simon (1967). Educational Record (American Council on Education) · link

    The founding document of learning engineering. Simon coined the term 'learning engineer' and made the case that teaching effectiveness is a distinct, learnable expertise grounded in cognitive science — not a byproduct of subject-matter expertise. The philosophical foundation for everything that followed. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-001 · confidence: low · expert-validated: false

  • Online Education: A Catalyst for Higher Education Reforms — Karen Willcox, Sanjay Sarma, Philip Lippel (2016). MIT Online Education Policy Initiative (MIT OEPI) · link

    Re-injected 'learning engineer' into higher education discourse at the moment MOOCs reached 58M+ global students. Recommended universities create dedicated learning engineering roles. Catalyzed institutional adoption of LE language and practices across US higher education. Widely cited as the document that triggered the modern LE movement. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-002 · confidence: low · expert-validated: false

  • Learning Engineering Toolkit: Evidence-Based Practices from the Learning Sciences, Instructional Design, and Beyond — Jim Goodell, Janet Kolodner (2023). Taylor & Francis / Routledge · link

    The field's primary practitioner handbook. Frames LE as a verb — an iterative, evidence-based problem-solving process — not a technology set or a job title. Covers lean-agile methods, HCI design, data instrumentation, motivation modeling, and predictive analytics. Explicitly technology-agnostic: the principles apply equally to AI-driven software and low-tech interventions. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-007 · confidence: low · expert-validated: false

  • Learning Engineering for Online Education: Theoretical Contexts and Design-Based Examples — Chris Dede, John Richards, Bror Saxberg (2019). Routledge · link

    First book-length treatment of learning engineering applied to digital and distributed contexts. Saxberg's formulation — LE as applying learning science at 'massive, affordable, data-rich scale' — became widely adopted. Documents how online architectures enable continuous telemetry that peer-reviewed journals document years later. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-008 · confidence: low · expert-validated: false

  • Learning Engineering Adoption Maturity Model (LEAMM) — Shawn Blake-Plock, Scotty D. Craig, Emily Czerwinski et al. (2025). IEEE ICICLE · link

    Organizational capability matrix allowing enterprises to measure fidelity of their LE adoption across human-centered design, learning science integration, and data-driven iteration dimensions. Provides a structured pathway from ad-hoc instructional design to mature learning engineering practice. The field's primary organizational self-assessment tool. Source: lecommons/landscape/data/grey_literature.json · ID: LE-LS-GL-009 · confidence: low · expert-validated: false

Key People

  • John Sweller (profile), University of New South Wales (1946–present) — Developer of Cognitive Load Theory

    Formalized Cognitive Load Theory (CLT) distinguishing intrinsic, extraneous, and germane load Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-004 · confidence: medium · expert-validated: false

  • Ann L. Brown (profile), University of California, Berkeley (1943–1999) — Pioneer of design-based research and collaborative learning

    Established design-based research (DBR) as the canonical methodology for testing learning interventions in authentic contexts Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-006 · confidence: medium · expert-validated: false

  • Bror Saxberg (profile), Formerly Chan Zuckerberg Initiative / Kaplan; founder of LearningForge (active 2000s–present) — Learning science practitioner; LE thought leader; industry-academic bridge

    Led learning science at Chan Zuckerberg Initiative, framing LE as 'precision education' analogous to precision medicine Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-009 · confidence: medium · expert-validated: false

  • Jim Goodell (profile), Quality Information Partners (QIP); IEEE ICICLE (active 2010s–present) — LE standards architect; Learning Engineering Toolkit co-editor; ICICLE community builder

    Co-edited the Learning Engineering Toolkit (Taylor & Francis, 2023), the field's primary practitioner handbook Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-012 · confidence: medium · expert-validated: false

  • Karen Willcox (profile), University of Texas at Austin, Oden Institute (active 2000s–present) — MIT OEPI lead; LE in higher education advocate

    Led MIT's Online Education Policy Initiative (OEPI), producing the 2016 catalyst report that re-introduced 'learning engineer' to higher education discourse Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-013 · confidence: medium · expert-validated: false

Organizations, Conferences & Journals

  • IEEE ICICLE — Industry Consortium on Learning Engineering (consortium) · link

    Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CO-001 · confidence: medium · expert-validated: false

  • HCII — Human-Computer Interaction Institute, Carnegie Mellon (research_center) · link

    Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CO-005 · confidence: medium · expert-validated: false

  • International Society of the Learning Sciences Annual Meeting (conference) · link

    Source: lecommons/landscape/data/organizations.json · ID: LE-LS-CE-005 · confidence: medium · expert-validated: false

  • Journal of the Learning Sciences (journal) · link

    Source: lecommons/landscape/data/organizations.json · ID: LE-LS-JO-001 · confidence: medium · expert-validated: false

  • Journal of Learning Engineering (journal) · link

    Source: lecommons/landscape/data/organizations.json · ID: LE-LS-JO-006 · confidence: medium · expert-validated: false

Programs & Initiatives

  • Stanford GSE — M.S. Learning Design and Technology (LDT) (PC) · link

    Residential master’s focused on designing and evaluating technology-enhanced learning environments using learning sciences, design methods, and empirical study—one of the clearest Stanford pathways aligned with learning engineering practice (even when the degree title says LDT, not LE). Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-085 · confidence: medium · expert-validated: false

Lecommons enrichment applied 2026-04-17. All items pending expert validation. See wrgr/lecommons for source data and curation methodology.

←9.4.2 Managing Team and End-User Alignment
Glossary of Terms→