4.7.3 Ensuring Evidence-Based Practices

This topic covers methods to ensure evidence-based practices, including but not limited to:

• Data Collection and Evaluation: The engineering approach in learning engineering is supported by rigorous data collection and evaluation practices that inform evidence-based decisions.

• Integration with Learning Sciences: Engineering foundations in learning engineering are tightly integrated with learning science principles, ensuring that design decisions are rooted in cognitive, social, and motivational research.

  ---

From the Learning Engineering Toolkit

Engineering is the use of creativity and science to solve problems, and learning engineering is the use of the learning sciences to solve problems creatively for learners and for learning [LET-04]. Even though learning engineering rests on the science of learning, it does not stop at mechanically applying what has already been found; like any engineering work, it calls on practitioners to ask which science stands behind an approach and how that science can be put to work to build robust learning environments [LET-04].

Anchoring practice in evidence also means reaching past settled findings. Learning engineers take up fresh experimentation and gather and analyze data to pinpoint where a current approach falls short for particular learners and to check whether alternative solutions actually work [LET-04]. A crucial task for any learning engineer is telling fact from fiction—separating what has been shown to work from what remains unknown [LET-04].

Instead of venturing into new territory the way basic research does, sound learning engineering builds on a broad set of tested, well-established techniques drawn from earlier work with well-replicated results [LET-04]. Piotr Mitros's original Open edX design, for instance, concentrated on a small number of proven learning approaches known to suit the courses at hand [LET-04]. Engineers may also treat data differently from research scientists: where some studies screen outliers out as noise, engineers look hard at outlier data and at the specific conditions under which a system is prone to fail [LET-04].

Testability is a principle shared across every engineering domain, and the platform enabled A/B testing of learning activities—randomly assigning alternatives and collecting data to see which performed best [LET-04]. A professional engineer, through education and training, is equipped to bring the scientific method to bear on analyzing and solving engineering problems [LET-04].

Further Reading

Source: wrgr/lecommons — curated by the learning engineering community. Confidence: medium — lecommons-curated; not yet independently expert-validated in this context. To validate or challenge any item: use the Edit button on this page. Upgrading confidence from medium → high requires expert sign-off.

Landmark Academic Papers

• Design principles for virtual humans in educational technology environments — Scotty D. Craig, Nicolaus L. Schroeder (2018). International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0148-y · ~150 citations · tier: contemporary

  Synthesized human-factors engineering principles for designing pedagogical agents. Showed that voice, appearance, and persona of virtual instructors systematically affect learner cognitive load and perceived credibility — establishing that LE must integrate psychological design alongside algorithmic design. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-009 · confidence: medium · expert-validated: false

• The voice quality of pedagogical agent impacts learning and agent perceptions — Ryan F. Siegle, Scotty D. Craig (2024). Journal of Computer Assisted Learning · doi:10.1111/jcal.12997 · ~20 citations · tier: contemporary

  Empirically demonstrated that voice quality (prosody, naturalness, warmth) of a pedagogical agent significantly affects both learning outcomes and learner perception. Illustrates the meticulous human factors engineering required in modern AI-driven learning systems. Source: lecommons/landscape/data/papers.json · ID: LE-LS-AP-010 · confidence: medium · expert-validated: false

Key People

• Scotty D. Craig (profile), Arizona State University (active 2000s–present) — Human factors researcher; virtual humans and pedagogical agents specialist

  Systematically investigated design principles for virtual humans and pedagogical agents in learning environments Source: lecommons/landscape/data/people.json · ID: LE-LS-PP-014 · confidence: medium · expert-validated: false

Programs & Initiatives

• Human Factors and Ergonomics Society (CO) · link

  Professional society for human factors and ergonomics. Publishes Human Factors journal and organizes the annual HFES conference. The Training Systems technical group directly addresses LE-relevant human factors in instructional systems. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-133 · confidence: medium · expert-validated: false

• INCOSE Systems Engineering Body of Knowledge (SEBoK) (CO) · link

  Open wiki-based body of knowledge for systems engineering. Includes the Human Systems Integration knowledge area. The SE process framework (requirements, architecture, integration, verification) is the engineering process backbone that LE adapts for learning system development. Source: lecommons/site/src/data/programs_people_registry.json · ID: LE-PP-134 · confidence: medium · expert-validated: false

Lecommons enrichment applied 2026-04-17. All items pending expert validation. See wrgr/lecommons for source data and curation methodology.