11.2.3 Ethical Use of AI in Learning Engineering

The ethical use of AI in learning engineering encompasses a cluster of principles that must be addressed before AI-powered systems are deployed with real learners: transparency, explainability, human oversight, and accountability.

Core Responsible AI Principles

Transparency means that learners, instructors, and institutional administrators can understand that an AI system is operating, what data it uses, and what types of decisions it makes or influences. The U.S. Department of Education's AI report established transparency as a foundational requirement — learners should not be subject to consequential AI decisions without knowledge that such systems are in operation. [LE-LS-GL-004]

Explainability extends transparency to the decision level: can a practitioner explain why a system recommended a particular intervention, flagged a learner as at-risk, or assigned a particular difficulty level? For ITS and adaptive systems, this typically means interpretable models are preferred over black-box deep learning when the decision is high-stakes. [LE-LS-GL-007]

Human oversight preserves instructor and learner agency. Algorithmic recommendations should be presented as suggestions that instructors can evaluate and override — not as automated actions that bypass human judgment. The AIED research community has extensively studied human-in-the-loop architectures that balance automation efficiency with instructor control. [LE-LS-CE-004]

Accountability addresses the question of who is responsible when an AI-powered learning system causes harm. The Journal of Learning Analytics has developed governance frameworks assigning accountability across developers, deploying institutions, and system operators. [LE-LS-JO-004]

The Practitioner's Obligation

IEEE ICICLE's standards require that LE practitioners document the AI components in their systems, their known limitations, and the governance processes in place to prevent misuse. [LE-LS-CO-001] The Generalizable LEAMM includes AI governance as a maturity indicator — mature organizations have ethics review processes embedded in design cycles, not applied post-hoc. [LE-LS-AP-013]

From the Learning Engineering Toolkit

The Toolkit takes up the ethics of AI and algorithms head-on. Jordan Richard Schoenherr writes that, as far as possible, the algorithms and designs inside a learning engineering solution should be transparent, with a design's affordances and anti-affordances both made plain to users. [LET-07]

The book reviews standards for ethical AI. It draws on UNESCO's recommendation on the ethics of artificial intelligence, which sets out four core values — among them respecting, protecting, and advancing human dignity, rights, and fundamental freedoms, and securing diversity and inclusion — that in turn shape ten design principles, such as human oversight and determination, transparency and explainability, and awareness and literacy. [LET-07] It likewise cites the IEEE's principles for ethically aligned design of trustworthy autonomous and intelligent systems — human rights, well-being, data agency, effectiveness, transparency, accountability, awareness of misuse, and designer competence; transparency, for instance, calls for the data-processing steps to be discoverable, and awareness of misuse calls for preventive measures that curb the risk of data being misused. [LET-07]

Because a learning engineer's read on a product's intended use cannot foresee every unintended one — each of which may raise fresh ethical questions — the Toolkit argues for ongoing ethical sense-making rather than a single compliance check. [LET-07] It also cautions that as adaptive and intelligent learning systems spread, learners need to be able to reach and make sense of their own data and the methods that produced it. [LET-07]

Chapter 17's SEEM-ED decision-making tool supports this for AI-driven designs, asking, under integrity, whether any deliberate or accidental errors or biases have been fixed and how, and whether the product or process has been described accurately enough for users to understand its features. [LET-17]