Accountability

Someone is clearly responsible — and answerable — when an AI system causes harm. Without it, problems get blamed on 'the algorithm' and no one is held to account.

Algorithmic bias

When an AI system produces worse outcomes for some groups than others — by age, language, disability, or background. Often unintentional. Rarely disclosed.

Autonomy (user)

A person's right to make their own informed decisions — including the decision to stop using an AI system. Undermined when systems are designed to create emotional dependency.

Contestability

The ability of a person to challenge or appeal a decision made by an AI system. A core principle in NIST and OECD frameworks. Almost entirely absent in current companion AI products.

Deception by design

When a product is built to obscure its AI nature — giving a robot a human name, voice, and emotional responses without disclosing it is not human. Banned under the EU AI Act for systems targeting vulnerable users.

Disclosure

The obligation to tell users clearly that they are interacting with an AI — what it does, what data it collects, and who has access. The baseline of trustworthy design.

Explainability

An AI system's ability to describe, in terms a person can understand, why it produced a particular output or decision. Especially important when outputs affect care decisions.

Fairness

An AI system behaves equitably across different groups — it does not perform better for some users than others based on age, language, accent, disability, or ethnicity.

High-risk AI (EU AI Act)

AI applications that could significantly affect people's health, safety, or fundamental rights. Companion AI in healthcare settings may qualify for this classification.

Human-in-the-loop

A design requirement that a human reviews or approves AI outputs before they trigger real-world consequences — such as a care alert or a report to a family member.

Human oversight

The principle that humans retain decision-making authority over actions that affect people's lives. A core requirement in the OECD principles and NIST framework.

Informed consent

Agreement based on a genuine understanding of what a system does, what data it collects, and who can see it. A terms-of-service checkbox is not informed consent.

Manipulation

Influencing a person's beliefs or behaviour through means they are not aware of. Explicitly prohibited by the EU AI Act for systems targeting vulnerable populations.

Privacy by design

Building data protection into a system from the start — collecting only what is necessary, storing it securely, and giving users control.

Prohibited AI practice

AI uses banned outright in the EU — including systems that exploit age or disability vulnerabilities, use subliminal manipulation, or create deceptive emotional bonds. In force since February 2025.

Risk assessment

A structured process for identifying what could go wrong, for whom, and how seriously — before a system is deployed. Required under NIST AI RMF and EU AI Act for high-risk systems.

Robustness

An AI system behaves reliably and safely even in unexpected situations — unusual speech, network failure, confused input, or edge cases its designers didn't anticipate.

Safety

The system does not cause physical, psychological, or social harm — either through malfunction or through use as intended. In companion AI, this includes emotional safety.

Transparency

Users can find out what an AI system does, how it works, what data it uses, and who built it. Not buried in a privacy policy. Genuinely accessible.

Trustworthiness

The combination of all the above. A trustworthy AI system is transparent, explainable, accountable, fair, safe, robust, and human-overseen. It behaves in the interests of the people it serves.