Jump to:Model InventoryGovernance GapsISO 42001NIST AI RMF
Back to resources

Explainer

Inside Social Companion Robots

This is an educational resource based on a hypothetical product. It does not describe any specific device or company.

By Ezra Schwartz

Living documentThis resource is updated as new research and community input emerges.
Share feedbackConnect on LinkedIn

Last updated: 15 July 2026, 23:43 UTC/15 July 2026, 18:43 CST

AI Governance Analysis

Model inventory, risk assessment, and compliance gaps

Critical Risk
7

Models with no documented human oversight or explainability

High Risk
6

Models with incomplete audit trail or bias testing

Third-Party
12

Supply chain dependencies with separate data retention policies

Indefinite Storage
3

Biometric data with no deletion policy — cannot be revoked

Model Inventory

Risk level
Vendor type

Wake Word Detection

HIGH

Always-on listening for trigger phrase

Vendor
Amazon Alexa SDK3rd party
Versionv3.2
Data RetentionNot stored (processed on-device)
Update FrequencyQuarterly
Explainable
No
Human Oversight
Automated
Data Inputs
Raw audio stream (continuous)
Vendor DocumentationAPI / Product Docs

Speech-to-Text Engine

HIGH

Transcribe voice to text

Vendor
Google Cloud Speech API3rd party
Versionv2
Data Retention30 days (Google servers)
Update FrequencyMonthly
Explainable
No
Human Oversight
Automated
Data Inputs
Audio clips (post-wake)
Vendor DocumentationAPI / Product Docs

Face Recognition Model

CRITICAL

Identify user via facial biometrics

Vendor
AWS Rekognition3rd party
VersionLatest
Data RetentionIndefinite (facial signature stored)
Update FrequencyContinuous
Explainable
No
Human Oversight
Automated
Data Inputs
Video framesFacial landmarks
Vendor DocumentationModel Card

Voiceprint Identification

CRITICAL

Match voice to user profile

Vendor
Nuance Communications3rd party
Versionv4.1
Data RetentionIndefinite (voiceprint stored)
Update FrequencyAnnual
Explainable
No
Human Oversight
Automated
Data Inputs
Voice audioVocal characteristics
Vendor DocumentationAPI / Product Docs

Fall Detection CNN

HIGH

Detect falls from video

Vendor
Custom (OpenCV + MobileNet)
Versionv2.3
Data Retention7 days video / 90 days alerts
Update FrequencyQuarterly
Explainable
Yes
Human Oversight
Required
Data Inputs
Video streamPose estimation data
Vendor DocumentationOpen Source / Paper

Activity Recognition

MEDIUM

Identify daily activities

Vendor
Microsoft Azure Custom Vision3rd party
Versionv3.0
Data Retention14 days
Update FrequencyMonthly
Explainable
Yes
Human Oversight
Automated
Data Inputs
Video framesObject detection
Vendor DocumentationTransparency Note

Facial Expression Analysis

HIGH

Read emotional state from face

Vendor
Affectiva Emotion AI3rd party
Versionv6.0
Data Retention30 days
Update FrequencyQuarterly
Explainable
No
Human Oversight
Automated
Data Inputs
Facial landmarksMicro-expressions
Vendor DocumentationAPI / Product Docs

Voice Biomarker Analysis

CRITICAL

Detect health changes in voice

Vendor
Sonde Health API3rd party
Versionv2.1
Data Retention180 days (PHI classification)
Update FrequencyMonthly
Explainable
No
Human Oversight
Required
Data Inputs
Voice audioVocal tremorSpeech patterns
Vendor DocumentationAPI / Product Docs

Sentiment Analysis

MEDIUM

Detect emotional tone

Vendor
OpenAI Moderation API3rd party
VersionLatest
Data Retention30 days
Update FrequencyContinuous
Explainable
Yes
Human Oversight
Automated
Data Inputs
Transcribed textVoice audio
Vendor DocumentationModel Card

Intent Classification

HIGH

Understand user requests

Vendor
Anthropic Claude3rd party
VersionClaude 3.5 Sonnet
Data RetentionPer Anthropic policy (unspecified)
Update FrequencyContinuous
Explainable
Yes
Human Oversight
Automated
Data Inputs
Transcribed textConversation context
Vendor DocumentationModel Card

Large Language Model

CRITICAL

Generate conversational responses

Vendor
OpenAI GPT-43rd party
Versiongpt-4-turbo
Data RetentionPer OpenAI policy (30 days minimum)
Update FrequencyContinuous
Explainable
No
Human Oversight
Required
Data Inputs
User textContext memoryMedical historyPreferences
Vendor DocumentationModel Card

Episodic Memory Model

CRITICAL

Remember personal details across sessions

Vendor
Pinecone (vector DB) + Custom3rd party
VersionCustom v1.0
Data RetentionIndefinite (long-term memory)
Update FrequencyContinuous
Explainable
Yes
Human Oversight
Required
Data Inputs
All conversationsActivity logsPreferences
Vendor DocumentationAPI / Product Docs

Medical Record Integration

CRITICAL

Load electronic health records into context

Vendor
Epic FHIR API3rd party
VersionR4
Data RetentionPer HIPAA (minimum 6 years)
Update FrequencyReal-time sync
Explainable
Yes
Human Oversight
Required
Data Inputs
Electronic Health Records (PHI)
Vendor DocumentationAPI / Product Docs

Behavioral Pattern Matching

HIGH

Detect deviations from behavioural baseline

Vendor
Custom (TensorFlow)
Versionv1.5
Data Retention365 days (rolling)
Update FrequencyWeekly retraining
Explainable
Yes
Human Oversight
Required
Data Inputs
Activity logsConversation frequencySleep patterns
Vendor DocumentationOpen Source / Paper

Emergency Detection

CRITICAL

Identify distress signals in speech

Vendor
Custom (BERT fine-tuned)
Versionv2.0
Data Retention90 days
Update FrequencyMonthly
Explainable
Yes
Human Oversight
Required
Data Inputs
Text transcriptsAudio toneFall detection output
Vendor DocumentationOpen Source / Paper

Governance Gaps Identified

  • 7 critical-risk models with permanent biometric retention (voiceprint, facial signature) — policy: indefinite, no expiration or deletion mechanism documented
  • 12 third-party dependencies in the AI — several carry no documented policy (listed as 'per vendor policy'), making impossible
  • Limited : 7 of 15 models operate as — outputs cannot be interpreted or contested by the person affected
  • Automated decision-making without in 8 models — including emotion detection, intent classification, and facial expression analysis affecting a
  • No documented — no analysis across age, ethnicity, language, or health status has been conducted or published
  • No implemented — system decisions, data accesses, and model outputs are not logged in a form available for review or
  • absent — a single terms-of-service agreement covers all data types; no granular per-category consent exists

Requirements

  • No AI system impact assessment documented for this deployment context or target population
  • Stakeholder identification incomplete — the primary (older adults, people with cognitive impairment) is not represented in any documented design or risk process
  • partially maintained — technical risks logged, but ethical, social, and autonomy-related risks absent
  • No procedure documented — no defined process for when the system causes harm, fails, or behaves unexpectedly
  • not implemented — requires continuous monitoring and a traceable record of system decisions
  • does not meet requirements — no granular consent per data type, no right to withdraw, no data deletion mechanism
  • Third-party not fully documented — vendor data handling obligations under (Third-party and Customer Relationships) are unmet for 9 of 15 vendors

Requirements

  • not assessed — no documented evaluation of validity, reliability, safety, or fairness across the system's components
  • not conducted — no analysis across the this system serves
  • Transparency partially implemented — some model purposes are disclosed; autonomy level and are not communicated to users
  • structure absent — no defined roles, escalation paths, or redress mechanisms for AI system failures
  • not monitored — no documented process for detecting when model behaviour changes over time, particularly relevant for emotion and behaviour models
  • not conducted — no independent review of system outputs for bias, accuracy, or disparate impact

A note on these examples

The AI services and models listed in this inventory are real, commercially available products used for illustrative purposes only. They were chosen to reflect the kinds of components commonly found in AI companion systems — not because I tested, evaluated, or vetted them. I have no relationship with any of these vendors.

The goal is to give readers a realistic exercise: what would a model inventory actually look like? What documentation can you find? What's missing? You'll notice that some vendors publish formal model cards or responsible AI documentation, while others offer little more than marketing materials. That gap is itself part of what this page is meant to surface.

This is not a product guide. Do not interpret any entry here as a recommendation.

Note: This analysis applies and the to a representative AI companion system. requires organisations to conduct an , maintain a , implement an , and document procedures before deployment. The requires evaluation, , and ongoing monitoring. The governance gaps identified here — absent , undocumented obligations, and no analysis for a — reflect patterns common across eldercare AI products currently on the market.