Caltech Research
Multimodal AI for Proactive Health Experiences
Led AI/ML research with Caltech postdoctoral researchers, combining computer vision, LLMs, and biosensor data to create proactive health experiences across robotics, smart home, and autonomous mobility.
Client
Independent Research Project, Caltech
Role
Product Designer / PM
Team
1 PM / 2 Post Docs
Period
6 months, 2025
Overview
Led an independent AI/ML research project with Caltech postdoctoral researchers, exploring proactive health experiences that detect and respond to changes in user health. Combined computer vision, LLMs, AI/ML, and biosensor data to translate health signals into service scenarios and interactions across robotics, smart home, and autonomous mobility.
Led independent AI/ML research with Caltech postdoctoral researchers.
Defined product strategy for AI-powered in-vehicle health monitoring.
Researched computer vision technologies and multimodal biosensors for driver health monitoring.
Planned AI-driven health monitoring experiences integrating vision, physiological sensing, and medical robotics.
Developed an early prototype bridging mobility and healthcare.
Challenge
While working on Mercedes-Benz’s AI-powered driver monitoring system, I identified the limitations of relying on digital sensing alone to understand a user’s physical condition. This led to a broader challenge: how might computer vision, biosensors, and AI work together to detect subtle health changes and enable more proactive support across different environments?
Objective
Develop a multimodal AI framework that combines computer vision, LLMs, AI/ML, and biosensor data to recognize changes in user health and translate them into adaptive service scenarios and interactions across robotics, smart home, and autonomous mobility.
Result
Defined a cross-environment AI experience model that translates health signals into proactive, context-aware interactions across robotics, smart home, and autonomous mobility, with the work advanced toward patent preparation.
More detail
Due to confidentiality, only an overview can be shown publicly. Additional context can be discussed briefly in person.