nytlabs

We all experience the world in a highly personalized fashion. And as the physical world becomes increasingly digital, computing is becoming more connected to our physical selves.

We've seen the rapid development of "natural user interfaces" that invoke the digital world by recognizing our voices, our gestures and even our faces, creating a more seamless integration of computers into our everyday environments. Additionally, a flurry of new consumer health and lifestyle devices that measure anything from how much we move to how deeply we sleep are evidence that biometric data are becoming accessible as a means to reflect about our personal wellbeing.

The R&D Lab's interactive mirror is a platform that we've designed to explore how the relationship between information and the self is evolving and how media content from The New York Times and others might play a part.

By using a special semi-reflective glass surface, the users of the mirror are able to see both a normal reflection of the real world as well as overlaid, high-contrast graphics. We've dubbed this "augmented reflection". Conceptually, the idea is that our mirror can reveal the halos of data around real-world objects, including ourselves.

Envisioned as a key fixture in your home, the mirror uses face recognition to call up personalized data, including health stats, a calendar, news feeds, and other information relevant to your morning routine. Voice commands switch between views, and gestures (via an embedded Kinect) activate content, including fullscreen video messages from other mirror users. An RFID-enabled shelf responds to objects that are placed on it, such as medications and personal care products, revealing personalized data. The mirror will recognize certain behaviors, such as when you schedule a trip or fail to get enough exercise, and recommend contextually-relevant content. If you're interested, you can tap your phone on the mirror to sync the article for reading on the run or on our Surface Reader application.

R&D developed the hardware and software for the mirror in-house. It is an ongoing project, with many new modules and modes of interaction in the works.

Press Coverage