Smart Objects

Reflex is a mirrored camera mobile training application for persons with Neuro-Developmental Disorder (NDD). The game, offered through a cross-platform application for smart-phones and tablets, bridges the digital and the physical worlds by tracking, via a bottom-looking mirror positioned on the device camera, physical items placed on the table. This interaction paradigm defined as phygital, its co-designed features and the first pilot study reveal an unexplored potential for learning. 

We chose to adopt a hybrid approach defined as “phygital” as it digitally virtualizes a physical environment of Tangible Objects (TOs) manipulable by the user. This approach uses a platform which augments a handheld computing device, such as a phone or tablet and utilizes novel computer vision algorithms to sense user interaction with the tangible objects. This technology yields numerous advantages including, but not limited to providing a low-cost alternative for developing a nearly limitless range of applications that blend both physical and digital mediums by reusing existing hardware (e.g., device camera); leveraging novel lightweight detection and recognition algorithms; having low implementation costs; being cross-compatible with existing computing device hardware; operating in real-time to provide for a rich, virtual experience; processing numerous interactions with one or more TOs simultaneously without overwhelming the computing device; recognizing TOs with substantially perfect recall and precision (e.g., 99% and 99.5%, respectively); being capable of adapting to lighting changes and wear and tear of TOs; providing a collaborative tangible experience between users; being natural and intuitive to use; requiring few or no constraints on the types of TOs that can be processed (no specialized markers or symbols are required to be included on the TOs in order for the platform to recognize them