The ability to interact with smartphones and other mobile devices through three-dimensional gestures with hands and fingers will substantially bring the digital world to reality by sensation. And the new project of the University of Linnaeus will help in the development of the next generation interface, which makes it possible to use the familiar movements in real life.

The real world is three-dimensional and is characterized by its width, length and depth (height). Why do users have to be content with two-dimensional gestures in interaction with their mobile devices?

Of course, touch screens greatly simplified the work with devices. But came new categories of devices – smart watches and helmets of virtual reality, for interaction with which there is clearly not enough two-dimensionality of the current interface of user interaction with the usual touchscreens of their digital assistants.

Users would like to be able to carry out the actions by moving their hands over the device, and around it. This is exactly what the scientists are striving to realize in the framework of the new project Real-Time 3D Gesture Analysis for Natural Interaction with Smart Devices, whose task, as its name implies, is to provide natural interaction with smart devices in real time via 3D gestures. This project is another significant step towards the development of interface technology.

The goal of this project is to make the user experience with the interaction with digital devices more natural and the object of the virtual world could, for example, be taken and rotated, explains project manager from the University of Linneus, Shahrouz Yousefi.

There are many applications of the developed technology, and they are different – virtual (VR) and supplemented (AR) realities, medicine, robotics, electronic learning technologies, 3D games and much more.

In order to be able to analyze the movements of the hands and fingers in real time, of course, requires high power, and high intelligence system, which is designed for this. It includes large amounts of data and advanced analysis methods, especially when the system needs to track simultaneous movements of several hands.

The main difficulty of the task is to develop and implement new approaches to the analysis of large amounts of data (Big Data) about gestures and movements. The second and no less difficult task is to integrate these new solutions with existing technologies of computer «vision» and recognition.

Will it be more convenient to manage a smartphone with 3D gestures?

Based on sciencedaily.com