In the schema above shows how B10MOUSE works. The system is composed by a normal personal computer and a Microsoft Kinect for Windows sensor, used as vision device. Born as a gaming controller, the Kinect has arrived lately in his desktop configuration and can be bought from many online stores. In the figure it is depicted the environment in which our system is used. Once the Kinect is connected to the PC and B10MOUSE is launched, it is possible to move the mouse pointer tilting the head and clicking and/or executing other tasks using voice commands (completely customizable and multi-language) and eye-blinks. Adding an optional USB device, it is also possible for the user to control the most common appliances such as televisions, hi-fi systems, air conditioning, and so on. Thus, thanks to B10MOUSE, people can have their own low-cost home automation system.
B10MOUSE is composed of a main program, which captures the face of the user and tracks his movement and his commands, and of a (increasing) number of connected applications studied to help impaired people in using all the normal features of a computer.
The figure above shows how the interaction to move the mouse pointer works. As soon as the user is in front of the monitor, he can start tilting his head in order to start using the computer. A pitch movement (up or down) drives an equivalent movement of the cursor up or down the screen. A rolling movement of the head causes a left or right movement of the pointer. It is possible to combine both movements in order to shift the mouse freely in all directions. It is possible to customize how the mouse behaves depending on the user mobility and on his resting position. Moreover, it is possible to change the settings relative to the mouse sensibility and speed. Using vocal commands, it is possible to perform different actions, such as mouse clicks (left, right, double-click…) or closing/opening a window. The vocal commands are completely customizable and multi-language. The eye-blink detection feature permits people who cannot use the vocal commands to use the system as well. Users can right-click or left-click literally in a blink of an eye! The whole system is managed through a simple and intuitive interface, which permits to explore and modify all the feature of the B10MOUSE.