The main functionality of the B10Mouse is making computers easier to use for people with various motor disabilities, who cannot move arms or hands. The user can move the mouse tilting his head along two axes:
When the B10Mouse starts, the main interface will appear and will remain superimposed over the other windows. By default, the interface is positioned in the lower-right corner of the screen, but it is always possible to shift it in the favored position. From the main window, the user can receive a feedback on how the software is tracking his face and can access various functionalities and applications:
In the "Emoticon feedback" area, the user can check the system status through some simple and easy-to-understand emoticons. Every different emoticon represents a different state captured by the system:
The graphic user interface has been studied in order to be as little intrusive as possible and easy to use. Moreover, the buttons are well visible and spaced, in order to be easily reachable and clickable. The final solutions have been implemented following the advices of the Italian Union for the Fight against Muscolar Dystrophy of Milan.
B10Mouse offers the possibility of commanding the computer with your voice, through customizable keywords. The speech recognition inferential engine exploits the array of microphones integrated in the Kinect for Windows to filter possible external noise, focusing only on the user's voice. The system does not need to be trained on the user voice and is ready-to-use since the first start. The default commands are the following:
All vocal commands can be edited and can be made of one, two or more words. It is advised to use a specific word (not used often) as a keyword to start the vocal recognition (like the word "biomouse" in the default commands).
If the user needs it, B10Mouse offers also the possibility of performing mouse clicks through eye-blinks. After a first calibration phase, the user can perform clicks as shown in the image. Closing the left eye causes a left-click; closing the right eye causes a right-click; closing both eyes causes a double left-click. The calibration phase is an operation needed sporadically and is the only one for which an operator is needed. It lasts about 30 seconds and it is necessary for the system to work at high precision level. During the calibration phase, selected features are extracted from the images taken, invariant to distance or light conditions. A machine learning system is then trained with these features and the result is the system capable of recognizing in real-time if the user is closing his eyes. Thanks to the stability of the face recognition (and as a consequence of the eyes position) and the use of the selected features, we are able to obtain an hit rate around 94%, with a percentage of false positive around 2%.
The user can modify how the mouse behaves according to his needs, using the settings window. As soon as the user moves one of the sliders, the new settings are saved and used by the program. In the upper part of the Settings window, inside the "Mouse velocity" area, there are two sliders that the user can move in order to modify the speed of the mouse, on the vertical axes (Y) or on the horizontal axes(X). The two speeds are independent between them as users can have different grades of neck mobility. In the box "Resting area", the user can calibrate –using the six sliders- the area inside with he can move his head without moving the mouse pointer. A bigger polygon means that the user has to tilt the head with a wider angle in order to move the cursor. A smaller (a point in an extreme situation) polygon means that even a small movement will involve a movement of the mouse. Using this menus, the user can (by himself) change the settings of the software, without the help of anybody else or the need for trained personal to invade his privacy. The third area, "Kinect angle", permits to modify the tilt angle of the Kinect sensor and thus to find the best inclination depending on the Kinect's or the user's position. The box "Vocal Commands", contains a button that opens the "Vocal commands settings", the window from which it is possible to modify the keywords understood by the program.
In some particular situations, it may be necessary to block the movement of the mouse using the head and the clicking using eye-blinks. In order to block B10Mouse, all that is necessary to do is press the F5 button on the physical keyboard, having the focus on the main window. A window warning that the system is blocked will pop up. Once the system is paused, the operator will be able to open the Settings window to change the software parameters or leave it inactive. The only unauthorized operation during the pause is the calibration of the eyes. When the system is paused, the red face with the crossed eyes is shown in the "Emoticon feedback area". To exit pause mode, it is necessary to press F5 button again.
B10Mouse offers also a vocal synthesizer system and an on-screen keyboard. The main features are three:
The on-screen keyboard can be used to write in the text-so-speech box or like a normal keyboard. To use it, all the user has to do is pointing at the letter he wants to write and click. While typing, in the upper part of the keyboard, thanks to the automatic completion feature, some suggested words will be visualized. Clicking on the suggestions, the software will complete the word automatically. Note that the suggestion language is based on the operating system language. In the Text-to-speech window it is possible to see the following commands:
Another application available inside B10Mouse is the simplified PDF reader. Thanks to this program, it is possible to open and read easily every pdf file. Moreover, it can be used as a simplified web browser as well, simply clicking on one of the favorite websites (and from then on navigate as in all normal browser, with the help of the big icons). The PDF Reader has been added following the requests of many users who wanted to read books or comics, but could not do that easily anymore. Thus, we have implemented a software intuitive to use and which integrates well with B10Mouse. Taking a look at the PDF Reader interface, it is possible to see that it is divided in two parts: on the left there is the navigation bar, while on the right there is a larger zone in which the pdf (or the website) itself is shown. Clicking with a double click on the icons, it will be possible to navigate the document or the site.
|"Go" button||opens the classical dialog window, from which it is possible to load a pdf file.|
|Predefined sites button||open the chosen site.|
|"+" and "-" buttons||magnify or minimize the view of the document or of the site|
|"<" and ">" buttons||go to the previous/next page|
|"^" and "v"||scroll up/down the page|
This application is optional and the user has to buy an additional hardware in order for it to work. The B10Remote Control is a virtual remote control, completely customizable and able to send and receive commands from many electronic appliances, such as televisions, hi-fi, etc... B10Remote Control looks like a common remote control, with a numbered keyboard, the buttons for changing volume and channels, for shutting down/turning on the appliance and the space to compose channels with two or more digits. In order to use the B10Remote Control with one specific appliance, it is necessary to register the specific remote control codes. In order to do that, all it is necessary to do is set the application in learn mode, selecting the voice "learn" from the File menu. The background will become green and it will be possible to register the different commands to different buttons, simply selecting the virtual button, clicking on it once and then pushing the button on the physical remote. It is possible to save and load more configurations, relative to different appliances. Thanks to the B10Remote Control, users will be able to control all the appliances, which can be controlled with a remote in their house, and thus they will be more independent, thanks to a low-cost home automation.