Support +39 0287 188 441

Mouse movement

The main functionality of the B10Mouse is making computers easier to use for people with various motor disabilities, who cannot move arms or hands. The user can move the mouse tilting his head along two axes:

  • Pitch: tilting his head downwards or upwards, the user can move the mouse pointer up or down on the screen
  • Roll: tilting his head towards his shoulders (right or left), the user can move the mouse pointer right or left on the screen.
  • These two movements can be combined in order to move the mouse obliquely. These movements have been studied in order to be as little tiring for the user as possible and also to allow the user to keep watching the screen while moving the mouse. Using a menu in the Settings window, it is possible to tune the sensibility of the mouse pointer to the user's movements in every direction. This way the software can be easily customized to meet the needs of the different users, which might have a different mobility. For example, a user with a good head mobility will prefer to have a larger area in which he can move without moving the mouse pointer, while a user with a bad mobility can set the system to work with every little movement he does with his head. Thanks to the advanced information given by the infrared technology of the Kinect for Windows, it has been possible to implement a robust algorithm capable of tracking the user's face without errors, even in dim light conditions.

    The main interface

    When the B10Mouse starts, the main interface will appear and will remain superimposed over the other windows. By default, the interface is positioned in the lower-right corner of the screen, but it is always possible to shift it in the favored position. From the main window, the user can receive a feedback on how the software is tracking his face and can access various functionalities and applications:

  • Settings button: pressing this button, it opens another window, from which it will be possible to modify the mouse behavior and edit the vocal commands.
  • Shut down button: pressing this button will cause the closing of B10Mouse.
  • Emoticon feedback area: in this area the user receives a visual feedback on whether the camera is tracking him correctly or not and on the fact that he is blinking or not
  • Live video area: here the user can see in real-time what the Kinect camera is watching
  • Alarm button: pushing this button an alarm signal will spread from the computer speakers
  • Text-to-speech button: pressing this button will cause the text-to-speech application to start
  • PDF Reader button: pressing this button will cause the Simplified PDF Reader application to start
  • B10Remote Control button (optional): pressing this button, if the optional USB device is connected to the PC, the remote control window will pop-up
  • In the "Emoticon feedback" area, the user can check the system status through some simple and easy-to-understand emoticons. Every different emoticon represents a different state captured by the system:

  • Green, normal face with open eyes: the user is tracked correctly and is standing still. In this state he can perform mouse clicks using eye-blinks
  • Yellow, happy face: the user is tracked correctly and is tilting his head, causing the mouse pointer to move on the screen
  • Red, unhappy face with open eyes: no user face is currently tracked
  • Red, unhappy face with crossed eyes: B10Mouse is currently in pause mode, and it is not possible to move the mouse or perform clicks using it. This is the result of pressing the F5 button
  • Green, normal face with closed eyes: the system has captured the closing of both eyes. The effect is a mouse double-click
  • Green, normal face with closed right eye: the system has captured the closing of the right eye. The effect is a mouse right-click
  • Green, normal face with closed left eye: the system has captured the closing of the left eye. The effect is a mouse left-click
  • The graphic user interface has been studied in order to be as little intrusive as possible and easy to use. Moreover, the buttons are well visible and spaced, in order to be easily reachable and clickable. The final solutions have been implemented following the advices of the Italian Union for the Fight against Muscolar Dystrophy of Milan.

    Vocal commands

    B10Mouse offers the possibility of commanding the computer with your voice, through customizable keywords. The speech recognition inferential engine exploits the array of microphones integrated in the Kinect for Windows to filter possible external noise, focusing only on the user's voice. The system does not need to be trained on the user voice and is ready-to-use since the first start. The default commands are the following:

  • Biomouse push: single mouse left-click
  • Biomouse double: mouse double left-click
  • Biomouse right: single mouse right-click
  • Biomouse left: press and keep pressed the left mouse button (e.g. for dragging an icon)
  • Biomouse release: release the mouse left button
  • Biomouse close: close the current selected window
  • Biomouse center: bring the mouse to the center of the screen
  • All vocal commands can be edited and can be made of one, two or more words. It is advised to use a specific word (not used often) as a keyword to start the vocal recognition (like the word "biomouse" in the default commands).

    Eye-blink detector

    If the user needs it, B10Mouse offers also the possibility of performing mouse clicks through eye-blinks. After a first calibration phase, the user can perform clicks as shown in the image. Closing the left eye causes a left-click; closing the right eye causes a right-click; closing both eyes causes a double left-click. The calibration phase is an operation needed sporadically and is the only one for which an operator is needed. It lasts about 30 seconds and it is necessary for the system to work at high precision level. During the calibration phase, selected features are extracted from the images taken, invariant to distance or light conditions. A machine learning system is then trained with these features and the result is the system capable of recognizing in real-time if the user is closing his eyes. Thanks to the stability of the face recognition (and as a consequence of the eyes position) and the use of the selected features, we are able to obtain an hit rate around 94%, with a percentage of false positive around 2%.

    Modify settings

    The user can modify how the mouse behaves according to his needs, using the settings window. As soon as the user moves one of the sliders, the new settings are saved and used by the program. In the upper part of the Settings window, inside the "Mouse velocity" area, there are two sliders that the user can move in order to modify the speed of the mouse, on the vertical axes (Y) or on the horizontal axes(X). The two speeds are independent between them as users can have different grades of neck mobility. In the box "Resting area", the user can calibrate –using the six sliders- the area inside with he can move his head without moving the mouse pointer. A bigger polygon means that the user has to tilt the head with a wider angle in order to move the cursor. A smaller (a point in an extreme situation) polygon means that even a small movement will involve a movement of the mouse. Using this menus, the user can (by himself) change the settings of the software, without the help of anybody else or the need for trained personal to invade his privacy. The third area, "Kinect angle", permits to modify the tilt angle of the Kinect sensor and thus to find the best inclination depending on the Kinect's or the user's position. The box "Vocal Commands", contains a button that opens the "Vocal commands settings", the window from which it is possible to modify the keywords understood by the program.

    Pause the interaction (head and eyes)

    In some particular situations, it may be necessary to block the movement of the mouse using the head and the clicking using eye-blinks. In order to block B10Mouse, all that is necessary to do is press the F5 button on the physical keyboard, having the focus on the main window. A window warning that the system is blocked will pop up. Once the system is paused, the operator will be able to open the Settings window to change the software parameters or leave it inactive. The only unauthorized operation during the pause is the calibration of the eyes. When the system is paused, the red face with the crossed eyes is shown in the "Emoticon feedback area". To exit pause mode, it is necessary to press F5 button again.

    Text-to-speech

    B10Mouse offers also a vocal synthesizer system and an on-screen keyboard. The main features are three:

  • Write using the on-screen keyboard and have the computer "speak" for you, through its speakers, using the default (female) voice
  • Use the user's pre-configured (customizable) phrases and have the computer synthesize them for you simply clicking a button
  • Send the text written inside the text-to-speech box via e-mail with a click of the mouse
  • The on-screen keyboard can be used to write in the text-so-speech box or like a normal keyboard. To use it, all the user has to do is pointing at the letter he wants to write and click. While typing, in the upper part of the keyboard, thanks to the automatic completion feature, some suggested words will be visualized. Clicking on the suggestions, the software will complete the word automatically. Note that the suggestion language is based on the operating system language. In the Text-to-speech window it is possible to see the following commands:

  • Text box: where the written words/phrases are visualized
  • Receiver e-mail: two boxes composing the e-mail address. In the first one the user has to insert the name of the receiver (e.g. info), in the second one the domain (e.g.b10nix.com)
  • Slider 1: to modify the dimension of the characters
  • Slider 2: to control the volume
  • "E-mail" button: to send the e-mail, using the basic mail program, to the written address and using as main text what is written in the text box.
  • "Teach phrases" button: when this button is pressed, the background of the window becomes red. It is then possible to write a phrase normally inside the text box and register it inside one of the pre-configured phrase buttons, just by pushing it. The pressed button will change its name and from now on it will contain the new phrase.
  • PDF Reader

    Another application available inside B10Mouse is the simplified PDF reader. Thanks to this program, it is possible to open and read easily every pdf file. Moreover, it can be used as a simplified web browser as well, simply clicking on one of the favorite websites (and from then on navigate as in all normal browser, with the help of the big icons). The PDF Reader has been added following the requests of many users who wanted to read books or comics, but could not do that easily anymore. Thus, we have implemented a software intuitive to use and which integrates well with B10Mouse. Taking a look at the PDF Reader interface, it is possible to see that it is divided in two parts: on the left there is the navigation bar, while on the right there is a larger zone in which the pdf (or the website) itself is shown. Clicking with a double click on the icons, it will be possible to navigate the document or the site.

    Buttons Description
    "Go" button opens the classical dialog window, from which it is possible to load a pdf file.
    Predefined sites button open the chosen site.
    "+" and "-" buttons magnify or minimize the view of the document or of the site
    "<" and ">" buttons go to the previous/next page
    "^" and "v" scroll up/down the page

    B10Remote control (optional)

    This application is optional and the user has to buy an additional hardware in order for it to work. The B10Remote Control is a virtual remote control, completely customizable and able to send and receive commands from many electronic appliances, such as televisions, hi-fi, etc... B10Remote Control looks like a common remote control, with a numbered keyboard, the buttons for changing volume and channels, for shutting down/turning on the appliance and the space to compose channels with two or more digits. In order to use the B10Remote Control with one specific appliance, it is necessary to register the specific remote control codes. In order to do that, all it is necessary to do is set the application in learn mode, selecting the voice "learn" from the File menu. The background will become green and it will be possible to register the different commands to different buttons, simply selecting the virtual button, clicking on it once and then pushing the button on the physical remote. It is possible to save and load more configurations, relative to different appliances. Thanks to the B10Remote Control, users will be able to control all the appliances, which can be controlled with a remote in their house, and thus they will be more independent, thanks to a low-cost home automation.