This page lists the documentation, progress and results for Open Source User Interface Project of Open Source Hardware Enterprise at Michigan Technological University
Existing Solutions[edit | edit source]
Software[edit | edit source]
OptiKey[edit | edit source]
Is a functional out of the box keyboard control, mouse control, and speech software designed specifically for people with motor and speech limitations.
This software is open source and free. But it only supports windows. It will work using mouse, keyboard, or several eye tracker device such as Tobii EyeX, Eye Tribe and ITU Gaze Tracker
Gazespeaker[edit | edit source]
Is a software to help people with disabilities to communicate and interact with their environment and web. It has a lot of features, such as text to speech, preset sentence and word, read and write emails, web, videos, music, ebooks, etc.
This is a free, open source and easily customizable. It only supports windows and can be used with mouse, webcam, or several eye tracker device such as Tobii EyeX, Eye Tribe and ITU Gaze Tracker
BKB[edit | edit source]
This is also a open source software to control keyboard and mouse using any device that could move a cursor (including eye tracker)
eViacam[edit | edit source]
This is a free and open source software that moves the cursor on computer screen as you move your head. It is easy to use and works with any webcam. It can run on windows and linux.
CameraMouse[edit | edit source]
This also another program to control the movement of cursor using our head. It is a free program that run only on windows.
Click2Speak[edit | edit source]
Another free, but not open source software for keyboard control, mouse control and text to speech. It features Swiftkey's prediction engine, movable and customizable on screen keyboard. It supports only windows 7 and 8. It supports any hardware that can control mouse cursor. It also supports eye tracking device such as Tobii EyeX, Tobii Rex, Tobii X2-30 (not the X2-60), Tobii PCEyeGo (with firmware 1.1.5 or later), and The Eye Tribe.
Hardware[edit | edit source]
Tobii EyeX Controller[edit | edit source]
Tobii EyeX Controller is low cost eye tracker to track the location of eye gaze for interactive and entertainment purposes. Together with the suitable software on computer, this device can accurately track the movement of eye.
It can only operate on microsoft windows 7, 8 or 10 with USB 3.0. Unfortunately, it needs at least core i5 and 8GB RAM for optimal performance. This hardware could be used with open source software such as OptiKey, Click2speak, Gazespeaker, and BKB
Eye Tribe Tracker Pro[edit | edit source]
This is also low cost commercial eye tracker that used infrared illumination to detect the movement of eye. It also require at least core i5, 1GB RAM, USB 3.0. Also it will only run on Windows 7, Windows 8, Windows 10, or MAC mountain lion 10.8.4. This hardware could be used with open source software such as OptiKey, Click2speak, Gazespeaker, and BKB.
EyeWriter[edit | edit source]
This is an open source hardware project designed to help artists and graffiti writers suffering from ALS to draw using only their eyes. There are clear instructions how to build the head mounted head tracker on the website. On their website, they said the project has already been used by graffiti writers with suceess.
They claim the hardware would only costs $50 with PS3 Eye and a pair of sunglasses. But the software provided with this project only support drawing functionality, and it is unclear whether we could use established software such as OptiKey, Click2speak, Gazespeaker, or BKB with this hardware.
An improved version of the device, EyeWriter 2.0, adds accuracy and allows people to move their head while using the device. In comparison, the original EyeWriter is designed to be used in motionless head. The new version is not mounted on the head, rather near the computer screen. It costs about $200.
OpenEyes[edit | edit source]
OpenEyes is an open source hardware and open source software project for Eye Tracking. Several design for the hardware and software could be found on their website. The parts for the hardware costs about $500. Their hardware is a head gear eye tracker using infrared illumination technique.
Pupil[edit | edit source]
Pupil is a open source head mounted mobile eye tracking platform. They used 3D Printed head gear and two cameras with infrared illumination technique.
Their hardware construction is modular and they prepared a guide to construct their hardware from component parts. The drawback is that their hardware is considerably more expensive than other solutions.
eyeCan[edit | edit source]
eyeCan is an open source hardware and software project for head mounted Eye Tracking. They used the same hardware as the EyeWriter version 1 project. The difference is that they developed their own software. It seems they only developed the software to control mouse cursor, but no keyboard. Video instruction and source code is provided on their project homepage.
Results[edit | edit source]
Tobii Eyex with Optikey[edit | edit source]
At first try, we found Tobii Eyex with Optikey hard to use and inaccurate. But after experimenting with several options and configurations we found several way to make them easier to use. After using them after a while and become quite adept at it, we are sure this is an acceptable solution with just a few drawbacks as we will discuss below shortly.
How to use Tobii Eyex with OptiKey:
- Use larger screen with larger resolution. The purpose of this is to make button on OptiKey appear larger. If you use larger screen, you can increase the size of button which will make inaccuracy of Tobii Eyex less of a problem
- Increase the OptiKey size to about 60% of the screen. This will improve accuracy, but will cover up things on the screen
- When not using the OptiKey keyboard, hide it by switching to mouse mode. This will reduce the portion of OptiKey covering the screen to about 15 % of the screen
- Always turn on the magnifier feature of the mouse mode. This will make clicking icons and menus much more easier
- When you switch from mouse mode to keyboard mode, sometimes you will lose focus of text box. Thus it became impossible to type to that text box. There is a way to do this. Before switching to keyboard mode, expand the mouse dock first. After that you can switch to keyboard mode. You will not lose focus of the text box this way.
We have a video to demonstrate how to use Tobii EyeX with Optikey to write email in the following link. Youtube
Drawback of this solution:
- It is not a real substitute of keyboard and mouse because you need the windows to boot up first, wait for Tobii Eyex engine to start, and wait for Optikey to start until you can use your eyes on the OptiKey
- OptiKey cover up big portion of screen when typing
Our Development[edit | edit source]
Eye Tracking Camera and Off-Screen Keyboard[edit | edit source]
This semester we worked on making our own open-source version of an eye tracking camera. The design concept was inspired by the EyeWriter project mentioned above, and we used the EyeWriter as the basic development platform for our system. The main improvement that has been made to the eye-tracking system takes the form of an off-screen keyboard. A traditional on-screen keyboard that would be used with an eye tracker is large and takes up a lot of space on the screen that could be used for other things. We decided a good solution to this problem would be a physical keyboard that could extend out of the back of the monitor or laptop screen. This will work nicely with an eye tracking camera and in the future we plan to combine the camera that we built and the off-screen keyboard into one design. If you are interested in this project check out the report we wrote about it. It has step-by step instructions on how to build our eye tracker and off-screen keyboard prototypes.
The build instructions in the report call for several CAD models to be 3D printed. Here are the links for the files you'll need!
Eye Tracker Parts the Internet Modeled: M12 Lens Mount PS3Eye Casing
Since we've only been working on this project for one semester there is still a lot of work that needs to be done to polish it up and finish it. Here are some of the things we have yet to do for the eye tracker:
- Write custom accessibility software that allows the device to be used as a standalone eye tracker.
- Incorporate onboard device drivers so that the device can be used without setup beyond a one-time initial installation
- Move the camera and associated hardware into the off-screen keyboard design package.
And for the off-screen keyboard:
- Incorporate physical design into eye tracker design so that the two designs turn into one device
- Increase the height of the screen by removing endstops from sliding rods and replacing spring mechanism with one that allows for greater motion
Head Tracking Camera[edit | edit source]
This is our completed head tracking system made out of Raspberry Pi, Arduino, and Pi Noir Camera.
How this works is that user will wear a round infrared reflective patch on their forehead. When they move their head in front of the camera, the system will detect the movement of the head and translates that into the movement of the mouse cursor.
It works by using infrared illumination technique. We used Pi Noir Camera with added filter that blocks visible light but transmit infrared light. So the Pi Noir camera will only see infrared light. Then we add our own infrared lighting using infrared LED. This method will ensure the system will work without dependency on the condition of external light.
We used a round IR reflective patch as the object to be tracked. Because the system has its own infrared lighting source, the camera will see the reflective patch as a round bright object while the other surrounding dark because no visible light is going through the filter. This will make tracking the object easier. This also ensure robust detection where no other object will be detected.
File links:
- Source code, firmware, and files needed to configure the system tinyupload.com
- 3D design file for camera case tinyupload.com
Please see project report in the "Eye Tracking Camera and Off-Screen Keyboard" section in this page for instruction how to build this system. Full step by step instructions is described there to allow complete recreation of the system
Future directions:
- Design a 3D printed case that could be mounted on top of the screen. All of the components should fit inside the case
- Reduce the number of components required and the cost of the system. One way of doing that is to choose another single board computer that could also function as a USB device, so we don't need Arduino Uno
- Port the head tracker program to C language, so it would run faster and allow higher resolution video to be processed
- Implement filtering for smooth tracking