Projects/MouseTrap/Design

The Mousetrap Design Documentation is based on a subset of the IEEE 1016 Software Design Description Standard.

Contents

1. Introduction
2. Architectural Description
3. Interface Description
4. Detailed Design
4.6 Image Detection Module MISSING!!!
4.7 OpenCV Framework
1. 1. 4.7.1 Entity 15 - OcvfwPython
    1. 4.7.1.1 Type
    2. 4.7.1.3 Description

1. Introduction

1.1 Purpose

This document is the Software Design Specification for MouseTrap. It contains an architectural description, an interface description and a detailed design of how Mousetrap will be implemented. The interface description is broken down into user interface, data interface and programming interface. This document will serve as the basis for implementation.

1.2 Scope

MouseTrap is a GNOME Accessibility tool that aids users with disabilities. MouseTrap is for users who cannot operate a hardware mouse due to a physical disability, but who can move their head left to right, up and down, in a deliberate manner, and can see well enough to see the cursor and other objects on the screen. MouseTrap tracks features of the user’s head using a web camera, and moves the cursor accordingly. MouseTrap is designed to allow users who can’t operate a hardware mouse to still be able to interact with the computer.

1.3 Definitions, Acronyms, and Abbreviations

AT Layer – Assistive Technology Layer of the GNOME Accessibility Architecture.
GDK- library used by GTK+ to create system windows. Provides abstraction from system to GTK+ (Described below)
GNOME Accessibility – The suite of software services and support in GNOME that allows people with disabilities to utilize all of the functionality of the GNOME user environment.
GTK- (GIMP Toolkit)- a cross-platform widget toolkit for creating graphical user interfaces.
HAAR points- points used for detection of ridged objects in image processing. Used by MouseTrap to detect facial features.
IDM – Image Detection Module. IDMs allow the user to select which feature they would like to track.
OpenCV- a library of programming functions mainly aimed at real-time computer vision. It focuses mainly on real-time image processing.

1.4 References

"GNOME Accessibility Developers Guide." GNOME Accessibility Developers Guide. N.p., n.d. Web. 14 Nov. 2014. <https://developer.gnome.org/accessibility-devel-guide/stable/ >.

"MouseTrap Dev Help." Foss 2 Serve. N.p., n.d. Web. 25 Apr. 2014. <http://foss2serve.org/index.php/MouseTrap_Dev_Help >.

"Wiki.gnome.org." MouseTrap Requirements. N.p., n.d. Web. 21 Oct. 2014. <https://wiki.gnome.org/Projects/MouseTrap/Requirements >.

1.5 Overview

Section 2 describes MouseTrap’s overall architectural design. Section 3 describes the User, Data, and Programming interfaces for MouseTrap. Section 4 describes MouseTrap at a more detailed level by describing the individual classes of MouseTrap. Section 5 describes requirements that have been identified during the development of this document, but were not documented in the Software Requirements Specification. Section 6 lists some special notes about the development of this document.

2. Architectural Description

3. Interface Description

3.1 User Interface

Mousetrap currently has only one user interface, which is the main screen. This is the window that is displayed when MouseTrap starts up. It appears on the upper right side of the monitor and displays the image captured by the web camera. With the haartraining method, OpenCV library provides MouseTrap with Haar-link features around the user’s head. This enables the software method to follow the user’s head and track the movements. The product will interface with GNOME as a GTK+ application to move the cursor about the screen. Based on the x & y coordinates of the output, MouseTrap can tell how far the cursor needs to be moved across the screen relative to the user’s head movements. The user may select sensitivity to users head movements relative to cursor movements on the interface to their own personal liking.

3.2 Data Interface

MouseTrap gets all of its input from the web camera. Using an open source library, named OpenCV, MouseTrap captures the current frame from the web camera. Using the haartraining method, the OpenCV library provides MouseTrap with Haar-like features around the users head. This enables the software to follow the user's head and track its movement. The product will interface with GNOME as a GTK+ application in order to move the cursor about the screen. Based upon the x & y coordinates that are output by the program we will be able to tell how far the cursor needs to be moved across the screen in accordance with the user's head.

3.3 Programming Interface

MouseTrap uses Python for its implementation, and makes use of OpenCV libraries for tracking. The software interacts with the GNOME desktop environment through the GTK & GDK.

4. Detailed Design

4.1 Controller

4.1.1 Entity 1 - Vision

4.1.1.1 Type = Collection

4.1.1.2 Requirement

3.2.1 Start Up MouseTrap
3.2.6 Close MouseTrap

4.1.1.3 Description

Vision is a collection of classes that are responsible for starting the camera , is the class that is called when MouseTrap is started. Contains the Camera class which allows the dimensions of the camera view to be set as well as retrieving an image from the camera which is stored in image.py.

4.1.4 Entity 4 - Core

4.1.4.1 Type - Collection

4.1.4.2 Requirement

3.2.1 Start Up MouseTrap
3.2.2 Follow Tracking Feature
3.2.6 Close MouseTrap

4.1.4.3 Description

Core is a collection of classes that are responsible for beginning execution of MouseTrap. Core contains classes that will load and set MouseTrap settings, load and initialize plugins, and start MouseTrap's primary loop and graphical user interface.

4.1.5 Entity 5 - App

4.1.5.1 Type - Class

4.1.5.2 Requirement

3.2.1 Start Up MouseTrap
3.2.6 Close MouseTrap

4.1.5.3 Description

The App class is responsible for loading all of the correct plugin files and registering them with the Loop class. It is also tasked with loading MouseTrap configurations, and initializing the Loop and GUI classes.

4.1.6 Entity 6 - Observable

4.1.6.1 Type - Class

4.1.6.2 Requirement

3.2.1 Start Up MouseTrap
3.2.2 Follow Tracking Feature
3.2.6 Close MouseTrap

4.1.6.3 Description

The Observable class is responsible for appending observers and adding argument values. It is a supertype of the Loop class.

4.1.7 Entity 7 - Loop

4.1.7.1 Type - Class

4.1.7.2 Requirement

3.2.1 Start Up MouseTrap
3.2.2 Follow Tracking Feature
3.2.6 Close MouseTrap

4.1.7.3 Description

The Loop class is MouseTrap's primary loop and constantly runs throughout the programs execution. It is a subtype of the Observable class.

4.1.8 Entity 8 - Plugin

4.1.8.1 Type - Class

4.1.8.2 Requirement

3.2.1 Start Up MouseTrap
3.2.2 Follow Tracking Feature
3.2.6 Close MouseTrap

4.1.8.3 Description

The Plugin class is the abstract base class for all plugins. It provides a common interface for all plugins.

4.1.9 Entity 9 - CameraPlugin

4.1.9.1 Type - Class

4.1.9.2 Requirement

3.2.2 Follow Tracking Feature

4.1.9.3 Description

The CameraPlugin class is the interface between the Camera and the image. It reads the image from the Camera and sets the image in the application.

4.1.10 Entity 10 - DisplayPlugin

4.1.10.1 Type - Class

4.1.10.2 Requirement

3.2.2 Follow Tracking Feature

4.1.10.3 Description

The DisplayPlugin class is the class that helps display the image in the application(GUI) window.

4.1.16 Entity 16 - NosePlugin

4.1.16.1 Type - Class

4.1.16.2 Requirement

3.2.3.5 Tracking Recalculation of User's Head
3.3.3 Follow Tracking Feature

4.1.16.3 Description

The NosePlugin class gets information on the user's nose position and is able to identify it from the image that the camera receives.

4.1.17 Entity 17 - NoseLocator

4.1.17.1 Type - Class

4.1.17.2 Requirement

3.2.3.5 Tracking Recalculation of User's Head
3.3.3 Follow Tracking Feature

4.1.17.3 Description

The NoseLocator class works with the FeatureDetector to locate the user's nose.

4.1.18 Entity 18 - NoseJoystickPlugin

4.1.18.1 Type - Class

4.1.18.2 Requirement

3.2.3.5 Tracking Recalculation of User's Head
3.2.4.3 Move Cursor - Joystick
3.3.3 Follow Tracking Feature

4.1.17.3 Description

The NoseJoystickPlugin class gets the nose's location from the NoseLocator class and will get the image received to point at the nose feature.

4.2 Interface

4.2.1 Entity 19 - Config

4.2.1.1 Type- Class

4.2.2.1 Requirement

3.2.3.1 Select Camera
3.2.3.2 Select Tracking Method
3.2.3.3 Toggle Self-View
3.2.3.7 Set Cursor Sensitivity
3.2.3.9 Toggle Face Outline
3.2.4.1 Select Cursor Movement Method
3.2.6 Close MouseTrap

4.2.3.1 Description

This class creates a data structure which holds the configuration. The class allows for the ability to load a default path or paths which are passed in to the program. The class also allows for specific access to a certain configuration. Values from the class are the targets and they are able to recursively be copied from a given source.

4.2.2 Entity 20-Gui

4.2.2.1 Type-Class

4.2.2.2 Requirement

3.2.1 Start Up MouseTrap
3.2.6 Close MouseTrap
3.2.3 Camera Preferences
3.2.3.10 Access Help Manual
3.2.5 Usage Features

4.2.2.3 Description

This class initializes a data structure of windows. For each window with a new window name, the Image Window class (Entity 7) is used to create the new window which is added to the data structure. Then it displays the image in the window that is named by what is given. This allows for reuse of named windows. This class also uses GTK to start handling events in the window as well as returning the height and width of the window.

4.2.3 Entity 21-ImageWindow

4.2.3.1 Type-Class

4.2.3.2 Requirement

3.2.5.2 Recalculate Face Location of the Tracking Feature
3.2.3.9 Toggle Face Outline
3.2.3.4 Flip Camera Image Orientation
3.2.3.3 Toggle Self-View

4.2.3.3 Description

This class uses GTK to initialize a window with a given title and configuration. A canvas is created using GTK and then the canvas is added to the window. This class also takes a given image and sets it to a canvas. The canvas is then drawn to the window.

4.2.4 Entity 22-Pointer

4.2.4.1 Type-Class

4.2.4.2 Requirement

3.2.2 Follow Tracking Feature
3.2.3.7 Set Cursor Sensitivity
3.2.4.2 Move Cursor -Screen
3.2.4.3 Move Cursor - Joystick
3.2.4.4 Move Cursor – Drag and Drop
3.2.4.5 Click Left Mouse
3.2.4.6 Click Right Mouse
3.2.4.7 Scroll Cursor
3.2.4.8 Click Drag and Drop Cursor
3.2.5.2 Recalculate Face Location of the Tracking Feature

4.2.4.3 Description

This class uses GDK to initialize a display and device manager. A pointer is created from the device manager and the device manager is created from the GDK display. This class also moves the pointer to a given position otherwise no change is made. This class has the ability to determine if the pointer is moving based on if a position was given previously. It also has the ability to determine the current position of the pointer. By using imports from Xlib, the class checks for clicking events of a button and syncs the display according to the event.

4.2.5 Entity 23-Image

4.2.5.1 Type-Class

4.2.5.2 Requirement

3.2.3.4 Flip Camera Image Orientation

4.2.5.3 Description

This class allows for manipulation on the image provided such as manipulating the image to cv grayscale and manipulating the image to pixel buffer. The class uses the image provided as well as information such as height, width, color space, etc. to manipulate pixel buffer which is from the imported GdkPixbuf.

4.2.6 Entity 25 - Haarloader

4.2.6.1 Type- Class

4.2.6.2 Requirement

3.2.4 Cursor Movement

4.2.6.3 Description

This class uses the haar files to allow the program to determine where on the face the nose and eyes on the user is.

4.2.7 Entity 26 - HaarNameError

4.2.7.1 Type- Class

4.2.7.2 Requirement

3.2.4 Cursor Movement

4.2.7.3 Description

This class will keep track of errors when the name of a Haar is incorrect. Then the class will print out an error message according to the problem.

4.2.8 Entity 27 - FeatureDetector

4.2.8.1 Type- Class

4.2.8.2 Requirement

3.2.4 Cursor Movement

4.2.8.3 Description

The feature detector is able to keep track of the image and clear the cache when necessary. It will cache the image id and result. If a feature is not found, an error message related to the problem would be print out.

4.2.9 Entity 28 - FeatureDetectorClearCachePlugin

4.2.9.1 Type- Class

4.2.9.2 Requirement

3.2.4 Cursor Movement

4.2.9.3 Description

This class allows for all of the detection caches that keep track of the features to be cleared.

4.2.10 Entity 29 - FeatureNotFoundException

4.2.10.1 Type- Class

4.2.10.2 Requirement

3.2.4 Cursor Movement

4.2.10.3 Description

This class allows for error handling when a feature is trying to be detected. If a feature is not found then an error message will be created related to the cause of the error that occurred.

4.3 Camera

4.3.1 Entity 30 - il8n

4.3.1.1 Type- Library

4.3.1.2 Requirement

3.2.1 Start Up MouseTrap
3.2.3 Camera Preferences
3.2.5 Usage Features
3.2.6 Close MouseTrap

4.3.1.3 Description

This is the library that contains the internationalization of the software. It is needed in order to initialize necessary inputs before starting MouseTrap.

4.3.2 Entity 31 - compat

4.3.2.1 Type- Library

4.3.2.2 Requirement

3.2.1 Start Up MouseTrap

4.3.2.3 Description

This Library deals with compatibility issues between Python 2 and 3. The Library first determines which version of Python MouseTrap is being run with and then sets up MouseTrap to run in the determined version, either Python 2 or Python 3.

4.4 Cursor

4.4.1 Entity 1 - Mouse Event Handler

4.4.1.1 Type

Collection

4.4.1.2 Requirement

3.2.4.1 Select Cursor Movement Method
3.2.4.2 Move Cursor - Screen
3.2.4.3 Move Cursor - Joystick
3.2.4.4 Move Cursor - Drag and Drop
3.2.4.5 Click Left Mouse Button
3.2.4.6 Click Right Mouse Button
3.2.4.7 Scroll
3.2.4.8 Click Drag and Drop

4.4.1.3 Description

The Mouse Event Handler will control how the mouse moves and how it is displayed on the screen. This class will act as an interface between the different cursor movement methods and GNOME.

4.4.2 Entity 2 - ScriptClass (Joystick)

4.4.2.1 Type

Class

4.4.2.2 Requirement

3.2.4.3 Move Cursor - Joystick

4.4.2.3 Description

The Joystick cursor movement class is first of the two movement classes included with MouseTrap. When the user's head is in a central location, it is considered neutral and thus, no movement occurs. The cursor will move when the user rotates their head in any direction and will stop once they return their head to the central location.

4.4.3 Entity 3 - ScriptClass (Screen)

4.4.3.1 Type

Class

4.4.3.2 Requirement

3.2.4.2 Move Cursor - Screen

4.4.3.3 Description

The Screen cursor movement class is the second of the two movement classes included with MouseTrap. The Screen movement method mimics that of a traditional mouse; as the user's head rotates, the cursor will follow accordingly.

4.5 Addons

4.5.1 Entity 1 - AddonsHandler

4.5.1.1 Type

Class

4.5.1.2 Requirement

3.2.5.1 Check CPU Usage
3.2.5.2 Recalculate Location on Tracking Feature

4.5.1.3 Description

This AddonsHandler class receives a list of Addons that the user has chosen to enable and loads the selected items.

4.5.2 Entity 2 - AddonsBase

4.5.2.1 Type

Class

4.5.2.2 Requirement

3.2.5.1 Check CPU Usage
3.2.5.2 Recalculate Location on Tracking Feature

4.5.2.3 Description

The AddonsBase class adds the selected Addons to MouseTrap's main menu.

4.5.3 Entity 3 - Addon (CPU)

4.5.3.1 Type

Class

4.5.3.2 Requirement

3.2.5.1 Check CPU Usage

4.5.3.3 Description

One of the Addons included with MouseTrap is Check CPU Usage. On the main menu, this Addon will display the percentage of the CPU that MouseTrap is currently using.

4.5.4 Entity 4 - Addon (Recalc)

4.5.4.1 Type

Class

4.5.4.2 Requirement

3.2.5.2 Recalculate Location on Tracking Feature

4.5.4.3 Description

Another Addon included with MouseTrap is the Recalculate button. On the main menu, this Addon will display a button and when clicked, MouseTrap will recalculate the tracking point.