Pointing device

Classification

To classify several pointing devices, a certain number of features can be considered. For example, the device's movement, controlling, positioning or resistance. The following points should provide an overview of the different classifications.^[1]

• direct vs. indirect input

In case of a direct-input pointing device, the on-screen pointer is at the same physical position as the pointing device (e.g., finger on a touch screen, stylus on a tablet computer). An indirect-input pointing device is not at the same physical position as the pointer but translates its movement onto the screen (e.g., computer mouse, joystick, stylus on a graphics tablet).

• absolute vs. relative movement

An absolute-movement input device (e.g., stylus, finger on touch screen) provides a consistent mapping between a point in the input space (location/state of the input device) and a point in the output space (position of pointer on screen). A relative-movement input device (e.g., mouse, joystick) maps displacement in the input space to displacement in the output state. It therefore controls the relative position of the cursor compared to its initial position.

• isotonic vs. elastic vs. isometric

An isotonic pointing device is movable and measures its displacement (mouse, pen, human arm) whereas an isometric device is fixed and measures the force which acts on it (trackpoint, force-sensing touch screen). An elastic device increases its force resistance with displacement (joystick).

• position control vs. rate control

A position-control input device (e.g., mouse, finger on touch screen) directly changes the absolute or relative position of the on-screen pointer. A rate-control input device (e.g., trackpoint, joystick) changes the speed and direction of the movement of the on-screen pointer.

• translation vs. rotation

Another classification is the differentiation between whether the device is physically translated or rotated.

• degrees of freedom

Different pointing devices have different degrees of freedom (DOF). A computer mouse has two degrees of freedom, namely its movement on the x- and y-axis. However the Wiimote has 6 degrees of freedom: x-, y- and z-axis for movement as well as for rotation.

• possible states

As mentioned later in this article, pointing devices have different possible states. Examples for these states are out of range, tracking or dragging.

Examples

• a computer mouse is an indirect, relative, isotonic, position-control, translational input device with two degrees of freedom (x, y position) and two states (tracking, dragging).
• a touch screen is a direct, absolute, isometric, position-control input device with two or more degrees of freedom (x, y position and optionally pressure) and two states (out of range, dragging).
• a joystick is an indirect, relative, elastic, rate-control, translational input device with two degrees of freedom (x, y angle) and two states (tracked, dragging).
• a Wiimote is an indirect, relative, elastic, rate-control, translational input device with six degrees of freedom (x, y, z orientation and x, y, z position) and two or three states (tracking, dragging for orientation and position; out-of-range for position).

Buxton's taxonomy

The following table shows a classification of pointing devices by their number of dimensions (columns) and which property is sensed (rows) introduced by Bill Buxton. The sub-rows distinguish between mechanical intermediary (i.e. stylus) (M) and touch-sensitive (T). It is rooted in the human motor/sensory system. Continuous manual input devices are categorized. Sub-columns distinguish devices that use comparable motor control for their operation. The table is based on the original graphic of Bill Buxton's work on "Taxonomies of Input".^[2]

Buxton's Three-State-Model

This model describes different states that a pointing device can assume. The three common states as described by Buxton are out of range, tracking and dragging. Not every pointing device can switch to all states.^[3]

Model	Explanation
2 State Transaction	A mouse is moved without the button being pushed. This state can be called tracking, meaning the user just moves the mouse without further interacting with the system. If the mouse is pointed at an icon and the button is pressed while moving the mouse, a new state called dragging is entered. These states are shown in the image "2 State Transaction".
Out of Range & Tracking	If a touch tablet, which is able to sense touch or no-touch, is used instead of a mouse, the state model looks different. More precisely this means that any movement of the finger off the display is out of range and has no effect on the system. Only when the finger touches the display, the state switches to tracking (Figure: "Out of Range & Tracking").
Out of Range, Tracking & Dragging	If a graphics tablet with a stylus is used, it is possible to sense all three states. When the stylus is lifted, it is out of range. When it is in range, the state switches to tracking and the pointer follows the stylus' movement. Performing extra pressure on the stylus initiates state 2 dragging (Figure: "Out of Range, Tracking & Dragging").
State 2 Set	By using a multiple-button mouse or multiple clicks, State 2 can be split into a set of states, i.e. pressing different buttons leads to different states. For example, selecting an object with Button 1 switches to the state Drag Original whereas Button 2 switches to Drag Copy. The same goes for multiple clicks known from Windows: one click on an object selects an object, double-clicks open it (Figure: "State 2 Set").

Fitts' Law

Fitts's law (often cited as Fitts' law) is a predictive model of human movement primarily used in human–computer interaction and ergonomics. This scientific law predicts that the time required to rapidly move to a target area is a function of the ratio between the distance to the target and the width of the target.^[4] Fitts's law is used to model the act of pointing, either by physically touching an object with a hand or finger, or virtually, by pointing to an object on a computer monitor using a pointing device. In other words, this means for example, that the user needs more time to click on a small button which is distant to the cursor, than he needs to click a large button near the cursor. Thereby it is generally possible to predict the speed which is needed for a selective movement to a certain target.

Mathematical formulation

The common metric to calculate the average time to complete the movement is the following:

${\displaystyle {\text{MT}}=a+b\cdot {\text{ID}}=a+b\cdot \log _{2}{\Bigg (}{\frac {2D}{W}}{\Bigg )}}$

where:

• MT is the average time to complete the movement.
• a and b are constants that depend on the choice of input device and are usually determined empirically by regression analysis.
• ID is the index of difficulty.
• D is the distance from the starting point to the center of the target.
• W is the width of the target measured along the axis of motion. W can also be thought of as the allowed error tolerance in the final position, since the final point of the motion must fall within ±W⁄2 of the target's center.

This results in the interpretation that, as mentioned before, large and close targets can be reached faster than little, distant targets.

Applying Fitts' Law in user interface design

As mentioned above, the size and distance of an object influence its selection. Additionally this effects the user experience. Therefore, it is important, that Fitts' Law is considered while designing user interfaces. Below some basic principles are mentioned.^[5]

• Interactive elements

Command buttons for example should have different sizes than non-interactive elements. Larger interactive objects are easier to select with any pointing device.

• Edges and corners

Due to the fact, that the cursor gets pinned on the edges and corners of a graphical user interface, those points can be accessed faster than other spots on the display.

• Pop-up menus

They should support immediate selection of interactive elements in order to reduce the user's "travel time".

• Options for selecting

Within menus like dropdown menus or top-level navigation, the distance increases the further the user goes down the list. However in pie menus, the distance to the different buttons is always the same. In addition, the target areas in pie menus are larger.

• Task bars

To operate a task bar, the user needs a higher level of precision, thus more time. Generally they hinder the movement through the interface.

Control-Display Gain

The Control-Display Gain (or CD gain) describes the proportion between movements in the control space to the movements in the display space. For example, a hardware mouse moves in another speed or distance than the cursor on the screen. Even if these movements take place in two different spaces, the units for measurement have to be the same in order to be meaningful (e.g. meters instead of pixels). The CD gain refers to the scale factor of these two movements:

${\displaystyle CDgain=V_{Display}/V_{Control}}$

The CD gain settings can be adjusted in most cases. However, a compromise has to be found: with high gains it is easier to approach a distant target, with low gains this takes longer. High gains hinder the selection of targets, whereas low gains facilitate this process.^[6] The Microsoft, macOS and X window systems have implemented mechanisms which adapt the CD gain to the user's needs. e.g. the CD gain increases when the user's movement velocity increases^[7] (historically referred to as "mouse acceleration").

Common pointing devices

Motion-tracking pointing devices

Mouse

A mouse is a small handheld device pushed over a horizontal surface.

A mouse moves the graphical pointer by being slid across a smooth surface. The conventional roller-ball mouse uses a ball to create this action: the ball is in contact with two small shafts that are set at right angles to each other. As the ball moves these shafts rotate, and the rotation is measured by sensors within the mouse. The distance and direction information from the sensors is then transmitted to the computer, and the computer moves the graphical pointer on the screen by following the movements of the mouse. Another common mouse is the optical mouse. This device is very similar to the conventional mouse but uses visible or infrared light instead of a roller-ball to detect the changes in position.^[8] Additionally there is the mini-mouse, which is a small egg-sized mouse for use with laptop computers; usually small enough for use on a free area of the laptop body itself, it is typically optical, includes a retractable cord and uses a USB port to save battery life.

Trackball

A trackball is a pointing device consisting of a ball housed in a socket containing sensors to detect rotation of the ball about two axis, similar to an upside-down mouse: as the user rolls the ball with a thumb, fingers, or palm the pointer on the screen will also move. Tracker balls are commonly used on CAD workstations for ease of use, where there may be no desk space on which to use a mouse. Some are able to clip onto the side of the keyboard and have buttons with the same functionality as mouse buttons.^[9] There are also wireless trackballs which offer a wider range of ergonomic positions to the user.

Joystick

Isotonic joysticks are handle sticks where the user can freely change the position of the stick, with more or less constant force.

Isometric joysticks are where the user controls the stick by varying the amount of force they push with, and the position of the stick remains more or less constant. Isometric joysticks are often cited as more difficult to use due to the lack of tactile feedback provided by an actual moving joystick.

Pointing stick

A pointing stick is a pressure-sensitive small nub used like a joystick. It is usually found on laptops embedded between the G, H, and B keys. It operates by sensing the force applied by the user. The corresponding "mouse" buttons are commonly placed just below the space bar. It is also found on mice and some desktop keyboards.

Wii Remote

The Wii Remote, also known colloquially as the Wiimote, is the primary controller for Nintendo's Wii console. A main feature of the Wii Remote is its motion sensing capability, which allows the user to interact with and manipulate items on screen via gesture recognition and pointing through the use of accelerometer and optical sensor technology.

Finger tracking

A finger tracking device tracks fingers in the 3D space or close to the surface without contact with a screen. Fingers are triangulated by technologies like stereo camera, time-of-flight and laser. Good examples of finger tracking pointing devices are LM3LABS' Ubiq'window and AirStrike

Position-tracking pointing devices

Graphics tablet

A graphics tablet with a pen

A graphics tablet or digitizing tablet is a special tablet similar to a touchpad, but controlled with a pen or stylus that is held and used like a normal pen or pencil. The thumb usually controls the clicking via a two-way button on the top of the pen, or by tapping on the tablet's surface.

A cursor (also called a puck) is similar to a mouse, except that it has a window with cross hairs for pinpoint placement, and it can have as many as 16 buttons. A pen (also called a stylus) looks like a simple ballpoint pen but uses an electronic head instead of ink. The tablet contains electronics that enable it to detect movement of the cursor or pen and translate the movements into digital signals that it sends to the computer."^[10] This is different from a mouse because each point on the tablet represents a point on the screen.

Stylus

A smartphone being operated with a stylus

A stylus is a small pen-shaped instrument that is used to input commands to a computer screen, mobile device or graphics tablet.

The stylus is the primary input device for personal digital assistants and smartphones that require accurate input, although devices featuring multi-touch finger-input with capacitive touchscreens have become more popular than stylus-driven devices in the smartphone market.

Touchpad

Trackpad on an Apple MacBook Pro

A touchpad or trackpad is a flat surface that can detect finger contact. It is a stationary pointing device, commonly used on laptop computers. At least one physical button normally comes with the touchpad, but the user can also generate a mouse click by tapping on the pad. Advanced features include pressure sensitivity and special gestures such as scrolling by moving one's finger along an edge.

It uses a two-layer grid of electrodes to measure finger movement: one layer has vertical electrode strips that handle vertical movement, and the other layer has horizontal electrode strips to handle horizontal movements.^[11]

Touchscreen

A virtual keyboard on an iPad

A touchscreen is a device embedded into the screen of the TV monitor, or system LCD monitor screens of laptop computers. Users interact with the device by physically pressing items shown on the screen, either with their fingers or some helping tool.

Several technologies can be used to detect touch. Resistive and capacitive touchscreens have conductive materials embedded in the glass and detect the position of the touch by measuring changes in electric current. Infrared controllers project a grid of infrared beams inserted into the frame surrounding the monitor screen itself, and detect where an object intercepts the beams.

Modern touchscreens could be used in conjunction with stylus pointing devices, while those powered by infrared do not require physical touch, but just recognize the movement of hand and fingers in some minimum range distance from the real screen.

Touchscreens are becoming popular with the introduction of palmtop computers like those sold by the Palm, Inc. hardware manufacturer, some high range classes of laptop computers, mobile smartphone like HTC or the Apple Inc. iPhone, and the availability of standard touchscreen device drivers into the Symbian, Palm OS, Mac OS X, and Microsoft Windows operating systems.

Pressure-tracking pointing devices

Isometric Joystick

In contrast to a 3D Joystick, the stick itself doesn't move or just moves very little and is mounted in the device chassis. To move the pointer, the user has to apply force to the stick. Typical representatives can be found on notebook's keyboards between the "G" and "H" keys. By performing pressure on the TrackPoint, the cursor moves on the display.^[12]

Other devices

• A light pen is a device similar to a touch screen, but uses a special light-sensitive pen instead of the finger, which allows for more accurate screen input. As the tip of the light pen makes contact with the screen, it sends a signal back to the computer containing the coordinates of the pixels at that point. It can be used to draw on the computer screen or make menu selections, and does not require a special touch screen because it can work with any CRT display.
• Light gun
• Palm mouse – held in the palm and operated with only two buttons; the movements across the screen correspond to a feather touch, and pressure increases the speed of movement
• Footmouse – sometimes called a mole – a mouse variant for those who do not wish to or cannot use the hands or the head; instead, it provides footclicks
• Similar to a mouse is a puck, which, rather than tracking the speed of the device, tracks the absolute position of a point on the device (typically a set of crosshairs painted on a transparent plastic tab sticking out from the top of the puck). Pucks are typically used for tracing in CAD/CAM/CAE work, and are often accessories for larger graphics tablets.
• Eye tracking devices – a mouse controlled by the user's retinal movements, allowing cursor-manipulation without touch
• Finger-mouse – An extremely small mouse controlled by two fingers only; the user can hold it in any position
• Gyroscopic mouse – a gyroscope senses the movement of the mouse as it moves through the air. Users can operate a gyroscopic mouse when they have no room for a regular mouse or must give commands while standing up. This input device needs no cleaning and can have many extra buttons, in fact, some laptops doubling as TVs come with gyroscopic mice that resemble, and double as, remotes with LCD screens built in.
• Steering wheel – can be thought of as a 1D pointing device – see also steering wheel section of game controller article
• Paddle – another 1D pointing device
• Jog dial – another 1D pointing device
• Yoke (aircraft)
• Some high-degree-of-freedom input devices
• 3Dconnexion – six-degree controller
• Discrete pointing devices
• directional pad – a very simple keyboard
• Dance pad – used to point at gross locations in space with feet
• Soap mouse – a handheld, position-based pointing device based on existing wireless optical mouse technology
• Laser pen – can be used in presentations as a pointing device

References