Pointer And Keyboard

For discussion of benefits and conflicts between pointer (usually a mouse) and keyboard usage. Related topics such as menus, function keys, and command strings are also appropriate. This topic is often the "input" side of the CLI/GUI arguments, as opposed to the "output" topic of GraphicAndCharacterDisplay?. (Many character-based interfaces use a pointer. Many people treat their GUI systems as a collection of character-based windows and use the pointer for tasks such as text selection.)

Antipattern:

Too often programs have *some* things that can only be done with a mouse (say, typing in commands), while *other* things can only be done with the keyboard (say, clicking OK when a "dangerous" command pops up the "Do you really want to do that ?" dialog box). This forces people to switch back and forth. Most people find this annoying.
In reaction, too often programmers say "OK, we're going to do *everything* with the mouse". Some people find this annoying.
In reaction, too often programmers say "OK, we're going to do *everything* with the keyboard". A different group of people find this annoying.

Solution:

we've agreed on:

Every command should be available from a menu, because beginning users with long acquisition times will find them fastest that way.
Each mouse command can have at least one keyboard equivalent, so that users who have no mouse, whose mouse is broken, or who refuse to use a mouse can still use the system (albeit inefficiently).

we've agreed on:

Some tasks, which users have memorized and which can be performed very quickly, are optimally invoked through keyboard equivalents.
Some tasks are ideally suited for the mouse, even given keyboard equivalents.

See also: GraphicAndCharacterDisplay?, GuiAndCli?, TheDumbingDownOfProgramming, MustEveryComparisonBeVs? (:-)

From TheDumbingDownOfProgramming:

I suspect that those who are more productive in CUIs are those who type very well, and those who are more productive in GUIs are those who type less well; for them, clicking and pointing is faster than typing. For a very good typist, clicking and pointing can be much, much slower (and less expressive). Of course, it depends upon what you're doing. I'd hate to draw using a CUI just as much as I hate to compile using a GUI.

I don't mean to pick on anyone; I mean only that in 1999 we can and should do better then to "suspect" and speculate. Moving your hands from home position takes about five keystrokes. If you don't touch-type, everything is slower. Even if you touch type, finding and reaching the F12 key is as expensive as a mouseclick. Finding and reaching a meta-key combination that pulls your fingers off home is as expensive as a mouse click. Any mouse operation can have a keyboard equivalent specified for it.

I would prefer us know and cite the literature, instead of saying "What studies?". Please see, for example, "Card, Moran, and Newell (1980), The Keystroke-Level Model for User Performance Time with Interactive Systems, Comm. ACM, 23(7), 396-410".

I think you're overstating things. The studies have shown some things to be true about the interaction of menu items and GUIs. But that's far from saying the subject's closed in all areas. Time to find and perform a command isn't all there is to say about productivity. Acquisition times vary depending on the user (and can be very slow for beginning users, which argues for having all commands on a menu).

All mouse commands can indeed have a keyboard equivalent, but there's no way that a keyboard command sequence is going to beat my mouse when moving a image from "somewhere in the top left ish" to "somewhere in the bottom right, there, a little bit above the text" (which is more or less the spec. my daughter gave me recently while editing a birthday card).

Now this isn't an excuse for ignorance of the literature in areas where it is relevant, of course.

So it sounds to me like you agree with my observations, even if I have overstated them. Let my try and summarize what I think we've agreed on: Every command should be available from a menu, because beginning users with long acquisition times will find them fastest that way. Each mouse command can have at least one keyboard equivalent, so that users who have no mouse, whose mouse is broken, or who refuse to use a mouse can still use the system (albeit inefficiently). Some tasks, which users have memorized and which can be performed very quickly, are optimally invoked through keyboard equivalents. Some tasks are ideally suited for the mouse, even given keyboard equivalents. And there is no excuse for ignorance of the literature. Which part have I overstated?

Perhaps more significantly, I'm disappointed that, to judge from comments such as those posted recently, the old prejudice that GUI's-are-for-dummies and that real-programmers-use-command-lines still lives, even when it has been shown to be unsupportable by relevant science.

I was thinking about what it is about mice I don't like, and I think it's something more important than speed. When I have to move my hand off the keyboard to the mouse, it feels like I'm switching modes in my brain. This mode switch is more disturbing than lost speed. It has the same feel as the telephone ringing while concentrating on something, an unwanted interruption that interferes with flow. I think that even if you can come up with bunches of science to prove that the mouse is, in some measurable way, better for what I do, you won't be able to change one bit the fact that mice feel bad to me. You can call it prejudice, but I prefer to think of it as a personal human factor.

I appreciate your candor. You embrace a venerable tradition that I have no problem with. You said, in effect, "I prefer the keyboard". That is perfectly legitimate, and has nothing to do with prejudice. What I call "prejudice" is if you were to accompany it with a statement along the lines of "and you're dumb if feel otherwise". Further, you have explicitly separated your feelings from the relevant science -- another perfectly reasonable stance. The position I object to is the position that says, in effect, "I like the keyboard, and so the keyboard is better, and everyone who prefers the mouse is dumb". That is not what I hear you say. A "good" user interface has keyboard equivalents in order to be responsive to users who feel as you do. The comparable stance, in the audiophile community, is "I prefer the sound of tubes and vinyl. I know it is not as 'accurate'; nevertheless, I prefer the warmth and coloration of my analog system." It is the effort to generalize personal experience, in the face of contrary relevant science, that I have a problem with.

You're both dumb to think that either the mouse alone or the keyboard alone, are any use to anyone at all.

I cite DougEngelbart's NLS, which had a chording keyboard for the left hand and a mouse for the right hand.

I cite 99% of all games created in the last 10 years which uniformly have keyboard control for the left hand and mouse control for the right.

I cite the fact that, excluding games programmers, all programmers are complete morons at interface design. The fact that morons, including you two, go for either 1) pure mouse, 2) pure keyboard, or 3) complete equivalence of mouse and keyboard, is all the proof I need to condemn all three options.

There should be only one mode of input, a perfect mouse/keyboard combination. That's the way I feel and you're dumb if you think otherwise.

No... I want it to be that if your mouse is broken, you can still use it, and if your keyboard is broken, you can still use it, so long as you are OK with using the on-screen keyboard for entering actual text (as opposed to hitting ctrl-c, ctrl-v, left, right, ctrl-shift-P).

Richard, I am a fan of DougEngelbart's chord keyboard plus mouse idea, but I have to wonder, are you using one right now, or is this just all talk?

Consider it part and parcel of being a contrarian. When people invest so much effort in polite and proper behaviour that it makes their writing stilted, I feel an unrestrainable impulse to slap some bad manners into them. Hence the gratuitous offense.

So I hope that answered your implied question. Unless you seriously wonder about chording. If you do then I can tell you that chording doesn't seem optimal to me (and I'm not up to spending 100+ hours learning a completely alien keymap when I still spend half my time typing qwerty). I prefer the light-glove and ring mouse instead, neither of which I have because I'm just too damned lazy. Then again, I don't have one of those 400$ CDN Kinesys keyboards either.

Why not using the natural parallelity of our motoric surface, outsourcing the pointer to our toetips {or something similar}. See: ShoeKeyboard. Then there is {probably} no {significant} mode-switching. FridemarPache

Preferences for keyboards are also supported by relevant science.

See "Individual Differences in the Use of Command Line and Menu Computer Interfaces" by S. J. Westerman in the International Journal of Human-Computer Interaction 1997 v.9 n.2 p.183-198. (abstract at http://www.hcibib.org/gs.cgi?word=checked&terms=J.IJHCI.9.2.183 )

: This article presents an experimental investigation of the process of computer-based command generation. The comparative cognitive demands imposed by menu and command line interfaces are examined in relation to individual differences in expertise and cognitive ability. Three-way interactions between associative memory, expertise, and command generation method indicated similarities in the performance of expert participants with low associative memory and that of novices. Spatial memory also interacted with expertise, with novices with low spatial memory performing more poorly than any other group. Implications for interface design are considered.

Also see "Social Influence and Preference of Direct-Manipulation and Keyboard-Command Computer Interfaces" in Personality and Individual Differences in Human Performance: Individual Differences in Man-Machine Interaction by Michael S. Wogalter and Richard L. Frei from the Proceedings of the Human Factors Society 34th Annual Meeting 1990 v.2 p.907-911 (abstract at http://www.hcibib.org/gs.cgi?word=checked&terms=C.HFS.90.907 )

: Direct-manipulation and command-based computer interfaces have each found their own following among microcomputer users. This study explores some of the differences between these two groups of computer users. Participants completed a questionnaire that requested their microcomputer usage and ownership, usage and preference of various command methods and pointing devices, the microcomputers most of their friends use, the microcomputer they would be most willing to purchase next, and their preference for several models of microcomputers. The results showed that participants preferred pointing devices (e.g., mouse) compared to other input methods (e.g., arrow keys) regardless of their prior usage. They tended to use an interface similar to that of their friends' and they reported greater willingness to purchase a computer with an interface similar to the one they most often use. In general, the results suggest that social influence and interface familiarity are important factors in determining which interface people choose to use. Being surrounded by others who use a similar computer interface eases the burden (in terms of effort, time, and expense) of obtaining relevant computer information. An implication of this work is that these variables may hinder approval and acceptance of improved computer interface designs offered by human factors specialists.

Finally, the following might be interesting: "Theoretical Models for System Design" in Computer Systems: Panel with Michael E. Atwood, Gerhard Fischer, Wayne D. Gray, and Peter G. Polson in the Proceedings of the Human Factors Society 33rd Annual Meeting 1989 v.1 p.278-280 (abstract at http://www.hcibib.org/gs.cgi?word=checked&terms=C.HFS.89.278 )

: In the history of human factors in computer systems, one of the most significant events of the past decade was the work on GOMS and keystroke models (cf. Card, Moran, and Newell, 1983). While a clear success in causing software developers to focus on the importance of interface design and attracting researchers to this areas, GOMS approaches have not significantly improved the quality of the systems that are developed.
: Why has this work, that has a great theoretical impact, had so little practical impact on existing systems? Is it that the GOMS formalism is not valid outside of laboratory contexts? Is it that it misses important aspects of behavior such as how people learn to use systems? Is it that GOMS was developed in the context of computer systems that are less powerful and interactive than we have today? Or, are there other reasons?
: In this panel, we argue that additional cognitive science approaches are needed to improve the quality of developed system. Dr. Gray extends this approach by reporting the first "real world": test of the GOMS-style of system modeling. Dr. Polson extends these models to how people learn to use systems. Dr. Fischer extends this style of research by focusing on cooperative, rather than passive computer systems. Audience members will have an opportunity to describe other approaches to developing theoretical models of system design.

The HCIBIB site at http://www.hcibib.org/ has over 19000 searchable bibliography entries--it's a good place to start looking for modern research. From a few hours of looking around it seems that voice and touch-interfaces are hot topics, with several studies showing improvements over mice and keyboards.

Does it let you actually read the article? I wouldn't want to back up an argument with only an abstract. Citing the literature is a great idea, but many of us don't have easy access to it.

Tog has done actual research on this. At http://www.asktog.com/SunWorldColumns/S02KeyboardVMouse3.html he describes how the stopwatch found all users faster using the mouse, but the same users all thought they were faster using the keyboard. There is an interesting psychological effect going on here.

''This is somewhat of a StrawMan test case. If I were going to replace every bar with an e like he said, using the keyboard, I would use vi and set up search pattern for bar, do one replace, and then do a rapid repeat of . and n. In other words, the test was skewed to favor the mouse because the text-only option was weaker than the mouse-only. I wonder if the other tests comparing keyboards and mice usage are done with vi/emacs or Word/Notepad?'' --Pete Hardie

Agreed. In OpenOffice, that's Ctrl-F, Shift-\ ("|"), Tab, E ("e"), Alt-L, Enter, Alt-C. For those who can't simulate OO.org in their mind, here's the breakdown:

Ctrl-F brings up the find dialog box.
The backslash key is also the bar key. The shift is included for keystroke completeness.
Tab moves the cursor to the "replace with:" box.
I type "e".
Alt-L activates "replace all".
Enter again closes the dialog box telling me how many times the selection was replaced.
Alt-C is the quickest way of closing the dialog box.

Total keystrokes: 11.

Just for kicks, here's the equivalent with mouse only:

Select "edit" menu.
Click on "Find and replace".
Type "|" (the "Search For" box is already selected).
Click the "Replace With" box.
Type "e".
Click "Find all".
Click "OK" to close the notification box.
Close the "Find and Replace" dialog box.

(If someone could total up the number of pixels the mouse has to move for that, that would be awesome.)

--AnonymousDonor.