Human-Computer Interaction

Summary

Human-computer interaction (HCI) is a field concerned with the study, design, implementation, evaluation, and improvement of the ways in which human beings use or interact with computer systems. The importance of human-computer interaction within the field of computer science has grown in tandem with technology's potential to help people accomplish an increasing number and variety of personal, professional, and social goals. For example, the development of user-friendly interactive computer interfaces, websites, games, home appliances, office equipment, art installations, and information distribution systems such as advertising and public awareness campaigns are all applications that fall within the realm of HCI.

Definition and Basic Principles

Human-computer interaction is an interdisciplinary science with the primary goal of harnessing the full potential of computer and communication systems for the benefit of individuals and groups. HCI researchers design and implement innovative interactive technologies that are not only useful but also easy and pleasurable to use and anticipate and satisfy the specific needs of the user. The study of HCI has applications throughout every realm of modern life, including work, education, communications, health care, and recreation.

The fundamental philosophy that guides HCI is the principle of user-centered design. This philosophy proposes that the development of any product or interface should be driven by the needs of the person or people who will ultimately use it, rather than by any design considerations that center around the object itself. A key element of usability is affordance, the notion that the appearance of any interactive element should suggest the ways in which it can be manipulated. For example, the use of shadowing around a button on a website might help make it look three-dimensional, thus suggesting that it can be pushed or clicked. Visibility is closely related to affordance. It is the notion that the function of all the controls with which a user interacts should be clearly mapped to their effects. For example, a label such as “Volume Up” beneath a button might indicate exactly what it does. Various protocols facilitate the creation of highly usable applications. A cornerstone of HCI is iterative design, a method of development that uses repeated cycles of feedback and analysis to improve each prototype version of a product, instead of simply creating a single design and launching it immediately. To learn more about the people who will eventually use a product and how they will use it, designers also make use of ethnographic field studies and usability tests.

Background and History

Before the advent of the personal computer, those who interacted with computers were largely technology specialists. In the 1980s, however, more and more individual users began making use of software such as word-processing programs, computer games, and spreadsheets. HCI as a field emerged from the growing need to redesign such tools to make them practical and useful to ordinary people with no technical training. The first HCI researchers came from a variety of related fieldscognitive science, psychology, computer graphics, human factors (the study of how human capabilities affect the design of mechanical systems), and technology. Among the thinkers and researchers whose ideas have shaped the formation of HCI as a science are John M. Carroll, best known for his theory of minimalism (an approach to instruction that emphasizes real-life applications and the chunking of new material into logical parts), and Adele Goldberg, whose work on early software interfaces at the Palo Alto Research Center (PARC) was instrumental in the development of the modern graphical user interface.

In the early days of HCI, the notion of usability was simply defined as the degree to which a computer system was easy and effective to use. However, usability has come to encompass a number of other qualities, including whether an interface is enjoyable, encourages creativity, relieves tension, anticipates points of confusion, and facilitates the combined efforts of multiple users. In addition, there has been a shift in HCI away from a reliance on theoretical findings from cognitive science and toward a more hands-on approach that prioritizes field studies and usability testing by real participants.

How It Works

Input and Output Devices. The essential goal of HCI is to improve the ways in which information is transferred between a user and the machine they are using. Input and output devices are the basic tools HCI researchers and professionals use for this purpose. The more sophisticated the interaction between input and output devices—the more complex the feedback loop between the two directions of information flow—the more the human user will be able to accomplish with the machine.

An input device is any tool that delivers data of some kind from a human to a machine. The most familiar input devices are the ones associated with personal computerskeyboards and mice. Other commonly used devices include joysticks, trackballs, pen styluses, and tablets. Still more unconventional or elaborate input devices might take the shape of headgear designed to track the movements of a user's head and neck, video cameras that track the movements of a user's eyes, skin sensors that detect changes in body temperature or heart rate, wearable gloves that precisely track hand gestures, or automatic speech recognition devices that translate spoken commands into instructions that a machine can understand. Some input devices, such as the sensors that open automatic doors at the fronts of banks or supermarkets, are designed to record information passively, without the user having to take any action.

An output device is any tool that delivers information from a machine to a human. Again, the most familiar output devices are those associated with personal computersmonitors, flat-panel displays, and audio speakers. Other output devices include wearable head-mounted displays or goggles that provide visual feedback directly in front of the user's field of vision and full-body suits that provide tactile feedback to the user in the form of pressure.

Perceptual-Motor Interaction. When HCI theorists speak about perceptual-motor interaction, they are referring to the notion that users' perceptions—the information they gather from the machine—are inextricably linked to their physical actions or how they relate to the machine. Computer systems can use input and output devices to provide feedback about the user's actions that help them make the next move. For example, a word on a website may change color when a user hovers the mouse over it, indicating it is a functional link. A joystick used in a racing game may exert what feels like muscular tension or pressure against the user's hand in response to the user steering the device left or right. Ideally, system feedback should align with the direction of the user's movements of the input device. For example, the direction in which a cursor moves on the screen should be the same as the direction in which the user moves the mouse. This is known as kinesthetic correspondence.

Another technique HCI researchers have devised to facilitate the feedback loop between a user's perceptions and actions is known as augmented reality. With this approach, rather than providing the user with data from a single source, the output device projects digital information, such as labels, descriptions, charts, and outlines, on the physical world. When an engineer is looking at a complex mechanical system, for example, the display might show what each part in the system is called and enable them to call up additional troubleshooting or repair information.

Applications and Products

Computers. At one time, interacting with a personal computer required knowing how to use a command-line interface in which the user typed in instructions—often worded in abstract technical language—for a computer to execute. A graphical user interface, based on HCI principles, supplements or replaces text-based commands with visual elements such as icons, labels, windows, widgets, menus, and control buttons. These elements are controlled using a physical pointing device such as a mouse. For instance, a user may use a mouse to open, close, or resize a window or to pull down a list of options in a menu in order to select one. The major advantage graphical user interfaces have over text-based interfaces is that they make completing tasks far simpler and more intuitive. Using graphic images rather than text reduces the amount of time it takes to interpret and use a control, even for a novice user. This enables users to focus on the task at hand rather than to spend time figuring out how to manipulate the technology itself. For instance, rather than having to recall and then correctly type in a complicated command, a user can print a particular file by selecting its name in a window, opening it, and clicking on an icon designed to look like a printer. Similarly, rather than choosing options from a menu in order to open a certain file within an application, a user might drag and drop the icon for the file onto the icon for the application.

Besides helping individuals navigate through and execute commands in operating systems, software engineers also use HCI principles to increase the usability of specific computer programs. One example is the way pop-up windows appear in the word-processing program Microsoft Word when a user types in the salutation in a letter or the beginning item in a list. The program is designed to recognize the user's task, anticipate the needs of that task, and offer assistance with formatting customized to that particular kind of writing.

Consumer Appliances. Besides computers, a host of consumer appliances use aspects of HCI design to improve usability. Graphic icons are ubiquitous parts of the interfaces commonly found on cameras, stereos, microwave ovens, refrigerators, and televisions. Smartphones like Apple's iPhones rely on the same graphic displays and direct manipulation techniques used in full-sized computers. Most have extra tactile, or haptic, dimensions such as touchscreen keyboards and the ability to rotate windows on the device by physically rotating the device. Some video game consoles have moved away from keyboard and joystick interfaces, which may not have kinesthetic correspondence, toward far more sophisticated controls. The hand-held Nintendo Wii controller allows players to control the game by moving their bodies. Finally, HCI research influences the physical design of many household devices. For example, an appliance plug designed with the user in mind might be shaped so it can be inserted into an outlet in any orientation based on the understanding that a user may have to fit several plugs into a limited space. Many appliances have bulky plugs that take up the space of more than one plug.

Increasingly, HCI research is helping appliance designers move toward multimodal user interfaces. These systems engage the array of human senses and physical capabilities, match particular tasks to the modalities that are easiest and most effective for people to use, and respond in tangible ways to users’ actions and behaviors. Multimodal interfaces combine input devices for collecting data from the user (such as video cameras, sound recording devices, and pressure sensors) with software tools that use statistical analysis or artificial intelligence to interpret these data (such as natural language processing programs and computer vision applications). For example, a multimodal interface for a GPS installed in an automobile allows the user to speak the name of a destination rather than typing it in while driving. The system might use auditory processing of the user's voice and visual processing of their lip movements to more accurately interpret speech. It might also use a camera to closely follow the movements of the user's eyes, tracking their gaze from one part of the screen to another and using this information to helpfully zoom in on particular parts of the map or automatically select a particular item in a menu.

Similarly, in 2015, Amazon released its Echo device with the Alexa virtual assistant, advancing Bluetooth technology. Built-in programs allow Echo users to give voice commands instructing the device to play music or sync with other devices. In the 2020s, Amazon continued to expand the Echo program and the devices with which it is compatible. Smart home technologies allow users to control lighting, thermostats, security alarms, and door locks. Later generations of Echo technology were smaller and more portable with additional features, such as Wi-Fi extension, motion sensing capabilities, and better-quality speakers. Echo Look uses artificial intelligence to suggest outfits. Echo Show pairs with other devices to play videos, display photos, or make video calls. Wearable devices, like Echo Frames and Echo Loops, allow users access to Alexa technology by wearing glasses or a ring. Similar devices were also developed by companies such as Google and Apple.

Workplace Information Systems. HCI research plays an important role in many products that enable people to perform workplace tasks more effectively. For example, modern computer systems and software increase safety and efficiency in air traffic control. Some systems collect data about the operator's pupil size, facial expression, heart rate, and the forward momentum and intensity of their mouse movements and clicks. This information helps the computer interpret the operator's behavior and state of mind and respond accordingly. When an airplane drifts slightly off its course, the system analyzes the operator's physical modalities. If their gaze travels quickly over the relevant area of the screen, with no change in pupil size or mouse click intensity, the computer might conclude that the operator has missed the anomaly and attempt to draw attention to it by using a flashing light or an alarm.

Other common workplace applications of HCI include products that facilitate communication and collaboration between team members, such as messaging programs like Microsoft Teams and videoconferencing tools like Zoom. Additionally, HCI principles have contributed to many project management tools that enable groups to schedule and track their progress on a shared task or to make changes to shared documents without overriding someone else's work.

Education and Training. Schools, museums, and businesses all make use of HCI principles when designing educational and training curricula for students, visitors, and staff. For example, many school districts have been moving away from printed textbooks and toward interactive electronic programs that target a variety of information-processing modalities through multimedia. Unlike paper and pencil worksheets, such programs also provide instant feedback, making it easier for students to learn and understand new concepts. Businesses use similar programs to train employees in such areas as the use of new software and the company's policies on issues of workplace ethics. Many art and science museums have installed electronic kiosks with touchscreens that visitors can use to learn more about a particular exhibit. HCI principles underlie the design of such kiosks. For example, rather than using a text-heavy interface, the screen on an interactive kiosk at a science museum might display a video of a museum staff member talking to the visitor about each available option.

Careers and Course Work

The paths toward becoming an HCI professional are extraordinarily varied. Bachelor's degrees in cognitive science, neuroscience, computer science, graphic design, psychology, engineering, art, and many other fields could serve as appropriate preparation for a career as someone who uses HCI principles. No matter which concentration an aspiring HCI researcher or student chooses, acquiring basic programming skills, a broad understanding of human psychophysiology, and some practical experience or training with graphic or product design is important. Common areas of work include developing websites; computer operating systems; interfaces for consumer appliances such as cell phones, printers, or cameras; and educational materials such as interactive employee training courses, advertising campaigns, or any other applications that demand accessible, learnable, and usable computer systems. Although a graduate degree is optional to enter the field, many universities offer specialized HCI master's programs.

Social Context and Future Prospects

As HCI advances research into multimodal interfaces and ubiquitous computing, notions of the computer as an object separate from the user may eventually be relegated to the archives of technological history, to be replaced by wearable machine interfaces that can be worn like clothing on the user's head, arm, or torso. 

Modern smartwatches, like Apple, Google, and Samsung devices, are designed to have all the features of smartphones and more in a wearable, theoretically more convenient format. Much like other wearable gadgets such as Fitbit, playing into society's increased concern with exercise and overall health, smartwatches can track heart rate and blood oxygen levels and serve as a GPS that maps running, walking, and biking routes. Modern virtual reality interfaces can immerse the user in a 360-degree space that looks, sounds, feels, and perhaps smells like a natural environment—with which they can interact naturally and intuitively, using their whole bodies. As the capacity to measure the physical properties of humans becomes ever more sophisticated, input devices may grow more sensitive. 

Future machines might “listen in” to the synaptic firings of the neurons in a user's brain and respond accordingly. Indeed, it is not beyond the realm of possibility that a means could stimulate a user's neurons to produce direct visual or auditory sensations. The future of HCI research is essential in the workplace, home, recreational spaces, and society.

Bibliography

Bogdon, Razuan, et al. "A Practical Experience on the Amazon Alexa Integration in Smart Offices." Sensors, vol. 21, no. 3, 22 Jan. 2021. MDPI, doi.org/10.3390/s21030734. Accessed 16 July 2024.

Cojocar, Grigoreta-Sofia, and Adriana-Mihaela Guran. Computational Intelligence and Human- Computer Interaction: Modern Methods and Applications. MDPI, 2022.

"Introduction to Human Computer Interaction (HCI) with Examples." Analytics Vidhya, 10 Apr. 2024, www.analyticsvidhya.com/blog/2023/05/human-computer-interaction. Accessed 16 July 2024.

Jokinen, Jussi P. P. "Emotional User Experience: Traits, Events, and States." International Journal of Human-Computer Studies, vol. 76, 2015, pp. 67–77. ScienceDirect, doi.org/10.1016/j.ijhcs.2014.12.006. Accessed 16 July 2024.

Kumar, Sandeep, et al. Cognitive Behavior and Human Computer Interaction Based on Machine Learning Algorithms. Scrivener Publishing Wiley, 2022.

Pressman, Aaron. "Alexa Turns 5: What Amazon's Kindergarten-Aged Assistant Can Teach the Rest of Tech." Fortune, 6 Nov. 2019, fortune.com/2019/11/06/amazon-alexa-echo-5-anniversary-dave-limp-interview. Accessed 16 July 2024.

Rapp, Amon. “Human-Computer Interaction.” Oxford Research Encyclopedia of Psychology, 2023, doi.org/10.1093/acrefore/9780190236557.013.47. Accessed 16 July 2024.

Sears, Andrew, and Julie A. Jacko, editors. The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. 2nd ed., Erlbaum, 2008.

Sharp, Heken, et al. Interaction Design: Beyond Human-Computer Interaction. 6th ed., John Wiley & Sons, 2023.

Soegaard, Mads, and Rikke Friis Dam, editors. The Encyclopedia of Human-Computer Interaction. 2nd ed., Interaction Design Foundation, 2014.

Tufte, Edward R. The Visual Display of Quantitative Information. 2nd ed., Graphics, 2018.