Big data
Big data refers to the vast volume of data generated daily by various global computer networks, surpassing the capacity of traditional databases and software systems to manage efficiently. This phenomenon has emerged since the mid-1980s, driven by advancements in computer technology that enhance storage and processing capabilities. Organizations from diverse sectors, including government, business, healthcare, and social media, face significant challenges in collecting, analyzing, and storing this massive influx of information, which is estimated to have grown substantially since 2010.
The concept of big data is characterized by four key dimensions: variety, volume, velocity, and veracity of the information. Effectively harnessing this data presents opportunities for improved decision-making, enhanced organizational efficiency, and innovative applications such as global health research and national security monitoring. However, the rise of big data also raises concerns about privacy and surveillance, with critics warning of potential misuse and the risk of individuals being reduced to mere data points. These discussions highlight the importance of thoughtful and ethical approaches in navigating the complexities of big data in a culturally sensitive manner.
On this Page
Subject Terms
Big data
In the cutting-edge world of data retrieval systems, big data is in-field shorthand that refers to the sheer mass of data produced daily by and within global computer networks at a pace that far exceeds the capacity of current databases and software programs to organize and process. Virtually any organization that gathers information—government, businesses, retail stores, services, hospitals, social media outlets—faces an enormous challenge in gathering information that is generated at massive rates every minute, reading that data into meaningful and coherent information, and then storing it efficiently and effectively. Given that the number of qualified computer software engineers savvy in this burgeoning field is far below industry needs, the strategies for directing and controlling big data represent a primary challenge to those who rely on computers for operating information.
Overview
The amount of information processed by computer networks globally every day is measured in terabytes (one trillion bytes) and petabytes (one quadrillion bytes). It is estimated that 90 percent of the sum total of data in the world today, incorporating general information, statistics, medical records, science and research data, phone records, business records, government records, etc., has been generated just since 2010. Data generation on that scale is measured four ways: 1) the variety of the information available (from cell phone records to air travel information, from social network inputs to stock trading); 2) the volume of information produced, measured per second; 3) the velocity at which the information is produced; and 4) its veracity (that is, its meaningfulness).
The ability to harness and direct that information represents an entirely new paradigm for virtually any organization from governments to businesses, from hospitals to school systems. The potential rewards are significant. Any interaction with computers—smartphones, e-readers, digital music players, tablets, laptops, Internet searches—generates data about individuals. This data is useful to businesses seeking to direct their resources most favorably, from accumulating customer profiles to directing product development and marketing. More importantly, big data research could be directed toward the global monitoring of potential terrorist activities, given that chatter from such sources is nearly constant. Medical records could be synthesized and coordinated globally with and against available research data about treatments. Government records could be streamlined and government services could be made far more efficient. Organization performance would be boosted, records would be more accessible, and networks could communicate effectively across traditional boundaries.
Information itself would become fluid and transparent. Critics of big data research, however, point to a Big Brother scenario, in which intrusion into privacy could proceed unchecked. All computer activity could be monitored and theoretically stored by the government, by employers, by businesses—really by anyone with access to what would become unprecedentedly massive data reservoirs. In addition, such widespread harvesting of information creates new threats to intellectual property rights and copyright infringements. And gathering data, these critics point out, is not effectively analyzing it, thus stripping the data from the sociocultural context that provides its true meaning. People, they argue, are more than data points.
Bibliography
Big Data Now: Current Perspectives from O’Reilly Radar. Sebastopol: O’Reilly Media, 2012. Ebook.
Cukier, Kenneth. Big Data: A Revolution that Will Transform How We Live, Work, and Think. Boston: Dolan, 2013. Print.
Franks, Bill. Taming the Big Data Trial Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics. New York: Wiley, 2012. Print.
Kolb, Jason, and Jeremy Kolb. The Big Data Revolution. Applied Data Labs, 2013. Print.
Nielsen, Lars. Hadoop: The Engine That Drives Big Data. Wickford: New Street Communications, 2013. Print.
Pentland, Alex. "Saving Big Data from Itself." Scientific American 311.2 (2014): 65–67. Print.
Sathi, Arvind. Big Data Analytics: Disruptive Technology for Changing the Game. Boise: MC Press Online. MC P, 2013. Print.
Schmidt, Eric, and Jared Cohen. The New Digital Age: Reshaping the Future of People, Nations, andBusiness. New York: Knopf, 2013. Print.
Smolan, Rick, and Jennifer Erwitt. The Human Face of Big Data. Sausalito: Against All Odds, 2012. Print.