For Today’s Graduate, Just One Word: Statistics, Essay Example

Since the creation of the internet, information has begun to expand exponentially. On the surface, this would not be a problem. And to be totally honest, the problem is not the fact that data is being generated at an ever increasing pace. The problem is also not how much data is currently in existence. The issue is where to store all of this data. This necessitated to emergence of big data that refers to the exponential nature of growth coupled with availability of structured as well as unstructured data. Big data plays a crucial role in business as well as the entire society and it is associated with the growth that has been witnessed in the internet. Big data also increases the accuracy of analysis that in turn results to higher degree of confidence in the process of decision making. Enhanced decision making consequently leads to higher echelons of operation efficiency, reduction in risks as well as reduction in the costs of doing business.

Amazon and other companies like it must store tens of terabytes of information. This means that the available information at Amazon is significantly huge. ChoicePoint is a company that stores information about the entire population of the United States of America. ChoicePoint stores hundreds of terabytes of information. The National Research Scientific Computing Center stores over 2.8 petabytes of information. Even more amazingly, the World Data Center for Climate stores over 6 petabytes of information. As recent as fifteen years ago, 256 megabytes of information was considered a lot, and now the data storage capabilities have increased exponentially.

The increasing demands for data storage has caused the creation of a term that is used for it—big data. Big data literally means groups of data that are so large that traditional data processing applications cannot keep up with the demands for such a large amount of data. The two main problems of such large groups of data that will be focused on in this paper are the analysis and the storage—which has been discussed—of such large amounts of data. However, beyond those two issues, there are others that will be briefly touched upon. Some of those other issues are gathering that much data, efficiently organizing and searching it, sharing it, and information privacy.

Analyzing large amounts of data requires thinking outside of the box, which it can be argued is an advantage of big data. Newer and better data analysis techniques were required to keep up with the increased needs of analyzing large amounts of data. However, the reason that it would still be beneficial to have smaller data sets is because it takes less time to analyze smaller data sets. The new and improved data analysis techniques are beneficial, undoubtedly, but the smaller the data set, the faster the analysis.

Gathering multiple terabytes of information is also time consuming. It is quite obvious that, if the data sets being worked with were smaller, the information gathering process would be faster as well.

Searching through vast amounts of information can be difficult as well. Anyone that has ever “defragmented” a hard drive can tell you that the more organized the information is, the faster the search through it goes when you need to find a particular file. That being said, even the most organized database that is multiple petabytes is going to take an extremely long time to search through. Consider this: if you have a small filing cabinet that is well organized and clearly marked, finding a particular file should take mere seconds. On the other hand, if you have an entire building full of files—no matter how well organized it is—finding a file in the building will take exponentially as long to complete.

Sharing vast amounts of data also becomes a problem when the data sets are as large as have been discussed in this paper. The speed of one’s internet and the upload and download capabilities of one’s equipment will obviously play a key role in the amount of time that it takes to transfer large sets of data, but no matter how fast one’s internet is, the transfer of hundreds of gigabytes takes an excruciating amount of time.

The last issue with large data sets is the complication of keeping it secure. Obviously, if the data is stored on equipment that is not connected to the internet, it is very secure. However, often that is not practical because the data needs to be accessible. Also, storing that much data in one centralized place takes up physical space. Storing the data on “the cloud” is an option, but at that point, the security of the information is much worse.

With all of the points mentioned above, big data is an issue. In order to keep up with the influx of information, effort needs to be exerted to crunch the data, to make it smaller. Effort also needs to be exerted to minimize the physical space that said data occupies so that more of it can fit into smaller spaces.

Works Cited

Lohr, S. (2009). For today’s graduate, just one word: statistics. NYTimes.com. Retrieved on November 28, 2012, from http://www.nytimes.com/2009/08/06/technology/06stats.html?_r=2&em

Longbottom, C. (2011) Big data: large problems or huge opportunities? Computerweekly.com. Retrieved on November 28, 2012, from http://www.computerweekly.com/news/2240105424/Big-data-large-problems-or-huge-opportunities

Shen, G. (2011). Unplugged: the disconnect of intelligence and analytics. Information Management, 21(1), 14. Retrieved on November 28, 2012, from http://www.information-management.com/issues/21_1/unplugged-10019478-1.html

Trembly, A. C. (2010). The problem with data storage: way too much information. Information Management. Retrieved on November 28, 2012, from http://www.information-management.com/news/data_storage-10016887-1.html