Have you ever thought of your contribution to the growing digital data? Usually, we upload pictures and videos, publish blogs, etc to our hangout social media channels. Similarly, enterprises, scientific research institutions, government organizations and all sections of the society contribute in one way or the other to the tons and tons of unstructured data. Do you have any idea what all kinds of data is getting generated? Business data, enterprise data, personal data, customer data, confidential data etc. Have you ever given a thought where at all this tons of data do get stored? “Data is everywhere and there isn’t enough room to store it”. Enterprises, individuals and everybody & anyone who is online have a major role in contributing to this immense data growth. That is why the relevance of the term ‘big data’ is going to remain for long.
Information is like gold. For an enterprise, data is the lifeblood of the business. Businesses collect raw data everyday from various sources (online/offline). This raw data then goes through many layers of structured processing (data/information processing) to be converted into meaningful information, considered as the most valuable assets by a Business. This wealth of information has a major role for an enterprise to achieve business goals. As we all know, data processing has eveloved as an industry in itself. This discussion, hence, becomes all the way important when we know that the data is growing.
Tons and tons of unstructured data are mounting in an explosive manner. Many terms have been floating to represent the exponential growth Digital world is challenged by – ‘data growth’ or ‘big data’ or ‘data deluge’. This is becoming a cause of concern for Global organizations as they too experience the implications of ‘big data’, which is beyond the capacity of commonly used tools to capture, store, manage, and process it. From tera bytes to peta bytes to zeta bytes and further to yottabyte, expansion of data continues to outpace the available storage capacity in enterprises. “Enterprise data in all forms is expected to grow around 650% over the next five years”, as per Gartner prediction.
What contributes to this explosive growth of data in recent years?
Many factors contributed to this data revolution. Internet revolution, mobile communication revolution etc are few among them. The proliferation of broadband internet and mobile communication channels brought in a sea of changes in the way people access and share data. Data is gathered in every fraction of a second from ubiquitous media equipments and channels like computers, mobile phones, smart phones, laptops, web logs, social networks, cloud environments, scientific researches, cameras, microphones, wireless sensor networks, etc. Information is flooding whether or not you want it.
A few cases:
- In Dublin, Ireland, an IBM InfoSphere Streams project has been collecting traffic data from buses and sensors at intersections. With 4,000 detectors in place in a road system with 700 intersections, the project is receiving 20,000 data records per minute, a pace of more than 300 per second. A thousand Dublin buses engaged in the project are sending 3,000 GPS readings per minute, a rate of 50 per second. On average, each bus sends location data every 20 seconds.
Why so much info from the streets of Dublin? One benefit the project aims to deliver is a system that triggers traffic signals to give any bus that approaches an intersection a green light. In addition to trimming operational costs for diesel fuel and electricity consumed while idling, the project could boost ridership, as citizens opt for timely bus routes to avoid traffic jams.
- In Canada, Project Artemis, a collaboration of IBM, the University of Ontario Institute of Technology and Toronto’s Hospital for Sick Children is collecting data from bedside devices and notes from doctors and nurses to help newborns. The goal is to use data to spot potential signs of life-threatening infection 24 hours in advance.
“Close to 200 pieces of information get generated per second for every baby,” Bhambhri says. It is humanly impossible to properly analyze all this information without technology. Project Artemis uses IBM InfoSphere Streams, a new processing architecture that employs targeted algorithms, to give doctors information in near-real time to make potentially life-saving decisions.
Source: IBM Destination z
The data is growing too fast and in huge quantity, to be effectively managed, searched, filed, indexed, briefed, organised or numbered.
What are the implications of data growth?
The implications of big data in how businesses, organizations and Governments function are massive. Too much data makes it difficult to process or manage it which in turns means making it difficult to get the right information in right time to be used for a specific purpose. Growth of data in an uncategorised and unstructured manner also demands the need for additional storage, new technologies for processing it and more human resources for managing it. The data growth has severe implications regarding security too. Organizations find it difficult to introduce appropriate security measures, which can lead to data breach, data loss etc.
Global business world is looking for solutions to address the big data and how to strategically use it for business benefits. Latest Mainframe systems with added capabilities and business models like cloud computing, ‘data storage as a service’ etc are expected to play a key role in dealing with the data deluge in the coming days.