A Data Science Central Community
If I told you all of our modern activities–posts to social media sites, transaction records, online activities, digital pictures and videos, and mobile GPS signals to name a few– leave a trail of data behind that is estimated to produce about 2.5 quintillion bytes of data each day, would you have a real feel for how much data that is? Even if I defined that number more clearly as 1 followed by 18 zeros, many tech veterans and savvy business leaders still have trouble digesting the scope of that amount of data, and even more importantly, knowing how to prepare for it.
While understanding the information data provides has become an integral part of the global economy, transforming business intelligence software into a booming market totaling in 14.4 billion just in 2013, many businesses today are still getting stumped on one critical point: calculating how much data they will accumulate in the near future.
Big Data, Big Decisions
Because data has gotten so big, it’s hard to truly grasp the data requirements of your business and its five-year plan when choosing a Business Intelligence (BI) solution to responsibly prepare for the future. This includes securing the ability to realize all the economic and business possibilities BI offers without bumping into software, budget, or data limitations–which is the most unfortunate stumbling block, since it is those businesses that have already done the painful part of shelling out the resources needed for a big data solution.
Yet, regardless of time and money expenditures, if a business chooses a solution that can’t scale to its future data needs, or becomes a black-hole for new investments in expensive hardware, third-party software, or a dedicated team for maintenance, they still end up with an obsolete tool just years later. To bring some clarity to the business intelligence buying process, here are two ways to more accurately measure your data and ensure your BI solutions isn’t worthless in the next several years:
1. The amount of data you have today is only an indication of how much data you will have in the future
If you’re excited at the prospect of finding a BI tool that can handle the exact amount of data your business has today, thank the heavens you are reading this blog. Choosing a BI tool that holds around the amount of data you have today means this BI tool will mostly likely be useless to your organization before you can even get the BI software implemented.
No matter what sector you work in, valuable digital information is being created at such an astonishing rate it is reported that 90% of the world's data has been produced in just the last two years, and the amount of data created by businesses doubles every 1.2 years.Think: that means all the historical data your business has already gathered, plus every row, column, and byte of data you have scattered across your billing system, web analytics system, feedback system, from places like Salesforce, Google AdWords, or just your Excel sheets, may only be a minuscule 10-40% of what you have in another year or so from now. That’s some pretty big data in just months, my friend.
Ultimately, how much data you have today is a road-map to more accurately predict how much data you will have in 3 years–and you must plan accordingly by making sure the BI solution can scale! Otherwise, vendors can mislead buyers into thinking they've found a solution that provides “the perfect match” to how much data they have today, when really they need to invest in a more robust solution that can scale down the road.
Bottom line: You need a way to gather, manage, and easily understand all that data, and now you understand you need to buy the shoes a size (or 2) up! In-memory solutions can support gigabytes of data and were the newest big data technology just years ago. Today, with the exorbitant rate of accumulating growth of data, occasionally even in-memory technology has become a more temporary resolution to the big data problem.
Other more scalable alternatives, such as the more expensive (and cumbersome) OLAP, or innovative technology such as In-Chip Technology that is build to easily handle terabytes of data, have successfully allowed businesses to grow at a fast pace and provide a long-term solution.
2. Measure your organizational data as well as valuable external data sources
Another rule to grasping the full scope and potential of your data is to understand that analyzing organizational data, however much you have of it, is only achieving part of business intelligence. Businesses are mashing up external data sources such as government data, external contact data, demographic data, and more to gain a competitive advantage by seeing their audience with a new angle.
For example, Wal-Mart combined historical weather data with sales histories to pinpoint the products people bought in bulks right before a hurricane’s arrival. They found that strawberry Pop-Tarts sales are seven times higher than normal, while beer is the top-selling pre-hurricane item. Likewise, in the days following a storm, stores typically ran out of a number of certain dry food products. Wal-Mart now increases the stocks of those products and have increase sales during severe weather conditions.
Similarly, sales and marketing departments have much to gain by combining their data with third-party data to learn about their prospects. For example, many businesses today incorporate data from CrunchBase, one of the most widely used databases of technology companies, people and investors, to build a target list for specific niches. There is even a BI investor dashboard on Crunchbase that analyzes investments, acquisitions and start-ups for financial companies and entrepreneurs, helping them make the smartest investments possible.
Bottom line: While this step in a BI strategy needs your creative thinking the most, in a few years from now, it will be a standard for businesses to think outside the proverbial box and take external data sources into account–simply because the competitive advantages to be had are tremendous.
Consider now for a moment: What sources are available for you to gather data about your competitors, your audience and customers, and your industry as a whole? Gain a general account for amount of external data you may use in the next several years, such as public information databases and projections.
Comments are closed for this blog post