If I told you all of our modern activities–posts to social media sites, transaction records, online activities, digital pictures and videos, and mobile GPS signals to name a few– leave a trail of data behind that is estimated to produce about 2.5 quintillion bytes of data each day, would you have a real feel for how much data that is? Even if I defined that number more clearly as 1 followed by 18 zeros, many tech veterans and savvy business leaders still have trouble digesting the scope of that amount of data, and even more importantly, knowing how to prepare for it. While understanding the information data provides has become an integral part of the global economy, transforming business intelligence software into a booming market totaling in $14.4 billion just in 2013, many businesses today are still getting stumped on one critical point: calculating how much data they will accumulate in the near future.
Big Data, Big Decisions
Because data has gotten so big, it’s hard to truly grasp the data requirements of your business and its five-year plan when choosing a Business Intelligence (BI) solution and responsibly prepare for the future. This includes securing the ability to realize all the economic and business possibilities BI offers without bumping into software, finance, or data limitations–which is the most unfortunate stumbling block, since it is those businesses that have already done the painful part of shelling out the resources needed for a big data solution.
Yet, regardless of time and money expenditures, if a business chooses a solution that can’t scale to its future data needs or becomes a blackhole for new investments in expensive hardware, third-party software, or a dedicated team for maintenance, they still end up with an obsolete tool just years later. To bring some clarity to the business intelligence buying process, here are two ways to more accurately measure your data and ensure your BI solutions isn’t worthless in the next three years:
2 Rules to More Accurately Measure Your Data
1. The amount of data you have today is only an indication of how much data you will have in the future
If you’re excited at the prospect of finding a BI tool that can handle the exact amount of data your business has today, thank the heavens you are reading this guide. Choosing a BI tool that holds around the amount of data you have today means this BI tool will mostly likely be useless to your organization before you can even get the BI software implemented. No matter what sector you work in, valuable digital information is being created at such an astonishing rate it is reported that 90% of the world’s data has been produced in just the last two years, and the amount of data created by businesses doubles every 1.2 years.
Think: that means all the historical data your business has already gathered, plus every row, column, and byte of data you have scattered across your billing system, web analytics system, feedback system from places like Salesforce, Google AdWords, or just your Excel sheets, may only be a minuscule 10-40% of what you have in another year or so from now. That’s some pretty big data in just months, my friend.
Ultimately, how much data you have today is a road-map to more accurately predict how much data you will have in 3 years–and you must plan accordingly by making sure the BI solution can scale! Otherwise, vendors can mislead buyers into thinking they’ve found a solution that provides “the perfect match” to how much data they have today, when really they need to invest in a more robust solution that can scale down the road.
Bottom line: You need a way to gather, manage, and easily understand all that data, and now you understand you need to buy the shoes a size (or 2) up. In-memory solutions can support gigabytes of data and were the newest big data technology just years ago. Today, with the exorbitant rate of accumulating growth of data, occasionally even in-memory technology has become a more temporary resolution to the big data problem. Other more scalable alternatives, such as the more expensive (and cumbersome) OLAP, or innovative technology such as In-Chip technology that is build to easily handle terabytes of data, have successfully allowed businesses to grow at a fast pace and provide a long-term solution.
2. Measure your organizational data as well as valuable external data sources
Another rule to grasping the full scope and potential of your data is to understand that analyzing organizational data, however much you have of it, is only achieving part of the beauty of business intelligence. Businesses are mashing up external data sources such as government data, external contact data, demographic data, and more to gain a competitive advantage by seeing their audience with a new angle.
For example, Wal-Mart combined historical weather data with sales histories to pinpoint the products people bought in bulks right before a hurricane’s arrival. They found that strawberry Pop-Tarts sales are seven times higher than normal, while beer is the top-selling pre-hurricane item. Likewise, in the days following a storm, stores typically ran out of a number of certain dry food products. Wal-Mart now increases the stocks of those products and have increase sales during severe weather conditions.
Similarly, sales and marketing departments have much to gain by combining their data with third-party data to learn about their prospects. For example, many businesses today incorporate data from CrunchBase, one of the most widely used databases of technology companies, people and investors, to build a target list for specific niches. There is even a BI investor dashboard that analyzes investments, acquisitions and start-ups for financial companies and entrepreneurs, helping them make the smartest investments possible.
Bottom line: While this step in a BI strategy needs your creative thinking the most, in a few years from now, it will be a standard for businesses to think outside the proverbial box and take external data sources into account–simply because the competitive advantages to be had are tremendous. Consider now for a moment: What sources are available for you to gather data about your competitors, your audience and customers, and your industry as a whole? Gain a general account for amount of external data you may use in the next several years, such as public information databases and projections.
Ensure Your BI Software Isn’t Worthless Next Year
Now, you don’t have to be a prophet to forecast a realistic, even probable data growth number. Once you do, select a BI solution that can easily scale as your business grows and can handle the exponential amounts of data you will have in the future. Also, as your company gains speed, more and more departments and employees want to crunch their numbers to gain their own business intelligence, make sure the BI tool can also scale to users and their queries–without getting sluggish or breaking your bank account.
To take your business intelligence to the next level, the level that gives you the upper hand among your competitors, is to transition to a BI system that can handle your future big data requirements. BI can enable you to anticipate market paradigms, scale your business up (or down) to meet its changing needs, and be prepared for the future–all while your competitors are still busy analyzing their spreadsheets.