The Costs of Business Intelligence (BI) in Perspective

Logically, the most successful business intelligence (BI) or analytics programs are the ones that incorporate as many users as possible,…

Logically, the most successful business intelligence (BI) or analytics programs are the ones that incorporate as many users as possible, from different roles and departments within an organization. The more users share data and insights, the more benefits of business intelligence a company can realize. Strangely enough, most old-school BI solutions don’t seem to follow this line of reasoning. As the number of users increases, so does the complexity of the system (and by extension, the cost). A number of startups founded in the past few years are attempting to change this aspect of the data-driven business model, in a variety of creative ways. But this still leaves one question unanswered—why is it that incorporating more users means a building a more complicated and expensive system?

Perspective #1: There is no “free lunch” (software costs)

Many people think that this is the simple answer to the question of cost—more software licenses for more people means more money. In fact, licenses are commonly used as a red herring in business intelligence pricing models. It’s fairly obvious why getting licenses for a greater number of users increases the total cost of licensing, but this is in turn only part of the total cost of a business intelligence software.

This may be a little hard to believe, but the proof is found in open source. Open source BI is free, and there’s no shortage of choices if you’re dead set on getting open source software. But none of these free options are as popular as established non-open (paid) BI tools from for-profit vendors, even though they’ve been around for as long or longer than the paid options. If the software licensing were the most costly element of BI implementation, it would follow that open source vendors would be far more prominent.

If you want even more proof, consider the BI platform pricing that major BI vendors offer for licenses; most large companies can easily negotiate a volume discount on licenses. Besides being a well-known marketing technique to increase sales, discounted bundled pricing is also evidence that software licenses are only part of the total cost of business intelligence, especially for larger deployments. As the number of licenses grows, so does the support infrastructure—DBAs and other IT personnel, preparation projects, administration, etc.

TL,DR: Pay attention to software costs, but there are hidden costs associated with most BI solutions. If you must compare the cost per license, factor in additional systemic costs as well.

Perspective #2: Money can’t buy happiness (hardware costs)

Depending on which technology you choose, each additional user can demand a marginal increase of anywhere from 10-50% in hardware resources, which includes disk, RAM, and CPU required to run BI software. The volume of data being queried directly has a major impact on hardware requirements, but so does the number of users expected to be performing the queries. The real point of no return is reached when data processing requirements outgrow commodity hardware (any cheap, off-the-shelf box) and other solutions, none of which is low-cost or simple, must be evaluated. There are three main options for organizations with box-breaking processing needs:

  1. Purchase a high-end proprietary server
  2. Build a data warehouse or pre-processed OLAP cubes
  3. Clustering/sharding

None of these options is particularly simple, and they’re all time- and resource-intensive. Most well-known BI technologies (OLAP, in-memory databases, RDBMS) were developed a decade ago or more—your mobile phone* probably has more advanced processing capabilities than most computers did when these database solutions were being developed. The problem is that modern business data is higher volume, higher velocity, and more complex than before, and older technology just can’t support the desired usage levels in a reasonable timeframe. The good news is, newer technologies are designed to work on almost any 64-bit commodity machine; by using modern chipsets to their fullest potential, it’s even possible to run queries on hundreds of gigabytes of data, for hundreds of users, on just one of today’s commodity hardware boxes.

TL,DR: If you’re looking for a solution that doesn’t require a massive budget for proprietary hardware or IT resources, you should look for technologies that let you run your business intelligence from a single commodity box.

*To be fair, I’ll also point out that a mid-market smartphone today has more computing power than the Apollo 11 shuttle.

Perspective #3: Goldilocks Syndrome (starting too big or too small)

There’s one additional fatal mistake in choosing a BI vendor that can mean unexpected spikes in cost down the road or wasted money in the short term. Choosing a solution based solely on current needs (3 terabytes of data, 50 users) will keep immediate costs more reasonable, but what happens when you have 8 terabytes of data and twice as many users? How can you anticipate project costs when you have trouble predicting growth in team size, data growth, or both?

The first challenge is to train your current employees to use the software you’ve chosen. For instance, if you’ve purchased 100 licenses, you might be disappointed (but not shocked) to learn only 15 of them know how to use it. So your total cost, even with the “bargain” pricing on a purchase of 100 licenses, is still only divided among the 15 users of the product. It’s a massive waste of resources, and yet it’s quite a common situation.

An alternative is to start by purchasing only a handful of licenses and then expanding as the user base starts to grow. But deploying a solution for 10 users is often vastly different from deploying the same solution for 100 users, and shifting from one model to the other likely requires a change in solution architecture. So while you might initially save money on licensing, the longer-term costs of expanding are going to be higher when you add users.

TL, DR: The ideal solution is one that scales without requiring major changes to architecture to support new users. Buying more software or hardware on an as- needed basis is cheap compared to the cost of rebuilding the entire system from scratch periodically.

Tags: |