Why does this matter?The complexity of your data is likely to indicate the level of difficulty you’ll face when trying to translate it into business value – a complex data set is typically more difficult to prepare and analyze than simple data, and often will require a different set of BI tools to do so. Complex data necessitates additional work to prepare and model the data before it is “ripe” for analysis and visualization. Hence it is important to understand the current complexity of your data, and its potential complexity in the future, to assess whether your business intelligence project will be up to the task.
The simple test: big or disparate dataIn high-level terms, there are two basic indications that your data might be considered complex:
- Your data is “big”: We’ve placed the word big in quotes because of the seemingly infinite meanings of the term “big data.” However, the fact of the matter remains that dealing with larger amounts of data poses a challenge in terms of the computational resources needed to process massive datasets, as well as the difficulty of separating the wheat from the chaff, i.e. distinguishing between signal and noise amid a huge deposit of raw information.
- Your data is coming from many disparate sources: Multiple data sources can often mean messy data, or simply multiple data sets that follow a different internal logic or structure. Data must, therefore, be transformed, or consolidated into a central repository in order to ensure your sources are all speaking the same language.