Analytics and Business Intelligence Are Changing

(and That’s a Good Thing!)

We live in an era of data. Not just data, but Big Data: datasets have become so huge, complex, and fastmoving, that traditional BI solutions just can’t handle them. They all either fail in getting the data, dealing with the data, preparing the data, or just understanding the data… But handle it we must! Data is everywhere and more of it is being produced all the time.

Spotify, Netflix, Google, Facebook, and Amazon crunch immense amounts of user data and mix it with your own unique profile to surface new content and products. Hospitals, governments, and charities use augmented analytics to find new ways to administer services and help more people.

Your organization needs to uncover the insights hiding in your data if it’s going to thrive. Digging through this data is tough, but doable with the right tools. But how will you identify the solution to your changing data needs? And what does all this have to do with augmented analytics?

In this whitepaper, we’ll look at the history of analytics and business intelligence, the state of the industry now, and where it’s going in the future, to help you determine what the right BI tools to handle your business’s future needs will look like.

Below is the classical data value chain of BI and Analytics. Augmented analytics will make immense changes to every step.

A diagram describing the traditional business intelligence process: from data collection and ingestion to performing analysis to sharing insights.

What Is Augmented Analytics?

First off, we should probably talk about what “augmented analytics” actually means. According to Gartner, in their October 2018 research “Augmented Analytics Is the Future of Data and Analytics,” “Augmented analytics uses machine learning/ artificial intelligence (ML/AI) techniques to automate data preparation, insight discovery and sharing.

It also automates data science and ML model development, management and deployment.” Jen Underwood says that “augmented analytics uses machine-learning automation to supplement human intelligence across the entire analytics life-cycle.” Bill Su, writing on Medium, says that augmented analytics “[automates] insight generation… through the use of advanced machine learning and artificial intelligence algorithms.”

The bottom line on augmented analytics is that artificial intelligence will change everything about the analytics and business intelligence process, simplifying or eliminating some steps and radically changing and improving others. Before we dig in to all the specific ways augmented analytics will change the analysis and business intelligence process, let’s look at the evolution of business intelligence.

Machine-Generated Business Intelligence

What Came Before: Traditional and Self-Service BI

A timeline of the history of business intelligence (BI), from IT-generated to business-generated to machine-generated BI.

Traditional BI

Analytics and business intelligence have been around for longer than many people realize. In 1958, IBM researcher Hans Peter Luhn published his white paper “A Business Intelligence System.” He posited that because “information is now being generated and utilized at an ever-increasing rate because of the accelerated pace and scope of human activities” (IBM Journal, October 1958), new technological tools would be required to process all that information and make sense of it, if users were to make better decisions for their organizations.

Traditional BI was the first iteration of general-usage tools that organizations could use to make better decisions based on their data. This stage of BI focused primarily on connecting to single databases and generating basic reports.

Analysis was unsophisticated and usually not very timely and it was under the purview of a small class of dedicated analysts and data professionals. There was room for improvement and it came in the form of the next generation of analytics and BI: self-service tools.

Self-Service BI

Since the main drawbacks of traditional BI were the need for highly-skilled technical workers, lengthy times-to-insights, and poor quality of the data being analyzed, self-service BI set about addressing these, with some success. Today, pretty much every BI tool on the market claims to be self-service. They’d be fools not to—no one would even consider them otherwise! Modern self-service BI solutions mostly possess user-friendly graphical interfaces. With enough trial and error, users are often able to get to the answers they want without calling IT for help or learning advanced database language skills. These systems also have the capacity to handle millions and billions of rows, drawn from multiple data sources: in-house databases, cloud storage, apps, Excel spreadsheets, and more. Even non-technical users are usually able to choose and manipulate data sources and get them ready for analysis, speeding up time to insights and helping eliminate the IT bottleneck. Modern BI solutions also make data governance, security, and access control simpler for IT teams.

Approach Data Differently

Lastly, once users have those answers, self-service BI systems have dashboard-building features that offer a wide array of charts and graphs and easy color selection to help users make their insights look good and tell a story. The ability to embed analytics in standalone widgets, workflows, or a native app gives users across the board access to insights from the data that matters most to them (and if the solution is white-labeled, it’ll even retain the company’s look and feel). Being able to easily tag other users and add them to dashboards and reports makes the whole organization smarter by leveraging the work other users have already done. No more reinventing the wheel. But even these enhancements aren’t enough. We need to approach data differently.

Business Intelligence Changes Over Time

BI tools are evolving from monitoring performance and KPI analysis to being sophisticated analytics platforms driven by Artificial Intelligence.

A timeline of analytics, from traditional to self-service to augmented.

Thinking About Data and Analytics Differently

Modern analytics and BI systems have a lot going for them, but there are still places where we need to think about data and analytics differently. Data preparation could be simplified, new ways of beating user bias need to be developed, and the business-led aspects of the industry have to be countered. We also need to think about data itself in a whole new way.

While modern BI solutions are capable of handling more types of data and a greater volume than ever, cleaning up that data before it can be used is still a highly manual process, even with simplified, self-service systems. This introduces the possibility of human error before the analysis has begun! Augmented analytics systems will have AI components that simplify and improve this process. Human error also pops up around user bias when performing the analysis itself. Modern BI tools are great at showing users the insights they’re looking for. The problem is, that’s all they show. If a user knows exactly what they’re looking for when they sit down to use a BI platform, chances are they’re going to find it. This leaves very little room for surprising and unexpected results that the user might not have been thinking of, which are exactly the kind that can have a massive impact on an organization. This is another place that an AI-assisted system can help human users get more out of their analyses. Democratized, easy-to-use analytics tools, made easier-to-use by AI elements can change organizations from the ground up, will allow users in every department to make smarter decisions.

Another huge reason to change how we think about data itself is that today data is Big Data. Human activities and the Internet of Things never stop making new data to analyze. Data comes from countless human and machine-made sources. It can be a huge challenge for IT departments to marry these disparate data sources and get them into an analytical platform.

Users who rely on BI to make smarter decisions are understanding more and more that they can’t rely on a small cluster of analysts and IT professionals to do all the number crunching and database management for the entire company. Modern companies are flatter, meaning insights can positively impact workers at all levels, and the next big idea or opportunity could come from anywhere inside the organization. The old way of doing BI doesn’t work for the new world. So, what should the future of analytics and BI look like?

Reevaluating BI, Looking Toward the Future

Data has been called the “new oil,” because many companies see it as a powerful, money-making resource. And it’s not enough to have just a small team able to get at that data and plumb it for insights. Data access has to be democratized, with stakeholders from across the company able to access intelligent, self-service solutions and find the answers that matter most to them, as well as perform surprising and game-changing analyses. Ingesting the data stored in proprietary, off-site, and third-party databases has to be easy, secure, and not resource-intensive for IT. Insights should be revealed quickly (ideally, instantly—not in days or weeks) and easily shared with others in the company.

First off, democratization and self-service: Frontline workers are more tech-savvy than ever and there just aren’t enough database experts and IT staffers to handle the behind-the-scenes tasks needed to govern, connect, and clean the data. To get insights on the scale modern businesses need them, where they need them, and when they need them, users need an AI-assisted self-service tool with an easy-to-understand interface.

Delivering Maximum Value

Adding data to analytics software was one of the most annoying parts of getting insights in the traditional BI world. Emails, phone calls, and late-night trouble tickets abounded as overworked techs and analysts went back and forth trying to get information into the analytical systems to hopefully run the reports that would maybe tell them something. The Modern BI era saw solutions capable of handling multiple disparate sources: in-house databases, cloud storage, live data streams, app APIs, CSVs, and more, and pull it together to perform complex analyses. But what if your BI tool was smart enough to understand how to connect to data sources seamlessly, what the information in them meant, and how to combine them in new and interesting ways? It’d be a real game changer.

Users also crave easy options for analyzing their data and sharing results. The interface needs to make sense, and the platform needs to have multiple ways to access it, especially via mobile apps. Users don’t want to be shackled to a desk. They also want a simple, social way to share their insights with teammates. Tagging, sharing, and commenting within the app should also be simple and intuitive to deliver maximum value. More advanced modern BI systems also let users interact with them via chatbots and integrations with AI assistants like Alexa. AI assistance within the system will even suggest people share findings with, smoothing and speeding the process of disseminating findings to key team members.

What’s Next for BI: Augmented Analytics

The next wave of analytics and BI tools, augmented analytics, will feel noticeably different than the current era’s tools. Augmented analytics integrate AI elements into the analytics and BI process to help users prepare their data, discover new insights, and easily share them with everyone in the organization. This new paradigm will feel different because augmented analytics’ subtle integration of artificial intelligence and natural language processing elements will change the user experience across the entire BI process. Data ingestion, insight discovery, understanding correlations in data, and interacting with the platform will all become more streamlined and powerful than their modern counterparts in a self-service paradigm that really is self-service.

The cycle of augmented analytics: prepare the data, find patterns in it, then share and operationalize those findings.

A truly intelligent augmented analytics BI system starts helping the user from the moment they begin interacting with it. Data ingestion will be radically simplified: Instead of having humans clean up data, fit together data sources, and choose which datasets to mashup for their analysis, AI components will do much of the heavy lifting. Within a few years, the AI elements will be able to surface these relationships via visualizations that allow human users to begin drilling down and looking for more insights immediately. Going smoothly from data ingestion to searching for insights will be a huge time saver from even streamlined modern BI systems.

Augmented analytics systems will also be built for Big Data. Modern enterprise companies already have data needs in the millions and billions of rows. That number is only going to keep going up, and the next wave of BI platforms will be ready for it. No matter how much data there is or where it comes from (in-house databases, cloud databases, live streams, application APIs, Excel spreadsheets, etc.), these smart systems will be able to handle them. The systems will also understand the differences between the datasets, how they interact with each other, and how best to query them for fastest results. Instead of being passive receivers and holders of data, waiting for the human users to perform analyses, these smart systems will be active in interweaving the data as it comes in and assisting in analysis.

Once the human user is actively searching for new insights into the data that the AI systems have helped them prepare, even this step will be different under the augmented analytics framework. Thanks to user bias, people only look for the things they want to find and only end up finding what they’re looking for. Looking for the obvious isn’t the way to radically change a business or the world. The AI in augmented analytics platforms will be free of human biases and will reveal insights that the humans never realized they needed or how crucial they are.

They will “see” connections in the massive ocean of data and suggest relationships and insights without the user even thinking to ask for them because their algorithms can consider so many more relationships than a human can.

Moreover, advanced machine learning algorithms aren’t bound by preconceived notions, biases, or misconceptions. They simply deliver insights where they are found. Augmented analytics systems will even become spatially aware, understanding how datasets from different parts of the world interact.

Understanding the insights being surfaced by the system and digging in for more will also change under augmented analytics. Natural Language Processing (NLP) literally gives these futuristic BI systems a voice: When new data is pulled into the system, the NLP components will be able to read and understand more of the data and draw conclusions from it.

Then, it’ll surface those new discoveries to the human user in a conversational manner. These conversational interactions will go even further, as humans will be able to query the system just by saying “show me my sales numbers for the last year” or asking “what was our revenue change over the last six months?”

Once they’ve served up the insights, intelligent systems can even walk their human users through the data, explaining it and helping them gain truer understanding, as opposed to just giving them the information.

This change to analytical systems that can actually talk with the users will extend beyond just using the platform at a computer. Many modern BI solutions have mobile apps. Some even have chatbot or Amazon Alexa integrations that allow users to interact with the data via more than just keystrokes and mouse clicks. Augmented analytics will keep building on this trend.

Analytics and BI will become an immersive, always-on environment. New insights and data will be accessible via a wider array of devices and pulled from the air just by asking a question of a chatbot or voice interface.

Intelligent assistants will notify decision makers when something needs their attention, as it happens, instead of waiting to be asked. In many situations, trying to predict the future isn’t as useful as getting instant updates when changes are happening and then reacting in real time.

Benefits from the Right Insights

Augmented analytics platforms will also have a stronger social component. All tech users are familiar with social networks, where sharing images and stories and tagging friends and family creates a richer, shared experience.

Organizations will use a similar social network effect in augmented analytics platforms: when dashboards and visualizations are created and insights discovered, users will share them, tag others within the organization, add notes and stories, and begin building a larger narrative and fitting that data into the business’s mission, all within the platform.

Instead of creating dashboards and reports and waiting to present them in a meeting, letting precious time slip away and slowing decision making, this instant notification and interweaving of many different users’ efforts, augmented analytics systems will become a productivity tool.

Think of an augmented analytics platform as an always-on, immersive system that takes people from questions to insights to decisions within a persistent environment, across departments, teams, devices, and locations.

This type of simplified sharing may help boost adoption across teams, especially among non-technical users or teams that don’t perform analysis frequently but could still benefit from the right insight at the right time.

This is why augmented analytics platforms, smart ones anyway, will include the ability to embed analytics anywhere. This can take several forms: First off, standalone analytics widgets can pull insights out of dashboards and allow users to place them anywhere (on a desktop, in a workflow, etc.).

Some systems even allow users to embed actions like triggering a purchase or launching a marketing campaign right alongside the insights. Embedding could also empower a client’s software with analytical capabilities without them having to build the new functionality from scratch, serving internal teams and surfacing insights to end-users/customers as well.

This all contributes the augmented analytics’ overall feeling of an immersive, always-on environment, constantly serving up insights and improving users’ lives with data.

From Big Data to Smart Data: Data Cognition

Augmenting the traditional BI process with AI is happening, but it’s not the only change coming to how analytical platforms handle huge amounts of data. Simply scaling up computing power to query huge datasets isn’t enough. A complete rethinking of how to pull insights from Big Data sources is underway. Data Cognition Engines are the newest way to understand what’s happening inside these huge datasets, quickly and efficiently.

These engines are a radically different approach to data and analytics that tackle data and analytics challenges previously deemed insoluble. Rather than trying to process big data queries faster (which takes tons of money and tech), data cognition engines condense immense amounts of data (terabytes) into lightweight neural networks (megabytes!) for quick and easy analysis, processing hundreds of thousands of queries a second using the latest AI technologies.

Representing huge datasets in small neural networks saves tons of storage space and time when processing queries. Instead of querying the data sources directly, queries are sent to the data cognition engine, which answers questions at submillisecond speeds with 99% accuracy (and better).

This radical new approach opens up a huge array of possibilities that would be impossible given the expense and latency of working directly with the huge, underlying data sets.

This transformational opportunity in data analysis is delivered via a series of problem-agnostic advanced-AI algorithms and by constructing a deep neural network (DNN) which is used to replace large and cumbersome datasets.

Making the Impossible Possible

Data Cognition Engines will change the world of Big Data and analytics forever, giving users unprecedented abilities to work with these immense datasets. First off, they open the door for interactive data exploration with millisecond response times, only querying the more expensive, slower Big Data system directly when very precise detail is required.

They also compress terabytes of data into a model that occupies less than 5 megabytes for each terabyte. Once in place, the tiny DNNs require no access to the underlying data, eliminating the need for storage, processing power, and bandwidth.

Additionally, since systems like this don’t retain any knowledge about the lowest level of detail in the data, there’s zero risk that queries could return sensitive data that is prohibited by an organization’s policy or regulatory compliance requirements. In cases where the row-level detail is not needed or cannot be stored, DNNs can completely replace Big Data repositories with a nearly accurate solution that will satisfy most analytic needs. For organizations with Big Data who want to work with their datasets quickly and easily, this is a game changer.

Conclusion

Traditional BI

Humans manually connect data sources and painstakingly clean up the data (where errors can occur), then they pick the data sources that they want to analyze (further introducing error and human biases), then they have the platform perform the analysis, receive insights/reports/etc., and share them across the organization (maybe as emails, messages, or within the platform, if it’s sufficiently advanced).

Self-Service BI

User-friendly graphical interfaces that handle millions and billions of rows, drawn from multiple data sources: in-house databases, cloud storage, apps, Excel spreadsheets, and more. Non-technical users can choose and manipulate data sources, speeding up time-to-insights, eliminating the IT bottleneck. Data governance, security, and access control are simpler for IT teams. Dashboard-building features offer a wide array of charts and graphs and easy color-selection features.

Augmented Analytics

AI systems help ensure effortless data collection and connection to the system. Once the data is there, the smart system helps human users choose the right datasets based on relationships it has detected while bringing in the data. When it’s time for actual analysis, the AI lets the human “drive,” but will also suggest analyses that the human might not even know they want as they are setting up the analysis.

Once the insights have been gleaned, the human user can effortlessly share these insights across the system via a wide range of integrated functions: in-app messaging, mobile apps, chatbots, AI assistants, and more.

Summary

The world has changed: data is now Big Data. Countless devices and users are creating new digital records every second of every day. That data is processed and stored in increasingly complex ways. More powerful and robust analytical systems and AI assistance are needed to make sense of it all.

Every company, organization, and government will need an augmented analytics platform to connect to these databases and live data sources, find relationships within the data, create visualizations and aid in storytelling, and then help human users effortlessly share their findings across the entire organization. They will also need ways to work with Big Data that go beyond the usual analytics systems and completely reimagine how users can relate to data. Augmented analytics will change how users experience analytics and BI. They will also change the world by serving up insights that humans could never imagine.

About Sisense

Sisense takes a radically different approach to business analytics. We believe that insights should be easily accessible to everyone, everywhere regardless of technical ability, and back this belief with a solution that simplifies every step of the BI process for all users—from data preparation to delivery of insights. We pair innovation with an obsession for customer success, enabling both startups and global brands to instantly reveal insights from complex data.

Watch a Demo

Why Sisense?

Augmented Analytics: the Future of Business Intelligence